Small ovaries = bad news?

I had the first appointment with a fertility specialist today. Previous tests all looked fine - HSG, blood levels, etc. After reviewing all the records and paperwork she seemed optimistic. She did an ultrasound and casually mentioned that my ovaries were not as big as she would like to see and that I didn't have as many eggs/follicles as she would like. So I have been reading online all about small ovaries and am now concerned that this may be a big problem. Does anyone else know about the impacts of small ovaries??