When I want something aside from what I receive in my CSA basket, and it's not Saturday (thus precluding the UCI farmers' market as an option), this (Growers Ranch) is where I generally get my produce.
It is a tremendous value - dollar for dollar the best value in Costa Mesa, for sure - but the quality is just short of superb, and nothing is organic or too exotic. Thus, if it is for everyday cooking, it is generally where I will shop. For special occasions and purposes, such as entertaining, I may go elsewhere. At places like Whole Foods or Mothers you will get better stuff, generally, but you will pay about five times as much.
Also - one should note that this place is not to be confused with Grower's Direct, also in Costa Mesa but on 17th at Newport Blvd. This place is even cheaper than Grower's Ranch, but somewhat lower quality. I am, however, not above shopping here from time to time, as it is extremely close to my home and open on Sundays and later in the evening that Grower's Ranch. They also have some items that you can't get at Grower's Ranch - e.g. Grower's Ranch didn't used to carry rapini regularly.
Another option - although this might change/limit your options as far as variety - are CSA's - which stands for Community Supported Agriculture.
Basically, you subscribe to a weekly or bi-weekly basket - which you can usually pick up somewhere near you - from a local farm or co-op of farms. Many of them choose heirloom, rare, and exotic seed varieties that are bred for taste rather than shelf life or cosmetic appearance, fruit is actually allowed to ripen fully and naturally (on the vine), and everything is generally picked the day of, or day before the baskets are assembled. The three I mention below are also certified organic.
You can, of course, get this kind of stuff from the better farmers' markets, but I find that the CSA arrangement is more affordable and I like the excitement - and subsequent challenge - of not knowing precisely what I'm going to get.
In the OC, I know of three different CSA's I can recommend:
Morning Song Farm (in Rainbow) - the best citrus in the world, also great avocados and macadamias
South Coast Farms (San Juan Capistrano) - George is famous for the albion strawberrries and heirloom tomatoes
Garden of Eden (co-op of farms in north San Diego county, I beleive) - I don't know this one as well, but they always have terrific greens at the UCI farmers' market.
The three offer, of course, a much wider variety of fruits and veggies in the baskets than I mention above.
A Google search should avail you of all the details.
If you don't mind traveling as far as San Diego County, then I strongly recommend Chino Farm in Rancho Santa Fe -- an hour or so from OC -- one of America's finest producers and a favorite of great cooks, home and restaurant alike. I love their fruits, especially berries and melons, and their green vegetables. The heirloom plum tomatoes make an excellent pasta or pizza sauce, the peppers a fabulous salsa, the yellow-and-white corn a perfectly sweet side for steak or Southern barbecue. And their Concord grapes are the Platonic ideal for Welch's. It's all rather expensive, of course; this isn't your home-town farmer's market. More like Grandma's backyard garden, assuming your grandmother is Alice Waters (whose Chez Panisse was one of CF's first major restaurant customers).
I've never seen fiddleheads or ramps at OC farmer's markets... I'd love to have a source. Have others found them in the past? Which markets/vendors?
English peas already seen this year, at Irvine UCI and Lag Hls.
Asparagus, of course, is easy to find at the markets. Favas, another springtime treat, and artichokes, too.
Farmer's markets... the OCFB market at UCI on Saturday mornings has English peas. I didn't see any ramps, though -- too early?
You might also try a produce market, like Manassero Farms or Tanaka Farms, or maybe Farm Boy in HB or Produce Boy (name correction?) on State College and Placentia, or Tom's Farms on Euclid and Cerritos...