You can basically go about as low as you want, but the flight time is going to drop and the biggest thing to consider is the overall amperage the pack can handle. For example, let's assume we have three 50C packs; 3800mAh, 4400mAh and 5000mAh. The true amperage (assuming capacity and 'C' ratings are correct) would be 190, 220 and 250 Amps respectively.
If you want to run really low capacity packs, it would be worth offsetting that with a higher 'C' rated pack. Just keep in mind 'C' ratings aren't always true. The typical way manufacturers rate a pack is to take a single cell, do a test in optimum conditions and use that for all packs made from that type of cell. When you assemble packs, the 'C' rating is different from a single cell. On top of this you have the "average" of the tests conducted, then marketing comes in. I like having a real number based on the true capabilities of a pack based on the worse condition, not a "high average" result. That is why the Opti-Power packs are rated "so low" in comparison to others, but still perform as well or in many cases better.
Okay, almost off the soap-box... I firmly believe we could all benefit from a third party certification of batteries to give them their true 'C' rating. The mobile audio world adopted a system like that about eight years ago due to many companies coming up with crazy power ratings for their amplifiers that were no where near reality... much like our Li-Po market currently. It was insane to see how many "1000 watt" amplifiers were really 200 watt under the new cert process!!!