I've thought mainly about
The obvious thing is that you'll take much much longer to get high level spells. No way I can disagree there, and this is the big problem, obviously. How bad this problem is depends on whether you do an even advancement in each class, or leave one until later. The fact that mages are especially weedy at low levels makes this problem particularly acute. I won't hide that. On the other hand, if you ever reach level 40, who the hell wants to face a level 20/20 Sorceror/Wizard? And a level 15 sorc/level 5 wiz has a few advantages too... Overall though, until high epic, it is a disadvantage. Now we have that most obvious disadvantage out of the way
For seconds, you'll have to concentrate on two stats instead of one... Very true. You'll have to decide what to take at level up, and you'll have to decide what rings to wear, but I've never yet seen an amulet to improve charisma, nor a belt, cloak or helm that ups Intelligence. Careful juggling plus use of potions ought to get you pretty high up in both. Another way around this would be to multi-class in a way where the prime requisites are complimentary, such as bard and sorceror or sorceror and cleric.
Now, your caster level will always be lower. Not much can be done about that, but you can focus on taking spells that do not rely on caster level. This too has disadvantges, as you will be casting spells that give saves, and that relies on spell level, and you will not be casting high evel spells, sop DCs will be low, but with high enough enough stats to get those DCs up, and with some spell focuses, you could do well, and there are spells out there that give neither saves or are caster level determines (They just aren't very good...) Necromancy in particular has lots spells of use. And there are very very few spells that aren't cappped at some number, so eventually, you'll be hitting a max anyway on those spells.
If you take the cleric/other option, a big downside is that you will not be able to wear armour any more and take advantage of the fact you can wear armuor and cast divine spells.
One thing that is good is that if you take spell feats, you'll have many more spells to apply them to. Well, no you won't, because you will have less high level spells, but more spells at each level. This is where a level 15 sorceror, level 5 wizard would be nice. Or you could take Pale Master levels and get extra spells for both classes every two levels

Overall, I don't think I have been very convincing, but think about it, how useful are your Epic Spells really? It might be fun to try this, neh?