Magic!

If I were doing this by myself, and not with the group at large, I would just go ahead and start breaking into classes and how they work/how they're built, but since we're not even entirely sure as of yet whether or not there's going to be "classes" right now, I'm instead going to start throwing out ideas for how the magic system works.

Magic systems are odd, they're different in literally every game I've ever played.  Some of them, like the D&D system is very rigid (extremely rigid) and concrete.  Every spell has a use, every spell is explained in several paragraphs exactly what it does, what it can't do, etc.  Then there's games like Mage: The Awakening where the spells are completely created by the players themselves, they are given a school, so many points in that school, and then they say exactly what their spells do.  It's very versatile, and very broken.

Then there's middle grounds, such as The Dresden Files RPG, which supposedly has a good mix of the two, unfortunately I have not yet played it so I cannot draw from it.

So I'll stick with what I know.  Personally, I do like the D&D system, however it kind of boxes a player in a little bit, eventually though, with new levels, they can pick up plenty of interesting and very usable spells, however eventually they become gods themselves and we don't want anything like that in our game.  We want them to be powerful, we want there spells to be versatile, but we don't want them to be gods, we don't want them to have a solution to every problem.

I read somewhere online about a system for D&D called Epic 6, essentially in D&D, your character completely stops leveling at level 6.  They can't go any higher.  The reasoning for this is when analyzing powers of some of the great fantasy heroes from books like LotR, or the Belgariad, and so on, a group of people noticed something: the heroes were generally speaking not incredibly powerful.  They said that Gandalf, when he took down the demon, was only a level 4 Wizard.  Suramon was only level 6.  Sauron?  10.

Gandalf, one of the greatest heroes Middlearth had ever known, was only level 4.

So what does that mean for us?

Well, we are trying to go for a more gritty sort of game, the players are meant to earn their victories, not just be handed them.  So how do we fit in magic for all of this?  My suggestion, is we take a look at the Player's Handbook spells and the Spell Compendium spells, come up with some of our own, and give the characters options to choose them.

Now the question is, how many spells do they get?

One of the things mentioned earlier was the idea that players do not get new powers as they level up, does this mean that casters don't get new spells as they level?  If so, that could make them a lot easier to balance out with the party.  But they need enough spells to make them versatile and with enough tricks up their sleeves they can benefit the party if the party is stumped.

We could potentially give them an entire spellbook at their disposal, all spells of their magic type are automatically available to them.  However I fear that might make them too powerful.  But maybe not, as they still have to deal with action economy and the like.  So I'm not sure about the answer here, but limiting spells might be a solid option available.

Also as for the whole prepared casting/spontaneous thing, I've got to say one thing: no prepared casting.  Too much book work.  Too much time and effort spent out of character trying to think about the situations your DM normally faces you with and how to prepare them.  Add metamagic to it and it gets even worse.  All casters will be spontaneous casters if I have anything  to say about it.  End of story.

As for how spontaneous casting should work, I think that a character should get so many "cantrips" or simple spells and let them cast them an unlimited amount of times. (Like Pathfinder does)  I mean the spells aren't gamebreaking in any way (besides maybe Prestidigitation...) but it adds flavor and mystique to the Wizard.  As for the rest of the spells, potentially give them a set amount of points equal to x + their main casting stat.  Or Double/Triple their casting stat.

Now each spell costs, as written x amount of points to cast.  They can make it longer/shorter, bigger/smaller, different shapes, do more damage, a different kind of damage, etc. by adding more points into the mix of the spell.  So yeah, once a day they can do one incredible spell, but then they're spent for the day, besides the cantrips.  I think this follows more along the lines of the traditional Wizards that you read about, who dig deep into themselves sometimes, almost killing themselves, to do these amazing world changing spells.  The rest of the time they're slinging around minor spells without too much problem.

I don't know, magic isn't my forte, but I think that this may be a solid method to work out casting costs and the like.