Making the agility stat affect cast speed sounds stupid and not logical. What, does it make you stroke it faster so that you can high intensity discharge on someone? Here is a solution that would be unique but work really well.
The only common Stat between mage and fig classes should be Constitution.
For Mage types, get rid of the stats Agility and Strength and replace them with something else (maybe one of the stats can affect cast speed) while the other does something else that is mage specific and be useful to a mage.
For Fighter types, get rid of Intelligence and Wisdom and replace them with something else that is fighter specific and useful to a fighter. (Maybe replace one of them with an endurance stat that affects the damage you do with your spell attacks such as spin and run attack).
This would be a great change for the game. Right now the system is very simple, Mage types have 3 important stats (Cons, Int, Wis) while Fighters have three important stats as well (Cons, Str, Agl). No fighter type will invest points into Wisdom or Intelligence because they do poo all for them compared to the other stats.
Putting in my idea will make the other two stats more useful for the respective character type.