On the cusp...

[Unwise]

My system is a little better than yours, but not too much better to where I should get drastically superior turn load times, but I do.

Not necessarily. A 2GHz Intel (especially if it's a Celeron) might be struggling to process all the battles on even a modest map. There is a lot of data being crunched.

On the "Glory of Gods" map (a big one) with all Early Era races represented, I have seen processing times of about a minute and a half and I am running a fairly beefy rig.