Great as the game is, the AI is notoriously poor. Even at its best (and TDM is the best in the stock game) it is easy to beat. Only in exceptional circumstances will you get the challenge you desire.
Some things you can do:
- Use a small map
- Move all computer player settings to high
- Select the best empires rather than random selection -- United Flora and Space Monsters are two I know of
- Let the AI use 2000, 3000 or 5000 points while you use none.
- Play PBW http://seiv.pbw.cc/text/
You will eventually play PBW against other people; the only true competition you will find. It can be very humbling.
As for mods, I'll have to let others answer.