Perhaps the AI's you encoutered were Xenophobes? The Nuetrals I tend to encounter tend to be Neutral and Xenophobic, meaning they like to be left alone and not bothered. So it would make somewhat sense that they were warm to you since you hadn't contacted them or bothered them, they accepted your trade alliance since they had a want for the bonus resources they would get, and are now displeased since they had to be bothered. It is a stretch but possible perhaps?
If they weren't neutral, perhaps only when you established a Trade Alliance did one of your diplomatic representatives let slip that your empire spanned 20 systems and resided on planets
just like the AI's homeworld...
Anyway, just a thought.
- b00tch