Well yes, playing a defensive game
and being deficent in agg/def is a bad combination. You can get by with a lower agg/def and a higher rate of construction, but you have to press the attack early and often to get the full use of your advantage. Swarm the opponent before he has a chance to get a lot of colonies and ships, and before he can get the ECM and sensors that in addition to his racial bonus make the fights really lopsoded. Otherwise you are just waiting till they get better ships and can...well I don't have to tell you, you've seen it yourself.
Geoschmo