You get experience for destroying enemy ships and colonies, too, though I never checked how much.
Wouldn't it be nice if you got some extra racial points to spend at game start with an experienced race? Even a 10-1 conVersion of racial experience to racial points would be good.