[Suicide Junkie]

Armor doesn't appear to be destroyed first - I've had vessels shot full of holes and still have most of the armor components intact.

No, it dosen't
What type of armor were you using, anyways?
The light armor has more hitpoints, but the heavy armor blocks a greater % of the damage.
The key thing about armor, is that it only REDUCES damage, and does not block it completely.
When you ship gets shot at, you get little holes in your armor, and weapons can slip through.
For better protection, add more/heavier armor. Researching tougher Versions of the armor you do have also helps a little bit.

If you are just using structural supports instead of components named armor, you will find they are often hit late in the battle, but can distract enemies. They may waste valuable shots trying to vaporize a crippled ship, instead of shooting at the live targets.