[Suicide Junkie]

From What I've Seen:

I believe that the "Damage until weapons gone" overrides the 10% damage rule.

Also, those rules only decide when to reconsider the targetted enemy. If the planet is still the best target according to the nearest/strongest/etc rules, your forces will choose it again as a target. If the planet is the only thing left, they will shoot at it just for something to do.

What you need to do is drop those troops.

Arrange the speed of your transports and the number of attack ships so that they arrive at the planet with cover fire.
Once the troops are down, an unarmed, contested planet is no longer considered hostile.