You need to transport a bunch of troops to the planet.
Just build a transport and load it up with troops. Enter into combat and park your transport right next to the planet. Click on "orders" and one of the options is "Drop Troops". Hit the "begin" button and if you have more troops than they have defences then the planet is yours
Naturally your going to want to plan this out a bit, send the transport in with support ships because the planet will likely have weapons platforms that will knock your troopship out of commision before you even get close.
BTW, if you are successful you'll want to build an "Urban Pacification Center" on your newly conquered planet right away because they won't be too happy about it.