Taking over replacment spots is cool. The other players in the game love you cause you are giving them a real live person to beat rather than an AI. If you manage to win or at least do well you get more credit than normal because you started from such a poor position. If you lose you can blame it on the previous owner, or the AI. It's a win for you either way.
If you are concerned with the amount of free time you have, well that's a legitimate concern. But keep in mind that many replacment spots are for smallish empires so there is less to do.
Geoschmo