Ok, new idea. I played around a bit with Fyron's suggestion of four sub-systems per system. I realized that four stars could be spaced so I could use two rings of planets around each star and not interfere with each other. This would actually permit more planets then you'd have with nine subsystems if you wanted to stuff it full. However, it's not very astheticly pleasing.
What four stars with two planetary rings does allow me to do is use less than the maximum number of planets and introduce some randomness to each subsystem. So they start to look more like tiny star systems instead of fuzzy dice.
The empty sectors between planets could could have asteroids or storms even.
Here's an example. This was created by moving planets around in the paint program. I haven't updated the mod data files to actually produce these system types. What I'd need to do is create a little script to make several system types that are all similer with some randomness in planet placment so they don't all look alike.
This gives us much better looking sub-systems, while still having almost as many planets as the 9-by format.