"But, the Talisman makes it 100 no matter what modifiers are in play, which is what makes it inherently unbalanced"
No, it just makes it overpowered for the cost to get it.
Not inherently unbalanced, because if the thing cost 50k I doubt you'd see many complaints about it being too powerful.
Phoenix-D