Bazookas aren't anti-building weapons, they're anti-tank weapons, and are really only useful in that capacity.
Also, bazookas are sensitive to location: a shot from the rear is more effective than a shot from the front, and a shot straight at it is better than a shot at an angle.