This begs a question: What if the user wants a billboard that is fixed on some other axis? I can think of other instances where having some other orientation would be preferred. One such example would be a space sim, where there’s no real specific orientation that is “up”.
First, I think it would be better to use local transforms instead of world transform. That way if I position the billboard at a 30 degree angle on the X-axis, it still won’t be limited.
Secondly, I think the API here should match closer to what Bullet Physics does for their 2D API:
body->setLinearFactor(btVector3(1,0,1));
body->setAngularFactor(btVector3(0,1,0));
The billboard API could be something like:
billboard->setAngularLimit(Vector3(1,1,1)); // Able to freely rotate
billboard->setAngularLimit(0,1,0); // Rotate by the Y-axis only