I wish to mask the part of the object that is outside of the billboard.
You may try this trick:
- add for yours billboard technique with additional pass let’s say - “bmask”
<technique vs="LitSolid" ps="LitSolid" psdefines="DIFFMAP">
<pass name="base" />
<pass name="litbase" psdefines="AMBIENT" />
<pass name="light" depthtest="equal" depthwrite="false" blend="add" />
<pass name="prepass" psdefines="PREPASS" />
<pass name="material" psdefines="MATERIAL" depthtest="equal" depthwrite="false" />
<pass name="deferred" psdefines="DEFERRED" />
<pass name="depth" vs="Depth" ps="Depth" />
<pass name="shadow" vs="Shadow" ps="Shadow" />
<pass name="bmask" vs="YourShaderToWriteBWColor" ps="YourShaderToWriteBWColor" />
</technique>
YourShaderToWriteBWColor.glsl
[spoiler][code]#include “Uniforms.glsl”
#include “Samplers.glsl”
#include “Transform.glsl”
#include “ScreenPos.glsl”
void VS()
{
mat4 modelMatrix = iModelMatrix;
vec3 worldPos = GetWorldPos(modelMatrix);
gl_Position = GetClipPos(worldPos);
}
void PS()
{
gl_FragColor.rgb = vec3(1.0);
}[/code][/spoiler]
-
add in RenderPath new RTT (with screen size or less) I think 8-bit for mask is enough let’s say it named as "bmaskRTT"
So you are add into head of RP:
-
also add scene pass to RenderPath what will be used for billboards with your custom material/technique what have “bmask” material pass
- at last you have mask and you still do not render your objects what need masking.
So, at this step you will need tweak shader of this objects(shader) to let use this mask. and add bmask RTT for scenepass into TU where this object are rendered
Prepare workaround for shader with put all needed data in RenderPath pass:
It’s may looked like this:
<command type="scenepass" pass="ThereisWillbeDrawObjectWhatMaskedByBMASKRTT" output="viewport">
<texture unit="diffuse" name="bmask" />
</command>
so and shader for this will be quite simple it fetch from diffuse(bmask).a < 0.5 you write result color of objects, if not you doing discard.
there maybe needed little fixes/polishing but main concept of using this trick I think simple.
In this case you do not need second scenes and other stuff, you just works only with RTT and custom shaders/passes in RenderPath.
- also don’t forget clear yours custom RTT by clear command for properly rendering working in some cases.