Cutting holes in the terrain

Does current Terrain component support cutting holes in it? Say for making cave entrance. API does not suggest anything obvious, but maybe there is some smart trick to achieve this?

Graphics is simple.
Physics is average.
Navigation etc - I don’t know.

FYI: How to make Bullet work with holes in terrain:

1. Inherit from btHeightfieldTerrainShape
2. Override processAllTriangles
3. Somehow drop triangles: use mask array or sentinel height. I’ve just copy-pasted content of processAllTriangles.
4. Use your own height field shape.

Maybe you could use several terrains, leaving a gap between them?

I thought of that. It would likely be easiest approach, but then these two terrains would not be stitched together and lesser problem is gaps would be allowed only at terrain boundaries.

@Eugene thanks for pointers. Any idea if sentinel height value could be used with heightmap itself? I tried naive thing (discarding vertices here) and it ended up being a royal mess because clearly i have no idea what im doing.

Uhm, looking at terrain just these days…
I found, for instance, BlenderGis, which allows you to get multimapping; one could use heightmap as usual, and slope and aspect to get direction of movement for characters for recast/detour I think… or for intentional movement inside a cave…
Anyway terrain basically is a flat plane and doing st like cave sounds like voxels… the normal solution is using point of interests and teleporting somewhere else…
But of course this is not continuos movement like probably you’re looking for…

It’s hard to combine stable geometry discarding, sentinel heights and geomipmapping.
Even PhysX use material index instead of height to mask cells as gaps.
I recommend you to hold the idea about sentinel heights and revise it if it become important.

Another approach is to just use something other than heightmap terrain, like regular 3D models.
You can also mix it up with terrain, leaving a low height hole where you want to place your 3D model cave, so the terrain is below the cave.

a workaround with terrain is to have some sort of transition between the cave and the terrain, like a door.

@Enhex my primary reason why i looked at terrain component is that it handles stitching and all that. Using 3D model would probably be akin to making own terrain system.

On a closer look at terrain component i noticed these:

void SetNorthNeighbor(Terrain* north);
void SetSouthNeighbor(Terrain* south);
void SetWestNeighbor(Terrain* west);
void SetEastNeighbor(Terrain* east);

Does terrain component handle stitching of multiple Terrain components together? Not just TerrainPatches within Terrain? If so that brings me to your original suggestion.

Take a following picture as example:

Each rectangle represents Terrain component. Each green border represents neighbour relationship being set. Red border represents no neighbour relationship.

Am i right to think that in this setup terrain would not be stitched together at the red border? If so then adjacent tiles could have different height (which would be a very steep cliff if stitched together) and create a hole when no stitching is applied to that border.

AFAIK, terrain wouldn’t be stiched unless you stich it manually. I mean, it depends on values in heightmap.
But I find this approach very limited to be useful.

What does SetXXNeighbor(Terrain*) do then?

Ensures that geomipmapping won’t make any unexpected holes

Excuse my ignorance, but that sounds very much like stitching up terrains. Could you please elaborate?

Separate terrains don’t have any shared data, so they couldn’t stich themselves without explicit user intention.

Neigbors are used to ensure that terrain patches on the edges has valid topology and there won’t be any gap caused by mismatching LODs.

You don’t have to handle stitching with a 3D model, just let it intersect with the terrain and you won’t have gaps.

I am experimenting with setting up multiple terrains. Docs are sparse on this topic and there seem to be hidden requirements for making entire thing work.

What is supposed to be a heightmap format? I noticed that terrain skips 1 pixel at the edges of heightmap, thus for creating 16x16 units terrain i need to use 17x17 heigthmap. Whats the purpose of skipped edge?

How exactly am i supposed to use SetXXXNeighbor()? My test code:

        auto terrain = scene_->CreateChild()->CreateComponent<Terrain>();
        terrain->SetPatchSize(8);
        terrain->SetSpacing({1, 1.f / 255.f, 1.f});
        terrain->SetHeightMap(GetCache()->GetResource<Image>("Textures/heightmap.png"));

        auto terrain2 = scene_->CreateChild()->CreateComponent<Terrain>();
        terrain2->SetPatchSize(8);
        terrain2->SetSpacing({1, 1.f / 255.f, 1.f});
        terrain2->SetHeightMap(GetCache()->GetResource<Image>("Textures/heightmap.png"));
        terrain2->GetNode()->SetPosition({16, 0, 0});
        terrain->SetEastNeighbor(terrain2);

And a heightmap. I scaled it up by 800% for demo purposes. Actual image used by code is 17x17.

Test code aligns terrain tiles like so:

On the right edge of terrain i expected to get seamless transition between terrains, however it looks like this:

Notice crack between points painted in white on heightmap. Also neighbour terrains are not taken into account when calculating tiles.

So whats the right way to use terrain in this case?

Do you mean 16x16 quads?

If you want two terrains be seamlessly connected, you should set them as neighbors for each other.

I mean entire terrain being 16x16 quad.

Edit: i suppose 1 pixel in heightmap means one vertex. In order to get 1x1 tile terrain i would need 2x2 heightmap, because one quad needs 4 vertices at the corners. So 16x16 terrain needx 17x17 pixels. Right?

Like so?

        terrain->SetEastNeighbor(terrain2);
        terrain2->SetWestNeighbor(terrain);

Because it produces exactly same visual result.

I think so.

Well, that’s strange. Maybe terrains have different scale?
They must have connected geometry. Could you check the grid?

Do you mean seeing debug geometry? Terrain does not have built in debug info rendering on purpose. I added it myself bit it isnt exactly revealing.

They are of exactly same scale. Snippet i pasted previously - its pretty much all the code setting up scene. Other stuff is just setting up viewport, camera and creating scene object. You can take a look at entire thing: terrain.zip

Edit:
I just noticed that white pixel on the right was tiny bit darker than white pixel on the left. That caused height mismatch. But there is still normal issue which makes lighting look weird. Is terrain supposed to take into account neighbouring terrains when calculating normals?

I have just noticed too xD

Yeah, I don’t like this lighting. Terrain definetely could pick neighbor’s data when generating normal, but it doesn’t do it for now… It could be hard, but must be possible.

Looking at the code it seems like neighbours should be set before creating the terrain geometry:

void Terrain::HandleNeighborTerrainCreated(StringHash eventType, VariantMap& eventData)
{
    UpdateEdgePatchNeighbors();
}

    // Send event only if new geometry was generated, or the old was cleared
    if (patches_.Size() || prevNumPatches)
    {
        using namespace TerrainCreated;

        VariantMap& eventData = GetEventDataMap();
        eventData[P_NODE] = node_;
        node_->SendEvent(E_TERRAINCREATED, eventData);
    }

Huh, just recalled…
If you want high-quality terrain lighting, use world-space normal maps.
Bake true normals of terrain into texture and use them in the shader instead of vertex normals.
It has almost zero performance penalty, doesn’t requre changes in Terrain and has much much better quality.
Vertex normals are badly corrupted by geomipmapping and look nasty. Texture is always nice.
(just a hint)

I don’t think that it’s important because Terrain geometry is created via GetRawNormal/GetRawHeight that don’t care about neighbors.

Tried setting neighbours before setting heightmap too. @Eugene guessed right - it had no effect.

Sounds like normals in a texture are easy way around the problem, going to do just that. Thanks

The cool part about using a normal map for normals is you can go higher resolution, to add detail that isn’t there in the heightmap. Bake your normal map from a much higher resolution version of the heightmap, then downsample that heightmap for the actual heightmap, etc… It’s a process I’m working on implementing in my terrain editor.

Back to original issue… I found interesting idea on the internets: having depth mask object in a place where hole is supposed to be cut in the terrain. I created material that makes box transparent and all, tried ordering of box and terrain rendering but best i could get was a black or transparent box but no hole in the terrain. Any pointers would be greatly appreciated.

As for collisions one person on youtube mentioned using trigger objects disabling collision with the terrain when passing through masked area. This indeed sounds like perfect solution. Much simpler than altering collision/rendering meshes.

I don’t understand the idea, may you elaborate?

BTW, I never liked solutions with stencil and other pipeline hacks because it makes many things broken and manual control is required…

It is. If you could guarantee that:

1. All your holes have some “restricted area” around the hole edge that is never collided.
2. This restricted area is bigger that any dynamic object that could collide your hole.
   

As i understand rendering “mask” object is supposed to alter depth buffer making part of terrain covered by “mask” object transparent.

In video i mentioned mask object shader adds object to rendering queue at Geometry+10. link
Same thing is done with terrain material, except its added to queue at Geometry+100. link

I am not familiar with unity terms, but it seems this technique exploits rendering order.

Possibly:

1. Render mask object, depth buffer modification done here
2. Render terrain object, previous depth buffer modification prevents terrain from rendering at spot where mask object was visible.

Did i get it right?

I see. It must work if you set correct render order. Default is 128. Terrain hole and terrain itself should have bigger order than everything else. Hole technique must have depthwrite=true. Note that hole shader mustn’t discard pixels with zero alpha.

If I understand correctly, this hack won’t work if your Camera could move through the hole.

Why not? Considering camera can see through that spot and collisions would be disabled upon moving through…

Could you take a peek at my technique and material?

 <technique vs="Unlit" ps="Unlit">
    <pass name="alpha" depthwrite="true" depthtest="lessequal" blend="replace" />
</technique>

<material>
    <technique name="Techniques/DepthMask.xml" />
    <parameter name="MatDiffColor" value="0 0 0 0" />
    <renderorder value="130" />
</material> 

They result in a black mask object. For terrain i am using original Materials/Terrain.xml material by the way.

What about Terrain render order?

Huh, it would be fine if you set front face culling for your mask object material.

Terrain material does not set renderorder so it must be default value.

So it won’t work if you draw terrain before cutting the hole.

I tried changing renderorder of mask material to 0, 120, 130, 200. I also tried explicitly adding renderorder to terrain material and changing numbers so their order changes. Nothing has any effect which makes me think i messed up something else in material or technique.

I don’t know what could be wrong with materials.
The most important thing is that hole is rendered before terrain:

I also tried explicitly adding renderorder to terrain material and changing numbers so their order changes.

The second important thing is that hole material writes depth.

depthwrite=“true”

You should be able to see hole if you set alpha = 0.1.
If you turn lighting on and make your terrain semi-transparend, you should be able to see the actual order of objects: maybe Urho doesn’t handle render order properly.

Victory! This is how its done:

For depth mask object:

 <technique vs="Basic" ps="Basic">
    <pass name="alpha" depthwrite="true" blend="addalpha" />
</technique>

<material>
    <technique name="Techniques/DepthMask.xml" />
    <parameter name="MatDiffColor" value="0 0 0 0" />
    <renderorder value="129" />
</material> 

For terrain:

<material>
    <technique name="Techniques/DiffAlpha.xml" />
    <renderorder value="130" />
</material>

Important: terrain has to have technique supporting alpha. Result:

Thank you everyone for the help!

@rku Sorry, I was wrong twice. It won’t work in common case.
This hack won’t work if hole object is not rendered in front of terrain. E.g. if your camera is inside hole object.

PS. Rendering terrain after all other geometry will have negative performance impact due to overdraw.

PS2. Your terrain should also be rendered after semi-transparent things like particles.

@Eugene thank you for insights. Indeed this appears to be fitting for just some usecases. I tried making mask object as thin as possible, but even then camera passing through it results in a tiny few pixels strip across entire screen. That strip is terrain being no longer masked. And like you said, performance issues… I guess they could be solved by replacing material of those terrain patches that have holes. Not much point in that considering previous issue.

Back to the drawing board i guess. I took a closer look at terrain code and seems like sentinel height values are off the table as well. Terrain component does a smart thing reusing index buffers between patches. Going to try to experiment by allowing unique index buffers for patches.

I don’t think it’s good solution if you want to use geomipmapping. There is no change to generate nice automatic LODs for terrain with corrupted geometry.

Heightfield use R or RG channels for height.
You could use B channel to mask holes for physics.
Of course, you’ll need semi-transparent material for terrain.

My idea was to allow it generate and process all vertices and only to not render triangles where at least one vertex is at zero height.

But transparent objects are rendered after geometry, so it means overdraw like with previous approach, no?

1-bit transparency is rendered like solid, so it’s fine (ALPHAMASK pixel shared define in material).

You will either have your holes disappearing (if hole is fully covered with low-quality LOD) or growing x2-x4-x8-etc (if it’s not) as lod quality is decreased. Impossible to control, impossible to tune. The only way is to disable geomipmapping at all.