Written by Jesper Tingvall, Product Expert, Simplygon

Disclaimer: This post is written using version 10.2.10100.0 of Simplygon and Maya 2022.5. If you encounter this post at a later stage, some of the API calls might have changed. However, the core concepts should still remain valid.

Introduction

In this blog we'll showcase how to cull away triangles that are below terrain. This is an easy way to reduce overdraw with the aggregator pipeline. It is also applicable to remeshing and we will discuss some common pitfalls.

Prerequisites

This example will use the Simplygon integration in Maya, but the same concepts can be applied to all other integrations of the Simplygon API that has support for selection sets.

Problem to solve

We have a scene with objects and a terrain. Parts of the objects are below the terrain causing overdraw and unnecessary triangle count. We'd like to remove these.

Just like icebergs our rocks have huge parts of them below the surface. This means we have triangles that will never be visible in the game. Removing them would cause no visual impact.

The assets we'll use in this blog are.

• Rock moss set 02
• Coast Line 01

Solution

In our case we want to optimize even the LOD0 model so aggregator, which leaves the surface of our model intact, is suitable. We will use this with geometry culling. It is also possible to use terrain culling in the remesher. This is suitable when creating HLODs for far away usage. All lessons in this blog, especially including the trouble shooting ones are applicable to remeshing as well.

In most cases terrain culling is something we build into the game engine so it can be performed after the level has been created in the level editor. In this blog we'll use our Maya plugin's UI to showcase the concept. Once we understand the concept of it we can automate it through scripting. An example of how to automate this for the aggregator and remesher can be found in the blog Aggregation and remeshing - evaluating the best method for proxy creation.

Selection set

First, we need divide up the scene into terrain and objects to cull. In Simplygon this is done with selection sets. To create a selection set in Maya select the terrain, then to go Create → Sets → Set. Give it a name, in our case 'terrain'. If we would move over to a scripted pipeline in the future, creation of selection sets would be done programmatically. We could split the scene depending on materials, node names or any other criteria we can imagine.

Aggregation with culling

Now let's create our pipeline which will do the processing for us. We will use aggregator. Bring up the Simplygon UI and click Add LOD Component then pick Template → Basic → Aggregation.

If we wanted to perform more optimization on the LOD0 asset in future, like merge materials to reduce draw calls, we could also do this with the aggregator. But for now let's keep it simple.

• Since we do not want to merge our output models into one we disable MergeGeometries.
• We check EnableGeometryCulling, this will make so any triangle fully enclosed in the mesh will be removed. This is a very useful way of optimizing kit bashed asset which usually contain lots of triangles on the inside. For more information about this feature see our video tutorial on Hollow Shell.
• Lastly to make the geometry culling take our terrain into account we check UseClippingGeometry as well as specify ClippingGeometrySelectionSetName. We set this to the selection set we just created.

Geometry Culling Precision

Using the GeometryCullingPrecision parameter we can adjust what precision to use for culling. Higher precision means we will cull triangles closer to the terrain but give us increased memory usage and processing time. Let's test a few different values.

We can see that a low value gives us more triangles under the terrain. A value of 0.0 gives us a lot of remaining triangles. It is very hard to see difference between 0.5 and 1.0, but there are some more triangles removed for 1.0. In our case we'll go with default value of GeometryCullingPrecision = 0.5.

It is important to point out that the aggregator pipeline does not cut triangles. That means that if you have large triangles which are mostly below the terrain with only a small part visible above it will be kept. Since the remesher creates a new mesh, this will not be a problem with it.

To process the scene, select the assets we want to process then click the big yellow Simplygon logo.

Result

If we look on the surface of our asset, we can not see any difference. Which is expected as we do not touch any of the triangles above ground.

If we look below the ground, we can see that a lot of triangles are removed.

After processing we get fewer triangles, but on a modern system we do not expect this to be the main performance gain. The real optimization is less overdraw. Depending on technology used for shadow casting we also get the following benefits:

• If we use real time shadows then we get no shadows casted by, or on, triangles below ground.
• If we use lightmaps then we get better usage of texture space since we are not wasting it on invisible triangles below ground.

Troubleshooting

We are now going to cover some common issues when it comes to using geometry culling. This blog is written from the point of view that we are culling against a terrain, but we could use arbitrary geometry for culling. If your culling geometry has any of the following issues, then you will in most cases get an empty scene back.

Worth pointing out is that we are only concerned with the parts of clipping geometry that intersects the bounding box of our processing geometry. If you use the same terrain for geometry culling on several objects, you can get some of them processing as expected while objects in other areas processing incorrectly.

Hole in terrain

One pitfall we have seen some studios do with culling geometry is that the terrain they are using for culling have holes in it. The reason for this can for example be that an entrance to a cave system is placed there. The reason why this does not work is that we can not divide up the scene's volume into two parts; things on inside that should be removed and things on outside. Since we have a hole in the culling geometry everything will be marked as inside and thus everything will be removed.

How to solve this is to plug that hole. Here we have added a cork covering the hole used solely for geometry clipping. After covering the hole in the terrain completely it works as expected.

Outside bounds of terrain

It is important that the clipping geometry covers the entire bounding box of our processing geometry. If we would have geometry placed outside of the terrain, then we would discover that we get an empty scene back after processing. The issue is same as "hole in terrain" case above, we can not find a way to divide up the scene into volumes that are on the inside and volumes that are on the outside. There are two ways of solving this. First is to include more of the terrain so bounding box of all processing geometry is completely covered by the terrain. Another way is to add terrain skirts by extending the outermost edge loop.

Strictly speaking the terrain needs to cover a little bit more then the exact bounding box of the processing geometry, so give it some margin. This margin is dependant upon which precision you are using.

Backface triangle

Incorrectly facing triangles is not that common when culling against a terrain since those tend to be quite simple in geometry. It is of course always possible that exporting bugs causes incorrectly facing triangles in your terrain mesh. But this is more likely to happen if you include more objects in your clipping geometry then just plain terrain. Say for example that you've included all other static objects in the scene.

Then what you risk having is geometry where back faces of triangles are possible to reach from the top side of our terrain. If that is the case, then we would get an empty scene back. In the image we can see example of that. The back faces are coloured in red. If we would cull against this mesh, we would get an empty scene back. The scene having a back facing triangle above the terrain marks everything above ground for removal. One can think of it like back faces flood fills culling into our volume.

The way to address this is to ensure that no triangles with incorrect back face is present in the clipping geometry. Depending on what precision you use back faces hiding in cracks can, or can not, flood fill the entire volume. There is a higher risk of this happening at higher precisions. If a scene suddenly starts disappearing when culling at higher precision, then this can be the fault.