Written by Jesper Tingvall, Product Expert, Simplygon

Disclaimer: The code in this post is written using version 10.2.4000.0 of Simplygon and Blender 3.3.7. If you encounter this post at a later stage, some of the API calls might have changed. However, the core concepts should still remain valid.

Introduction

In this post we'll cover how to find the correct settings for a scripted pipeline by exporting a pipeline created using the user interface.

Prerequisites

This example will use the Simplygon integration in Blender, but the same concepts can be applied to all other integrations of the Simplygon API.

It is important to point out that we do not have full support for Blender shading networks. In this example the material consists of 3 textures; base color, roughness and normal. If your material shading networks are more complex Simplygon will not understand the material. That will give you black texture output.

Problem to solve

In this example we will use a chessboard model made by Riley Queen. In this asset each chess piece is their own model. We have 3 materials; one for the chess board and one for white pieces and one for black pieces. We would like to aggregate this model into one that can be rendered in a single draw call without loosing visual fidelity.

We have created a pipeline in UI and want to change this into a scripted pipeline. Reason for this could either be that we want to automate the pipeline and define it via scripting instead of a json file, or that we require access to some script only Simplygon API functions.

When we create pipelines using the UI Simplygon defaults to the correct settings for that integration. As we now are creating the pipeline via code we want to see which settings that are correct for this integration.

Solution

The solution is to export the pipeline file from user interface and look at the exported json file when we are recreating it via scripting.

Pipeline created in user interface

We start by creating an aggregation pipeline in our user interface. We'll specify that the meshes should be merged with MergeGeometries as well as any internal face should be removed with EnableGeometryCulling.

To merge the base color texture a color caster for it is added.

To merge the roughness texture a color caster for roughness is added. Notice that OutputSRGB is unchecked as we are using linear color space for roughness texture.

Lastly to merge the normal map a normal caster is added. Our Blender plug-in will default to the settings that are correct for how Blender encodes normal maps.

Export pipeline

To export the pipeline as a json file click the gear icon in the top right corner and select Save pipeline. This exported pipeline file can be of great use if you want to share it with your colleagues, more to read in this blog post.

Aggregation pipeline

We will now recreate the pipeline in Python script. Let's start by creating a function which creates an aggregation pipeline.

def create_aggregation_pipline(sg, base_color_channel_name, texture_size): 
    """Create aggregation pipeline with color material caster"""
    
    # Create aggregation pipeline
    aggregation_pipeline = sg.CreateAggregationPipeline()

To use the same settings as we exported from our user interface we can look at the exported json pipeline.

"AggregationSettings": {
    "MergeGeometries": true,
    "MergeGeometriesUI": {
        "Visible": true
    },
    "EnableGeometryCulling": true,
    "EnableGeometryCullingUI": {
        "Visible": true
    },
    "GeometryCullingPrecision": 0.5,
    "GeometryCullingPrecisionUI": {
        "Visible": true,
        "MinValue": 0.0,
        "MaxValue": 1.0,
        "TicksFrequencyValue": 0.10000000149011612
    },

The same settings can then be set on the AggregationSettings object.

    aggregator_settings = aggregation_pipeline.GetAggregationSettings()
    # Set aggregation settings.
    # Specify that we want to merge into one model and remove all internal geometry.
    aggregator_settings.SetMergeGeometries( True )
    aggregator_settings.SetEnableGeometryCulling( True )	
    aggregator_settings.SetGeometryCullingPrecision	( 0.5 )

To merge all materials into one we want to perform material casting. Thus we also need to setup a mapping image. These settings except output texture resolution where completely hidden in the user interface, but we can see them in the exported json file.

MappingImageSettings": {
    "GenerateMappingImage": true,
    "GenerateMappingImageUI": {
        "Visible": false
    },
    "GenerateTexCoords": true,
    "GenerateTexCoordsUI": {
        "Visible": false
    },
    "GenerateTangents": true,
    "GenerateTangentsUI": {
        "Visible": false
    },
    "UseFullRetexturing": true,
    "UseFullRetexturingUI": {
        "Visible": false
    },
    "ApplyNewMaterialIds": true,
    "ApplyNewMaterialIdsUI": {
        "Visible": false
    },
    ...
    "TexCoordGeneratorType": 1,
    "TexCoordGeneratorTypeUI": {
        "Visible": false
    },

The settings we are going to set is following.

• To generate a mapping image GenerateMappingImage is set to True.
• We want to generate texture coordinates so GenerateTexCoords is set to True.
• Tangents is also desired so GenerateTangents is set to True.
• As we are merging all materials into one we need to set UseFullRetexturing to True.
• We also need to specify that we should apply new material IDs by setting ApplyNewMaterialIDs to True.
• TexCoordLevel specifies which texture coordinate level to use for texture generation. In Blender we can read from the exported pipeline that it is set to 0.

For TexCoordGeneratorType we have 2 options.

• Parameterizer which unwraps a whole new set of unique UVs.
• Chart aggregator which uses the UV maps already present on the meshes and merges them into one chart. It is clever enough to reuse the same texture space if multiple models refers to the same location.

As our model already contains UV maps we go for 1 = ETexcoordGeneratorType_ChartAggregator. This is also what our json exported file refers to.

    # Set mapping image settings to generate new UV coordinates we can use for material casting.
    mapping_image_settings = aggregation_pipeline.GetMappingImageSettings()
    mapping_image_settings.SetGenerateMappingImage(True)
    mapping_image_settings.SetApplyNewMaterialIds(True)
    mapping_image_settings.SetGenerateTangents(True)
    mapping_image_settings.SetUseFullRetexturing(True)
    mapping_image_settings.SetTexCoordLevel(0)
    mapping_image_settings.SetTexCoordGeneratorType( Simplygon.ETexcoordGeneratorType_ChartAggregator )

As we are using the chart aggregator we need to set some settings on that one as well.

"ChartAggregatorSettings": {
    ...
    "OriginalTexCoordLevel": 0,
    "OriginalTexCoordLevelUI": {
        "Visible": true,
        "MinValue": -1,
        "MaxValue": 255,
        "TicksFrequencyValue": 1
    },
    ...
    "OriginalChartProportionsChannel": "Basecolor",
    "OriginalChartProportionsChannelUI": {
        "Visible": true
    },
},

Here we need to specify which UV channel to take our charts from with OriginalTexCoordLevel. To determine the relative size of each aggregated chart in output atlas we need to specify which material channel to look at. This is done with OriginalChartProportionsChannel. It is set to the base color channel, same as the we use in base color casting below.

# Look at base color channel to determine chart proportions
chart_aggregator_settings = mapping_image_settings.GetChartAggregatorSettings()
chart_aggregator_settings.SetOriginalChartProportionsChannel(base_color_channel_name)    
chart_aggregator_settings.SetOriginalTexCoordLevel(0)

Lastly we set the output texture size. Instead of looking at the exported pipeline we'll have this as a parameter in our script so it can easily be changed.

    material_settings = mapping_image_settings.GetOutputMaterialSettings(0)
    material_settings.SetTextureWidth(texture_size)
    material_settings.SetTextureHeight(texture_size)

Base color casting

We will now start to add material casters. We will start with a function for creating a color caster. It can be worth pointing out that there might be more settings that we want to set if we are using transparent materials. But for this example MaterialChannel and OutputSRGB is enough.

def create_color_caster(sg, channel_name, output_srgb):
    """Create a color caster using specified settings"""
    
    caster = sg.CreateColorCaster()
    caster_settings = caster.GetColorCasterSettings()
    
    # Set casting settings specific per integration
    caster_settings.SetMaterialChannel(channel_name)
    caster_settings.SetOutputSRGB(output_srgb)
    return caster

Every 3D creation tool and game engine has different names for texture channels. To find out the name of our base color channel and if it should cast in sRGB we can look at the pipeline file we just exported.

"ColorCasterSettings": {
    "MaterialChannel": "Basecolor",
    "MaterialChannelUI": {
        "Visible": true
    },
    ...
    "OutputSRGB": true,
    "OutputSRGBUI": {
        "Visible": true
    },
}

Based on this ColorCasterSettings we can see that Blender has texture channel name "Basecolor" for the base texture and it is in sRGB format. The code for creating a color caster for base color will thus be.

basecolor_caster = create_color_caster(sg, "Basecolor", True)

A good way of detecting if sRBG flag is set correctly is that if it is not we will get wrong color share after processing. More on this can be read in our blog post about troubleshooting gamma issues.

Roughness casting

Roughness is just another color channel so we can reuse create_color_caster function we created above. To figure out the corresponding casting settings we can again inspect the pipeline file.

"ColorCasterSettings": {
    "MaterialChannel": "Roughness",
    "MaterialChannelUI": {
        "Visible": true
    },
    ...
    "OutputSRGB": false,
    "OutputSRGBUI": {
        "Visible": true
    },
}

From this we can see that the material channel's name is Roughness and it is in linear color space, non sRGB. With that knowledge we can create a material caster for our roughness channel.

roughness_caster = create_color_caster(sg, "Roughness", False)

Normal casting

There is no standard way of saving 3D data into normal maps. This means that almost every 3D creation tool and game engine have their own format and way of handling them.

First we need to ensure DefaultTangentCalculatorType is set accordingly. This specifies which tangent space method is used. If this is not set correctly generated normal maps will not be correct and lighting will look wrong. The settings corresponding to our integration can be found in the pipeline file we exported earlier.

"GlobalSettings": {
    "DefaultTangentCalculatorType": 3,
    "DefaultTangentCalculatorTypeUI": {
        "Visible": true
    }
},

DefaultTangentCalculatorType": 3 indicates that in our case Blender uses MikkTSpace tangent space method. Thus we need to set the calculator type before start processing assets.

sg.SetGlobalDefaultTangentCalculatorTypeSetting(Simplygon.ETangentSpaceMethod_MikkTSpace)

Now let's create a function for creating a NormalCaster. In addition to material channel name there are some more settings that can differ depending on integration.

• GenerateTangentSpaceNormals
• FlipGreen
• CalculateBitangentPerFragment
• NormalizeInterpolatedTangentSpace

sRGB should never be set for normal maps. If we would have casted normals with sRGB we would get distorted and skewed normals, in essence normals would not point in the correct direction. Another sign of that sRGB is incorrectly set for normal maps is that we get visible normals borders.

def create_normal_caster(sg, channel_name, flip_green, generate_tangent_space_normals, normalize_interpolated_tangent_space, calculate_bitangent_per_fragment):
    """Create a normal caster using specified settings"""
    
    caster = sg.CreateNormalCaster()
    caster_settings = caster.GetNormalCasterSettings()
    
    # Set casting settings specific per integration
    caster_settings.SetMaterialChannel(channel_name)
    caster_settings.SetGenerateTangentSpaceNormals(generate_tangent_space_normals)
    caster_settings.SetFlipGreen(flip_green)
    caster_settings.SetCalculateBitangentPerFragment(calculate_bitangent_per_fragment)
    caster_settings.SetNormalizeInterpolatedTangentSpace(normalize_interpolated_tangent_space)

    return caster

With the function in place we can inspect the exported pipeline file and find the correct values.

"NormalCasterSettings": {
    "MaterialChannel": "Normals",
    "MaterialChannelUI": {
        "Visible": false
    },
    ...
    "GenerateTangentSpaceNormals": true,
    "GenerateTangentSpaceNormalsUI": {
        "Visible": true
    },
    "FlipGreen": false,
    "FlipGreenUI": {
        "Visible": true
    },
    "CalculateBitangentPerFragment": true,
    "CalculateBitangentPerFragmentUI": {
        "Visible": true
    },
    "NormalizeInterpolatedTangentSpace": false,
    "NormalizeInterpolatedTangentSpaceUI": {
        "Visible": true
    }
}

With that information in mind we can create a normal caster with correct settings for Blender.

normal_caster = create_normal_caster(sg, "Normals", flip_green = False, generate_tangent_space_normals = True, normalize_interpolated_tangent_space = False, calculate_bitangent_per_fragment = True)

Result

The result is a aggregated mesh with only one material. The material contains base color, roughness and normals baked from the three input materials.

It is worth to point out that in Blender we get incorrect metallic value in material after import unless we have baked a map to that texture channel. So after exporting we unhook metallic channel from the Blender material shading network and set it to 0.

Which model is original and which one is aggregation? The author of this article forgot, but it does not matter as no difference can be seen. The only difference is that the aggregated model only requires one draw call as all 3 materials are now merged into one with roughness, normal and color channel textures seen below.

Complete script

# Copyright (c) Microsoft Corporation. 
# Licensed under the MIT license. 

import os
import bpy
import gc

from simplygon10 import simplygon_loader
from simplygon10 import Simplygon

file = "scene.glb"
temp_path = "c:/tmp/"


# Change parameters for quality
texture_size = 4096


def export_selection(sg, file_path):
    """Export the current selected objects into Simplygon."""
    
    bpy.ops.export_scene.gltf(filepath = file_path, use_selection=True)
    sceneImporter = sg.CreateSceneImporter()
    sceneImporter.SetImportFilePath(file_path)
    sceneImporter.Run()
    scene = sceneImporter.GetScene()
    return scene


def import_results(sg, scene, file_path):
    """Import the Simplygon scene into Blender."""
    
    scene_exporter = sg.CreateSceneExporter()
    scene_exporter.SetExportFilePath(file_path)
    scene_exporter.SetScene(scene)
    scene_exporter.Run()
    bpy.ops.import_scene.gltf(filepath=file_path)
    

def create_aggregation_pipline(sg, base_color_channel_name, texture_size): 
    """Create aggregation pipeline with color material caster"""
    
    # Create aggregation pipeline
    aggregation_pipeline = sg.CreateAggregationPipeline()
    aggregator_settings = aggregation_pipeline.GetAggregationSettings()
    mapping_image_settings = aggregation_pipeline.GetMappingImageSettings()
    
    # Set aggregation settings.
    # Specify that we want to merge into one model and remove all internal geometry.
    aggregator_settings.SetMergeGeometries( True )
    aggregator_settings.SetEnableGeometryCulling( True )	
    aggregator_settings.SetGeometryCullingPrecision	( 0.5 )
    
    # Set mapping image settings to generate new UV coordinates we can use for material casting.
    mapping_image_settings.SetGenerateMappingImage(True)
    mapping_image_settings.SetApplyNewMaterialIds(True)
    mapping_image_settings.SetGenerateTangents(True)
    mapping_image_settings.SetUseFullRetexturing(True)
    mapping_image_settings.SetTexCoordGeneratorType( Simplygon.ETexcoordGeneratorType_ChartAggregator )
    mapping_image_settings.SetTexCoordLevel(0)
    
    # Look at base color channel to determine chart proportions
    chart_aggregator_settings = mapping_image_settings.GetChartAggregatorSettings()
    chart_aggregator_settings.SetOriginalChartProportionsChannel(base_color_channel_name)    
    chart_aggregator_settings.SetOriginalTexCoordLevel(0)
    
    material_settings = mapping_image_settings.GetOutputMaterialSettings(0)
    material_settings.SetTextureWidth(texture_size)
    material_settings.SetTextureHeight(texture_size)
    
    return aggregation_pipeline


def create_color_caster(sg, channel_name, output_srgb):
    """Create a color caster using specified settings"""
    
    caster = sg.CreateColorCaster()
    caster_settings = caster.GetColorCasterSettings()
    
    # Set casting settings specific per integration
    caster_settings.SetMaterialChannel(channel_name)
    caster_settings.SetOutputSRGB(output_srgb)
    return caster    
    
    
def create_normal_caster(sg, channel_name, flip_green, generate_tangent_space_normals, normalize_interpolated_tangent_space, calculate_bitangent_per_fragment):
    """Create a normal caster using specified settings"""
    
    caster = sg.CreateNormalCaster()
    caster_settings = caster.GetNormalCasterSettings()
    
    # Set casting settings specific per integration
    caster_settings.SetMaterialChannel(channel_name)
    caster_settings.SetGenerateTangentSpaceNormals(generate_tangent_space_normals)
    caster_settings.SetFlipGreen(flip_green)
    caster_settings.SetCalculateBitangentPerFragment(calculate_bitangent_per_fragment)
    caster_settings.SetNormalizeInterpolatedTangentSpace(normalize_interpolated_tangent_space)

    return caster   


def process_selection(sg):
    """Remove and bake decals on selected meshes."""
    
    sg.SetGlobalDefaultTangentCalculatorTypeSetting(Simplygon.ETangentSpaceMethod_MikkTSpace)

    # Export scene from Blender and import it 
    file_path = temp_path + file
    scene = export_selection(sg, file_path)
    
    # Aggregate optimized scene
    pipeline = create_aggregation_pipline(sg, "Basecolor", texture_size)
    
    # Create material casters
    basecolor_caster = create_color_caster(sg, "Basecolor", True)
    roughness_caster = create_color_caster(sg, "Roughness", False)
    normal_caster = create_normal_caster(sg, "Normals", flip_green = False, generate_tangent_space_normals = True, normalize_interpolated_tangent_space = False, calculate_bitangent_per_fragment = True)

    # Add material casters
    pipeline.AddMaterialCaster(basecolor_caster, 0)
    pipeline.AddMaterialCaster(roughness_caster, 0)
    pipeline.AddMaterialCaster(normal_caster, 0)
    
    # Run pipeline and perform material casting
    pipeline.RunScene(scene, Simplygon.EPipelineRunMode_RunInThisProcess)
    
    # Import result into Blender
    import_results(sg, scene, file_path)


def main():
    sg = simplygon_loader.init_simplygon()
    process_selection(sg)
    sg = None
    gc.collect()


if __name__== "__main__":
    main()