A Guide to Filament for Android

Filament is an open source rendering engine developed by Google. You can use it to render objects in real time. It is commonly used by games and AR applications.

In this tutorial I’ll show you how to use Filament to render a model created in Blender. I’ll also be using Jetpack Compose.

Setting up the project

To get started we need to add the dependencies. I’m using Filament 1.36, but you should use the latest version.

dependencies {
   implementation 'com.google.android.filament:filament-android:1.36.0'
   implementation 'com.google.android.filament:filament-utils-android:1.36.0'
   implementation 'com.google.android.filament:gltfio-android:1.36.0'
}

Now that we have Filament available, we need to initialize it. You can basically do this anywhere given it is before you use Filament. You can do it in your Application or MainActivity class. You just need to call Utils.init() and it will initialize Filament for you.

Utils.init()

Utils.init internally calls Filament.init, so you do not need to do that again. If Utils is not available, it is probably because you forgot to add the filament-utils-android dependency.

Filament may not work correctly on emulators, so try to use a physical device if you can.

Exporting the Blender model

You can create your models by defining all the vertices manually, but that is really difficult and time consuming. Another approach is to use a tool like Blender to create the models and then export them.

I will not teach you how to create models in Blender because there are hundreds of tutorials about that.

In this tutorial I’ll be using a cube I created in Blender. Pay attention to the two lights. If you do not add lights, your models will look dark. If you add lights, make sure you increase their power (W).

Go to File > Export > glTF 2.0.

Set the format to glTF Binary.

Under Include, select Punctual Lights if you included lights.

Under Data > Mesh, make sure UVs, Normals and Vertex Colors are selected.

After your model is exported you should have a .glb file.

Visualizing the model

Now that we have Filament initialized and the model, we can proceed to the next step.

To make sure your model was exported correctly, you can use websites like glb.ee or Babylon.js to visualize it. If it is rendered correctly on these websites, then Filament should have no problem rendering it.

Go back to your IDE and put the glb file under app/src/main/assets/models. If you do not have the assets or models folders, just create them.

I’m using Jetpack Compose and, to make my code more organized, I’m creating a class that is responsible only for rendering.

Let’s start by defining a few variables. We also need to observe the view lifecycle to dispose the components correctly.

class ModelRenderer {
    private lateinit var surfaceView: SurfaceView
    private lateinit var lifecycle: Lifecycle

    private lateinit var choreographer: Choreographer
    private lateinit var uiHelper: UiHelper

    private lateinit var modelViewer: ModelViewer

    private lateinit var assets: AssetManager

    private val frameScheduler = FrameCallback()

    private val lifecycleObserver = object : DefaultLifecycleObserver {
        override fun onResume(owner: LifecycleOwner) {
            choreographer.postFrameCallback(frameScheduler)
        }

        override fun onPause(owner: LifecycleOwner) {
            choreographer.removeFrameCallback(frameScheduler)
        }

        override fun onDestroy(owner: LifecycleOwner) {
            choreographer.removeFrameCallback(frameScheduler)
            lifecycle.removeObserver(this)
        }
    }
}

Below that, let’s define a method that will get called when the SurfaceView becomes available in the composition.

fun onSurfaceAvailable(surfaceView: SurfaceView, lifecycle: Lifecycle) {
    this.surfaceView = surfaceView
    this.lifecycle = lifecycle
    assets = surfaceView.context.assets

    lifecycle.addObserver(lifecycleObserver)
    ...
}

Below that, in the same method, let’s continue the configuration.

fun onSurfaceAvailable(surfaceView: SurfaceView, lifecycle: Lifecycle) {
    ...
    choreographer = Choreographer.getInstance()
    uiHelper = UiHelper(UiHelper.ContextErrorPolicy.DONT_CHECK).apply {
        // This is needed to make the background transparent
        isOpaque = false
    }

    modelViewer = ModelViewer(surfaceView = surfaceView, uiHelper = uiHelper)

    // This is needed so we can move the camera in the rendering
    surfaceView.setOnTouchListener { _, event ->
        modelViewer.onTouchEvent(event)
        true
    }

    // This is the other code needed to make the background transparent
    modelViewer.scene.skybox = null
    modelViewer.view.blendMode = View.BlendMode.TRANSLUCENT
    modelViewer.renderer.clearOptions = modelViewer.renderer.clearOptions.apply {
        clear = true
    }

    // This part defines the quality of your model. Feel free to change it or
    // add other options.
    modelViewer.view.apply {
        renderQuality = renderQuality.apply {
            hdrColorBuffer = View.QualityLevel.MEDIUM
        }
    }

    createRenderables()
}

The onSurfaceAvailable method is responsible for initializing the variables we need to render the model.

The createRenderables method loads the glb file we exported earlier. modelViewer.loadModelGlb parses the contents of the file and creates the entities in Filament. modelViewer.transformToUnitCube is needed to position and scale the model correctly.

private fun createRenderables() {
    val buffer = assets.open("models/tutorial.glb").use { input ->
        val bytes = ByteArray(input.available())
        input.read(bytes)
        ByteBuffer.allocateDirect(bytes.size).apply {
            order(ByteOrder.nativeOrder())
            put(bytes)
            rewind()
        }
    }

    modelViewer.loadModelGlb(buffer)
    modelViewer.transformToUnitCube()
}

Finally we define FrameCallback.

inner class FrameCallback : Choreographer.FrameCallback {
    override fun doFrame(frameTimeNanos: Long) {
        choreographer.postFrameCallback(this)
        modelViewer.render(frameTimeNanos)
    }
}

By default ModelViewer adds a directional light above the model, so even if you add no lights you will still see the top part of the model illuminated. Filament also only supports one directional light.

Now that ModelRenderer is finished, let’s go to the Compose part. I’m putting my composable inside my MainActivity.

class MainActivity : ComponentActivity() {

    companion object {
        init {
            Utils.init()
        }
    }

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContent {
            MaterialTheme {
                Surface(
                    modifier = Modifier.fillMaxSize(),
                    color = MaterialTheme.colorScheme.background
                ) {
                    AndroidView(factory = { context ->
                        val renderer = ModelRenderer()

                        SurfaceView(context).apply {
                            renderer.onSurfaceAvailable(this, lifecycle)
                        }
                    })
                }
            }
        }
    }
}

Now let’s run the app.

If your model looks like this, it is because it is missing lighting. Make your lights brighter in Blender. I’m using 9000 W for this example.

After making the lights brighter, the final result should look like this:

For some reason the background stays black on emulators, but if you run this on a physical device it will be transparent.

If you are using an emulator, make sure you enable OpenGL 2 or 3. Otherwise it will crash when rendering the material.

You can find the source code on GitHub.