Making a 3D shader outline in Godot

2021-03-07

Example from my big project Click for higher quality

For my big project I needed an outline shader for indicating things with errors and some other usecases like shown in the image above. However, no solution I found satisified my needs (all of them had artifacts of some sort or simply look wrong). I ended up writing my own solution which does give me a proper outline effect.

The code is available under the MIT license. It is also available on Github. I will describe below how the shader works.

Complications with Godot

Sadly, Godot doesn't make it easy to write a proper outline shader: it's impossible to customize the render pipeline, which means we can't add the extra information we need, let alone add a custom post-processing effect directly. Luckily however, we can use a second Viewport with a ViewportContainer, which allows us to use a custom canvas_item shader.

Gathering the necessary info

To determine what kind of info we need we have to determine what exactly we are trying to render. In this case, we want to draw a line around an object. To do that we need a method to detect the edges. What does have obvious edges? A white blob on a black surface!

Thus, we take all the meshes that need an outline and simply render them as plain white.

Output of rendering the meshes as plain white

Drawing the outline

We now need to detect the edges somehow. There are various ways to do this, but the method used in the screenshot works as follows:

From the blurred image we use the red (or any) channel as alpha value and set the color to whatever we like.

That's all there is to it!

Actually drawing the outline in Godot

Of course, things aren't that simple in Godot. To actually use it, we need to duplicate all the meshes that need an outline, put them in a Viewport and mirror the transforms of the camera and the MeshInstances every frame. I already did some of the work in the project I linked above.

Caveats

The shader as it is right now may not suit all needs. In particular, it disregards depth information as that isn't easily accessible (though it is possible to get this info with a custom Spatial shader) so the outline is always visible. It is also slow for thick outlines.

There are two alternative implementations present in the source code, one of which is much faster, but the results they produce are poor.

Addendum: drawing thick outlines

Instead of having a glow/emission outline you may want to have a thick outline. It is easy to do this with the above shader by simply rounding the alpha value (after multiplying it with 1.45...). Alternatively, you can comment out COLOR.a = ... in one of the alternative shaders.