Animating thousands of fish with MultiMeshInstance

Animating thousands of fish with MultiMeshInstance

ABZU for rendering and animating thousands of fish using vertex animation and static mesh instancing. Shader and a MultiMeshInstance. Using the following technique you can render thousands of animated objects, even on low end hardware. We will start by animating one fish. Then, we will see how to extend that animation to thousands of fish.

Animating one Fish

MeshInstance and add a new ShaderMaterial. Here is the fish we will be using for the example images, you can use any fish model you like. Note QuaterniusDev and is shared with a creative commons license. CC0 1.0 Universal (CC0 1.0) Public Domain Dedication https://creativecommons.org/publicdomain/zero/1.0/ Skeleton to animate objects. However, bones are animated on the CPU and so you end having to calculate thousands of operations every frame and it becomes impossible to have thousands of objects. Using vertex animation in a vertex shader, you avoid using bones and can instead calculate the full animation in a few lines of code and completely on the GPU. The animation will be made of four key motions:

A side to side motion
A pivot motion around the center of the fish
A panning wave motion
A panning twist motion All the code for the animation will be in the vertex shader with uniforms controlling the amount of motion. We use uniforms to control the strength of the motion so that you can tweak the animation in editor and see the results in real time, without the shader having to recompile. VERTEX in model space. We want the vertices to be in model space so that the motion is always relative to the orientation of the fish. For example, side-to-side will always move the fish back and forth in its left to right direction, instead of on the x axis in the world orientation. TIME.
//time_scale is a uniform floatfloat time = TIME * time_scale;
VERTEX.x by cos of TIME. Each time the mesh is rendered, all the vertices will move to the side by the amount of cos(time).
//side_to_side is a uniform floatVERTEX.x += cos(time) * side_to_side;
The resulting animation should look something like this: VERTEX by a rotation matrix for it to rotate around the center of the fish. We construct a rotation matrix like so:
//angle is scaled by 0.1 so that the fish only pivots and doesn't rotate all the way around//pivot is a uniform floatfloat pivot_angle = cos(time) * 0.1 * pivot;mat2 rotation_matrix = mat2(vec2(cos(pivot_angle), -sin(pivot_angle)), vec2(sin(pivot_angle), cos(pivot_angle)));
x and z axes by multiplying it by VERTEX.xz.
VERTEX.xz = rotation_matrix * VERTEX.xz;
With only the pivot applied you should see something like this: body. body is a float that is 0 at the tail of the fish and 1 at its head.
float body = (VERTEX.z + 1.0) / 2.0; //for a fish centered at (0, 0) with a length of 2
cos by the position along the spine, which is the variable we defined above, body.
//wave is a uniform floatVERTEX.x += cos(time + body) * wave;
body to offset cos each vertex along the spine has a different position in the wave making it look like a wave is moving along the fish. The last motion is the twist, which is a panning roll along the spine. Similarly to the pivot, we first construct a rotation matrix.
//twist is a uniform floatfloat twist_angle = cos(time + body) * 0.3 * twist;mat2 twist_matrix = mat2(vec2(cos(twist_angle), -sin(twist_angle)), vec2(sin(twist_angle), cos(twist_angle)));
xy axes so that the fish appears to roll around its spine. For this to work, the fish’s spine needs to be centered on the z axis.
VERTEX.xy = twist_matrix * VERTEX.xy;
Here is the fish with twist applied: If we apply all these motions one after another, we get a fluid jelly-like motion. mask. mask is a float that goes from 0 at the front of the fish to 1 at the end using smoothstep to control the point at which the transition from 0 to 1 happens.
//mask_black and mask_white are uniformsfloat mask = smoothstep(mask_black, mask_white, 1.0 - body);
mask used as COLOR: mask which will limit it to the back half.
//wave motion with maskVERTEX.x += cos(time + body) * mask * wave;
mix. mix allows us to mix the vertex position between a fully rotated vertex and one that is not rotated. We need to use mix instead of multiplying mask by the rotated VERTEX because we are not adding the motion to the VERTEX we are replacing the VERTEX with the rotated version. If we multiplied that by mask, we would shrink the fish.
//twist motion with maskVERTEX.xy = mix(VERTEX.xy, twist_matrix * VERTEX.xy, mask);
Putting the four motions together gives us the final animation. Go ahead and play with the uniforms in order to alter the swim cycle of the fish. You will find that you can create a wide variety of swim styles using these four motions.
Making a school of fish
Godot makes it easy to render thousands of the same object using a MultiMeshInstance node. School, because it will contain a school of fish. MultiMesh, and to that MultiMesh add your Mesh with the shader from above. Color. instance_count specifies how many instances of the mesh you want to draw. For now, leave instance_count at 0 because you cannot change any of the other parameters while instance_count is larger than 0. We will set instance count in GDScript later. transform_format specifies whether the transforms used are 3D or 2D. For this tutorial, select 3D. color_format and custom_data_format you can choose between None, Byte, and Float. None means you won’t be passing in that data (either a per-instance COLOR variable, or INSTANCE_CUSTOM) to the shader. Byte means each number making up the color you pass in will be stored with 8 bits while Float means each number will be stored in a floating-point number (32 bits). Float is slower but more precise, Byte will take less memory and be faster, but you may see some visual artifacts. instance_count to the number of fish you want to have. Next we need to set the per-instance transforms. MultiMeshInstance tutorial. The second is to loop over all the instances and set their transforms in code. Below, we use GDScript to loop over all the instances and set their transform to a random position.
for i in range($School.multimesh.instance_count):  var position = Transform()  position = position.translated(Vector3(randf() * 100 - 50, randf() * 50 - 25, randf() * 50 - 25))  $School.multimesh.set_instance_transform(i, position)
Running this script will place the fish in random positions in a box around the position of the MultiMeshInstance. Note If performance is an issue for you, try running the scene with GLES2 or with fewer fish. Notice how all the fish are all in the same position in their swim cycle? It makes them look very robotic. The next step is to give each fish a different position in the swim cycle so the entire school looks more organic.
Animating a school of fish
cos functions is that they are animated with one parameter, time. In order to give each fish a unique position in the swim cycle, we only need to offset time. INSTANCE_CUSTOM to time.
float time = (TIME * time_scale) + (6.28318 * INSTANCE_CUSTOM.x);
INSTANCE_CUSTOM. We do that by adding one line into the for loop from above. In the for loop we assign each instance a set of four random floats to use.
$School.multimesh.set_instance_custom_data(i, Color(randf(), randf(), randf(), randf()))
INSTANCE_CUSTOM to make them swim faster or slower by multiplying by TIME.
//set speed from 50% - 150% of regular speedfloat time = (TIME * (0.5 + INSTANCE_CUSTOM.y) * time_scale) + (6.28318 * INSTANCE_CUSTOM.x);
You can even experiment with changing the per-instance color the same way you changed the per-instance custom value. One problem that you will run into at this point is that the fish are animated, but they are not moving. You can move them by updating the per-instance transform for each fish every frame. Although doing so will be faster than moving thousands of MeshInstances per frame, it’ll still likely be slow. Particles to take advantage of the GPU and move each fish around individually while still receiving the benefits of instancing.