CG Basics

Triangulation

Technical background

We are used to doing our modeling with quads, meaning polygons that have exactly 4 sides. This, however, is only an abstraction and what we actually have are two triangles that are contained in this 4-sided polygon. All quads (and generally speaking, polygons with more than 3 sides) are always made up of triangles. You may not see it but your offline renderer, the viewport in your 3d application, and game engines all only use triangles.

Quads and consequently all larger n-gons (polygons with n sides) only exist as a modeling aid. It's more convenient to subdivide a quad than subdividing triangles where the direction of the flow of the mesh is not obvious. N-gons are also useful for capping flat holes where the exact way the polygon is triangulated doesn't necessarily matter. Using n-gons, however, is rather frowned upon and usually for good reasons. One reason is that the automatic triangulation can be very irregular and produce issues in certain circumstances.

All right, so far so good, but if we prefer to use quads anyways, why should we care about the underlying triangulation?
Let's look at a few examples and how the triangulation can affect our results.

When it suddenly matters

For this example, we will use a 2x2 quad plane and move the vertices of two opposing corners down. We can see that the triangulation of one of these corner quads is running towards us and one is rotated 90°. This has an important effect on the shape and thus the silhouette of our plane. The corner on the right is following the intended curvature, whereas the corner on the left produces a groove, originating at the middle of the plane. This breaks up our intended curvature, produces contradictory shadows and is thus most likely not what we want. So we need to flip this hidden edge to make it go the other way.
It not usually a problem with high-poly meshes and less extreme curvatures but it is much more likely to be a problem in game models, especially for mobile applications where the triangle budget is much more limited. So you should be aware that automatic triangulation can produce unintended results.

The next example depicts a typical example where n-gons can be problematic. The n-gon on the left is triangulated properly but the same n-gon on the right has improper triangulation which leads to overlapping polygons at the C-like cutout. This is very bad and should be avoided under all circumstances. Ngons with very few sides are easy to fix but it can be next to impossible to fix complicated shapes with all sorts of holes and cutouts (like in a typical CAD import).
In cases like this, it is advisable to add edges manually to prevent triangulation in concave shapes. In this example, you could simply connect the left corners to the corners in the middle and you would end up with quads again.

Another important case where triangulation matters is the export and import into other 3D applications combined with baked normals. The automatic triangulation between your content creation program and, for example, your game engine, will likely differ and suddenly you will end up with ugly normal map artifacts. This is due to resulting tangent space mismatches. The tangent space is calculated using those hidden triangles, so if those triangles change between your applications, the tangent space will change too. See the "Tangent space" chapter for a deeper explanation. This one of the reasons why you usually want to triangulate your model before baking and export