What Goes Into Compiling a Source Map?

(November 16, 2017)

Every time you want to play a map you're editing, you'll need to compile it. Compiling, as you may know, is where you run your map source against a set of tools to make it playable in the game. You might have a rough idea of what each tool does, but what are the specifics? What are the tools really doing while they grind away? Hopefully, this guide can clear up what all the time spent compiling is used for.

VBSP

VBSP (Valve Binary Space Partitioning) is the first step in compiling and the most important of the three. VBSP consists of two main processes—processing the level geometry and setting up the entities in the world. VBSP is what creates the playable map itself, though it'll be missing a few key components if you try to run it.

Technical info

One of VBSP's first tasks is to build the level itself. VBSP does a form of preprocessing we can fancily call backface culling, which means that it cuts out the faces of brushes you can't see. In some instances, only one single face of a brush will get compiled into a map. VBSP stores the original brushwork in the file regardless, though it isn't used for rendering. Source map decompilers like BSPSource extract this data in order to recreate the level.

The BSP format stores everything in an array of over 60 lumps, each containing a different set of data. Notably, Lump 0 (LUMP_ENTITIES) contains all the entities in a level and Lump 40 (LUMP_PAKFILE) are any loose assets you might have zipped into the BSP. Lump 18 is LUMP_BRUSHES, where all the original brushwork is kept.

VBSP makes a giant list of all entities in the level and stores it in Lump 0. Technically, the world is the first entity. If you die by falling in a game and check the console, you'll see something akin to:

Player was killed by worldspawn

Boredom, destroyer of worlds

It's not exactly useful, but you can try removing worldspawn in the console with the command ent_remove worldspawn. The game will then crash. This is because you literally just removed the world itself.

VBSP then prepares a portal file. This doesn't have anything to do with the game Portal, but it's a nice way to get high-detail maps to run well on sub-par hardware. Originally designed in the Quake days, each brush splits the world up into invisible boxes called visleaves. It's called a portal (PRT) file because each face of a visleaf is either the surface of a brush or a "portal" to another visleaf. Only the visleaves (and the things in them) that the player can see into will get rendered.

VBSP then begins to take care of the entities in the map. A certain class of entities known as internal entities get merged into the world and stop existing altogether. These are generally entities that make visual changes to the world, such as overlays.

As for entities that do exist after compile, VBSP also sets them up in their proper locations. VBSP also takes care of any cubemaps that exist. If a texture is set up to be shiny, VBSP links them to the nearest cubemap for when those are built in the engine.

Patchwork

When you compile a map, the original versions of any shiny textures you might've used don't actually exist in the final map. Instead, versions of the original material (VMT) file are embedded, each with their $envmap (linking a texture to an environment map, or the thing the texture reflects) parameter changed and the rest of their parameters inherited from the original. This is done through a special shader called the Patch shader, which allows textures to inherit values from other textures.

At this stage, VBSP's output looks a little like this.

There's no lighting and no visibility control present in the map. That's what the other two tools are for.

VVIS

When VBSP is done, it passes the map file onto VVIS (Valve Visible Information Set), the second tool in the process. VVIS makes maps run much faster by testing which parts of the level can be seen from where. If something can't be seen from another part of the map, the game won't render it when you're playing it.

Technical info

As we said, brushes (or more specifically, their planes) split the playable game world up into visleaves, which are essentially big, empty volumes. VVIS tests visibility between these volumes and writes that data to Lump 4, LUMP_VISIBILITY. When it comes time to render the world, the game can avoid rendering most of it by reading out of that lump and ignoring the visleaves players can't see in at that moment.

In the above diagram, visleaves are numbered. The yellow lines are the invisible portal boundaries where two visleaves meet. Because of the way this room is set up, visleaf 4 can't see visleaf 5. If the player is in visleaf 4, 5 won't be rendered, and vice versa. Visleaves 1, 2, and 3 are visible from anywhere in the level. Setting up your levels so the fewest number of visleaves are rendered at any time is a good idea.

Only world brushes split visleaves. Point entities, brush entities, displacements, and brushes that have any side of their figure textured with a $translucent texture are ignored when VBSP makes its portal file. In turn, these don't affect visibility. This is why the func_detail brush entity is so handy. Because VBSP completely ignores it when it creates its portal file, VVIS will work faster, and the game will have fewer visleaves to worry about. The entire process is sped up.

VVIS likes things simple. If you're looking to speed up your compile, be sure that you make good use of detail brushes, keep things neat and in dimensions of powers of two as much as you can, and make open areas only as large as they need to be. Long, unbroken sections of the map are notorious for slowing VVIS down.

VVIS' output can be best visualized like so:

VRAD

The final step in the compile process is VRAD (Valve Radiosity), which calculates lighting and light bounces. Because of the amount of calculations it requires, it takes the longest of the three tools.

Technical info

Basic light and light_spot entities don't actually light up the game world. Instead, their effects are compiled into a new texture called a lightmap. There's a lightmap for every brush face visible in a map. The color values in each texture are multiplied by the color values in the lightmap, causing it to appear brighter or darker. If a lightmap has a perfectly dark patch (or a color value of zero), multiplying the color in the texture by zero will make that face appear perfectly black in-game. Lightmaps only apply to brushes, not props.

The above picture has removed all diffuse textures with mat_fullbright 2, leaving only the lightmaps on the walls. Lightmaps are written to Lump 8, LUMP_LIGHTING.

VRAD is different from Quake's LIGHT program in the way that it calculates radiosity, or light bouncing. In the real world, rays of light bounce around on nearby surfaces as they peter out, and a radiosity algorithm simulates those bounces.

In Source's implementation, faces are split up into smaller surfaces called patches, and all the patches in a scene are paired at random and their visibility from one another (their form factor) is calculated. From there, it's simply a matter of solving a system of equations, with the various form factors brightening patches if needed. A bright patch that can see another patch very well will "bounce" light to it.

In short, the floors and walls get cut up into tiny bits, and the visibility between bits is calculated. If a bright bit can see a dimmer bit, it'll brighten it up. This is how radiosity is calculated. The results are burned onto lightmaps and packaged into a map. Dynamic lights, such as the player's flashlight, work differently and VRAD doesn't control them. The game will brighten whatever the flashlight hits while the game is running.

You're just projecting

The flashlight is actually a texture projected onto world geometry using the same entity as any lamps you spawn in Garry's Mod: env_projectedtexture.

Raising lightmap scales and using fewer and brighter lights to achieve the effect you want can help in lowering the amount of time it takes for VRAD to do its thing. The final results of VRAD's work:

There's a lot that goes into compiling a map. Now you know why it takes so long.