State of the art for base building system in 2024

I'm really sorry that this article is that long, it summarize 5 months of work and tons of things done.

Ardaria is designed with builders in mind, emphasizing freedom of design and speed as core values in crafting our systems. Our base building system stands out for its flexibility and depth, enabling players to bring their creative visions to life fast and precisely, regardless of the design's size or complexity.

In this article, we will explore the reasoning behind our software design choices and present our solutions. Following this, we will provide technical details of the system. This will include discussions on procedural meshes, runtime boolean mesh operations, spatial queries for lights, visual and sound effects pooling. In a broader context, we will explore techniques for seamlessly displaying hundreds of runtime custom designs at a smooth 60 frames per second within Unreal Engine 5 with 0 baking.

Our ultimate vision is to establish an online platform that helps players create and share schematics, enabling the development of large cities that are fully interactive and populated with NPCs.

Imagine a scenario where a group of players collaboratively builds a city as vast and majestic as Minas Tirith from "The Lord of the Rings." In this virtual city, visitors have the opportunity to explore every building and interact with hundreds of players and NPCs simultaneously.

Ardaria’s approach overview

Traditional building systems usually follow a component-based approach, requiring players to put together blocks of different sizes. However, this method has its own limitations, making it difficult to create detailed structures. In games like Minecraft, architectural designs often look more like pixel art than technical drawings. Similarly, building in Lego Fortnite, and Rust involve trying to fit predefined components together.

In contrast, our procedural component system empowers players to create adaptive parts of any size and shapes at ease, with literally just two clicks.
The concept of fixed block sizes, such as the 1 cubic meter blocks in Minecraft or 2m x 2m panels in Rust, is transcended. Players can modularly shape their structures with the precision they desire, breaking free from the constraints imposed by predefined block dimensions. The freedom philosophy in designing aligns with our commitment to encourage creativity and allowing players to bring their architectural visions to life.

Adaptive walls, roofs and floors

The philosophy driving our approach to floors, roofs, windows, and doors is centered on speed and flexibility in design and edition. We draw inspiration from level design blocking out, the key distinction lies in the outcome—users seamlessly transition from a rapid design phase to the creation of a aesthetically pleasing building. This ensures that the creative journey is both efficient and rewarding.

This is where the procedural mesh technology excels. Unreal Engine 5 introduced a new kind of tools called “Geometry Scripting Tools” along with dynamic meshes. It's a set of tools designed to easily create procedural meshes and edit them during runtime. This is part of the effort to integrate 3D modeling tools into Unreal Engine 5 which prevent artists to go back and forth between Blender ↔ UE.

With this new toolset, we can effortlessly create polygons (or any shape like curved stairs) and then apply a series of optimizations that we will detail later in the article. This capability empowers us to generate meshes of any size and shape with just 2 clicks.

Optimization involves baking procedural meshes into static mesh for rendering performance improvement.
Further development involves triangulation and use of GPU instanced nanite meshes.

Materials customization

We added a set of 40 different materials to customize walls, floors and roofs to be able to create different architectural styles.
We used the excellent materials from freestylized.com.

Optimization include using shader primitive data and texture atlases.
Future development involve color customization and direct shader attributes edition.

Windows and doors

Traditionally, incorporating doors involves designing specific spaces or using predefined door frame panels. However, with the introduction of geometry scripting tools in Unreal Engine 5, we've adopted a revolutionary approach. These tools empower us to dynamically edit meshes at runtime, enabling complex operations like mesh boolean (subtracting two meshes).

This shift in approach provides a fully modular method for building creation. Doors and windows can be positioned anywhere on a wall, giving players unprecedented freedom in precise placement. The process is as simple as aiming, clicking, and witnessing the transformation—a window appears while the wall seamlessly adjusts to accommodate the new part. This not only enhances the precision of placement but also elevates the overall user experience, making building design a dynamic and accessible endeavor. Removing a door is also extremely easy; the wall gets regenerated, and the hole gets filled.

Props - weapons / food / decoration

In our vision of a dynamic and immersive game world, we acknowledge that a house or base involves more than walls and roof —it's about the life within. To bring this vision to reality, we've incorporated over 300+ props into the game, covering crucial elements like food, tools, weapons, and a variety of decoration items such as lights, carpets, books, etc… What distinguishes our system is its centimeter precision location-based approach, enabling players to position these props exactly where they want to place them with absolute freedom.

The optimization process will be discussed further in the article and includes GPU instancing using hierarchical instanced meshes registries and spatial pooling.
Further development involves modding by letting player add props from modpacks at runtime by using Unreal Engine new feature: mesh server streaming.

Interactive complex compositions

Certain components can't be represented as a single prop. This happens with torches, storage, or any item with particles, sound, light, or moving parts like doors or lights.

Optimization process include pooling components using spatial queries to only display close to the player effect and disable everything else.

Import / Export aka collaborative construction

First step is to create data structure which can hold all the necessary information to rebuild houses and cities from serialized data. Such a format make it easily shareable as a JSON file.

The current export tool let the player selects a 2D area in the game which will scan for any components within its bounds and creates an array of serialized components to finally export it to a JSON file.

The import tool will let you select a JSON file, parse it and import each component into the world by saving it into the world chunks. Before import, you can rotate the schematic.

Establishing an online library where players can share and access plans is crucial to initiate collaborative community driven cities. For now, the rudimentary version of the library is hosted on steam and discord.

Optimization will be compression and further dev is a online marketplace where players can monetize their schematics.

System overview conclusion

By using procedural parts, props, static meshes, lights, VFX and SFX, we can create an absolute infinity of design variety and let the players create the worlds they will play in.

Technical insights

Challenges

While prototyping the system was completed within a week or so, the initial polishing stage and scale up have proven to be quite challenging in terms of software design for a team creating their first game.

1. Unreal's approach to runtime components involves using actors for everything. However, this method results in a massive tree with tens of thousands of actors, making it very challenging to track, save, load, export, and share. ECS design patern is our solution to this issue.
2. Geometry Scripts generate dynamic meshes intended to be altered each frame and uses the dynamic render path instead of the static render path. As a result, vertex caching is neglected, making draw calls very expensive. Also, lumen does not work with dynamic meshes. Geometry Scripting FAQ
3. Attempting to spawn thousands of lights, VFX, and SFX is not feasible at all. The volume of ticks and draw calls becomes immense and component pooling must be used.
4. Every procedural part must be rebuild from the simplest set of data, in our case we use the component class which has a set of overridden function depending on the part and an array of vertices to rebuild the polygon with the geometry script tools.

Data structure and ECS design pattern + components pools

Actor vs Scene Components

In Unreal Engine everything is an actor. An actor is a very wide class which is used to display an element in the world. An actor has many things we don’t need to instanciate when creating a build component while a component (Unreal documentation). The simplest class Unreal offers for an object with visual representation in the world is a scene component. Every build component is a scene component which get attached to it’s chunk actor and positioned relatively to the chunk.

ECS design pattern

Having explored Bevy.rs ECS in the past, it's clear that an efficient multithreaded system is crucial for scalability. Ardaria's goal is to populate cities with NPCs that can make decisions, engage in business, and establish an emergent economy. Part of the crafting system involves creating factories with tons of automation. These game design choices necessitate substantial processing power and call for highly modular game components.

Unreal Engine introduced an ECS framework called “Mass,” as demonstrated in the matrix demo of Unreal Engine 5. Mass offers a comprehensive set of tools to design cities teeming with thousands of NPCs. It serves as a robust alternative to the existing UE AI system, with full multithreading capabilities. Mass Framework overview

Building components can possess specific functionalities, such as doors that can open and close or torches that can be ignited. Additionally, static elements like walls, floors, and decorations could incorporate durability or integrity systems, making them a good fit for an ECS.

By utilizing Scene Components instead of actors, it's essential to maintain a clear and developer-friendly hierarchy tree. To streamline the development process, we opted to emulate an ECS system using blueprints, reducing the transition time from the initial blueprint-based iteration.

This approach results in a unified build component (Entity) that can manifest as different types—static mesh, light, VFX, and SFX (Component). These components can execute specific tick functions (Systems), such as rotating a static mesh by 110 degrees to simulate a door opening.

Data Structure

We must efficiently store the data in saves but also as an exportable and exchangeable format. We can’t store meshes triangles and must reconstruct them every time we load the game.
We choose JSON as data format as it’s extremelly common and understood format for data. It’s easily parseable and plenty of tools exists for it.
Data tables uses internal ids (GUID) to refer to component types and each individual component is stored using the following format.

{
  spawnableId: GUID, // refers to the datatable row id
  chunkId: GUID, // which chunk in the world the component is relative to
  worldTransform: { // where the element is on the plot
    location: Vector3D,
    rotation: Quaternion,
    scale: Vector3D
  },
  vertices: Vector3D[], // reconstruct a procedural part (walls, floors and roofs)
  materialId: GUID, // custimazation for procedural components
  parentId: GUID, // windows and doors are attached to a wall
  elementId: GUID // the id of the element itself
}

Saving/loading Entities

Our initial strategy involved letting the player place circle plots on the map, restricting construction within these designated areas. However, this approach encountered challenges, especially when attempting to position adjacent plots. To address these issues, we have transitioned to a more flexible system. Players can now position components freely across the map, each assigned a unique chunk ID. These chunks measure 32x32m and are saved into memory only when a component is added to them. The workflow entails find or create a chunk, assigning an ID if not already present, adding the component, and then saving the chunk.

Unreal save system is extremely fast to serialize structs into binary data. Nonetheless, as our world expands significantly, potential challenges may arise. Adopting a by-chunk saving approach, rather than saving the entire world at once, could offer a viable solution.

During the world-loading phase, we stagger component creation across multiple ticks to expedite the interactive experience for players. Currently, all chunks load simultaneously, resulting in a surge of component creations for expansive worlds. Our next strategy involves prioritizing the loading of chunks based on proximity, loading those nearest to the player first.

Component pools

Certain components within Ardaria, such as lights, VFX, and SFX, consume a considerable amount of resources. All lights in Ardaria operate as dynamic lights, aligning with the game's runtime level design. Upon creation, each light component registers with a designated pool. This pool enforces a maximum light count at any given moment, with the option to adjust this setting for enhanced performance on high-end systems. Periodically, the pool calculates the distance between each light and the player, determining the optimal placement and configuration based on our component configuration cache, which will be detailed in the subsequent section.

Currently, this system operates sub-optimally, consuming approximately 6ms every second. Significant enhancements could be achieved by transitioning to a full C++ implementation and leveraging Unreal's built-in classes for spatial queries, rather than recalculating the distance to the player continuously.
Geometry Scripting FAQ

Relying solely on actors for spawning components would quickly become chaotic, especially considering that a single house could feature dozens of torches.

Component configuration

In Ardaria, we have two main types of items: static meshes and compositions. Think of compositions like a recipe to create a complex component. They're made using blueprints in Unreal Engine. For example, a torch isn't just a single mesh. It's made of a static mesh, a light, fire particles, and a fire sound.

As we feel configuring in the blueprint’s viewport is the fastest and easiest way to adjust each component, we kept the Unreal Engine way to do but we can’t spawn a blueprint actor for every torch. Thus, we spawn every kind of composition at game starts, but make them invisible, and we cache the configuration as a struct.

{
  intensity: float
  attenuationRadius: float
  castShadows: boolean
  lightColor: Vector4
  vfxSystem: objectReference
  sfxSystem: objectReference
}

When spawning a new composition, we query the cached settings and recreate components in the new entity and then configuring them. In the case of a component which should be using the pool, we don’t directly create the component but rather add an entry to the corresponding pool.

To make it easier for us, we set up these blueprints below the game world and then save how they're set up. This saved setup is like a checklist that tells the game how each part of the blueprint should act.

This way, technical artists can create blueprints directly in the editor without having to worry about the code.

Hierarchical Instanced Static Meshes aka GPU instancing

In a similar way to pools, static meshes are not directly rendered within the Entity’s main Scene Component as a Static Mesh Child Component but they are rather sent to a Hierarchical Instanced Static Mesh (HISM) owned by the city actor (the main parent actor in the game). Every static mesh has nanite enabled and is an instanced static mesh. It means that the number of draw calls we are making to render every single prop in the game is linear with the amount of unique different props we have. Rendering 1 door or rendering 10,000 doors comes at very similar performance cost.

This way, we can render cities with a virtually unlimited amount of props. For instance, HISM is what Unreal team decided to use for their Procedural Content Generation (PCG) framework. So it’s battle tested and will receive optimizations in the future.

Procedural meshes

Procedural meshes are the killer feature of Ardaria base building system. They are the ultimate precision designing tool to easily create complex architectural shapes.
Geometry script tools provide an easy way to create runtime procedural meshes. The process is as follow:

1. Create a dynamic mesh
2. Create a polygon
3. Edit vertices of the polygon depending on the shape choosen by the player
4. Finished !

But dynamic meshes are the biggest performance sink and an optimization nightmare since the very first day of development.

Using dynamic meshes is terrible for performances as they use the RHI dynamic render path which will send all the vertex positions to the GPU for every single drawcall. It means that dynamic meshes don’t use caching for rendering, which make the render path very slow and impractical for large structures. A big house with 200 procedural mesh component would bring down the FPS to 30/40. Geometry Scripting FAQ - Will DynamicMeshComponent be as efficient as StaticMeshComponent in my Game?

The only solution is to bake dynamic meshes into static meshes. The solution to do this at runtime is to use mesh descriptions, a specific class used to describe a static mesh.

The process is straight forward:

1. List all the vertices in the dynamic mesh
2. Create vertices in the mesh description
3. Create vertex instances and UVs
4. Triangulate and recompute normals
5. Init the bake

It’s a game of looping and copying information from one format to another.

Static mesh baking from static mesh description is a multi threaded operation and is blazing fast. In the editor, it’s possible to also bake the nanite mesh, but not at runtime in a packaged game. Which lead us to a huge limitation: draw calls.

Every single procedural part become a unique static mesh and thus, inevitably leads to an additional draw call. As the mesh is not nanite baked, it also adds more occlusion tests and will eventually affect performances.

Our current benchmarks are giving a max city size of 300/400 structures thanks to frustrum culling which do not render anything outside of the camera bounds. So 100 structures on screen at any given time.

Once the bake is done, we don’t delete the dynamic mesh as it’s the source of truth for collisions. As we are doing mesh boolean operations, simple collisions are not working anymore and we must use the complex collisions. The dynamic mesh is used solely as a collider component (ECS) and the static mesh is used solely as a renderer component (ECS).

We have an idea to push the procedural parts further and making an ultimate optimization that we will discuss later in the article.

Procedural mesh textures

Procedural meshes can be customized by changing their texture, we added 40 different textures to the game.

In Unreal Engine, to display a texture on a mesh, we use an object type which is called a Material which is an API to write shader code. Using a different material for each custom texture was hard to manage and prevented us to further optimize the procedural meshes.

Instead, they all share the same shader (material) and are using primitive data per mesh to inject variables into the shader. Combining this technique with texture arrays, we can render every single custom texture with a single material/shader.

At game start, we cache the parameters of each instance to create a library of 40 material settings which can then be referenced by their id and get their parameters applied to the shader following a player action of changing the texture.

More optimizations

Some optimizations are on the conception stage.

Procedural parts into nanite GPU instanced static meshes

While our approach to building offers unparalleled flexibility and precision, it comes with a notable downside as our method generates a substantial number of individual meshes. Each mesh translates to a distinct draw call and to an important amount of occlusion tests.

The ultimate step in the optimization of procedural parts is to re-triangulate every static mesh into right triangles, which could then be rendered using GPU instancing in a single draw call. The goal is to represent any shape with a single deformed shape. As any triangle is the sum of 2 right-triangles, this shape is the elementary brick.

When using a single shape, and thus, a single static mesh, coupled with our single-shader approach for customizing textures, we can then use GPU instancing to render a whole city walls, floors and roofs.

I can’t wait to render a whole city within a single draw call, while expectations are high, we can’t benchmark yet and give numbers, but as UE can render hundred of billions of triangles at 120 FPS, there is definitively a lot of potential.

By embracing this forward-looking optimization, we aim to strike a balance between the intricacy of our building system and the demand for rendering efficiency. The goal is to provide players with expansive and visually stunning virtual environments, ensuring that the creative potential of our system is not hindered by performance constraints.

The downside of this optimization step is the high memory footprint and the initial computational power necessary to re-triangulate and cache right-triangles. We favor the FPS metric over the memory footprint, software programming is always all about trade-offs.

Light LODs and aggressive caching

Unreal Engine uses 3 systems for runtime dynamic lighting.

1. lights from DirectX and Vulkan API: point lights and spot lights. They are hiting performances very fast when overlapping. 100 non-overlaping points lights render at 120 FPS, 100 overlaping point lights render at 30 FPS (on RTX 2070).
2. Global illumination which can be screen space or lumen.
3. New virtual shadow map which caches the shadows. Dynamic lights can force usage of virtual shadow map.

As dynamic point lights are pretty expensive, culling strategies are aggressive and lights will fade out with distance pretty quickly. It lets structures distant from the player in the dark. Imagine a street in the night where houses are not lit 100 meters away. This is the current situation in Ardaria and prevent us to create a convincing and immersive day/night cycle.

One imagined solution would be to create Level of Details (LODs) for lights. It would consist in a merging algorithm for near lights in order to always illuminate a house from outside with a limited quantity of point lights. This algorithm would also take care of overlapping by computing the optimal quantity of light and their attenuation radius.

Lighting in video games is already a tough subject which is crossing between art and engineering, but user generated dynamic runtime lights are a order of magnitude harder.

Static mesh merging and LODs

We currently render wall cuts (a wall with a hole for a window) at far distance while we cant see in the hole of the wall. A simpler mesh could be used at far distance to reduce draw call complexity and consequent occlusion culling tests. Transfering everything to nanite enabled gpu instanced meshes should also do the trick. This is where our biggest effort must be done to increase scalability in the system and create hyper or infinite scale cities.

Textures and shaders

While props deriving from different packs (full list in the credits screen), they utilize a variety of shaders and textures. To streamline performance, we're actively working on merging and combining these textures using texture arrays and replacing multiple shaders into a single shader.

This optimization stage is pivotal in ensuring a seamless and visually cohesive experience for players, allowing them to enjoy the diversity of props without compromising on performance.

More features 2024/25

While the building system is usable, optimized and not considered in the prototyping phase, more developments must be done to make it the best base building system in the video game history.

Schematics Marketplace

Ardaria aims to be the ultimate sandbox where community tools are integrated in the base game. Our vision as game developers is to offer a rewarding and fun experience to players. Our next mission is to create a cooperative building experience and while the schematics sharing is the first step towards collaborative building, we lack a platform where players can share, rate, comment and recommend other players skills as builders and architects.

The marketplace will be seamlessly integrated into the game with the ability for creators to upload their schematics and participate in the curating and reviewing system, directly from the game interface.

Also, players can crawl the online marketplace library to find schematics they want to download or buy as we will offer monetization opportunities for builders. A preview tool will teleport the player on a separate map in order to explore the schematics before deciding to import it in their map.

Creative flatmap

Current Ardaria map is an island with procedural forest (PCG) in a mountain circus. This is not optimal to build large structure and an effort must be done to provide players with a flatmap, accessible within the schematic tool in order to create structure independently from their survival map.

Gizmo tool for placement adjustments

A gizmo tool will let the players precisely adjust the location and rotation and the components if they feel the need to do so. It’s a quality of life feature which has been asked by the community.

Schematics UX

We plan to introduce a set of tools which let the player preview the schematics and adjust precisely its position and rotation before validating the import. A rollback feature will be implemented in case of a mistake in order to completely revert the import.

More tools and procedural parts

What about cutting a wall with a custom mesh boolean? Or directly design a polygon to make a very specific shape? We could extend our procedural component system further to give more freedom for the most exigent creators.

As you read before, lighting is a critical part of game performance. A light complexity feedback tool to check to let the player know about performance impact will be valued.

High customization

The color of the light is not orange enough? Or maybe you just want some blue? Should the light more intense? Do you want to dye that wooden wall?

Edit history

Nothing is most frustrating than making a mistake which would take a long time to fix. Imagine destroying a wall with 10 windows, you would have to replace the wall and place all the windows once again. It’s precious minutes lost where you could have been improving your building instead.

A Ctrl+Z style edit history is mandatory for a smooth and rewarding player experience. It would pave the way for more control over the loading of schematics, letting players create time-lapse videos of their creations.

Multiplayer

Mutliplayer is mandatory to build large scale (1000 of buildings) collaboratively and it introduces the possibility to transform Ardaria into a MMO, which is our ultimate goal with our game.

Current multiplayer system in UE only supports few players (10_20) but Epic effort to bring their Fortnite multiplayer system in the engine (Iris) would enable 100 concurrent players.

We will start to replicate the build system and bring coop servers first before any upscaling.

Modding

This is the ultimate feature for the Ardaria building system. Modders could create packs of props to be imported into the game so Ardaria would become a platform by itself, allowing players to create worlds with a totally different artistic direction. Or simply improve some aspects of the game they feel incomplete.

The current idea we have is to use the upcoming mesh streaming Unreal Engine feature which let the meshes to be hosted on a remote server and downloaded by the player at runtime.

As we are community and player maximalists, we will use the marketplace to let the community choose which modpacks are worth the integration into Ardaria.

Job opportunity !

We are hiring a senior engineer to be in charge of the next 2 years of development of the building system. This might be you! FILL OUR FORM

Your role will be to lead the development of the best building system ever made in the video game history in every single metric. Here is the roadmap you will be part of:

• Schematics ingame library and integration with online marketplace
• Transitioning from baked static meshes to HISM for procedural parts
• Migration from Blueprints to C++
• Improving component pooling performance
• UX improvement for intuitive usage
• Edit history - Ctrl+Z
• Integration of Mass ECS framework
• Compression algorithm for schematics files
• Scaling up city system - create multiple cities on an infinite procedural world
• Optimizing world saving and loading
• Integration of survival mode for building system
• Material customization features
• Shaders and texture optimization
• Mesh LODs system
• Lights LODs system
• Modding API
• Integration of more building components such as machines and industry system
• System replication for multiplayer
• Navmesh and NPC navigation in user generated cities

Conclusion

Ardaria offers players the opportunity to create detailed cities and structures with ease. With its advanced technology, including tools like procedural meshes and dynamic component configurations in Unreal Engine 5, Ardaria ensures a rich and immersive experience. We are dedicated to building a supportive community where creativity flourishes and every player's architectural vision comes to life.