World Building in VR/VRChat - Vol 1 - Into Unity

Feb 1811 min read
A : Introduction
What is this all about ?

Building a “virtual” place is one of the great exercises for innovation and problem solving. For this document, we will be focusing on the main process of VRChat World Creation and Building, however, these steps or implementations can be applied to other platforms and environments.

B : Contents and Formats
What does this document have ? What to expect and how to interpret it ?

This document can be separated into the following sections : 
Setting up the Unity Project
Importing Assets into Unity
Unity Material creation
Unity GameObjects Configurations
Optimization
Uploading the project and Maintenance

Throughout this document, there will be 2 types of “links”, or citings, which will be denoted with “[Note:#]” and “[Ref:#]” where “#” is a number. “[Note:#]” represents that a certain segment requires more context or explanation. “[Ref:#]” represents that a certain segment is a reference from an external source, this can be treated as “citing a source”.
Throughout this document, there will be examples provided, these examples will be colored green.

1 : Setting up the Unity Project
Time to put everything together

Before opening Unity, a better way to set up a VRChat project (in this case a world) is to use “VRChat Creator Companion”, or for short, “VCC”. VCC is a software provided by VRChat in order to handle all your VRChat projects in one place [Ref:1] .

All VRChat Projects can be seen as multiple parts, some parts require other parts in order to work, some parts may have available updates. VCC handles all the needed parts, downloads them, and installs them. If a part needs to be updated, VCC can also update the selected part, and every process will be taken care.

Once VCC is installed and initiated, the application should look like Figure 101. On the top right corner, the button “Create New Project” allows the creator to initiate a new VRC project, this will bring the creator to Figure 102. 

[Figure 101 : The Main Layout of VCC]

In Figure 102, there are 4 modules, “Worlds”, “Avatars”, “Avatars 2019” and “Worlds 2019”. For this document, we will be focusing on “Worlds”, or more accurately, “Worlds 2022”, since this module requires Unity 2022.3.22 [Ref:2] in order to fully function. Next, the creator will have to decide the name and the location of the project. When all of the mandatory information is set, the creator can continue by clicking “Create Project”. This will bring the creator to Figure 103

[Figure 102 : The page to configure the information for the new Project]

In Figure 103, this page contains all the information of the project, and the parts (mentioned earlier) of the project, when all the required parts are installed, the creator can go back to Figure 101 by clicking the “Projects” button on the top left and then start having fun by clicking “Open Project” in the row of the created project.

[Figure 103 : the different components of a VRChat World 2022 Project]
[Ref:1] 
VCC is available for all VRChat Users to download. ||
VRChat Home (Date Not Available). VRChat Home. Available : https://vrchat.com/home/download

[Ref:2]
All VRChat Projects will require a SPECIFIC Unity version to work with, in this case, Unity 2022.3.22 is required. || 
Unity (No Dates Available). Unity Download Archive. Available : https://unity.com/releases/editor/archive

2 : Importing Assets into Unity
Time to put everything together

A typical Unity Project should appear to be like Figure 201. In the figure, every panel (or sections) can be identified by the label (or name) on their buttons (or tabs). The panels in the figure that can be identified are “Scene”, “Project”, “Hierarchy” and “Inspector”. There are other panels that are hidden, but available. However these 4 panels are the most important panels through out the project.

[Figure 201 : A typical Unity Project opened in Unity 2022.3.22 editor]
Scene panel
Shows the overall layout and view of every object in the World. It is also where objects can be placed visually and intuitively by the creator.

Project panel
Lists all the assets that can be used in the project. This is where the creator can import the assets that they have gathered (mentioned in Sections 5,6, and 7)

Hierarchy panel
Lists all the objects in the world in a tree structure to show that objects can be affected by another object based on the structure of the hierarchy. This also allows the creator to group certain objects as a whole.

Inspector panel
Displays the parameters of different components that make up or are applied to the object. This is also where some of the parameters can be adjusted or configured exactly by inserting exact numerical values.

3 : Unity Material creation
I'm gonna make that thing look SICK

In Unity, every mesh can take up a visual appearance, based on the Shader. Every “look” will be treated as a “Material”. Material in Unity is a file such that it defines how an object will appear on the screen. 

Figures 301, 302, and 303, are all examples of what a material can be and stored in the project folder and displayed on the Projects panel. These materials are soon to be applied onto Mesh objects afterwards.

[Figure 301]

[Figure 302]

[Figure 303]

Figures 304, 305 and 306 shows that these Materials are configurations of parameters based on different shaders [Note:1] .

[Figure 304]

[Figure 305]

[Figure 306]

Figure 307, 308 and 309 shows an example how the object with that material will appear to the visitors. 

[Figure 307]

[Figure 308]

[Figure 309]
[Note:1] Detailed Material Configuration will not be covered in the document as this might derail from the original purpose of this document.  

4 : Unity GameObject Configuration
Time to put everything together

In Unity, every item in the Hierarchy panel is called an “GameObject”. Normally, there are 5 types of GameObjects : 
Statics
Dynamics
Instantiated
Pickups
Interactables

Statics
These are GameObjects that do not move or change. Usually, these are objects to represent architectural structures, terrain, and furniture. One of the important components [Note:1] [Ref:1] of such game objects is a “Collider”, by adding a Collider to the Object, visitors can then collide or be supported by the object. A classic example is to let the floors and walls to have colliders so that the visitors will not pass through walls and drop through floors.

Dynamics
These are GameObjects that move around in a periodic or predictable matter. These objects can be animated by using Unity Animation, or Code Animation, or represented by using Particle Systems. In a game world, such game objects can be a moving platform, or sometimes, the sun and the moon, so that the world has a daylight time cycle system.

Instantiated
These are game objects generated during runtime [Note:2]. Usually, these objects need to be generated in a massive quantity. Due to how VRChat achieves synchronization, instantiated objects cannot be synced. In my world “Relativity Express”, one of the applications of this is the “balls Lab”. [Ref:2]

Pickups
These are game Objects that can be “picked up” during Runtime by the visitors. Under normal circumstances, these object will require 4 additional components :
Colliders : so that players can pick it up
Rigidbody : the Physical Behavior of the Object and VRCPickup needs it
VRC Pickup : a component to let players pickup an object
VRC Object Sync : optional component, to sync the object's position, rotation, owner

Interactables
These are game Objects that can be interacted by visitors, and often provides an output for the one who trigger it. Most of the time, these game Objects are buttons, doors, chairs, and more. These objects will require further coding, which will not be covered by this document.

[Note:1] In Unity, a component can be treated as a function, or a part of a gameObject, so that the gameObject may behave in a certain matter.

[Note:2] Runtime refers to the moment when the program is executing rather than being edited.

[Ref:1] 
For more information about “component” in Unity, documentation is available ||
Unity (2024, Oct, 24). Introduction to components. Available :  https://docs.unity3d.com/Manual/Components.html

[Ref:2] 
During one of the community meetups, participants get to experience such mechanism : this is time stamped from 1:04:17 to 1:05:32 ||
Community Meetup (2024, Feb, 25) VRChat Community Meetup 2024-02-25 - PhaxeNor. Available : 
https://www.youtube.com/watch?v=hZBAkC0Xtzo

5 : Optimization
Where did my frames go ?

One ordinary problem when it comes to world creation like this is that players are struggling to have a decent experience, usually, this is caused by ineffective algorithms or calculations.

In this section, the document will be discussing some of the most often issues and fixe : 
Real-time lights
Particle Systems
Occlusion
Mesh Optimization
Hierarchy Structure

Real time lights

Real time lights are light sources that cast light and/or shadows for objects that are within it’s radius. These casting calculations will be performed for every frame [Note:1].

Real-time lights can have no shadows or with shadows [Figure 501]. If a light source do not cast shadows, then the calculations will not be too intensive. In this case, only the distance between the point of another object and the light source will be the parameter for shading the object.

According to MissStabby [Ref:1], whenever a light source casts real-time shadows, it uses depth buffer(s) to calculate the required parts of other objects that are in the range of the light source that need to be shaded by the light source.

A depth buffer can be understood as a camera but each pixel only has 1 channel, and that channel will contain the distance between the location of that pixel is “looking” and the camera. It is vital to know that any mechanism that uses “camera”s like this one is frame expensive. 

[Figure 501 : A light source (the yellow sphere) without shadow shading a cube and its surroundings]

[Figure 502 : A light source with shadow casting shading a cube and its surroundings]
Based on the type of light source, different numbers of depth buffers will be performed, as different light sources will be shading the scene with different ranges and directions.

[Figure 503]
Light Source Type
Depth Buffer Quantity
Spot
1
Directional
1
Point
6
[Figure 503 & Table 501 : the allowed real light types with shadows and the number of depth buffers ]

One of the main solutions is Light Baking [Figure 504] which is a clever built-in Unity Operation, the main idea is to record the light data on the surface of the objects and save the data as a texture which is called a light map.

So instead of calculating the data for light casting, the lightmaps are referenced instead which is more efficient from a hardware standpoint. Note that light baking will only work for static light source and static objects. Light Count Limit will be covered in the part “Occlusion”.

[Figure 505 : an example of a light map]

🟢 Lightmap ? WTH is that ?

Back in the days, when we were younger, when we have to perform multiplications, we will need to perform repeated addition, this is slow and inefficient. One way is to create a time table, and every time when we need to do multiplication, we can lookup the table, this is fast.

In this case, when Unity tries to calculate Realtime lights and shadows, its like us doing repeated addition, yes it gets the job done but it is slow. Light Baking the process of making the times table, and the Lightmap is like our Times table. Afterwards, Unity will just need to lookup the final data, just like us looking up the Times table.
Particle Systems

A particle system is a built-in Unity Component which allows a gameObject to instantiate multiple numbers of BillBoards [Note:2] or meshes, each instantiated object is a particle, which has a set life time, more parameters can be set [Ref:2].

Here, the straightforward solution is to limit the number of particles at a certain time. This can be done by adjusting the emission rate and particle lifetime.

[Figure 506 : An example of particle systems used for flying yellow specs as a visual effect]

[Figure 507 : A classic “XR_XharpRazor Particle Usage” is to use Particle systems are plants, here it’s a particle tree]
Occlusion

The main Idea of Occlusion is “If the camera doesn't see it, then don’t render it”. In this case, there are 2 types of Occlusion : Unity built in Occlusion and Manual Occlusion. 

Unity built in Occlusion is similar to light baking, which is a built in operation which records the positions for a specific object to be seen so that Unity knows when to spend the necessary resources to render the object. However, this will only apply to gameObjects which has a mesh representation. Sometimes, Unity also struggles with edge cases, which will cause some meshes to glitch out.

Manual Occlusion is done based on scripts. The main mechanism is that when the player enters an area, a certain group of objects will be activated, and deactivated when the player leaves. Although the group of objects need to be assigned manually by the creator, using manual occlusion allows flexibility and applies to any gameObjects. Note that this method can be applied to Real time lights and particle systems or any mechanisms that are intensive.

Mesh Optimization

Some models has up to millions of polygons, these are extremely intensive to render due to how different game engines renders meshes. According to SimonDev [Ref:3], In short, when rendering a triangle smaller than a pixel, 0.75 of the work done by the GPU will be thrown away.

One solution is to lower the number of polygons, by doing this, the number of triangles will be decreased, and the size of the triangle will increase such that it will be larger than a pixel, which is advantageous for the renderer.
Another solution is to have multiple LODs [Note:3] for different models, so that when the models are far enough from the player, a lower quality version for the mesh will be used, thus using less resources.

Hierarchy Structure

If a single gameObject is a parent of a large number of gameObject, the structure of the Hierarchy becomes “render intensive”. This is pointed out by BeanMChocolate [Note:4], it is better for creators to have simpler object hierarchy.

Assume that we have 2 groups, group A : a bookshelf with 256 books, and each book is a child; group B : the same bookshelf with the same 256 books but it is a single mesh. By running a simple experiment, group B performs better.

[Note:1] A frame is an image on a screen, usually, multiple frames are displayed one after another to mimic movements and changes, also known as animations. By referring to a flipbook, each page is a frame, a flip book is an animation.

[Note:2] In Unity, BillBoards in Particle sense refers to a single square plane.

[Note:3] LOD means “Level of Details”, which represents the different models with different level of details

[Note:4] This is a conversation, no citing source available.

[Ref:1]
This citation is recorded as a Twitter Post, Link ||
M2007U-VR (2024, Feb, 16) 1 realtime point light with shadow = 6 spotlights + 6 cameras. Available : https://x.com/M2007U_V2021/status/1758660006495547501

[Ref:2]
In Unity, for a particle system, the lifetime of a particle refers to the allowed duration for a particle to stay before it gets destroyed. There are more parameters that can be set for a particle system, however, this document will not cover this as this is not the main goal of this document. Documentation are available ||
Unity (2024, Oct, 24) Particle System. Available :  https://docs.unity3d.com/2022.3/Documentation/Manual/class-ParticleSystem.html

[Ref:3]
SimonDev (2024, Jan, 29) When Optimizations Work, But for the Wrong Reasons. Available :  https://www.youtube.com/watch?v=hf27qsQPRLQ

6 : Uploading the project and Maintenance
Hey, wanna hang out at my place ?

With all the basic parts of the world, the creator can now upload the project to VRChat’s server, and if the world is set to public, visitors can visit the world.
One mandatory prefab is the scene descriptor [Figure 601]. This prefab is mandatory in the hierarchy for a VRC World to be uploaded. This prefab provides the server some of the essential information about the world.  
The one final step is to fill up the fields in the VRC SDK Build Panel, tick the box of acknowledgement, and click Build and Upload.

[Figure 1301 : the components of a Scene Descriptor]

[Figure 1302 : a screenshot of the VRC SDK Build Panel]

Z : Conclusion
Wrap it up

This document discusses the main steps and concepts of Creating a VRChat World. Keep in mind that there are still a lot to cover that this document didn’t consider. Some of the Information are meant to be separated and digested in smaller parts. Afterall, World building is not meant to fit entirely inside one single document.

This document suggests a specific type of workflow. However, as a world creator, feel free to alter some of the recommendations from this document when necessary and where it seems fit.

[Figure Z01 : A gathering held in a VRChat World]

REFERENCES

[1] “WHERE ARE ALL THE CITINGS ? you are plagiarizing stuffs~”
If you skipped everything and jump to the last page over here just to check for citings,
This document is a long one, in this case, it is better to chop the citing list into several gray lists for every section so that those referenced materials can be navigated easier. ||
XR_XharpRazor (2024, Oct, 24) World Building in Virtual Reality / VRChat. Available : you are reading it