Watching someone play a simulation tells you more than any survey. The problem is that you cannot sit beside every student, and the interesting moments in a field exercise are the ones nobody expected. Trailhead exists so that a session can be replayed later, in 3D, by whoever needs to see it.

It was built for the Extended Reality team at Unity Environmental University to replace a third-party recording service, first for Virtual Forest, where students estimate rabbit populations and investigate the origin of a fire. The questions instructors had were things like "where did they go first" and "did they ever look at the evidence". Those are spatial questions, and a replay answers them directly.

What gets recorded

A recording is a set of tracked subjects plus the events that happened around them. In Virtual Forest that means the player, where they were looking, and each rabbit. The gaze subject is a raycast from the camera, so the replay can show not just where the player stood but what they were pointed at.

Each subject's transform is sampled ten times a second, and a sample is skipped if the object has not moved more than a centimetre since the last one. A rabbit sitting still costs almost nothing. Gameplay events such as a rabbit being sighted or caught are stamped into the same timeline, either on the recording as a whole or on the subject they belong to.

Alongside the samples, the game sets session and simulation metadata: the game mode, time of day,weather, duration, average framerate, and the end-of-game stats. That metadata is what the gallery lists and searches, so nobody has to open a payload to find the session they want.

Staying out of the game loop

The non-negotiable requirement was that recording never causes a frame hitch. A simulation that stutters because it is being observed is not teaching what it is supposed to teach.

The Unity client is a single component. Samples go into plain in-memory arrays, and nothing is serialised until the session ends. Upload runs as a coroutine with a short timeout, and the game carries on whether or not it succeeds; on success the server returns a recording ID, and the post-game screen shows a link to the replay. The API the game sees is small: start a recording, add subjects, add events, set metadata, finish.

Browser builds get a different path. A WebGL page can be torn down at any moment, so those clients upload compressed deltas as they go, and the server appends each one to the recording. A closed tab loses the last few seconds rather than the whole session.

Privacy

All user data is annonomized and obscured to prevent identifiability. User experiance and user trust are our most important assets. While we discuss data collection, its important to clearify the type of data collected. We never collect Personal Identifying Information. The only information we collect is directly related to the session events.

Where it goes

Payloads travel to an ingest endpoint guarded by an API key. The server writes the raw JSON to disk and indexes the game and metadata in Postgres, so the dashboard can list, search, and sort recordings without reading every file. Trailhead ships with a self-hosted sidecar Postgres and a Docker volume, which is all a single-host tool needs.

Summaries are also pushed into Summit, the analytics platform that holds sessions, events, and performance samples. The push happens after the upload has been acknowledged and never blocks it. The result is that an analytics view of a session can link straight to its 3D replay, which closes the gap between "this number looks odd" and "here is what the player was doing when it happened". Player feedback is entangled to their sessions, adding tangible evidence to close the loop.

The viewer

The replay viewer is a Next.js app with a React Three Fiber scene. It loads the terrain, picks a skybox to match the session's time of day and weather, places the tracked subjects with their models, and scrubs through the timeline with the key events marked along it. The camera can fly freely, follow a subject, or sit in the player's point of view. Stretches where nothing was recorded are skipped automatically.

Each subject can also be rendered as a heatmap, which turns a single session into a picture of where a player spent their time. That view has done more to answer "where did they go first" than any amount of scrubbing.

What I would keep

Two decisions held up well and I would make them again on any similar tool.

Treat recording as a separate concern with a tiny surface area. The game should be able to add a subject in one line and never think about it again.

Store the raw payload and the searchable summary separately. The summary is what people browse; the payload is what they open once they have found the session they care about. Mixing the two makes both slower.

Trailhead is an internal tool and the repository is private, but the general shape is reusable for any real-time project that wants to know what players did rather than guess.