Project-XVR

Overview

Project-XVR is a multiplayer VR assessment platform featuring four cooperative missions where teams of players work together to complete objectives while being evaluated on coordination, communication, and task performance. The missions range from collaborative symbol-matching exercises to physics-based payload navigation, all connected through a custom mission lifecycle framework (XVRAssessmentSuite) that manages server-authoritative initialization, state tracking, and completion events.

Developed as a capstone project at Shenandoah University for Air University, the platform is built in Unity 6 for Meta Quest with Netcode for GameObjects handling networked physics and object ownership, Vivox providing spatial voice chat, and XR Hands enabling hand tracking alongside controller input. It represents the most architecturally complex XR project in the portfolio, combining multiplayer networking, VR interaction, and structured assessment in a single production-targeted system.

Non-Technical Summary

The application guides players through four different missions, each with its own style of challenge. In one mission, players sort colored objects into matching slots. In another, they work together to fire cannons and push a payload to a goal. Each mission has a narrator that explains the rules and a timer counting down, creating an engaging and slightly competitive atmosphere.

Behind the scenes, the application manages all the complexity of keeping everyone's view of the world synchronized. When one player picks up an object, everyone else sees it move in real time. A significant amount of engineering went into making this feel seamless, including three complete rewrites of the networking system before settling on the final approach.

Highlights

• Designed and built the XVRAssessmentSuite framework, a reusable, namespace-organized mission lifecycle system with async PreInit/Init/Ready phases, interface-driven contracts (IInitialization, IPostInitialization, IXVRTask, IXVRSpawnHandler), and a mediator-pattern XR interaction pipeline
• Migrated networking stack from Photon PUN 2 through Photon Fusion 2 to Unity Netcode for GameObjects (v2.3.2) across 5 iterative attempts, ultimately delivering server-authoritative gameplay with NetworkVariable synchronization, ownership transfer on grab, and ServerRpc/ClientRpc communication
• Integrated Vivox voice chat (v16.6.0), XR hand tracking (v1.5.1), and Timeline-driven narration with captions to create an immersive multiplayer VR experience targeting Meta Quest via OpenXR
• Implemented 4 distinct cooperative mission types (symbol matching with room-based distribution, team cannon physics with payload delivery, obstacle navigation with puzzle gates, and color-matching tutorials), each with networked state, UI Toolkit interfaces, and task completion tracking

Quick Highlights

• Server-authoritative multiplayer via Netcode for GameObjects with ownership transfer for true collaborative object manipulation
• Custom XVRAssessmentSuite framework with async initialization pipeline and interface-driven mission contracts
• Vivox voice chat integration for real-time player communication during missions
• XR hand tracking and controller support via OpenXR with mediator-pattern interaction filtering
• Timeline-driven narration system with audio captions for guided mission introductions

Technical Breakdown

The core framework is organized into namespaces under XVRAssessmentSuite:

• CoreSubsystems.Base - MissionHandlerBase provides the abstract lifecycle: PreInit() for pre-spawn setup, Init() for scene initialization, and Ready() for server-side validation before gameplay begins. Fires OnMissionInitComplete to signal downstream systems.
• CoreSubsystems.Interfaces - IInitialization, IPostInitialization, IXVRTask (with OnTaskComplete event), and IXVRSpawnHandler define contracts for mission components.
• CoreSubsystems.Enums - MissionState enum tracks lifecycle phases.
• MissionSubsystem - Houses MissionHandlerBase and mission-specific handlers.
• TaskSubsystem - Task completion tracking with event-driven callbacks.
• ServiceLocation - Scene-scoped ServiceLocator for dependency injection across networked scenes.
• UIToolkit - UI Toolkit integration layer for mission-specific UI panels.
• Networking - Network state management (renamed from NetworkingRework during v2.0 cleanup).
• XR - XVRBaseInteractorMediator and XVRBaseInteractableMediator providing mediator-pattern XR interaction filtering.

Networking Architecture

The multiplayer stack uses Netcode for GameObjects 2.3.2 with a server-client topology:

• NetworkObject ownership model: GrabTransferOwnership changes NetworkObject.OwnerClientId on XR grab events via ChangeOwnershipServerRpc(), enabling any player to manipulate any interactable.
• NetworkVariable synchronization: mission state (obstacle visibility in PuzzleOne, flashlight state in FlashlightInteractableHandler, color assignments in ColorHandler) is synced automatically.
• NetworkList: MissionOneSpawnHandler uses NetworkList<Vector3> to manage spawn point allocation across clients.
• ServerRpc/ClientRpc: All authoritative actions (cannon firing in CannonManager, catapult rotation in MoveCatapult, puzzle activation in MissionThree) route through ServerRpc with ClientRpc for broadcast.

Mission Implementations

Mission 0 - Color Matching (Tutorial): ColorHandler assigns random materials to objects and targets via server-authoritative NetworkVariable. Players grab objects and place them in matching color slots. Correct placements lock via LockSocketInteractorMediator.

Mission 1 - Symbol Matching: MissionHandler initializes rooms based on player count, uses SymbolObjectFactory to spawn symbol objects across 6 cube faces (Front, Back, Left, Right, Top, Bottom). NewRoomHandler tracks local and global correct interactables. MissionOneSpawnHandler allocates spawn points via NetworkList. MissionOneUIHandler displays status via UI Toolkit with TimelineHandler countdown.

Mission 2 - Cannon Payload: CannonManager spawns networked ammo with NetworkObject.Spawn() and applies Rigidbody.AddForce() impulse. MoveCatapult handles server-controlled rotation in 4 directions. MovePayload applies physics forces to push the payload left/right with Y-axis frozen. CollisionDetector fires win condition. UIManager displays team scores and 60-second countdown.

Mission 3 - Obstacle Navigation: WaypointMover moves a payload along a Waypoint hierarchy path with pause/resume via ServerRpc. PuzzleOne, PuzzleTwo, PuzzleThree each manage obstacle visibility through NetworkVariable<bool> arrays. PuzzleManager implements IXVRTask to detect payload boundary triggers and fire OnTaskComplete. MissionThree orchestrates puzzle progression and timer management.

XR Interaction Pipeline

The interaction system extends XR Interaction Toolkit with a mediator pattern:

• LockSocketInteractorMediator extends XVRBaseInteractorMediator, implements IXRHoverFilter and IXRSelectFilter to restrict socket interactions to a configured allowed-interactable set. On correct interaction, fires OnTaskComplete via IXVRTask.
• FlashlightInteractableHandler extends XVRBaseInteractableMediator with NetworkVariable<bool> light state synchronized via ToggleLightServerRpc().

Scene Flow

0B_MainMenu_Scene -> 1_Intermission_Scene -> M_Mission_0_Scene (tutorial) -> M_Mission_1_Scene -> M_Mission_2_Scene -> M_Mission_3_Scene. Each mission scene is loaded with its own MissionHandler registered in the ServiceLocator.

Voice Communication

Vivox 16.6.0 provides in-game voice chat with channel management tied to the multiplayer session lifecycle.

Audio System

Main_AudioMixer routes mission-specific SFX (collision sounds, button presses, door effects, lighting SFX, UI hover/click sounds) and narration audio through dedicated mixer groups. Timeline-driven narration plays mission introductions and instructions with synchronized captions.

Systems Used

• Mission Handler Base* - Abstract base class providing standardized mission lifecycle management with OnMissionInitComplete events and interface-driven initialization
• XVR Interactor Mediator* - Mediator-pattern XR interaction filtering system that restricts socket interactions to allowed interactable sets and fires task completion events
• Service Locator* - Scene-scoped service locator for dependency injection of mission handlers, UI managers, and spawn handlers across networked scenes
• Grab Transfer Ownership* - NetworkObject ownership transfer system enabling true multiplayer object manipulation by changing ownership on grab events
• Symbol Object Factory* - Factory-pattern object creation system that instantiates and configures symbol objects across 6 cube faces with networked spawning
• Timeline Handler* - Timeline-driven narration and caption system for mission introductions and instructions with audio mixer integration
• Network Spawn Handler* - Server-authoritative spawn point allocation using NetworkList to manage player teleportation positions across mission scenes
• Waypoint Navigation System* - Physics-based payload movement along waypoint paths with networked pause/resume and gizmo visualization for level design

Deep Dive

The most significant technical challenge was networking. The project went through 5 distinct networking iterations:

1. Photon PUN 2 (Dec 2024 - Feb 2025): Initial import of Photon PUN and Voice. Built client/server scripts and a BootstrapHandler. Abandoned due to limitations with VR object synchronization.

2. Photon Fusion 2 (Feb 2025): Imported after removing PUN 2. Explored room-based networking. Created a MissionSubsystem and PlayerServices layer. Still insufficient for the server-authoritative model needed for assessment integrity.

3. Photon Fusion Attempt 3 (Feb 2025): Clean restart with "base project with no multiplayer," importing TimelineExtensions, Polygon asset packs, and UI prefabs. Established the visual foundation but networking remained problematic.

4. Netcode Attempt 4 (Feb 2025): First Unity Netcode attempt. Configured for Dedicated Server topology. Achieved a "working implementation" but dedicated server approach was overly complex for the use case.

5. Netcode Attempt 5 (Feb 2025 - May 2025): Final and successful approach. Client-hosted Netcode for GameObjects with NetworkVariable synchronization, ownership transfer, and ServerRpc/ClientRpc communication. This became the main branch and was tagged V1.0.0.

Each migration required re-architecting how player state, object ownership, and mission progression were synchronized. The final Netcode approach was chosen for its tight Unity integration, simpler topology, and first-party support for NetworkObject ownership transfer, critical for VR grab interactions.

Framework Design Decisions

Why a mediator pattern for XR interactions? XR Interaction Toolkit provides generic grab/socket interactions, but assessment missions need filtered interactions: only the "correct" object should lock into a socket. LockSocketInteractorMediator wraps XRI's filter interfaces (IXRHoverFilter, IXRSelectFilter) to reject non-matching interactables before the physics layer processes them. This keeps mission logic decoupled from XRI internals.

Why async initialization (PreInit/Init/Ready)? Networked VR scenes have ordering dependencies: the server must spawn objects before clients can interact, and clients must be at their spawn points before gameplay begins. The three-phase lifecycle ensures: (1) PreInit configures server-side state, (2) Init spawns and positions everything, (3) Ready validates all clients are synchronized before releasing control.

Why Service Locator over dependency injection? Unity's scene-based architecture makes constructor injection impractical for MonoBehaviour components. The ServiceLocator provides scene-scoped registration/resolution that survives Netcode's spawning lifecycle, where objects are instantiated by the networking layer rather than by application code.

Performance Considerations

• Ownership transfer is used sparingly, only on grab events, to minimize authority handoff overhead
• NetworkVariable is preferred over RPCs for continuous state (flashlight on/off, puzzle visibility) to leverage Netcode's delta compression
• NetworkList for spawn points avoids repeated RPC broadcasts when players connect sequentially
• Physics on the payload (MovePayload) freezes the Y-axis and disables gravity to prevent drift in networked physics simulation