Natural-Interaction VR Action Adventure

Puzzlehand

A controller-free VR action-adventure that turns the player's hands into the movement, climbing, magic, combat, and puzzle interface. Built as my Bachelor of Engineering thesis, the project combines Unity, HTC Vive, and Noitom Hi5 2.0 gloves in three progressively layered levels.

Unity SteamVR Noitom Hi5 2.0 HTC Vive

Interaction Systems Level Design Hardware Testing

EngineUnity

HardwareHTC Vive + Hi5 Gloves

RoleDesign + Programming

FormatBachelor's Thesis, 2025

01 / Project Intent

Remove the controller and make physical gestures the game's core language.

Puzzlehand explores how a VR game changes when grabbing, pulling, climbing, casting, and operating machinery are performed through tracked hands instead of handheld controllers. The goal was not to add gesture input on top of a conventional game, but to design traversal and puzzles around the affordances of natural hand movement.

I designed and programmed the gameplay, interaction systems, and three-level progression. Visual and audio assets were third-party or production resources; they are not presented as my original art.

Core design question

Can one consistent hand-based interaction language support locomotion, traversal, combat, resource decisions, and environmental puzzles without relying on a controller?

02 / Interaction Systems

Every major verb begins with the player's hands.

Hand-drag Locomotion

Players press and drag against the virtual ground to propel themselves. Motion thresholds, direction sampling, gravity, and inertia smoothing translate hand movement into stable traversal.

Two-hand Climbing

Grabbable surfaces switch the character between grounded and climbing states. Releasing either hand transfers tracked displacement into body movement while collision logic prevents invalid motion.

Magic, Combat & Resources

Gesture-cast abilities connect ranged interaction, enemy pressure, and environmental problem solving. Resource limits make magic a traversal and planning decision rather than unlimited spectacle.

Physical Mechanisms

Levers, buttons, rotating cranks, keycard validators, and joystick-style controls reuse consistent grabbing, collision, rotation, and state-change rules.

Puzzlehand interaction mechanism designs — Mechanism studies for physical puzzle interactions and feedback states.

Puzzlehand VR interaction mechanisms — Additional interactable forms built from the same hand-driven input rules.

03 / Level Design

Three levels teach, combine, and pressure-test the interaction vocabulary.

Puzzlehand level one design diagram — **Level 1 — Foundations.** Introduces hand-drag movement, basic climbing, magic management, and a keycard-validator sequence in controlled spaces.

Puzzlehand level two design diagram — **Level 2 — Mechanical Challenge.** Extends traversal into ceiling climbing and combines validators, physical mechanisms, and enemy interference.

Puzzlehand level three design diagram — **Level 3 — Combined Mastery.** Layers multi-step mechanisms, advanced gesture use, traversal decisions, and combat pressure into a final integrated route.

04 / Technical Implementation

Modular controllers keep gesture data, player state, collision, and feedback readable.

The project separates tracking input from gameplay state changes. Hand position and gesture data feed movement and interaction controllers; those controllers then coordinate gravity, CharacterController collision, climbing state, object constraints, and visual feedback.

This structure made the same low-level grabbing and motion rules reusable across traversal, rotating mechanisms, validators, and combat interactions, while allowing each object to expose its own state and completion logic.

Implementation priorities

Stable movement, reusable interaction components, visible state changes, and hardware calibration that could be tested one behavior at a time.

Puzzlehand movement and gravity code — Movement update logic coordinating hand-driven locomotion, gravity, and the character controller.

Puzzlehand climbing system code — Climbing-state logic translating tracked hand displacement into controlled player movement.

05 / Hardware Validation

Each interaction was calibrated and tested on the target hardware.

Input Setup

Noitom Hi5 2.0 gloves provided tracked hand input while HTC Vive supplied the headset and VR environment. Calibration aligned the physical hands, avatar hands, and interaction volumes.

Functional Testing

Climbing, cranks, levers, buttons, validators, and joystick interactions were tested independently before being combined inside full level routes.

Iteration

Thresholds, collision ranges, release behavior, and mechanism constraints were tuned to reduce accidental activation and make intended hand motion easier to read.

Noitom Hi5 motion capture gloves used for Puzzlehand — Noitom Hi5 2.0 motion-capture gloves used for hand input.

HTC Vive headset used for Puzzlehand — HTC Vive target headset and tracking environment.

Puzzlehand hand calibration test — Hand-model calibration before interaction testing.

Puzzlehand climbing interaction test — Two-hand climbing test in the playable environment.

Puzzlehand rotating crank interaction test — Rotating crank test for constrained hand-driven mechanisms.

Puzzlehand lever interaction test — Lever activation and state feedback test.

Puzzlehand button interaction test — Button collision and activation test.

Puzzlehand keycard validator interaction test — Keycard placement and validator-state test.

Puzzlehand joystick interaction test — Joystick-style input test using constrained hand movement.

06 / Outcome

A thesis-scale project that connects interaction research to a complete playable progression.

Puzzlehand demonstrates my strongest working pattern: define a player-facing design question, build the underlying system, author content that teaches it, and validate the result on real hardware. The final prototype supports controller-free movement, climbing, magic, combat, physical mechanisms, and three escalating levels within one coherent gesture language.