Overview
An experiment connecting a physical glove controller to a Google Cardboard VR headset. The glove reads finger contact combinations via Arduino and transmits the data wirelessly to a Unity application on the phone. In VR, the player sees colored spheres approaching and must respond with the correct hand gesture – each of ten colors mapped to a different finger combination.
Tools: Arduino UNO, Unity, Python, Google Cardboard, WiFi bridge
Context: Independent experiment, 2026
An experiment connecting a physical glove controller to a Google Cardboard VR headset. The glove reads finger contact combinations via Arduino and transmits the data wirelessly to a Unity application on the phone. In VR, the player sees colored spheres approaching and must respond with the correct hand gesture – each of ten colors mapped to a different finger combination.
Tools: Arduino UNO, Unity, Python, Google Cardboard, WiFi bridge
Context: Independent experiment, 2026
Concept
The project is primarily a technical demonstration – an exploration of what it takes to make a physical body legible to a VR system. The chain between gesture and response is deliberately exposed: finger contact becomes electrical signal, signal becomes serial data, data crosses WiFi as UDP packets, packets become Unity input, input becomes visual event. The glove does not translate intention directly – it translates what can be measured.
The game mechanic is intentionally minimal: the cognitive load of memorizing ten color-gesture pairs is a side effect of the system's logic, not a designed experience – however, it was gamified into a record system, with balls appearing faster and faster as the player progresses.
The game mechanic is intentionally minimal: the cognitive load of memorizing ten color-gesture pairs is a side effect of the system's logic, not a designed experience – however, it was gamified into a record system, with balls appearing faster and faster as the player progresses.
Framework
The project draws on three thinkers working at the intersection of technology, body, and action.
Gilbert Simondon provides the core: the technical object is not a passive tool but a process. Stability is not a final state but a temporary resolution of tensions. The machine does not work in the world – it works with it, and the environment becomes part of what the machine is.
Bernard Stiegler adds the question of translation: continuous bodily movement is grammatized – broken into discrete, technically readable elements. The hand does not enter the system as it is; it arrives as angle, delay, deviation, threshold. Something is always lost or smoothed in the conversion.
Yuk Hui frames the political dimension: technology is never neutral. Every system creates an ontology – decides what counts as movement, what counts as error, what counts as acceptable input. To pass through the interface, the body must first become legible to it.
Together, these form the question the game plays out: not "who controls whom" but how a shared condition of action is produced between a body, a device, and a system of measurement.
The project draws on three thinkers working at the intersection of technology, body, and action.
Gilbert Simondon provides the core: the technical object is not a passive tool but a process. Stability is not a final state but a temporary resolution of tensions. The machine does not work in the world – it works with it, and the environment becomes part of what the machine is.
Bernard Stiegler adds the question of translation: continuous bodily movement is grammatized – broken into discrete, technically readable elements. The hand does not enter the system as it is; it arrives as angle, delay, deviation, threshold. Something is always lost or smoothed in the conversion.
Yuk Hui frames the political dimension: technology is never neutral. Every system creates an ontology – decides what counts as movement, what counts as error, what counts as acceptable input. To pass through the interface, the body must first become legible to it.
Together, these form the question the game plays out: not "who controls whom" but how a shared condition of action is produced between a body, a device, and a system of measurement.
Process
he technical chain required an unexpected workaround: without a WiFi module for the Arduino, and with direct Arduino-to-phone USB connection requiring a fight with Android permissions, a laptop was introduced as an intermediary. Its sole function in the system is to act as a WiFi bridge – it connects to the phone's mobile hotspot and forwards serial data from the Arduino as UDP packets, which the Unity application on the phone reads. During demonstration, the laptop sits in the player's backpack.
The glove itself was built from proxy materials: a textile glove, aluminium foil as a flexible conductor, and electrical tape. This kind of assembly does not function as a proper device, but proved reliable enough for continuous testing.
The rest of the build involved setting up the Arduino to read finger contact combinations, writing a Python bridge script to handle the serial-to-UDP conversion, and configuring Unity to receive and parse the incoming data. Due to persistent issues with the Google Cardboard SDK, the stereoscopic split-screen and accelerometer-based head tracking were implemented manually in Unity (with AI code assistance) – a faster solution than continuing to debug the SDK integration for a short-form experinment.
The glove itself was built from proxy materials: a textile glove, aluminium foil as a flexible conductor, and electrical tape. This kind of assembly does not function as a proper device, but proved reliable enough for continuous testing.
The rest of the build involved setting up the Arduino to read finger contact combinations, writing a Python bridge script to handle the serial-to-UDP conversion, and configuring Unity to receive and parse the incoming data. Due to persistent issues with the Google Cardboard SDK, the stereoscopic split-screen and accelerometer-based head tracking were implemented manually in Unity (with AI code assistance) – a faster solution than continuing to debug the SDK integration for a short-form experinment.
