Blog: Physical Computing

ITPG-GT 2301-005

Tutor: Tom


Assignment II: 

An Immersive Jazz Drum Experience: See in VR and play in reality

Designing the device & circuit

In this part I'll record the thinking process of the device, which has shifted from the original plan of a DJ board to the drum we have in the end.

I first drew some sketches of the composition.


Then I sketched the structure of the individual drum, went to the material shop, bought the plywood and sketched the model in Illustrator for laser cut.

屏幕截图 2020-11-11 023738.png



Designing the communication & Coding

Because the serial communication has to be connected with the USB wire which is not available in Oculus Quest, and also the bluetooth is not supported, we decide to use the WiFi for the communication between arduino and Oculus, the VR set we use.

Before Shin used the Arduino ESP32 and the ESP8266WiFi library, I tried to communicate with my Arduino Nano Iot and the WiFiMina included. Here's the code which also worked:

屏幕截图 2020-11-11 024449.png

Later Shin taught me to build the communication in Unity end to receive the message (OSC type) and I tried to build the communication in Unity. For the model part, it's just the same with other Unity projects I've done, setting the trigger and placing it to the right place.

Also here I attached the Android setting in Unity to fit the Oculus environment:

屏幕截图 2020-11-11 042325.png
屏幕截图 2020-11-11 042258.png

Trim & The End

After all these works, I trimmed the wire in the breadboard, and attached and reinforced each part for the drum. And it's good to go.

(I'll attach the user experience later.)