Introduction: through the looking glass

Naively, the glass of our computer or mobile phone screen can be seen as the boundary between that which is traditionally called the 'virtual' and the 'physical'. When we post text on a social network, create files on our computer, or talk with collegues using Teams, we seem to leave our physicality behind and enter the domain of the virtual: the online realm with its owwn logic, boundaries, and customs. On the other hand, when we close our laptops and leave our phones in our pockets in order to go for a walk, enjoy a nice dinner, or play the piano, we are again in the domain of the physical: the hard fleshy reality with its own logic, boundaries, and customs.

In this sense, we could speak of two different worlds: the online world of the virtual and the offline world of the physical. The edge between these two worlds is defined by our physical keyboards, trackpads and game consoled with which we manipulate the objects in the virtual world, whose state and reactions we observe by looking at the screen and listening to the sounds they make (and, sparsely, feeling some haptic feedback). There remains, however, always some kind of unbridgable abyss between the two. To borrow Donald Norman's phrase: the gulf of expectation and the gulf of evaluation are never completely closed.

Arduino and Unity

During the module on Unity, you have learned how to create a virtual world. You know how to make and position objects, how to give them shapes and color, and how to move them around in space. This way, you give meaning to specific parts of space in order to differentiate these parts from the principally endless cartesian space. Paraphrasing Espen Aarseth, we could state that creating a virtual reality is “essentially concerned with spatial representation and negotiation" – both of which we do employing the physical structures we mentioned above.

On the other hand, we have learned to work with the Arduino. Using this small microprocessor, we were able to use some parts of the physical realm to change the brightness of an LED, make a motor rotate faster or have a microchip emit some kind of sound for us. We have worked with physical sensors to change the state and representation of physical actuators.

We also showed how we can use serial communication to connect the virtual with the physical. In a way one could state that the Arduino gives us possibilities to finally bridge the gulfs that Norman is talking about. We can finally go through the looking glass.

Assignment

Create a world

In this assignment, you are asked to do exactly that. Create a world that contains both virtual and physical elements, taking into consideration the pros and cons of both modalities. Think about reasons why some parts of your world are located in the virtual realm, and other parts in the physical. Things in the virtual parts have find their effects in the physical, and the other way around.

Create the virtual part of this world using Unity and use serial communication with the Arduino to connect this virtual part with the physical. Connect sensors to your Arduino in order to bring aspects of the physical into the virtual part of this world, use specific events in Unity to drive actuators in the physical. Changes and effects in one part should have repercussions in the other. In this way, reality becomes one complete whole again, when the difference between the virtual and the physical are obliterated.

During the last half of this module you will get time to work on this assignment.

No Game

Though your work must have a narrative, a nice design and an interactive component (something must react on something), it is prohibited to make a game (a game in this case being every interactive installation in which a specific goal is to be reached, and/or in which a certain form of competition is present).

The interaction with the installation must be its own reward. The design, flow and narrative of the project needs to be such that visitors want to interact with it for the sheer pleasure of interacting.

Documentation

It is imperative that you keep track of your process, using images, movies, and text. In the end, you are asked to write a small report discussing your project: its initialisation, its reason of being, the procedures and developments that you made, and your final reflection. Integral part of this report is the technical documentation of your project, using schematics and technical drawings.

This document is an integral part of the project and will be assessed as well as the product.

Assessment

There are four aspects on which your project is reviewed

• The Narrative: what is the story you tell with regards to your reality? Why exactly did you choose this form of interaction, why are these parts in Unity and those parts in reality? Why did you choose these sensors, these actuators, these models...?
• The Design: is the installation finished? Is the design thought-through, complete and convincing? Is the realisation based on concrete design-decisions? Does the design fit the narrative?
• The Technical realisation: does the installation work? Does the realisation fit the narrative?
• The Documentation: is the documentation complete and thorough? Can we follow the (creative, intellectual, collaborative) steps you have taken throughout the project?