Interactive Devices - Unit Conclusion

The unit overall has taught me much, especially from the hardware side of things. It was very interesting to see how different pieces of software could connect in order to transmit hardware signals. In our case:

Arduino board -> Arduino program -> OpenFramworks program -> Windows

This course also helped to reinforce what I already suspected: hardware is not the side of computing that I am interested in. It's too messy and low-level for my liking. What I do like though is the idea of creating something from scratch in a team which works as a pipeline where team members specialised in hardware create an initial device/machine which I can then build on.

I personally found it challenging to write a research paper for a project which could be shown at a conference such as CHI. Finding the appropriate wording was not easy and it took a long time. In addition, all the extra parts a Work In Progress CHI paper needs, to be presented, augmented the time taken to write the report.

Finally, I think the unit would be beneficial to all students in Computer Science which have never tasted the adventure that is hardware. The reason is not to only encourage them to pick up products such as Arduino and make something creative/useful but also to see if hardware is something that they enjoy and never had the time to try, possibly changing their career path.

Interactive Devices: Final Lab - Rubikon Demo

On the morning of the demo, the group gathered to decide in which order the applications would be shown; this was critical for a good presentation flow.

Under my suggestion, we decided to start from authentication because usually a computer session starts from logging in. After authentication through our custom login screen, we would switch applications to go to the Desktop. Here we would use the folder navigation features to travel through the icons to select Google Chrome and launch it. Once in the browser, features such as: switching tabs, scrolling down/up and going back and forward in a page's history would be highlighted. These actions would all show how Rubikon works as an input device.

Even though we had a prototype, our final product would have been a personal device. This meant output would be required as well. To highlight this fact at the end of the presentation, a specific button would be pressed to play a Christmas song and light the LEDs on the cube, with different colours, according to the song's rhythm.

With the demo order finalised, we concluded the meeting by deciding what to say during the presentation to emphasise the potential of our device.

The feedback from the people that came to see and try out our device during the demo was very positive. In addition, everyone seemed to enjoy using the cube as an alternative means of input. However it was hard to convey how Rubikon would be, ideally, a personal device due to the early stages of the prototype. For this reason I brought to the demo a mini touch screen to give an idea of the feasibility of the final product.

Interactive Devices: Lab 5 Individual Post - Implementing the software features

From the last lab we had an Arduino program capable of transmitting rotary encoder inputs to our laptop. In order to make use of these inputs myself and Vlad, after much research, decided to use the OpenFrameworks library to implement the features we set out at the beginning of the project i.e. folder navigation, browser navigation and authentication.

We started by sending the device's rotations from Arduino to OpenFrameworks and subsequently created a Node JS server, listening for commands, in order to perform actions on the browser. As we implemented the new system we quickly remembered that the most important, if not all, computer commands are mapped to certain keyboard shortcuts, even better, these shortcuts are often consistent across programs. Luckily Windows provides a C++ function called keybd_event() which simulates key strokes.

Example:

keybd_event(VK_TAB, 0, 0, 0);                                  // Presses tab key
::Sleep(1000);
keybd_event(VK_TAB, 0, KEYEVENTF_KEYUP, 0);   // Releases tab key

For our authentication we decided to start from the Windows lock screen. The locking action on Windows has a shortcut, Windows Key + L, so we simulated this combination with our program. This first attempt didn't work so we decided to try other combinations to see where the problem lied. After a few successful attempts I quickly realised Windows had a security protocol in place which clearly disabled us from locking the computer programmatically.
As a solution we decided to make our own authentication screen which would record key presses in the background and on pressing enter, if the password was correct, it would let us perform the other previously mapped actions, basically locking down any cube functionality until access was granted.

Using a similar method, myself and Vlad then implemented the following applications for the demo: folder navigation, browser navigation and application switching.

Interactive Devices: Lab 4 Individual Post - Building the prototype

The day before the lab we received the electronic components required to start building our cube; seven rotary encoders. Each works as a rotary encoder, push button and LED. The degree of rotation is not restricted to 360 and it knows whether it's rotating clockwise or counter-clockwise while the button is a simple push down, i.e. it has a binary state. The LED is an RGB LED capable of displaying light of any RGB combination, this means vibration feedback was not needed anymore as we could just give feedback by lighting the LEDs.

With the components in our possession myself and Amir went to the electronic labs to solder each of the eight pins. I soldered the first potentiometer to its circuit board but after thinking about it, we
decided to do away with the boards because otherwise we would never be able to fit all the rotary encoders inside the cube.
When we got back to our lab the whole team started looking at how to connect the rotary encoder to the Arduino board by following a Bildr guide provided by the rotary encoder's page on Sparkfun.

After a fair bit of trial and error with Arduino coding we eventually managed to write a program which received movement data when rotating the encoder. Following from our success we quickly decided to try to get the button to work as well so we connected the push button wires to the Arduino board and edited our program. Soon enough our program could receive both rotation and push data.