Evaluation

In our prototype presentation, we outline our success criteria as "By creating a space that fosters habit-forming behaviours, we aim to increase the number of people coming through a low-traffic area by masking exercise as fun." More specifically,

  • Currently 25 - 30 people an hour use the bridge (during peak hour times on weekdays).
  • We're aiming for a 25% increase. This would be a mean of 34 people an hour.
  • Based on Brian K. Smith's model, we're aiming for an average heart rate of 115BPM (in comparison, Need for Speed is 120BPM and Dance Dance Revolution is 125BPM)

Unfortunately, due to the project presentation being moved from at UQ, to the Edge at South Bank, our budgeted one-day time-frame for testing it at the bridge location, had to be removed to complete the project by the deadline.

Test Plan

If we had been able to completed our evaluation, our proposed test plan such evaluation is described below.

Setting Up

At 7am in the morning, we planned to transport and set up our game in the bridge. This would hopefully be finished by 8am so as not to inconvenience people who needed to use the bridge during University hours (8am-9pm). Also by 8am, we hoped to have secured any potential safety problems e.g. Taping down power leads and securing ramps either side of the game floor.

A3 informational posters would be attached to the walls at the start and end of the game floor. These posters would inform pedestrians that the floor is a project constructed for the ITEE subject Physical Computing. It would inform them that it is 100% safe to walk on, and any depression on the floor surface they may feel while stepping on it is simply the pressure sensors registering their presence. These posters would not mention instructions on how to play game or what they could expect when stepping on it. Ideally these posters would not be there in a pure ethnographic study, however, we felt it necessary to address any pedestrians safety concerns about the project.

Observing

From 8am onwards, we planned to use ethnographic techniques to observe people (from a distance) interacting with the device. We planned to record how many pedestrians crossed the bridge and how long each pedestrian spent on the game. We would then compare these results to the average number of pedestrians and average duration spent on the bridge which we had recorded previously for the Prototype presentation. We hoped that we would at least see a 25% increase on the number of people using the bridge and also, a 25% increase of the amount of time spent on the bridge. Though 25% may only seem like a small increase, our game was designed so that it could introduce a small amount of exercise, that would otherwise not be there, into a persons day.

For recording heart rate, this area would be more difficult to test. Two kinds of heart rate monitors are available - chest strap monitors, and strap free monitors. The chest strap monitors consist generally of a chest strap and a wrist watch. The chest strap is placed below the users chest muscles and is very accurate. They work by detecting heart rate. The strap free monitors work by placing one or two fingers on a sensors and detecting the users pulse. They are considered less accurate and can give no or wild readings for people with cold fingers etc. For our tests, we would have used the strap free monitors as they would be far less intrusive for our users. We would measure a pedestrians standing pulse rate, and then ask them to use the game. We would then ask them to come off mid-game and have their pulse rate measured again. We would then be able to determine if we achieved our goal of achieving an average heart rate of 115bpm.

The Exhibit

When exhibiting our game at the Edge, we did ask people to fill out a survey after playing our game. Most users felt that the game did raise their heart rate, and they would at least interact with the game, if it was on their path of travel. Some others felt they would go out of their way to use it, or use it while waiting nearby the game. Based the qualitative results of the surveys, we believe we have achieved our goals (as accurately as we could measure them). However, these kind of qualitative survey results are not always reliable, as some of the participants may have 'just wanted to be nice', and were overly eager to give us positive results. To fully evaluate our project, a procedure like the Test Plan described above could be used.

Reflection

Reflecting on the project, I was extremely pleased with our results. With a number of technical setbacks throughout the semester, we were still able to produce a product that really showed off our original idea. We utilised two different kinds of technology for each half, the first half was completely 'self engineered' using hacked keyboards, shift registers, transistor arrays and a whole lot of LEDs. The second half was more 'prefab'; it utilised the built-in inputs of the Phidget, and a projector. Though I still prefer the self engineered design, many people actually preferred the projected display as it produced a much cleaner, neater looking game floor. I believe that this preference is mainly due to the dimness of our green LEDs. This was a problem we faced throughout the semester and could not be overcome due to budgetary constraints. With a larger budget, I believe we could have created a game that was really visually spectacular. Throughout the semester, I learned that there were many solutions to a problem and it is important to not get stuck in a way of thinking. Spending more time brainstorming and thinking about ways to approach a problem was always more beneficial than diving in with the first idea.