I’ve been asked recently about the role of biometrics testing in User Experience.
Biometrics in general refers to measuring biological processes and characteristics. Any technology which measures an aspect of human physiology can be referred to as biometrics. Biometrics technology has a range of applications from security (think fingerprint and retina scans) through medical (blood pressure monitors) and now has found its way into the world of product design.
The mainstream User Experience industry has evolved over the past ten years or so to include qualitative and quantitative testing methods to assess what customers want. Quants methods have largely been limited to products that are already in the field, e.g. analysis of web analytics information, A/B testing, Net Promoter Score etc. Some quants methods do apply during product development, primarily methods such as surveys, kano analysis etc. Qual methods have primarily centred around what have traditionally been market research methods such as focus groups and user testing.
There’s been an increasing trend in user experience, (particularly for B2C products) to focus not on just whether a customer “can” use a product (i.e. usability), but whether the customer “will” use a product. The latter has been referred to by organisations such as the HFI as “PET Design” (Persuasion Emotion Trust), however I prefer the term “Persuasion architecture”.
Biometrics is a method to allow us to measure the physiological impact of our products on customers. Biometrics includes eye tracking, galvanic skin response (GSR), and electroencephalogram (EEG). Recently, I’ve been working with UX Researcher Louise MacAulay, on a pilot scheme to assess the viability of biometrics methods for games testing.
We’re currently using the following equipment:
- Mirametrix S2 eye tracker
- Emotive EEG headset
We also bought an “Affectiva q-sensor” GSR bracelet to measure electrodermal activity (changes in skin conductivity). However after trialling this with iMotions Attention Tool software, we found that we couldn’t get all three sources of biometrics equipment working simultaneously. I also found that the the GSR wrist bracelet took several minutes to “normalise” on my wrist, and that response latency (i.e. time between the emotional stimulus like me getting excited in a game and the time actually seeing a GSR response on screen was just too high (several seconds)). Furthermore, the GSR response took a long time (the order of minutes) to normalise back to a usable level. Hence we dismissed GSR as a viable biometrics method for games testing.
iMotions Attention Tool software allows us to integrate data from the eye tracker, GSR and EEG simulataneously. We did have some difficulties combining data from all three sources, but that was possibly due to the laptop that we were using. I recommend having a decent graphics card – this will help with the fast i/o needed for concurrently running this sort of kit. With a regular powered laptop however, we were able to run eye tracking and EEG equipment in parallel into iMotions.
EEG
Electroencephalogram works by picking up small electrical signals in the brain. It is not as sophisticated as methods such as Magnetic Resonance Imaging used in medicine. MRI, allows doctors to see detailed synaptic activity in functional areas of the brain. For example, using MRI, it would be possible to see activity in “Broca’s area” of the brain, responsible for speech.
EEG is more simplistic, in that it uses a number of sensors (typically of the order of 10 and 100) to measure the frequency of brainwave patterns in regions of the brain. This allows an approximation of the mental state of the person being tested as follows:
- Alpha band (8-14 hz) that reflects calm, mental work.
- Beta band (14-30 hz) that reflects focused, engaged mental work
- Delta band (1-4 hz) that reflects sleep, relaxation and fatigue
- Theta band (4-8 hz) that reflects emotions and sensations
The lack of resolution in comparison to MRI means that we can only approximate the mental state experience by the respondent. Relaxation and boredom can have similar electrical reactions. Similarly, excitement and fear can have similar responses. However, the software combines information from the different frequency bands to estimate the emotional state of the respondent and chart how it fluctuates over time.
Test Session
We have planned test sessions with the following structure:
- Setup (15 minutes)
- Gameplay (15 minutes)
- Recall (5 minutes)
- Playback and speak aloud (15 minutes)
Setup
Allow 15 minutes to configure the eye tracking and EEG for each test subject. There are eye tracking solutions out there that involve wearing goggles that track eye movements – I haven’t tried these as yet, but they may be worth investigating. The Mirametrix eye tracker is a piece of hardware that sits just underneath the monitor that will be used to display the game being tested. I recommend having the monitor relatively high with respect to the subject’s eye level … that means that the eye tracking unit can also be placed relatively high (e.g. chin level) – this minimises the angle that the device has to the subject’s eyes. It’s recommended not to include subjects who wear glasses or contact lenses as this may interfere with the eye tracking.
The emotive EEG that we use is relatively simple to setup – it has 14 “biopotential” sensors built into a headset. Make sure it’s fully charged over USB before use! It’s wireless, and uses Bluetooth to send data to the laptop. Each of the sensors has removable pads which need to be soaked in saline solution – this ensures that there’s good electrical conductivity between the skull and the sensors. You should warn subjects in advance that their hair may get a little wet. I’ve found that wearing the headset in general is pretty comfortable, and you can even forget you’re wearing it, although wearing it for prolonged periods (over half an hour) can cause a little discomfort. Once the headset is on and correctly positioned, you will need to make slight changes to the position of individual sensors to ensure that a good reading is shown on screen for that sensor. We’ve found that it’s usually possible to get strong “green” signals for most sensors, but that it’s acceptable to have medium “yellow” signals on some sensors. We haven’t had any issues with hair length as yet, although it’s recommended to try to get the sensor as close as possible to the skull, pushing hair aside if required.
Test Session
Allow the customer to play the game (or perform whatever testing tasks you need) without interruption. This is significantly different to standard qual testing – the customer should not speak aloud what they’re doing, and shouldn’t be interrupted with questions, or indeed instructions if at all possible. Doing so will interrupt normal brainwave activity and may skew results. Allow significant time to allow the customer to get into the “flow” (term coined by Mihaly Csikszentmihalyi) of the game. Use iMotions Attention Tool to record the session – what’s on screen, eye tracking and EEG data.
Recall and Playback
Once you’ve recorded the test data you need (in our case 15 minutes), then we’re asking the customer to recall specific elements of gameplay. This is a standard qual technique just to check what specific elements of play stood out for the customer. This may help identify features that bore or excite the customer.
Once that’s done, our technique is to playback the gameplay, with eye tracking and EEG activity to the subject. Get them to talk through what they were looking at, what they were feeling and thinking. This allows you to assess ambiguities such as whether the subject was bored or relaxed.
Analysis
The key with this form of user testing is try to plot emotional state over time, and relate this graph to specific events in the game, such as a win, or tension building. We’re still relatively new at this particular technique, but I’m guessing that in addition to looking at specific stimulus / response patterns, we’ll need to start looking for repeating patterns of behaviour. Back at the beginning of my career while designing Windows applications, I used the concept of Maximal Repeating Patterns .. using logging quants data in applications to identify the largest repeating loops of behaviour in an application. This could help identify areas of improvement in the application, e.g. creation of macros, wizards etc. Looking for such repeating loops of behaviour and emotional states in game play could lead to deeper insights into game mechanics, not just interaction design. Watch this space!
Further reading:
http://www.gamasutra.com/view/feature/134710/game_testing_and_research_the_.php
