Emotiv Headset and Locked In Syndrome - Part 2

Today I visited Shah again to find out how he had got on using the Emotiv Headset, and the results were interesting. Shah was not using it very much now - because he had regained some movement in his finger - and this means that he will now have access to more traditional technology, such as a switch or maybe a computer mouse - very good news!

The inevitable question is - did the Emotiv headset help Shah get his finger movement back? Shah is skeptical that it had any impact. However general feedback from some of those involved was that there was a possibility that he may have recovered some movement because of the increased brain activity from using the headset.

This was not a clinical trial - it was an experiment to see whether such a device would help - so we will never know the true impact of it, or whether his finger movement would have recovered at the same rate anyway. But as his family told me, the headset demonstrated that it *was* possible for him to get some degree of personal control back, and that in itself was encouraging.

And what about the headset? Would Shah recommend trying it to others in a similar position? Yes! Has he given up using it now he has another option? Nah - he's a techy at heart and likes to play with it - just like the rest of us!!

Emotiv Headset and Locked In Syndrome

In March, a salesman working for IBM had a stroke, which left him with complete paralysis, unable to use his muscles, and without the ability to speak. His brain however is working fine - a condition called Locked-In Syndrome. His means of communicating is by his eyes - looking up for yes, and down for no. He has to wait for someone to ask him if he'd like to speak before being able to do so. Then, using a letter chart, that someone must point at letters one by one, until a confirmation is received. This is the first letter of the sentence. The process must be repeated for every other letter in the sentence, until the full sentence is spelt out.

The salesman in question is called Shah, and Sarah (my wife) is his Occupational Therapist (OT), who saw the mind-reading headset from Emotiv that I was using in IBM and thought about the potential this could have for Shah. The headset was designed for the gaming industry, and measures facial expressions, excitement/boredom levels and can be trained to listen for particular thoughts, which can perform an action on the computer.

Sarah asked me to demonstrate it to the consultant, speech therapist, psychologist and hospital ward manager - who agreed it may have some potential, and who took it to the ethics committee. Shah was informed about the headset and what it can do - but that we weren't aware of any person with the symptom who had tried it before. Shah, being a bit of a techy, was up for pioneering it - so I met him last week for his first go.

Just like the many other people I have demonstrated the headset with, Shah was instantly able to train the system so that one action - the "push" action - would push the floating "Emotiv cube" into the distance. As we trained, the ability to push (and stop pushing) at will, improved.

Adding a second action adds a complication. Now the unit has to distinguish not just whether you think or don't think, but which thought you are having - much harder! And like everyone, it will take a bit of time to practise getting sufficiently good at this to control. However Shah is up for the challenge, and has a fantastically supportive family who are willing to help him train.

Last week, after one week of use, Shah had managed to train the headset very well - though when tired the Emotiv skill can go down as well as up. Working with the speech therapist, we have connected the output of the headset to the input of "The Grid 2" - a piece of software by Sensory Software which allows a user, normally via eye tracking, keyboard, mouse or switches, to control their environment, write emails and surf the internet. We have initially set up 3 different menu items, and thinking "push" starts highlighted these options in turn at 10 second intervals. Thinking push again, will select one of the options. Sounds easy perhaps - but if I tell you to *not* think of a rabbit - you can't help but think of a rabbit! It's hard to think about when to start and stop thinking, and then change to think about the thought you are supposed to be thinking about! I think this will take some practice to achieve, but once achieved could be widened to many more options, with very little extra practice.

Often people ask about using the headset for people with different conditions, and in different places. What with Shah being a fellow IBMer who likes gadgets, and his OT being my wife, and with the particular medical condition he had, conditions were ideal to try out the headset. It will take time for us all to get it right - but for now it is looking good. I will try to keep you updated!

Wearable computing devices

Time to do a bit of blogging again. I've recently been asked about various wearable computing devices... here's a starter list, but I'd be interested in any other links you may have.

AR Glasses: These plug into a phone and can overlay data like directions / emails etc whilst travelling.

Gesture Gloves: Very accurate hand movement recognition.

Gesture Watch: Accelerometer, temperature, pressure sensors - also can act as wireless hub for other sensors.

Sixth Sense (wearable video/projector): Gesture recognition, ability to detect objects and augment real objects with data by projecting back on to them. Very cool.

Emotiv Headset: Brain reading device built for gamers which reads facial expression, excitement/engagement, trained actions and head movement.

NeuroSky Mindset: Brain reading device which measures excitement/engagement. Chip has the ability to add other sensors.

Mobile Phone Jewellery: Article from 10 years ago about IBM going into jewellery that can be used as a mobile phone.


We have used some of these sensors in the ETS Lab... more in a follow up post!

Using Brain Computer Interfaces for something useful

There are now a whole collection of Brain Computer Interfaces around... and this latest one from the games company Mattel uses the power of concentration to make a ping pong ball hover. The technology is supplied by NeuroSky, who have also provided the technology for a Star Wars game - $50-$100 for the fun of using "brain waves to allow players to manipulate a sphere within a clear 10-inch-tall training tower..." - something I could imagine myself doing just the once. Never-the-less it's quite good to see the technology coming down in price.

What fascinates me however is how we might be able to use brain waves for real personal benefit and integrate it more with our every day computer interaction experience. People who have severe motor difficulties could use it as a rehabilitative aid. Performing product research, it can be used for detecting when people are excited or not when they experience a particular product or website. When we are browsing on-line, we could combine it with eye tracking to provide feedback on what really interests us, and then tailor our online experience accordingly. When we go to the shops... the cinema... driving in the car... feedback can be provided to enterprise systems in all sorts of situations, which in turn can affect the environment around us. I'm really looking forward to talking with a number of customers about how they can integrate this and other interface technology in their own innovative projects in 2009!

Calculating energy usage is harder than it seems

In my last post, I said a Christmas project was to monitor my usage - but I confess I've got a problem of how to measure my energy usage which is so far unsolved.

As I said, I have a currentcost meter (m1) which reads the current going into and out of the house. Of course, it cannot determine which way current flows, so it reads 50W regardless of whether the 50W is imported to, or exported from, my house. Therefore, I put a second meter (m2) round the wire from the solar panel, which told me how much the panel was generating and was thinking that if the value on m2 > m1 I must be exporting, otherwise I would be importing. However this thinking is flawed:

If I am generating 100W, (m2=100) and I am using 50W, then I will export 50W (so m1=50).
If I am generating 100W, (m2=100) but I am using 150W, then I will import 50W (so m1=50).

So from my two readings, I cannot conclusively say whether I am importing or exporting electricity, unless I am missing something obvious? The bright green import/export meter which Southern Electric fitted manages to work out what is being imported and what is being exported - but there doesn't seem to be a nice little serial port anywhere obvious! The other option is to buy a wireless unit as an addition to the inverter but this costs £120 and doesn't appear to give me open access to the data.

BTW - in 20 days we've generated 16.2 kWh and exported 9 of them. A grey day typically generates 200Wh the whole day - a sunny winter's day like today seems to generate about 2kWh.

The solar works

In the loft
The solar panels are connected to an inverter (right - click photos to enlarge) , which converts the DC generated current into AC, using the "only clever bit" of the entire installation, according to the installer. This is mounted just inside the entrance to the loft. The interesting thing about the blue box is the "knock interface"... to switch on the unit and light up the screen you have to knock on the grey area. The screen tells you how many watts the panels are generating, and how much have been produced during the day, and how much in total.

In the kitchen
Downstairs, next to the consumer unit is a fantastic red knob, which isolates the panels. Above it is a meter which also shows how much electricity has been generated by the panels in total. During the first day of operation, the unit generated a total of 700W - disappointing considering it was bright sunshine outside all day, and the unit is spec'd at 1000W per hour. The installation engineer reassured me that as the sun got higher during the summer months the unit would generate much more - we shall see! During the second day of operation, when it was cloudy, the unit generated a total of just 200W during the day. Not good.

Metering
The electricity meter has been switched to an import / export meter. The meter gives the amount exported to the National Grid (ie total produced by the solar panel minus total consumed at the time in the house), and also the amount imported from the Grid. You can see the current cost clamp in the picture. This is interesting, because the current cost meter measures the flow of electricity regardless of which way it flows... so it reads 100W when the house consumes 100W, and it reads 100W when the house exports 100W. It was highly confusing turning on a light to see the amount on the meter decrease! So one of my Christmas projects is to connect another meter to the solar panel cable, and add the two readings together in order to get a sensible output... more on that later.

Solar Panel installation

Here's a photo of the solar panels which were fitted to our roof today by Rayotec. They should provide us with 1kW electricity per hour in peak sunshine, but just around 100W per hour in horrible foggy weather like today. Tomorrow morning they will be grid connected, so any extra electricity we don't use will go back into the grid. With Southern Electricity giving us slightly more per unit than we pay for our electricity, there's still an incentive to use as little as we can. We expect the unit to just about cover our usage, over a yearly period. It cost around the same as having all our doors and windows replaced a few years ago and we're promised a similar length of service. So, we're just looking forward to the grand switching-on ceremony tomorrow morning!