Alternative (Machine Learning) Detection Algorithm

Why?

The current Seizure Detection Algorithm works better than I had expected it to, given that it is quite simple – it has detected real tonic clonic seizures and gives (for us) an acceptable level of false alarms.

But it is far from perfect – it gives false alarms for several quite normal activities such as brushing teeth, and the fall detection algorithm is not reliable. For our own use, this is good enough – false alarms are not a particularly big issue because we monitor Benjamin using CCTV, so the warning or alarm beep is just a prompt for us to check on him. For more able users though that are using the SMS alert system, false alarms are a more significant issue as they will have to ring around the people notified to tell them that they are ok.

So it would be good to be able to reduce the false alarm rate and maintain the detection reliability.

Options

There are a few different machine learning techniques that we could use as an alternative to the deterministic algorithms we are using at the moment.

1. A ‘conventional’ neural network classifier: Trained to analyse a snippet of accelerometer data and assign a probability of the data representing one of a number of classes of movement (ie. Seizure, Fall, Normal). There are several variations of this option – Dense Neural Network, Convolution Neural Network, different depths and sizes of network etc.
   • Pros:
     • Easy to code
     • low computation cost for live data.
   • Cons:
     • Tricky to get settings right to train it.
     • Need lots of data – in particular we will need real seizure data for training, which is difficult to get hold of.
2. Autoencoder Anomaly Detector: Teach an autoencoder network to recognise ‘normal’ behaviour. Then analyse live accelerometer data to see if it appears ‘normal’ and alarm abnormal behaviour, which could be a seizure or a fall (or an activity that was not in the ‘normal’ dataset). We will differentiate between ‘normal’ and ‘abnormal’ based on how well the autoencoder can reproduce the input data – if it can reproduce it well, it has been trained on it, so it must be normal. If not it is abnormal – same method as in this paper. Similar variations to the conventional neural network option above.
   • Pros:
     • Does not require real seizure data for training.
     • low computation cost for live data.
   • Cons:
     • Tricky to get settings right to train it.
3. A Nearest Neighbour (KNN) anomaly detector. Similar to the autoencoder approach, but a simpler algorithm (the disadvantage is the amount of computation required on the live data)
   • Pros:
     • Does not require real seizure data for training.
     • simple to train
   • Cons:
     • High computation cost for live data.
     • Not sure how to do the anomaly detection part once we have the autoencoder working – we may need a KNN technique on the back of it?

Data Available

• DIY Data: The OpenSeizureDetector Garmin Data source has all of the accelerometer data on the phone, so we can record this for later analysis – I have a few days of me wearing the watch all day and recording data – but it is not classified into different activities – it is all just ‘normal’ Classifying it is tricky – we’d need an app to easily classify different data as different activities in pretty much real time to make classification feasible.
• SmartFall Dataset: Accelerometer data that is marked as either a fall or ‘normal’ – would be useful for testing the anomaly detector.
• Other Fall Detection Datasets: There are a few more published fall detection datasets that I have identified, but haven’t looked at the data yet.

The Plan

Collect Some Data

• Essential
  • Use the Garmin Seizure Detector to collect
    • some ‘normal’ data (just wear it for a few days while it logs raw data)
    • some simulated seizure data – simulate several minutes of seizure like movements that we would like it to be able to detect.
    • [We have some of this data here]
• Optional
  • Modify the Pebble Watch seizure detector (and associated phone app code) to make it transmit raw rather than processed data, so that more users can contribute data [Not done yet – I have an issue with de-coding the data sent – probably ‘endiness’ of the data].
  • If we are going to ask users to contribute data we are likely to need to automate it – make a web app and database to receive the data, and modify the Android app to send to it [The reason I haven’t done this is that I am a bit worried about how to handle privacy issues, and to be really useful we would want the data tagged with some information about what sort of activity it was, which is not that easy to do].

Assess Reliability

We will need a way of evaluating the alternative methods to see if they are any better than the one we already have. We will do this by:

• Make a software interface to the available data that reads it and pre-processes it in a standard way so we have exactly the same data for each algorithm we test.
• Split the data into a ‘training’ and a ‘test’ dataset
• Train the new algorithm using the ‘trainining’ dataset, and use the ‘test’ dataset to calculate the detection reliability and false alarm rate. Again we will use the same calculation method for all algorithms under tests.
• Decide on what statistic(s) to measure to evaluate detection and false alarm reliability to be consistent with published papers on the subject – need to check what other researchers use.

Evaluate Existing Algorithm

I am somewhat ashamed to admit that I have never tried to do a formal evaluation of the existing algorithm – I just worked on it until it worked well enough for us! We will first evaluate the existing algorithm by:

• Collect several days of normal living accelerometer data (at the moment we need to use the Garmin seizure detector for this because the Pebble one processes the data on the watch and sends processed data to the phone to save radio communications to conserve battery)
• We should really do this for several test subjects because they will have different movement natural frequencies.
• Generate several minutes of simulated seizure like movements (just because real seizure data is so hard to come by because (fortunately) users do not have seizures that often.
• If a user does have frequent seizures and would be willing to contribute data, this would be much, much better though.
• Set up python implementation of the seizure detection algorithm (I currently have a C version that runs on the Pebble watch and a Java one that runs on the android phone). The reason for this is that the machine learning techniques are much easier to develop in python.
• Run the normal life and seizure data through the algorithm to work out the detection reliability and false alarm rate.

Visualise Data

Implement a system to allow us to visualise the data to help with developing new algorithms. Things like:

• Plot the raw accelerometer data and fourier transform of it for each data point so we can see if there are any ‘odd’ features in the data.
• Use statistical methods such as Principal Component Analysis (PCA) to collapse the multi dimensional dataset (either 10 frequency bins per sample, or 125 individual accelerometer readings per sample) into something we can visualise to produce a point cloud of data so we can see where seizure data sits in relation to the background of ‘normal’ data points.

Prototype Alternative Algorithms

• Implement a prototype of the aternative algorithm in python
• Evaluate its reliability compared to the current detection algorithm.
• Adjust parameters of the algorithm to see what improves it.

Autoencoder Based Anomaly Detector

Prototype implemented as autoencoder_anomaly.py.

The first objective is to train a network that will take accelerometer data as input and produce a good representation of the input data at its output. The idea is that it will be able to do this well for ‘normal’ data, but will not do so well for ‘abnormal’ data, so we will be able to detect abnormality by the deviation between the input and output data.
After quite a bit of fiddling to get an activation method that would work for our data (which has both positive and negative values, so the common ‘relu’ method fails), it now seems to train reasonably, reaching a minimum loss function after about 20 epochs.

Using some ‘normal’ test data shows that the model gives reasonable reproductions of the input data (below the red lines are the model output and the blue lines the original input data).

Implement New Algorithm in OpenSeizureDetector

• Once we have identified a better alternative algorithm, update the OpenSeizureDetector Android App to include an option to use the new algorithm instead of the original.
• This will require porting the new algorithm to Java, as it will be written in Python for prototyping.
• Give it to users to test and seek feedback.

How to Help

• If you have any experience of Machine Learning, please get in touch with graham@openseizuredetector.org.uk and make any suggestions for improvements to the above! (Thank you, Iain for doing this already!).
• If you know anyone that is into machine learning, please see if you can interest them in this.
• Contribute Data
  • If you use the Gramin Seizure Detector, in the settings switch on the ‘Log Data to SD Card’ option in the ‘General Settings’ page.
  • This will record the accelerometer data onto your pone.
  • If you have a seizure while this, I’d really like the log file output around the time that it happened if you don’t mind – We can come up with a way of making the data anonymous if you prefer.
• Testing
  • Quite a way off, but we will need to do real-world testing if we find an algorithm that appears to be better than what we have at the moment.

Progress

Autoencoder Based Anomaly Detector

First attempt was a simple neural network using just dense layers. It looked sort-of promising – the network initially trained well with the error decreasing, but the error hit a plateau and no extra training improved it. When I finally looked at the input and output waveforms, I got this:

Here it seems that positive signals are being reproduced reasonably (Red line is the output, blue the input), but negative signals are being clipped at zero. It seemed that this the use of ‘Relu’ activation which does truncate at zero. So we changed it to ‘tanh’ activation instead, and normalised the data to be -1 to +1 (which is needed by tanh), and it looks more promising:

Source Code

All the source code for this work will be on Github at: https://github.com/OpenSeizureDetector/ANN_SD

If you have any suggestions or want to get involved, please drop me a line at graham@openseizuredetector.org.uk or our face book page.