Author: graham

A user reported having problems with the bluetooth connection between the watch and the phone failing after a while on a fairly new phone (Android 13)

I experienced something similar overnight on my test system using a Samsung A20e phone (Android 11) and my Vivoactive 3 watch – the only way I found to get it working again was to switch the bluetooth off and on again on the phone. This makes me think it is an issue with Android rather than Garmin software (and I am confident it is not OpenSeizureDetector doing it).

This is quite a worry because two failures overnight is an unacceptable reliability.

Update 22/09/2022: I used an old phone (Samsung J5) and a VenuSQ watch for 24 hours without issue, so it must be a specific hardware or software issue. I am currently testing my normal Phone (Samsung A20e) with the VenuSQ to see if that shows the same problem.

Update 23/09/2022: My Samsung A20e has worked fine with the VenuSQ now for 24 hours, so the issue may have been specific to my Vivoactive 3 watch.

Has anyone else experienced this problem? If so, which phone, watch etc. were you using? Please get in touch (graham@openseizuredetector.org.uk, or comment on the post about this issue on facebook.com/openseizuredetector).
Thanks,

Graham.

I have been quiet for quite a while, so thought I should share an update…..

Executive Summary

The post below is quite long, but it can be summarised as follows if you do not want to read it all!:

• The data that users are contributing to the Data Sharing system is giving us the ability to assess the performance of OpenSeizureDetector and look at improving it – thank you to everyone who has contributed data so far!
• We have used the contributed data to demonstrate with confidence that OpenSeizureDetector does work and detects a large majority of seizures that involve shaking.
• Initial results from the Machine Learning (Artificial Intelligence, Neural Network) algorithm development are very promising to help improve false alarm performance – we hope to have a test version of OpenSeizureDetetctor that uses it in a few weeks.
• We always need more data to test and improve the system, so if you are not signed up to the Data Sharing system, please do, so you can help improve OpenSeizureDetector. Users who have regular seizures are particularly important to help us build a database of real seizure data.
• I am trying to decide how to publish the OpenSeizureDatabase for use by researchers – I would welcome users’ thoughts on this.

Data Sharing

Firstly, thank you so much to all the users who have been sharing their OpenSeizureDetector data using the ‘Data Sharing’ system. We are building a really useful database of data for both genuine seizures and also ‘seizure like activities’ that cause false alarms. This data is essential if we are to develop an improved seizure detection system that will both improve our seizure detection capability AND reduce false alarms.

Over the last few weeks I have been working with Jamie Pordoy (who is researching seizure detection for his PhD) to look at how best to use this data. This involves checking the data, filtering out events that appear to be tests of the system and working out how to collapse the dataset into unique events to avoid duplication.

I am pleased to say we are making good progress, and now have a system where we can test different seizure detection algorithms by running them on the data contributed to users so we can work out how the different algorithms perform compared to the original OpenSeizureDetector algorithm.

Current OpenSeizureDetector Performance (as of Version 4.0.x)

What we are finding is that the original detection algorithm which is in use by OpenSeizureDetector at the moment (Version 4.0.x of the phone app):

• has a detection reliability of about 81% for tonic-clonic seizures. (it is actually slightly higher after we were woken by an alarm at 0420 this morning, but that seizure is not included in the analysis).
• has a detection reliability of about 75% for all seizures.
• …does give quite a lot of false alarms (I still need to decide how best to calculate a false alarm rate from the available data)

I was very pleased with these results, because it is the first time we have had sufficient data that we can demonstrate with confidence that OpenSeizureDetector does work and detects a large majority of seizures that involve shaking.

We appreciate that the false alarms from the system are an issue for a lot of users, and would also like to get the detection reliability even higher, without increasing the false alarm rate.

Possible Tweaks to Improve OpenSeizureDetector

We have used the test system to try looking at different settings for the OpenSeizureDetector algorithm to try to reduce the false alarm rate, but what we found was that

• Increasing the ‘Alarm Threshold’ parameter sufficiently to reduce the number of false alarms (to ~5000) started to impact on the seizure detection reliability. We do find that setting it to about 900 reduces the number of WARNING events which makes the system quieter.
• Increasing the ‘Alarm Ratio Threshold’ parameter to 61 would halve the number of false alarms BUT would reduce the seizure detection reliability to 65% and 59% for tonic clonic and all seizure events respectively. This reduction in seizure detection reliability is significant, so the change is not recommended.

So it appears that the default settings in OpenSeizureDetector are close to optimum, so we are not likely to see significant improvements using the existing algorithm, so we will need something more complex.

Machine Learning Seizure Detection

Jamie and I are looking at using the data that users have contributed to train a Machine Learning (neural network, ‘Artificial Intelligence’) based seizure detector. There are several potential ways of designing and training this system. We have made some very good progress in the last couple of weeks, so that we have two potential models at the moment:

• One model is giving us a very good improvement in the seizure detection reliability, but is increasing the false alarm rate significantly, so will need further development.
• The other is giving modest reduction in the seizure detection reliability, but is giving a very large (factor of 3) improvement in false alarm performance.

So the two models we have at the moment are bracketing the target of having high seizure detection reliability with low false alarms.

I think there is scope to adjust the training method to sacrifice some of the false alarm performance to get the seizure detection reliability to be comparable to the existing OpenSeizureDetector reliability. If we can achieve this it will be a big improvement in the system performance.

If we can get a model that looks like it achieves this objective we will look at incorporating it into an update to OpenSeizureDetector to give users the chance to try it and see how its ‘real world’ performance compares. It is likely to be available to users who are contributing to the Data Sharing system initially, as we will need to use that infrastructure to get updates to the model to you as it is improved by more data being uploaded into the system.

Open Seizure Database Licence

The data that we have collected of measurements during seizures and seizure-like movements should be really useful for research purposes.

I am looking at making the anonymised dataset available for research purposes, so am trying to think of how to do this. The principle of OpenSeizueDetector has always been of complete openness – the source code and description of the algorithm are published for anyone to make use of, subject to an Open Source Licence.

I am looking for a suitable licence for the database but have not found one that feels quite right yet, so if anyone has suggestions, please let me know (graham@openseizuredetector.org.uk). The options I have looked at for ‘standard’ liences are:

• A very ‘open’ license that just requires users to credit OpenSeizureDetector contributors in the work that uses it. This would mean that researchers/companies could use the data without contributing back to OpenSeizureDetector.
• A ‘non commercial’ license that prevents commercial use of the data. This has the same issue as above, but would mean the data could only be used for non-commercial research purposes.

Neither of the above options quite achieves what I would like, which is that the users of OpenSeizureDetector who have contributed the data have the opportunity to benefit from it in terms of improvements to OpenSeizureDetector.

What I am proposing is a variation to the very ‘open’ licence by including a condition on the use of the data that the results from using the data must be published (as well as crediting OpenSeizureDetector for the data) – draft is here.

I would welcome users thoughts on this approach.

Conclusions

• The data that users are contributing to the Data Sharing system is giving us the ability to assess the performance of OpenSeizureDetector and look at improving it – thank you to everyone who has contributed data so far!
• We have used the contributed data to demonstrate with confidence that OpenSeizureDetector does work and detects a large majority of seizures that involve shaking.
• Initial results from the Machine Learning (Artificial Intelligence, Neural Network) algorithm development are very promising to help improve false alarm performance – we hope to have a test version of OpenSeizureDetetctor that uses it in a few weeks.
• We always need more data to test and improve the system, so if you are not signed up to the Data Sharing system, please do, so you can help improve OpenSeizureDetector. Users who have regular seizures are particularly important to help us build a database of real seizure data.
• I am trying to decide how to publish the OpenSeizureDatabase for use by researchers – I would welcome users’ thoughts on this.

V4.0.7 of the Android Phone App should be released for testing shortly (subject to Google approving the release…). The changes relate to improving the user interface for marking events as either real seizures or false alarms in the Data Sharing system. The specific changes are:

V4.0.7 - Improvements to Data Sharing data log manager screen
    - Removed automatic refresh of shared data events list (Issue #62)
    - Added option to include warnings in shared data events list (Issue #64)
    - Added catch of NullPointerException in LogManager in case of network disruption (Issue #61)
    - Added 'demo mode' on main screen when phone data source selected and highighted it in warning colour (Issue #56)
    - Changed default local data retention time to 7 days (from 28) - works around #63

Anyone registered as a beta tester should be invited to upgrade to this version over the next day or so – please let me know if you have any issues.

Graham (graham@openseizuredetetor.org.uk)

We are finding that users are uploading a lot of events to the database (either genuine alarms/warnings or false alarms), but we have a lot which have not been marked to show if they are real seizures or false alarms – and we can not use the data without that. I am going to implement some improvements in a new version of the app. The ones I intend to do are:

• Allow the events list to be filtered to only show alarm events and not warnings (because it is much more important to mark the alarms as seizure or false alarm than it is the warnings). (https://github.com/OpenSeizureDetector/Android_Pebble_SD/issues/64)
• Switch off the automatic updating of the events list, because this makes it difficult to scroll down the list without it jumping back to the top when it refreshes (https://github.com/OpenSeizureDetector/Android_Pebble_SD/issues/62)
• Fix issue with the data log manager screen getting very slow after a while (make the local database access happen in the background, and change the default data retention period to 7 days rather than 28) – https://github.com/OpenSeizureDetector/Android_Pebble_SD/issues/63
• Make the ‘Unvalidated Events’ notification clear if there are no longer unvalidated events – this will make the notification appearing have more significance for the user than if it stands constantly – https://github.com/OpenSeizureDetector/Android_Pebble_SD/issues/58

Please let me know if there are any other data sharing related issues that you would like to see addressed in this update (graham@openseizuredetector.org.uk)

Thanks, Graham.

It has been a while since we released Version 4.x of OpenSeizureDetector, which includes the ‘Data Sharing‘ feature. This feature allows users to contribute to the development of OpenSeizureDetector by sharing data recorded during seizures or false alarms anonymously with researchers and developers to help improve the detection algorithms.

Since we went live with the feature, over 100 users have signed up and the database now contains over 10000 individual ‘events’. An event is an instance where OpenSeizureDetector goes into a warning or alarm state, or the user reports a seizure manually. We then upload data points covering a 5 minute period either side of the event – so we are generating a good amount of ‘real world’ data which will be really useful for testing new seizure detection algorithms.

A key part of the Data Sharing system is the ability for users to categorise the events to say if they are real seizures or false alarms – doing this is really, really important to us, because without that we do not know what to do with the data – do we develop the algorithm to make it more likely to detect events like the uploaded one or less likely to detect it?

Now an activity which generates a warning or alarm condition may well generate several warnings or alarms over a short period, so we have been working on a way of condensing all the contributed data down into a more manageable set of ‘unique events’ which covers a period of activity that generated one or more alarms. Based on the current version of the data condensing code, we have the following numbers of unique alarm events in the database*:

• 24 Tonic-Clonic Seizues
• 62 Seizures in total (of various types)
• 121 classified false alarms (that is the user has confirmed the false alarm and said what caused it).
• 167 events marked as unknown
• 589 unclassified events.

(* Note that this still needs some more checking so the numbers are likely to change – the numbers are lower than you might expect, but this is because unless the event is marked by the user as a seizure, we have ignored events that only produced a warning state, not a full alarm at the moment)

I have a test system that I can use to determine the detection reliability and false alarm rate based on this contributed data – I will report those results in the next week or so once I have checked it some more (the results I have at the moment look a bit too good to be true, so I am looking for a mistake!).

It is clear from the data above that we are building a very useful database of both real seizures and false alarms generated from users’ normal activities. The ‘unclassified events’ list is by far the largest though. Jamie Pordoy, who is using this data to support his PhD research reports that there are quite a few events in the ‘unclassified’ list that look very like real seizures to him – so if you have been contributing data and have had a seizure that you have not reported, please will you get in touch so we can try to identify it in the database?

Note that real seizure data is vitally important to developing better algorithms, so if you know anyone who does have regular seizures, please will you encourage them to use OpenSeizureDetector and contribute data, even if they are not particularly interested in the alarm function for themselves.

I know that marking the events as real seizures or false alarms can be difficult, especially once you have a lot in the list, so please get in touch with any suggestions that you think would make it easier for you to do. The things I have thought of as options so far are:

• Filter the events to list only ALARM conditions (as these are the ones we are most keen on you marking.
• Provide a web page based list of your events that you can scroll through more easily on a bigger screen.
• Provide a feedback function where researchers can ask you explicitly about an event that looks ‘interesting’ and ask you to describe it in more detail – either through a notification in the app, or an email?

Other ideas and feedback would be appreciated!

Thanks for all your efforts in contributing data – keep it up!

Graham (graham@openseizuredetector.org.uk)

Version 4.0.6 of the Android Phone App went live on Google Play Store today so existing users should receive upgrade notifications from Play Store. There is also a new version of the watch app (V1.2) to go with it. There are a few things to be aware of when upgrading:

• The system is a bit more sensitive than you are used to (this is because I fixed a calculation error). This means you probably want to change the Alarm Ratio Threshold to the new default value of 57 rather than 50 to reduce false alarms.
• Depending on your watch, you might find it too sensitive to very small movements. We increased the Alarm Threshold to 900 on Benjamin’s watch to reduce false alarms when he was asleep and not really moving.
• You may see a warning about the android system optimising the battery usage. This will give rise to faults when running on battery. See the Troubleshooting page for instructions on how to avoid it being ‘optimised’.
• You will see three dialog boxes about permissions when you enable SMS alerts – this is a requirement to get it published on Play Store so I had to have that many, sorry!

Finally, please enable the Data Sharing feature and report alarm activations as either real seizures or false alarms – We are hoping to have a neural network based detection algorithm to give better discrimination between real seizures and other activities, but we need as many seizures recording as possible to train it – Jamie Pordoy is working on this as part of his PhD project and if successful we will incorporate it into a future release of OpenSeizureDetector – but we need your help to do it, so please enable Data Sharing.!

Please get in touch if you have any issues.

Graham (graham@openseizuredetector.org.uk)

Thank you to those users who have installed V4.0.x of OpenSeizureDetector for testing and provided feedback over the last few months. V4.0.6 is waiting for final approval from Google, then it will be released on Play store over the next day or so. I have also released a new version of the Garmin Watch app to go with it.

Phone App

All new users will receive V4.0 and existing users will be asked to upgrade – please upgrade to this latest version as there are quite a lot of small fixes in it as well as some major changes. The main changes are:

• Introduce a ‘Data Sharing’ facility to upload data that causes alarm or warnings to help us improve the OpenSeizureDetector algorithms. Please make use of this, and use it to report real seizures and false alarms so we can build a better detection system.
• Fixed an issue that meant that SMS alerts did not contain the link to the user’s location on some versions of Android.
• The system checks if it is being ‘optimised’ by the Android system and warns the user to disable battery optimisation if it is (optimisation results in the bluetooth radio shutting down and the watch displaying ERR: -xxx errors).
• Resolved quite a lot of issues that prevented the system shutting down properly sometimes, which gave unpredictable behaviour.
• Corrected a mathematical error that had been there for a long time (a bracket in the wrong place). This has made the system more sensitive, so you are likely to want to increase the value of the “Alarm Ratio Threshold” setting to the new default value of 57.

Watch App

There is also a new version of the Garmin Watch app available here: The new version of the watch app is available here: https://github.com/OpenSeizureDetector/Garmin_SD/releases/tag/v1.2

Please upgrade to the latest release of the watch app. It provides the following additional features:

• It sends the 3D data to the phone for logging – this has been identified as being beneficial for seizure detection by some of the research work using the Data Sharing results.
• Fixed a crash on some devices if you have multiple alarms in rapid succession.
• Provides user feedback of an alarm condition or communications fault by buzzing or beeping from the watch.

As usual, please let me know if you have any issues with this release – I intend to roll this out to all users in about a week, so please let me know as soon as possible if you find an issue with it!

Regards

Graham (graham@openseizuredetector.org.uk)

Thank you to those users who have installed V4.0.x of OpenSeizureDetector for testing and provided feedback.

I have taken advantage of the long weekend in the UK to do an update which will be available on Google Play Store for people who register as Beta Testers in the next few hours (assuming Google do not reject it!).

The main changes are:

• Fixed a bug that crashed the app if you used quotation marks in the description of a seizure-like event.
• Added the ability to save 3D acceleration data as well as the vector magnitude value that we have been saving up till now. This is to support Jamie Pordoy who is developing a machine learning (neural network) based seizure detector as part of his PhD and expects to be able to improve the model using 3D data.

Existing Beta testers should receive an upgrade notification automatically. If other users would like to try this please use the links below to register as a Beta Tester:

• Web browser: https://play.google.com/apps/testing/uk.org.openseizuredetector
• Android Device: https://play.google.com/store/apps/details?id=uk.org.openseizuredetector

Note that to accompany this version of OpenSeizureDetector there is a new version of the Garmin Watch app (V1.2) which has a few significant improvements:

• It sends the 3D data to the phone for logging, so is necessary for full compatibility with V4.0.x of OpenSeizureDetector
• Fixed a crash on some devices if you have multiple alarms in rapid succession.
• Provides user feedback of an alarm condition or communications fault by buzzing or beeping from the watch.

The new version of the watch app is available here: https://github.com/OpenSeizureDetector/Garmin_SD/releases/tag/v1.2

As usual, please let me know if you have any issues with this release – I intend to roll this out to all users in about a week, so please let me know as soon as possible if you find an issue with it!

Regards

Graham (graham@openseizuredetector.org.uk)Poste

Has anyone received some spam email from a graham@openseizuredetector.org.uk email address?
I just got a failed delivery notification for an email I didn’t send….
If so, please let me know and I will investigate further…..
Thanks,
Graham.

OpenSeizureDetector V4.0.4 is now available on Google Play Store for anyone who signs up for ‘Beta’ Testing by using one of the following links:

• Web browser: https://play.google.com/apps/testing/uk.org.openseizuredetector
• Android Device: https://play.google.com/store/apps/details?id=uk.org.openseizuredetector

The main change is the introduction of the Data Sharing feature that we have been promising for ages :). It is probably best if users start from a fresh installation by selecting apps->OpenSeizureDetector->Clear Data in the phone settings to make sure you use the new default settings, especially if you get unexpected results.

I have written a set of instructions, which are now available on the Installation, Set-Up and Testing page which is visible on the web site main menu. I am thinking I might make a video walk-through of the installation process if that would be useful?

It would be very helpful if existing users sign up and test this version and feed back their experiences – in particular about the Data Sharing system where the user interface is new, so I am sure there are improvements to be made to make it easier for the user to record when the system detects genuine seizures and false alarms.

Regards

Graham (graham@openseizuredetector.org.uk)