Our smartphones are spying on us all, and I am now subject to a new kind of surveillance. TapCounter tracks every time I touch my phone screen. My swipes and jabs average around 1,000 per day. However, I notice that this is decreasing as I avoid social media in order to meet my deadline. QuantActions, a European company, claims it can detect important indicators of mental/neurological health by analysing and capturing data.
Arko Ghosh, the cofounder of the company, is a neuroscientist from Leiden University in The Netherlands. Tappigraphy patterns can be used to determine slumber habits (tapping during the early hours of the morning means that you are not asleep) and mental performance level (the small intervals between key-presses indicate a proxy for reaction speed), he said.
QuantActions will launch features that are based on these insights in the next year. Ghosh hopes to make use of the technology in the future for medical purposes, including the prediction of seizures in epilepsy patients. Ghosh published this year a small clinical study with epilepsy patients that showed how subtle changes in smartphone appigraphy could be used to predict abnormal brainwaves. Ghosh says that we hope to be able to predict the next episodes.
My taps and Ghosh's work are part of a rapidly growing field called digital Phenotyping. This project aims to analyse the vast amounts of data available from smartphones, wearables, and other digital devices using artificial intelligence (AI). It will then use this data to draw conclusions about health and disease behaviours.
It could be a new way to diagnose and monitor a variety of medical conditions, especially those related to brain or mental health, if digital biomarkers, which are symptom-related signals, can be properly analyzed. Early research has shown that patterns in geolocation data could correlate with episodes and relapses of schizophrenia. Certain keystroke patterns could also predict bipolar disorder mania. And toddlers' gazes at smartphones could be used as a way to detect autism.
Data streams can include smartphone activity logs and measurements from any of the built-in sensors on a phone (such as the GPS or accelerometer) as well user-generated content that can be used to mine for words or phrases. Brit Davidson is an assistant professor of analytics at University of Bath. She has been closely following the development of the field.
Digital phenotyping allows for continuous behavioral data collection to capture a person's experiences
This technology is attracting the attention of major tech companies. The Wall Street Journal reported in September that Apple was working on iPhone features to diagnose cognitive decline and depression. Google and others are also interested. Apple may be hoping to be able correlate different phone indices with other indicators it considers to show mental health disorders. Helen Christensen is a professor of psychology at the University of New South Wales, Australia, and also heads the non-profit Black Dog Institute that focuses on the prevention, diagnosis, and treatment of mood disorders.
Silicon Valley-based startups in consumer health and wellbeing are already using aspects of the technology to improve their products. However, they have not yet been able to use it for clinical diagnosis. Mindstrong offers therapy and psychiatry online and has been funded with tens to millions of dollars, including by Jeff Bezos' venture capital firm. Mindstrong claims it uses patented technology that tracks how clients scroll, click and tap on their phones to provide better care. Ginger, which provides on-demand mental healthcare support, has a spokesperson who said that the company uses a very basic form of digital phenotyping to analyze text conversations between users and coaches in order to give insights. Nicole Martinez-Martin, a Stanford University bioethicist who studies digital health technology, says that transparency about the activities of some companies can be difficult to find.
Traditional methods of diagnosing mental illness have relied on the patient's own experiences and a medical assessment done at a clinic. It captures only one moment in time and is highly subjective. Digital phenotyping allows for continuous behavioral data to capture a person's lived experiences. It could help us diagnose people more accurately, according to Jukka-Pekka Onnela (a Harvard University biostatistician who helped pioneer digital Phenotyping). Onnela is also part the leadership team for a large collaboration with Apple to study women's health, including tracking their menstrual cycles through the iPhone.
Illustrations of Ginger, the emotional support app Ginger. Photograph: www.ginger.com
In a world with increasing mental health issues and stretched services, it could be possible. It could help people manage their mental health better, and anyone who isn't sure if they have a problem should be notified. Christensen says it is worth looking into. It would be a huge breakthrough if we could find the data to be relevant.
Cogito Companion shows how this technology could be used in the future. It is an experimental tool for clinicians to diagnose anxiety and mood disorders. Cogito, a Boston-based startup, developed the tool with funding from Defense Advanced Research Projects Agency. The company hopes to be able use it medically for military veterans and personnel. It is currently being tested in 750 sailors who returned from overseas deployments. The technology has been used by Cogito to create an AI coaching product that helps call centre staff become more empathic. It is also sold commercially. The tool is installed as an app on participants' phones and passively monitors for signs of social interaction collapses and indicators that activities are being avoided. It also examines changes in call patterns, text, and mobility data. The tool also analyzes the voice recordings of participants to find signs of depression. Skyler Place, chief of behavioural science for Cogito, said that about 200 signals are analysed from the voice, including energy, pauses, and intonation. A total risk score is then sent back to the person's clinician for support in diagnosis and treatment.
While there is much promise, digital phenotyping science still has a long way ahead. Privacy and whether it will be a technology that best serves society are some of the questions.
First, it takes a lot of work to show that medical information is possible. Many published academic studies were pilot studies. Ghoshs epilepsy research, for instance, only involved eight people. Mindstrongs only cites a single study involving 27 people as the scientific evidence supporting its product. Christensen says that the algorithms will be used in medical research and require large numbers of participants. Pilots are giving way to larger studies. This trial of the Cogito Companion is based on a smaller proof–of-concept study.
Research is now beginning to include healthy individuals. Pilot studies are often limited to those with certain conditions. BiAffect is a study that examines keystroke behaviour in order to predict bipolar episodes. It was developed by researchers at the University of Illinois at Chicago. The open science component of the BiAffect study allows anyone to download an app to take part. This will allow us to better understand the differences between bipolar disorder sufferers and healthy people. There are approximately 2,000 participants.
In action: The Cogito Companion App Photograph by Cogito
Apple's interest in smartphone diagnosis may be rooted in previously announced research collaborations with Biogen to study digital biomarkers of anxiety and depression with the University of California at Los Angeles (UCLA), and mild cognitive impairment with the pharmaceutical giant Biogen (which has a controversial new treatment). Both companies are collecting a variety of data from iPhones and Apple Watches, and then combining it with survey responses and cognitive tests. The UCLA study began with 150 participants in a pilot phase and will continue with a main phase that includes 3,000 participants. This phase will also include healthy individuals. Biogen's September study plans to enroll a mixture of participants (23,000) and recruit from the general public.
Experts say the primary scientific problem in the field is the noise of the data. Because people use their phones in such different ways, it can be difficult to compare the behaviour of individuals and even within the same person over time. It can be difficult to see the connections between offline and online behaviours. Brit Davidson suggests that a sudden drop of smartphone-based communication could indicate social withdrawal. However, it may also mean that someone is communicating face to face.
What happens if a doctor or patient views an algorithm assessment as objective?
The biases in AI-based technology can also lead to certain people being negatively affected or not benefitting from it. This is well known in AI-based technology. Stanfords Martinez-Martin notes that a lot of health-related research is done with white, educated, wealthy populations. She says it is difficult to know how it will transfer.
Privacy concerns are also a concern. Data gathered for academic research follows strict protocols. However, data gathered by private companies can lead to more complicated issues. Martinez-Martin says that companies can share data that includes digital phenotyping methods that make predictive inferences. This could have significant impacts. These inferences may be of interest to insurers, employers, and education providers, but they could also be used in a negative way, she states. Data that is anonymized and deidentified does not mean it cannot be reidentified.
While sensitive health information is protected under US law, it only applies to data collected by healthcare systems and not by tech companies. It is not clear, however, whether the established definitions for sensitive health information encompass the type of information digital phenotyping seeks to collect. Martinez-Martin says that the old system of protecting sensitive data is no longer appropriate in this digital age.
Digital phenotyping could also be disruptive or competitive with doctors.
What happens if a doctor or patient views an algorithm assessment as objective? What happens when a tool recommendation differs from that of a physician? Rosie Weatherley is the information content manager at Mind, and says technology has a place in mental health services. Software can be used to spot signs of mental illness. But, human interaction and professional clinical judgement are essential components of the patient's experience with diagnosis, treatment and support.
Lisa Cosgrove is a University of Massachusetts, Boston professor of counselling psychology who studies social justice in psychiatry. She raises a philosophical question. Digital phenotyping's focus on the individual distracts from the socio-political factors that can lead to mental health problems such as discrimination, job loss, and eviction. She says that while individual care is important, digital phenotyping overlooks the context where people experience emotional distress.
Ghosh is optimistic that this field will be a benefit to society. It is a relatively new phenomenon to have data for research. This requires time and effort. He says that we need to ensure that we are helping people and not disrupting their lives.
