Researchers at Dartmouth College have developed MoodCapture, a smartphone app that uses AI to research facial expression to detect depression early.

The stigma related to mental health is a giant reason why people placed on a brave face even once they are feeling depressed. This can be why we try harder to smile after we interact with people or take a selfie that we wish to post on social media.

When we glance right down to use our phone, we’re mostly unaware that there’s a camera looking back at us. MoodCapture uses a smartphone’s front camera to commonly capture authentic, unguarded facial expressions when a user unlocks their phone. An AI algorithm is then used to repeatedly monitor for early signs of depression.

The researchers trained their algorithm on 125,000 photos of 177 participants diagnosed with major depressive disorder.

The photos were taken over a period of 90 days while participants used an app on their phones to reply a brief patient health questionnaire (PHQ-8) 3 times a day.

When the participant responded to the query “I felt low, depressed, or hopeless,” the app took an image of the person’s face. The patients consented to the photo being taken, but didn’t know exactly when it happened.

In addition to facial expression, the AI ​​model was also trained on the angle of the phone, predominant color, lighting conditions, location of the photo, and background elements present within the photos.

Using machine learning and deep learning, the researchers created a model that learned which nuanced facial expressions and contextual background elements correlated with depression.

The AI ​​model that powers MoodCapture was trained using photos of the user’s face and background elements, in addition to machine learning and deep learning, to find out the extent of depression. Source: arXiv

The MoodCapture app prototype can reliably detect the onset of depression even before the user knows they’re in danger.

Millions of individuals worldwide suffer from depression, with severe cases answerable for 800,000 suicides annually. Early detection of depression could mean more people have access to appropriate and timely treatment.

The app functionality is impressive, but researchers note their paper that mental health monitoring apps like MoodCapture raise a variety of thorny ethical questions.

If my phone senses that I’m feeling down, it could be nice to have an AI assistant give me a pep talk or recommend a health option. However, I would really like to maintain this information private.

The prospect of being the goal of mood-based promoting or Big Brother mental health surveillance by authorities is not so far-fetched if the app’s user data were exposed. The more on-device AI is feasible, the simpler these privacy issues will probably be to deal with.

Thanks to improved cameras and built-in AI, our phones could soon have the option to inform whether we’re having a very good day or not, even when our friends cannot at all times tell.

This article was originally published at