The Need to Evaluate Mobile Health Tools

Private companies, government institutions, academics and even individuals have produced thousands of mobile tools aimed at improving people’s health. With a couple of clicks on their smartphone, people can access apps that help them do everything from lose weight to manage their diabetes to have more constructive conflicts with their romantic partner.

These developments are both exciting and timely. The average visit with a primary care doctor runs a meager 15-20 minutes, and the traditional model of care (i.e., sick person sees a provider face-to-face until they feel better) can’t meet the growing demand for health services. This is particularly true as the U.S. population increasingly struggles with chronic conditions that will need a lifetime of daily management. Also, while up to 50 percent of Americans will experience a mental health condition at some point in their lives, the vast majority of those people will never see a behavioral health provider. Mobile health (mHealth) interventions, such as mobile apps and websites, help fill these gaps in care by giving people 24-hour access to health care and health-related information. Research is showing that this can reduce the cost of care, and, most importantly, improve health outcomes.

Yet, as with any new technology, we must be careful. Unlike medications or other kinds of medical devices, there’s practically no government regulation of mHealth tools. While it has long been recognized that there is a need to evaluate mHealth apps, research in this field is in its earliest stages. While some studies show that mHealth technologies hold significant promise in helping people manage health conditions as diverse as HIV and depression, other studies should make us take have more concerning results.

For example, the Swedish government’s alcohol monopoly recently released an app intended to help university students curb binge drinking by measuring their blood alcohol level and warning when it got too high. Unfortunately, a study found that, instead of reducing problematic drinking patterns among male users, the app motivated some users to drink more frequently. Other studies have found that the drop-out rate for mHealth intervention studies is incredibly high. In one computerized treatment for depression, only 97 of over 19,000 enrolled participants actually completed the treatment; another mHealth study only had 12 people complete the study out over 1000 enrolled participants. These findings raise important research—as well as ethical—questions.

The National Center for Telehealth & Technology (T2), part of the Defense Health Agency (DHA), has been developing free mHealth interventions for service members, veterans and their families since 2008. T2 evaluates our products to better understand how they’re used on a daily basis. To that end, we have just developed the T2 Study Enrollment Generator (T2SEG) that provides app analytics for several of our products (including Breathe2Relax, Virtual Hope Box, and T2 Mood Tracker).

Usage information provides researchers with precise quantitative data, which allows them to determine whether product usage relates to user’s health outcomes. Types of data include:

  • Objective measure of compliance rates (i.e., how many people are using the app as intended?)
  • Potential to determine dose of intervention (i.e., are people using the app enough to make a meaningful change in their behavior?)

Researchers who are interested in using the T2SEG to study our products are encouraged to contact us. We are also always interested in building partnerships with researchers who want to develop and evaluate mHealth tools for service members.