Virtual Health Assistants for Healthy Lifestyles and Virtual Counseling
There is a growing societal need to address the increasing prevalence - and even epidemics - of lifestyle behavioral health issues such as overeating, excessive alcohol consumption, unsafe sex, drug use, and general lack of treatment adherence for a variety of health problems. These behavioral issues place people at risk of serious diseases. For example, obesity can lead to diabetes, alcoholism to cirrhosis, physical inactivity to heart disease.
Since almost all psychosocial interventions on the internet and other electronic modalities are delivered via text (or at best video), using supportive virtual health coaches is a novel modality for healthcare interventions. For many, virtual health coaches who converse with people over the long term about their intimate health issues and provide them with effective personalized and tailored counseling is still part of science fiction. This research, however, has become within the realm of computational possibilities due to latest progress in affective computing, and has the potential to transform access to and delivery of effective healthcare interventions for diverse populations.
The projects described below continue toward our long-term goals of on-demand conversations with virtual agents as personalized health and well-being helpers for people in need. Our approach is programmable for a variety of lifestyle target behaviors such as overeating, excessive alcohol consumption, narcotic drug use, non-adherence to treatment, unsafe sex, among others.
Since almost all psychosocial interventions on the internet and other electronic modalities are delivered via text (or at best video), using supportive virtual health coaches is a novel modality for healthcare interventions. For many, virtual health coaches who converse with people over the long term about their intimate health issues and provide them with effective personalized and tailored counseling is still part of science fiction. This research, however, has become within the realm of computational possibilities due to latest progress in affective computing, and has the potential to transform access to and delivery of effective healthcare interventions for diverse populations.
The projects described below continue toward our long-term goals of on-demand conversations with virtual agents as personalized health and well-being helpers for people in need. Our approach is programmable for a variety of lifestyle target behaviors such as overeating, excessive alcohol consumption, narcotic drug use, non-adherence to treatment, unsafe sex, among others.
Virtual screening and counseling
Our virtual health coaches deliver a variety of health interventions based on Motivational Interviewing, an evidence-based patient-centered communication style designed to help people change at-risk behaviors, which is recognized as effective in the healthcare domain.
They adapt to current needs of diverse populations, and increase access to effective health interventions for people without such access.
Over multiple interactions based on an individual's evolving readiness to change, our virtual coaches provide people with tailored personalized assessment, feedback and advice about what can place them at-risk.
They adapt to current needs of diverse populations, and increase access to effective health interventions for people without such access.
Over multiple interactions based on an individual's evolving readiness to change, our virtual coaches provide people with tailored personalized assessment, feedback and advice about what can place them at-risk.
A sample intervention session demo: The following video provides an example of how the virtual agent delivers the brief intervention on alcohol consumption.
As seen in the video above, our virtual health coaches are 3-dimensional expressive and empathic virtual agents who use their speaking abilities combined with with dynamic expressive animations of subtle facial expressions and gestures to deliver spoken motivational health interventions that are:
The dialog system processes the persons’ spoken answers and facial expressions to control the flow of the conversation based on effective motivational healthcare interventions.
Our research involves the integration of computer vision, dialog management, and dynamic animations for 3D virtual characters. The agent performs real-time recognition of the user’s facial expressions and speech during the dialog. Using reinforcement learning, the dialog manager controls the flow of the conversation based on the user’s spoken responses to an effective motivational behavior change intervention. The agent uses a novel computational model of nonverbal communication to decide what expressions and gestures to portray as it engages the user on at-risk behaviors.
Our virtual coaches demonstrate that they can lower dropout rates. Our virtual health coaches are effective in increasing people’s ongoing interest in computer-based health interventions, thereby increasing their ability to change toward healthy lifestyles. Our virtual health coaches can help address the nation epidemic of behavioral issues, and help people find and maintain lifestyles.
This project is further explained in the following articles:
- low-cost and effective
- accessible via the Internet or mobiles,
- reproducible on personal computers
- tailored to the individual
- engaging and empathic using multimodal conversations
- preferred by users over current computer-based interventions using text-only (Lisetti et al., 2013).
The dialog system processes the persons’ spoken answers and facial expressions to control the flow of the conversation based on effective motivational healthcare interventions.
Our research involves the integration of computer vision, dialog management, and dynamic animations for 3D virtual characters. The agent performs real-time recognition of the user’s facial expressions and speech during the dialog. Using reinforcement learning, the dialog manager controls the flow of the conversation based on the user’s spoken responses to an effective motivational behavior change intervention. The agent uses a novel computational model of nonverbal communication to decide what expressions and gestures to portray as it engages the user on at-risk behaviors.
Our virtual coaches demonstrate that they can lower dropout rates. Our virtual health coaches are effective in increasing people’s ongoing interest in computer-based health interventions, thereby increasing their ability to change toward healthy lifestyles. Our virtual health coaches can help address the nation epidemic of behavioral issues, and help people find and maintain lifestyles.
This project is further explained in the following articles:
- C. Lisetti, A. Amini, and U. Yasavur (2015). Now All Together: Overview of Virtual Health Assistants Emulating Face-to-Face Health Interview Experience. Künstliche Intelligenz (DOI 10.1007/s13218-015-0357-0) [PDF].
- U. Yasavur, and C. L. Lisetti, and N. Rishe (2013). Modeling Brief Alcohol Intervention Dialogue with MDPs for Delivery by ECAs. In Proceedings of the 13th International Conference on Intelligent Virtual Agents (IVA’13) (Edinburgh, SCOTLAND, August 2013) [PDF].
- U. Yasavur, R. Amini, C. L. Lisetti, and N. Rishe (2013). Ontology-based Named Entity Recognizer for Behavioral Health. In Proceedings of the 26th International FLAIRS Conference, www.aaai.org, (St Petersburg, Florida, USA, May 2013) [PDF].
- C. L. Lisetti, R. Amini, U. Yasavur, and N. Rishe (2013). I Can Help You Change! An Empathic Virtual Agent Delivers Behavior Change Health Interventions. ACM Transactions on Management Information Systems, Vol. 4, No. 4, Article 19, 2013 [PDF].
Since almost all psychosocial interventions on the web and other electronic modalities are delivered via text (or at best video), using supportive virtual health coaches is a novel modality for healthcare interventions. For many, virtual health coaches who converse with people over the long term about their intimate health issues and provide them with effective personalized counseling is still part of science fiction. This research has the potential to transform access to and delivery of effective healthcare interventions for diverse populations.
The research involved the integration of computer vision, dialog management, and dynamic animations for 3D virtual characters. The agent performed real-time recognition of the user’s facial expressions and speech during the dialog. Using reinforcement learning, the dialog manager controlled the flow of the conversation based on the user’s spoken responses to an effective motivational behavior change intervention. The agent used a novel computational model of nonverbal communication to decide what expressions and gestures to portray as it engaged the user on at-risk behaviors.
Our virtual coaches demonstrated that they can lower dropout rates. Our virtual health coaches are effective in increasing people’s ongoing interest in computer-based health interventions, thereby increasing their ability to change toward healthy lifestyles. Our virtual health coaches can help address the nation epidemic of behavioral issues, and help people find and maintain lifestyles.
This project is further explained in the following articles:
- U. Yasavur, and C. L. Lisetti, and N. Rishe (2013). Modeling Brief Alcohol Intervention Dialogue with MDPs for Delivery by ECAs. In Proceedings of the 13th International Conference on Intelligent Virtual Agents (IVA’13) (Edinburgh, SCOTLAND, August 2013).
- U. Yasavur, R. Amini, C. L. Lisetti, and N. Rishe (2013). Ontology-based Named Entity Recognizer for Behavioral Health. In Proceedings of the 26th International FLAIRS Conference, www.aaai.org, (St Petersburg, Florida, USA, May 2013).
On-Demand Virtual Counselor for Motivational Dialogs on Lifestyles
In this project, we found that the delivery of computer-based interventions by empathic virtual agents increases by 31% users' intention to reuse the intervention compared with the same content delivered via text-only (Lisetti et al., 2013) [PDF].
Our approach was to build a virtual agent that delivers the
exact same content as an existing effective behavior change intervention for alcohol consumption: the Drinker’s Check-Up (DCU). The DCU has been found effective in reducing alcohol consumption in problem drinkers by 50% at 12-month follow-up (Hester, 2005).
As shown in the ODVIC architecture figure and described in our article, our On-Demand VIrtual Counselor (ODVIC), accessed over the internet, is a multimodal Embodied Conversational Agent (ECA) that adapts its non-verbal (e.g. facial expressions, head nods) and verbal communication based on the user's facial expressions and answers to questions.
Our work is described in the following article: C. L. Lisetti, R. Amini, U. Yasavur, and N. Rishe (2013). I Can Help You Change! An Empathic Virtual Agent Delivers Behavior Change Health Interventions. ACM Transactions on Management Information Systems, Vol. 4, No. 4, Article 19, 2013 [PDF].
Our approach was to build a virtual agent that delivers the
exact same content as an existing effective behavior change intervention for alcohol consumption: the Drinker’s Check-Up (DCU). The DCU has been found effective in reducing alcohol consumption in problem drinkers by 50% at 12-month follow-up (Hester, 2005).
As shown in the ODVIC architecture figure and described in our article, our On-Demand VIrtual Counselor (ODVIC), accessed over the internet, is a multimodal Embodied Conversational Agent (ECA) that adapts its non-verbal (e.g. facial expressions, head nods) and verbal communication based on the user's facial expressions and answers to questions.
Our work is described in the following article: C. L. Lisetti, R. Amini, U. Yasavur, and N. Rishe (2013). I Can Help You Change! An Empathic Virtual Agent Delivers Behavior Change Health Interventions. ACM Transactions on Management Information Systems, Vol. 4, No. 4, Article 19, 2013 [PDF].
Ontology for Behavioral Health
We have developed the first named-entity recognizer designed for the lifestyle change domain. It aims at enabling smart health applications to recognize relevant concepts.
We created the first ontology for behavioral health (shown below) based on which we developed an NER augmented with lexical resources. Our NER automatically tags words and phrases in sentences with relevant (lifestyle) domain-specific tags. For example, it can tag:
Named-Entity Recognizers (NERs) are an important part of information extraction systems in annotation tasks. Although substantial progress has been made in recognizing domain-independent named entities (e.g. location, organization and person), there is a need to recognize named entities for domain-specific applications in order to extract relevant concepts.
We created the first ontology for behavioral health (shown below) based on which we developed an NER augmented with lexical resources. Our NER automatically tags words and phrases in sentences with relevant (lifestyle) domain-specific tags. For example, it can tag:
- healthy food
- unhealthy food
- potentially-risky
- healthy activity,
- drugs,
- tobacco,
- alcoholic beverages.
Named-Entity Recognizers (NERs) are an important part of information extraction systems in annotation tasks. Although substantial progress has been made in recognizing domain-independent named entities (e.g. location, organization and person), there is a need to recognize named entities for domain-specific applications in order to extract relevant concepts.
