Undernutrition is a major public health challenge worldwide and has significant effects on birthweight, mortality, brain development, and future earnings. The prevalence of poor nutrition is high in India, with 38% of children stunted and 53% of women of childbearing age being iron deficient. In spite of an existing knowledge base about which nutrition interventions are most effective to improve health outcomes, there is an urgent need for cost-effective, scalable approaches to deliver these interventions. India has one of the largest government-led Community Health Worker (CHW) programs globally with over 1.4 million trained CHWs providing nutrition-related services. The village-level CHWs provide a range of health and nutrition services to pregnant women, children less than six, and their mothers in their communities. Digital health interventions, including mobile phones, are increasingly being used to empower community health workers (CHWs) through increased access to health information and decision support, as well as support to their service delivery. Despite a proliferation of digital health interventions, existing evidence is limited to feasibility studies and efficacy trials of small-scale pilots. There is scant evidence on the effectiveness of these interventions at-scale. This evaluation aims to address this critical evidence gap by studying the effectiveness of an at- scale mHealth integrated with a large-scale nutrition program in India. The evaluation sought to examine whether the mHealth intervention improves the timeliness of home visits and life-stage appropriate counseling by the CHWs to pregnant women and mothers of children <12m. and to assess whether mHealth intervention works to improve infant and young child feeding practices.
In partnership with the implementing stakeholders including the Ministry of Woman and Child Development, Government of India, CARE International, and Dimagi, researchers independently
evaluated a mHealth intervention that was rolled at-scale in two northern states in India. The mHealth intervention included a smartphone-based application for CHWs to function as a job aid and alerts for home visits, with accompanying multi-level data web-enabled dashboards for supervisors and managers to ensure better service delivery and supervision, and enable real-time monitoring and data-based decision making. Researchers conducted a comprehensive evaluation including impact, process, cost, and technology evaluation. The impact evaluation used a village-matched controlled design with repeated cross-sectional surveys to evaluate whether ICDS-CAS can enable more timely and appropriate services to pregnant women, children less than 12 months and their mothers, compared to the standard nutrition program. The study has recruited nearly 1500 AWWs and 6000+ mother-child dyads from 400+ matched pairs of villages in Bihar and Madhya Pradesh between 2017 and 2019.
The primary outcomes were the proportion of beneficiaries receiving (a) an adequate number of home visits and (b) an appropriate level of counseling by the AWWs. The process evaluation sought to examine barriers and facilitators in the ICDS-CAS implementation and pathways to impact while the technology evaluation included heuristics evaluation and usability testing to understand the user experience of the application. The costing study estimated the potential cost of scaling-up.
CHW home visits and counseling improved for mothers of children less than 12 months, but not for pregnant women
mHealth intervention had an impact on the hypothesized short-term primary outcomes of an adequate number of home visits and recall of life-stage specific counseling as per the nutrition program guidelines among mothers of children less than 12m in Madhya Pradesh (MP) and Bihar but not for pregnant women in both the states as shown in the figures below.
mHealth intervention did not increase the provision of take-home rations (THR)
Take-Home Ration (THR) provisions depend on the availability of the supply stock from the state or district. In Bihar, 34 percent and in MP 72 percent of mothers reported receiving THR monthly, but the mHealth intervention does not have any module to report any supply or stock-related problems and consequently had no effect on THR provisions.
mHealth intervention alone is not enough to improve critical infant and young child feeding practices
Exclusive breastfeeding and timely initiation of complementary feeding were prevalent in MP (82% and 71%) and in Bihar (49% and 68%) and we found a strong correlation between the number of home visits and recall of counseling with these practices. Therefore, it appears that larger changes in the nutrition service delivery by using mHealth intervention were necessary to improve practices. Only 7 percent of the children aged 6-12 months in MP and 12 percent of them in Bihar received adequate diet which highlights the prevailing constraints of diet preferences, social norms, and food access. It is likely that mHealth interventions lack the agency to address these underlying factors.
This study provides evidence of the effectiveness of a mHealth intervention integrated into a large-scale program, with the mHealth component deployed to more than 500,000 CHW. The technology
evaluation identified user interface-related challenges and provided recommendations to redesign certain modules of the application. The cost of implementing mHealth intervention across India could be ₹12,563 ($179.47) per CHW in the first year due to investments in devices and training; approx. $215 million for 1.2 million CHWs across India. However, the cost can taper down to ₹5,169 ($72.48) per CHW in the third year before it spikes again in the fourth year due to the replacement of devices.
The findings suggest that a range of successes were seen on outcomes that were hypothesized to change in the short 18-month implementation timeframe.
A crucial conclusion of this study was that technology alone cannot immediately change persistent and pervasive community-, family-, or individual-level conditions that drive human behavior. Future research should focus on why and how technology would work under different contexts and whether the changes in the process- and system-level outcomes are large enough to eventually improve health and nutrition outcomes among beneficiaries. Future implementers and policymakers looking to implement mHealth interventions should lay out plans for timely device procurement, replacement, data hosting, and retrieval capacity, data ownership agreements between implementing partners and government functionaries.
“If anyone asks about family planning, we instantly tell them by using mobile. Earlier we had to search in the growth chart that which child has less weight or in which group I need to find it. Now I enter it in the app and everything is shown automatically. The app also automatically indicates where to go for a home visit” – CHW in MP
“Earlier we had to plan which lady should be visited on which day. Now the mobile tells us automatically which house we have to visit on which date. Earlier we had to speak, now we explain everything from mobile.” – CHW in MP
“Earlier we [AWWs] planned [for home visit] using the register, now we get to know from the mobile. Earlier we needed to explain verbally, now we show the video on mobile and then we explain. Because of this it has become easy, and we also update the information immediately.” – CHW in Bihar