Building a content playbook for adolescent sexual and reproductive health interventions (commissioned by the Gates Foundation)
The Client: The Gates Foundation and Population Council
Context: The Gates Foundation and Population Council jointly commissioned a project to build a playbook of content guidelines & UX best practices for adolescent sexual and reproductive health (ASRH) interventions. The final playbook offered editorial and data ethics guidelines to help interventions inform and engage adolescents about their reproductive health
Approach
The project commenced with qualitative interviews with:
1. Program and platform stakeholders of 3 digital interventions: Agents of Ishq, Kahi Ankahi Baatein and Love Matters
2. ASRH subject-matter experts (SMEs)
3. Content, Analytics and Design Experts from big-tech and e-commerce industries
Insights from our research were synthesized and iterated on through a dissemination workshop with the Gates Foundation, Population Council and program stakeholders. The workshop helped us test and refine the activities and guidelines before drafting the final playbook: Love in the time of digital
Scale
Interviews with employees from the 3 digital interventions, 5 SMEs, 3 technology experts (product manager, engineering lead and head of analytics). The dissemination workshop had ~80 participants who were SMEs, program and platform stakeholders, the Gates foundation and Population council.
Identifying content pillars by analysing existing user queries
A subset of anonymised data from a live IVRS helpline was analysed to illustrate use cases beyond the basic metrics that programs were already looking into. The graph below illustrates the potential of data to enable teams to reflect about the trajectories of content consumption among adolescent helpline users. It could, for example inform user learning journeys: do queries begin to move from survival to thriving, from preliminary health topics to more nuanced topics around pleasure and rights
The graph below is for illustration purpose only. Actual graphs cannot be shown owing to client confidentiality
Sample Insights - Gaps in leveraging existing digital best practices
Cultivating an ecosystem of trust around data
Aligning the needs of multiple stakeholders was a challenging precursor to data-driven decision making. Program stakeholders tended towards being decision-driven (how can data help me take decision Y?) while platform stakeholders preferred a data-driven approach (what can we learn from the existing data? What other data could we collect and learn from?).
How might we enable program and platform stakeholders to air their concerns and opportunities in making data-driven decisions for their platforms?
Ethical apprehensions restricted teams' curiosity around exploring their data to improve program outcomes. Crucial constraints around user anonymity, privacy and explicit consent governed decisions around capturing additional data to unlock new content generation ideas. There was a need to define and align on the foundational ethics of data collection above which decision-making capabilities could exist.
How might we enable program and platform stakeholders to develop and align on their ethical values for data collection and analysis?
The paradox of attribution
Subject matter experts and program stakeholders opined that while a data-driven approach to impact could expedite program outcomes and enhance accountability, it could pose the risk of using data to oversimplify the intangible outcomes around behavior change and attitudinal shifts
How might we balance data-driven accountability with the qualitative nature of behavior change and attitudinal shifts around sexual and reproductive health?
Recommendations & Workshop Reflections
How might we enable program and platform stakeholders to dialogue better, airing their concerns and opportunities in making ethical, insightful data-driven decisions for their platforms?
Cultivating a sense of trust between program and platform stakeholders required healthy dialogue around expectations and apprehensions with data. Program stakeholders' needs were anchored around what data could do for them, while platform stakeholders' needs were focused on maximizing the opportunities that data unlocked for the program. The diagram below illustrates this relationship that translated into stakeholder communication activities at the workshop
How might we enable program and platform stakeholders to discover and align on their ethical principles for data collection?
The workshop featured an activity that focused on ethical considerations in capturing and utilizing data. Provocations around consent, scale, anonymization, privacy and secure storage helped teams create a hierarchy of data aspirations from ethics to actionability.
How might we balance data-driven accountability with the qualitative nature of behavior change and attitudinal shifts around sexual and reproductive health?
Addressing the paradox of attribution that emerged from research, teams reflected about an exhaustive roadmap of data opportunities: from measuring the effectiveness of the intervention to the eventual impact it created among adolescent users. The activities helped highlight the analytical opportunities across all stages of program and platform maturity
Outcome
The final report "Love in the Time of Digital" (including the final platform journey framework and detailed guidelines for implementing them) was shared with the Gates Foundation to help them implement and measure current and upcoming digital interventions for ASRH. Templates and frameworks that were tested at the workshop underwent further iteration based on feedback, before they were included in the report.
The report has been referenced in ASRH research papers by subject matter experts within the Gates Foundation: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7887897/
Note: The actual framework we built is client confidential and cannot be disclosed in further detail
The cover page of the final playbook: Love in the time of digital