|
Request for proposals for Evaluation of the WAZZII Chatbot Digital Health and AI Evaluation, Kenya Girl Effect Kenya Issue Date: 14th October 2026. Closing Date: 26 Oct 2026 Type: Consultancy Procurement Summary Item Details RFP Title Evaluation of the WAZZII Chatbot across Three Kenya Programmes Competition Type Open competitive bid Supplier Type Independent research or evaluation agency Programme Scope MSD 4.0, Tukisonga 2.0 and AI Mobiliser Geographic Scope Nairobi and Nakuru Counties, plus approved digital and service data Assignment Scope Evaluation design, product analytics, AI performance and safety testing, user research, service-pathway analysis and reporting Completion Deadline All final deliverables completed by 31 December 2026 Clarification Deadline 14th October Proposal Deadline 26th October 2026 Submission Email suppliers@girleffect.org Proposal Validity 90 days Who We Are Girl Effect is an international non-profit that builds media and technology that girls want, trust and need. We use youth-centred design and behaviour-change approaches to help adolescent girls and young women access trusted information, services and support and make informed choices about their health and futures. Background Girl Effect is implementing three programmes in Kenya that use WAZZII, its digital chatbot, alongside multimedia, community engagement and service linkages to improve access to sexual and reproductive health (SRH) and maternal and newborn health (MNH) information and services. MSD 4.0: Nairobi and Nakuru, focusing on maternal and newborn health. Tukisonga 2.0: Nairobi and Nakuru, focusing on SRH among adolescent girls and young women (AGYW) aged 1524, including contraception, DMPA-SC/self-injection, heavy menstrual bleeding and STIs. AI Mobiliser: Nairobi, focusing on AI-enabled SRH information and support to facilitate informed decision-making and progression to contraceptive services. Girl Effect seeks an independent research partner to evaluate WAZZII across the three programmes, assessing user experience, AI performance and safety, behavioural outcomes, equity and progression from digital engagement to service uptake. Evaluation Purpose and objectives The evaluation will assess WAZZIIs performance, usefulness and safety, and its contribution to SRH/MNH outcomes and service pathways. Specifically, it will: Assess reach, engagement, repeat use and user experience. Evaluate AI accuracy, reliability, trust, safety and handling of sensitive or higher-risk situations. Examine associations between chatbot use and knowledge, confidence, intentions and service-seeking. Assess referrals, completed referrals and service uptake. Identify differences in access and outcomes across user groups. Generate recommendations to improve product performance, safeguarding, service linkages and cost-efficiency. What You will do Evaluation design and analytics Develop an evaluation protocol, indicator framework, sampling strategy and analysis plan. Review chatbot analytics to assess user characteristics, engagement, repeat use, conversation patterns and drop-off points. Map the pathway from programme exposure and chatbot engagement to human support, referral and service uptake. Integrate platform analytics, user surveys, qualitative research and available referral/service data. Where feasible, follow chatbot users over time. Clearly document data gaps, selection bias and limitations to attributing outcomes to chatbot use. AI performance, safety and user experience Assess WAZZIIs accuracy, reliability, relevance, comprehension, accessibility, privacy and trust. For AI Mobiliser, conduct structured testing of representative SRH questions, including contraception, DMPA-SC/self-injection, sensitive disclosures, misinformation risks and referral information. Review available AI performance monitoring, safeguarding alerts, failure records and escalation processes. Assess whether inappropriate responses and higher-risk situations are identified and managed correctly. Apply the existing 22-metric Product-Level Scorecard where appropriate. Use user surveys, interviews or usability sessions to identify barriers and opportunities for improvement. Equity and digital access Assess whether users with different levels of digital access can engage with and benefit from WAZZII. Consider shared-device access, connectivity, digital literacy, privacy, language, disability and geography. Analyse differences in engagement, experience, referral and outcomes by relevant user characteristics, where data permits. Examine how chatbot use interacts with multimedia, community mobilisation and human support, particularly within AI Mobiliser. Behavioural outcomes and service pathways Assess reported knowledge, confidence, decision-making, intentions and service-seeking using programme-relevant measures. For AI Mobiliser, examine progression along the contraceptive-care pathway, including referrals offered, completed visits and service uptake. Use referral codes, provider records, voucher redemption or other approved verification methods where available. Clearly distinguish verified service uptake from self-reported uptake. Where feasible, assess outcomes by engagement intensity and combinations of programme exposure. Any claims about effectiveness must be supported by the evaluation design; associations should not automatically be interpreted as causal effects. Analysis and recommendations Triangulate findings across analytics, AI testing, user research and service data.. Identify priority improvements in AI safety, product design, accessibility, referral systems and programme implementation. Deliverables Inception report, evaluation protocol and indicator framework. Analytics review and user-journey map, including data gaps. User research tools and AI performance and safety assessment framework. Required ethics approvals and research authorisations. Clean, de-identified datasets, codebooks and analysis syntax, subject to data governance requirements. Draft and final evaluation reports, including findings on user experience, AI performance, safety, outcomes, equity and service uptake. Prioritised recommendations and presentation of findings to Girl Effect. Duration and Management The assignment will cover evaluation design, data-feasibility assessment, product analytics, AI performance and safety testing, participant research, service-pathway analysis and reporting. All final deliverables must be completed and accepted by 31 December 2026. The agency will report to Girl Effects designated Evidence and Insights focal point and work closely with the relevant product, programme, safeguarding, operations and country teams. Any risk to the agreed scope, quality, budget or final deadline must be reported immediately with a proposed mitigation plan. Who You Are Required expertise The research partner should demonstrate experience in digital health or AI evaluation, mixed-methods research, usability testing, digital analytics, AI safety assessment, SRH/MNH, safeguarding and research ethics. Experience conducting research in Kenya and measuring referral and service uptake is essential. Proposal Submission Invited agencies must submit separate technical and financial proposals. The technical proposal should not exceed 10 pages, including CVs, references, company documents and supporting evidence. Technical Proposal The technical proposal should include: Evaluation design, sampling, and analysis approach. AI performance and safety testing methods. Methods for measuring user experience and service uptake. Ethical considerations and data-protection measures. Team composition and relevant experience. Ethics, safeguarding, privacy, security and data-governance approach. Detailed work plan demonstrating completion by 31 December 2026. Three comparable assignments. Three contactable client references. Financial Proposal The financial proposal must be submitted in Kenya shillings and include: A detailed fixed-price budget by phase and deliverable. Personnel roles, professional rates and level of effort. Applicable taxes shown separately. Proposed reimbursable expenses shown separately. Clear assumptions, dependencies, exclusions and conditions. Prices must remain valid for 90 days from the proposal deadline. Any cost not expressly excluded or identified as conditional may be treated as included in the proposed fixed price. Evaluation Criteria Proposals will be evaluated in the following order: Pre-screening Technical evaluation Financial evaluation. Evaluation Criterion Understanding of WAZZII and the programme and evaluation requirements: 10% Evaluation design, sampling, outcomes and service pathway methodology: 15% Product analytics, AI performance and safety-testing approach: 15% User experience, equity, ethics, safeguarding and data-protection approach: 10% Relevant organisational experience and referees: 10% Team capability and work plan feasibility :10% Technical Total: 70% Financial proposal and value for money: 30% Total: 100% Girl Effect may request: Written clarifications; A methodology presentation; Interviews with proposed key personnel; Reference checks; Due diligence information; or A best-and-final technical or financial offer. The lowest-priced proposal will not necessarily be selected. Procurement Timelines Terms of reference published: 14th October 2026 Questions/Clarification Deadline: 19th October 2026 Deadline for proposal submission: 26th October 2026 WAZZII Chatbot Evaluation Kenya Clarification questions must be submitted to the stated procurement contact by the clarification deadline. Proposals must be received by 26 October 2026 Clarification questions must be submitted to the stated procurement contact by the clarif Tender Link : https://reliefweb.int/job/4234177/request-proposal-evaluation-wazzii-chatbot-digital-health-and-ai-evaluation-kenya-girl-effect-kenya-issue-date-14th-october-2026
|
