5. Development phase: Model evaluation
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed do eiusmod tempor incididunt ut labore et dolore magna.
Why should I do this?
Evaluate whether the developed AI model efficiently performs against defined metrics and requirements.
- Workflow activities Miro Board
Target users
- CTOs and technical leaders
- Data scientists and ML engineers
- Product managers
- Research organizations, think tanks
Checklists in play
- Checklist 1: Risk Identification & Mitigation (Q13, Q14)
Questions to consider
Checklist 1 · Q13
What evidence demonstrates the model's performance against all predefined benchmarks and success criteria?
Checklist 1 · Q14
Which red team exercises (stress tests, adversarial tests, and failure mode analysis) have you done to mitigate risks before deployment?
Investment types
Every investment project is unique
The application of the six steps will vary accordingly. To provide examples that align with your project, common characteristics of AgDev investments were researched and three ‘investment types’ were developed.
AgriConnect: a digital solutions investment
The team benchmarked the assistant against real farmer and extension-officer queries, testing intent recognition, data retrieval, numerical accuracy, groundedness and source attribution.
They then used adversarial testing to identify and correct failures. Once performance thresholds were met, controlled user/shadow testing with extension officers and farmers was done to assess usability and task completion.
Participatory co-creation, feedback, and model refinement
- What was the process followed to design the benchmark test? How did the process ensure it was representative of different user groups' needs?
- Are users informed of the AI’s limitations and could they provide anonymous feedback?
Lessons learned and challenges that arose
The benchmark needed to consider different levels of digital literacy, local terminology and more ambiguous questions. Adversarial testing identified cases where the assistant produced plausible answers when data were incomplete or ambiguous. Source attribution and uncertainty indicators were not sufficiently prominent for some users during shadow testing. Users tended to interpret authoritative-sounding responses as government-endorsed advice, even when uncertainty was indicated.
Decisions taken and documentation maintained
An inclusion and exclusion log was maintained to justify how the pilot groups were selected. Domain experts were consulted to the benchmark test. The interface was redesigned to make uncertainty, limitations and source attribution more prominent, and to prompt users to consult extension officers for higher-risk topics. Serious failure scenarios and escalation/rollback criteria were documented before deployment. Benchmark results, adversarial test cases, user/shadow-test findings, mitigation decisions and system changes were maintained in evaluation reports, test logs and change logs.
AgroThrive: a policy and advocacy investment
The team initially evaluated the system in a sandbox using a controlled document corpus, focusing on factuality, evidence coverage and citations.
It was followed by red-team testing to highlight hallucinations and misleading conclusions. Finally, the team adopted a human-in-the-loop shadow pilot with policy analysts to compare AI-assisted and conventional briefing workflows before wider deployment.
Participatory co-creation, feedback, and model refinement
- Do policymakers understand how synthesis was generated?
- Are users encouraged to review source material directly?
Lessons learned and challenges that arose
Red-team testing highlighted risks of hallucinated evidence, misleading synthesis and overconfident conclusions where source material was incomplete or conflicting. The evaluation highlighted the importance of maintaining traceability from individual claims back to source documents, particularly for policy and investment-related content. Users did not always understand how AI-generated synthesis was derived or distinguish between AI-generated interpretation and source evidence. Shadow testing showed a tendency to trust AI-generated content without independently checking the underlying sources.
Decisions taken and documentation maintained
Mandatory source traceability and citation panels were added to all outputs. Exported summaries included disclaimers requiring human validation before external use. Pilot findings and mitigation responses were documented in the model evaluation reports and deployment approval was signed off by the assigned team member. Mandatory source traceability and citation panels were introduced, allowing analysts to inspect the underlying evidence for generated claims. Outputs intended for external or official use were configured to require human review and validation before export or publication. The team established escalation criteria for unsupported, conflicting or insufficient evidence and guidance on when analysts should consult source documents directly. Sandbox results, red-team findings, shadow-pilot observations, identified failure modes, mitigation decisions and deployment approvals were recorded in model evaluation reports, test logs, risk/issue logs and change records.
NGBT: a field research investment
The team began with scenario-based testing using questions validated by experts, focusing on accuracy, safety and appropriate handling of uncertainty.
Stress testing was used to probe potentially unsafe or out-of-scope advice. They then progressed to a supervised pilot with extension officers and farmers, with A/B user testing where appropriate to assess comprehension, usability and learning outcomes.
Participatory co-creation, feedback, and model refinement
- Did farmers understand the tool was AI-generated?
- Are low-literacy and offline users included in this group?
- Are accessible feedback mechanisms available?
- Can communities contest misleading or culturally inappropriate outputs?
Lessons learned and challenges that arose
Expert scenario testing showed that accuracy alone was insufficient; the assistant needed to communicate uncertainty and recognize when advice was outside its scope. Stress testing identified scenarios where the system could provide overly confident or potentially unsafe agronomic advice. The A/B pilot testing showed that some farmers interpreted the AI as a replacement for extension officers' advice, rather than a supplementary source of information.
Decisions taken and documentation maintained
The system was configured to refuse, warn or escalate to an extension officer for defined high-risk, uncertain or out-of-scope questions. The deployment approach was revised to explicitly position the AI as supporting rather than replacing extension services. Additional community engagement and feedback mechanisms were introduced, including channels accessible to lower-literacy and offline users. Source traceability and escalation procedures were strengthened so that advice could be checked against approved agronomic materials. A/B and pilot results, expert assessments, stress-test findings, community feedback, identified risks and mitigation actions were documented in evaluation reports, stakeholder review records, test logs and deployment/change records.