How well do frontier AI models handle African work? The Index tests them on tasks written by practising African professionals and language specialists, and publishes the scores.
An invoice totals 11,600, including VAT at 16%. How much VAT is due, and what is the net amount?
Rubric written and validated by registered accountants.
Global benchmarks rarely test African law, tax rules, clinical guidance or the way people in African cities actually write. The Index fills that gap.
African accountants, clinicians, lawyers, teachers and language specialists write each task from real work.
Each task has pass/fail criteria and a reference answer, checked by a second expert.
Scores appear on a public leaderboard with the method, and a sample of tasks is released.
The full test set stays private so models cannot be trained on the answers.
The first release covers 200 tasks in three domains, run on at least six frontier models. The second grows to 600 tasks in five domains.
| Domain | Example task | Written by | Release |
|---|---|---|---|
| Swahili & Sheng customer support | Resolve a failed mobile-money transfer reported in Sheng | Language specialists | First |
| Tax & accounting | Compute VAT and payroll tax for a small business’s invoices | Registered accountants | First |
| Health | Give triage advice consistent with national health guidelines | Registered clinicians | First |
| Law | Advise on notice periods under national employment law | Practising lawyers | Second |
| Education | Mark a learner’s answer against a national curriculum rubric | Registered teachers | Second |
Built by our employed team between client projects, documented with a datasheet, and licensed non-exclusively to labs and model developers.
Customer messages from banking and telco support, labelled for intent, sentiment and urgency by native speakers.
Questions on finance, health and education with rubrics and reference answers grounded in African practice.
Accounting and tax questions answered and graded by registered African accountants.
Transcribed conversational speech for speech recognition and voice-quality testing.
