Google's AI Tutor Raised Maths Scores in Sierra Leone. It Raised Them Most for Students Who Were Already Ahead.

Somewhere in Port Loko District, 1,763 schoolchildren spent eight weeks learning maths with Gemini. It worked: their scores went up, and the trial that proved it is more rigorous than almost anything else in this field.

It worked best for the children who needed it least.

That sentence is not my inference, and it is not a hostile reading. It is in Google's own technical report, at p = 0.002, a more robust result than the headline figure it qualifies. Google published it. I have not yet found a write-up that leads with it.

Here is what the trial found, what it means for anyone buying an AI tutor for an African classroom, and the one question I now think you should ask every vendor who comes to your ministry.

Start with the good news, because it's real

In October–December 2025, Google's LearnLM team and Fab AI ran a preregistered randomised controlled trial of Gemini's Guided Learning in Sierra Leonean maths classrooms. It was registered with the AEA RCT Registry (AEARCTR-0016651) and cleared by the Sierra Leone Ethics and Scientific Review Committee, with the support of the Ministry of Basic and Senior Secondary Education.

The design:

• 1,763 students, aged 13+, in 48 Grade 7 and 8 classrooms across 12 government-supported junior secondary schools in Port Loko District.
• 24 classrooms treatment, 24 control. A real counterfactual.
• All teachers got the same 5–6 hour training, delivered before randomisation, so training can't confound the result.
• Two of four weekly maths periods, ~90 minutes a week, eight weeks of instruction (nine calendar weeks, staggered starts).
• Students shared devices, tablets or desktops, at two students per device.
• Crucially: outcome assessments were built and administered by Oxford MeasurEd, an outside contractor, using item response theory, blind to treatment assignment.

The headline result: an intent-to-treat effect of +0.258 standard deviations on maths scores (95% CI [0.027, 0.488], p = 0.029). Google's own intuitive translation: a student at the 50th percentile at baseline moves to roughly the 60th percentile by endline. And in 91.4% of conversations, students were working to build understanding rather than extract answers.

I want to be unambiguous, because the rest of this piece is going to be critical and I don't want that mistaken for cynicism: this is better evidence than almost anything else in the AI-for-African-education space. Preregistration, randomisation, a control group, blinded third-party outcome measurement, published technical report, published teacher guide, published replication playbook. Most claims you will read this year about AI transforming African education have none of these. Google did the work, and they published the methodology so you could check it. Credit where it is due, and it is due here.

Now let's read the report.

The finding I haven't seen reported

Buried in the results, Google discloses this:

Students who entered the trial with stronger mathematics skills benefited more. For each additional standard deviation of mathematics proficiency a student demonstrated at baseline, the treatment effect increased by +0.195 SD (p = 0.002).

Read that again, slowly.

The tool worked. But it worked disproportionately for the students who were already ahead. A student one standard deviation above the baseline mean got an effect roughly 0.195 SD larger than the average student did. Run that in the other direction and the students furthest behind, the ones an education intervention in Port Loko presumably exists to reach, captured the least benefit.

This is a heterogeneous treatment effect that widens the achievement gap, and it is statistically stronger (p = 0.002) than the headline result it qualifies (p = 0.029).

Google disclosed this. It is in their report, in plain language. But every write-up I have so far found leads with "1.2 to 1.7 years of learning progress," and I have not yet found one that leads with the gap. I haven't surveyed the coverage systematically, so I'll put that no stronger than it deserves, if you find a piece that led with this finding, send it to me and I'll link it.

For a development intervention, this is not a footnote. It is arguably the whole finding. An AI tutor that amplifies existing advantage is still useful, but it is a fundamentally different policy object from one that closes gaps, and a ministry deciding how to spend a constrained budget needs to know which one it is buying.

Two more things in the report you should know

The model changed in the middle of the trial. From the report: a Gemini update rolled out on 18 November, so students used Gemini 2.5 Pro for the first six weeks and Gemini 3.0 Pro for the final three. Google disclosed this, to their credit. But it means the "intervention" was not a fixed object, you are measuring the effect of a moving target, and there is no clean way to attribute the result to either model. In a pharmaceutical trial, changing the compound two-thirds of the way through would be disqualifying. In AI research it is a Tuesday, and that mismatch is worth sitting with.

The confidence interval nearly touches zero. The ITT effect is +0.258 SD with a 95% CI of [0.027, 0.488]. The lower bound is 0.027, very close to no effect at all. The result is statistically significant and I am not disputing it. But the honest reading is "somewhere between a barely-detectable effect and a large one, most likely a moderate one," not "1.2 to 1.7 years, confirmed."

About that "1.2 to 1.7 years"

The conversion is real and it is Google's, and it deserves a careful reading rather than a dismissal.

0.258 SD is a good effect size. Converting it to "years of learning" requires an assumption about how much a typical student learns in a year, and the report is explicit that it benchmarks against "typical academic progress in low- and middle-income countries." That is the correct benchmark to use. It is also a benchmark where annual learning gains are low, which is precisely why a moderate SD gain converts into a headline-grabbing number of "years." The same 0.258 SD, benchmarked against a high-performing school system, would produce a far less exciting number.

That is not a criticism of the researchers. They reported the standard deviation, they reported the confidence interval, and they also volunteered the conservative framing (50th → 60th percentile). They gave you everything you need.

But note what happens downstream. Google's DeepMind blog also reports a treatment-on-treated effect of +0.380 SD for students who hit the full 12-hour dose, which converts to "roughly 1.8 to 2.5 years." That bigger number is now in circulation. TOT effects on the students who complied most are not the same thing as what you'd get rolling this out across a district, and the gap between the ITT and the TOT framing is exactly where honest research turns into a press release.

The rule: when someone quotes you "years of learning," ask for the standard deviation. When they quote the SD, ask for the confidence interval. When they quote the average effect, ask who it worked best for. That last question is the one that broke this story open, and almost nobody asks it.

Meanwhile, in Johannesburg

On 1 July, Google held its inaugural "Building for Africa" Cloud Summit, 3,000 attendees, President Ramaphosa photographed with Google leaders, and announced five initiatives:

1. South Africa Digital Exchange Port (Eastern Cape), the first of four connectivity hubs Google has committed to the continent, linking to Australia via the Umoja subsea cable and to India.
2. Google Africa Applied AI Lab: in Ghana, at the Accra AI Community Centre. Google calls it Africa's first Applied AI Lab.
3. Digital Innovation Centre: with WeThinkCode, at South West Gauteng TVET College in Soweto. R3 million.
4. Creative AI education: with The Akuna Group, "more than $1 million (R17 million)" in Google.org funding.
5. Google for Startups Accelerator South Africa: 15 South African startups, equity-free funding.

Three of the five are physically sited in South Africa, and one of those restricts eligibility to South African startups. The Applied AI Lab is in Ghana but is open to founders from across the continent, and the Exchange Port is international infrastructure whose benefit isn't confined to one country, so this is not quite as narrow as a quick glance suggests, and I won't pretend otherwise.

The money that carries a number is small. On the exchange rate implied by Google's own gloss (R17m ≈ >$1m), R3 million is somewhere under $200,000. The Exchange Port is real capex and Google doesn't disclose it. The Applied AI Lab has no disclosed budget.

And the Applied AI Lab is not a skills programme. Read Google's own FAQ: it's a "zero-to-one commercialization platform" aiming to build "Africa's first generation of AI-native unicorn startups." Selected teams get Gemini, Gemma and Veo before general release. Participation requires using Google's models "in a materially impactful way", that's FAQ 9, and the answer is "Yes." There are VC partners (4DX, Norrsken22, Novastar, Ventures Platform, independent firms, not Google's) and a demo day in front of investors.

That is a good programme and a smart commercial play, and those are compatible. If you are an African AI founder or a researcher with a startup idea, including pre-seed and unfunded, both of which Google says are eligible, apply. Applications close 31 August 2026. Early access to unreleased frontier models is a real edge and you should take it.

It is simply a different kind of thing from an intervention that reaches a child in a classroom, and it should be read as what it is.

What I got wrong

When I first scanned this news cycle, I flagged the Accra lab and missed Port Loko entirely.

I'll just say what happened to me: the summit had a president, a stage, and a launch video. The trial had a PDF. I went for the summit, and the PDF was the story.

That is a bias worth naming in yourself, because the incentive structure that produced it isn't going away. The announcement with the most production value is rarely the one with the most evidence in it. The report you have to download is where the findings live, including the findings that complicate the announcement.

On the "3 million Africans" number, don't repeat it

You may have seen a pledge to train three million Africans in AI skills attached to this news cycle. Be careful with it, because there is more than one such pledge in circulation and they are being blurred together:

• Microsoft has publicly committed to training 3 million people across Africa in AI skills.
• Separately, TechAfrica News reports that the Africa AI Council's two-year strategic plan, validated at the Council's second meeting at ITU headquarters in Geneva, aims to equip "at least three million Africans with AI skills over the next three years."

I could not locate the Africa AI Council's strategic plan as a public document, so I can't verify that second figure at source, and I'm not going to pretend I can. (Note also the oddity: a three-year skills target inside a two-year strategic plan.)

What I can say is that neither pledge, as far as I can find, publishes a definition of what counts as "trained." A completed two-hour webinar and a deployed production model would both satisfy the sentence. Until someone publishes the definition and the completion data, three million is a press release, not a result, whoever's press release it is.

Two smaller corrections to things circulating: the 15 startups in the accelerator are South African, not pan-African. And Google states it has "already met and surpassed" its $1 billion commitment to Africa's digital transformation, ahead of schedule. That may well be true, but I could not find a public accounting of what the billion bought, broken down by country and programme. For a number that size, in a region this data-poor, asking for the breakdown seems reasonable. I'd rather ask than assume in either direction.

What to actually do

• If you're a founder or researcher with a startup idea: apply to the Accra lab. 31 August. Pre-seed and unfunded are eligible. Take the model access.
• If you're a ministry official or educator: read the Port Loko technical report and the RCT playbook, Google published them so you could. Then ask the vendor selling you an AI tutor for their heterogeneous treatment effects. If they can't tell you who it worked worst for, they haven't measured it.
• The device maths matters more than the model. Port Loko ran at two students per device, with 5–6 hours of teacher training per teacher. That's the real cost of replication, and I couldn't find anyone who has costed it at national scale. Gemini being good doesn't buy a tablet.

---

Sources

Primary

• Google DeepMind & Fab AI, Measuring the impact of Guided Learning on student outcomes (technical report, PDF), trial design, N=1,763, 0.258 SD ITT, 95% CI [0.027, 0.488], the +0.195 SD baseline-interaction finding, the mid-trial Gemini 2.5→3.0 Pro rollout, the two-students-per-device ratio, Oxford MeasurEd blinding. AEA registry AEARCTR-0016651.
• Google DeepMind, Gemini's guided learning: results from a randomized controlled trial in Sierra Leone, the TOT effect and the "1.8–2.5 years" framing
• Google Africa, Measuring the impact of learning with AI in Sierra Leone and beyond (9 June 2026)
• Google, Accelerating Africa's digital renaissance (1 July 2026), the five initiatives, R3m, >$1m, the $1bn claim
• Google Labs, Google Africa Applied AI Lab, eligibility, the Google-AI usage requirement, VC partners, 31 August deadline

Secondary

• TechAfrica News, Africa AI Council second meeting, the "at least three million" figure, unverified at source

Ibrahim Denis Fofanah is a data scientist and AI researcher at Pace University, founder of the Rise Africa Foundation for STEM and Innovation, and author of Understanding Agentic AI.