The Everyday Brief #7: five leaks that inflated AUC by up to 0.26, a true tool result that was still wrong, and $10,000 awards closing November 13

From this week on the site: five data leaks, each measured in a controlled experiment with honest and leaky AUC on the same data.

Five leaks, every gain measured, not guessed. Target leakage moved AUC from 0.833 to 0.989. Selecting features before the split moved it from 0.534 to 0.798, and that test ran on pure noise. Keep the honest note on the card in mind: fitting a scaler on all the data measured exactly zero in these tests. Rank your audit by damage, not by how often a tutorial warns about a leak.

Read the full piece →

Split by the customer, patient or account, not by the row, whenever production will meet new groups:

```
from sklearn.model_selection import GroupShuffleSplit

splitter = GroupShuffleSplit(n_splits=1, test_size=0.25, random_state=42)
train_idx, test_idx = next(splitter.split(X, y, groups=customer_id))
```

In the test on the site, a random split across 500 repeat groups scored 0.823, while a group split scored 0.676 on the same data. The random split let the model memorise group fingerprints. If your test rows come from entities the training set has already seen, and production brings new entities, the higher score is a fiction.

TRAP and ReplayBench (arXiv:2610.16333, submitted October 12).

The researchers took real, unmodified tool results and replayed them into the wrong task, an earlier state, a second time, or out of order. Across the cross-task test, covering eight models and three frameworks, the mean final-state error rate was 0.649. In 64 of 168 configurations it reached 1.0, meaning every valid trial ended in a wrong persistent state. Model choice moved the result far more than framework choice: model means ranged from 0.019 to 0.967, while framework means stayed between 0.607 and 0.686.

Why it matters for builders: the content was clean. The plumbing delivered it to the wrong place, and the agent did the damage itself. Freshness hints in the payload helped only a little. Runtime checks at the tool gateway, call-ID binding, one-shot tickets and world-state epochs, brought the error rate to zero in the defense test the authors ran, with no false refusals in that test. That defense test was narrow, one model and one framework, so treat zero as a direction, not a guarantee. Our full breakdown is on the site today.

Read the full breakdown →

Small models and paid speech data were the real African AI story of early October.

Vambo AI in South Africa released MORENA, a 1.5-billion-parameter open-weight model built for 12 African languages, plus English and French. Read the launch claim with care: the company has not yet published its licence terms or benchmark results, so MORENA is a promising release, not a proven one. In Togo, the Voix du Togo programme began paying contributors to record, transcribe and translate phrases in Ewe, Kabiye and Moba, targeting 50 hours of validated speech and 6,000 translated sentence pairs per language. Fifty hours sounds small. For a language with almost no validated speech data, it is a starting asset researchers can build on. Ocular AI, founded by two Zimbabweans, raised $2 million in pre-seed funding to sell voice datasets and evaluations to large labs, which proves the market value of African speech data and raises the usual question: who keeps the long-term value?

Our full fortnight roundup, including the World Bank advice to back low-cost AI and Nigeria naming power as its binding constraint, is on the site.

Read our roundup →

Lacuna Fund open datasets call, via ACTS: full proposals close November 13, 2026.

The African Centre for Technology Studies (ACTS) is running a request for proposals funded by Wellcome Trust: train a new machine learning model, scale an existing one, or build an application or tool using one or more of the 67-plus open datasets already created with Lacuna Fund support. Three awards of up to $10,000 each are available, $30,000 in total. Work runs for six months, December 2026 to May 2027, and a pilot-stage minimum viable product is the minimum final deliverable. Models should be hosted for inference, for example on Hugging Face, with documentation.

Read the eligibility closely before you apply. Lead organisations must be headquartered in a low- or middle-income country in the Global South, and must be a non-profit, a research institution, a for-profit social enterprise, or a team of such organisations. Individuals must apply through an institutional sponsor. Any SDG domain represented in the Lacuna Fund catalogue is eligible, including health, agriculture, education, climate and governance. Domain competency is a core requirement and will be assessed by the review panel. Questions are due by October 28, 2026, with answers posted November 4, so look at the dataset catalogue this week and send questions early.

Apply here →