Study finds AI accuracy at just 5% for advanced sports broadcast tasks

Sports broadcasters can breathe easy — for now. A landmark new study has found that even the most capable artificial intelligence models fall apart the moment they are asked to do anything beyond a surface-level account of what is happening on a sports pitch.

The benchmark

Researchers at the University of North Carolina at Chapel Hill and Northeastern University put together a new evaluation framework called SVI-bench — short for strategic video intelligence benchmark — designed to assess four cognitive skills that prior AI assessments have consistently struggled to measure: perception, causation, simulation, and agency.

The dataset used in the study was substantial. It comprised videos of basketball, football, and hockey matches, 15 million tagged game plays, 15,000 hours of professional commentary and analysis, 23,000 post-match reports, and 103,000 statistical records. The AI models assessed were ChatGPT, Google's Gemini, and the open-source model Qwen. The study has not yet undergone peer review.

Where AI held up — and where it did not

Even at the most basic level — straightforward visual perception of events on screen — the models achieved success rates no higher than 74%, a threshold the researchers noted would be wholly insufficient for even a youth-league broadcast.

Performance fell sharply when the models were tested on causal reasoning — the ability to identify why a particular play unfolded as it did. Average accuracy at this level dropped to just 40%.

ChatGPT, for instance, when posed a question about what made a particular shot from the top of the backboard unusual, identified it as the "player's first made three of the game" — a response that missed the point entirely.

The simulation tests, which required the models to predict a player's next physical movement based on an analysis of their trajectory, produced results no better than chance — the best-performing model hovered around the equivalent of a coin flip.

Agency: the sharpest failure

The most striking findings came from the agency tests, where models were asked to carry out the kind of complex, multi-layered post-match statistical analysis that professional broadcasters routinely perform. Accuracy collapsed to just 5 per cent.

What this means for AI and jobs

Lorenzo Torresani, a computer science researcher at Northeastern University and co-author of the study, explained the gap between what AI can and cannot currently do in broadcasting.

"A good sportscaster does much more than describe what's on screen; they explain why a play worked, anticipate what's next, and decide which moments matter. Our study shows AI is already reasonably good at the descriptive part but collapses on the rest."

Torresani also drew wider conclusions about what the findings mean for industries beyond sport, at a time when concern over AI-driven job losses is running high.

"The same gap shows up in any job whose value lies not in describing what's visible, but in understanding why events unfold, anticipating what comes next, deciding what matters, and recommending what to do about it."

For those anxious about where AI poses the most immediate threat to employment, the study offers a more precise diagnosis: the technology is capable where tasks are descriptive, but it remains far from ready to replicate the kind of contextual judgement and strategic reasoning that define skilled human commentary.