A recent analysis of AI benchmark coverage reveals that 90% of American working hours are devoted to tasks insufficiently measured by current AI evaluation tools. The study, which mapped 923 occupations across 22 job families using the U.S. Department of Labor’s O*NET taxonomy, found that only 208 occupations have at least five benchmark prompts tied to their core tasks. Just six of the 22 job families showed any occupations meeting this threshold.

The researchers measured coverage by linking benchmark prompts to specific occupational tasks. Their methodology combined task-level coverage estimates with labor time data to calculate the share of national work hours exposed to benchmark evaluation. Results indicated that coverage is neither uniform nor random. Instead, it clusters around higher-wage occupations and tasks deemed more economically significant.

For example, AI-exposed legal tasks such as drafting contracts or advising clients received extensive benchmark attention, while similar tasks performed by legal secretaries or court clerks were largely overlooked. This disparity reflects a pattern where benchmarks prioritize roles with larger wage bills and higher perceived AI exposure, as defined by prior research on automation potential.

The study also found that tasks rated as more important within an occupation were six to ten times more likely to be benchmarked than those considered less critical. Furthermore, occupations with higher annual wage bills demonstrated significantly greater likelihood of task coverage, even after accounting for AI exposure levels.

While the initial benchmark set included 29 high-quality, publicly available evaluations focused on economically relevant and agentic tasks, the researchers acknowledged limitations. Their selection intentionally avoided overrepresented domains like software engineering to improve breadth, which may result in underestimating coverage in already-well-studied fields. They also noted that saturated benchmarks could inflate perceived model performance, while unsaturated ones might understate it.

The authors emphasized that their findings are provisional. As they expand the benchmark set and refine methods for weighting tasks by importance and exposure, they aim to improve the accuracy of economic coverage estimates. They also cautioned that current benchmarks may not fully capture augmentation effects — how AI supports human workers rather than replacing entire job functions.

Efforts are underway to incorporate O*NET-based weighting into occupation-level performance metrics, which could offer a more nuanced view of AI capabilities across diverse roles. However, challenges remain, including limited statistical power for low-frequency tasks and the difficulty of measuring real-world task fidelity.

The study underscores a growing gap between the tasks that dominate the U.S. workforce and the tools available to evaluate AI’s impact on them. Without broader, more representative benchmarking, understanding AI’s true economic role will remain incomplete.