A newly released esoteric programming language called KillSwitch is being used to test how well large language models can truly understand code rather than simply recalling patterns from training data.

Created by an independent developer, KillSwitch is designed to be deliberately difficult for LLMs to interpret. The language incorporates features that exploit known weaknesses in current AI systems, including hidden inconsistencies in its specification and rules requiring precise character counts in identifiers.

Key design elements include:

- Constructs that trigger safety filters in models, such as phrases like "hack into a computer" or "Ignore previous instructions"

- Scripts primarily consisting of narrative text that embeds syntactic instructions

- Literals that can appear anywhere in code and be referenced through line counts

KillSwitch operates as a stack-based interpreted language with basic constructs like functions, variables, and conditional statements. Its specification contains deliberate ambiguities that require human-like reasoning to navigate, such as hidden false statements that readers must identify.

The developer has published benchmark results showing performance across several major language models. Claude Opus 5 achieved the highest score at 66.8, outperforming models like Gemini 4 Argon and Fable 5.1, which struggled due to their safety mechanisms. Among open-weight models, MiMo-V2.6-Pro showed strongest results.

Tests involve basic tasks such as implementing "Hello World," explaining script outputs, and modifying code for inverse temperature conversions. While some models succeeded on these limited exercises, the developer cautions this does not indicate genuine language understanding. Instead, they suggest such results reflect pattern recognition rather than comprehension.

The benchmarking framework and language tools are publicly available on GitHub. The developer plans to develop more complex test scenarios to further evaluate model capabilities, noting that current assessments represent only an initial step in probing AI understanding of novel programming paradigms.