A new open-source initiative called Jevstiller is developing a method to run smaller, faster versions of the Jev AI system directly on users' devices. Jev, introduced earlier this year, is designed to answer structured queries such as choice, score, and noul questions without generating free-form text. Its creators describe it as a lightweight alternative to full-scale large language models for tasks that require rapid, low-cost responses.

Jevstiller aims to improve efficiency by creating a local model that handles queries it recognizes as familiar. When a request matches known patterns, the local model provides an answer without contacting Jev's servers. Uncertain or complex queries are forwarded to Jev for processing, with the interaction used to refine the local model over time. The system uses continuous auditing to maintain alignment with Jev, routing a fixed portion of requests to the upstream service regardless of local confidence levels.

According to the project's documentation, the local model begins with no knowledge of Jev's responses and learns through repeated interactions. Each query and its corresponding answer are collected to train the model and update routing rules. A background process periodically retrains the model based on new data, adjusting its confidence thresholds. If agreement with Jev drops below a user-defined target, the audit rate increases to accelerate retraining. In cases where performance collapses entirely, all queries are redirected back to Jev, and training restarts from that point.

In internal testing, the developers observed that when Jev altered its behavior mid-session, the local model detected the shift within minutes and adapted using only post-change data, without manual intervention. The system maintained functionality despite such changes, though the team emphasized that high agreement rates do not guarantee factual accuracy. They noted that Jevstiller reduces reliance on Jev's API for routine queries, potentially lowering operational costs and latency, with response times as low as 15 milliseconds on a device's CPU.

The project remains experimental, and its long-term viability depends on continued alignment with Jev's evolving behavior. Users interested in testing the tool can access documentation and setup guides through the project's public repository.