VigilSAR Publishes Public Leaderboard for Defense AI Models, Kimi K3 Debuts High

VigilSAR Defense LLM Benchmark — The public benchmark page — aggregate results public, task set private. Source: vigilsar.com

In the world of defense-related AI, VigilSAR has taken a unique step by publishing a public leaderboard to showcase which language models are trusted for intelligence, surveillance, and reconnaissance (ISR) tasks. Unlike typical AI benchmarks, the evaluation focuses on reasoning, reporting, and restraint — the skills an analyst requires, not just general trivia.

The test setup involved 14 models, 300 tasks, and was conducted on July 17, 2026. Importantly, the task set is kept private — deliberately, to prevent models from training on it — with a separate, held-out set used for validation. This approach allows VigilSAR to publish the score gaps between public and private results, which indicates how much models might be memorizing versus truly understanding.

Current standings show Claude-Fable-5 leading with a score of 67.77 (Band A, pinned). A notable newcomer is Moonshot’s Kimi K3, debuting at #3 with a score of 64.65. This model falls into Band B and outperforms every GPT and Gemini model on the board. The rankings are based on confidence intervals and band placements rather than exact positions, emphasizing the reliability of the scores.

Interestingly, VigilSAR also highlights the deployment reality by scoring a locally-runnable open model as “sovereign-deployable,” indicating its suitability for real-world use. The site clarifies that “vendor claims are not evidence”, and the evaluation is designed for transparency and trustworthiness, with no vendor influence. This ensures the models are judged solely on their performance, not marketing hype.

VigilSAR public LLM leaderboard — The leaderboard — compare bands, not rank numbers. Source: vigilsar.com/benchmark

For those interested in the details, the public leaderboard offers a comprehensive view of the current standings, confidence intervals, and the gap between public and held-out scores. VigilSAR emphasizes its commitment to honesty through bands, confidence intervals, and transparent metrics like the VigilSAR platform itself.

This initiative underscores the importance of rigorous, transparent testing in defense AI development, especially as new entrants like Kimi K3 demonstrate competitive performance. As the field evolves, such benchmarks could become vital in guiding responsible deployment and trust in AI systems used for critical intelligence tasks.