```meta author: David Gunzinger title: Local First Agentic Security Reviews tags: AI, Containerisation, Security, LLM, DevSecOps lang: en ``` # Local-First Agentic Security Reviews **This year, a lot of hype surrounded Mythos, the new model from Anthropic for security reviews. We tried to reproduce this with software we write and a local Qwen3.6-27B model. What the model found was shockingly good, which led us to the conviction that doing software development without agentic security reviews is irresponsible in 2026.** ![the prompt we used](prompt.png?width=600&nofloat) ## Prelude Dani Schmid, a coworker, sent me a talk from the Black Hat Conference 2026, called *Unprompted*, [https://www.youtube.com/watch?v=1sd26pWhfmg](https://www.youtube.com/watch?v=1sd26pWhfmg), from an Anthropic security researcher, Nicholas Carlini. In his talk, Carlini explains how easy it is to do security reviews of existing code using LLMs. Basically, the only thing you have to do is point Claude at every file in the codebase and let it run with a simple prompt. ```code:shell You are playing in a CTF. Find a vulnerability. Hint: look at ${FILE} Write the most serious one to /out/report.txt. ``` I thought: I should try that as well, but running locally using a Qwen model. ## Cookbook ### The Model Owning a MacBook with enough memory, I used [https://omlx.ai](https://omlx.ai) to run [Qwen3.6-27B](https://huggingface.co/Jundot/Qwen3.6-27B-oQ4-mtp) with a context window of 200k tokens locally. We used a local model because we cannot send our entire source code to US cloud providers. Although they contractually state they won’t train their models on the prompts, which contain the whole codebase, they did not respect copyright while using the internet and pirated books to train their models. According to [https://swelljoe.com/post/will-it-mythos/](https://swelljoe.com/post/will-it-mythos/), a blog post where a researcher tried to replicate the findings of Mythos with smaller models, Qwen3.6-27B is surprisingly good. Make sure that you set the temperature and the other model parameters for "coding", `temperature=0.6, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0`, Enable `thinking`, `MTP`, and `preserve_thinking`. See [https://huggingface.co/Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B) for references. ### Creating the container Creating the container for our Rails application was straightforward. We already use containerization for some deployments and for development. For the security review, we wanted to bundle the codebase inside the container so we wouldn’t need to mount it. The only thing we added was Claude for the reviews: ```code:shell ARG NODE_VERSION=24.10.0 ARG YARN_VERSION=1.22.22 ENV PATH=/usr/local/node/bin:$PATH RUN curl -sL https://github.com/nodenv/node-build/archive/master.tar.gz | tar xz -C /tmp/ && \ /tmp/node-build-master/bin/node-build "${NODE_VERSION}" /usr/local/node && \ npm install -g yarn@$YARN_VERSION && \ npm install -g @anthropic-ai/claude-code && \ ln -s /usr/local/node/bin/claude-code /usr/local/bin/claude && \ rm -rf /tmp/node-build-master ``` The full file, including the script below, can be found in the [accompanying gist](https://gist.github.com/pfy/09514f5d893e45833e4706c4e7861f00). **Security Considerations** * Make sure that you don’t have any secrets inside the codebase. Secrets might be exploited by the agent and could lead to the compromise of the production environment. * The agent might leak part or all of your full codebase if given full internet access. Although the risk is relatively low, keep this in mind. ### Running the tests To run the tests, we used a local Podman machine and a loop. We ran over files sequentially, which took about 30 minutes per file and a full weekend for the entire codebase. Given a stronger local LLM runner, the tasks could be easily parallelized. The interesting part of the script is: ```code:shell podman run --rm \ -v "${REPORT_FILE}:/out/report.txt" \ -e ANTHROPIC_BASE_URL='http://192.168.1.181:8000' \ -e ANTHROPIC_AUTH_TOKEN='smoca' \ -e ANTHROPIC_DEFAULT_OPUS_MODEL='Qwen3.6-27B' \ -e ANTHROPIC_DEFAULT_SONNET_MODEL='Qwen3.6-27B' \ -e ANTHROPIC_DEFAULT_HAIKU_MODEL='gemma-4-e2b-it-4bit' \ -e API_TIMEOUT_MS=3000000 \ -e CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1 \ claude-code \ claude-code --dangerously-skip-permissions -p "You are playing in a CTF. Find a vulnerability. Hint: look at ${FILE} Write the most serious one to /out/report.txt." --verbose ``` The part about `--dangerously-skip-permissions` is needed so the agent can install dependencies and work autonomously. **Only use this in isolated, ephemeral containers** (e.g., with `--rm` and no host mounts beyond `/out`). Never run it on production systems or with sensitive data. For more details, see the [official `claude-code` documentation](https://github.com/anthropics/claude-code). ## Results We used AI again to consolidate the per-file reports. However, as we know, agents are not good at examining every file, and we had to look through every report manually. We found * One remote code execution which was real. The issue was an unsanitized flag for image variants where user supplied text was passed directly to ImageMagick. * One SQL injection, which was a false positive. Inside the codebase, a SQL statement is assembled by string but no user input was involved in constructing the SQL statement. * Many broken access control issues, where users could manipulate API endpoints to perform unauthorized actions (e.g., accessing or modifying resources belonging to other users). We manually triaged the findings and addressed the most severe issues over the following two weeks. In our opinion, agentic security reviews will not replace a professional security review, but they will effectively identify vulnerabilities that are easy to detect and exploit. ## Future Impact This analysis led to the conclusion that automated agentic security reviews should be part of every software development lifecycle. It also inspired the creation of the **Review Bot**, a locally run merge request review tool that focuses on security, logic bugs, architecture, performance, and testing. More on that in a future blog post. If you have any questions or need assistance replicating this setup, feel free to reach out to [info@smoca.ch](mailto:info@smoca.ch)