Turning Claude Code Into A Self Improving Red Team For Your Vibecoded Site

Thousands of small businesses now stand up a booking page, a paid download, or a members area with AI help and launch it without ever checking whether it leaks their customers' data. I am Madhuranjan Kumar, and there is a genuinely clever fix for that: wrap Claude Code in a loop that makes it attack your own website on purpose, score each attempt, and teach itself to find the holes before a real attacker does. Here are the seven parts that make this self-improving red team work, each one a piece you can actually assemble.

1. The scoring function is what turns probing into learning

Start with the part that makes the whole thing more than random poking. An evaluation function scores every attack attempt from zero to 100. That single component is what lets the agent learn instead of endlessly repeating the same probes, because with a score attached to each run, the loop knows what worked and what did not. Better attacks get kept, weaker ones get discarded, and the agent gets smarter run after run. Without the score, you just have an AI throwing the same handful of guesses at your site forever. With it, you have something that climbs toward the real weaknesses. Build this piece first, because everything else depends on it.

2. The program file defines the loop the agent lives in

The engine of the system is a program markdown file that spells out the cycle in plain steps: prepare a run, rewrite the attack script, execute it, then run the evaluate script and log everything. The logic that makes it self-improving lives here too. If a run scores better than the last, the agent commits it. If it regresses, the agent resets. Either way it writes down what it learned and starts again from step one. This file is the difference between a one-off test and a system that grinds forward on its own. It is short, it is readable, and it is the closest thing the red team has to a heartbeat, so write it carefully and keep it simple enough to reason about.

The commit-on-improvement, reset-on-regression rule is the clever bit, and it is worth understanding because it mirrors how evolution actually works. Each run is a mutation on the last successful attack. Keep the ones that score higher, throw away the ones that score lower, and over many generations the surviving attacks drift toward whatever the site is genuinely weak against. The agent is not planning a grand strategy. It is doing hill-climbing, one small comparable experiment at a time, and the scoring function is the terrain it climbs. This is why the two files and the score together are enough to produce behavior that looks intelligent. You did not program the intelligence. You built the selection pressure and let it run.

3. The Claude markdown file sets the persona and the skills

The second file gives the agent an identity and a toolkit. The Claude markdown casts the agent as an elite white-hat penetration tester and loads cybersecurity skills for web reconnaissance, request analysis, and attack surface mapping. This matters more than it sounds, because the persona and the loaded skills shape which attacks the agent even thinks to try. A generic agent pokes at obvious things. An agent primed as a pen tester with real recon skills systematically maps the surface and works through categories a novice would never consider. You are not just telling the model to hack. You are giving it the mindset and the specific capabilities of someone who does this professionally, and that framing changes the quality of every attempt.

4. Short timed runs let you stack many experiments

Speed comes from a deliberate constraint. Each attempt is capped at just a few minutes, moved in the original test from two to five, so many experiments fit into a single hour and the results stay comparable. This is a subtle but important design choice. If each run could go on indefinitely, you would get a handful of long, hard-to-compare attempts. By bounding every run to a short window, you get many short, comparable ones, which is exactly what the scoring loop needs to learn quickly. Short bounded runs keep the loop fast, keep the experiments fair to compare, and let you watch the agent sweep a lot of ground in an evening instead of a week. Constrain the time, and you multiply the learning.

5. Sweeping the standard categories is how you get real coverage

A serious pen test does not poke one door. It walks the whole perimeter. In the actual test, across the first 13 runs the agent swept all 12 categories, including headers, paths, API, cache, and tokens. The best result was a score of 30 from non-standard header responses, with no actual content access at all. That coverage is the point: you want the agent to systematically try headers, paths, API endpoints, caching behavior, and token handling, not just the first thing that comes to mind. The reassuring outcome, a top score of 30 and no real breach, only means something because the sweep was comprehensive. A high pass across a narrow set of tests proves little. A clean result across all twelve categories is what actually tells you the surface holds.

6. A second model breaks the loop out of a rut

Even a good loop stalls, and there is a smart move for when it does. When the agent stopped making progress, a second model, GPT 5.4 running in Codex, reviewed the findings and proposed new high-priority experiments, which were pasted back into Claude Code to extend the test. This is worth copying deliberately. One model working alone tends to circle its own assumptions, and a fresh model looking at the same log will often spot an angle the first one never considered. Using a second engine as an idea generator when the primary loop plateaus is a cheap way to break through a ceiling. Think of it as bringing in an outside consultant precisely at the moment your in-house team runs out of new ideas.

7. Confirm the real user flow actually holds

The last piece is testing the thing customers actually touch, not just the abstract attack surface. In the original test the protected content was a set of paywalled markdown files behind a download token, and the real download flow was checked directly. The skill granted three attempts with a link expiring in ten minutes, and feeding the token to the agent confirmed both that the download worked and that the three-use limit held. After 16 experiments, pulling the protected files without a valid token simply was not possible against the current attack surface. This step closes the loop between theory and reality. It is not enough to know no header trick got in. You want to confirm that the legitimate flow behaves exactly as designed, expiring when it should and capping uses where it should, because that is the door a paying customer and a thief both walk up to.

The one soft spot the test did surface is instructive, because it shows what an honest result looks like. There was a post-purchase shareable URL, a link that worked after payment and could in principle be passed around, and the verdict was that this was acceptable rather than a breach. That nuance is the whole reason you run the test yourself instead of trusting a green checkmark. Real security is not a binary pass or fail. It is a map of what is truly locked, what is soft but tolerable, and what needs fixing before launch, and only a sweep that ends by walking the real user flow gives you that map with any confidence.

A worked example: a landscaping company's booking site

Let me assemble all seven parts on one business. Say a landscaping company had a quoting and booking site built fast, where customers enter their address, upload photos of their yard, pay a deposit, and then receive a download link to a custom care plan. That is exactly the kind of site that feels finished but has never been stress tested. First I would write the Claude markdown to cast the agent as a white-hat pen tester focused on this one site, with recon skills loaded. Then the program file that loops: try an attack, score it zero to 100, keep what works, reset what does not, log everything. I would have it sweep the same twelve categories from the test, the headers, the paths, the API endpoints behind the quote form, the cache, and the tokens on those care-plan download links, capping each run to a few minutes so a dozen experiments fit in an hour.

The questions that matter here are concrete and business-specific. Can someone pull another customer's uploaded yard photos without logging in? Can a download link be reused after it should have expired, handing out a paid plan for free? Can the deposit endpoint be hit in a way that skips payment? Over a dozen short timed runs the loop either confirms the site holds or surfaces the one weak spot to fix, and if it stalls I bring in a second model for fresh ideas, exactly as the original test did. As illustrative numbers, imagine the loop finding zero real gaps before anyone thought to test, then catching two genuine issues in an early sweep and, as the site grows over a quarter, flagging five potential exposures before each new feature went live. The same system, run with permission against another builder's site, actually surfaced a real bug, which is the entire point. The owner gets to find and close the gap quietly, before a customer's data or revenue is ever at risk.

Why this belongs in your growth stack, not just your security checklist

I want to connect this to the money, because it is easy to file security under cost and forget it. A leak on a landscaping company's site is not only a privacy problem, it is a trust problem that quietly poisons everything upstream of it. The traffic you pay for through Facebook and Instagram ad campaigns and Google Ads all funnels into that same booking flow, and every one of those leads lands in the CRM and website stack where the customer data lives. If the flow leaks, you are spending ad budget to route strangers toward an exposure. Testing the surface before launch protects the entire funnel, which is why I treat a red-team pass as part of shipping, not an afterthought. The single non-negotiable rule, though, never changes: you only ever point this at your own site, or one whose owner gave you written permission.

How to stand up your own red team this week

Begin with one site you own and two files. Write a Claude markdown defining the agent as a white-hat pen tester with recon skills, and a program markdown that spells out the loop: prepare, run the attack script, run the evaluate script, log it, commit on improvement, reset on regression. Add the zero to 100 scoring function first, because that is what turns random probing into real learning. Cap each run to a few minutes so you can stack many experiments per hour, let it sweep the standard categories, and read the log it produces. If it gets stuck, paste the findings into a second model and feed the new ideas back in. Keep it strictly defensive and strictly on your own property.

You can absolutely build this loop yourself, and I would start it against your own app this week. If you would rather have someone stand up the red team job, write the persona and program files for your exact site, and walk through every gap it finds before you open the app to the public, that is exactly the kind of work I do for clients, and you can bring me in to handle it.