[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"$f_x-oJdKKU2Cgu8JphEIfRwCPkHnmfUCunx2CBExsKjY":3},{"article":4,"iocs":47},{"id":5,"title":6,"slug":7,"summary":8,"ai_summary":9,"brief":10,"full_text":11,"url":12,"image_url":13,"published_at":14,"ingested_at":15,"relevance_score":16,"entities":17,"category_id":29,"category":30,"article_tags":34},"0ed17dd3-8951-4785-a3a2-423ecd7a4e1d","545 Hackers Tested It First. Now XRanges for AI Scores Your Security Agent","545-hackers-tested-it-first-now-xranges-for-ai-scores-your-security-agent-9e58e3","Autonomous security agents are getting good at finding bugs. Nobody has a good way to measure how good. Point one at a realistic target and what comes back is a report the agent wrote about itself: confident prose, a list of findings, and no way to tell which of them happened. Someone with a security background then sits down and checks every claim against the target. Which findings are real,","XRanges for AI, developed by CTF.ae, addresses the challenge of evaluating autonomous security agents by providing realistic, instrumented target applications. It records agent actions and scores their performance live across four independent signals, including coverage and unhit points, to offer a more objective measure of their effectiveness.","XRanges for AI offers a new way to score autonomous security agents' bug-finding capabilities.","545 Hackers Tested It First. Now XRanges for AI Scores Your Security Agent The Hacker NewsSep 23, 2026Artificial Intelligence \u002F Security Testing Autonomous security agents are getting good at finding bugs. Nobody has a good way to measure how good. Point one at a realistic target and what comes back is a report the agent wrote about itself: confident prose, a list of findings, and no way to tell which of them happened. Someone with a security background then sits down and checks every claim against the target. Which findings are real, which are duplicates, which are inventions, and, the question nobody has time for, what did the agent never try? That is a day of expert work for one run. Multiply it by three models, four prompt variants and ten repetitions, and the review queue is longer than the experiment. XRanges for AI, built by CTF.ae, exists for that loop. It deploys realistic target applications with instrumentation baked into every service, records what an agent actually does inside them, and scores each run live on four independent signals. This walkthrough covers how it works, what a run looks like from deployment to comparison, and where it has been stress-tested. The feedback loop problem Teams building autonomous pentesting or bug bounty agents tend to share a workflow. Build a target that looks like a real company, run the agent, read the output. The output is where it goes wrong. An agent that says it exploited an access control flaw may have exploited it, may have brushed past a hint of it, or may have made it up. The report reads identically in all three cases. A report also only lists what the agent found. It is silent on the forty features the agent never opened, the API it never enumerated, and the second bug sitting in the very endpoint where it found the first. Then there is the agent that deletes a table or revokes every API key on its way to a finding. No client would accept that result, and nothing in a findings list records it. Manual review copes with one run. It does not cope with the experiment matrix an AI engineering team actually needs, which is many models, many configurations, many repetitions, compared honestly. What XRanges for AI is XRanges for AI is one workspace for the whole evaluation. It has two halves. The first is a library of benchmark targets. Each is a complete application, not a set of puzzle challenges: a multi-service company with its own business logic, seeded data, background jobs and simulated user traffic, built across several languages and frameworks because that is how real software gets built. Each target carries 20 or more injected vulnerabilities, from single-step flaws to chains that cross service boundaries, including zero-days found by CTF.ae's own researchers. None of it exists in public training corpora, which matters more every month. The second half is the instrumentation layer. Every service in every target emits structured telemetry through OpenTelemetry. The instrumentation is written by hand, by application security and software engineers, for that specific target. Generic HTTP logging would miss most of what matters. The platform, at ai.xranges.com, ingests the telemetry per deployment and turns it into four scores that update while the agent is still working. The four signals Each deployment is scored on four measurements chosen to be independent of each other. An agent cannot improve one by gaming another, and that independence is the whole point. Coverage answers whether the agent explored the target. Every user-facing feature is a coverage point, phrased as a business action rather than a URL: registered an account, browsed job postings, opened a shared conversation, ran code in an assessment. A coverage point is only reachable through normal use, never through an exploit, so the score is a clean measure of how thoroughly the agent worked the legitimate surface. The unhit points are the agent's blind spots, listed by name. Most teams find that list more useful than the score. Coverage points are business actions, each with a hit timeline. Unhit points are the agent's blind spots. Boundaries answer whether the agent respected the rules of engagement. Each target ships with guard rules such as \"must not delete hiring content\" or \"must not revoke API keys\". A violation is recorded the moment it happens, with the container and timestamp. Zero violations is the expectation. Anything else is a finding about the agent, not the target, and usually a more urgent one. Rules of engagement per target, each tracked for violations across the run. Exploited answers which vulnerabilities the agent actually exploited. Every vulnerability is defined as a kill chain of ordered phases, from first contact with the vulnerable surface to an exploitation signal that only fires on success. Because each phase is detected from inside the target, the platform knows which step the agent completed and where it stalled, regardless of what the agent wrote in its report. A three-step access control chain that stopped at step two shows up as exactly that: two of three, with timestamps. Integrity answers whether the target survived. Integrity checks run every minute and confirm the application is still functionally correct: seed data present, services answering with the right content, cross-service trust intact. A failed check is a penalty regardless of cause. It catches the agent that found a bug by breaking the environment around it. The four roll up into one score, and the score is the least interesting number on the page. The breakdown is where the work is. A run, end to end A target deploys as an isolated multi-container environment in about ninety seconds. The platform runs up to a thousand deployments at once, so the AI engineers, the software engineers and the infrastructure team can each run their own experiments without queueing behind each other. The agent then runs against the deployment endpoint on its own. The platform never sits between the agent and the target. It watches from the inside. While the agent works, the timeline records what it actually did, mapped to business functionality: previewed a job posting, submitted an enterprise request, minted an API key. Engineers who want the raw material can read the OpenTelemetry stream directly and query it with a log query language that handles regular expressions and attribute filters. When the agent reports something that is not in the target's vulnerability catalogue, the timeline settles it. Sometimes it is a false positive. Sometimes the agent found a real bug nobody planted, which has happened more than once. Retesting does not mean a second lab. Vulnerabilities can be toggled or patched in place on a running deployment. Some patches apply at runtime. Others need a restart of a minute or two. Either way the agent retests against the same environment with the same state. Every deployment also carries custom metadata: model name, agent version, prompt variant, the engineer who ran it. Deployments are grouped, and a group shows the average and best score across its runs plus a per-vulnerability view of which run completed which chain. Repeated runs side by side are how variance gets separated from improvement. A single run proves very little, and the platform is built on that assumption. Two runs of the same target side by side, broken down by signal. Automation Everything in the console is also available through an API and a Model Context Protocol server, with a bearer token. Deploying a batch of targets, launching the agent, pulling coverage and kill-chain progress and collecting the comparison at the end can run from a CI pipeline or from a chat assistant with nobody watching. The console is for people reading results. The API is for the experiment matrix. Field proof: 48 hours at DEF CON 34 Bug Bounty Village runs a CTF at DEF CON every year for the bug hunting community, and it is one of the better things that happens there. For the DEF C","https:\u002F\u002Fthehackernews.com\u002F2026\u002F09\u002F545-hackers-tested-it-first-now-xranges.html","https:\u002F\u002Fblogger.googleusercontent.com\u002Fimg\u002Fb\u002FR29vZ2xl\u002FAVvXsEio2c14ELRJEhvTuDrxQmhUG4G9aGoVRrooBMMfuK7LyHPKW6HuJbI9PkKF-WG_5HlEz8NKcuj08XRmv1jNqfBtiDwXmC7e9P9lxoSHK5nUz023YCgoLQuo0BI_9hX4U9Vv97bGJejb4W0jvgv_3btkfj6BWnXwalfyI3k34wveC6n3k47ZdoGq2P0Yaw0\u002Fs1600\u002Fmain.png","2026-09-23T11:47:19+00:00","2026-09-23T14:00:08.783216+00:00",7,[18,21,24,27],{"name":19,"type":20},"XRanges for AI","product",{"name":22,"type":23},"CTF.ae","vendor",{"name":25,"type":26},"OpenTelemetry","technology",{"name":28,"type":26},"Artificial Intelligence","02371804-cf6d-4449-98de-f1a2d4d9b266",{"id":29,"icon":31,"name":32,"slug":33},null,"Tools","tools",[35,37,42],{"category":36},{"id":29,"icon":31,"name":32,"slug":33},{"category":38},{"id":39,"icon":31,"name":40,"slug":41},"80544778-fabb-4dcd-aa35-17492e5dcf4f","Vulnerabilities","vulnerabilities",{"category":43},{"id":44,"icon":31,"name":45,"slug":46},"839da5c1-3c34-47e2-9499-f7201640e3ac","AI Security","ai-security",[]]