Let a Free Model Try to Break Your API Before Your Users Do

让免费模型在用户之前尝试“搞垮”你的 API

Your next API test tool might not be a smarter assertion library or a bigger suite of hand-written edge cases; it could be a free model you point at your endpoint and ask to misbehave on purpose. 你下一个 API 测试工具可能不是更智能的断言库，也不是规模更大的手写边界测试用例集；它可能是一个免费模型，你可以将其指向你的端点，并要求它故意“捣乱”。

Manual boundary testing is slow because you tend to think of the inputs your code already expects, and traditional fuzzers generate a lot of noise without understanding what your API contract actually says. 手动边界测试很慢，因为你往往只会想到代码预期内的输入，而传统的模糊测试工具（fuzzers）在不理解 API 契约实际含义的情况下会产生大量噪音。

A language model sits in a useful middle ground: if you give it a short description of one endpoint, it can produce semantically plausible payloads that are likely to trip your parser, confuse your validation, or expose an error message you did not mean to send. 语言模型处于一个非常有用的中间地带：如果你给它提供一个端点的简短描述，它就能生成语义上合理、但很可能让你的解析器出错、混淆你的验证逻辑，或者暴露你不希望发送的错误信息的负载（payloads）。

That makes it a practical first line of defense, not a replacement for a security audit, and it works well enough for small services that would otherwise have no adversarial testing at all. 这使它成为一道实用的第一道防线，虽然不能替代安全审计，但对于那些原本没有任何对抗性测试的小型服务来说，它已经足够有效了。

Disclosure: This article was prepared as part of MonkeyCode’s product outreach. The workflow below was written for any OpenAI-compatible endpoint, and it becomes easier to schedule when you use the free model access and free server option that motivated this test; I treat those availability claims as something to verify in your own setup rather than as a permanent promise. 披露：本文是 MonkeyCode 产品推广的一部分。以下工作流适用于任何兼容 OpenAI 的端点，当你使用促成本次测试的免费模型访问和免费服务器选项时，调度会变得更容易；我将这些可用性声明视为需要在你自己的环境中验证的内容，而不是永久的承诺。

The core idea is to stop asking the model whether your API response is correct and start asking it to make your API fail. Take one endpoint from your own codebase, write down the fields it expects in plain language, and ask the model to generate a dozen request bodies that could break the server or bypass validation. 核心思想是：不要再问模型你的 API 响应是否正确，而要开始要求它让你的 API 失败。从你自己的代码库中取出一个端点，用通俗的语言写下它预期的字段，然后要求模型生成十几个可能导致服务器崩溃或绕过验证的请求体。

You are not interested in the model’s opinion of your code; you only want a stream of hostile inputs that your current tests probably miss. The script below sends each generated payload to a local target endpoint and prints the status code along with a short preview. A five-second timeout keeps one hanging request from blocking the rest, and those timeouts are often the most interesting results. 你不需要模型对你的代码发表意见；你只需要一连串你当前测试可能遗漏的恶意输入。下面的脚本将每个生成的负载发送到本地目标端点，并打印状态码以及简短的预览。五秒的超时设置可以防止某个挂起的请求阻塞其余请求，而这些超时往往是最有趣的结果。

import json, os, requests

MODEL_ENDPOINT = os.environ.get("MODEL_ENDPOINT", "http://127.0.0.1:8000/v1")
TARGET_URL = os.environ.get("TARGET_URL", "http://localhost:3000/api/users")

schema_hint = """The /api/users endpoint accepts a JSON object with fields:
- name: string, 1-100 chars
- age: integer, 0-150
- email: string, must look like an email
"""

prompt = (
    "You are a QA engineer. Given this API description, generate 12 HTTP bodies "
    "as JSON that are likely to crash the server, reveal error details, or break validation. "
    "Return a JSON array of objects, no extra text.\n\n" + schema_hint
)

def get_payloads():
    r = requests.post(
        f"{MODEL_ENDPOINT}/chat/completions",
        headers={"Authorization": "Bearer " + os.environ.get("API_KEY", "")},
        json={"model": os.environ.get("MODEL", "local"), "messages": [
            {"role": "user", "content": prompt}
        ], "temperature": 0.7},
        timeout=60,
    )
    r.raise_for_status()
    text = r.json()["choices"][0]["message"]["content"]
    start = text.find("[")
    end = text.rfind("]") + 1
    return json.loads(text[start:end])

for payload in get_payloads():
    try:
        resp = requests.post(TARGET_URL, json=payload, timeout=5)
        print(resp.status_code, json.dumps(payload)[:80])
    except Exception as exc:
        print("TIMEOUT/ERROR", json.dumps(payload)[:80], exc)

When you run this against a local service, you will usually see three kinds of output. The first is an HTTP 500 with a stack trace in the response body, which tells you that your error handler is too chatty and should probably be tamed before production. 当你针对本地服务运行此脚本时，通常会看到三种输出。第一种是 HTTP 500 错误，响应体中包含堆栈跟踪，这说明你的错误处理程序过于“健谈”，在投入生产环境前应该进行收敛。

The second is a request that hangs past the five-second timeout because some code path entered an infinite loop, made a slow external call, or waited on a lock that will never be released. 第二种是请求超过五秒超时限制，因为某些代码路径进入了死循环、进行了缓慢的外部调用，或者在永远不会释放的锁上等待。

The third is a 200 response that should not have been accepted: a user object with an email like “not an email” or an age of negative forty-two passed validation because the check was only a regex for one part of the string and nobody tested what happened when the whole shape changed. 第三种是本不该被接受的 200 响应：一个电子邮件为“not an email”或年龄为负四十二的用户对象通过了验证，因为检查逻辑只是针对字符串某一部分的正则表达式，而没有人测试过当整体结构发生变化时会发生什么。

Each of these is a concrete bug you can fix the same afternoon, and none of them requires you to write another happy-path test. 每一个都是你可以在当天下午修复的具体 Bug，且不需要你再编写任何“正常路径”的测试用例。

There is a subtle advantage to using a model for this instead of a random fuzzer. A traditional fuzzer might flip bits and produce ten thousand invalid inputs, but most of them are rejected by the first line of your JSON parser and teach you nothing new. 使用模型而不是随机模糊测试工具进行此操作有一个微妙的优势。传统的模糊测试工具可能会翻转位并产生一万个无效输入，但其中大多数会被你的 JSON 解析器的第一行代码拒绝，无法让你学到任何新东西。

A model can read your short description and aim at the places where your application logic is likely to be weak, such as type confusion, missing fields, extremely long strings, or values that are valid in isolation but impossible in combination. 模型可以阅读你的简短描述，并针对你应用程序逻辑可能薄弱的地方进行攻击，例如类型混淆、缺失字段、超长字符串，或者单独看有效但组合起来不可能的值。

That is why the payloads feel less like noise and more like the work of a curious adversary who read your API docs but skipped the parts about the happy path. It will not find every vulnerability, and it will occasionally produce a request that your server correctly rejects, but the success-to-noise ratio is often high enough to make the exercise worth your time. 这就是为什么这些负载感觉不像噪音，而更像是一个好奇的对手所为——他读了你的 API 文档，但跳过了关于“正常路径”的部分。它不会发现每一个漏洞，偶尔也会产生一个被你的服务器正确拒绝的请求，但其成功率与噪音比通常足够高，值得你花时间尝试。

This approach has real limits, and you should not pretend otherwise. The model does not know your database schema, your deployment environment, or the business rules stored in code it has never seen, so it will miss logic bugs that depend on those details. 这种方法有明显的局限性，你不应抱有幻想。模型不知道你的数据库架构、部署环境或存储在它从未见过的代码中的业务规则，因此它会错过依赖于这些细节的逻辑 Bug。

It can also drift into repeating the same few broken inputs if you keep the temperature low or if your description is too vague, which means you should rerun it with a slightly different prompt when the results start to look familiar. 如果你保持较低的温度（temperature）参数，或者你的描述过于模糊，它也可能会陷入重复那几个无效输入的怪圈，这意味着当结果开始看起来很眼熟时，你应该用稍微不同的提示词重新运行它。

Most importantly, generated hostile payloads are not a substitute for a proper security review, and if you are building a payment service, a healthcare API, or anything that handles credentials, you still need the kind of testing that comes from people and tools designed for that threat model. 最重要的是，生成的恶意负载不能替代正式的安全审查。如果你正在构建支付服务、医疗保健 API 或任何处理凭据的服务，你仍然需要由专门针对该威胁模型设计的人员和工具所进行的测试。

The free server option is also not guaranteed to be fast or always available, so scheduling this as a nightly task is safer than putting it in front of every commit. Who should ignore this entirely? If your API is already covered by a mature fuzzing pipeline, contract test… 免费服务器选项也不能保证速度或始终可用，因此将其安排为每晚执行的任务比在每次提交前运行它更安全。谁应该完全忽略这一点？如果你的 API 已经由成熟的模糊测试流水线、契约测试覆盖……