I Got Tired of AI Quiz Tools Making Up Facts That Weren't In My Notes, So I Built One That Can't

I Got Tired of AI Quiz Tools Making Up Facts That Weren’t In My Notes, So I Built One That Can’t

我厌倦了 AI 测验工具编造笔记中不存在的事实,于是我亲手做了一个不会乱编的

Two nights before a chemistry final, I pasted my notes into an AI quiz generator to test myself. One question asked about a reaction I never studied. I got it wrong, looked it up afterward, and it wasn’t in my notes at all. The tool had just made it up. 在化学期末考试的前两个晚上,我把笔记粘贴到一个 AI 测验生成器里进行自我测试。其中一个问题问到了一个我从未学过的反应。我答错了,事后查阅资料才发现,笔记里根本没有提到这个内容。那个工具纯粹是编造出来的。

I went looking for a better one and hit the same wall three more times with three different tools. Paste your notes, get a quiz, and somewhere in the output is a question built on a fact your source material never mentioned. Nobody flags it. You just find out when you’re wrong about something you were sure you’d studied. 我试图寻找更好的工具,结果又在另外三个不同的工具上碰了三次壁。粘贴笔记,生成测验,输出结果中总会混入基于源材料中从未提及的事实所提出的问题。没人会标记出这些错误。你只有在答错那些你确信自己已经复习过的内容时,才会发现问题。

The failure mode makes sense once you think about how these tools are built. Most of them prompt an LLM with something like “generate 10 quiz questions from this text” and print whatever comes back. The model is good at sounding right. It is not naturally good at staying inside the boundary of what you actually gave it, and a prompt that says “don’t hallucinate” is a request, not a constraint. The model can ignore it and you’d never know from the output alone. 一旦你思考这些工具的构建方式,这种故障模式就很好理解了。它们大多数只是给大语言模型(LLM)发送类似“根据这段文字生成 10 个测验问题”的指令,然后直接输出结果。模型擅长让内容听起来正确,但它并不擅长自动限制在用户提供的范围内。所谓的“不要产生幻觉”指令仅仅是一个请求,而非强制约束。模型完全可以忽略它,而你仅从输出结果中根本无法察觉。

So I built QuizPaste around a different idea: don’t ask the model to be honest, check it. When it generates a question, it has to also point at the sentence in your source text the question came from. Before that question ever gets shown to you, the code tries to actually locate that sentence in your original text. If it can’t find it (wrong wording, a made-up detail, or the line just isn’t there), the question gets thrown out silently and never reaches you. You only ever see questions the tool can prove came from your own material. Open any question and you can see the exact line highlighted. 因此,我围绕一个不同的理念构建了 QuizPaste:不要指望模型诚实,而是要验证它。当它生成一个问题时,必须同时指出该问题源自源文本中的哪一句话。在问题展示给你之前,代码会尝试在你的原始文本中定位该句子。如果找不到(措辞错误、细节编造或根本没有这一行),该问题会被静默丢弃,绝不会出现在你面前。你看到的每一个问题,都是工具能证明源自你自身材料的。打开任何一个问题,你都能看到高亮显示的原文。

Using it is the boring part, on purpose. Paste lecture notes or a block of text, or grab a YouTube video’s transcript (open the video, three dots, “Show transcript,” copy the panel text, paste it in). There’s no scraping involved, so it doesn’t break when YouTube changes something on their end. You get a practice quiz plus flashcards in about five seconds. 使用它的过程很“无聊”,这是我刻意为之的。粘贴讲义笔记或一段文本,或者获取 YouTube 视频的转录稿(打开视频,点击三个点,选择“显示转录稿”,复制面板文字并粘贴)。这里不涉及任何爬虫技术,所以即使 YouTube 更改了后端,它也不会失效。你可以在五秒钟内得到一份练习测验和抽认卡。

You don’t need an account for your first two runs a day. If you want more than that or want your history saved across devices, that’s what the free account is for. Everything you generate exports to a tab-separated file Anki imports directly, plus a CSV if you’d rather handle it yourself, no watermark either way. 每天前两次使用无需注册账号。如果你需要更多次数或希望在不同设备间同步历史记录,可以注册一个免费账号。你生成的所有内容都可以导出为 Anki 可直接导入的制表符分隔文件(TSV),或者如果你想自己处理,也可以导出为 CSV 格式,且均无水印。

What it doesn’t do yet: there’s no spaced-repetition scheduling built in. That’s what exporting to Anki is for; I’m not trying to rebuild Anki’s scheduler badly. No packaged .apkg file yet, just the .tsv/CSV, though a real .apkg is on the list. No account sync unless you actually make an account. I’d rather ship the honest version of a feature than a half version that looks finished. 目前尚未实现的功能:没有内置间隔重复调度功能。那是导出到 Anki 的用途;我不想拙劣地重造 Anki 的调度器。目前还没有打包好的 .apkg 文件,只有 .tsv/CSV,不过制作真正的 .apkg 文件已在计划中。除非你注册账号,否则不会进行账号同步。我宁愿发布一个诚实的功能版本,也不愿发布一个看起来完成但实际上半吊子的版本。

The site’s about four days old as I’m writing this. I don’t have user numbers worth quoting and I’m not going to make any up. What I do have is a specific, fixable problem I kept hitting personally, and a fix that’s a real constraint in the code rather than a line in the marketing copy. If you study from your own notes or a lecture recording and want to self-test without wondering whether a question is testing something you never actually wrote down, it’s free to try at https://quizpaste.com/. 写这篇文章时,这个网站上线大约四天。我没有值得引用的用户数据,也不会编造任何数据。我所拥有的是一个我个人不断遇到的、可解决的具体问题,以及一个在代码层面真正实现约束的解决方案,而不是营销文案里的空话。如果你习惯通过自己的笔记或讲座录音学习,并且想在自我测试时不必担心问题是否超纲,欢迎免费试用:https://quizpaste.com/。