nemotron-halloween

本文为原文前 6,000 字符的节选翻译，完整内容请查看原文。

Back to Articles nemotron-halloween Enterprise + Article Published October 30, 2026 Upvote 6 Rick Grandy rgrandy Follow nvidia Maryam Motamedi maryameee Follow nvidia Building a Haunted House with NVIDIA Nemotron Open Models Or rather… Boo! Now Get Off My Lawn! TL;DR: MonsterHouse is an interactive projection I created for Halloween. He listens to trick-or-treaters, answers in the voice of a grumpy old pirate ship and softens when visitors remember their manners. He is built on the NVIDIA Nemotron Voice Agent with NVIDIA Nemotron ASR, Nemotron 3.5 Bolt E4B 4GB, Magpie TTS Zeroshot and Audio2Face-3D, and he runs on a single workstation.

返回文章 nemotron-halloween 企业版 + 文章发布于 2026 年 10 月 30 日 点赞 6 Rick Grandy rgrandy 关注 nvidia Maryam Motamedi maryameee 关注 nvidia 使用 NVIDIA Nemotron 开源模型建造鬼屋，或者更确切地说……“呜！快从我的草坪上滚开！”简而言之：MonsterHouse 是我为万圣节创作的一个互动投影。它能倾听“不给糖就捣蛋”的孩子们的声音，用一艘脾气暴躁的旧海盗船的声音回答，并在访客表现出礼貌时变得温和。它基于 NVIDIA Nemotron 语音代理构建，集成了 NVIDIA Nemotron ASR、Nemotron 3.5 Bolt E4B 4GB、Magpie TTS Zeroshot 和 Audio2Face-3D，且运行在单台工作站上。

I wanted the kids visiting my house at Halloween to interact in a fun and unique way. The house was going to be in charge and giving out the treats, but more grumpy than scary. Somehow, my solution was to give the garage door a mouth and teach him to complain. That became MonsterHouse: a grumpy old building with pirate stories, a fiery grin and a suspicious weakness for “please.” NVIDIA Nemotron Voice Agent supplies the conversation foundation. Nemotron Speech does the listening, Bolt E4B is the brain, NVIDIA Magpie TTS Zeroshot is the live voice, and NVIDIA Audio2Face-3D makes the wooden mouth talk.

我希望万圣节来我家玩的孩子们能以一种有趣且独特的方式进行互动。房子将负责分发糖果，但比起恐怖，它更倾向于暴躁。不知怎的，我的解决方案是给车库门装上一张嘴，并教它抱怨。这就是 MonsterHouse 的由来：一座脾气暴躁的老建筑，有着海盗故事、火热的咧嘴笑，以及对“请”字毫无抵抗力的弱点。NVIDIA Nemotron 语音代理提供了对话基础。Nemotron Speech 负责倾听，Bolt E4B 是大脑，NVIDIA Magpie TTS Zeroshot 提供实时语音，而 NVIDIA Audio2Face-3D 则让木制的嘴巴动起来说话。

How Monster House works Listening: NVIDIA Nemotron 3 ASR turns visitor speech into text. [nemotron-speech-streaming-en-0.6b.q8_0.gguf ] Thinking: NVIDIA Nemotron 3.5 Bolt E4B 4GB, an open small language model for conversational agents, writes each reply. A LoRA adapter supplies the personality. Speaking: NVIDIA Magpie TTS Zeroshot voices each reply from a short reference sample of the character. Animating: NVIDIA Audio2Face-3D turns clean speech into mouth animation in Unreal Engine 5. Seeing: NVIDIA Nemotron 3 Nano Omni describes costumes, while YOLO and ByteTrack count and track visitors. Running it: NVIDIA Nemotron Voice Agent coordinates everything on one workstation with an NVIDIA RTX PRO 6000 Blackwell Max-Q and an NVIDIA RTX PRO 4000 Blackwell.

Monster House 的工作原理：倾听：NVIDIA Nemotron 3 ASR 将访客的语音转换为文本。[nemotron-speech-streaming-en-0.6b.q8_0.gguf ] 思考：NVIDIA Nemotron 3.5 Bolt E4B 4GB，一种用于对话代理的开源小型语言模型，负责编写每一条回复。LoRA 适配器提供了个性。说话：NVIDIA Magpie TTS Zeroshot 根据角色的简短参考样本为每条回复配音。动画：NVIDIA Audio2Face-3D 将清晰的语音转换为虚幻引擎 5 中的嘴部动画。视觉：NVIDIA Nemotron 3 Nano Omni 负责描述服装，而 YOLO 和 ByteTrack 则负责计数和追踪访客。运行：NVIDIA Nemotron 语音代理在配备 NVIDIA RTX PRO 6000 Blackwell Max-Q 和 NVIDIA RTX PRO 4000 Blackwell 的单台工作站上协调一切。

A grin full of splinters The mouth came first. I imagined the wooden garage door boards splitting into jagged lips, with the burning flames of hell illuminating the driveway from beyond whenever he opened his cursed mouth. He needed to look ancient, expressive and mildly offended by the existence of visitors. A few expressions from a house with plenty to say. Creating a custom animation rig was one of the more labor intensive parts of the build. After bashing it into submission, I could export the facial expression shapes that Audio2Face-3D could later animate. The character SkelMesh was built in Autodesk Maya then exported into Unreal Engine where virtual fire and effects were added behind the door. With no eyes, no eyebrows and no blinking, he’s a surprisingly economical monster.

满是木刺的咧嘴笑：嘴巴是第一步。我设想木制的车库门板裂开形成锯齿状的嘴唇，每当它张开那张被诅咒的嘴时，地狱的烈火就会从深处照亮车道。它需要看起来古老、富有表现力，并且对访客的存在感到轻微的冒犯。这是一座有很多话要说的房子的几种表情。创建自定义动画绑定是构建过程中最耗费劳动力的部分之一。在将其调整到位后，我可以导出面部表情形状，供 Audio2Face-3D 稍后进行动画处理。角色 SkelMesh 是在 Autodesk Maya 中构建的，然后导出到虚幻引擎中，并在门后添加了虚拟火焰和特效。由于没有眼睛、眉毛和眨眼动作，它是一个出奇经济的怪物。

While all the nonsense of building the character was going on, NVIDIA built the Nemotron Voice Agent. What was supposed to be the hard part of the build now became rather fun, iterative and easy. I had a foundation for listening, conversation and speech right out of the box. The house already had a mouth. Now he could finally find something to complain about. Teaching bad manners The house needed a past as cantankerous as its personality so I devised a backstory that the house had been built from the timbers of a sunken pirate ship, haunted by its stormy past. The nautical complaints now had a reason. This was the ship, reincarnated as real estate and not entirely pleased about it.

在构建角色的所有琐事进行的同时，NVIDIA 构建了 Nemotron 语音代理。原本应该是构建中最困难的部分，现在变得相当有趣、可迭代且简单。我开箱即用就拥有了倾听、对话和语音的基础。房子已经有了嘴巴，现在它终于可以找到抱怨的对象了。教授坏习惯：房子需要一个与其个性一样乖戾的过去，所以我设计了一个背景故事：这座房子是由一艘沉没的海盗船的木材建造的，被其风暴般的过去所困扰。航海方面的抱怨现在有了理由。这就是那艘船，转世成了房地产，而且对此并不完全满意。

The performance needed a guide, a path to follow from greeting to goodbye while providing treats along the way. First, the house would brush visitors off with remarks like “I’m busy grumbling, go bother the neighbors!” But persistence would wear him down. A “Please” could break through immediately, and “thank you” earned a warm farewell. Once a visitor won him over, I wanted him to stay won over. Nobody wants to negotiate with the same garage door twice. Here is where the Voice Agent really shines. You can choose any LLM you like. Because I wanted the system to be off-grid and completely self-contained, I had a memory and compute budget.

表演需要一个引导，一条从问候到告别并沿途提供糖果的路径。起初，房子会用“我正忙着发牢骚，去烦邻居吧！”之类的话打发访客。但坚持不懈会让他妥协。一句“请”可以立即打破僵局，而“谢谢”则能换来温暖的告别。一旦访客赢得了它的好感，我希望它能保持这种状态。没人想和同一个车库门谈判两次。这就是语音代理真正闪光的地方。你可以选择任何你喜欢的 LLM。因为我希望系统是离线且完全自包含的，所以我必须考虑内存和计算预算。

The character needs more than a prompt that says “be grumpy.” It had to follow a master conversation flow, respond to what visitors actually said, track encounter state, and leave enough compute for the rest of the pipeline. After testing more than a dozen models, I landed on Bolt E4B to supply the live dialogue. It is an open small language model purpose-built for local conversational agents and interactive characters, which is a fair description of a talking garage door. I use the 4-bit quantized release (the “4GB” in its name) as the default response model, with the BF16 checkpoint available for comparison.

这个角色需要的不仅仅是一个写着“要暴躁”的提示词。它必须遵循主对话流程，回应访客实际说的话，跟踪遭遇状态，并为管道的其余部分留出足够的计算资源。在测试了十几个模型后，我选择了 Bolt E4B 来提供实时对话。它是一个专为本地对话代理和交互式角色构建的开源小型语言模型，这对于一个会说话的车库门来说是一个恰当的描述。我使用 4 位量化版本（名称中的“4GB”）作为默认响应模型，并提供 BF16 检查点以供比较。

If it’s not obvious by now, I work at NVIDIA and due to this I was able to test Bolt E4B before its public release and found it to be a solid choice, with a smaller footprint than other frontrunners- Qwen3 30B-A3B and Qwen 3.6 35B-A3B. It works inside the Nemotron Voice Agent pipeline, turning what visitors say into the next suitably reluctant reply. Bolt provides the conversational foundation. My prompts, application logic and LoRA adapter trained on thousands of reviewed example conversations shape it into this particular stubborn ship.

如果现在还不明显的话，我在 NVIDIA 工作，因此我能够在 Bolt E4B 公开发布之前对其进行测试，并发现它是一个可靠的选择，其占用空间比其他领先模型（如 Qwen3 30B-A3B 和 Qwen 3.6 35B-A3B）更小。它在 Nemotron 语音代理管道内工作，将访客所说的话转化为下一个恰当的、不情愿的回复。Bolt 提供了对话基础。我的提示词、应用程序逻辑以及在数千个经过审查的对话示例上训练的 LoRA 适配器，将其塑造成了这艘固执的船。

Creating a dataset of over 5000 multi-turn conversations meant I could train LoRAs for each model I felt good enough to be an option. Some took about an hour to train, some took a couple. A local agent ran everything and benchmarked the results. That agent and a very large LLM created and judged the dataset to begin with. This happened at night or while I was doing the day job. With the LoRA in place I check the replies for continuity, timing and tone, benchmark models and adapters for quality, repetition and speed, and then listen to the performance. The house should give in eventually, without turning into a cheerful concierge. A little resistance is part of the charm. An endless ar

创建一个包含超过 5000 次多轮对话的数据集意味着我可以为每个我认为足够好的模型训练 LoRA。有些训练大约需要一个小时，有些则需要几个小时。一个本地代理运行了一切并对结果进行了基准测试。该代理和一个非常大的 LLM 首先创建并评估了数据集。这通常发生在晚上或我做日常工作的时候。有了 LoRA，我会检查回复的连贯性、时机和语气，对模型和适配器的质量、重复性和速度进行基准测试，然后聆听表演效果。房子最终应该会妥协，但不会变成一个开朗的礼宾员。一点点抵抗力正是其魅力的一部分。无尽的 ar