I Put Real Planes in the Sky With AR. The 3D Was the Easy Part.

我用 AR 把真实的飞机放到了天空中，但 3D 建模其实是最简单的部分。

A few years ago I would have told you AR was a graphics problem. You get a 3D model, you get a camera feed, you draw one on top of the other. Ship it. Then I built location-based AR for real, and the graphics turned out to be the least interesting part of the whole thing. The build that taught me this was Planes XR, one of the location-based AR experiences I worked on for ARCortex. 几年前，我会告诉你 AR 是一个图形学问题。你获取一个 3D 模型，获取一个摄像头画面，然后把模型叠加在画面上。发布就行了。但当我真正构建了基于地理位置的 AR 后，我发现图形渲染反而是整个过程中最无趣的部分。让我明白这一点的项目是 Planes XR，这是我为 ARCortex 开发的基于地理位置的 AR 体验之一。

The pitch is simple to say and brutal to deliver: point your phone at the sky, and the actual aircraft flying overhead show up, positioned where they really are. Not a canned demo of a plane. The plane that is genuinely up there right now, pulled from live flight data, rendered in the correct patch of sky. If you have only ever done marker-based or surface AR, that one requirement changes almost every engineering decision you make. So I want to walk through why, because the lesson generalizes well past AR. 这个产品的构想说起来简单，但实现起来却极其残酷：将手机对准天空，头顶上真实的飞机就会显示出来，并定位在它们实际所在的位置。这不是预设的飞机演示，而是此时此刻真正在你头顶上飞行的飞机，通过实时飞行数据获取，并渲染在天空对应的位置。如果你只做过基于标记（Marker-based）或平面（Surface）的 AR，那么这一个需求几乎会改变你所有的工程决策。因此，我想带大家了解一下原因，因为这个经验教训完全可以推广到 AR 之外的领域。

The three flavors of AR are not the same job

AR 的三种形态并非同一类工作

It is worth being precise, because “AR” gets used for three genuinely different builds: 我们需要精确定义，因为“AR”一词被用于三种截然不同的构建方式：

• Marker-based AR shows content when the camera recognizes a specific image or QR code. Reliable, but something physical has to be in frame.
• 基于标记的 AR：当摄像头识别出特定的图像或二维码时显示内容。这种方式很可靠，但画面中必须包含实物。
• Surface AR drops content onto a detected flat surface near you, a floor or a table. This is your furniture-in-your-living-room app.
• 基于平面的 AR：将内容放置在你附近的检测到的平面上，比如地板或桌子。这就是你常用的“客厅家具摆放”类应用。
• Location-based AR anchors content to real-world coordinates, so it works across a whole site, a city, or the open sky, with no marker at all.
• 基于地理位置的 AR：将内容锚定在现实世界的坐标上，因此它可以在整个场地、城市甚至广阔的天空中运行，完全不需要任何标记。

Marker and surface AR are, engineering-wise, mostly solved problems with good SDK support. The moment your experience has to span something bigger than a tabletop, you cross into location-based territory, and the difficulty steps up hard. 从工程角度来看，基于标记和基于平面的 AR 基本上是已解决的问题，且有良好的 SDK 支持。但一旦你的体验范围超过了桌面大小，你就进入了基于地理位置的领域，难度会陡然上升。

Location-based AR is really three systems pretending to be one

基于地理位置的 AR 实际上是三个系统在“假装”成一个

A location-based AR app has to know three things at the same instant: where the user is, which way they are facing, and where the content is supposed to live in the world. Get all three aligned and a digital object sits convincingly in a real street or a real sky. Get any one wrong and the illusion collapses instantly, because humans are shockingly good at noticing when something floats in the wrong spot. That splits into three subsystems, and each one is a place the whole thing can quietly fall apart. 一个基于地理位置的 AR 应用必须在同一瞬间知道三件事：用户在哪里、他们朝向哪里，以及内容应该存在于世界的什么位置。如果这三者对齐，数字对象就能令人信服地出现在真实的街道或天空中。如果其中任何一个出错，幻觉会瞬间破灭，因为人类对于物体漂浮在错误位置的感知能力强得惊人。这被拆分为三个子系统，每一个都可能导致整个系统悄无声息地崩溃。

Positioning 定位 The app fuses several signals to place you in the world: GPS for a rough fix, the compass and motion sensors for heading, and the camera itself for fine correction. The thing nobody tells you up front is that raw GPS is only accurate to several meters. That is completely fine for “you are near this landmark” and completely useless for “this arrow points at that specific doorway.” Closing that gap, from good-enough-for-a-map to good-enough-to-overlay, is one of the core engineering problems in the entire category. You do not solve it with a better model. You solve it with sensor fusion. 应用通过融合多种信号来确定你在世界中的位置：GPS 用于粗略定位，指南针和运动传感器用于确定朝向，摄像头本身用于精细校正。没人会提前告诉你的是，原始 GPS 的精度只有几米。这对于“你在某个地标附近”来说完全没问题，但对于“这个箭头指向那个特定的门口”来说则毫无用处。缩小这一差距——从“地图够用”到“叠加够用”——是整个领域的核心工程问题之一。你无法通过更好的模型来解决它，你必须通过传感器融合来解决。

Anchoring 锚定 Once you know where the user is, you have to lock the content to the world and hold it there as they walk, turn, and pan around. If the anchor drifts even slightly, the object slides off its spot and the experience reads as broken. Stable anchoring is the whole difference between an overlay that feels physically present and one that visibly wobbles, and it gets harder outdoors, at distance, and across large areas, which is to say exactly the conditions location-based AR lives in. 一旦你知道了用户的位置，你就必须将内容锁定在世界中，并在用户行走、转动和环顾四周时保持其位置不变。如果锚点哪怕有轻微的漂移，物体就会偏离位置，体验就会显得很糟糕。稳定的锚定是“感觉真实存在”的叠加层与“明显晃动”的叠加层之间的全部区别。在户外、远距离和大范围场景下，这种难度会进一步增加，而这恰恰是基于地理位置的 AR 所处的环境。

Data 数据 Here is the part that ate the most of my time. The interesting location-based experiences are almost never showing a static model. They are showing something tied to what is happening right now at that place, which means a live-data pipeline feeding the immersive layer. For Planes XR that meant pulling live aircraft positions from OpenSky, keeping them current, and rendering each plane where it actually is in the sky relative to where the user is standing and pointing. That pipeline, fetching the data, cleaning it, keeping it fresh, and handling the moments it is slow or simply unavailable, was as much work as the AR rendering itself. The overlay is the part users see. The data plumbing is the part that decides whether they trust it. 这是最耗费我时间的部分。有趣的基于地理位置的体验几乎从不展示静态模型。它们展示的是与该地点当前发生的事情相关的内容，这意味着需要一个实时数据管道来为沉浸式层提供支持。对于 Planes XR 来说，这意味着从 OpenSky 获取实时飞机位置，保持数据更新，并根据用户站立和指向的位置，将每架飞机渲染在天空中它实际所在的地方。那个获取数据、清洗数据、保持数据新鲜，以及处理数据缓慢或不可用情况的管道，其工作量与 AR 渲染本身一样大。叠加层是用户看到的部分，而数据管道则是决定用户是否信任它的部分。

The failure modes live in the seams, not the scene

故障模式存在于“接缝”中，而不是场景中

That sentence is the thing I actually took away from the build, and it is why I keep bringing it up in other work. Every part of Planes XR that was hard to get right lived in a seam between two systems, not inside any one of them. The 3D plane model rendered fine on day one. What broke was the plane rendering in the correct spot only when positioning, heading, and the data feed all agreed at the same moment. When a GPS reading was stale, or the compass drifted a few degrees, or the OpenSky feed lagged, no single component was “wrong,” but the plane ended up in the wrong piece of sky and the whole thing felt fake. You debug those problems by staring at the boundaries between subsystems, not by staring at any subsystem alone. That is a very different debugging mindset from a normal app, where a bug usually lives in one function you can put a breakpoint in. 这句话是我从这个项目中真正学到的东西，也是我不断在其他工作中提及它的原因。Planes XR 中所有难以实现的部分都存在于两个系统之间的“接缝”处，而不是任何一个系统内部。3D 飞机模型在第一天就渲染得很完美。真正出问题的是：只有当定位、朝向和数据流在同一时刻达成一致时，飞机才能渲染在正确的位置。当 GPS 读数过时、指南针偏移了几度，或者 OpenSky 数据流延迟时，没有任何一个单一组件是“错误”的，但飞机最终出现在了错误的天空位置，整个体验就显得很假。调试这些问题时，你需要盯着子系统之间的边界，而不是盯着任何一个单独的子系统。这与普通应用的调试思维截然不同，在普通应用中，Bug 通常存在于你可以设置断点的某个函数里。

How I scope one now, so it does not become a science project

我现在如何规划项目，以防它变成一个“科学实验”

The most expensive mistake in this category is commissioning a broad, city-scale, multi-feature experience before proving the core alignment even feels right. So the way I approach it now is deliberately narrow at the start: Prototype the hardest alignment in week one. If the whole thing hinges on content landing accurately at a real place, build exactly that first, at one single location, and then go physically stand in the real spot with a real device. Everything else is comparatively predictable once positioning and anchoring feel right. Prove one location before you build a hundred. Nail a single site, or a single sky, measure whether the overlay holds and the data stays trustworthy, then expand. Coverage is a multiplier, not a starting point. Treat the data pipeline as its own project. If live data is part of the value, budget for it as a first-class workstream, including the unglamorous parts: freshness. 在这个领域，最昂贵的错误就是在证明核心对齐效果良好之前，就启动一个覆盖城市规模、具备多种功能的宏大体验。所以我现在的方法是：一开始就刻意缩小范围。第一周就对最难的对齐部分进行原型设计。如果整个项目的关键在于内容能否准确落在现实地点，那就先只做一个地点，然后带着真实设备亲自去那个地方测试。一旦定位和锚定感觉对了，其他一切都是相对可预测的。在构建一百个地点之前，先证明一个地点是可行的。搞定一个场地或一片天空，测量叠加层是否稳定、数据是否可靠，然后再进行扩展。覆盖范围是乘数，而不是起点。将数据管道视为一个独立的项目。如果实时数据是价值的一部分，请将其作为一流的工作流进行预算，包括那些枯燥的部分：数据新鲜度。