GPT 5.6 Sol is the best “vision” model OpenAI ever released

GPT 5.6 Sol 是 OpenAI 迄今为止发布的最强“视觉”模型

Last week, OpenAI announced the GPT-5.6 lineup, introducing the Sol, Terra, and Luna models. During the release stream, the team focused heavily on computer use, showing models capable of navigating and operating desktop applications. OpenAI highlighted UI agents and detailed 3D visualizations, but both depend on stronger visual understanding.

上周，OpenAI 发布了 GPT-5.6 系列，推出了 Sol、Terra 和 Luna 模型。在发布会直播中，团队重点展示了计算机使用能力，演示了模型如何导航并操作桌面应用程序。OpenAI 强调了 UI 智能体和详细的 3D 可视化效果，但这两者都依赖于更强大的视觉理解能力。

To measure their vision capabilities, we ran the models through our upcoming VLM benchmark, which we plan to release in the next few weeks. The benchmark covers common vision tasks, including detection, counting, OCR, and data extraction. In this post, we take a closer look at how GPT-5.6 performs across each of them.

为了衡量它们的视觉能力，我们通过即将发布的 VLM（视觉语言模型）基准测试对这些模型进行了评估，该基准测试计划在未来几周内发布。该基准涵盖了常见的视觉任务，包括检测、计数、OCR（光学字符识别）和数据提取。在本文中，我们将深入探讨 GPT-5.6 在各项任务中的表现。

Sol is clearly the best vision model OpenAI has released so far. The jump is especially visible in object detection and counting, where GPT-5.5 was far behind the strongest VLMs. Terra and Luna are not as strong as Sol, but both show meaningful progress over GPT-5.5.

Sol 显然是 OpenAI 迄今为止发布的最强视觉模型。这种飞跃在目标检测和计数方面尤为明显，而 GPT-5.5 在这些领域曾远落后于最强的 VLM。Terra 和 Luna 虽然不如 Sol 强大，但相比 GPT-5.5 都有显著进步。

Object Detection

目标检测

Detection is where GPT-5.6 shows the clearest jump. GPT-5.5 scored 13.8 mAP@50 in our benchmark, while Sol reached 46.2. Terra and Luna followed closely at 44.7 and 43.3, moving object detection from a major weakness to a practical capability.

检测是 GPT-5.6 进步最明显的领域。在我们的基准测试中，GPT-5.5 的得分为 13.8 mAP@50，而 Sol 达到了 46.2。Terra 和 Luna 分别以 44.7 和 43.3 紧随其后，使目标检测能力从一个主要短板转变为一项实用的功能。

Document layout detection is one of the clearest strengths of GPT-5.6. Sol handled titles, paragraphs, tables, images, and signatures well. Many document workflows start with locating the relevant parts of a page before OCR or data extraction begins.

文档布局检测是 GPT-5.6 最显著的优势之一。Sol 能很好地处理标题、段落、表格、图像和签名。许多文档工作流在开始 OCR 或数据提取之前，都需要先定位页面中的相关部分。

GPT-5.6 also performed well on dense scenes. The pills and eggs examples contain many similar objects packed closely together, a common weakness for VLM-based detection. Unlike traditional detectors, VLMs generate each class label and set of coordinates as text. As object count grows, the response becomes longer and the risk of missed objects, duplicates, or coordinate errors increases. Despite this, Sol detected most objects across both scenes.

GPT-5.6 在密集场景中也表现出色。药丸和鸡蛋的示例中包含许多紧密排列的相似物体，这是基于 VLM 的检测常见的弱点。与传统检测器不同，VLM 将每个类别标签和坐标集作为文本生成。随着物体数量的增加，响应变得更长，漏检、重复或坐标错误的风险也会增加。尽管如此，Sol 还是检测到了两个场景中的大部分物体。

For the best detection results, prompt GPT-5.6 models to return absolute XYXY coordinates in image pixels. This differs from Gemini 3.5 Flash, which performed best with YXYX coordinates normalized to a 0–1000 range. Using the wrong coordinate format reduced GPT-5.6 detection performance by around 15 mAP points in our benchmark.

为了获得最佳检测结果，请提示 GPT-5.6 模型返回图像像素中的绝对 XYXY 坐标。这与 Gemini 3.5 Flash 不同，后者在归一化到 0–1000 范围的 YXYX 坐标下表现最佳。在我们的基准测试中，使用错误的坐标格式会导致 GPT-5.6 的检测性能下降约 15 个 mAP 点。

In a few cases, GPT-5.6 Sol returned boxes in seemingly random parts of the image. Many had no overlap, or almost no overlap, with the ground truth. Instead of matching the visible objects, the boxes often formed unnatural layouts, such as straight rows or evenly spaced groups.

在少数情况下，GPT-5.6 Sol 会在图像中看似随机的位置返回边框。许多边框与真实标注（ground truth）没有重叠或几乎没有重叠。这些边框往往没有匹配可见物体，而是形成了不自然的布局，例如直线排列或均匀分布的组。

We shared those examples with OpenAI. Their team confirmed that Sol becomes less stable on images around 2,000 by 2,000 pixels or larger, especially at lower reasoning effort. Higher reasoning effort improves stability, but also increases token use, latency, and cost. Resizing or cropping large images before sending them to the OpenAI API is the most practical workaround.

我们将这些示例分享给了 OpenAI。他们的团队确认，Sol 在 2000x2000 像素或更大的图像上稳定性会下降，尤其是在较低的推理强度下。提高推理强度可以改善稳定性，但也会增加 Token 使用量、延迟和成本。在将大图发送到 OpenAI API 之前进行缩放或裁剪是最实用的解决方法。

Object Counting

目标计数

Counting improved across the full GPT-5.6 lineup. Sol scored 73.0% in our benchmark, up from 64.9% for GPT-5.5, while Terra and Luna reached 67.6% and 66.2%. Luna, the cheapest model in the lineup, still outperformed the previous OpenAI baseline.

整个 GPT-5.6 系列的计数能力都有所提升。Sol 在我们的基准测试中得分为 73.0%，高于 GPT-5.5 的 64.9%，而 Terra 和 Luna 分别达到了 67.6% 和 66.2%。作为系列中最便宜的模型，Luna 的表现依然超过了 OpenAI 此前的基准水平。

As part of the benchmark, we tested cases requiring more than spotting objects and returning a total. Sol counted heavily overlapping metal brackets, a difficult case for both traditional object detectors and VLMs. Sol also counted bullet holes only inside selected scoring zones, showing an understanding of both which objects to count and where the rule applied.

作为基准测试的一部分，我们测试了那些不仅需要识别物体并返回总数的情况。Sol 统计了严重重叠的金属支架，这对传统目标检测器和 VLM 来说都是一个难题。Sol 还统计了仅在选定计分区域内的弹孔，这表明它既理解要统计哪些物体，也理解规则适用的范围。

Blister packs proved much harder. In separate prompts, we asked Sol to count the empty slots and the pills still sealed inside the package. The repeated layout, reflections, and small visual differences between filled and empty slots made both tasks difficult.

泡罩包装（Blister packs）的测试难度要大得多。在不同的提示中，我们要求 Sol 统计空槽和包装内仍密封的药丸。重复的布局、反光以及已填充槽位与空槽位之间细微的视觉差异，使得这两项任务都变得非常困难。

The abnormal candy example exposed a different type of failure. Sol gave the wrong count, though it is unclear whether the model miscounted the candies or misunderstood the target category.

异常糖果的示例暴露了另一种类型的失败。Sol 给出了错误的计数，尽管尚不清楚模型是数错了糖果，还是误解了目标类别。

OCR and Data Extraction

OCR 与数据提取

OCR performance stayed close to GPT-5.5. Sol achieved a 90.7% mean similarity score, only 0.5 points behind GPT-5.5 at 91.2%, while Terra and Luna reached 88.8% and 88.4%. The gap was larger in text extraction, where Sol scored 82.5% compared with 87.6% for GPT-5.5. Luna and Terra followed at 81.4% and 79.4%.

OCR 性能与 GPT-5.5 保持接近。Sol 的平均相似度得分为 90.7%，仅比 GPT-5.5 的 91.2% 低 0.5 个百分点，而 Terra 和 Luna 分别达到 88.8% 和 88.4%。在文本提取方面差距较大，Sol 得分为 82.5%，而 GPT-5.5 为 87.6%。Luna 和 Terra 分别为 81.4% 和 79.4%。

As part of the benchmark, we separated full transcription from targeted extraction. OCR asks the model to transcribe all visible text, while text extraction asks for a specific piece of information. Sol performed well on handwritten notes in both settings, producing a full transcription in one case and extracting a requested date in another.

作为基准测试的一部分，我们将全文转录与定向提取区分开来。OCR 要求模型转录所有可见文本，而文本提取则要求获取特定信息。Sol 在这两种设置下对手写笔记的处理表现良好，在一种情况下完成了全文转录，在另一种情况下提取了所需的日期。

Sol performed well on text embedded in complex visual scenes. It read a tire size sequence printed along the curved surface of a dirty, worn tire. In another example, it extracted the live score from a hockey broadcast and returned the answer in the requested format, testing both visual reading and instruction following.

Sol 在处理嵌入复杂视觉场景的文本时表现出色。它读取了印在脏污、磨损轮胎曲面上的轮胎尺寸序列。在另一个示例中，它从冰球比赛直播中提取了实时比分，并以要求的格式返回答案，这同时测试了视觉阅读和指令遵循能力。

Some simple-looking extraction tasks still failed. Sol could not read the expiration date printed on a blister pack. The text was small, vertical, low contrast, and affected by reflections, which may explain the error.

一些看起来简单的提取任务仍然失败了。Sol 无法读取印在泡罩包装上的过期日期。文本很小、垂直排列、对比度低且受到反光影响，这或许可以解释出错的原因。

Trade-offs

权衡

The vision gains come with higher token usage across the GPT-5.6 lineup. The difference matters less in small tests, but becomes more important at scale, where token volume directly increases processing costs.

视觉能力的提升伴随着 GPT-5.6 系列更高的 Token 使用量。这种差异在小规模测试中影响不大，但在大规模应用中变得更为重要，因为 Token 总量会直接增加处理成本。

Sol averaged close to 10 seconds per image in our benchmark. Terra reduced that to around 6 seconds, while Luna finished in slightly over 5 seconds. Luna offers the strongest latency-quality balance in the lineup, with speed close to Gemini 3.5 Flash while still outperforming GPT-5.5 on detection and counting.

在我们的基准测试中，Sol 平均每张图片耗时接近 10 秒。Terra 将时间缩短至约 6 秒，而 Luna 则在 5 秒多一点完成。Luna 提供了该系列中延迟与质量的最佳平衡，其速度接近 Gemini 3.5 Flash，同时在检测和计数方面仍优于 GPT-5.5。