How to migrate an online store’s product catalog in minutes (not weeks)

如何在几分钟（而非几周）内迁移在线商店的产品目录

Migrating a store to a new platform is mostly one boring, expensive problem: getting the product catalog out of the old site. The checkout, the theme, the domain — those are afternoons. The catalog is weeks. Here is how to do it in minutes instead, and why the three obvious approaches usually don’t get you there. 将商店迁移到新平台，主要面临一个枯燥且昂贵的难题：如何将产品目录从旧网站中导出。结账流程、主题、域名——这些只需一个下午就能搞定，但产品目录却需要几周时间。以下是如何在几分钟内完成迁移的方法，以及为什么通常的三种常规方案往往行不通。

Why catalog migration eats weeks

为什么目录迁移会耗费数周时间

1. Retyping by hand 1. 手动重新录入 The default. Someone opens the old store in one tab and the new admin in another, and copies 600 products across. At a realistic 3-4 minutes per product — name, description, price, SKU, category, download the photo, re-upload it — that’s about 35 hours of work. It is also the version that silently loses data: whoever gets tired at product 400 starts skipping the specs table. 这是最常见的方式。某人打开一个标签页查看旧商店，在另一个标签页打开新后台，然后将 600 个产品逐一复制过去。按每个产品 3-4 分钟的实际耗时计算（包括名称、描述、价格、SKU、分类、下载图片、重新上传），这大约需要 35 个小时的工作量。这也是最容易悄无声息丢失数据的版本：当处理到第 400 个产品时，操作者感到疲惫，就会开始跳过规格表。

2. The CSV export that “already exists” 2. “现成”的 CSV 导出 Every platform has an export button, so this feels solved. It isn’t: The export gives you their column names, not the new platform’s. Photos come as URLs pointing at the old CDN, which dies with the old plan. Variants, bundles and specs get flattened into one unparseable text blob. Legacy or custom-built stores (a huge share of small B2B sites) have no export at all. You end up writing a transform script anyway — and you can only start once you have someone’s admin credentials, which for an agency migrating a client is often the long pole. 每个平台都有导出按钮，所以这看起来是个已解决的问题。其实不然：导出的文件使用的是旧平台的列名，而非新平台的。图片链接指向的是旧的 CDN，一旦旧套餐过期，这些链接就会失效。变体、捆绑包和规格参数会被压缩成一团无法解析的文本。遗留系统或定制开发的商店（占小型 B2B 网站的很大一部分）根本没有导出功能。最终你还是得编写转换脚本——而且只有在拿到对方的管理员凭据后才能开始，对于代理商来说，这往往是迁移过程中最耗时的环节。

3. A scraper with CSS selectors 3. 基于 CSS 选择器的爬虫 The engineer’s answer, and it works — for exactly one site. You open DevTools, find that the price lives in .product-info__price > span, write the selector, and it holds until the store’s theme updates. Multiply by every client you migrate and you are maintaining a per-site template library. Selector-based scrapers don’t generalize; that’s their whole problem. 这是工程师的方案，它确实有效——但仅限于单个网站。你打开开发者工具，找到价格所在的 .product-info__price > span，写好选择器，它在商店主题更新前都能用。如果你要为每个客户迁移，你就得维护一个针对每个站点的模板库。基于选择器的爬虫无法通用，这就是它们的核心问题。

The AI approach: no selectors at all

AI 方案：完全无需选择器

Instead of teaching a script where the price is, you let a model read the page and tell you what’s on it. The crawler is generic — follow links, find category pages, follow pagination, collect candidate product URLs — and the extraction is a prompt. The non-obvious part is not extraction. It’s rejection. E-commerce pages lie: a category page carries a name, a price and a photo, exactly like a product page. So the single most important instruction in the prompt is that the model must answer null when the page is a listing, not a product. Teaching the AI to say “this isn’t a product” mattered more than teaching it to extract. 与其教脚本去哪里找价格，不如让模型直接阅读页面并告诉你上面有什么。爬虫是通用的——跟踪链接、查找分类页面、跟踪分页、收集候选产品 URL——而提取过程则通过提示词（Prompt）完成。最关键的部分不在于提取，而在于“拒绝”。电商页面会“撒谎”：分类页面也包含名称、价格和图片，看起来和产品页面一模一样。因此，提示词中最重要的一条指令是：当页面是列表页而非产品页时，模型必须返回 null。教 AI 说“这不是产品”比教它如何提取数据更重要。

Step by step, with curl

使用 curl 分步操作

I packaged the engine as an API. Subscribe on RapidAPI (free tier, no card), grab your key, and it’s two calls: submit a job, poll it. 我将该引擎封装成了 API。在 RapidAPI 上订阅（有免费层级，无需信用卡），获取你的密钥，只需两次调用：提交任务，然后轮询结果。

1. Submit the store URL 1. 提交商店 URL

curl -X POST 'https://ai-product-catalog-extractor.p.rapidapi.com/v1/extract' \
 -H 'x-rapidapi-key: YOUR_KEY' \
 -H 'x-rapidapi-host: ai-product-catalog-extractor.p.rapidapi.com' \
 -H 'content-type: application/json' \
 -d '{ "url": "https://the-old-store.com/", "limites": { "max_paginas": 200 } }'

You get a 202 back: 你会收到 202 响应：

{ "job_id": "9f2c...", "poll": "/v1/jobs/9f2c..." }

2. Poll until it’s done 2. 轮询直到完成

curl 'https://ai-product-catalog-extractor.p.rapidapi.com/v1/jobs/9f2c...' \
 -H 'x-rapidapi-key: YOUR_KEY' \
 -H 'x-rapidapi-host: ai-product-catalog-extractor.p.rapidapi.com'

While it runs you get live progress — fase (phase), paginas_visitadas, itens_extraidos — so you can render a progress bar instead of a spinner. 在运行过程中，你会获得实时进度——fase（阶段）、paginas_visitadas（已访问页面）、itens_extraidos（已提取项目）——这样你就可以渲染进度条，而不是只显示一个加载转圈。

3. Read the catalog 3. 读取目录

{
  "status": "done",
  "result": {
    "itens": [
      {
        "nome": "Rebite Repuxo Alumínio 4,8 x 12mm",
        "marca": "CRV",
        "categoria": "Rebites",
        "sku": "RA-4812",
        "preco": 42.9,
        "descricao": "Rebite de repuxo em alumínio com haste de aço...",
        "foto": "https://old-store.com/img/ra-4812.jpg",
        "_url": "https://old-store.com/produtos/rebite-repuxo-4812"
      }
    ],
    "total_candidatos": 126,
    "truncado": false,
    "usage": { "tokens_entrada": 113559, "chamadas_ia": 60 }
  }
}

About those Portuguese field names

关于那些葡萄牙语字段名

The engine was born inside a Brazilian ERP, so the default preset ships nome, preco, foto, descricao. That’s cosmetic, not structural — you can send your own schema and get whatever keys your target platform wants: 该引擎诞生于一个巴西 ERP 系统中，因此默认预设输出的是 nome（名称）、preco（价格）、foto（图片）、descricao（描述）。这只是表面现象，而非结构限制——你可以发送自己的 schema，获取目标平台所需的任何键名：

"schema": [
  { "key": "title", "tipo": "texto", "label": "product name" },
  { "key": "price", "tipo": "number" },
  { "key": "image", "tipo": "foto" }
]

Same engine, keys ready to POST straight into Shopify or WooCommerce. 同样的引擎，键名已准备好直接 POST 到 Shopify 或 WooCommerce。

Real numbers

真实数据

A cold run against a Brazilian industrial-tools store, no configuration, no selectors: 160 pages crawled → 126 candidates → 50 clean products in 72 seconds. Zero duplicates, zero category pages leaking into the results, ~113k input tokens (a few cents of model cost). Compare that with 35 hours of retyping. 对一家巴西工业工具商店进行了一次冷测试，无需配置，无需选择器：爬取 160 个页面 → 126 个候选 → 72 秒内提取 50 个干净的产品数据。零重复，零分类页面混入结果，约 113k 输入 Token（模型成本仅几美分）。对比一下那 35 小时的手动录入吧。

Honest limitations

诚实的局限性

• SPA storefronts don’t work yet. If the products only exist after JavaScript runs (VTEX, headless Shopify, client-side Next.js), you’ll get zero. Static HTML only, for now.
• SPA 商店暂不支持。 如果产品信息仅在 JavaScript 运行后才显示（如 VTEX、无头 Shopify、客户端渲染的 Next.js），你将一无所获。目前仅支持静态 HTML。
• One page = one product is assumed. Stores where SKUs exist only inside a filterable listing, with no individual URL, return nothing.
• 假设一页对应一个产品。 如果 SKU 仅存在于可过滤的列表页中，且没有独立 URL 的商店，则无法返回任何内容。
• robots.txt is respected by default. There’s an owner-authorization flag for when it’s your own store.
• 默认遵守 robots.txt。 如果是你自己的商店，可以使用所有者授权标志。
• Crawling is rate-limited per host on purpose. So a large catalog takes minutes, not seconds. Pointing at the listing page instead of the homepage speeds things up a lot.
• 爬取速度有意针对每个主机进行了限速。 因此大型目录需要几分钟而非几秒钟。直接指向列表页而非首页可以大幅提高速度。

Try it on your own migration: AI Product Catalog Extractor on RapidAPI (free tier, 50 calls/month). 尝试你的迁移任务：RapidAPI 上的 AI 产品目录提取器（免费层级，每月 50 次调用）。