Measuring Engagement Without Conversions: How We Evaluate MVPs at Inithouse
Measuring Engagement Without Conversions: How We Evaluate MVPs at Inithouse
在没有转化的情况下衡量参与度：我们如何在 Inithouse 评估 MVP
When you’re running early-stage MVPs, the standard playbook breaks down fast. ROAS? Meaningless when you have 3 signups a week. Conversion rate? Statistically irrelevant on 40 sessions a day. We ran into this across our portfolio at Inithouse — 16+ products, most pre-PMF — and needed a measurement framework that actually works before the numbers get big. Here’s what we built: a weighted engagement scoring system that gives us a PMF signal from behavior, not transactions.
当你运营早期 MVP（最小可行性产品）时，标准的运营手册很快就会失效。ROAS（广告支出回报率）？每周只有 3 个注册时，这个指标毫无意义。转化率？每天 40 次会话时，统计学上根本不具备参考价值。我们在 Inithouse 的产品组合中遇到了同样的问题——超过 16 款产品，大多数处于 PMF（产品市场契合）前阶段——我们需要一套在数据量变大之前就能真正发挥作用的衡量框架。于是我们构建了这套系统：一个加权参与度评分系统，通过用户行为而非交易行为，为我们提供 PMF 信号。
Why ROAS Doesn’t Work for MVPs
为什么 ROAS 不适用于 MVP
ROAS assumes a functioning funnel: ad spend in, revenue out, ratio tells the story. But at the MVP stage, the funnel barely exists. At Magical Song — our AI personalized song generator — we had weeks with solid traffic but zero purchases. Does that mean the product is dead? Not necessarily. Users were generating songs, sharing previews, coming back. The engagement was real; the monetization path just wasn’t optimized yet. Optimizing for ROAS at this stage is premature optimization. You’d kill promising products and keep ones that happen to convert on a tiny, non-representative sample.
ROAS 假设存在一个运作良好的漏斗：投入广告费，产出收入，比率说明一切。但在 MVP 阶段，漏斗几乎不存在。以我们的 AI 个性化歌曲生成器 Magical Song 为例，我们曾有几周流量很稳，但购买量为零。这是否意味着产品“死”了？不一定。用户在生成歌曲、分享预览并不断回访。参与度是真实的，只是变现路径尚未优化。在这个阶段优化 ROAS 是过早优化。你可能会扼杀掉有潜力的产品，却保留那些仅在极小、无代表性的样本中产生转化的产品。
Engagement Events with Weighted Values
带有加权值的参与事件
Instead of tracking a single conversion event, we define 5-8 engagement events per product and assign each a fractional value that sums to 1.0. The idea: distribute “one unit of PMF signal” across the user journey. For Verdict Buddy — our AI decision-making tool — the setup looks roughly like this:
我们不再追踪单一的转化事件，而是为每个产品定义 5-8 个参与事件，并为每个事件分配一个总和为 1.0 的分数值。核心思路是：将“一个单位的 PMF 信号”分配到整个用户旅程中。以我们的 AI 决策工具 Verdict Buddy 为例，设置大致如下：
| Event (事件) | Weight (权重) | What It Signals (信号含义) |
|---|---|---|
| Page load (>10s) | 0.05 | Basic interest (基本兴趣) |
| First question entered | 0.20 | Active engagement (主动参与) |
| Analysis generated | 0.25 | Core value delivered (核心价值交付) |
| Second question (same session) | 0.25 | Retention signal (留存信号) |
| Share or bookmark | 0.15 | Word-of-mouth potential (口碑潜力) |
| Return visit (7d) | 0.10 | Sticky behavior (粘性行为) |
The weights are opinionated — and that’s the point. They encode what we think PMF looks like for this specific product, before we have conversion data to prove it. At Party Challenges, the weights look completely different. There, “challenge completed” and “shared with friends” carry most of the score, because the product’s value is inherently social.
这些权重是主观设定的——这正是关键所在。在没有转化数据证明之前，它们编码了我们认为该特定产品的 PMF 应该是什么样子。在 Party Challenges 项目中，权重设置则完全不同。在那里，“挑战完成”和“分享给朋友”占据了大部分分数，因为该产品的价值本质上是社交性的。
Setting This Up in GA4
在 GA4 中进行设置
The implementation is straightforward:
- Define custom events for each engagement milestone. Use
gtag('event', 'engagement_milestone', { milestone_name: 'analysis_generated', weight: 0.25 }). - Create a calculated metric in GA4 called “Engagement Score” that sums the weighted values per session. Under Admin → Custom definitions → Calculated metrics, define it as the sum of weight parameter values.
- Build an Exploration report with Session as the dimension and Engagement Score as the metric. Sort descending — high-scoring sessions are your power users.
- Set up a segment for sessions with score > 0.5 (we call these “meaningful sessions”). Track what percentage of total sessions are meaningful — that’s your engagement rate. The beauty: this runs on GA4’s free tier. No BigQuery export needed, no custom infrastructure.
实现过程很简单：
- 为每个参与里程碑定义自定义事件。使用
gtag('event', 'engagement_milestone', { milestone_name: 'analysis_generated', weight: 0.25 })。 - 在 GA4 中创建一个名为“参与度评分 (Engagement Score)”的计算指标，将每次会话的加权值相加。在“管理”→“自定义定义”→“计算指标”中，将其定义为权重参数值的总和。
- 构建一个“探索”报告，以“会话”为维度，以“参与度评分”为指标。按降序排列——高分会话即为你的核心用户。
- 为评分 > 0.5 的会话设置一个细分（我们称之为“有意义的会话”）。追踪总会话中有多少比例是“有意义的”——这就是你的参与率。其妙处在于：这可以在 GA4 的免费版上运行。无需 BigQuery 导出，也无需自定义基础设施。
Reading the Numbers
解读数据
After running this across our portfolio for several months, here’s what we’ve learned to look for:
- Engagement score per session is the headline metric. If this trends up week-over-week without changes to the product, you’re seeing organic pull. If it’s flat or declining despite traffic growth, the product is leaking interest — people show up but don’t engage deeply.
- Meaningful session rate (sessions scoring > 0.5) tells you what fraction of visitors actually experience the product’s core value. At Živá Fotka — our AI photo animation tool — this metric jumped from 31% to 58% after we simplified the upload flow. That was a stronger signal than any conversion metric could have given us at that traffic level.
- Distribution matters. If your engagement scores cluster at 0.05 (page load only) and 0.85+ (power users) with nothing in between, you have a discovery problem — people who find the value love it, but most visitors bounce before they get there. A healthy distribution shows a gradual curve, meaning users are progressing through your engagement milestones.
在我们的产品组合中运行几个月后，我们总结出以下观察重点：
- 单次会话参与度评分是核心指标。如果产品没有变动但该指标周环比上升，说明你获得了自然增长动力。如果流量增长但该指标持平或下降，说明产品正在流失兴趣——用户来了，但没有深度参与。
- 有意义的会话率（评分 > 0.5 的会话）告诉你到底有多少比例的访问者真正体验到了产品的核心价值。在我们的 AI 照片动画工具 Živá Fotka 上，简化上传流程后，该指标从 31% 跃升至 58%。在当时的流量水平下，这比任何转化指标提供的信号都要强烈。
- 分布很重要。 如果你的参与度评分集中在 0.05（仅页面加载）和 0.85+（核心用户），而中间没有过渡，说明你存在发现问题——找到价值的人很喜欢它，但大多数访问者在体验到价值前就离开了。健康的分布应该呈现平缓的曲线，意味着用户正在逐步完成你的参与里程碑。
When to Switch to Conversion Optimization
何时转向转化优化
This framework is a bridge, not a destination. You graduate from engagement scoring to conversion optimization when:
- Meaningful session rate exceeds 40% — enough users experience core value that conversion becomes a real lever.
- Weekly active sessions exceed ~200 — the numbers are statistically meaningful for A/B tests.
- The engagement score distribution stabilizes — you’ve squeezed out the easy UX wins and the funnel shape is consistent.
At that point, add a real conversion event (purchase, signup, subscription), set it as your GA4 key event, and start optimizing. The engagement score doesn’t disappear — it becomes a diagnostic tool to understand why conversions go up or down.
这套框架是一座桥梁，而非终点。当你满足以下条件时，就可以从参与度评分转向转化优化：
- 有意义的会话率超过 40% —— 足够多的用户体验到了核心价值，转化才成为一个真正的杠杆。
- 每周活跃会话超过约 200 次 —— 数据在统计学上对 A/B 测试才有意义。
- 参与度评分分布趋于稳定 —— 你已经榨干了简单的 UX 优化红利，且漏斗形态保持一致。
此时，添加一个真实的转化事件（购买、注册、订阅），将其设为 GA4 的关键事件，并开始优化。参与度评分并不会消失——它将成为诊断工具，帮助你理解转化率波动的原因。
The Framework in Practice
框架实践
We’ve been running this system across all products in the Inithouse portfolio. It’s not perfect — the weight assignments are subjective, and you’ll re-calibrate them as you learn more about your users. But it beats the alternative: staring at a conversion rate of 0% and wondering whether your product has potential. The core principle: in early MVP stages, measure depth of engagement, not completion of transactions. Transactions will come if the engagement is real. If it’s not, no amount of funnel optimization will save you.
我们已经在 Inithouse 的所有产品中运行了这套系统。它并不完美——权重分配是主观的，随着对用户了解的加深，你需要重新校准它们。但它总比盯着 0% 的转化率、怀疑产品是否有潜力要好得多。核心原则是：在 MVP 早期阶段，衡量参与深度，而非交易完成情况。如果参与度是真实的，交易自然会到来。如果不是，任何漏斗优化都救不了你。