Backpropagation Explained for Beginners (Part 1): Building the Intuition

反向传播初学者指南（第一部分）：构建直觉

Deep Learning Backpropagation Explained for Beginners (Part 1): Building the Intuition. Let’s discover how neural networks learn, step by step. 深度学习反向传播初学者指南（第一部分）：构建直觉。让我们一步步探索神经网络是如何学习的。

Have you ever struggled to understand backpropagation? If you’re trying to understand how modern AI systems like large language models (LLMs) are trained, backpropagation is one of the most important concepts to understand. But if you ask me how I felt when I encountered it, I was completely lost by looking at the math equations. It felt like a mental block for me. 你是否曾为理解反向传播而苦恼？如果你正试图理解像大语言模型（LLMs）这样的现代人工智能系统是如何训练的，反向传播是最重要的概念之一。但如果你问我初次接触它时的感受，我当时看着那些数学公式完全是一头雾水，感觉像撞上了一堵思维之墙。

I realized and wanted to start from scratch and build my understanding one step at a time. That journey began with my previous article, where we built a neural network from scratch using a simple dataset and understood how it makes predictions. The blog received a great response. Thank you for that! Now, let’s continue with the same approach. We will break down backpropagation step-by-step, keeping it as simple and intuitive as before. 我意识到我需要从零开始，一步一个脚印地建立理解。这段旅程始于我的上一篇文章，我们在文中利用一个简单的数据集从头构建了一个神经网络，并理解了它是如何进行预测的。那篇博客收到了很好的反响，非常感谢大家！现在，让我们沿用同样的方法。我们将把反向传播拆解开来，像之前一样保持简单直观。

Before we begin, I just want to say one thing. We’ll take this one step at a time. Topics like backpropagation can feel overwhelming at first, but once we build a strong foundation, everything else becomes much easier to understand. So, let’s get started. 在开始之前，我想说一件事：我们将循序渐进。像反向传播这样的主题起初可能会让人感到不知所措，但一旦我们打下了坚实的基础，其他一切都会变得容易理解得多。那么，让我们开始吧。

Welcome back! Let’s continue our learning journey through deep learning. We already have a basic understanding of neural networks, which we explored using a simple dataset in the previous blog. Now, let’s first recall what we learned in the previous blog on neural networks. 欢迎回来！让我们继续深度学习的学习之旅。我们已经对神经网络有了基本的了解，这是我们在上一篇博客中通过一个简单的数据集探索出来的。现在，让我们先回顾一下上一篇关于神经网络的博客中学到的内容。

Quick Recap

快速回顾

We considered this simple dataset. We observed that a single line was not enough to fit it. So, we decided to solve it using neural networks. Next, we got to know about the equation of a single neuron, and after that, we learned about the different layers in a neural network. For simplicity, we considered one hidden layer with two hidden neurons. 我们考虑了这个简单的数据集。我们观察到单条直线不足以拟合它，因此我们决定使用神经网络来解决。接着，我们了解了单个神经元的方程，随后学习了神经网络中的不同层。为了简化，我们考虑了一个包含两个隐藏神经元的隐藏层。

Next, we observed how the two hidden neurons produced two different linear transformations, and then we wanted to combine them in the output layer. However, we found out that combining two lines produced another line, not the curve that could fit the data. This is where we realized the significance of activation functions, as they introduce non-linearity into the model. 接下来，我们观察了这两个隐藏神经元如何产生两种不同的线性变换，然后我们试图在输出层将它们结合起来。然而，我们发现结合两条直线只会产生另一条直线，而不是能够拟合数据的曲线。这时我们意识到了激活函数的重要性，因为它们为模型引入了非线性。

So, we passed the outputs from the hidden neurons through the activation function (ReLU) and then combined them in the output layer. In other words, we took the linear combination of the outputs from the activation function in the output layer, and finally, we obtained the curve. 因此，我们将隐藏神经元的输出通过激活函数（ReLU），然后在输出层进行组合。换句话说，我们对激活函数的输出进行了线性组合，最终得到了这条曲线。

In the previous blog we built the neural network architecture and saw how it makes predictions through forward propagation. Before we learn how backpropagation works, let’s first look at the values produced at each layer during the forward pass which we discussed in earlier blog. We’ll use these values throughout the blog to understand how the network learns by updating its parameters. 在上一篇博客中，我们构建了神经网络架构，并了解了它如何通过前向传播进行预测。在学习反向传播的工作原理之前，让我们先看看在前向传播过程中每一层产生的值，这些我们在之前的博客中已经讨论过。我们将在整篇博客中使用这些值，以理解网络如何通过更新参数来学习。

Why Does the Network Need to Learn?

为什么网络需要学习？

When we look at the final curve produced by our neural network, we can see that it is not a good fit. For example, when the hours studied (x) is 1, the actual exam score is 55, but our neural network predicts it as 28, which is a huge difference. Now, we need to make our neural network perform better, which means it should predict values that are much closer to the actual exam scores. To do that, the neural network needs to learn. By learning, we mean figuring out which parameters should be increased and which should be decreased to reduce the loss. 当我们观察神经网络生成的最终曲线时，可以看到它拟合得并不好。例如，当学习时长 (x) 为 1 时，实际考试成绩为 55，但我们的神经网络预测为 28，差距巨大。现在，我们需要让神经网络表现得更好，这意味着它预测的值应该更接近实际考试成绩。为了做到这一点，神经网络需要学习。所谓的学习，就是找出哪些参数应该增加，哪些应该减少，从而降低损失。

Learning from a Familiar Example

从熟悉的例子中学习

Now, how can we do this? At this point, we don’t know how to do that. Let’s do one thing. Let’s proceed with what we already know. But what do we already know? We already have an idea about simple linear regression, how the loss is calculated, and how the bowl curve looks. Maybe we can learn something from it. 那么，我们该怎么做呢？目前我们还不知道。让我们这样做：从我们已知的内容入手。但我们已经知道了什么？我们已经了解了简单线性回归、损失是如何计算的，以及碗状曲线（损失函数曲线）的样子。也许我们可以从中得到一些启发。

In simple linear regression, we need to find the optimal values for β0 (intercept) and β1 (slope). Of course, we already have formulas, but we also derived them ourselves. What we did was plot a graph with three axes. One axis represented (β0), the second represented (β1), and the third represented the loss. We plotted the loss values for different (β0) and (β1) values and observed a bowl-shaped curve. We then understood that the minimum loss occurs at the bottom of the curve, where the slope of the loss surface becomes zero. 在简单线性回归中，我们需要找到 β0（截距）和 β1（斜率）的最优值。当然，我们已经有了公式，而且我们也亲自推导过它们。我们所做的是绘制一个三轴图：一个轴代表 β0，第二个轴代表 β1，第三个轴代表损失。我们绘制了不同 β0 和 β1 值下的损失值，并观察到了一个碗状曲线。我们随后理解到，最小损失出现在曲线的底部，即损失曲面的斜率为零的地方。

To find that point, we used partial differentiation and eventually solved the resulting equations to obtain the formulas. In simple linear regression, we can use different loss functions such as the Sum of Squared Errors (SSE), Mean Squared Error (MSE), or other suitable loss functions depending on the problem. Here, we’ll consider the Mean Squared Error (MSE) as our loss function. 为了找到那个点，我们使用了偏微分，并最终通过求解所得方程得到了公式。在简单线性回归中，我们可以根据问题使用不同的损失函数，例如误差平方和 (SSE)、均方误差 (MSE) 或其他合适的损失函数。在这里，我们将使用均方误差 (MSE) 作为我们的损失函数。

For simple linear regression, the loss function is: 对于简单线性回归，损失函数为： [ L(\beta_0,\beta_1)=\frac{1}{n}\sum_{i=1}^{n}\left(y_i-\hat{y}_i\right)^2 ] where [\hat{y}_i=\beta_0+\beta_1x_i.] 其中 [\hat{y}_i=\beta_0+\beta_1x_i.]

Notice that the loss depends only on two parameters, [\beta_0] and [\beta_1]. We have to find the values of [\beta_0] and [\beta_1] that minimize this loss. Now, let’s look at our neural network. Since our current problem is a non-linear regression problem, we can continue using the same Mean Squared Error (MSE). The loss function can now be written as: 注意，损失仅取决于两个参数：[\beta_0] 和 [\beta_1]。我们必须找到使该损失最小化的 [\beta_0] 和 [\beta_1] 值。现在，让我们看看我们的神经网络。由于我们当前的问题是一个非线性回归问题，我们可以继续使用相同的均方误差 (MSE)。损失函数现在可以写成： [ L(w_1,w_2,w_3,w_4,b_1,b_2,b_3) = \frac{1}{n} \sum_{i=1}^{n} \left(y_i-\hat{y}_i\right)^2. ]

However, unlike simple linear regression, our prediction is no longer given by [\hat{y}=\beta_0+\beta_1x.] Instead, it is produced by the entire neural network. For our neural network: 然而，与简单线性回归不同，我们的预测不再由 [\hat{y}=\beta_0+\beta_1x] 给出，而是由整个神经网络产生。对于我们的神经网络： [ \hat{y}_i = w_3,\mathrm{ReLU}(w_1x_i+b_1) + w_4,\mathrm{ReLU}(w_2x_i+b_2) + b_3. ]

As a result, the loss no longer depends on just two parameters. It now depends on all seven parameters of the neural network, which are [w_1, w_2, w_3, w_4, b_1, b_2, b_3.] 因此，损失不再仅仅取决于两个参数，而是取决于神经网络的所有七个参数，即 [w_1, w_2, w_3, w_4, b_1, b_2, b_3.]