CBAM Paper Walkthrough: The Double-Attention Mechanism
CBAM Paper Walkthrough: The Double-Attention Mechanism
CBAM 论文解读：双重注意力机制
Introduction
In this article, I am going to review and implement the deep learning paper titled “CBAM: Convolutional Block Attention Module” by Woo et al. [1]. As the name suggests, this is essentially a block we can attach to a CNN-based model to enhance feature quality by performing an attention mechanism. Despite the name attention, it is completely different from the one in the ViT (Vision Transformer) architecture. Keep in mind that CBAM was first released in 2018, while ViT was only introduced in 2020. So, we can simply say that CBAM is the older approach to apply an attention mechanism to image data. Despite being older, we should not worry about its relevance since CBAM is a lot more lightweight as compared to ViT, which makes it suitable to be used as a starting point for deployment on low-power devices.
引言
在本文中，我将回顾并实现由 Woo 等人 [1] 撰写的深度学习论文《CBAM: Convolutional Block Attention Module》（卷积块注意力模块）。顾名思义，这本质上是一个可以附加到基于 CNN 的模型上的模块，通过执行注意力机制来增强特征质量。尽管名字里带有“注意力”，但它与 ViT（视觉 Transformer）架构中的注意力机制完全不同。请记住，CBAM 于 2018 年首次发布，而 ViT 直到 2020 年才被引入。因此，我们可以简单地说，CBAM 是一种较早的将注意力机制应用于图像数据的方法。尽管它出现得更早，但我们不必担心它的相关性，因为与 ViT 相比，CBAM 轻量得多，这使其非常适合作为在低功耗设备上部署的起点。
Better Than SENet
If we were to talk about the history, CBAM was actually proposed as the improvement of SENet (Squeeze-and-Excitation Network), which was introduced a year before CBAM. If you remember the SENet architecture, it essentially works by performing attention across the channel dimension. By doing so, the channels that seem unimportant would be weighted less than that of the more important ones. — I actually got a separate article talking more thoroughly about SENet, which you can access through the link given in reference number [2]. Instead of just assigning weights to each channel, CBAM also gives weights to every single pixel in the spatial dimension of the image. So, we can say that CBAM has two attention mechanisms, which the authors refer to as the CAM (Channel Attention Module) and the SAM (Spatial Attention Module). So, based solely on this theory, CBAM should perform better than SENet.
优于 SENet
谈到历史，CBAM 实际上是作为 SENet（挤压与激励网络）的改进版提出的，而 SENet 在 CBAM 问世前一年被引入。如果你还记得 SENet 的架构，它本质上是通过在通道维度上执行注意力机制来工作的。通过这种方式，看起来不重要的通道权重会低于更重要的通道。——我实际上有另一篇文章更详尽地讨论了 SENet，你可以通过参考资料 [2] 中给出的链接访问它。CBAM 不仅为每个通道分配权重，还为图像空间维度中的每一个像素分配权重。因此，我们可以说 CBAM 拥有两种注意力机制，作者将其称为 CAM（通道注意力模块）和 SAM（空间注意力模块）。因此，仅从理论上讲，CBAM 的表现应该优于 SENet。
CBAM Architecture
Let’s talk more specifically about the CBAM architecture which I display in Figure 1 below. As I’ve mentioned earlier, CBAM consists of CAM and SAM. These two sub-blocks are responsible for creating attention weights, which will then be applied to the original tensor by multiplication. The output tensor of this block (the one referred to as Refined Features) has the exact same dimension as the input (Input Feature), meaning that we can easily plug CBAM to any backbone model without needing to worry about altering the tensor shapes.
CBAM 架构
让我们更具体地谈谈我在下方图 1 中展示的 CBAM 架构。正如我之前提到的，CBAM 由 CAM 和 SAM 组成。这两个子模块负责创建注意力权重，然后通过乘法将其应用于原始张量。该模块的输出张量（即所谓的“精炼特征”）与输入（输入特征）具有完全相同的维度，这意味着我们可以轻松地将 CBAM 插入到任何骨干模型中，而无需担心改变张量形状。
(Figure 1. The high-level view of the CBAM architecture [1]. / 图 1. CBAM 架构的高层视图 [1])
Channel Attention Module (CAM)
Now let’s take a closer look at the channel attention module in Figure 2 below. This component is actually very similar to the SENet block, except that CAM also uses global maxpooling layer in addition to the global average-pooling layer. It is explained in the paper that the two operations capture different kind of information, allowing the tensor produced by CAM to be more informative as compared to that of the SENet block.
通道注意力模块 (CAM)
现在让我们仔细看看下方图 2 中的通道注意力模块。该组件实际上与 SENet 模块非常相似，不同之处在于 CAM 除了全局平均池化层外，还使用了全局最大池化层。论文中解释说，这两种操作捕获了不同类型的信息，使得 CAM 产生的张量比 SENet 模块产生的张量包含更多信息。
(Figure 2. The structure of the channel attention module [1]. / 图 2. 通道注意力模块的结构 [1])
Remember that the spatial dimension of the tensor collapses to 1×1 when we apply global pooling operation to it. This essentially means that the input tensor, which has the original shape of C×H×W, now becomes C×1×1, allowing us to process it further easily with the MLP in the subsequent step. There are two linear layers in this MLP, where the first one is used to shrink the number of features according to the reduction ratio parameter, whereas the second one works by expanding the feature vector back to the original length (i.e., the number of channels C). These two linear layers in the MLP are together responsible to learn the importance of each channel. Also, keep in mind that this MLP is shared for the tensor produced by the maxpooling and the average-pooling operations, meaning that these two tensors will be processed by the exact same MLP.
请记住，当我们对张量应用全局池化操作时，其空间维度会坍缩为 1×1。这意味着原始形状为 C×H×W 的输入张量现在变成了 C×1×1，从而使我们能够在后续步骤中轻松地使用 MLP 对其进行进一步处理。该 MLP 中有两个线性层，第一个用于根据缩减比参数缩小特征数量，而第二个则将特征向量扩展回原始长度（即通道数 C）。MLP 中的这两个线性层共同负责学习每个通道的重要性。此外，请记住，此 MLP 对于最大池化和平均池化操作产生的张量是共享的，这意味着这两个张量将由同一个 MLP 处理。
As these two tensors have been processed, we then combine them by element-wise summation and pass it through a sigmoid function. Since this function shrinks any number to the range of 0 to 1, we can now perceive the resulting tensor as the channel attention weight. The elements that correspond to the more important channels will have the value close to 1, indicating that these channels will be weighted more than the others. According to the paper, this kind of mechanism helps the model to understand what kind of features to attend.
当这两个张量被处理后，我们通过逐元素相加将它们结合起来，并通过一个 Sigmoid 函数。由于该函数将任何数字压缩到 0 到 1 的范围内，我们现在可以将生成的张量视为通道注意力权重。对应于更重要通道的元素值将接近 1，表明这些通道的权重将高于其他通道。根据论文，这种机制有助于模型理解应该关注什么样的特征。
You can see the formal mathematical expression of the channel attention module in Figure 3 below, where F is an arbitrary intermediate tensor within a network. One thing you need to keep in mind here is that there should be a ReLU activation function placed between the two linear layers (i.e., W₀ and W₁) yet is somehow not written in this equation.
你可以在下方的图 3 中看到通道注意力模块的正式数学表达式，其中 F 是网络中的任意中间张量。这里需要记住的一点是，在两个线性层（即 W₀ 和 W₁）之间应该有一个 ReLU 激活函数，但不知何故它没有写在这个公式中。
(Figure 3. The formal mathematical expression of the channel attention module [1]. / 图 3. 通道注意力模块的正式数学表达式 [1])
Spatial Attention Module (SAM)
The spatial attention module is conceptually similar to the channel attention module. Take a look at the illustration of this sub-block in Figure 4 below.
空间注意力模块 (SAM)
空间注意力模块在概念上与通道注意力模块相似。请看下方图 4 中该子模块的示意图。
(Figure 4. The structure of the spatial attention module [1]. / 图 4. 空间注意力模块的结构 [1])
What essentially differentiates SAM from CAM is the axis where the pooling operation is performed. Previously in CAM the pooling is done across the spatial dimension, allowing each channel to have a single value representing that channel. Meanwhile, here in SAM the pooling is done across the channel dimension for each spatial pixel location. Thus, every pixel now contains a single value that represents all channels at once. By doing so, the input tensor that initially has the shape of C×H×W will collapse to 1×H×W. Remember that since we use maximum and average-pooling operations, we will thus have two tensors of that size. These two tensors are then concatenated, forming a new tensor of shape 2×H×W. This tensor is then processed with a 7×7 convolution layer of a single kernel, which effectively combines the information from the two channels into one. So, at this point the tensor becomes 1×H×W again and is then forwarded to the sigmoid function. Similar to CAM, the tensor produced by this sigmoid acts as the spatial attention weight.
SAM 与 CAM 的本质区别在于执行池化操作的轴。在之前的 CAM 中，池化是在空间维度上进行的，允许每个通道拥有一个代表该通道的单一值。而在 SAM 中，池化是针对每个空间像素位置在通道维度上进行的。因此，每个像素现在包含一个同时代表所有通道的单一值。通过这种方式，最初形状为 C×H×W 的输入张量将坍缩为 1×H×W。请记住，由于我们使用了最大池化和平均池化操作，因此我们将得到两个该尺寸的张量。这两个张量随后被拼接，形成一个形状为 2×H×W 的新张量。该张量随后由一个单核的 7×7 卷积层处理，有效地将来自两个通道的信息合并为一个。因此，此时张量再次变为 1×H×W，然后被转发到 Sigmoid 函数。与 CAM 类似，由该 Sigmoid 函数产生的张量充当空间注意力权重。