RT-DETR改进策略【Neck】| SEAM：分离和增强注意模块，解决复杂场景下的小目标遮挡问题
一、本文介绍
本文记录的是
利用 SEAM 模块优化 RT-DETR 的目标检测网络模型
。
SEAM（Separated and Enhancement Attention Module）
的设计出发点在于
解决复杂场景下的人脸遮挡问题
，相当于是
小目标
被其他物体部分遮挡时，传统方法因特征缺失导致检测精度下降的问题。该模块
通过增强未遮挡区域的特征响应并补偿被遮挡区域的信息损失
，提升模型对遮挡
小目标
的检测能力。
二、SEAM模块介绍
YOLO-FaceV2: A Scale and Occlusion Aware Face Detector
2.1 出发点
SEAM模块（Separated and Enhancement Attention Module）的设计出发点是解决复杂场景下的人脸遮挡问题，特别是当人脸被其他物体部分遮挡时，传统方法容易因特征缺失导致检测精度下降。该模块通过增强未遮挡区域的特征响应并补偿被遮挡区域的信息损失，提升模型对遮挡人脸的检测能力。
2.2 结构
SEAM模块包含以下核心组件：
- 深度可分离卷积与残差连接 ：通过逐通道卷积和点卷积（1×1）分离空间和通道维度的特征学习，减少参数冗余并保留多尺度信息。
- 通道融合全连接层 ：使用两层全连接网络融合通道间的依赖关系，强化被遮挡区域与周围区域的关联。
- 指数归一化 ：将全连接层的输出通过指数函数映射到[1, e]区间，增强模型对位置误差的鲁棒性，最终与原始特征相乘生成注意力权重。
2.3 优势
- 遮挡适应性 ：通过通道和空间特征的联合优化，显著提升被遮挡目标的特征响应，减少漏检率。
- 轻量高效 ：深度可分离卷积降低计算复杂度，同时残差结构避免梯度消失问题。
- 多尺度融合 ：结合不同patch大小的多尺度特征提取（如CSMM模块），增强对不同尺度遮挡的泛化能力。
论文： https://arxiv.org/pdf/2208.02019
源码： https://github.com/Krasjet-Yu/YOLO-FaceV2
三、SEAM的实现代码
SEAM
的实现代码如下：
import torch
import torch.nn as nn
class Residual(nn.Module):
def __init__(self, fn):
super(Residual, self).__init__()
self.fn = fn
def forward(self, x):
return self.fn(x) + x
class ConvMixer(nn.Module):
def __init__(self, c1, c2, depth, kernel_size=3, patch_size=4, reduction=16):
super(ConvMixer, self).__init__()
if c1 != c2:
c2 = c1
self.DConvN = nn.Sequential(
nn.Conv2d(c1, c2, kernel_size=patch_size, stride=patch_size),
nn.GELU(),
nn.BatchNorm2d(c2),
*[nn.Sequential(
Residual(nn.Sequential(
nn.Conv2d(c2, c2, kernel_size, groups=c2, padding=1),
nn.GELU(),
nn.BatchNorm2d(c2)
)),
nn.Conv2d(c2, c1, kernel_size=1),
nn.GELU(),
nn.BatchNorm2d(c2)
) for i in range(depth)]
)
self.avg_pool = torch.nn.AdaptiveAvgPool2d(1)
self.fc = nn.Sequential(
nn.Linear(c2, c2 // reduction, bias=False),
nn.ReLU(inplace=True),
nn.Linear(c2 // reduction, c2, bias=False),
nn.Sigmoid()
)
def forward(self, x):
b, c, _, _ = x.size()
y = self.DConvN(x)
y = self.avg_pool(y).view(b, c)
y = self.fc(y).view(b, c, 1, 1)
y = torch.exp(y)
return x * y.expand_as(x)
class SEAM(nn.Module):
def __init__(self, c1, c2, n, reduction=16):
super(SEAM, self).__init__()
if c1 != c2:
c2 = c1
self.DCovN = nn.Sequential(
# nn.Conv2d(c1, c2, kernel_size=3, stride=1, padding=1, groups=c1),
# nn.GELU(),
# nn.BatchNorm2d(c2),
*[nn.Sequential(
Residual(nn.Sequential(
nn.Conv2d(in_channels=c2, out_channels=c2, kernel_size=3, stride=1, padding=1, groups=c2),
nn.GELU(),
nn.BatchNorm2d(c2)
)),
nn.Conv2d(in_channels=c2, out_channels=c2, kernel_size=1, stride=1, padding=0, groups=1),
nn.GELU(),
nn.BatchNorm2d(c2)
) for i in range(n)]
)
self.avg_pool = torch.nn.AdaptiveAvgPool2d(1)
self.fc = nn.Sequential(
nn.Linear(c2, c2 // reduction, bias=False),
nn.ReLU(inplace=True),
nn.Linear(c2 // reduction, c2, bias=False),
nn.Sigmoid()
)
self._initialize_weights()
# self.initialize_layer(self.avg_pool)
self.initialize_layer(self.fc)
def forward(self, x):
b, c, _, _ = x.size()
y = self.DCovN(x)
y = self.avg_pool(y).view(b, c)
y = self.fc(y).view(b, c, 1, 1)
y = torch.exp(y)
return x * y.expand_as(x)
def _initialize_weights(self):
for m in self.modules():
if isinstance(m, nn.Conv2d):
nn.init.xavier_uniform_(m.weight, gain=1)
elif isinstance(m, nn.BatchNorm2d):
nn.init.constant_(m.weight, 1)
nn.init.constant_(m.bias, 0)
def initialize_layer(self, layer):
if isinstance(layer, (nn.Conv2d, nn.Linear)):
torch.nn.init.normal_(layer.weight, mean=0., std=0.001)
if layer.bias is not None:
torch.nn.init.constant_(layer.bias, 0)
def DcovN(c1, c2, depth, kernel_size=3, patch_size=3):
dcovn = nn.Sequential(
nn.Conv2d(c1, c2, kernel_size=patch_size, stride=patch_size),
nn.SiLU(),
nn.BatchNorm2d(c2),
*[nn.Sequential(
Residual(nn.Sequential(
nn.Conv2d(in_channels=c2, out_channels=c2, kernel_size=kernel_size, stride=1, padding=1, groups=c2),
nn.SiLU(),
nn.BatchNorm2d(c2)
)),
nn.Conv2d(in_channels=c2, out_channels=c2, kernel_size=1, stride=1, padding=0, groups=1),
nn.SiLU(),
nn.BatchNorm2d(c2)
) for i in range(depth)]
)
return dcovn
class MultiSEAM(nn.Module):
def __init__(self, c1, c2, depth, kernel_size=3, patch_size=[3, 5, 7], reduction=16):
super(MultiSEAM, self).__init__()
if c1 != c2:
c2 = c1
self.DCovN0 = DcovN(c1, c2, depth, kernel_size=kernel_size, patch_size=patch_size[0])
self.DCovN1 = DcovN(c1, c2, depth, kernel_size=kernel_size, patch_size=patch_size[1])
self.DCovN2 = DcovN(c1, c2, depth, kernel_size=kernel_size, patch_size=patch_size[2])
self.avg_pool = torch.nn.AdaptiveAvgPool2d(1)
self.fc = nn.Sequential(
nn.Linear(c2, c2 // reduction, bias=False),
nn.ReLU(inplace=True),
nn.Linear(c2 // reduction, c2, bias=False),
nn.Sigmoid()
)
def forward(self, x):
b, c, _, _ = x.size()
y0 = self.DCovN0(x)
y1 = self.DCovN1(x)
y2 = self.DCovN2(x)
y0 = self.avg_pool(y0).view(b, c)
y1 = self.avg_pool(y1).view(b, c)
y2 = self.avg_pool(y2).view(b, c)
y4 = self.avg_pool(x).view(b, c)
y = (y0 + y1 + y2 + y4) / 4
y = self.fc(y).view(b, c, 1, 1)
y = torch.exp(y)
return x * y.expand_as(x)
四、创新模块
4.1 改进点1⭐
模块改进方法
：加入
SEAM模块
（
第五节讲解添加步骤
）。
SEAM模块
添加后如下：
4.2 改进点2⭐
模块改进方法
：加入
MultiSEAM模块
（
第五节讲解添加步骤
）。
MultiSEAM模块
添加后如下：
注意❗：在
第五小节
中需要声明的模块名称为：
SEAM
和
MultiSEAM
。
五、添加步骤
5.1 修改一
① 在
ultralytics/nn/
目录下新建
AddModules
文件夹用于存放模块代码
② 在
AddModules
文件夹下新建
SEAM.py
，将
第三节
中的代码粘贴到此处
5.2 修改二
在
AddModules
文件夹下新建
__init__.py
（已有则不用新建），在文件内导入模块：
from .SEAM import *
5.3 修改三
在
ultralytics/nn/modules/tasks.py
文件中，需要在两处位置添加各模块类名称。
首先：导入模块
其次：在
parse_model函数
中注册
SEAM
和
MultiSEAM
模块
六、yaml模型文件
6.1 模型改进版本1
此处以
ultralytics/cfg/models/rt-detr/rtdetr-l.yaml
为例，在同目录下创建一个用于自己数据集训练的模型文件
rtdetr-l-SEAM.yaml
。
将
rtdetr-l.yaml
中的内容复制到
rtdetr-l-SEAM.yaml
文件下，修改
nc
数量等于自己数据中目标的数量。
📌 模型的修改方法是在
颈部网络中
添加
SEAM模块
。
# Ultralytics YOLO 🚀, AGPL-3.0 license
# RT-DETR-l object detection model with P3-P5 outputs. For details see https://docs.ultralytics.com/models/rtdetr
# Parameters
nc: 1 # number of classes
scales: # model compound scaling constants, i.e. 'model=yolov8n-cls.yaml' will call yolov8-cls.yaml with scale 'n'
# [depth, width, max_channels]
l: [1.00, 1.00, 1024]
backbone:
# [from, repeats, module, args]
- [-1, 1, HGStem, [32, 48]] # 0-P2/4
- [-1, 6, HGBlock, [48, 128, 3]] # stage 1
- [-1, 1, DWConv, [128, 3, 2, 1, False]] # 2-P3/8
- [-1, 6, HGBlock, [96, 512, 3]] # stage 2
- [-1, 1, DWConv, [512, 3, 2, 1, False]] # 4-P4/16
- [-1, 6, HGBlock, [192, 1024, 5, True, False]] # cm, c2, k, light, shortcut
- [-1, 6, HGBlock, [192, 1024, 5, True, True]]
- [-1, 6, HGBlock, [192, 1024, 5, True, True]] # stage 3
- [-1, 1, DWConv, [1024, 3, 2, 1, False]] # 8-P5/32
- [-1, 6, HGBlock, [384, 2048, 5, True, False]] # stage 4
head:
- [-1, 1, Conv, [256, 1, 1, None, 1, 1, False]] # 10 input_proj.2
- [-1, 1, AIFI, [1024, 8]]
- [-1, 1, Conv, [256, 1, 1]] # 12, Y5, lateral_convs.0
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [7, 1, Conv, [256, 1, 1, None, 1, 1, False]] # 14 input_proj.1
- [[-2, -1], 1, Concat, [1]]
- [-1, 3, RepC3, [256]] # 16, fpn_blocks.0
- [-1, 1, Conv, [256, 1, 1]] # 17, Y4, lateral_convs.1
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [3, 1, Conv, [256, 1, 1, None, 1, 1, False]] # 19 input_proj.0
- [[-2, -1], 1, Concat, [1]] # cat backbone P4
- [-1, 3, RepC3, [256]] # X3 (21), fpn_blocks.1
- [-1, 1, Conv, [256, 3, 2]] # 22, downsample_convs.0
- [[-1, 17], 1, Concat, [1]] # cat Y4
- [-1, 3, RepC3, [256]] # F4 (24), pan_blocks.0
- [-1, 1, Conv, [256, 3, 2]] # 25, downsample_convs.1
- [[-1, 12], 1, Concat, [1]] # cat Y5
- [-1, 3, RepC3, [256]] # F5 (27), pan_blocks.1
- [21, 1, SEAM, [256, 1, 16]]
- [24, 1, SEAM, [256, 1, 16]]
- [27, 1, SEAM, [256, 1, 16]]
- [[28, 29, 30], 1, RTDETRDecoder, [nc]] # Detect(P3, P4, P5)
6.2 模型改进版本2⭐
此处以
ultralytics/cfg/models/rt-detr/rtdetr-resnet50.yaml
为例，在同目录下创建一个用于自己数据集训练的模型文件
rtdetr-resnet50-MultiSEAM.yaml
。
将
rtdetr-resnet50.yaml
中的内容复制到
rtdetr-resnet50-MultiSEAM.yaml
文件下，修改
nc
数量等于自己数据中目标的数量。
📌 模型的修改方法是在
颈部网络中
添加
MultiSEAM模块
。
# Ultralytics YOLO 🚀, AGPL-3.0 license
# RT-DETR-ResNet50 object detection model with P3-P5 outputs.
# Parameters
nc: 80 # number of classes
scales: # model compound scaling constants, i.e. 'model=yolov8n-cls.yaml' will call yolov8-cls.yaml with scale 'n'
# [depth, width, max_channels]
l: [1.00, 1.00, 1024]
backbone:
# [from, repeats, module, args]
- [-1, 1, ResNetLayer, [3, 64, 1, True, 1]] # 0
- [-1, 1, ResNetLayer, [64, 64, 1, False, 3]] # 1
- [-1, 1, ResNetLayer, [256, 128, 2, False, 4]] # 2
- [-1, 1, ResNetLayer, [512, 256, 2, False, 6]] # 3
- [-1, 1, ResNetLayer, [1024, 512, 2, False, 3]] # 4
head:
- [-1, 1, Conv, [256, 1, 1, None, 1, 1, False]] # 5
- [-1, 1, AIFI, [1024, 8]]
- [-1, 1, Conv, [256, 1, 1]] # 7
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [3, 1, Conv, [256, 1, 1, None, 1, 1, False]] # 9
- [[-2, -1], 1, Concat, [1]]
- [-1, 3, RepC3, [256]] # 11
- [-1, 1, Conv, [256, 1, 1]] # 12
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [2, 1, Conv, [256, 1, 1, None, 1, 1, False]] # 14
- [[-2, -1], 1, Concat, [1]] # cat backbone P4
- [-1, 3, RepC3, [256]] # X3 (16), fpn_blocks.1
- [-1, 1, Conv, [256, 3, 2]] # 17, downsample_convs.0
- [[-1, 12], 1, Concat, [1]] # cat Y4
- [-1, 3, RepC3, [256]] # F4 (19), pan_blocks.0
- [-1, 1, Conv, [256, 3, 2]] # 20, downsample_convs.1
- [[-1, 7], 1, Concat, [1]] # cat Y5
- [-1, 3, RepC3, [256]] # F5 (22), pan_blocks.1
- [16, 1, MultiSEAM, [256, 1, 3, [6, 7, 8], 16]]
- [-1, 1, Conv, [256, 1, 1]] # 24
- [19, 1, MultiSEAM, [256, 1, 3, [6, 7, 8], 16]]
- [-1, 1, Conv, [256, 1, 1]] # 26
- [22, 1, MultiSEAM, [256, 1, 3, [6, 7, 8], 16]]
- [-1, 1, Conv, [256, 1, 1]] # 28
- [[24, 26, 28], 1, RTDETRDecoder, [nc]] # Detect(P3, P4, P5)
七、成功运行结果
打印网络模型可以看到
SEAM
和
MultiSEAM
已经加入到模型中，并可以进行训练了。
rtdetr-l-SEAM ：
rtdetr-l-SEAM summary: 732 layers, 33,040,835 parameters, 33,040,835 gradients, 109.2 GFLOPs
from n params module arguments
0 -1 1 25248 ultralytics.nn.modules.block.HGStem [3, 32, 48]
1 -1 6 155072 ultralytics.nn.modules.block.HGBlock [48, 48, 128, 3, 6]
2 -1 1 1408 ultralytics.nn.modules.conv.DWConv [128, 128, 3, 2, 1, False]
3 -1 6 839296 ultralytics.nn.modules.block.HGBlock [128, 96, 512, 3, 6]
4 -1 1 5632 ultralytics.nn.modules.conv.DWConv [512, 512, 3, 2, 1, False]
5 -1 6 1695360 ultralytics.nn.modules.block.HGBlock [512, 192, 1024, 5, 6, True, False]
6 -1 6 2055808 ultralytics.nn.modules.block.HGBlock [1024, 192, 1024, 5, 6, True, True]
7 -1 6 2055808 ultralytics.nn.modules.block.HGBlock [1024, 192, 1024, 5, 6, True, True]
8 -1 1 11264 ultralytics.nn.modules.conv.DWConv [1024, 1024, 3, 2, 1, False]
9 -1 6 6708480 ultralytics.nn.modules.block.HGBlock [1024, 384, 2048, 5, 6, True, False]
10 -1 1 524800 ultralytics.nn.modules.conv.Conv [2048, 256, 1, 1, None, 1, 1, False]
11 -1 1 789760 ultralytics.nn.modules.transformer.AIFI [256, 1024, 8]
12 -1 1 66048 ultralytics.nn.modules.conv.Conv [256, 256, 1, 1]
13 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
14 7 1 262656 ultralytics.nn.modules.conv.Conv [1024, 256, 1, 1, None, 1, 1, False]
15 [-2, -1] 1 0 ultralytics.nn.modules.conv.Concat [1]
16 -1 3 2232320 ultralytics.nn.modules.block.RepC3 [512, 256, 3]
17 -1 1 66048 ultralytics.nn.modules.conv.Conv [256, 256, 1, 1]
18 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
19 3 1 131584 ultralytics.nn.modules.conv.Conv [512, 256, 1, 1, None, 1, 1, False]
20 [-2, -1] 1 0 ultralytics.nn.modules.conv.Concat [1]
21 -1 3 2232320 ultralytics.nn.modules.block.RepC3 [512, 256, 3]
22 -1 1 590336 ultralytics.nn.modules.conv.Conv [256, 256, 3, 2]
23 [-1, 17] 1 0 ultralytics.nn.modules.conv.Concat [1]
24 -1 3 2232320 ultralytics.nn.modules.block.RepC3 [512, 256, 3]
25 -1 1 590336 ultralytics.nn.modules.conv.Conv [256, 256, 3, 2]
26 [-1, 12] 1 0 ultralytics.nn.modules.conv.Concat [1]
27 -1 3 2232320 ultralytics.nn.modules.block.RepC3 [512, 256, 3]
28 21 1 77568 ultralytics.nn.AddModules.SEAM.SEAM [256, 256, 1, 16]
29 24 1 77568 ultralytics.nn.AddModules.SEAM.SEAM [256, 256, 1, 16]
30 27 1 77568 ultralytics.nn.AddModules.SEAM.SEAM [256, 256, 1, 16]
31 [28, 29, 30] 1 7303907 ultralytics.nn.modules.head.RTDETRDecoder [1, [256, 256, 256]]
rtdetr-l-SEAM summary: 732 layers, 33,040,835 parameters, 33,040,835 gradients, 109.2 GFLOPs
rtdetr-resnet50-MultiSEAM ：
rtdetr-resnet50-MultiSEAM summary: 739 layers, 72,911,395 parameters, 72,911,395 gradients, 134.7 GFLOPs
from n params module arguments
0 -1 1 9536 ultralytics.nn.modules.block.ResNetLayer [3, 64, 1, True, 1]
1 -1 1 215808 ultralytics.nn.modules.block.ResNetLayer [64, 64, 1, False, 3]
2 -1 1 1219584 ultralytics.nn.modules.block.ResNetLayer [256, 128, 2, False, 4]
3 -1 1 7098368 ultralytics.nn.modules.block.ResNetLayer [512, 256, 2, False, 6]
4 -1 1 14964736 ultralytics.nn.modules.block.ResNetLayer [1024, 512, 2, False, 3]
5 -1 1 524800 ultralytics.nn.modules.conv.Conv [2048, 256, 1, 1, None, 1, 1, False]
6 -1 1 789760 ultralytics.nn.modules.transformer.AIFI [256, 1024, 8]
7 -1 1 66048 ultralytics.nn.modules.conv.Conv [256, 256, 1, 1]
8 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
9 3 1 262656 ultralytics.nn.modules.conv.Conv [1024, 256, 1, 1, None, 1, 1, False]
10 [-2, -1] 1 0 ultralytics.nn.modules.conv.Concat [1]
11 -1 3 2232320 ultralytics.nn.modules.block.RepC3 [512, 256, 3]
12 -1 1 66048 ultralytics.nn.modules.conv.Conv [256, 256, 1, 1]
13 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
14 2 1 131584 ultralytics.nn.modules.conv.Conv [512, 256, 1, 1, None, 1, 1, False]
15 [-2, -1] 1 0 ultralytics.nn.modules.conv.Concat [1]
16 -1 3 2232320 ultralytics.nn.modules.block.RepC3 [512, 256, 3]
17 -1 1 590336 ultralytics.nn.modules.conv.Conv [256, 256, 3, 2]
18 [-1, 12] 1 0 ultralytics.nn.modules.conv.Concat [1]
19 -1 3 2232320 ultralytics.nn.modules.block.RepC3 [512, 256, 3]
20 -1 1 590336 ultralytics.nn.modules.conv.Conv [256, 256, 3, 2]
21 [-1, 7] 1 0 ultralytics.nn.modules.conv.Concat [1]
22 -1 3 2232320 ultralytics.nn.modules.block.RepC3 [512, 256, 3]
23 16 1 9983488 ultralytics.nn.AddModules.SEAM.MultiSEAM [256, 256, 1, 3, [6, 7, 8], 16]
24 -1 1 66048 ultralytics.nn.modules.conv.Conv [256, 256, 1, 1]
25 19 1 9983488 ultralytics.nn.AddModules.SEAM.MultiSEAM [256, 256, 1, 3, [6, 7, 8], 16]
26 -1 1 66048 ultralytics.nn.modules.conv.Conv [256, 256, 1, 1]
27 22 1 9983488 ultralytics.nn.AddModules.SEAM.MultiSEAM [256, 256, 1, 3, [6, 7, 8], 16]
28 -1 1 66048 ultralytics.nn.modules.conv.Conv [256, 256, 1, 1]
29 [24, 26, 28] 1 7303907 ultralytics.nn.modules.head.RTDETRDecoder [1, [256, 256, 256]]
rtdetr-resnet50-MultiSEAM summary: 739 layers, 72,911,395 parameters, 72,911,395 gradients, 134.7 GFLOPs