RT-DETR改进策略【注意力机制篇】| Large Separable Kernel Attention (LSKA) 大核可分离卷积注意力 二次创新HGBlock、ResNetLayer
一、本文介绍
本文记录的是
利用
LSKA 大核可分离卷积注意力
模块优化
RT-DETR
的目标检测网络模型
。
LSKA
结合了
大卷积核的广阔感受野
和
可分离卷积的高效性
，不仅
降低计算复杂度和内存占用
，而且提高了模型对
不同卷积核大小的适应性
。本文将其应用到
RT-DETR
中，利用
LSKA
提高模型对不同尺度目标的检测能力。
二、LSKA介绍
2.1 设计出发点
-
在视觉注意力网络（VAN）中，
大核注意力（LKA）模块虽在视觉任务中表现出色，但深度卷积层随卷积核增大， 计算和内存消耗呈二次增长 。为解决此问题，使VAN的注意力模块能使用 极大卷积核 ，提出了LSKA模块。
2.2 原理
-
LSKA将 深度卷积层 的 2D卷积核 分解为 级联的水平和垂直1D卷积核 。通过这种分解方式，能在注意力模块中直接使用大核的深度卷积层，无需额外模块，且相比标准LKA设计， 能降低计算复杂度和内存占用 。
2.3 结构
2.3.1 基本LKA模块（不使用扩张深度卷积）
-
输入特征图
F
∈
R
C
×
H
×
W
F \in \mathbb{R}^{C ×H ×W}
F
∈
R
C
×
H
×
W
，设计LKA的简单方式是在
2D深度卷积中使用大卷积核
，计算公式：
Z
C
=
∑
H
,
W
W
k
×
k
C
∗
F
C
Z^{C}=\sum_{H, W} W_{k × k}^{C} * F^{C}
Z
C
=
H
,
W
∑
W
k
×
k
C
∗
F
C
A
C
=
W
1
×
1
∗
Z
C
A^{C}=W_{1 × 1} * Z^{C}
A
C
=
W
1
×
1
∗
Z
C
F
C
=
A
C
⊗
F
C
F^{C}=A^{C} \otimes F^{C}
F
C
=
A
C
⊗
F
C
这里
Z
C
Z^{C}
Z
C
是
深度卷积输出， A C A^{C} A C 是注意力图， ⊗ \otimes ⊗ 是哈达玛积。 此结构中深度卷积计算成本随核增大呈二次增长。
2.3.2 原始LKA模块（VAN 中）
-
为缓解上述问题，原始LKA模块将
大核深度卷积
分解为
小核深度卷积和扩张的大核深度卷积，计算公式： Z ‾ C = ∑ H , W W ( 2 d − 1 ) × ( 2 d − 1 ) C ∗ F C \overline{Z}^{C}=\sum_{H, W} W_{(2 d-1) \times(2 d-1)}^{C} * F^{C} Z C = H , W ∑ W ( 2 d − 1 ) × ( 2 d − 1 ) C ∗ F C Z C = ∑ H , W W [ k d ] × [ k d ] C ∗ Z ‾ C Z^{C}=\sum_{H, W} W_{\left[\frac{k}{d}\right] \times\left[\frac{k}{d}\right]}^{C} * \overline{Z}^{C} Z C = H , W ∑ W [ d k ] × [ d k ] C ∗ Z C A C = W 1 × 1 ∗ Z C A^{C}=W_{1 × 1} * Z^{C} A C = W 1 × 1 ∗ Z C F ‾ C = A C ⊗ F C \overline{F}^{C}=A^{C} \otimes F^{C} F C = A C ⊗ F C
2.3.3 LSKA模块
-
如
图d所示，将LKA的前两层分解为四层，每层LKA由两个1D卷积层组成。输出计算如公式： Z ‾ C = ∑ H , W W ( 2 d − 1 ) × 1 C ∗ ( ∑ H , W W 1 × ( 2 d − 1 ) C ∗ F C ) \overline{Z}^{C}=\sum_{H, W} W_{(2 d-1) × 1}^{C} *\left(\sum_{H, W} W_{1 \times(2 d-1)}^{C} * F^{C}\right) Z C = H , W ∑ W ( 2 d − 1 ) × 1 C ∗ H , W ∑ W 1 × ( 2 d − 1 ) C ∗ F C Z C = ∑ H , W W [ k d ] × 1 C ∗ ( ∑ H , W W 1 × [ k d ] C ∗ Z ‾ C ) Z^{C}=\sum_{H, W} W_{\left[\frac{k}{d}\right] × 1}^{C} *\left(\sum_{H, W} W_{1 \times\left[\frac{k}{d}\right]}^{C} * \overline{Z}^{C}\right) Z C = H , W ∑ W [ d k ] × 1 C ∗ H , W ∑ W 1 × [ d k ] C ∗ Z C A C = W 1 × 1 ∗ Z C A^{C}=W_{1 × 1} * Z^{C} A C = W 1 × 1 ∗ Z C F ‾ C = A C ⊗ F C \overline{F}^{C}=A^{C} \otimes F^{C} F C = A C ⊗ F C
2.4 优势
-
计算复杂度和内存占用方面
-
从图可知，相比
LKA - trivial和LKA，LSKA - trivial和LSKA显著降低了VAN的计算复杂度。通过分析 FLOPs 和 参数 计算公式，LSKA在 深度卷积层 和 扩张深度卷积层 都能节省参数，计算更有效。
-
从图可知，相比
-
性能方面
-
长程依赖捕捉
：通过
有效感受野（ERF）生成方法
验证，如图4所示，从核大小
7到65，LSKA方法能有效捕捉图像长程依赖。 - 空间和通道适应性 ：继承LKA设计，包含空间和通道注意力特性，且 采用级联水平和垂直内核 进一步降低内存和计算复杂度。
-
对大核的可扩展性
：在VAN中，
LKA - trivial随核增大计算成本二次增长，LKA虽降低但核超 23 × 23 23×23 23 × 23 时参数增长。而LSKA - trivial和LSKA不仅降低计算成本，还能保持模型参数相对稳定，且随核增大从23到53，LSKA - Base在参数大小、GFLOPs和精度上都表现出可扩展性。
-
长程依赖捕捉
：通过
有效感受野（ERF）生成方法
验证，如图4所示，从核大小
论文： https://arxiv.org/pdf/2309.01439
源码： https://github.com/StevenLauHKHK/Large-Separable-Kernel-Attention
三、LSKA的实现代码
LSKA
及其改进的实现代码如下：
import torch
import torch.nn as nn
import torch.nn.functional as F
from ultralytics.nn.modules.conv import LightConv
class LSKA(nn.Module):
def __init__(self, dim, k_size):
super().__init__()
self.k_size = k_size
if k_size == 7:
self.conv0h = nn.Conv2d(dim, dim, kernel_size=(1, 3), stride=(1,1), padding=(0,(3-1)//2), groups=dim)
self.conv0v = nn.Conv2d(dim, dim, kernel_size=(3, 1), stride=(1,1), padding=((3-1)//2,0), groups=dim)
self.conv_spatial_h = nn.Conv2d(dim, dim, kernel_size=(1, 3), stride=(1,1), padding=(0,2), groups=dim, dilation=2)
self.conv_spatial_v = nn.Conv2d(dim, dim, kernel_size=(3, 1), stride=(1,1), padding=(2,0), groups=dim, dilation=2)
elif k_size == 11:
self.conv0h = nn.Conv2d(dim, dim, kernel_size=(1, 3), stride=(1,1), padding=(0,(3-1)//2), groups=dim)
self.conv0v = nn.Conv2d(dim, dim, kernel_size=(3, 1), stride=(1,1), padding=((3-1)//2,0), groups=dim)
self.conv_spatial_h = nn.Conv2d(dim, dim, kernel_size=(1, 5), stride=(1,1), padding=(0,4), groups=dim, dilation=2)
self.conv_spatial_v = nn.Conv2d(dim, dim, kernel_size=(5, 1), stride=(1,1), padding=(4,0), groups=dim, dilation=2)
elif k_size == 23:
self.conv0h = nn.Conv2d(dim, dim, kernel_size=(1, 5), stride=(1,1), padding=(0,(5-1)//2), groups=dim)
self.conv0v = nn.Conv2d(dim, dim, kernel_size=(5, 1), stride=(1,1), padding=((5-1)//2,0), groups=dim)
self.conv_spatial_h = nn.Conv2d(dim, dim, kernel_size=(1, 7), stride=(1,1), padding=(0,9), groups=dim, dilation=3)
self.conv_spatial_v = nn.Conv2d(dim, dim, kernel_size=(7, 1), stride=(1,1), padding=(9,0), groups=dim, dilation=3)
elif k_size == 35:
self.conv0h = nn.Conv2d(dim, dim, kernel_size=(1, 5), stride=(1,1), padding=(0,(5-1)//2), groups=dim)
self.conv0v = nn.Conv2d(dim, dim, kernel_size=(5, 1), stride=(1,1), padding=((5-1)//2,0), groups=dim)
self.conv_spatial_h = nn.Conv2d(dim, dim, kernel_size=(1, 11), stride=(1,1), padding=(0,15), groups=dim, dilation=3)
self.conv_spatial_v = nn.Conv2d(dim, dim, kernel_size=(11, 1), stride=(1,1), padding=(15,0), groups=dim, dilation=3)
elif k_size == 41:
self.conv0h = nn.Conv2d(dim, dim, kernel_size=(1, 5), stride=(1,1), padding=(0,(5-1)//2), groups=dim)
self.conv0v = nn.Conv2d(dim, dim, kernel_size=(5, 1), stride=(1,1), padding=((5-1)//2,0), groups=dim)
self.conv_spatial_h = nn.Conv2d(dim, dim, kernel_size=(1, 13), stride=(1,1), padding=(0,18), groups=dim, dilation=3)
self.conv_spatial_v = nn.Conv2d(dim, dim, kernel_size=(13, 1), stride=(1,1), padding=(18,0), groups=dim, dilation=3)
elif k_size == 53:
self.conv0h = nn.Conv2d(dim, dim, kernel_size=(1, 5), stride=(1,1), padding=(0,(5-1)//2), groups=dim)
self.conv0v = nn.Conv2d(dim, dim, kernel_size=(5, 1), stride=(1,1), padding=((5-1)//2,0), groups=dim)
self.conv_spatial_h = nn.Conv2d(dim, dim, kernel_size=(1, 17), stride=(1,1), padding=(0,24), groups=dim, dilation=3)
self.conv_spatial_v = nn.Conv2d(dim, dim, kernel_size=(17, 1), stride=(1,1), padding=(24,0), groups=dim, dilation=3)
self.conv1 = nn.Conv2d(dim, dim, 1)
def forward(self, x):
u = x.clone()
attn = self.conv0h(x)
attn = self.conv0v(attn)
attn = self.conv_spatial_h(attn)
attn = self.conv_spatial_v(attn)
attn = self.conv1(attn)
return u * attn
def autopad(k, p=None, d=1): # kernel, padding, dilation
"""Pad to 'same' shape outputs."""
if d > 1:
k = d * (k - 1) + 1 if isinstance(k, int) else [d * (x - 1) + 1 for x in k] # actual kernel-size
if p is None:
p = k // 2 if isinstance(k, int) else [x // 2 for x in k] # auto-pad
return p
class Conv(nn.Module):
"""Standard convolution with args(ch_in, ch_out, kernel, stride, padding, groups, dilation, activation)."""
default_act = nn.SiLU() # default activation
def __init__(self, c1, c2, k=1, s=1, p=None, g=1, d=1, act=True):
"""Initialize Conv layer with given arguments including activation."""
super().__init__()
self.conv = nn.Conv2d(c1, c2, k, s, autopad(k, p, d), groups=g, dilation=d, bias=False)
self.bn = nn.BatchNorm2d(c2)
self.act = self.default_act if act is True else act if isinstance(act, nn.Module) else nn.Identity()
def forward(self, x):
"""Apply convolution, batch normalization and activation to input tensor."""
return self.act(self.bn(self.conv(x)))
def forward_fuse(self, x):
"""Perform transposed convolution of 2D data."""
return self.act(self.conv(x))
class HGBlock_LSKA(nn.Module):
"""
HG_Block of PPHGNetV2 with 2 convolutions and LightConv.
https://github.com/PaddlePaddle/PaddleDetection/blob/develop/ppdet/modeling/backbones/hgnet_v2.py
"""
def __init__(self, c1, cm, c2, k=3, n=6, lightconv=False, shortcut=False, act=nn.ReLU()):
"""Initializes a CSP Bottleneck with 1 convolution using specified input and output channels."""
super().__init__()
block = LightConv if lightconv else Conv
self.m = nn.ModuleList(block(c1 if i == 0 else cm, cm, k=k, act=act) for i in range(n))
self.sc = Conv(c1 + n * cm, c2 // 2, 1, 1, act=act) # squeeze conv
self.ec = Conv(c2 // 2, c2, 1, 1, act=act) # excitation conv
self.add = shortcut and c1 == c2
self.cv = LSKA(c2, 11)
def forward(self, x):
"""Forward pass of a PPHGNetV2 backbone layer."""
y = [x]
y.extend(m(y[-1]) for m in self.m)
y = self.cv(self.ec(self.sc(torch.cat(y, 1))))
return y + x if self.add else y
class ResNetBlock(nn.Module):
"""ResNet block with standard convolution layers."""
def __init__(self, c1, c2, s=1, e=4):
"""Initialize convolution with given parameters."""
super().__init__()
c3 = e * c2
self.cv1 = Conv(c1, c2, k=1, s=1, act=True)
self.cv2 = Conv(c2, c2, k=3, s=s, p=1, act=True)
self.cv3 = Conv(c2, c3, k=1, act=False)
self.cv4 = LSKA(c2, 11)
self.shortcut = nn.Sequential(Conv(c1, c3, k=1, s=s, act=False)) if s != 1 or c1 != c3 else nn.Identity()
def forward(self, x):
"""Forward pass through the ResNet block."""
return F.relu(self.cv3(self.cv4(self.cv2(self.cv1(x)))) + self.shortcut(x))
class ResNetLayer_LSKA(nn.Module):
"""ResNet layer with multiple ResNet blocks."""
def __init__(self, c1, c2, s=1, is_first=False, n=1, e=4):
"""Initializes the ResNetLayer given arguments."""
super().__init__()
self.is_first = is_first
if self.is_first:
self.layer = nn.Sequential(
Conv(c1, c2, k=7, s=2, p=3, act=True), nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
)
else:
blocks = [ResNetBlock(c1, c2, s, e=e)]
blocks.extend([ResNetBlock(e * c2, c2, 1, e=e) for _ in range(n - 1)])
self.layer = nn.Sequential(*blocks)
def forward(self, x):
"""Forward pass through the ResNet layer."""
return self.layer(x)
四、创新模块
4.1 改进点1⭐
模块改进方法
：基于
LSKA模块
的
HGBlock
（
第五节讲解添加步骤
）。
第一种改进方法是对
RT-DETR
中的
HGBlock模块
进行改进，并将
LSKA
在加入到
HGBlock
模块中。
改进代码如下：
对
HGBlock
模块进行改进，加入
LSKA模块
并重命名为
HGBlock_LSKA
class HGBlock_LSKA(nn.Module):
"""
HG_Block of PPHGNetV2 with 2 convolutions and LightConv.
https://github.com/PaddlePaddle/PaddleDetection/blob/develop/ppdet/modeling/backbones/hgnet_v2.py
"""
def __init__(self, c1, cm, c2, k=3, n=6, lightconv=False, shortcut=False, act=nn.ReLU()):
"""Initializes a CSP Bottleneck with 1 convolution using specified input and output channels."""
super().__init__()
block = LightConv if lightconv else Conv
self.m = nn.ModuleList(block(c1 if i == 0 else cm, cm, k=k, act=act) for i in range(n))
self.sc = Conv(c1 + n * cm, c2 // 2, 1, 1, act=act) # squeeze conv
self.ec = Conv(c2 // 2, c2, 1, 1, act=act) # excitation conv
self.add = shortcut and c1 == c2
self.cv = LSKA(c2, 11)
def forward(self, x):
"""Forward pass of a PPHGNetV2 backbone layer."""
y = [x]
y.extend(m(y[-1]) for m in self.m)
y = self.cv(self.ec(self.sc(torch.cat(y, 1))))
return y + x if self.add else y
4.2 改进点2⭐
模块改进方法
：基于
LSKA模块
的
ResNetLayer
（
第五节讲解添加步骤
）。
第二种改进方法是对
RT-DETR
中的
ResNetLayer模块
进行改进，并将
LSKA
在加入到
ResNetLayer
模块中。
改进代码如下：
对
ResNetLayer
模块进行改进，加入
LSKA模块
。
class ResNetBlock(nn.Module):
"""ResNet block with standard convolution layers."""
def __init__(self, c1, c2, s=1, e=4):
"""Initialize convolution with given parameters."""
super().__init__()
c3 = e * c2
self.cv1 = Conv(c1, c2, k=1, s=1, act=True)
self.cv2 = Conv(c2, c2, k=3, s=s, p=1, act=True)
self.cv3 = Conv(c2, c3, k=1, act=False)
self.cv4 = LSKA(c2, 11)
self.shortcut = nn.Sequential(Conv(c1, c3, k=1, s=s, act=False)) if s != 1 or c1 != c3 else nn.Identity()
def forward(self, x):
"""Forward pass through the ResNet block."""
return F.relu(self.cv3(self.cv4(self.cv2(self.cv1(x)))) + self.shortcut(x))
class ResNetLayer_LSKA(nn.Module):
"""ResNet layer with multiple ResNet blocks."""
def __init__(self, c1, c2, s=1, is_first=False, n=1, e=4):
"""Initializes the ResNetLayer given arguments."""
super().__init__()
self.is_first = is_first
if self.is_first:
self.layer = nn.Sequential(
Conv(c1, c2, k=7, s=2, p=3, act=True), nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
)
else:
blocks = [ResNetBlock(c1, c2, s, e=e)]
blocks.extend([ResNetBlock(e * c2, c2, 1, e=e) for _ in range(n - 1)])
self.layer = nn.Sequential(*blocks)
def forward(self, x):
"""Forward pass through the ResNet layer."""
return self.layer(x)
注意❗：在
第五小节
中需要声明的模块名称为：
HGBlock_LSKA
和
ResNetLayer_LSKA
。
五、添加步骤
5.1 修改一
① 在
ultralytics/nn/
目录下新建
AddModules
文件夹用于存放模块代码
② 在
AddModules
文件夹下新建
LSKA.py
，将
第三节
中的代码粘贴到此处
5.2 修改二
在
AddModules
文件夹下新建
__init__.py
（已有则不用新建），在文件内导入模块：
from .LSKA import *
5.3 修改三
在
ultralytics/nn/modules/tasks.py
文件中，需要在两处位置添加各模块类名称。
首先：导入模块
其次：在
parse_model函数
中注册
HGBlock_LSKA
和
ResNetLayer_LSKA
模块
六、yaml模型文件
6.1 模型改进版本1
此处以
ultralytics/cfg/models/rt-detr/rtdetr-l.yaml
为例，在同目录下创建一个用于自己数据集训练的模型文件
rtdetr-l-HGBlock_LSKA.yaml
。
将
rtdetr-l.yaml
中的内容复制到
rtdetr-l-HGBlock_LSKA.yaml
文件下，修改
nc
数量等于自己数据中目标的数量。
📌 模型的修改方法是将
骨干网络
中的
HGBlock
替换成
HGBlock_LSKA
。
# Ultralytics YOLO 🚀, AGPL-3.0 license
# RT-DETR-l object detection model with P3-P5 outputs. For details see https://docs.ultralytics.com/models/rtdetr
# Parameters
nc: 1 # number of classes
scales: # model compound scaling constants, i.e. 'model=yolov8n-cls.yaml' will call yolov8-cls.yaml with scale 'n'
# [depth, width, max_channels]
l: [1.00, 1.00, 1024]
backbone:
# [from, repeats, module, args]
- [-1, 1, HGStem, [32, 48]] # 0-P2/4
- [-1, 6, HGBlock, [48, 128, 3]] # stage 1
- [-1, 1, DWConv, [128, 3, 2, 1, False]] # 2-P3/8
- [-1, 6, HGBlock, [96, 512, 3]] # stage 2
- [-1, 1, DWConv, [512, 3, 2, 1, False]] # 4-P4/16
- [-1, 6, HGBlock_LSKA, [192, 1024, 5, True, False]] # cm, c2, k, light, shortcut
- [-1, 6, HGBlock_LSKA, [192, 1024, 5, True, True]]
- [-1, 6, HGBlock_LSKA, [192, 1024, 5, True, True]] # stage 3
- [-1, 1, DWConv, [1024, 3, 2, 1, False]] # 8-P5/32
- [-1, 6, HGBlock, [384, 2048, 5, True, False]] # stage 4
head:
- [-1, 1, Conv, [256, 1, 1, None, 1, 1, False]] # 10 input_proj.2
- [-1, 1, AIFI, [1024, 8]]
- [-1, 1, Conv, [256, 1, 1]] # 12, Y5, lateral_convs.0
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [7, 1, Conv, [256, 1, 1, None, 1, 1, False]] # 14 input_proj.1
- [[-2, -1], 1, Concat, [1]]
- [-1, 3, RepC3, [256]] # 16, fpn_blocks.0
- [-1, 1, Conv, [256, 1, 1]] # 17, Y4, lateral_convs.1
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [3, 1, Conv, [256, 1, 1, None, 1, 1, False]] # 19 input_proj.0
- [[-2, -1], 1, Concat, [1]] # cat backbone P4
- [-1, 3, RepC3, [256]] # X3 (21), fpn_blocks.1
- [-1, 1, Conv, [256, 3, 2]] # 22, downsample_convs.0
- [[-1, 17], 1, Concat, [1]] # cat Y4
- [-1, 3, RepC3, [256]] # F4 (24), pan_blocks.0
- [-1, 1, Conv, [256, 3, 2]] # 25, downsample_convs.1
- [[-1, 12], 1, Concat, [1]] # cat Y5
- [-1, 3, RepC3, [256]] # F5 (27), pan_blocks.1
- [[21, 24, 27], 1, RTDETRDecoder, [nc]] # Detect(P3, P4, P5)
6.2 模型改进版本2⭐
此处以
ultralytics/cfg/models/rt-detr/rtdetr-resnet50.yaml
为例，在同目录下创建一个用于自己数据集训练的模型文件
rtdetr-ResNetLayer_LSKA.yaml
。
将
rtdetr-resnet50.yaml
中的内容复制到
rtdetr-ResNetLayer_LSKA.yaml
文件下，修改
nc
数量等于自己数据中目标的数量。
📌 模型的修改方法是将
骨干网络
中的
ResNetLayer模块
替换成
ResNetLayer_LSKA模块
。
# Ultralytics YOLO 🚀, AGPL-3.0 license
# RT-DETR-ResNet50 object detection model with P3-P5 outputs.
# Parameters
nc: 1 # number of classes
scales: # model compound scaling constants, i.e. 'model=yolov8n-cls.yaml' will call yolov8-cls.yaml with scale 'n'
# [depth, width, max_channels]
l: [1.00, 1.00, 1024]
backbone:
# [from, repeats, module, args]
- [-1, 1, ResNetLayer_LSKA, [3, 64, 1, True, 1]] # 0
- [-1, 1, ResNetLayer_LSKA, [64, 64, 1, False, 3]] # 1
- [-1, 1, ResNetLayer_LSKA, [256, 128, 2, False, 4]] # 2
- [-1, 1, ResNetLayer_LSKA, [512, 256, 2, False, 6]] # 3
- [-1, 1, ResNetLayer_LSKA, [1024, 512, 2, False, 3]] # 4
head:
- [-1, 1, Conv, [256, 1, 1, None, 1, 1, False]] # 5
- [-1, 1, AIFI, [1024, 8]]
- [-1, 1, Conv, [256, 1, 1]] # 7
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [3, 1, Conv, [256, 1, 1, None, 1, 1, False]] # 9
- [[-2, -1], 1, Concat, [1]]
- [-1, 3, RepC3, [256]] # 11
- [-1, 1, Conv, [256, 1, 1]] # 12
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [2, 1, Conv, [256, 1, 1, None, 1, 1, False]] # 14
- [[-2, -1], 1, Concat, [1]] # cat backbone P4
- [-1, 3, RepC3, [256]] # X3 (16), fpn_blocks.1
- [-1, 1, Conv, [256, 3, 2]] # 17, downsample_convs.0
- [[-1, 12], 1, Concat, [1]] # cat Y4
- [-1, 3, RepC3, [256]] # F4 (19), pan_blocks.0
- [-1, 1, Conv, [256, 3, 2]] # 20, downsample_convs.1
- [[-1, 7], 1, Concat, [1]] # cat Y5
- [-1, 3, RepC3, [256]] # F5 (22), pan_blocks.1
- [[16, 19, 22], 1, RTDETRDecoder, [nc]] # Detect(P3, P4, P5)
七、成功运行结果
打印网络模型可以看到
HGBlock_LSKA
和
ResNetLayer_LSKA
已经加入到模型中，并可以进行训练了。
rtdetr-l-HGBlock_LSKA ：
rtdetr-l-HGBlock_LSKA summary: 700 layers, 36,018,371 parameters, 36,018,371 gradients, 118.2 GFLOPs
from n params module arguments
0 -1 1 25248 ultralytics.nn.modules.block.HGStem [3, 32, 48]
1 -1 6 155072 ultralytics.nn.modules.block.HGBlock [48, 48, 128, 3, 6]
2 -1 1 1408 ultralytics.nn.modules.conv.DWConv [128, 128, 3, 2, 1, False]
3 -1 6 839296 ultralytics.nn.modules.block.HGBlock [128, 96, 512, 3, 6]
4 -1 1 5632 ultralytics.nn.modules.conv.DWConv [512, 512, 3, 2, 1, False]
5 -1 6 2765440 ultralytics.nn.AddModules.LSKA.HGBlock_LSKA [512, 192, 1024, 5, 6, True, False]
6 -1 6 3125888 ultralytics.nn.AddModules.LSKA.HGBlock_LSKA [1024, 192, 1024, 5, 6, True, True]
7 -1 6 3125888 ultralytics.nn.AddModules.LSKA.HGBlock_LSKA [1024, 192, 1024, 5, 6, True, True]
8 -1 1 11264 ultralytics.nn.modules.conv.DWConv [1024, 1024, 3, 2, 1, False]
9 -1 6 6708480 ultralytics.nn.modules.block.HGBlock [1024, 384, 2048, 5, 6, True, False]
10 -1 1 524800 ultralytics.nn.modules.conv.Conv [2048, 256, 1, 1, None, 1, 1, False]
11 -1 1 789760 ultralytics.nn.modules.transformer.AIFI [256, 1024, 8]
12 -1 1 66048 ultralytics.nn.modules.conv.Conv [256, 256, 1, 1]
13 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
14 7 1 262656 ultralytics.nn.modules.conv.Conv [1024, 256, 1, 1, None, 1, 1, False]
15 [-2, -1] 1 0 ultralytics.nn.modules.conv.Concat [1]
16 -1 3 2232320 ultralytics.nn.modules.block.RepC3 [512, 256, 3]
17 -1 1 66048 ultralytics.nn.modules.conv.Conv [256, 256, 1, 1]
18 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
19 3 1 131584 ultralytics.nn.modules.conv.Conv [512, 256, 1, 1, None, 1, 1, False]
20 [-2, -1] 1 0 ultralytics.nn.modules.conv.Concat [1]
21 -1 3 2232320 ultralytics.nn.modules.block.RepC3 [512, 256, 3]
22 -1 1 590336 ultralytics.nn.modules.conv.Conv [256, 256, 3, 2]
23 [-1, 17] 1 0 ultralytics.nn.modules.conv.Concat [1]
24 -1 3 2232320 ultralytics.nn.modules.block.RepC3 [512, 256, 3]
25 -1 1 590336 ultralytics.nn.modules.conv.Conv [256, 256, 3, 2]
26 [-1, 12] 1 0 ultralytics.nn.modules.conv.Concat [1]
27 -1 3 2232320 ultralytics.nn.modules.block.RepC3 [512, 256, 3]
28 [21, 24, 27] 1 7303907 ultralytics.nn.modules.head.RTDETRDecoder [1, [256, 256, 256]]
rtdetr-l-HGBlock_LSKA summary: 700 layers, 36,018,371 parameters, 36,018,371 gradients, 118.2 GFLOPs
rtdetr-ResNetLayer_LSKA ：
rtdetr-ResNetLayer_LSKA summary: 689 layers, 44,099,555 parameters, 44,099,555 gradients, 134.2 GFLOPs
from n params module arguments
0 -1 1 9536 ultralytics.nn.AddModules.LSKA.ResNetLayer_LSKA[3, 64, 1, True, 1]
1 -1 1 232128 ultralytics.nn.AddModules.LSKA.ResNetLayer_LSKA[64, 64, 1, False, 3]
2 -1 1 1295872 ultralytics.nn.AddModules.LSKA.ResNetLayer_LSKA[256, 128, 2, False, 4]
3 -1 1 7523840 ultralytics.nn.AddModules.LSKA.ResNetLayer_LSKA[512, 256, 2, False, 6]
4 -1 1 15783424 ultralytics.nn.AddModules.LSKA.ResNetLayer_LSKA[1024, 512, 2, False, 3]
5 -1 1 524800 ultralytics.nn.modules.conv.Conv [2048, 256, 1, 1, None, 1, 1, False]
6 -1 1 789760 ultralytics.nn.modules.transformer.AIFI [256, 1024, 8]
7 -1 1 66048 ultralytics.nn.modules.conv.Conv [256, 256, 1, 1]
8 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
9 3 1 262656 ultralytics.nn.modules.conv.Conv [1024, 256, 1, 1, None, 1, 1, False]
10 [-2, -1] 1 0 ultralytics.nn.modules.conv.Concat [1]
11 -1 3 2232320 ultralytics.nn.modules.block.RepC3 [512, 256, 3]
12 -1 1 66048 ultralytics.nn.modules.conv.Conv [256, 256, 1, 1]
13 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
14 2 1 131584 ultralytics.nn.modules.conv.Conv [512, 256, 1, 1, None, 1, 1, False]
15 [-2, -1] 1 0 ultralytics.nn.modules.conv.Concat [1]
16 -1 3 2232320 ultralytics.nn.modules.block.RepC3 [512, 256, 3]
17 -1 1 590336 ultralytics.nn.modules.conv.Conv [256, 256, 3, 2]
18 [-1, 12] 1 0 ultralytics.nn.modules.conv.Concat [1]
19 -1 3 2232320 ultralytics.nn.modules.block.RepC3 [512, 256, 3]
20 -1 1 590336 ultralytics.nn.modules.conv.Conv [256, 256, 3, 2]
21 [-1, 7] 1 0 ultralytics.nn.modules.conv.Concat [1]
22 -1 3 2232320 ultralytics.nn.modules.block.RepC3 [512, 256, 3]
23 [16, 19, 22] 1 7303907 ultralytics.nn.modules.head.RTDETRDecoder [1, [256, 256, 256]]
rtdetr-ResNetLayer_LSKA summary: 689 layers, 44,099,555 parameters, 44,099,555 gradients, 134.2 GFLOPs