RT-DETR改进策略【卷积层】| AKConv： 具有任意采样形状和任意参数数量的卷积核
一、本文介绍
本文记录的是
利用
AKConv
优化
RT-DETR
的目标检测网络模型
。标准卷积操作的卷积运算局限于局部窗口，无法捕获其他位置的信息，且采样形状固定，无法适应不同数据集和位置中目标形状的变化。而
AKConv
旨在为卷积核提供任意数量的参数和任意采样形状，以在网络开销和性能之间提供更丰富的权衡选择
。本文利用
AKConv
模块改进
RT-DETR
，来提高网络性能。
二、AKConv介绍
AKConv
： 具有任意采样形状和任意参数数量的卷积核
2.1、AKConv原理
-
定义初始采样位置
：
-
通过新的坐标生成算法为任意大小的卷积核定义初始位置。具体来说，先生成规则采样网格，再为剩余采样点创建不规则网格，最后拼接生成整体采样网格。以
3
×
3
3×3
3
×
3
卷积操作为例，其采样网格
R
=
{
(
−
1
,
−
1
)
,
(
−
1
,
0
)
,
.
.
.
,
(
0
,
1
)
,
(
1
,
1
)
}
R = \{(-1,-1),(-1,0),...,(0,1),(1,1)\}
R
=
{(
−
1
,
−
1
)
,
(
−
1
,
0
)
,
...
,
(
0
,
1
)
,
(
1
,
1
)}
，但
AKConv针对不规则形状的卷积核，通过算法生成卷积核 P n P_n P n 的初始采样坐标。在算法中，将左上角 ( 0 , 0 ) (0, 0) ( 0 , 0 ) 点设为采样原点。定义在位置 P 0 P_0 P 0 的相应卷积运算为 C o n v ( P 0 ) = ∑ w × ( P 0 + P n ) Conv(P_0) = \sum w \times (P_0 + P_n) C o n v ( P 0 ) = ∑ w × ( P 0 + P n ) ，其中 w w w 表示卷积参数。
-
通过新的坐标生成算法为任意大小的卷积核定义初始位置。具体来说，先生成规则采样网格，再为剩余采样点创建不规则网格，最后拼接生成整体采样网格。以
3
×
3
3×3
3
×
3
卷积操作为例，其采样网格
R
=
{
(
−
1
,
−
1
)
,
(
−
1
,
0
)
,
.
.
.
,
(
0
,
1
)
,
(
1
,
1
)
}
R = \{(-1,-1),(-1,0),...,(0,1),(1,1)\}
R
=
{(
−
1
,
−
1
)
,
(
−
1
,
0
)
,
...
,
(
0
,
1
)
,
(
1
,
1
)}
，但
-
可变卷积操作
：
-
标准卷积采样位置固定，只能提取当前窗口的局部信息，无法捕获其他位置的信息。
Deformable Conv通过学习偏移来调整初始规则模式的采样网格，以弥补卷积操作的不足，但它和标准卷积不允许卷积核有任意数量的参数，且卷积参数随卷积核大小呈平方增长，对硬件环境不友好。 -
AKConv类似于Deformable Conv，先通过卷积操作获得对应核的偏移，其维度为 ( B , 2 N , H , W ) (B, 2N, H, W) ( B , 2 N , H , W ) （ N N N 为卷积核大小），然后通过偏移和原始坐标求和得到修改后的坐标，最后通过插值和重采样获得对应位置的特征。 -
对于不规则卷积核难以提取对应采样位置特征的问题，可采用多种方法解决。例如，在
Deformable Conv和RFAConv中，通过在空间维度堆叠 3 × 3 3×3 3 × 3 卷积特征，然后用步长为3的卷积操作提取特征，但此方法针对正方形采样形状。因此，可以将特征按行或列堆叠，使用列卷积或行卷积来提取对应不规则采样形状的特征；也可以将特征转换为四维 ( C , N , H , W ) (C, N, H, W) ( C , N , H , W ) ，然后用步长和卷积大小为 ( N , 1 , 1 ) (N,1,1) ( N , 1 , 1 ) 的Conv3d提取特征；还可以将特征在通道维度堆叠为 ( C N , H , W ) (CN, H, W) ( CN , H , W ) ，然后用(1×1)卷积降维为 ( C , H , W ) (C, H, W) ( C , H , W ) 。在AKConv中，按照上述方法对特征进行重塑并使用相应卷积操作即可提取对应特征。最终，AKConv通过不规则卷积完成特征提取过程，能根据偏移灵活调整样本形状，为卷积采样形状带来更多探索选项。
-
标准卷积采样位置固定，只能提取当前窗口的局部信息，无法捕获其他位置的信息。
-
扩展AKConv
：
-
AKConv可以通过重新采样初始坐标呈现多种变化，即使不使用Deformable Conv中的偏移思想，也能实现多种卷积核形状。 - 根据数据集目标形状的变化，设计对应采样形状的卷积操作，通过设计特定形状的初始采样形状来实现。例如，为长管状结构分割任务设计具有相应形状的采样坐标，但形状选择仅针对长管状结构。
-
AKConv真正实现了卷积核操作具有任意形状和数量的过程，能够使卷积核呈现多种形状。而Deformable Conv旨在弥补常规卷积的不足，DSConv针对特定对象形状设计，它们都没有探索任意大小和形状的卷积。AKConv通过Offset使卷积操作能高效提取不规则样本形状的特征，允许卷积有任意数量的卷积参数和多种形状。
-
2.2、AKConv优势
- 提高检测性能 ：在COCO2017、VOC 7 + 12和VisDrone - DET2021等数据集的目标检测实验中，AKConv显著提高了YOLOv5等模型的目标检测性能。例如，在COCO2017数据集上，当AKConv大小为5时，不仅使模型所需的参数和计算开销减少，还显著提高了YOLOv5n的检测精度， A P 50 AP_{50} A P 50 、 A P 75 AP_{75} A P 75 和 A P AP A P 均提高了三个百分点，且对大物体的检测精度提升更为明显。
- 灵活的参数选择 ：与标准卷积和Deformable Conv相比，AKConv允许卷积参数数量呈线性增减，有利于硬件环境，可作为轻量级模型的替代选择，减少模型参数和计算开销。同时，在大内核且资源充足的情况下，它有更多选项来提高网络性能。
- 丰富的选择 ：与Deformable Conv不同，AKConv为网络提供了更丰富的选择，它可以使用规则和不规则卷积操作。当AKConv大小设置为(K)的平方时，它可以成为Deformable Conv，但Deformable Conv没有探索不规则卷积核大小，而AKConv可以实现参数为5和11等的卷积操作。
论文： https://arxiv.org/pdf/2311.11587v2
源码： https://github.com/CV-ZhangXin/AKConv
三、AKConv的实现代码
AKConv模块
的实现代码如下：
import math
from einops import rearrange
import torch
import torch.nn as nn
import torch.nn.functional as F
def autopad(k, p=None, d=1): # kernel, padding, dilation
"""Pad to 'same' shape outputs."""
if d > 1:
k = d * (k - 1) + 1 if isinstance(k, int) else [d * (x - 1) + 1 for x in k] # actual kernel-size
if p is None:
p = k // 2 if isinstance(k, int) else [x // 2 for x in k] # auto-pad
return p
class Conv(nn.Module):
"""Standard convolution with args(ch_in, ch_out, kernel, stride, padding, groups, dilation, activation)."""
default_act = nn.SiLU() # default activation
def __init__(self, c1, c2, k=1, s=1, p=None, g=1, d=1, act=True):
"""Initialize Conv layer with given arguments including activation."""
super().__init__()
self.conv = nn.Conv2d(c1, c2, k, s, autopad(k, p, d), groups=g, dilation=d, bias=False)
self.bn = nn.BatchNorm2d(c2)
self.act = self.default_act if act is True else act if isinstance(act, nn.Module) else nn.Identity()
def forward(self, x):
"""Apply convolution, batch normalization and activation to input tensor."""
return self.act(self.bn(self.conv(x)))
def forward_fuse(self, x):
"""Perform transposed convolution of 2D data."""
return self.act(self.conv(x))
class AKConv(nn.Module):
def __init__(self, inc, outc, num_param=5, stride=1):
super(AKConv, self).__init__()
self.num_param = num_param
self.stride = stride
self.conv = Conv(inc, outc, k=(num_param, 1), s=(num_param, 1) )
self.p_conv = nn.Conv2d(inc, 2 * num_param, kernel_size=3, padding=1, stride=stride)
nn.init.constant_(self.p_conv.weight, 0)
self.p_conv.register_full_backward_hook(self._set_lr)
@staticmethod
def _set_lr(module, grad_input, grad_output):
grad_input = (grad_input[i] * 0.1 for i in range(len(grad_input)))
grad_output = (grad_output[i] * 0.1 for i in range(len(grad_output)))
def forward(self, x):
# N is num_param.
offset = self.p_conv(x)
dtype = offset.data.type()
N = offset.size(1) // 2
# (b, 2N, h, w)
p = self._get_p(offset, dtype)
# (b, h, w, 2N)
p = p.contiguous().permute(0, 2, 3, 1)
q_lt = p.detach().floor()
q_rb = q_lt + 1
q_lt = torch.cat([torch.clamp(q_lt[..., :N], 0, x.size(2) - 1), torch.clamp(q_lt[..., N:], 0, x.size(3) - 1)],
dim=-1).long()
q_rb = torch.cat([torch.clamp(q_rb[..., :N], 0, x.size(2) - 1), torch.clamp(q_rb[..., N:], 0, x.size(3) - 1)],
dim=-1).long()
q_lb = torch.cat([q_lt[..., :N], q_rb[..., N:]], dim=-1)
q_rt = torch.cat([q_rb[..., :N], q_lt[..., N:]], dim=-1)
# clip p
p = torch.cat([torch.clamp(p[..., :N], 0, x.size(2) - 1), torch.clamp(p[..., N:], 0, x.size(3) - 1)], dim=-1)
# bilinear kernel (b, h, w, N)
g_lt = (1 + (q_lt[..., :N].type_as(p) - p[..., :N])) * (1 + (q_lt[..., N:].type_as(p) - p[..., N:]))
g_rb = (1 - (q_rb[..., :N].type_as(p) - p[..., :N])) * (1 - (q_rb[..., N:].type_as(p) - p[..., N:]))
g_lb = (1 + (q_lb[..., :N].type_as(p) - p[..., :N])) * (1 - (q_lb[..., N:].type_as(p) - p[..., N:]))
g_rt = (1 - (q_rt[..., :N].type_as(p) - p[..., :N])) * (1 + (q_rt[..., N:].type_as(p) - p[..., N:]))
# resampling the features based on the modified coordinates.
x_q_lt = self._get_x_q(x, q_lt, N)
x_q_rb = self._get_x_q(x, q_rb, N)
x_q_lb = self._get_x_q(x, q_lb, N)
x_q_rt = self._get_x_q(x, q_rt, N)
# bilinear
x_offset = g_lt.unsqueeze(dim=1) * x_q_lt + \
g_rb.unsqueeze(dim=1) * x_q_rb + \
g_lb.unsqueeze(dim=1) * x_q_lb + \
g_rt.unsqueeze(dim=1) * x_q_rt
x_offset = self._reshape_x_offset(x_offset, self.num_param)
out = self.conv(x_offset)
return out
# generating the inital sampled shapes for the AKConv with different sizes.
def _get_p_n(self, N, dtype):
base_int = round(math.sqrt(self.num_param))
row_number = self.num_param // base_int
mod_number = self.num_param % base_int
p_n_x, p_n_y = torch.meshgrid(
torch.arange(0, row_number),
torch.arange(0, base_int), indexing='xy')
p_n_x = torch.flatten(p_n_x)
p_n_y = torch.flatten(p_n_y)
if mod_number > 0:
mod_p_n_x, mod_p_n_y = torch.meshgrid(
torch.arange(row_number, row_number + 1),
torch.arange(0, mod_number), indexing='xy')
mod_p_n_x = torch.flatten(mod_p_n_x)
mod_p_n_y = torch.flatten(mod_p_n_y)
p_n_x, p_n_y = torch.cat((p_n_x, mod_p_n_x)), torch.cat((p_n_y, mod_p_n_y))
p_n = torch.cat([p_n_x, p_n_y], 0)
p_n = p_n.view(1, 2 * N, 1, 1).type(dtype)
return p_n
# no zero-padding
def _get_p_0(self, h, w, N, dtype):
p_0_x, p_0_y = torch.meshgrid(
torch.arange(0, h * self.stride, self.stride),
torch.arange(0, w * self.stride, self.stride), indexing='xy')
p_0_x = torch.flatten(p_0_x).view(1, 1, h, w).repeat(1, N, 1, 1)
p_0_y = torch.flatten(p_0_y).view(1, 1, h, w).repeat(1, N, 1, 1)
p_0 = torch.cat([p_0_x, p_0_y], 1).type(dtype)
return p_0
def _get_p(self, offset, dtype):
N, h, w = offset.size(1) // 2, offset.size(2), offset.size(3)
# (1, 2N, 1, 1)
p_n = self._get_p_n(N, dtype)
# (1, 2N, h, w)
p_0 = self._get_p_0(h, w, N, dtype)
p = p_0 + p_n + offset
return p
def _get_x_q(self, x, q, N):
b, h, w, _ = q.size()
padded_w = x.size(3)
c = x.size(1)
# (b, c, h*w)
x = x.contiguous().view(b, c, -1)
# (b, h, w, N)
index = q[..., :N] * padded_w + q[..., N:] # offset_x*w + offset_y
# (b, c, h*w*N)
index = index.contiguous().unsqueeze(dim=1).expand(-1, c, -1, -1, -1).contiguous().view(b, c, -1)
x_offset = x.gather(dim=-1, index=index).contiguous().view(b, c, h, w, N)
return x_offset
# Stacking resampled features in the row direction.
@staticmethod
def _reshape_x_offset(x_offset, num_param):
b, c, h, w, n = x_offset.size()
x_offset = rearrange(x_offset, 'b c h w n -> b c (h n) w')
return x_offset
class ResNetBlock(nn.Module):
"""ResNet block with standard convolution layers."""
def __init__(self, c1, c2, s=1, e=4):
"""Initialize convolution with given parameters."""
super().__init__()
c3 = e * c2
self.cv1 = Conv(c1, c2, k=1, s=1, act=True)
self.cv2 = Conv(c2, c2, k=3, s=s, p=1, act=True)
self.cv3 = AKConv(c2, c3)
self.shortcut = nn.Sequential(Conv(c1, c3, k=1, s=s, act=False)) if s != 1 or c1 != c3 else nn.Identity()
def forward(self, x):
"""Forward pass through the ResNet block."""
return F.relu(self.cv3(self.cv2(self.cv1(x))) + self.shortcut(x))
class ResNetLayer_AKConv(nn.Module):
"""ResNet layer with multiple ResNet blocks."""
def __init__(self, c1, c2, s=1, is_first=False, n=1, e=4):
"""Initializes the ResNetLayer given arguments."""
super().__init__()
self.is_first = is_first
if self.is_first:
self.layer = nn.Sequential(
Conv(c1, c2, k=7, s=2, p=3, act=True), nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
)
else:
blocks = [ResNetBlock(c1, c2, s, e=e)]
blocks.extend([ResNetBlock(e * c2, c2, 1, e=e) for _ in range(n - 1)])
self.layer = nn.Sequential(*blocks)
def forward(self, x):
"""Forward pass through the ResNet layer."""
return self.layer(x)
四、创新模块
4.1 改进点1⭐
模块改进方法
：直接加入
AKConv 模块
（
第五节讲解添加步骤
）。
AKConv 模块
添加后如下：
4.2 改进点2⭐
模块改进方法
：基于
AKConv 模块
的
ResNetLayer
（
第五节讲解添加步骤
）。
第二种改进方法是对
RT-DETR
中的
ResNetLayer模块
进行改进，并将
AKConv
在加入到
ResNetLayer
模块中。
改进代码如下：
首先添加如下代码改进
ResNetBlock
模块，并将
ResNetLayer
重命名为
ResNetLayer_AKConv
class ResNetBlock(nn.Module):
"""ResNet block with standard convolution layers."""
def __init__(self, c1, c2, s=1, e=4):
"""Initialize convolution with given parameters."""
super().__init__()
c3 = e * c2
self.cv1 = Conv(c1, c2, k=1, s=1, act=True)
self.cv2 = Conv(c2, c2, k=3, s=s, p=1, act=True)
self.cv3 = AKConv(c2, c3)
self.shortcut = nn.Sequential(Conv(c1, c3, k=1, s=s, act=False)) if s != 1 or c1 != c3 else nn.Identity()
def forward(self, x):
"""Forward pass through the ResNet block."""
return F.relu(self.cv3(self.cv2(self.cv1(x))) + self.shortcut(x))
class ResNetLayer_AKConv(nn.Module):
"""ResNet layer with multiple ResNet blocks."""
def __init__(self, c1, c2, s=1, is_first=False, n=1, e=4):
"""Initializes the ResNetLayer given arguments."""
super().__init__()
self.is_first = is_first
if self.is_first:
self.layer = nn.Sequential(
Conv(c1, c2, k=7, s=2, p=3, act=True), nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
)
else:
blocks = [ResNetBlock(c1, c2, s, e=e)]
blocks.extend([ResNetBlock(e * c2, c2, 1, e=e) for _ in range(n - 1)])
self.layer = nn.Sequential(*blocks)
def forward(self, x):
"""Forward pass through the ResNet layer."""
return self.layer(x)
注意❗：在
第五小节
中需要声明的模块名称为：
AKConv
和
ResNetLayer_AKConv
。
五、添加步骤
5.1 修改一
① 在
ultralytics/nn/
目录下新建
AddModules
文件夹用于存放模块代码
② 在
AddModules
文件夹下新建
AKConv .py
，将
第三节
中的代码粘贴到此处
5.2 修改二
在
AddModules
文件夹下新建
__init__.py
（已有则不用新建），在文件内导入模块：
from .AKConv import *
5.3 修改三
在
ultralytics/nn/modules/tasks.py
文件中，需要在两处位置添加各模块类名称。
首先：导入模块
其次：在
parse_model函数
中注册
AKConv
和
ResNetLayer_AKConv
模块
六、yaml模型文件
6.1 模型改进版本1
此处以
ultralytics/cfg/models/rt-detr/rtdetr-l.yaml
为例，在同目录下创建一个用于自己数据集训练的模型文件
rtdetr-l-AKConv.yaml
。
将
rtdetr-l.yaml
中的内容复制到
rtdetr-l-AKConv.yaml
文件下，修改
nc
数量等于自己数据中目标的数量。
📌 模型的修改方法是将
骨干网络
中添加
AKConv模块
。
# Ultralytics YOLO 🚀, AGPL-3.0 license
# RT-DETR-l object detection model with P3-P5 outputs. For details see https://docs.ultralytics.com/models/rtdetr
# Parameters
nc: 1 # number of classes
scales: # model compound scaling constants, i.e. 'model=yolov8n-cls.yaml' will call yolov8-cls.yaml with scale 'n'
# [depth, width, max_channels]
l: [1.00, 1.00, 1024]
backbone:
# [from, repeats, module, args]
- [-1, 1, HGStem, [32, 48]] # 0-P2/4
- [-1, 6, HGBlock, [48, 128, 3]] # stage 1
- [-1, 1, DWConv, [128, 3, 2, 1, False]] # 2-P3/8
- [-1, 6, HGBlock, [96, 512, 3]] # stage 2
- [-1, 1, DWConv, [512, 3, 2, 1, False]] # 4-P4/16
- [-1, 6, AKConv, [512]] # cm, c2, k, light, shortcut
- [-1, 6, AKConv, [512]]
- [-1, 6, AKConv, [512]] # stage 3
- [-1, 1, DWConv, [1024, 3, 2, 1, False]] # 8-P5/32
- [-1, 6, HGBlock, [384, 2048, 5, True, False]] # stage 4
head:
- [-1, 1, Conv, [256, 1, 1, None, 1, 1, False]] # 10 input_proj.2
- [-1, 1, AIFI, [1024, 8]]
- [-1, 1, Conv, [256, 1, 1]] # 12, Y5, lateral_convs.0
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [7, 1, Conv, [256, 1, 1, None, 1, 1, False]] # 14 input_proj.1
- [[-2, -1], 1, Concat, [1]]
- [-1, 3, RepC3, [256]] # 16, fpn_blocks.0
- [-1, 1, Conv, [256, 1, 1]] # 17, Y4, lateral_convs.1
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [3, 1, Conv, [256, 1, 1, None, 1, 1, False]] # 19 input_proj.0
- [[-2, -1], 1, Concat, [1]] # cat backbone P4
- [-1, 3, RepC3, [256]] # X3 (21), fpn_blocks.1
- [-1, 1, Conv, [256, 3, 2]] # 22, downsample_convs.0
- [[-1, 17], 1, Concat, [1]] # cat Y4
- [-1, 3, RepC3, [256]] # F4 (24), pan_blocks.0
- [-1, 1, Conv, [256, 3, 2]] # 25, downsample_convs.1
- [[-1, 12], 1, Concat, [1]] # cat Y5
- [-1, 3, RepC3, [256]] # F5 (27), pan_blocks.1
- [[21, 24, 27], 1, RTDETRDecoder, [nc]] # Detect(P3, P4, P5)
6.2 模型改进版本2⭐
此处以
ultralytics/cfg/models/rt-detr/rtdetr-resnet50.yaml
为例，在同目录下创建一个用于自己数据集训练的模型文件
rtdetr-ResNetLayer_AKConv.yaml
。
将
rtdetr-resnet50.yaml
中的内容复制到
rtdetr-ResNetLayer_AKConv.yaml
文件下，修改
nc
数量等于自己数据中目标的数量。
📌 模型的修改方法是将
骨干网络
中的所有
ResNetLayer模块
替换成
ResNetLayer_AKConv模块
。
# Ultralytics YOLO 🚀, AGPL-3.0 license
# RT-DETR-ResNet50 object detection model with P3-P5 outputs.
# Parameters
nc: 1 # number of classes
scales: # model compound scaling constants, i.e. 'model=yolov8n-cls.yaml' will call yolov8-cls.yaml with scale 'n'
# [depth, width, max_channels]
l: [1.00, 1.00, 1024]
backbone:
# [from, repeats, module, args]
- [-1, 1, ResNetLayer_AKConv, [3, 64, 1, True, 1]] # 0
- [-1, 1, ResNetLayer_AKConv, [64, 64, 1, False, 3]] # 1
- [-1, 1, ResNetLayer_AKConv, [256, 128, 2, False, 4]] # 2
- [-1, 1, ResNetLayer_AKConv, [512, 256, 2, False, 6]] # 3
- [-1, 1, ResNetLayer_AKConv, [1024, 512, 2, False, 3]] # 4
head:
- [-1, 1, Conv, [256, 1, 1, None, 1, 1, False]] # 5
- [-1, 1, AIFI, [1024, 8]]
- [-1, 1, Conv, [256, 1, 1]] # 7
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [3, 1, Conv, [256, 1, 1, None, 1, 1, False]] # 9
- [[-2, -1], 1, Concat, [1]]
- [-1, 3, RepC3, [256]] # 11
- [-1, 1, Conv, [256, 1, 1]] # 12
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [2, 1, Conv, [256, 1, 1, None, 1, 1, False]] # 14
- [[-2, -1], 1, Concat, [1]] # cat backbone P4
- [-1, 3, RepC3, [256]] # X3 (16), fpn_blocks.1
- [-1, 1, Conv, [256, 3, 2]] # 17, downsample_convs.0
- [[-1, 12], 1, Concat, [1]] # cat Y4
- [-1, 3, RepC3, [256]] # F4 (19), pan_blocks.0
- [-1, 1, Conv, [256, 3, 2]] # 20, downsample_convs.1
- [[-1, 7], 1, Concat, [1]] # cat Y5
- [-1, 3, RepC3, [256]] # F5 (22), pan_blocks.1
- [[16, 19, 22], 1, RTDETRDecoder, [nc]] # Detect(P3, P4, P5)
七、成功运行结果
打印网络模型可以看到
AKConv
和
ResNetLayer_RFAConv
已经加入到模型中，并可以进行训练了。
**rtdetr-l-AKConv **：
rtdetr-l-AKConv summary: 607 layers, 51,311,095 parameters, 51,311,095 gradients, 167.2 GFLOPs
from n params module arguments
0 -1 1 25248 ultralytics.nn.modules.block.HGStem [3, 32, 48]
1 -1 6 155072 ultralytics.nn.modules.block.HGBlock [48, 48, 128, 3, 6]
2 -1 1 1408 ultralytics.nn.modules.conv.DWConv [128, 128, 3, 2, 1, False]
3 -1 6 839296 ultralytics.nn.modules.block.HGBlock [128, 96, 512, 3, 6]
4 -1 1 5632 ultralytics.nn.modules.conv.DWConv [512, 512, 3, 2, 1, False]
5 -1 6 8147004 ultralytics.nn.AddModules.AKConv.AKConv [512, 512]
6 -1 6 8147004 ultralytics.nn.AddModules.AKConv.AKConv [512, 512]
7 -1 6 8147004 ultralytics.nn.AddModules.AKConv.AKConv [512, 512]
8 -1 1 11264 ultralytics.nn.modules.conv.DWConv [512, 1024, 3, 2, 1, False]
9 -1 6 6708480 ultralytics.nn.modules.block.HGBlock [1024, 384, 2048, 5, 6, True, False]
10 -1 1 524800 ultralytics.nn.modules.conv.Conv [2048, 256, 1, 1, None, 1, 1, False]
11 -1 1 789760 ultralytics.nn.modules.transformer.AIFI [256, 1024, 8]
12 -1 1 66048 ultralytics.nn.modules.conv.Conv [256, 256, 1, 1]
13 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
14 7 1 131584 ultralytics.nn.modules.conv.Conv [512, 256, 1, 1, None, 1, 1, False]
15 [-2, -1] 1 0 ultralytics.nn.modules.conv.Concat [1]
16 -1 3 2232320 ultralytics.nn.modules.block.RepC3 [512, 256, 3]
17 -1 1 66048 ultralytics.nn.modules.conv.Conv [256, 256, 1, 1]
18 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
19 3 1 131584 ultralytics.nn.modules.conv.Conv [512, 256, 1, 1, None, 1, 1, False]
20 [-2, -1] 1 0 ultralytics.nn.modules.conv.Concat [1]
21 -1 3 2232320 ultralytics.nn.modules.block.RepC3 [512, 256, 3]
22 -1 1 590336 ultralytics.nn.modules.conv.Conv [256, 256, 3, 2]
23 [-1, 17] 1 0 ultralytics.nn.modules.conv.Concat [1]
24 -1 3 2232320 ultralytics.nn.modules.block.RepC3 [512, 256, 3]
25 -1 1 590336 ultralytics.nn.modules.conv.Conv [256, 256, 3, 2]
26 [-1, 12] 1 0 ultralytics.nn.modules.conv.Concat [1]
27 -1 3 2232320 ultralytics.nn.modules.block.RepC3 [512, 256, 3]
28 [21, 24, 27] 1 7303907 ultralytics.nn.modules.head.RTDETRDecoder [1, [256, 256, 256]]
rtdetr-l-AKConv summary: 607 layers, 51,311,095 parameters, 51,311,095 gradients, 167.2 GFLOPs
rtdetr-ResNetLayer_RFAConv ：
rtdetr-ResNetLayer_RFAConv summary: 705 layers, 83,409,699 parameters, 83,409,699 gradients, 240.7 GFLOPs
from n params module arguments
0 -1 1 9536 ultralytics.nn.AddModules.RFAConv.ResNetLayer_RFAConv[3, 64, 1, True, 1]
1 -1 1 629760 ultralytics.nn.AddModules.RFAConv.ResNetLayer_RFAConv[64, 64, 1, False, 3]
2 -1 1 3372032 ultralytics.nn.AddModules.RFAConv.ResNetLayer_RFAConv[256, 128, 2, False, 4]
3 -1 1 19847168 ultralytics.nn.AddModules.RFAConv.ResNetLayer_RFAConv[512, 256, 2, False, 6]
4 -1 1 40296448 ultralytics.nn.AddModules.RFAConv.ResNetLayer_RFAConv[1024, 512, 2, False, 3]
5 -1 1 524800 ultralytics.nn.modules.conv.Conv [2048, 256, 1, 1, None, 1, 1, False]
6 -1 1 789760 ultralytics.nn.modules.transformer.AIFI [256, 1024, 8]
7 -1 1 66048 ultralytics.nn.modules.conv.Conv [256, 256, 1, 1]
8 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
9 3 1 262656 ultralytics.nn.modules.conv.Conv [1024, 256, 1, 1, None, 1, 1, False]
10 [-2, -1] 1 0 ultralytics.nn.modules.conv.Concat [1]
11 -1 3 2232320 ultralytics.nn.modules.block.RepC3 [512, 256, 3]
12 -1 1 66048 ultralytics.nn.modules.conv.Conv [256, 256, 1, 1]
13 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
14 2 1 131584 ultralytics.nn.modules.conv.Conv [512, 256, 1, 1, None, 1, 1, False]
15 [-2, -1] 1 0 ultralytics.nn.modules.conv.Concat [1]
16 -1 3 2232320 ultralytics.nn.modules.block.RepC3 [512, 256, 3]
17 -1 1 590336 ultralytics.nn.modules.conv.Conv [256, 256, 3, 2]
18 [-1, 12] 1 0 ultralytics.nn.modules.conv.Concat [1]
19 -1 3 2232320 ultralytics.nn.modules.block.RepC3 [512, 256, 3]
20 -1 1 590336 ultralytics.nn.modules.conv.Conv [256, 256, 3, 2]
21 [-1, 7] 1 0 ultralytics.nn.modules.conv.Concat [1]
22 -1 3 2232320 ultralytics.nn.modules.block.RepC3 [512, 256, 3]
23 [16, 19, 22] 1 7303907 ultralytics.nn.modules.head.RTDETRDecoder [1, [256, 256, 256]]
rtdetr-ResNetLayer_RFAConv summary: 705 layers, 83,409,699 parameters, 83,409,699 gradients, 240.7 GFLOPs