RT-DETR改进策略【Conv和Transformer】| ECCV-2024 Histogram Transformer 直方图自注意力 适用于噪声大，图像质量低的检测任务
一、本文介绍
本文记录的是
利用
直方图自注意力
优化
RT-DETR
的目标检测方法研究
。
在目标检测任务中，清晰准确的图像对于目标检测至关重要，本文创新方法通过恢复图像质量，可以减少因图像质量低导致的误检和漏检，实现有效涨点
。
二、直方图自注意力介绍
2.1 设计出发点
- 解决现有Transformer方法的局限 ：现有的基于Transformer的方法在处理恶劣天气图像恢复时，为了提高内存利用效率，通常将自注意力操作限制在固定的空间范围或仅仅在通道维度上，这种限制阻碍了Transformer对长距离空间特征的捕捉能力，从而影响了图像恢复的性能。
- 利用天气退化特征 ：观察到天气引起的退化因素主要导致相似的遮挡和亮度变化，因此希望设计一种能够更好地处理这些特征的模块。
2.2 原理
2.2.1 动态范围直方图自注意力（DHSA）
- 动态范围卷积 ：传统卷积操作的感受野范围有限，主要关注局部信息，与自注意力机制的长距离依赖建模能力不匹配。通过在传统卷积操作之前对输入特征进行重新排序，将其分为两个分支，对第一个分支的特征进行水平和垂直排序，然后与第二个分支的特征连接，再通过可分离卷积。这样可以将高强度和低强度的像素组织成矩阵对角线上的规则模式，使卷积能够在动态范围内进行计算，从而部分聚焦于保留干净信息和分别恢复退化特征。
- 直方图自注意力机制 ：注意到天气引起的退化会导致相似的模式，不同强度的包含背景特征或天气退化的像素应给予不同程度的注意力。因此提出将空间元素分类到不同的bin中，并在bin内和bin间分配不同的注意力。
2.2.2 双尺度门控前馈（DGFF）模块
- 考虑到之前的方法在标准前馈网络中通常使用单范围或单尺度卷积来增强局部上下文，但忽略了动态分布的天气引起的退化之间的相关性。因此设计了DGFF模块，它在传输过程中集成了两个不同的多范围和多尺度深度卷积路径，通过不同的卷积操作和门控机制来增强对多尺度和多范围信息的提取能力。
2.3 结构
-
包含两个关键模块
- DHSA模块 ：由动态范围卷积和直方图自注意力机制组成。动态范围卷积对输入特征进行重新排序，直方图自注意力机制对重新排序后的特征进行处理，包括将特征分为Value特征和Query - Key对，对Value特征进行排序并根据其索引对Query - Key对进行排列，然后将特征重塑为两种类型（bin - wise直方图重塑和frequency - wise直方图重塑），分别通过两种重塑方式和后续的自注意力过程，最后将输出元素相乘得到最终输出。
- DGFF模块 ：输入张量首先经过点卷积操作增加通道维度，然后分为两个并行分支。在特征转换过程中，一个分支使用5×5深度卷积，另一个分支使用扩张的3×3深度卷积来增强多范围和多尺度信息的提取。第二个分支的输出经过激活后作为门控图作用于第一个分支，最后通过像素重排和逆重排操作以及点卷积得到输出并传递到下一个阶段。
2.4 优势
-
有效捕捉动态范围的特征
：
DHSA模块通过动态范围卷积和直方图自注意力机制，能够有效地捕捉天气引起的动态空间退化特征，实现对长距离空间特征的建模，克服了现有方法的局限性。 -
提取多尺度和多范围信息
：
DGFF模块通过集成两个不同的多范围和多尺度深度卷积路径，能够更好地提取图像中的多尺度和多范围信息，增强了对天气退化图像的恢复能力。 - 提高图像恢复性能 ：通过上述两个模块的协同作用， Histogram Transformer Block 能够提高恶劣天气图像恢复的性能，在多个数据集上取得了较好的效果。
论文： https://arxiv.org/pdf/2407.10172
源码： https://github.com/sunshangquan/Histoformer
三、HTB的实现代码
HTB模块
的实现代码如下：
import numbers
import torch
import torch.nn as nn
import torch.nn.functional as F
from einops import rearrange
from ultralytics.nn.modules.conv import LightConv
Conv2d = nn.Conv2d
## Layer Norm
def to_2d(x):
return rearrange(x, 'b c h w -> b (h w c)')
def to_3d(x):
# return rearrange(x, 'b c h w -> b c (h w)')
return rearrange(x, 'b c h w -> b (h w) c')
def to_4d(x,h,w):
# return rearrange(x, 'b c (h w) -> b c h w',h=h,w=w)
return rearrange(x, 'b (h w) c -> b c h w',h=h,w=w)
class BiasFree_LayerNorm(nn.Module):
def __init__(self, normalized_shape):
super(BiasFree_LayerNorm, self).__init__()
if isinstance(normalized_shape, numbers.Integral):
normalized_shape = (normalized_shape,)
normalized_shape = torch.Size(normalized_shape)
assert len(normalized_shape) == 1
self.normalized_shape = normalized_shape
def forward(self, x):
sigma = x.var(-1, keepdim=True, unbiased=False)
return x / torch.sqrt(sigma+1e-5) #* self.weight
class WithBias_LayerNorm(nn.Module):
def __init__(self, normalized_shape):
super(WithBias_LayerNorm, self).__init__()
if isinstance(normalized_shape, numbers.Integral):
normalized_shape = (normalized_shape,)
normalized_shape = torch.Size(normalized_shape)
assert len(normalized_shape) == 1
self.normalized_shape = normalized_shape
def forward(self, x):
mu = x.mean(-1, keepdim=True)
sigma = x.var(-1, keepdim=True, unbiased=False)
return (x - mu) / torch.sqrt(sigma+1e-5) #* self.weight + self.bias
class LayerNorm(nn.Module):
def __init__(self, dim, LayerNorm_type="WithBias"):
super(LayerNorm, self).__init__()
if LayerNorm_type =='BiasFree':
self.body = BiasFree_LayerNorm(dim)
else:
self.body = WithBias_LayerNorm(dim)
def forward(self, x):
h, w = x.shape[-2:]
return to_4d(self.body(to_3d(x)), h, w)
## Dual-scale Gated Feed-Forward Network (DGFF)
class FeedForward(nn.Module):
def __init__(self, dim, ffn_expansion_factor, bias):
super(FeedForward, self).__init__()
hidden_features = int(dim * ffn_expansion_factor)
self.project_in = Conv2d(dim, hidden_features * 2, kernel_size=1, bias=bias)
self.dwconv_5 = Conv2d(hidden_features // 4, hidden_features // 4, kernel_size=5, stride=1, padding=2,
groups=hidden_features // 4, bias=bias)
self.dwconv_dilated2_1 = Conv2d(hidden_features // 4, hidden_features // 4, kernel_size=3, stride=1, padding=2,
groups=hidden_features // 4, bias=bias, dilation=2)
self.p_unshuffle = nn.PixelUnshuffle(2)
self.p_shuffle = nn.PixelShuffle(2)
self.project_out = Conv2d(hidden_features, dim, kernel_size=1, bias=bias)
def forward(self, x):
x = self.project_in(x)
x = self.p_shuffle(x)
x1, x2 = x.chunk(2, dim=1)
x1 = self.dwconv_5(x1)
x2 = self.dwconv_dilated2_1(x2)
x = F.mish(x2) * x1
x = self.p_unshuffle(x)
x = self.project_out(x)
return x
##Dynamic-range Histogram Self-Attention (DHSA)
class Attention_histogram(nn.Module):
def __init__(self, dim, num_heads=4, bias=False, ifBox=True):
super(Attention_histogram, self).__init__()
self.factor = num_heads
self.ifBox = ifBox
self.num_heads = num_heads
self.temperature = nn.Parameter(torch.ones(num_heads, 1, 1))
self.qkv = Conv2d(dim, dim * 5, kernel_size=1, bias=bias)
self.qkv_dwconv = Conv2d(dim * 5, dim * 5, kernel_size=3, stride=1, padding=1, groups=dim * 5, bias=bias)
self.project_out = Conv2d(dim, dim, kernel_size=1, bias=bias)
def pad(self, x, factor):
hw = x.shape[-1]
t_pad = [0, 0] if hw % factor == 0 else [0, (hw // factor + 1) * factor - hw]
x = F.pad(x, t_pad, 'constant', 0)
return x, t_pad
def unpad(self, x, t_pad):
_, _, hw = x.shape
return x[:, :, t_pad[0]:hw - t_pad[1]]
def softmax_1(self, x, dim=-1):
logit = x.exp()
logit = logit / (logit.sum(dim, keepdim=True) + 1)
return logit
def normalize(self, x):
mu = x.mean(-2, keepdim=True)
sigma = x.var(-2, keepdim=True, unbiased=False)
return (x - mu) / torch.sqrt(sigma + 1e-5) # * self.weight + self.bias
def reshape_attn(self, q, k, v, ifBox):
b, c = q.shape[:2]
q, t_pad = self.pad(q, self.factor)
k, t_pad = self.pad(k, self.factor)
v, t_pad = self.pad(v, self.factor)
hw = q.shape[-1] // self.factor
shape_ori = "b (head c) (factor hw)" if ifBox else "b (head c) (hw factor)"
shape_tar = "b head (c factor) hw"
q = rearrange(q, '{} -> {}'.format(shape_ori, shape_tar), factor=self.factor, hw=hw, head=self.num_heads)
k = rearrange(k, '{} -> {}'.format(shape_ori, shape_tar), factor=self.factor, hw=hw, head=self.num_heads)
v = rearrange(v, '{} -> {}'.format(shape_ori, shape_tar), factor=self.factor, hw=hw, head=self.num_heads)
q = torch.nn.functional.normalize(q, dim=-1)
k = torch.nn.functional.normalize(k, dim=-1)
attn = (q @ k.transpose(-2, -1)) * self.temperature
attn = self.softmax_1(attn, dim=-1)
out = (attn @ v)
out = rearrange(out, '{} -> {}'.format(shape_tar, shape_ori), factor=self.factor, hw=hw, b=b,
head=self.num_heads)
out = self.unpad(out, t_pad)
return out
def forward(self, x):
b, c, h, w = x.shape
x_sort, idx_h = x[:, :c // 2].sort(-2)
x_sort, idx_w = x_sort.sort(-1)
x[:, :c // 2] = x_sort
qkv = self.qkv_dwconv(self.qkv(x))
q1, k1, q2, k2, v = qkv.chunk(5, dim=1) # b,c,x,x
v, idx = v.view(b, c, -1).sort(dim=-1)
q1 = torch.gather(q1.view(b, c, -1), dim=2, index=idx)
k1 = torch.gather(k1.view(b, c, -1), dim=2, index=idx)
q2 = torch.gather(q2.view(b, c, -1), dim=2, index=idx)
k2 = torch.gather(k2.view(b, c, -1), dim=2, index=idx)
out1 = self.reshape_attn(q1, k1, v, True)
out2 = self.reshape_attn(q2, k2, v, False)
out1 = torch.scatter(out1, 2, idx, out1).view(b, c, h, w)
out2 = torch.scatter(out2, 2, idx, out2).view(b, c, h, w)
out = out1 * out2
out = self.project_out(out)
out_replace = out[:, :c // 2]
out_replace = torch.scatter(out_replace, -1, idx_w, out_replace)
out_replace = torch.scatter(out_replace, -2, idx_h, out_replace)
out[:, :c // 2] = out_replace
return out
##Histogram Transformer Block (HTB)
class HTB(nn.Module):
def __init__(self, dim, num_heads=1, ffn_expansion_factor=2.5, bias=False, LayerNorm_type='WithBias'):## Other option 'BiasFree'
super(HTB, self).__init__()
self.attn_g = Attention_histogram(dim, num_heads, bias, True)
self.norm_g = LayerNorm(dim, LayerNorm_type)
self.ffn = FeedForward(dim, ffn_expansion_factor, bias)
self.norm_ff1 = LayerNorm(dim, LayerNorm_type)
def forward(self, x):
x = x + self.attn_g(self.norm_g(x))
x_out = x + self.ffn(self.norm_ff1(x))
return x_out
def autopad(k, p=None, d=1): # kernel, padding, dilation
"""Pad to 'same' shape outputs."""
if d > 1:
k = d * (k - 1) + 1 if isinstance(k, int) else [d * (x - 1) + 1 for x in k] # actual kernel-size
if p is None:
p = k // 2 if isinstance(k, int) else [x // 2 for x in k] # auto-pad
return p
class Conv(nn.Module):
"""Standard convolution with args(ch_in, ch_out, kernel, stride, padding, groups, dilation, activation)."""
default_act = nn.SiLU() # default activation
def __init__(self, c1, c2, k=1, s=1, p=None, g=1, d=1, act=True):
"""Initialize Conv layer with given arguments including activation."""
super().__init__()
self.conv = nn.Conv2d(c1, c2, k, s, autopad(k, p, d), groups=g, dilation=d, bias=False)
self.bn = nn.BatchNorm2d(c2)
self.act = self.default_act if act is True else act if isinstance(act, nn.Module) else nn.Identity()
def forward(self, x):
"""Apply convolution, batch normalization and activation to input tensor."""
return self.act(self.bn(self.conv(x)))
def forward_fuse(self, x):
"""Perform transposed convolution of 2D data."""
return self.act(self.conv(x))
class HGBlock_HTB(nn.Module):
"""
HG_Block of PPHGNetV2 with 2 convolutions and LightConv.
https://github.com/PaddlePaddle/PaddleDetection/blob/develop/ppdet/modeling/backbones/hgnet_v2.py
"""
def __init__(self, c1, cm, c2, k=3, n=6, lightconv=False, shortcut=False, act=nn.ReLU()):
"""Initializes a CSP Bottleneck with 1 convolution using specified input and output channels."""
super().__init__()
block = LightConv if lightconv else Conv
self.m = nn.ModuleList(block(c1 if i == 0 else cm, cm, k=k, act=act) for i in range(n))
self.sc = Conv(c1 + n * cm, c2 // 2, 1, 1, act=act) # squeeze conv
self.ec = Conv(c2 // 2, c2, 1, 1, act=act) # excitation conv
self.add = shortcut and c1 == c2
self.cv = HTB(c2)
def forward(self, x):
"""Forward pass of a PPHGNetV2 backbone layer."""
y = [x]
y.extend(m(y[-1]) for m in self.m)
y = self.cv(self.ec(self.sc(torch.cat(y, 1))))
return y + x if self.add else y
四、创新模块
4.1 改进点⭐
模块改进方法
：直接加入
HTB
（
第五节讲解添加步骤
）。
HTB
模块加入如下：
4.2 改进点⭐
模块改进方法
：基于
HTB模块
的
HGBlock
（
第五节讲解添加步骤
）。
第二种改进方法是对
RT-DETR
中的
HGBlock模块
进行改进，并将
HTB
在加入到
HGBlock
模块中。
改进代码如下：
对
HGBlock
模块进行改进，加入
HTB模块
，并重命名为
HGBlock_HTB
class HGBlock_HTB(nn.Module):
"""
HG_Block of PPHGNetV2 with 2 convolutions and LightConv.
https://github.com/PaddlePaddle/PaddleDetection/blob/develop/ppdet/modeling/backbones/hgnet_v2.py
"""
def __init__(self, c1, cm, c2, k=3, n=6, lightconv=False, shortcut=False, act=nn.ReLU()):
"""Initializes a CSP Bottleneck with 1 convolution using specified input and output channels."""
super().__init__()
block = LightConv if lightconv else Conv
self.m = nn.ModuleList(block(c1 if i == 0 else cm, cm, k=k, act=act) for i in range(n))
self.sc = Conv(c1 + n * cm, c2 // 2, 1, 1, act=act) # squeeze conv
self.ec = Conv(c2 // 2, c2, 1, 1, act=act) # excitation conv
self.add = shortcut and c1 == c2
self.cv = HTB(c2)
def forward(self, x):
"""Forward pass of a PPHGNetV2 backbone layer."""
y = [x]
y.extend(m(y[-1]) for m in self.m)
y = self.cv(self.ec(self.sc(torch.cat(y, 1))))
return y + x if self.add else y
注意❗：在
第五小节
中需要声明的模块名称为：
HTB
和
HGBlock_HTB
。
五、添加步骤
5.1 修改一
① 在
ultralytics/nn/
目录下新建
AddModules
文件夹用于存放模块代码
② 在
AddModules
文件夹下新建
HTB.py
，将
第三节
中的代码粘贴到此处
5.2 修改二
在
AddModules
文件夹下新建
__init__.py
（已有则不用新建），在文件内导入模块：
from .HTB import *
5.3 修改三
在
ultralytics/nn/modules/tasks.py
文件中，需要在指定位置添加各模块类名称。
首先：导入模块
其次：在
parse_model函数
中注册
HTB
和
HGBlock_HTB
模块
六、yaml模型文件
6.1 模型改进版本1⭐
此处以
ultralytics/cfg/models/rt-detr/rtdetr-l.yaml
为例，在同目录下创建一个用于自己数据集训练的模型文件
rtdetr-l-HTB.yaml
。
将
rtdetr-l.yaml
中的内容复制到
rtdetr-l-HTB.yaml
文件下，修改
nc
数量等于自己数据中目标的数量。
📌 模型的修改方法是将
颈部网络
中的部分
Conv
模块替换成
HTB模块
。
# Ultralytics YOLO 🚀, AGPL-3.0 license
# RT-DETR-l object detection model with P3-P5 outputs. For details see https://docs.ultralytics.com/models/rtdetr
# Parameters
nc: 1 # number of classes
scales: # model compound scaling constants, i.e. 'model=yolov8n-cls.yaml' will call yolov8-cls.yaml with scale 'n'
# [depth, width, max_channels]
l: [1.00, 1.00, 1024]
backbone:
# [from, repeats, module, args]
- [-1, 1, HGStem, [32, 48]] # 0-P2/4
- [-1, 6, HGBlock, [48, 128, 3]] # stage 1
- [-1, 1, DWConv, [128, 3, 2, 1, False]] # 2-P3/8
- [-1, 6, HGBlock, [96, 512, 3]] # stage 2
- [-1, 1, DWConv, [512, 3, 2, 1, False]] # 4-P4/16
- [-1, 6, HGBlock, [192, 1024, 5, True, False]] # cm, c2, k, light, shortcut
- [-1, 6, HGBlock, [192, 1024, 5, True, True]]
- [-1, 6, HGBlock, [192, 1024, 5, True, True]] # stage 3
- [-1, 1, DWConv, [1024, 3, 2, 1, False]] # 8-P5/32
- [-1, 6, HGBlock, [384, 2048, 5, True, False]] # stage 4
head:
- [-1, 1, Conv, [256, 1, 1, None, 1, 1, False]] # 10 input_proj.2
- [-1, 1, AIFI, [1024, 8]]
- [-1, 1, HTB, [256]] # 12, Y5, lateral_convs.0
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [7, 1, Conv, [256, 1, 1, None, 1, 1, False]] # 14 input_proj.1
- [[-2, -1], 1, Concat, [1]]
- [-1, 3, RepC3, [256]] # 16, fpn_blocks.0
- [-1, 1, HTB, [256]] # 17, Y4, lateral_convs.1
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [3, 1, Conv, [256, 1, 1, None, 1, 1, False]] # 19 input_proj.0
- [[-2, -1], 1, Concat, [1]] # cat backbone P4
- [-1, 3, RepC3, [256]] # X3 (21), fpn_blocks.1
- [-1, 1, Conv, [256, 3, 2]] # 22, downsample_convs.0
- [[-1, 17], 1, Concat, [1]] # cat Y4
- [-1, 3, RepC3, [256]] # F4 (24), pan_blocks.0
- [-1, 1, Conv, [256, 3, 2]] # 25, downsample_convs.1
- [[-1, 12], 1, Concat, [1]] # cat Y5
- [-1, 3, RepC3, [256]] # F5 (27), pan_blocks.1
- [[21, 24, 27], 1, RTDETRDecoder, [nc]] # Detect(P3, P4, P5)
6.2 模型改进版本2⭐
此处以
ultralytics/cfg/models/rt-detr/rtdetr-l.yaml
为例，在同目录下创建一个用于自己数据集训练的模型文件
rtdetr-l-rtdetr-l-HGBlock_HTB.yaml
。
将
rtdetr-l.yaml
中的内容复制到
rtdetr-l-C3k2_HTB.yaml
文件下，修改
nc
数量等于自己数据中目标的数量。
📌 模型的修改方法是将
骨干网络
中的
HGBlock_HTB模块
替换成
rtdetr-l-HGBlock_HTB模块
。
# Ultralytics YOLO 🚀, AGPL-3.0 license
# RT-DETR-l object detection model with P3-P5 outputs. For details see https://docs.ultralytics.com/models/rtdetr
# Parameters
nc: 1 # number of classes
scales: # model compound scaling constants, i.e. 'model=yolov8n-cls.yaml' will call yolov8-cls.yaml with scale 'n'
# [depth, width, max_channels]
l: [1.00, 1.00, 1024]
backbone:
# [from, repeats, module, args]
- [-1, 1, HGStem, [32, 48]] # 0-P2/4
- [-1, 6, HGBlock, [48, 128, 3]] # stage 1
- [-1, 1, DWConv, [128, 3, 2, 1, False]] # 2-P3/8
- [-1, 6, HGBlock, [96, 512, 3]] # stage 2
- [-1, 1, DWConv, [512, 3, 2, 1, False]] # 4-P4/16
- [-1, 6, HGBlock_HTB, [192, 1024, 5, True, False]] # cm, c2, k, light, shortcut
- [-1, 6, HGBlock_HTB, [192, 1024, 5, True, True]]
- [-1, 6, HGBlock_HTB, [192, 1024, 5, True, True]] # stage 3
- [-1, 1, DWConv, [1024, 3, 2, 1, False]] # 8-P5/32
- [-1, 6, HGBlock, [384, 2048, 5, True, False]] # stage 4
head:
- [-1, 1, Conv, [256, 1, 1, None, 1, 1, False]] # 10 input_proj.2
- [-1, 1, AIFI, [1024, 8]]
- [-1, 1, Conv, [256, 1, 1]] # 12, Y5, lateral_convs.0
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [7, 1, Conv, [256, 1, 1, None, 1, 1, False]] # 14 input_proj.1
- [[-2, -1], 1, Concat, [1]]
- [-1, 3, RepC3, [256]] # 16, fpn_blocks.0
- [-1, 1, Conv, [256, 1, 1]] # 17, Y4, lateral_convs.1
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [3, 1, Conv, [256, 1, 1, None, 1, 1, False]] # 19 input_proj.0
- [[-2, -1], 1, Concat, [1]] # cat backbone P4
- [-1, 3, RepC3, [256]] # X3 (21), fpn_blocks.1
- [-1, 1, Conv, [256, 3, 2]] # 22, downsample_convs.0
- [[-1, 17], 1, Concat, [1]] # cat Y4
- [-1, 3, RepC3, [256]] # F4 (24), pan_blocks.0
- [-1, 1, Conv, [256, 3, 2]] # 25, downsample_convs.1
- [[-1, 12], 1, Concat, [1]] # cat Y5
- [-1, 3, RepC3, [256]] # F5 (27), pan_blocks.1
- [[21, 24, 27], 1, RTDETRDecoder, [nc]] # Detect(P3, P4, P5)
七、成功运行结果
打印网络模型可以看到
HTB
和
HGBlock_HTB
已经加入到模型中，并可以进行训练了。
rtdetr-l-HTB ：
rtdetr-l-HTB summary: 707 layers, 34,479,939 parameters, 34,479,939 gradients, 111.4 GFLOPs
from n params module arguments
0 -1 1 25248 ultralytics.nn.modules.block.HGStem [3, 32, 48]
1 -1 6 155072 ultralytics.nn.modules.block.HGBlock [48, 48, 128, 3, 6]
2 -1 1 1408 ultralytics.nn.modules.conv.DWConv [128, 128, 3, 2, 1, False]
3 -1 6 839296 ultralytics.nn.modules.block.HGBlock [128, 96, 512, 3, 6]
4 -1 1 5632 ultralytics.nn.modules.conv.DWConv [512, 512, 3, 2, 1, False]
5 -1 6 1695360 ultralytics.nn.modules.block.HGBlock [512, 192, 1024, 5, 6, True, False]
6 -1 6 2055808 ultralytics.nn.modules.block.HGBlock [1024, 192, 1024, 5, 6, True, True]
7 -1 6 2055808 ultralytics.nn.modules.block.HGBlock [1024, 192, 1024, 5, 6, True, True]
8 -1 1 11264 ultralytics.nn.modules.conv.DWConv [1024, 1024, 3, 2, 1, False]
9 -1 6 6708480 ultralytics.nn.modules.block.HGBlock [1024, 384, 2048, 5, 6, True, False]
10 -1 1 524800 ultralytics.nn.modules.conv.Conv [2048, 256, 1, 1, None, 1, 1, False]
11 -1 1 789760 ultralytics.nn.modules.transformer.AIFI [256, 1024, 8]
12 -1 1 901952 ultralytics.nn.AddModules.HTB.HTB [256, 256]
13 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
14 7 1 262656 ultralytics.nn.modules.conv.Conv [1024, 256, 1, 1, None, 1, 1, False]
15 [-2, -1] 1 0 ultralytics.nn.modules.conv.Concat [1]
16 -1 3 2232320 ultralytics.nn.modules.block.RepC3 [512, 256, 3]
17 -1 1 901952 ultralytics.nn.AddModules.HTB.HTB [256, 256]
18 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
19 3 1 131584 ultralytics.nn.modules.conv.Conv [512, 256, 1, 1, None, 1, 1, False]
20 [-2, -1] 1 0 ultralytics.nn.modules.conv.Concat [1]
21 -1 3 2232320 ultralytics.nn.modules.block.RepC3 [512, 256, 3]
22 -1 1 590336 ultralytics.nn.modules.conv.Conv [256, 256, 3, 2]
23 [-1, 17] 1 0 ultralytics.nn.modules.conv.Concat [1]
24 -1 3 2232320 ultralytics.nn.modules.block.RepC3 [512, 256, 3]
25 -1 1 590336 ultralytics.nn.modules.conv.Conv [256, 256, 3, 2]
26 [-1, 12] 1 0 ultralytics.nn.modules.conv.Concat [1]
27 -1 3 2232320 ultralytics.nn.modules.block.RepC3 [512, 256, 3]
28 [21, 24, 27] 1 7303907 ultralytics.nn.modules.head.RTDETRDecoder [1, [256, 256, 256]]
rtdetr-l-HTB summary: 707 layers, 34,479,939 parameters, 34,479,939 gradients, 111.4 GFLOPs
**rtdetr-l-HGBlock_HTB **：
rtdetr-l-HGBlock_HTB summary: 730 layers, 75,478,982 parameters, 75,478,982 gradients, 245.2 GFLOPs
from n params module arguments
0 -1 1 25248 ultralytics.nn.modules.block.HGStem [3, 32, 48]
1 -1 6 155072 ultralytics.nn.modules.block.HGBlock [48, 48, 128, 3, 6]
2 -1 1 1408 ultralytics.nn.modules.conv.DWConv [128, 128, 3, 2, 1, False]
3 -1 6 839296 ultralytics.nn.modules.block.HGBlock [128, 96, 512, 3, 6]
4 -1 1 5632 ultralytics.nn.modules.conv.DWConv [512, 512, 3, 2, 1, False]
5 -1 6 15918977 ultralytics.nn.AddModules.HTB.HGBlock_HTB [512, 192, 1024, 5, 6, True, False]
6 -1 6 16279425 ultralytics.nn.AddModules.HTB.HGBlock_HTB [1024, 192, 1024, 5, 6, True, True]
7 -1 6 16279425 ultralytics.nn.AddModules.HTB.HGBlock_HTB [1024, 192, 1024, 5, 6, True, True]
8 -1 1 11264 ultralytics.nn.modules.conv.DWConv [1024, 1024, 3, 2, 1, False]
9 -1 6 6708480 ultralytics.nn.modules.block.HGBlock [1024, 384, 2048, 5, 6, True, False]
10 -1 1 524800 ultralytics.nn.modules.conv.Conv [2048, 256, 1, 1, None, 1, 1, False]
11 -1 1 789760 ultralytics.nn.modules.transformer.AIFI [256, 1024, 8]
12 -1 1 66048 ultralytics.nn.modules.conv.Conv [256, 256, 1, 1]
13 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
14 7 1 262656 ultralytics.nn.modules.conv.Conv [1024, 256, 1, 1, None, 1, 1, False]
15 [-2, -1] 1 0 ultralytics.nn.modules.conv.Concat [1]
16 -1 3 2232320 ultralytics.nn.modules.block.RepC3 [512, 256, 3]
17 -1 1 66048 ultralytics.nn.modules.conv.Conv [256, 256, 1, 1]
18 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
19 3 1 131584 ultralytics.nn.modules.conv.Conv [512, 256, 1, 1, None, 1, 1, False]
20 [-2, -1] 1 0 ultralytics.nn.modules.conv.Concat [1]
21 -1 3 2232320 ultralytics.nn.modules.block.RepC3 [512, 256, 3]
22 -1 1 590336 ultralytics.nn.modules.conv.Conv [256, 256, 3, 2]
23 [-1, 17] 1 0 ultralytics.nn.modules.conv.Concat [1]
24 -1 3 2232320 ultralytics.nn.modules.block.RepC3 [512, 256, 3]
25 -1 1 590336 ultralytics.nn.modules.conv.Conv [256, 256, 3, 2]
26 [-1, 12] 1 0 ultralytics.nn.modules.conv.Concat [1]
27 -1 3 2232320 ultralytics.nn.modules.block.RepC3 [512, 256, 3]
28 [21, 24, 27] 1 7303907 ultralytics.nn.modules.head.RTDETRDecoder [1, [256, 256, 256]]
rtdetr-l-HGBlock_HTB summary: 730 layers, 75,478,982 parameters, 75,478,982 gradients, 245.2 GFLOPs