学习资源站

RT-DETR改进策略【Head】增加针对大目标的检测层（四个检测头）_rt-detr检测头改进-

RT-DETR改进策略【Head】| 增加针对 大目标 的检测层 （四个检测头）

前言

在目标检测领域， 大目标在图像中所占比例大 ，下采样不足容易缺少大目标的信息，其检测同样是一个具有挑战性的问题。本文将介绍如何在 RT-DETR 中 添加大目标检测层，以提高对大目标的检测能力。

一、RT-DETR原始模型结构介绍

rtdetr-l 原始模型结构如下： 

# Ultralytics YOLO 🚀, AGPL-3.0 license
# RT-DETR-l object detection model with P3-P5 outputs. For details see https://docs.ultralytics.com/models/rtdetr

# Parameters
nc: 1 # number of classes
scales: # model compound scaling constants, i.e. 'model=yolov8n-cls.yaml' will call yolov8-cls.yaml with scale 'n'
  # [depth, width, max_channels]
  l: [1.00, 1.00, 1024]

backbone:
  # [from, repeats, module, args]
  - [-1, 1, HGStem, [32, 48]] # 0-P2/4
  - [-1, 6, HGBlock, [48, 128, 3]] # stage 1

  - [-1, 1, DWConv, [128, 3, 2, 1, False]] # 2-P3/8
  - [-1, 6, HGBlock, [96, 512, 3]] # stage 2

  - [-1, 1, DWConv, [512, 3, 2, 1, False]] # 4-P4/16
  - [-1, 6, HGBlock, [192, 1024, 5, True, False]] # cm, c2, k, light, shortcut
  - [-1, 6, HGBlock, [192, 1024, 5, True, True]]
  - [-1, 6, HGBlock, [192, 1024, 5, True, True]] # stage 3

  - [-1, 1, DWConv, [1024, 3, 2, 1, False]] # 8-P5/32
  - [-1, 6, HGBlock, [384, 2048, 5, True, False]] # stage 4

head:
  - [-1, 1, Conv, [256, 1, 1, None, 1, 1, False]] # 10 input_proj.2
  - [-1, 1, AIFI, [1024, 8]]
  - [-1, 1, Conv, [256, 1, 1]] # 12, Y5, lateral_convs.0

  - [-1, 1, nn.Upsample, [None, 2, "nearest"]]
  - [7, 1, Conv, [256, 1, 1, None, 1, 1, False]] # 14 input_proj.1
  - [[-2, -1], 1, Concat, [1]]
  - [-1, 3, RepC3, [256]] # 16, fpn_blocks.0
  - [-1, 1, Conv, [256, 1, 1]] # 17, Y4, lateral_convs.1

  - [-1, 1, nn.Upsample, [None, 2, "nearest"]]
  - [3, 1, Conv, [256, 1, 1, None, 1, 1, False]] # 19 input_proj.0
  - [[-2, -1], 1, Concat, [1]] # cat backbone P4
  - [-1, 3, RepC3, [256]] # X3 (21), fpn_blocks.1

  - [-1, 1, Conv, [256, 3, 2]] # 22, downsample_convs.0
  - [[-1, 17], 1, Concat, [1]] # cat Y4
  - [-1, 3, RepC3, [256]] # F4 (24), pan_blocks.0

  - [-1, 1, Conv, [256, 3, 2]] # 25, downsample_convs.1
  - [[-1, 12], 1, Concat, [1]] # cat Y5
  - [-1, 3, RepC3, [256]] # F5 (27), pan_blocks.1

  - [[21, 24, 27], 1, RTDETRDecoder, [nc]] # Detect(P3, P4, P5)

二、有效特征层对应的检测头类别

2.1 P3/8 - small检测头

  • 原始模型中的 P3/8特征层 对应的检测头主要用于检测相对较小的目标。其特征图大小相对较大，空间分辨率较高。
  • 适合检测尺寸大概在 8x8 到 16x16 像素左右的目标。

2.2 P4/16 - medium检测头

  • 这个检测头对应的 P4/16特征层 经过了更多的下采样操作，相比P3/8特征图空间分辨率降低，但通道数增加，特征更抽象且有语义信息。
  • 它主要用于检测中等大小的目标，尺寸范围大概在 16x16 到 32x32 像素左右。

2.3 P5/32 - large检测头

  • P5/32 是经过最多下采样操作得到的特征层，其空间分辨率最低，但语义信息最强、全局感受野最大。
  • 该检测头适合检测较大尺寸的目标，一般是尺寸在 32x32 像素以上的目标。

2.4 新添加针对大目标的检测头

  • 新添加的检测头主要用于检测更大尺寸的目标。尺寸在 64x64 像素以上的超大目标。

💡这是因为在目标检测任务中，随着目标尺寸的增大，需要更能关注到整体轮廓的特征图来有效捕捉大目标特征。

三、添加大目标的检测层后的网络结构 

# Ultralytics YOLO 🚀, AGPL-3.0 license
# RT-DETR-l object detection model with P3-P5 outputs. For details see https://docs.ultralytics.com/models/rtdetr

# Parameters
nc: 1  # number of classes
scales: # model compound scaling constants, i.e. 'model=yolov8n-cls.yaml' will call yolov8-cls.yaml with scale 'n'
  # [depth, width, max_channels]
  l: [1.00, 1.00, 1024]

backbone:
  # [from, repeats, module, args]
  - [-1, 1, HGStem, [32, 48]]  # 0-P2/4
  - [-1, 6, HGBlock, [48, 128, 3]]  # stage 1

  - [-1, 1, DWConv, [128, 3, 2, 1, False]]  # 2-P3/8
  - [-1, 6, HGBlock, [96, 512, 3]]   # stage 2

  - [-1, 1, DWConv, [512, 3, 2, 1, False]]  # 4-P4/16
  - [-1, 6, HGBlock, [192, 1024, 5, True, False]]  # cm, c2, k, light, shortcut
  - [-1, 6, HGBlock, [192, 1024, 5, True, True]]
  - [-1, 6, HGBlock, [192, 1024, 5, True, True]]  # stage 3

  - [-1, 1, DWConv, [768, 3, 2, 1, False]]  # 8-P5/32
  - [-1, 6, HGBlock, [384, 1536, 5, True, False]]  # stage 4

  - [-1, 1, DWConv, [1024, 3, 2, 1, False]]  # 10-P6/64
  - [-1, 6, HGBlock, [768, 2048, 5, True, False]]  # stage 5

head:
  - [-1, 1, Conv, [256, 1, 1, None, 1, 1, False]]  # 10 input_proj.2
  - [-1, 1, AIFI, [1024, 8]]
  - [-1, 1, Conv, [256, 1, 1]]   # 14, Y5, lateral_convs.0

  - [-1, 1, nn.Upsample, [None, 2, 'nearest']]
  - [9, 1, Conv, [256, 1, 1, None, 1, 1, False]]  # 14 input_proj.1
  - [[-2, -1], 1, Concat, [1]]
  - [-1, 3, RepC3, [256]]  # 18, fpn_blocks.0
  - [-1, 1, Conv, [256, 1, 1]]   # 19, Y4, lateral_convs.1

  - [-1, 1, nn.Upsample, [None, 2, 'nearest']]
  - [7, 1, Conv, [256, 1, 1, None, 1, 1, False]]  # 19 input_proj.0
  - [[-2, -1], 1, Concat, [1]]  # cat backbone P4
  - [-1, 3, RepC3, [256]]    # X3 (23), fpn_blocks.1

  - [-1, 1, nn.Upsample, [None, 2, 'nearest']]
  - [3, 1, Conv, [256, 1, 1, None, 1, 1, False]]  # 19 input_proj.0
  - [[-2, -1], 1, Concat, [1]]  # cat backbone P4
  - [-1, 3, RepC3, [256]]    # X3 (27), fpn_blocks.1

  - [-1, 1, Conv, [256, 3, 2]]   # 28, downsample_convs.0
  - [[-1, 23], 1, Concat, [1]]  # cat Y4
  - [-1, 3, RepC3, [256]]    # F4 (30), pan_blocks.0

  - [-1, 1, Conv, [256, 3, 2]]   # 31, downsample_convs.0
  - [[-1, 19], 1, Concat, [1]]  # cat Y4
  - [-1, 3, RepC3, [256]]    # F4 (33), pan_blocks.0

  - [-1, 1, Conv, [256, 3, 2]]   # 34, downsample_convs.1
  - [[-1, 14], 1, Concat, [1]]  # cat Y5
  - [-1, 3, RepC3, [256]]    # F5 (36), pan_blocks.1

  - [[27, 30, 33, 36], 1, RTDETRDecoder, [nc]]  # Detect(P3, P4, P5)

四、修改及调用步骤

4.1 修改步骤

新建一个模型文件，将上方修改后的模型结构复制到其中，注意修改 nc: 80 # number of classes 。

无需任何其他地方的修改。

运行即可。

4.2 运行结果 

                   from  n    params  module                                       arguments                     
  0                  -1  1     25248  ultralytics.nn.modules.block.HGStem          [3, 32, 48]                   
  1                  -1  6    155072  ultralytics.nn.modules.block.HGBlock         [48, 48, 128, 3, 6]           
  2                  -1  1      1408  ultralytics.nn.modules.conv.DWConv           [128, 128, 3, 2, 1, False]    
  3                  -1  6    839296  ultralytics.nn.modules.block.HGBlock         [128, 96, 512, 3, 6]          
  4                  -1  1      5632  ultralytics.nn.modules.conv.DWConv           [512, 512, 3, 2, 1, False]    
  5                  -1  6   1695360  ultralytics.nn.modules.block.HGBlock         [512, 192, 1024, 5, 6, True, False]
  6                  -1  6   2055808  ultralytics.nn.modules.block.HGBlock         [1024, 192, 1024, 5, 6, True, True]
  7                  -1  6   2055808  ultralytics.nn.modules.block.HGBlock         [1024, 192, 1024, 5, 6, True, True]
  8                  -1  1     29184  ultralytics.nn.modules.conv.DWConv           [1024, 768, 3, 2, 1, False]   
  9                  -1  6   4642560  ultralytics.nn.modules.block.HGBlock         [768, 384, 1536, 5, 6, True, False]
 10                  -1  1     29696  ultralytics.nn.modules.conv.DWConv           [1536, 1024, 3, 2, 1, False]  
 11                  -1  6  11739648  ultralytics.nn.modules.block.HGBlock         [1024, 768, 2048, 5, 6, True, False]
 12                  -1  1    524800  ultralytics.nn.modules.conv.Conv             [2048, 256, 1, 1, None, 1, 1, False]
 13                  -1  1    789760  ultralytics.nn.modules.transformer.AIFI      [256, 1024, 8]                
 14                  -1  1     66048  ultralytics.nn.modules.conv.Conv             [256, 256, 1, 1]              
 15                  -1  1         0  torch.nn.modules.upsampling.Upsample         [None, 2, 'nearest']          
 16                   9  1    393728  ultralytics.nn.modules.conv.Conv             [1536, 256, 1, 1, None, 1, 1, False]
 17            [-2, -1]  1         0  ultralytics.nn.modules.conv.Concat           [1]                           
 18                  -1  3   2232320  ultralytics.nn.modules.block.RepC3           [512, 256, 3]                 
 19                  -1  1     66048  ultralytics.nn.modules.conv.Conv             [256, 256, 1, 1]              
 20                  -1  1         0  torch.nn.modules.upsampling.Upsample         [None, 2, 'nearest']          
 21                   7  1    262656  ultralytics.nn.modules.conv.Conv             [1024, 256, 1, 1, None, 1, 1, False]
 22            [-2, -1]  1         0  ultralytics.nn.modules.conv.Concat           [1]                           
 23                  -1  3   2232320  ultralytics.nn.modules.block.RepC3           [512, 256, 3]                 
 24                  -1  1         0  torch.nn.modules.upsampling.Upsample         [None, 2, 'nearest']          
 25                   3  1    131584  ultralytics.nn.modules.conv.Conv             [512, 256, 1, 1, None, 1, 1, False]
 26            [-2, -1]  1         0  ultralytics.nn.modules.conv.Concat           [1]                           
 27                  -1  3   2232320  ultralytics.nn.modules.block.RepC3           [512, 256, 3]                 
 28                  -1  1    590336  ultralytics.nn.modules.conv.Conv             [256, 256, 3, 2]              
 29            [-1, 23]  1         0  ultralytics.nn.modules.conv.Concat           [1]                           
 30                  -1  3   2232320  ultralytics.nn.modules.block.RepC3           [512, 256, 3]                 
 31                  -1  1    590336  ultralytics.nn.modules.conv.Conv             [256, 256, 3, 2]              
 32            [-1, 19]  1         0  ultralytics.nn.modules.conv.Concat           [1]                           
 33                  -1  3   2232320  ultralytics.nn.modules.block.RepC3           [512, 256, 3]                 
 34                  -1  1    590336  ultralytics.nn.modules.conv.Conv             [256, 256, 3, 2]              
 35            [-1, 14]  1         0  ultralytics.nn.modules.conv.Concat           [1]                           
 36                  -1  3   2232320  ultralytics.nn.modules.block.RepC3           [512, 256, 3]                 
 37    [27, 30, 33, 36]  1   7517987  ultralytics.nn.modules.head.RTDETRDecoder    [1, [256, 256, 256, 256]]     
rtdetr-l-xlarge summary: 826 layers, 48,192,259 parameters, 48,192,259 gradients, 110.7 GFLOPs

从打印的网络结果可以看出， Backbone 部分增加了一次下采样来，缩小特征图，并提取特征， 颈部 中加入了用于提取大目标特征的特征层， 检测头 中传入了 四 个特征层分别为 P3 、 P4 、 P5 、 P6 ，其中 P6 就是新加入的 针对大目标的检测层 。

💡第 36 层是专门针对更大尺寸目标设计的。
💡 新添加的针对大目标检测的检测头通过在骨干网络中添加额外的下采样操作和C2fCIB模块来提取目标关键特征，再结合特征融合生成适合大目标检测的特征图，从而提高模型对大目标的检测能力。