【YOLOv13多模态融合改进】（可见光+红外）涉及前期、中期、后期融合方式的完整配置步骤以及二次改进方案
前言
主题： YOLOv13的多模态融合改进
方式： 前期融合、中期融合、后期融合。
内容： 包含融合方式详解和完整的项目包和配置步骤以及二次改进建议，开箱即用，一键运行。
一、融合方式
1.1 前期融合方法及结构图
定义： 在网络输入阶段将多模态数据直接合并，形成统一的特征表示。
实现方式： 将 RGB（3 通道）与红外（3 通道）图像直接拼接为 6 通道输入，以保留原始模态的细节信息。
结构示意图：
1.2 中期融合方法及结构图
定义： 在网络中间层（骨干网络与颈部网络之间）对多模态特征进行融合。
实现方式： 每个模态通过独立的骨干网络提取特征，融合时采用Add操作合并特征图，送入颈部网络。
结构示意图：
1.3 后期融合方法及结构图
定义： 在网络输出阶段（如检测头或分类器前）对多模态特征进行融合。
实现方式： 每个模态通过独立的骨干网络和颈部网络提取特征，融合时采用Add操作合并特征图，送入检测头。
结构示意图：
二、完整配置步骤
！！！ 获取的项目包就已经把相关的多模态输入、训练等改动都已经配好了，只需要新建模型yaml文件，粘贴对应的模型，进行训练即可。 项目包获取及使用教程可参考链接： 《YOLO系列模型的多模态项目》配置使用教程
在什么地方新建，n,s,l,x,用哪个版本按自己的需求来即可，和普通的训练步骤一致。
除了模型结构方面的改动，在yaml文件中还传入了一个通道数
ch: 6
表示传入的是双模态，6通道 ，前三个是可见光，后三个是红外。
在default.yaml中也配置了这个参数。
！！！还需要注意的是在yolov13中，融合部分只能使用 Add ，对于Add的配置可参考专栏中 CFT 模块的介绍。
2.1 前期融合
ch: 6
nc: 80 # number of classes
scales: # model compound scaling constants, i.e. 'model=yolov13n.yaml' will call yolov13.yaml with scale 'n'
# [depth, width, max_channels]
n: [0.50, 0.25, 1024] # Nano
s: [0.50, 0.50, 1024] # Small
l: [1.00, 1.00, 512] # Large
x: [1.00, 1.50, 512] # Extra Large
backbone:
# [from, repeats, module, args]
- [-1, 1, MF, [64]] # 0
- [-1, 1, Conv, [64, 3, 2]] # 1-P1/2
- [-1, 1, Conv, [128, 3, 2, 1, 2]] # 2-P2/4
- [-1, 2, DSC3k2, [256, False, 0.25]]
- [-1, 1, Conv, [256, 3, 2, 1, 4]] # 4-P3/8
- [-1, 2, DSC3k2, [512, False, 0.25]]
- [-1, 1, DSConv, [512, 3, 2]] # 6-P4/16
- [-1, 4, A2C2f, [512, True, 4]]
- [-1, 1, DSConv, [1024, 3, 2]] # 8-P5/32
- [-1, 4, A2C2f, [1024, True, 1]] # 9
head:
- [[5, 7, 9], 2, HyperACE, [512, 8, True, True, 0.5, 1, "both"]]
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [ 10, 1, DownsampleConv, []]
- [[7, 10], 1, FullPAD_Tunnel, []] #13
- [[5, 11], 1, FullPAD_Tunnel, []] #14
- [[9, 12], 1, FullPAD_Tunnel, []] #15
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [[-1, 13], 1, Concat, [1]] # cat backbone P4
- [-1, 2, DSC3k2, [512, True]] # 18
- [[-1, 10], 1, FullPAD_Tunnel, []] #19
- [18, 1, nn.Upsample, [None, 2, "nearest"]]
- [[-1, 14], 1, Concat, [1]] # cat backbone P3
- [-1, 2, DSC3k2, [256, True]] # 22
- [11, 1, Conv, [256, 1, 1]]
- [[22, 23], 1, FullPAD_Tunnel, []] #24
- [-1, 1, Conv, [256, 3, 2]]
- [[-1, 19], 1, Concat, [1]] # cat head P4
- [-1, 2, DSC3k2, [512, True]] # 27
- [[-1, 10], 1, FullPAD_Tunnel, []]
- [27, 1, Conv, [512, 3, 2]]
- [[-1, 15], 1, Concat, [1]] # cat head P5
- [-1, 2, DSC3k2, [1024,True]] # 31 (P5/32-large)
- [[-1, 12], 1, FullPAD_Tunnel, []]
- [[24, 28, 32], 1, Detect, [nc]] # Detect(P3, P4, P5)
2.2 中期融合
ch: 6
nc: 80 # number of classes
scales: # model compound scaling constants, i.e. 'model=yolov13n.yaml' will call yolov13.yaml with scale 'n'
# [depth, width, max_channels]
n: [0.50, 0.25, 1024] # Nano
s: [0.50, 0.50, 1024] # Small
l: [1.00, 1.00, 512] # Large
x: [1.00, 1.50, 512] # Extra Large
backbone:
# [from, repeats, module, args]
- [-1, 1, IN, []] # 0
- [-1, 1, Multiin, [1]] # 1
- [-2, 1, Multiin, [2]] # 2
- [1, 1, Conv, [64, 3, 2]] # 3-P1/2
- [-1, 1, Conv, [128, 3, 2, 1, 2]] # 4-P2/4
- [-1, 2, DSC3k2, [256, False, 0.25]]
- [-1, 1, Conv, [256, 3, 2, 1, 4]] # 6-P3/8
- [-1, 2, DSC3k2, [512, True]]
- [-1, 1, DSConv, [512, 3, 2]] # 8-P4/16
- [-1, 4, A2C2f, [512, True, 4]]
- [-1, 1, DSConv, [1024, 3, 2]] # 10-P5/32
- [-1, 4, A2C2f, [1024, True, 1]] # 11
- [2, 1, Conv, [64, 3, 2]] # 12-P1/2
- [-1, 1, Conv, [128, 3, 2, 1, 2]] # 13-P2/4
- [-1, 2, DSC3k2, [256, False, 0.25]]
- [-1, 1, Conv, [256, 3, 2, 1, 4]] # 15-P3/8
- [-1, 2, DSC3k2, [512, True]]
- [-1, 1, DSConv, [512, 3, 2]] # 17-P4/16
- [-1, 4, A2C2f, [512, True, 4]]
- [-1, 1, DSConv, [1024, 3, 2]] # 19-P5/32
- [-1, 4, A2C2f, [1024, True, 1]] # 20
- [[7, 16], 1, Add, [1]] # 21 cat backbone P3
- [[9, 18], 1, Add, [1]] # 22 cat backbone P4
- [[11, 20], 1, Add, [1]] # 23 cat backbone P5
head:
- [[21, 22, 23], 2, HyperACE, [512, 8, True, True, 0.5, 1, "both"]]
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [ 24, 1, DownsampleConv, []]
- [[22, 24], 1, FullPAD_Tunnel, []] # 27
- [[21, 25], 1, FullPAD_Tunnel, []] # 28
- [[23, 26], 1, FullPAD_Tunnel, []] # 29
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [[-1, 27], 1, Concat, [1]] # cat backbone P4
- [-1, 2, DSC3k2, [512, True]] # 32
- [[-1, 24], 1, FullPAD_Tunnel, []] # 33
- [32, 1, nn.Upsample, [None, 2, "nearest"]]
- [[-1, 28], 1, Concat, [1]] # cat backbone P3
- [-1, 2, DSC3k2, [256, True]] # 36
- [25, 1, Conv, [256, 1, 1]]
- [[36, 37], 1, FullPAD_Tunnel, []] # 38
- [-1, 1, Conv, [256, 3, 2]]
- [[-1, 33], 1, Concat, [1]] # cat head P4
- [-1, 2, DSC3k2, [512, True]] # 41
- [[-1, 24], 1, FullPAD_Tunnel, []]
- [41, 1, Conv, [512, 3, 2]]
- [[-1, 29], 1, Concat, [1]] # cat head P5
- [-1, 2, DSC3k2, [1024,True]] # 45 (P5/32-large)
- [[-1, 26], 1, FullPAD_Tunnel, []]
- [[38, 42, 46], 1, Detect, [nc]] # Detect(P3, P4, P5)
2.3 后期融合
ch: 6
nc: 80 # number of classes
scales: # model compound scaling constants, i.e. 'model=yolov13n.yaml' will call yolov13.yaml with scale 'n'
# [depth, width, max_channels]
n: [0.50, 0.25, 1024] # Nano
s: [0.50, 0.50, 1024] # Small
l: [1.00, 1.00, 512] # Large
x: [1.00, 1.50, 512] # Extra Large
backbone:
# [from, repeats, module, args]
- [-1, 1, IN, []] # 0
- [-1, 1, Multiin, [1]] # 1
- [-2, 1, Multiin, [2]] # 2
- [1, 1, Conv, [64, 3, 2]] # 3-P1/2
- [-1, 1, Conv, [128, 3, 2, 1, 2]] # 4-P2/4
- [-1, 2, DSC3k2, [256, False, 0.25]]
- [-1, 1, Conv, [256, 3, 2, 1, 4]] # 6-P3/8
- [-1, 2, DSC3k2, [512, True]]
- [-1, 1, DSConv, [512, 3, 2]] # 8-P4/16
- [-1, 4, A2C2f, [512, True, 4]]
- [-1, 1, DSConv, [1024, 3, 2]] # 10-P5/32
- [-1, 4, A2C2f, [1024, True, 1]] # 11
- [2, 1, Conv, [64, 3, 2]] # 12-P1/2
- [-1, 1, Conv, [128, 3, 2, 1, 2]] # 13-P2/4
- [-1, 2, DSC3k2, [256, False, 0.25]]
- [-1, 1, Conv, [256, 3, 2, 1, 4]] # 15-P3/8
- [-1, 2, DSC3k2, [512, True]]
- [-1, 1, DSConv, [512, 3, 2]] # 17-P4/16
- [-1, 4, A2C2f, [512, True, 4]]
- [-1, 1, DSConv, [1024, 3, 2]] # 19-P5/32
- [-1, 4, A2C2f, [1024, True, 1]] # 20
head:
- [[7, 9, 11], 2, HyperACE, [512, 8, True, True, 0.5, 1, "both"]]
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [ 21, 1, DownsampleConv, []]
- [[9, 21], 1, FullPAD_Tunnel, []] # 24
- [[7, 22], 1, FullPAD_Tunnel, []] # 25
- [[11, 23], 1, FullPAD_Tunnel, []] # 26
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [[-1, 24], 1, Concat, [1]] # cat backbone P4
- [-1, 2, DSC3k2, [512, True]] # 29
- [[-1, 21], 1, FullPAD_Tunnel, []] # 30
- [29, 1, nn.Upsample, [None, 2, "nearest"]]
- [[-1, 25], 1, Concat, [1]] # cat backbone P3
- [-1, 2, DSC3k2, [256, True]] # 33
- [22, 1, Conv, [256, 1, 1]]
- [[33, 34], 1, FullPAD_Tunnel, []] # 35
- [-1, 1, Conv, [256, 3, 2]]
- [[-1, 30], 1, Concat, [1]] # cat head P4
- [-1, 2, DSC3k2, [512, True]] # 38
- [[-1, 21], 1, FullPAD_Tunnel, []]
- [38, 1, Conv, [512, 3, 2]]
- [[-1, 26], 1, Concat, [1]] # cat head P5
- [-1, 2, DSC3k2, [1024,True]] # 42 (P5/32-large)
- [[-1, 23], 1, FullPAD_Tunnel, []]
- [[16, 18, 20], 2, HyperACE, [512, 8, True, True, 0.5, 1, "both"]]
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [ 44, 1, DownsampleConv, []]
- [[18, 44], 1, FullPAD_Tunnel, []] # 47
- [[16, 45], 1, FullPAD_Tunnel, []] # 48
- [[20, 46], 1, FullPAD_Tunnel, []] # 49
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [[-1, 47], 1, Concat, [1]] # cat backbone P4
- [-1, 2, DSC3k2, [512, True]] # 52
- [[-1, 44], 1, FullPAD_Tunnel, []] # 53
- [52, 1, nn.Upsample, [None, 2, "nearest"]]
- [[-1, 48], 1, Concat, [1]] # cat backbone P3
- [-1, 2, DSC3k2, [256, True]] # 56
- [45, 1, Conv, [256, 1, 1]]
- [[56, 57], 1, FullPAD_Tunnel, []] # 58
- [-1, 1, Conv, [256, 3, 2]]
- [[-1, 53], 1, Concat, [1]] # cat head P4
- [-1, 2, DSC3k2, [512, True]] # 61
- [[-1, 44], 1, FullPAD_Tunnel, []]
- [61, 1, Conv, [512, 3, 2]]
- [[-1, 49], 1, Concat, [1]] # cat head P5
- [-1, 2, DSC3k2, [1024,True]] # 65 (P5/32-large)
- [[-1, 46], 1, FullPAD_Tunnel, []]
- [[35, 58], 1, Add, [1]] # 67 cat backbone P3
- [[39, 62], 1, Add, [1]] # 68 cat backbone P4
- [[43, 66], 1, Add, [1]] # 69 cat backbone P5
- [[67, 68, 69], 1, Detect, [nc]] # Detect(P3, P4, P5)
三、成功运行结果
前期融合结果： 可以看到输入的通道数为6，表明可见光图像和红外图像均输入到了模型中进行融合训练。
YOLOv13-earlyfusion summary: 630 layers, 2,482,442 parameters, 2,482,426 gradients, 7.1 GFLOPs
from n params module arguments
0 -1 1 472 ultralytics.nn.AddModules.multimodal.MF [6, 16]
1 -1 1 2336 ultralytics.nn.modules.conv.Conv [16, 16, 3, 2]
2 -1 1 2368 ultralytics.nn.modules.conv.Conv [16, 32, 3, 2, 1, 2]
3 -1 1 5792 ultralytics.nn.modules.block.DSC3k2 [32, 64, 1, False, 0.25]
4 -1 1 9344 ultralytics.nn.modules.conv.Conv [64, 64, 3, 2, 1, 4]
5 -1 1 20800 ultralytics.nn.modules.block.DSC3k2 [64, 128, 1, False, 0.25]
6 -1 1 17792 ultralytics.nn.modules.conv.DSConv [128, 128, 3, 2]
7 -1 2 180864 ultralytics.nn.AddModules.A2C2f.A2C2f [128, 128, 2, True, 4]
8 -1 1 34432 ultralytics.nn.modules.conv.DSConv [128, 256, 3, 2]
9 -1 2 689408 ultralytics.nn.AddModules.A2C2f.A2C2f [256, 256, 2, True, 1]
10 [5, 7, 9] 1 273536 ultralytics.nn.modules.block.HyperACE [128, 128, 1, 4, True, True, 0.5, 1, 'both']
11 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
12 10 1 33280 ultralytics.nn.modules.block.DownsampleConv [128]
13 [7, 10] 1 1 ultralytics.nn.modules.block.FullPAD_Tunnel []
14 [5, 11] 1 1 ultralytics.nn.modules.block.FullPAD_Tunnel []
15 [9, 12] 1 1 ultralytics.nn.modules.block.FullPAD_Tunnel []
16 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
17 [-1, 13] 1 0 ultralytics.nn.modules.conv.Concat [1]
18 -1 1 115328 ultralytics.nn.modules.block.DSC3k2 [384, 128, 1, True]
19 [-1, 10] 1 1 ultralytics.nn.modules.block.FullPAD_Tunnel []
20 18 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
21 [-1, 14] 1 0 ultralytics.nn.modules.conv.Concat [1]
22 -1 1 35136 ultralytics.nn.modules.block.DSC3k2 [256, 64, 1, True]
23 11 1 8320 ultralytics.nn.modules.conv.Conv [128, 64, 1, 1]
24 [22, 23] 1 1 ultralytics.nn.modules.block.FullPAD_Tunnel []
25 -1 1 36992 ultralytics.nn.modules.conv.Conv [64, 64, 3, 2]
26 [-1, 19] 1 0 ultralytics.nn.modules.conv.Concat [1]
27 -1 1 90752 ultralytics.nn.modules.block.DSC3k2 [192, 128, 1, True]
28 [-1, 10] 1 1 ultralytics.nn.modules.block.FullPAD_Tunnel []
29 27 1 147712 ultralytics.nn.modules.conv.Conv [128, 128, 3, 2]
30 [-1, 15] 1 0 ultralytics.nn.modules.conv.Concat [1]
31 -1 1 345344 ultralytics.nn.modules.block.DSC3k2 [384, 256, 1, True]
32 [-1, 12] 1 1 ultralytics.nn.modules.block.FullPAD_Tunnel []
33 [24, 28, 32] 1 432427 ultralytics.nn.modules.head.Detect [9, [64, 128, 256]]
YOLOv13-earlyfusion summary: 630 layers, 2,482,442 parameters, 2,482,426 gradients, 7.1 GFLOPs
中期融合结果：
YOLOv13-mid-fusion summary: 920 layers, 3,548,498 parameters, 3,548,482 gradients, 9.9 GFLOPs
from n params module arguments
0 -1 1 0 ultralytics.nn.AddModules.multimodal.IN []
1 -1 1 0 ultralytics.nn.AddModules.multimodal.Multiin [1]
2 -2 1 0 ultralytics.nn.AddModules.multimodal.Multiin [2]
3 1 1 464 ultralytics.nn.modules.conv.Conv [3, 16, 3, 2]
4 -1 1 2368 ultralytics.nn.modules.conv.Conv [16, 32, 3, 2, 1, 2]
5 -1 1 5792 ultralytics.nn.modules.block.DSC3k2 [32, 64, 1, False, 0.25]
6 -1 1 9344 ultralytics.nn.modules.conv.Conv [64, 64, 3, 2, 1, 4]
7 -1 1 74368 ultralytics.nn.modules.block.DSC3k2 [64, 128, 1, True]
8 -1 1 17792 ultralytics.nn.modules.conv.DSConv [128, 128, 3, 2]
9 -1 2 180864 ultralytics.nn.AddModules.A2C2f.A2C2f [128, 128, 2, True, 4]
10 -1 1 34432 ultralytics.nn.modules.conv.DSConv [128, 256, 3, 2]
11 -1 2 689408 ultralytics.nn.AddModules.A2C2f.A2C2f [256, 256, 2, True, 1]
12 2 1 464 ultralytics.nn.modules.conv.Conv [3, 16, 3, 2]
13 -1 1 2368 ultralytics.nn.modules.conv.Conv [16, 32, 3, 2, 1, 2]
14 -1 1 5792 ultralytics.nn.modules.block.DSC3k2 [32, 64, 1, False, 0.25]
15 -1 1 9344 ultralytics.nn.modules.conv.Conv [64, 64, 3, 2, 1, 4]
16 -1 1 74368 ultralytics.nn.modules.block.DSC3k2 [64, 128, 1, True]
17 -1 1 17792 ultralytics.nn.modules.conv.DSConv [128, 128, 3, 2]
18 -1 2 180864 ultralytics.nn.AddModules.A2C2f.A2C2f [128, 128, 2, True, 4]
19 -1 1 34432 ultralytics.nn.modules.conv.DSConv [128, 256, 3, 2]
20 -1 2 689408 ultralytics.nn.AddModules.A2C2f.A2C2f [256, 256, 2, True, 1]
21 [7, 16] 1 0 ultralytics.nn.AddModules.CFT.Add [128]
22 [9, 18] 1 0 ultralytics.nn.AddModules.CFT.Add [128]
23 [11, 20] 1 0 ultralytics.nn.AddModules.CFT.Add [256]
24 [21, 22, 23] 1 273536 ultralytics.nn.modules.block.HyperACE [128, 128, 1, 4, True, True, 0.5, 1, 'both']
25 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
26 24 1 33280 ultralytics.nn.modules.block.DownsampleConv [128]
27 [22, 24] 1 1 ultralytics.nn.modules.block.FullPAD_Tunnel []
28 [21, 25] 1 1 ultralytics.nn.modules.block.FullPAD_Tunnel []
29 [23, 26] 1 1 ultralytics.nn.modules.block.FullPAD_Tunnel []
30 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
31 [-1, 27] 1 0 ultralytics.nn.modules.conv.Concat [1]
32 -1 1 115328 ultralytics.nn.modules.block.DSC3k2 [384, 128, 1, True]
33 [-1, 24] 1 1 ultralytics.nn.modules.block.FullPAD_Tunnel []
34 32 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
35 [-1, 28] 1 0 ultralytics.nn.modules.conv.Concat [1]
36 -1 1 35136 ultralytics.nn.modules.block.DSC3k2 [256, 64, 1, True]
37 25 1 8320 ultralytics.nn.modules.conv.Conv [128, 64, 1, 1]
38 [36, 37] 1 1 ultralytics.nn.modules.block.FullPAD_Tunnel []
39 -1 1 36992 ultralytics.nn.modules.conv.Conv [64, 64, 3, 2]
40 [-1, 33] 1 0 ultralytics.nn.modules.conv.Concat [1]
41 -1 1 90752 ultralytics.nn.modules.block.DSC3k2 [192, 128, 1, True]
42 [-1, 24] 1 1 ultralytics.nn.modules.block.FullPAD_Tunnel []
43 41 1 147712 ultralytics.nn.modules.conv.Conv [128, 128, 3, 2]
44 [-1, 29] 1 0 ultralytics.nn.modules.conv.Concat [1]
45 -1 1 345344 ultralytics.nn.modules.block.DSC3k2 [384, 256, 1, True]
46 [-1, 26] 1 1 ultralytics.nn.modules.block.FullPAD_Tunnel []
47 [38, 42, 46] 1 432427 ultralytics.nn.modules.head.Detect [9, [64, 128, 256]]
YOLOv13-mid-fusion summary: 920 layers, 3,548,498 parameters, 3,548,482 gradients, 9.9 GFLOPs
后期融合结果：
YOLOv13-late-fusion summary: 1,203 layers, 4,634,905 parameters, 4,634,889 gradients, 12.3 GFLOPs
from n params module arguments
0 -1 1 0 ultralytics.nn.AddModules.multimodal.IN []
1 -1 1 0 ultralytics.nn.AddModules.multimodal.Multiin [1]
2 -2 1 0 ultralytics.nn.AddModules.multimodal.Multiin [2]
3 1 1 464 ultralytics.nn.modules.conv.Conv [3, 16, 3, 2]
4 -1 1 2368 ultralytics.nn.modules.conv.Conv [16, 32, 3, 2, 1, 2]
5 -1 1 5792 ultralytics.nn.modules.block.DSC3k2 [32, 64, 1, False, 0.25]
6 -1 1 9344 ultralytics.nn.modules.conv.Conv [64, 64, 3, 2, 1, 4]
7 -1 1 74368 ultralytics.nn.modules.block.DSC3k2 [64, 128, 1, True]
8 -1 1 17792 ultralytics.nn.modules.conv.DSConv [128, 128, 3, 2]
9 -1 2 180864 ultralytics.nn.AddModules.A2C2f.A2C2f [128, 128, 2, True, 4]
10 -1 1 34432 ultralytics.nn.modules.conv.DSConv [128, 256, 3, 2]
11 -1 2 689408 ultralytics.nn.AddModules.A2C2f.A2C2f [256, 256, 2, True, 1]
12 2 1 464 ultralytics.nn.modules.conv.Conv [3, 16, 3, 2]
13 -1 1 2368 ultralytics.nn.modules.conv.Conv [16, 32, 3, 2, 1, 2]
14 -1 1 5792 ultralytics.nn.modules.block.DSC3k2 [32, 64, 1, False, 0.25]
15 -1 1 9344 ultralytics.nn.modules.conv.Conv [64, 64, 3, 2, 1, 4]
16 -1 1 74368 ultralytics.nn.modules.block.DSC3k2 [64, 128, 1, True]
17 -1 1 17792 ultralytics.nn.modules.conv.DSConv [128, 128, 3, 2]
18 -1 2 180864 ultralytics.nn.AddModules.A2C2f.A2C2f [128, 128, 2, True, 4]
19 -1 1 34432 ultralytics.nn.modules.conv.DSConv [128, 256, 3, 2]
20 -1 2 689408 ultralytics.nn.AddModules.A2C2f.A2C2f [256, 256, 2, True, 1]
21 [7, 9, 11] 1 273536 ultralytics.nn.modules.block.HyperACE [128, 128, 1, 4, True, True, 0.5, 1, 'both']
22 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
23 21 1 33280 ultralytics.nn.modules.block.DownsampleConv [128]
24 [9, 21] 1 1 ultralytics.nn.modules.block.FullPAD_Tunnel []
25 [7, 22] 1 1 ultralytics.nn.modules.block.FullPAD_Tunnel []
26 [11, 23] 1 1 ultralytics.nn.modules.block.FullPAD_Tunnel []
27 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
28 [-1, 24] 1 0 ultralytics.nn.modules.conv.Concat [1]
29 -1 1 115328 ultralytics.nn.modules.block.DSC3k2 [384, 128, 1, True]
30 [-1, 21] 1 1 ultralytics.nn.modules.block.FullPAD_Tunnel []
31 29 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
32 [-1, 25] 1 0 ultralytics.nn.modules.conv.Concat [1]
33 -1 1 35136 ultralytics.nn.modules.block.DSC3k2 [256, 64, 1, True]
34 22 1 8320 ultralytics.nn.modules.conv.Conv [128, 64, 1, 1]
35 [33, 34] 1 1 ultralytics.nn.modules.block.FullPAD_Tunnel []
36 -1 1 36992 ultralytics.nn.modules.conv.Conv [64, 64, 3, 2]
37 [-1, 30] 1 0 ultralytics.nn.modules.conv.Concat [1]
38 -1 1 90752 ultralytics.nn.modules.block.DSC3k2 [192, 128, 1, True]
39 [-1, 21] 1 1 ultralytics.nn.modules.block.FullPAD_Tunnel []
40 38 1 147712 ultralytics.nn.modules.conv.Conv [128, 128, 3, 2]
41 [-1, 26] 1 0 ultralytics.nn.modules.conv.Concat [1]
42 -1 1 345344 ultralytics.nn.modules.block.DSC3k2 [384, 256, 1, True]
43 [-1, 23] 1 1 ultralytics.nn.modules.block.FullPAD_Tunnel []
44 [16, 18, 20] 1 273536 ultralytics.nn.modules.block.HyperACE [128, 128, 1, 4, True, True, 0.5, 1, 'both']
45 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
46 44 1 33280 ultralytics.nn.modules.block.DownsampleConv [128]
47 [18, 44] 1 1 ultralytics.nn.modules.block.FullPAD_Tunnel []
48 [16, 45] 1 1 ultralytics.nn.modules.block.FullPAD_Tunnel []
49 [20, 46] 1 1 ultralytics.nn.modules.block.FullPAD_Tunnel []
50 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
51 [-1, 47] 1 0 ultralytics.nn.modules.conv.Concat [1]
52 -1 1 115328 ultralytics.nn.modules.block.DSC3k2 [384, 128, 1, True]
53 [-1, 44] 1 1 ultralytics.nn.modules.block.FullPAD_Tunnel []
54 52 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
55 [-1, 48] 1 0 ultralytics.nn.modules.conv.Concat [1]
56 -1 1 35136 ultralytics.nn.modules.block.DSC3k2 [256, 64, 1, True]
57 45 1 8320 ultralytics.nn.modules.conv.Conv [128, 64, 1, 1]
58 [56, 57] 1 1 ultralytics.nn.modules.block.FullPAD_Tunnel []
59 -1 1 36992 ultralytics.nn.modules.conv.Conv [64, 64, 3, 2]
60 [-1, 53] 1 0 ultralytics.nn.modules.conv.Concat [1]
61 -1 1 90752 ultralytics.nn.modules.block.DSC3k2 [192, 128, 1, True]
62 [-1, 44] 1 1 ultralytics.nn.modules.block.FullPAD_Tunnel []
63 61 1 147712 ultralytics.nn.modules.conv.Conv [128, 128, 3, 2]
64 [-1, 49] 1 0 ultralytics.nn.modules.conv.Concat [1]
65 -1 1 345344 ultralytics.nn.modules.block.DSC3k2 [384, 256, 1, True]
66 [-1, 46] 1 1 ultralytics.nn.modules.block.FullPAD_Tunnel []
67 [35, 58] 1 0 ultralytics.nn.AddModules.CFT.Add [64]
68 [39, 62] 1 0 ultralytics.nn.AddModules.CFT.Add [128]
69 [43, 66] 1 0 ultralytics.nn.AddModules.CFT.Add [256]
70 [67, 68, 69] 1 432427 ultralytics.nn.modules.head.Detect [9, [64, 128, 256]]
YOLOv13-late-fusion summary: 1,203 layers, 4,634,905 parameters, 4,634,889 gradients, 12.3 GFLOPs
四、二次改进方案
-
多模态模型的二次改进和普通模型的改进一致，主要涉及到DSC3k2、A2C2f、颈部结构、上采样、下采样等，可以增加或替换成其它模块，可以换成其它的颈部结构在进行融合。
-
两个骨干中均可以再次添加其它模块，需要注意的是融合的时候层数要对应上，即两层的特征图大小要一致。