【YOLOv10多模态融合改进】（可见光+红外）涉及前期、中期、中后期、后期融合方式的完整配置步骤以及二次改进方案
前言
主题： YOLOv10的多模态融合改进
方式： 前期融合、中期融合、中-后期融合、后期融合。
内容： 包含融合方式详解和完整的项目包和配置步骤以及二次改进建议，开箱即用，一键运行。
一、融合方式
1.1 前期融合方法及结构图
定义： 在网络输入阶段将多模态数据直接合并，形成统一的特征表示。
实现方式： 将 RGB（3 通道）与红外（3 通道）图像直接拼接为 6 通道输入，以保留原始模态的细节信息。
结构示意图：
1.2 中期融合方法及结构图
定义： 在网络中间层（骨干网络与颈部网络之间）对多模态特征进行融合。
实现方式： 每个模态通过独立的骨干网络提取特征，融合时采用Concat操作合并特征图，送入颈部网络。
结构示意图：
1.3 中-后期融合方法及结构图
定义： 在颈部网络的上采样之后对多模态特征进行融合。
实现方式： 每个模态通过独立的骨干网络和颈部的FPN网络提取特征，融合时采用Concat操作合并特征图，送入检测头。
结构示意图：
1.4 后期融合方法及结构图
定义： 在网络输出阶段（如检测头或分类器前）对多模态特征进行融合。
实现方式： 每个模态通过独立的骨干网络和颈部网络提取特征，融合时采用Concat操作合并特征图，送入检测头。
结构示意图：
二、完整配置步骤
！！！ 私信获取的项目包就已经把相关的多模态输入、训练等改动都已经配好了，只需要新建模型yaml文件，粘贴对应的模型，进行训练即可。 项目包获取及使用教程可参考链接： 《YOLO系列模型的多模态项目》配置使用教程
在什么地方新建，n,s,m,l,x,用哪个版本按自己的需求来即可，和普通的训练步骤一致。
除了模型结构方面的改动，在yaml文件中还传入了一个通道数
ch: 6
表示传入的是双模态，6通道 ，前三个是可见光，后三个是红外。
在default.yaml中也配置了这个参数。
2.1 前期融合
# Ultralytics YOLO 🚀, AGPL-3.0 license
# YOLOv10 object detection model. For Usage examples see https://docs.ultralytics.com/tasks/detect
# Parameters
ch: 6
nc: 80 # number of classes
scales: # model compound scaling constants, i.e. 'model=yolov10n.yaml' will call yolov10.yaml with scale 'n'
# [depth, width, max_channels]
n: [0.33, 0.25, 1024]
backbone:
# [from, repeats, module, args]
- [-1, 1, MF, [64]] # 0-P1/2
- [-1, 1, Conv, [64, 3, 2]] # 1-P1/2
- [-1, 1, Conv, [128, 3, 2]] # 2-P2/4
- [-1, 3, C2f, [128, True]]
- [-1, 1, Conv, [256, 3, 2]] # 4-P3/8
- [-1, 6, C2f, [256, True]]
- [-1, 1, SCDown, [512, 3, 2]] # 6-P4/16
- [-1, 6, C2f, [512, True]]
- [-1, 1, SCDown, [1024, 3, 2]] # 8-P5/32
- [-1, 3, C2f, [1024, True]]
- [-1, 1, SPPF, [1024, 5]] # 10
- [-1, 1, PSA, [1024]] # 11
# YOLOv10.0n head
head:
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [[-1, 7], 1, Concat, [1]] # cat backbone P4
- [-1, 3, C2f, [512]] # 14
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [[-1, 5], 1, Concat, [1]] # cat backbone P3
- [-1, 3, C2f, [256]] # 17 (P3/8-small)
- [-1, 1, Conv, [256, 3, 2]]
- [[-1, 12], 1, Concat, [1]] # cat head P4
- [-1, 3, C2f, [512]] # 20 (P4/16-medium)
- [-1, 1, SCDown, [512, 3, 2]]
- [[-1, 11], 1, Concat, [1]] # cat head P5
- [-1, 3, C2fCIB, [1024, True, True]] # 23 (P5/32-large)
- [[17, 20, 23], 1, v10Detect, [nc]] # Detect(P3, P4, P5)
2.2 中期融合
# Ultralytics YOLO 🚀, AGPL-3.0 license
# YOLOv10 object detection model. For Usage examples see https://docs.ultralytics.com/tasks/detect
# Parameters
ch: 6
nc: 80 # number of classes
scales: # model compound scaling constants, i.e. 'model=yolov10n.yaml' will call yolov10.yaml with scale 'n'
# [depth, width, max_channels]
n: [0.33, 0.25, 1024]
backbone:
# [from, repeats, module, args]
- [-1, 1, IN, []] # 0
- [-1, 1, Multiin, [1]] # 1
- [-2, 1, Multiin, [2]] # 2
- [1, 1, Conv, [64, 3, 2]] # 3-P1/2
- [-1, 1, Conv, [128, 3, 2]] # 4-P2/4
- [-1, 3, C2f, [128, True]]
- [-1, 1, Conv, [256, 3, 2]] # 6-P3/8
- [-1, 6, C2f, [256, True]]
- [-1, 1, SCDown, [512, 3, 2]] # 8-P4/16
- [-1, 6, C2f, [512, True]]
- [-1, 1, SCDown, [1024, 3, 2]] # 10-P5/32
- [-1, 3, C2f, [1024, True]]
- [2, 1, Conv, [64, 3, 2]] # 12-P1/2
- [-1, 1, Conv, [128, 3, 2]] # 13-P2/4
- [-1, 3, C2f, [128, True]]
- [-1, 1, Conv, [256, 3, 2]] # 15-P3/8
- [-1, 6, C2f, [256, True]]
- [-1, 1, SCDown, [512, 3, 2]] # 17-P4/16
- [-1, 6, C2f, [512, True]]
- [-1, 1, SCDown, [1024, 3, 2]] # 19-P5/32
- [-1, 3, C2f, [1024, True]]
- [[7, 16], 1, Concat, [1]] # 21 cat backbone P3
- [[9, 18], 1, Concat, [1]] # 22 cat backbone P4
- [[11, 20], 1, Concat, [1]] # 23 cat backbone P5
- [-1, 1, SPPF, [1024, 5]] # 24
- [-1, 1, PSA, [1024]] # 25
# YOLOv10.0n head
head:
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [[-1, 22], 1, Concat, [1]] # cat backbone P4
- [-1, 3, C2f, [512]] # 28
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [[-1, 21], 1, Concat, [1]] # cat backbone P3
- [-1, 3, C2f, [256]] # 31 (P3/8-small)
- [-1, 1, Conv, [256, 3, 2]]
- [[-1, 28], 1, Concat, [1]] # cat head P4
- [-1, 3, C2f, [512]] # 34 (P4/16-medium)
- [-1, 1, SCDown, [512, 3, 2]]
- [[-1, 25], 1, Concat, [1]] # cat head P5
- [-1, 3, C2fCIB, [1024, True, True]] # 37 (P5/32-large)
- [[31, 34, 37], 1, v10Detect, [nc]] # Detect(P3, P4, P5)
2.3 中-后期融合
# Ultralytics YOLO 🚀, AGPL-3.0 license
# YOLOv10 object detection model. For Usage examples see https://docs.ultralytics.com/tasks/detect
# Parameters
ch: 6
nc: 80 # number of classes
scales: # model compound scaling constants, i.e. 'model=yolov10n.yaml' will call yolov10.yaml with scale 'n'
# [depth, width, max_channels]
n: [0.33, 0.25, 1024]
backbone:
# [from, repeats, module, args]
- [-1, 1, IN, []] # 0
- [-1, 1, Multiin, [1]] # 1
- [-2, 1, Multiin, [2]] # 2
- [1, 1, Conv, [64, 3, 2]] # 3-P1/2
- [-1, 1, Conv, [128, 3, 2]] # 4-P2/4
- [-1, 3, C2f, [128, True]]
- [-1, 1, Conv, [256, 3, 2]] # 6-P3/8
- [-1, 6, C2f, [256, True]]
- [-1, 1, SCDown, [512, 3, 2]] # 8-P4/16
- [-1, 6, C2f, [512, True]]
- [-1, 1, SCDown, [1024, 3, 2]] # 10-P5/32
- [-1, 3, C2f, [1024, True]]
- [-1, 1, SPPF, [1024, 5]] # 12
- [-1, 1, PSA, [1024]] # 13
- [2, 1, Conv, [64, 3, 2]] # 14-P1/2
- [-1, 1, Conv, [128, 3, 2]] # 15-P2/4
- [-1, 3, C2f, [128, True]]
- [-1, 1, Conv, [256, 3, 2]] # 17-P3/8
- [-1, 6, C2f, [256, True]]
- [-1, 1, SCDown, [512, 3, 2]] # 19-P4/16
- [-1, 6, C2f, [512, True]]
- [-1, 1, SCDown, [1024, 3, 2]] # 21-P5/32
- [-1, 3, C2f, [1024, True]]
- [-1, 1, SPPF, [1024, 5]] # 23
- [-1, 1, PSA, [1024]] # 24
# YOLOv10.0n head
head:
- [13, 1, nn.Upsample, [None, 2, "nearest"]]
- [[-1, 9], 1, Concat, [1]] # cat backbone P4
- [-1, 3, C2f, [512]] # 27
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [[-1, 7], 1, Concat, [1]] # cat backbone P3
- [-1, 3, C2f, [256]] # 30 (P3/8-small)
- [24, 1, nn.Upsample, [None, 2, "nearest"]]
- [[-1, 20], 1, Concat, [1]] # cat backbone P4
- [-1, 3, C2f, [512]] # 33
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [[-1, 18], 1, Concat, [1]] # cat backbone P3
- [-1, 3, C2f, [256]] # 36 (P3/8-small)
- [[13, 24], 1, Concat, [1]] # 37 cat backbone P3
- [[27, 33], 1, Concat, [1]] # 38 cat backbone P4
- [[30, 36], 1, Concat, [1]] # 39 cat backbone P5
- [-1, 1, Conv, [256, 3, 2]]
- [[-1, 38], 1, Concat, [1]] # cat head P4
- [-1, 3, C2f, [512]] # 42 (P4/16-medium)
- [-1, 1, SCDown, [512, 3, 2]]
- [[-1, 37], 1, Concat, [1]] # cat head P5
- [-1, 3, C2fCIB, [1024, True, True]] # 45 (P5/32-large)
- [[39, 42, 45], 1, v10Detect, [nc]] # Detect(P3, P4, P5)
2.4 后期融合
# Ultralytics YOLO 🚀, AGPL-3.0 license
# YOLOv10 object detection model. For Usage examples see https://docs.ultralytics.com/tasks/detect
# Parameters
ch: 6
nc: 80 # number of classes
scales: # model compound scaling constants, i.e. 'model=yolov10n.yaml' will call yolov10.yaml with scale 'n'
# [depth, width, max_channels]
n: [0.33, 0.25, 1024]
backbone:
# [from, repeats, module, args]
- [-1, 1, IN, []] # 0
- [-1, 1, Multiin, [1]] # 1
- [-2, 1, Multiin, [2]] # 2
- [1, 1, Conv, [64, 3, 2]] # 3-P1/2
- [-1, 1, Conv, [128, 3, 2]] # 4-P2/4
- [-1, 3, C2f, [128, True]]
- [-1, 1, Conv, [256, 3, 2]] # 6-P3/8
- [-1, 6, C2f, [256, True]]
- [-1, 1, SCDown, [512, 3, 2]] # 8-P4/16
- [-1, 6, C2f, [512, True]]
- [-1, 1, SCDown, [1024, 3, 2]] # 10-P5/32
- [-1, 3, C2f, [1024, True]]
- [-1, 1, SPPF, [1024, 5]] # 12
- [-1, 1, PSA, [1024]] # 13
- [2, 1, Conv, [64, 3, 2]] # 14-P1/2
- [-1, 1, Conv, [128, 3, 2]] # 15-P2/4
- [-1, 3, C2f, [128, True]]
- [-1, 1, Conv, [256, 3, 2]] # 17-P3/8
- [-1, 6, C2f, [256, True]]
- [-1, 1, SCDown, [512, 3, 2]] # 19-P4/16
- [-1, 6, C2f, [512, True]]
- [-1, 1, SCDown, [1024, 3, 2]] # 21-P5/32
- [-1, 3, C2f, [1024, True]]
- [-1, 1, SPPF, [1024, 5]] # 23
- [-1, 1, PSA, [1024]] # 24
# YOLOv10.0n head
head:
- [13, 1, nn.Upsample, [None, 2, "nearest"]]
- [[-1, 9], 1, Concat, [1]] # cat backbone P4
- [-1, 3, C2f, [512]] # 27
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [[-1, 7], 1, Concat, [1]] # cat backbone P3
- [-1, 3, C2f, [256]] # 30 (P3/8-small)
- [-1, 1, Conv, [256, 3, 2]]
- [[-1, 27], 1, Concat, [1]] # cat head P4
- [-1, 3, C2f, [512]] # 33 (P4/16-medium)
- [-1, 1, SCDown, [512, 3, 2]]
- [[-1, 13], 1, Concat, [1]] # cat head P5
- [-1, 3, C2fCIB, [1024, True, True]] # 36 (P5/32-large)
- [24, 1, nn.Upsample, [None, 2, "nearest"]]
- [[-1, 20], 1, Concat, [1]] # cat backbone P4
- [-1, 3, C2f, [512]] # 39
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [[-1, 18], 1, Concat, [1]] # cat backbone P3
- [-1, 3, C2f, [256]] # 42 (P3/8-small)
- [-1, 1, Conv, [256, 3, 2]]
- [[-1, 39], 1, Concat, [1]] # cat head P4
- [-1, 3, C2f, [512]] # 45 (P4/16-medium)
- [-1, 1, SCDown, [512, 3, 2]]
- [[-1, 24], 1, Concat, [1]] # cat head P5
- [-1, 3, C2fCIB, [1024, True, True]] # 48 (P5/32-large)
- [[30, 42], 1, Concat, [1]] # 49 cat backbone P3
- [[33, 45], 1, Concat, [1]] # 50 cat backbone P4
- [[36, 48], 1, Concat, [1]] # 51 cat backbone P5
- [[49, 50, 51], 1, v10Detect, [nc]] # Detect(P3, P4, P5)
三、成功运行结果
前期融合结果： 可以看到输入的通道数为6，表明可见光图像和红外图像均输入到了模型中进行融合训练。
YOLOv10n-earlyfusion summary: 398 layers, 2,726,158 parameters, 2,726,142 gradients, 9.2 GFLOPs
from n params module arguments
0 -1 1 472 ultralytics.nn.AddModules.multimodal.MF [6, 16]
1 -1 1 2336 ultralytics.nn.modules.conv.Conv [16, 16, 3, 2]
2 -1 1 4672 ultralytics.nn.modules.conv.Conv [16, 32, 3, 2]
3 -1 1 7360 ultralytics.nn.modules.block.C2f [32, 32, 1, True]
4 -1 1 18560 ultralytics.nn.modules.conv.Conv [32, 64, 3, 2]
5 -1 2 49664 ultralytics.nn.modules.block.C2f [64, 64, 2, True]
6 -1 1 9856 ultralytics.nn.modules.block.SCDown [64, 128, 3, 2]
7 -1 2 197632 ultralytics.nn.modules.block.C2f [128, 128, 2, True]
8 -1 1 36096 ultralytics.nn.modules.block.SCDown [128, 256, 3, 2]
9 -1 1 460288 ultralytics.nn.modules.block.C2f [256, 256, 1, True]
10 -1 1 164608 ultralytics.nn.modules.block.SPPF [256, 256, 5]
11 -1 1 249728 ultralytics.nn.modules.block.PSA [256, 256]
12 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
13 [-1, 7] 1 0 ultralytics.nn.modules.conv.Concat [1]
14 -1 1 148224 ultralytics.nn.modules.block.C2f [384, 128, 1]
15 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
16 [-1, 5] 1 0 ultralytics.nn.modules.conv.Concat [1]
17 -1 1 37248 ultralytics.nn.modules.block.C2f [192, 64, 1]
18 -1 1 36992 ultralytics.nn.modules.conv.Conv [64, 64, 3, 2]
19 [-1, 12] 1 0 ultralytics.nn.modules.conv.Concat [1]
20 -1 1 140032 ultralytics.nn.modules.block.C2f [320, 128, 1]
21 -1 1 18048 ultralytics.nn.modules.block.SCDown [128, 128, 3, 2]
22 [-1, 11] 1 0 ultralytics.nn.modules.conv.Concat [1]
23 -1 1 282624 ultralytics.nn.modules.block.C2fCIB [384, 256, 1, True, True]
24 [17, 20, 23] 1 861718 ultralytics.nn.modules.head.v10Detect [1, [64, 128, 256]]
YOLOv10n-earlyfusion summary: 398 layers, 2,726,158 parameters, 2,726,142 gradients, 9.2 GFLOPs
中期融合结果：
YOLOv10n-midfusion summary: 490 layers, 3,742,134 parameters, 3,742,118 gradients, 11.5 GFLOPs
from n params module arguments
0 -1 1 0 ultralytics.nn.AddModules.multimodal.IN []
1 -1 1 0 ultralytics.nn.AddModules.multimodal.Multiin [1]
2 -2 1 0 ultralytics.nn.AddModules.multimodal.Multiin [2]
3 1 1 464 ultralytics.nn.modules.conv.Conv [3, 16, 3, 2]
4 -1 1 4672 ultralytics.nn.modules.conv.Conv [16, 32, 3, 2]
5 -1 1 7360 ultralytics.nn.modules.block.C2f [32, 32, 1, True]
6 -1 1 18560 ultralytics.nn.modules.conv.Conv [32, 64, 3, 2]
7 -1 2 49664 ultralytics.nn.modules.block.C2f [64, 64, 2, True]
8 -1 1 9856 ultralytics.nn.modules.block.SCDown [64, 128, 3, 2]
9 -1 2 197632 ultralytics.nn.modules.block.C2f [128, 128, 2, True]
10 -1 1 36096 ultralytics.nn.modules.block.SCDown [128, 256, 3, 2]
11 -1 1 460288 ultralytics.nn.modules.block.C2f [256, 256, 1, True]
12 2 1 464 ultralytics.nn.modules.conv.Conv [3, 16, 3, 2]
13 -1 1 4672 ultralytics.nn.modules.conv.Conv [16, 32, 3, 2]
14 -1 1 7360 ultralytics.nn.modules.block.C2f [32, 32, 1, True]
15 -1 1 18560 ultralytics.nn.modules.conv.Conv [32, 64, 3, 2]
16 -1 2 49664 ultralytics.nn.modules.block.C2f [64, 64, 2, True]
17 -1 1 9856 ultralytics.nn.modules.block.SCDown [64, 128, 3, 2]
18 -1 2 197632 ultralytics.nn.modules.block.C2f [128, 128, 2, True]
19 -1 1 36096 ultralytics.nn.modules.block.SCDown [128, 256, 3, 2]
20 -1 1 460288 ultralytics.nn.modules.block.C2f [256, 256, 1, True]
21 [7, 16] 1 0 ultralytics.nn.modules.conv.Concat [1]
22 [9, 18] 1 0 ultralytics.nn.modules.conv.Concat [1]
23 [11, 20] 1 0 ultralytics.nn.modules.conv.Concat [1]
24 -1 1 394240 ultralytics.nn.modules.block.SPPF [512, 256, 5]
25 -1 1 249728 ultralytics.nn.modules.block.PSA [256, 256]
26 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
27 [-1, 22] 1 0 ultralytics.nn.modules.conv.Concat [1]
28 -1 1 164608 ultralytics.nn.modules.block.C2f [512, 128, 1]
29 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
30 [-1, 21] 1 0 ultralytics.nn.modules.conv.Concat [1]
31 -1 1 41344 ultralytics.nn.modules.block.C2f [256, 64, 1]
32 -1 1 36992 ultralytics.nn.modules.conv.Conv [64, 64, 3, 2]
33 [-1, 28] 1 0 ultralytics.nn.modules.conv.Concat [1]
34 -1 1 123648 ultralytics.nn.modules.block.C2f [192, 128, 1]
35 -1 1 18048 ultralytics.nn.modules.block.SCDown [128, 128, 3, 2]
36 [-1, 25] 1 0 ultralytics.nn.modules.conv.Concat [1]
37 -1 1 282624 ultralytics.nn.modules.block.C2fCIB [384, 256, 1, True, True]
38 [31, 34, 37] 1 861718 ultralytics.nn.modules.head.v10Detect [1, [64, 128, 256]]
YOLOv10n-midfusion summary: 490 layers, 3,742,134 parameters, 3,742,118 gradients, 11.5 GFLOPs
中-后期融合结果：
YOLOv10n-mid-to-late-fusion summary: 560 layers, 4,439,350 parameters, 4,439,334 gradients, 14.6 GFLOPs
from n params module arguments
0 -1 1 0 ultralytics.nn.AddModules.multimodal.IN []
1 -1 1 0 ultralytics.nn.AddModules.multimodal.Multiin [1]
2 -2 1 0 ultralytics.nn.AddModules.multimodal.Multiin [2]
3 1 1 464 ultralytics.nn.modules.conv.Conv [3, 16, 3, 2]
4 -1 1 4672 ultralytics.nn.modules.conv.Conv [16, 32, 3, 2]
5 -1 1 7360 ultralytics.nn.modules.block.C2f [32, 32, 1, True]
6 -1 1 18560 ultralytics.nn.modules.conv.Conv [32, 64, 3, 2]
7 -1 2 49664 ultralytics.nn.modules.block.C2f [64, 64, 2, True]
8 -1 1 9856 ultralytics.nn.modules.block.SCDown [64, 128, 3, 2]
9 -1 2 197632 ultralytics.nn.modules.block.C2f [128, 128, 2, True]
10 -1 1 36096 ultralytics.nn.modules.block.SCDown [128, 256, 3, 2]
11 -1 1 460288 ultralytics.nn.modules.block.C2f [256, 256, 1, True]
12 -1 1 164608 ultralytics.nn.modules.block.SPPF [256, 256, 5]
13 -1 1 249728 ultralytics.nn.modules.block.PSA [256, 256]
14 2 1 464 ultralytics.nn.modules.conv.Conv [3, 16, 3, 2]
15 -1 1 4672 ultralytics.nn.modules.conv.Conv [16, 32, 3, 2]
16 -1 1 7360 ultralytics.nn.modules.block.C2f [32, 32, 1, True]
17 -1 1 18560 ultralytics.nn.modules.conv.Conv [32, 64, 3, 2]
18 -1 2 49664 ultralytics.nn.modules.block.C2f [64, 64, 2, True]
19 -1 1 9856 ultralytics.nn.modules.block.SCDown [64, 128, 3, 2]
20 -1 2 197632 ultralytics.nn.modules.block.C2f [128, 128, 2, True]
21 -1 1 36096 ultralytics.nn.modules.block.SCDown [128, 256, 3, 2]
22 -1 1 460288 ultralytics.nn.modules.block.C2f [256, 256, 1, True]
23 -1 1 164608 ultralytics.nn.modules.block.SPPF [256, 256, 5]
24 -1 1 249728 ultralytics.nn.modules.block.PSA [256, 256]
25 13 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
26 [-1, 9] 1 0 ultralytics.nn.modules.conv.Concat [1]
27 -1 1 148224 ultralytics.nn.modules.block.C2f [384, 128, 1]
28 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
29 [-1, 7] 1 0 ultralytics.nn.modules.conv.Concat [1]
30 -1 1 37248 ultralytics.nn.modules.block.C2f [192, 64, 1]
31 24 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
32 [-1, 20] 1 0 ultralytics.nn.modules.conv.Concat [1]
33 -1 1 148224 ultralytics.nn.modules.block.C2f [384, 128, 1]
34 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
35 [-1, 18] 1 0 ultralytics.nn.modules.conv.Concat [1]
36 -1 1 37248 ultralytics.nn.modules.block.C2f [192, 64, 1]
37 [13, 24] 1 0 ultralytics.nn.modules.conv.Concat [1]
38 [27, 33] 1 0 ultralytics.nn.modules.conv.Concat [1]
39 [30, 36] 1 0 ultralytics.nn.modules.conv.Concat [1]
40 -1 1 73856 ultralytics.nn.modules.conv.Conv [128, 64, 3, 2]
41 [-1, 38] 1 0 ultralytics.nn.modules.conv.Concat [1]
42 -1 1 140032 ultralytics.nn.modules.block.C2f [320, 128, 1]
43 -1 1 18048 ultralytics.nn.modules.block.SCDown [128, 128, 3, 2]
44 [-1, 37] 1 0 ultralytics.nn.modules.conv.Concat [1]
45 -1 1 348160 ultralytics.nn.modules.block.C2fCIB [640, 256, 1, True, True]
46 [39, 42, 45] 1 1090454 ultralytics.nn.modules.head.v10Detect [1, [128, 128, 256]]
YOLOv10n-mid-to-late-fusion summary: 560 layers, 4,439,350 parameters, 4,439,334 gradients, 14.6 GFLOPs
后期融合结果：
YOLOv10n-latefusion summary: 620 layers, 5,330,998 parameters, 5,330,982 gradients, 16.0 GFLOPs
from n params module arguments
0 -1 1 0 ultralytics.nn.AddModules.multimodal.IN []
1 -1 1 0 ultralytics.nn.AddModules.multimodal.Multiin [1]
2 -2 1 0 ultralytics.nn.AddModules.multimodal.Multiin [2]
3 1 1 464 ultralytics.nn.modules.conv.Conv [3, 16, 3, 2]
4 -1 1 4672 ultralytics.nn.modules.conv.Conv [16, 32, 3, 2]
5 -1 1 7360 ultralytics.nn.modules.block.C2f [32, 32, 1, True]
6 -1 1 18560 ultralytics.nn.modules.conv.Conv [32, 64, 3, 2]
7 -1 2 49664 ultralytics.nn.modules.block.C2f [64, 64, 2, True]
8 -1 1 9856 ultralytics.nn.modules.block.SCDown [64, 128, 3, 2]
9 -1 2 197632 ultralytics.nn.modules.block.C2f [128, 128, 2, True]
10 -1 1 36096 ultralytics.nn.modules.block.SCDown [128, 256, 3, 2]
11 -1 1 460288 ultralytics.nn.modules.block.C2f [256, 256, 1, True]
12 -1 1 164608 ultralytics.nn.modules.block.SPPF [256, 256, 5]
13 -1 1 249728 ultralytics.nn.modules.block.PSA [256, 256]
14 2 1 464 ultralytics.nn.modules.conv.Conv [3, 16, 3, 2]
15 -1 1 4672 ultralytics.nn.modules.conv.Conv [16, 32, 3, 2]
16 -1 1 7360 ultralytics.nn.modules.block.C2f [32, 32, 1, True]
17 -1 1 18560 ultralytics.nn.modules.conv.Conv [32, 64, 3, 2]
18 -1 2 49664 ultralytics.nn.modules.block.C2f [64, 64, 2, True]
19 -1 1 9856 ultralytics.nn.modules.block.SCDown [64, 128, 3, 2]
20 -1 2 197632 ultralytics.nn.modules.block.C2f [128, 128, 2, True]
21 -1 1 36096 ultralytics.nn.modules.block.SCDown [128, 256, 3, 2]
22 -1 1 460288 ultralytics.nn.modules.block.C2f [256, 256, 1, True]
23 -1 1 164608 ultralytics.nn.modules.block.SPPF [256, 256, 5]
24 -1 1 249728 ultralytics.nn.modules.block.PSA [256, 256]
25 13 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
26 [-1, 9] 1 0 ultralytics.nn.modules.conv.Concat [1]
27 -1 1 148224 ultralytics.nn.modules.block.C2f [384, 128, 1]
28 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
29 [-1, 7] 1 0 ultralytics.nn.modules.conv.Concat [1]
30 -1 1 37248 ultralytics.nn.modules.block.C2f [192, 64, 1]
31 -1 1 36992 ultralytics.nn.modules.conv.Conv [64, 64, 3, 2]
32 [-1, 27] 1 0 ultralytics.nn.modules.conv.Concat [1]
33 -1 1 123648 ultralytics.nn.modules.block.C2f [192, 128, 1]
34 -1 1 18048 ultralytics.nn.modules.block.SCDown [128, 128, 3, 2]
35 [-1, 13] 1 0 ultralytics.nn.modules.conv.Concat [1]
36 -1 1 282624 ultralytics.nn.modules.block.C2fCIB [384, 256, 1, True, True]
37 24 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
38 [-1, 20] 1 0 ultralytics.nn.modules.conv.Concat [1]
39 -1 1 148224 ultralytics.nn.modules.block.C2f [384, 128, 1]
40 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
41 [-1, 18] 1 0 ultralytics.nn.modules.conv.Concat [1]
42 -1 1 37248 ultralytics.nn.modules.block.C2f [192, 64, 1]
43 -1 1 36992 ultralytics.nn.modules.conv.Conv [64, 64, 3, 2]
44 [-1, 39] 1 0 ultralytics.nn.modules.conv.Concat [1]
45 -1 1 123648 ultralytics.nn.modules.block.C2f [192, 128, 1]
46 -1 1 18048 ultralytics.nn.modules.block.SCDown [128, 128, 3, 2]
47 [-1, 24] 1 0 ultralytics.nn.modules.conv.Concat [1]
48 -1 1 282624 ultralytics.nn.modules.block.C2fCIB [384, 256, 1, True, True]
49 [30, 42] 1 0 ultralytics.nn.modules.conv.Concat [1]
50 [33, 45] 1 0 ultralytics.nn.modules.conv.Concat [1]
51 [36, 48] 1 0 ultralytics.nn.modules.conv.Concat [1]
52 [49, 50, 51] 1 1639574 ultralytics.nn.modules.head.v10Detect [1, [128, 256, 512]]
YOLOv10n-latefusion summary: 620 layers, 5,330,998 parameters, 5,330,982 gradients, 16.0 GFLOPs
四、二次改进方案
-
多模态模型的二次改进和普通模型的改进一致，主要涉及到C2fUIB、颈部结构、上采样、下采样等，可以增加或替换成其它模块，可以换成其它的颈部结构在进行融合。若有需要可查看主页的模块改进专栏。
-
两个骨干中均可以再次添加其它模块，需要注意的是融合的时候层数要对应上，即两层的特征图大小要一致。