【YOLOv8多模态融合改进】在前期、中期、中后期、后期多模态融合中添加P2小目标检测层，完整步骤及代码
前言
主题： YOLOv8 的多模态融合改进中增加P2小目标检测层
方式： 分别在前期融合、中期融合、中-后期融合、后期融合中增加P2多模态融合检测层。
内容： 包含融合方式详解以及完整配置步骤，开箱即用，一键运行。
一、YOLOv8原始模型结构介绍
YOLOv8
原始模型结构如下：
# Ultralytics YOLO 🚀, AGPL-3.0 license
# YOLOv8 object detection model with P3-P5 outputs. For Usage examples see https://docs.ultralytics.com/tasks/detect
# Parameters
nc: 1 # number of classes
scales: # model compound scaling constants, i.e. 'model=yolov8n.yaml' will call yolov8.yaml with scale 'n'
# [depth, width, max_channels]
n: [0.33, 0.25, 1024] # YOLOv8n summary: 225 layers, 3157200 parameters, 3157184 gradients, 8.9 GFLOPs
s: [0.33, 0.50, 1024] # YOLOv8s summary: 225 layers, 11166560 parameters, 11166544 gradients, 28.8 GFLOPs
m: [0.67, 0.75, 768] # YOLOv8m summary: 295 layers, 25902640 parameters, 25902624 gradients, 79.3 GFLOPs
l: [1.00, 1.00, 512] # YOLOv8l summary: 365 layers, 43691520 parameters, 43691504 gradients, 165.7 GFLOPs
x: [1.00, 1.25, 512] # YOLOv8x summary: 365 layers, 68229648 parameters, 68229632 gradients, 258.5 GFLOPs
# YOLOv8.0n backbone
backbone:
# [from, repeats, module, args]
- [-1, 1, Conv, [64, 3, 2]] # 0-P1/2
- [-1, 1, Conv, [128, 3, 2]] # 1-P2/4
- [-1, 3, C2f, [128, True]]
- [-1, 1, Conv, [256, 3, 2]] # 3-P3/8
- [-1, 6, C2f, [256, True]]
- [-1, 1, Conv, [512, 3, 2]] # 5-P4/16
- [-1, 6, C2f, [512, True]]
- [-1, 1, Conv, [1024, 3, 2]] # 7-P5/32
- [-1, 3, C2f, [1024, True]]
- [-1, 1, SPPF, [1024, 5]] # 9
# YOLOv8.0n head
head:
- [-1, 1, nn.Upsample, [None, 2, 'nearest']]
- [[-1, 6], 1, Concat, [1]] # cat backbone P4
- [-1, 3, C2f, [512]] # 12
- [-1, 1, nn.Upsample, [None, 2, 'nearest']]
- [[-1, 4], 1, Concat, [1]] # cat backbone P3
- [-1, 3, C2f, [256]] # 15 (P3/8-small)
- [-1, 1, Conv, [256, 3, 2]]
- [[-1, 12], 1, Concat, [1]] # cat head P4
- [-1, 3, C2f, [512]] # 18 (P4/16-medium)
- [-1, 1, Conv, [512, 3, 2]]
- [[-1, 9], 1, Concat, [1]] # cat head P5
- [-1, 3, C2f, [1024]] # 21 (P5/32-large)
- [[15, 18, 21], 1, Detect, [nc]] # Detect(P3, P4, P5)
二、有效特征层对应的检测头类别
2.1 P3/8 - small检测头
-
原始模型中的
P3/8特征层对应的检测头主要用于检测相对较小的目标。其特征图大小相对较大，空间分辨率较高。 -
适合检测尺寸大概在
8x8到32x32像素左右的目标。
2.2 P4/16 - medium检测头
-
这个检测头对应的
P4/16特征层经过了更多的下采样操作，相比P3/8特征图空间分辨率降低，但通道数增加，特征更抽象且有语义信息。 -
它主要用于检测中等大小的目标，尺寸范围大概在
32x32到64x64像素左右。
2.3 P5/32 - large检测头
-
P5/32是经过最多下采样操作得到的特征层，其空间分辨率最低，但语义信息最强、全局感受野最大。 -
该检测头适合检测较大尺寸的目标，一般是尺寸在
64x64像素以上的目标。
2.4 新添加针对小目标的检测头
-
新添加的检测头主要用于检测更小尺寸的目标。尺寸在
4x4到8x8像素左右的微小目标。
💡这是因为在目标检测任务中，随着目标尺寸的减小，需要更高分辨率的特征图来有效捕捉目标特征。新添加的检测头很可能是基于这样的考虑，通过一系列的卷积、上采样和拼接等操作生成适合微小目标检测的特征图，从而提高模型对微小目标的检测能力。
三、小目标检测头多模态融合方式
-
前期融合中，在网络输入阶段将多模态数据合并后，增加针对小目标的检测层。
-
中期融合中，在骨干网络中增加针对P2的多模态特征进行融合，以此引出小目标的检测层。
-
中-后期融合中，在颈部的FPN结构中，增加针对P2的多模态特征进行融合，以此引出小目标的检测层。
-
后期融合中，在检测头前增加P2多模态特征进行融合。
四、完整配置步骤
！！！ 私信获取的项目包就已经把相关的多模态输入、训练等改动都已经配好了，只需要新建模型yaml文件，粘贴对应的模型，进行训练即可。 项目包获取及使用教程可参考链接： 《YOLO系列模型的多模态项目》配置使用教程
在什么地方新建，n,s,m,l,x,用哪个版本按自己的需求来即可，和普通的训练步骤一致。
除了模型结构方面的改动，在yaml文件中还传入了一个通道数
ch: 6
表示传入的是双模态，6通道 ，前三个是可见光，后三个是红外。
在default.yaml中也配置了这个参数。
4.1 P2前期融合
# Ultralytics YOLO 🚀, AGPL-3.0 license
# YOLOv8 object detection model with P3-P5 outputs. For Usage examples see https://docs.ultralytics.com/tasks/detect
# Parameters
ch: 6
nc: 1 # number of classes
scales: # model compound scaling constants, i.e. 'model=yolov8n.yaml' will call yolov8.yaml with scale 'n'
# [depth, width, max_channels]
n: [0.33, 0.25, 1024] # YOLOv8n summary: 225 layers, 3157200 parameters, 3157184 gradients, 8.9 GFLOPs
s: [0.33, 0.50, 1024] # YOLOv8s summary: 225 layers, 11166560 parameters, 11166544 gradients, 28.8 GFLOPs
m: [0.67, 0.75, 768] # YOLOv8m summary: 295 layers, 25902640 parameters, 25902624 gradients, 79.3 GFLOPs
l: [1.00, 1.00, 512] # YOLOv8l summary: 365 layers, 43691520 parameters, 43691504 gradients, 165.7 GFLOPs
x: [1.00, 1.25, 512] # YOLOv8x summary: 365 layers, 68229648 parameters, 68229632 gradients, 258.5 GFLOPs
# YOLOv8.0n backbone
backbone:
# [from, repeats, module, args]
- [-1, 1, MF, [64]] # 0
- [-1, 1, Conv, [64, 3, 2]] # 1-P1/2
- [-1, 1, Conv, [128, 3, 2]] # 2-P2/4
- [-1, 3, C2f, [128, True]]
- [-1, 1, Conv, [256, 3, 2]] # 4-P3/8
- [-1, 6, C2f, [256, True]]
- [-1, 1, Conv, [512, 3, 2]] # 6-P4/16
- [-1, 6, C2f, [512, True]]
- [-1, 1, Conv, [1024, 3, 2]] # 8-P5/32
- [-1, 3, C2f, [1024, True]]
- [-1, 1, SPPF, [1024, 5]] # 10
# YOLOv8.0n head
head:
- [-1, 1, nn.Upsample, [None, 2, 'nearest']]
- [[-1, 7], 1, Concat, [1]] # cat backbone P4
- [-1, 3, C2f, [512]] # 13
- [-1, 1, nn.Upsample, [None, 2, 'nearest']]
- [[-1, 5], 1, Concat, [1]] # cat backbone P3
- [-1, 3, C2f, [256]] # 16
- [-1, 1, nn.Upsample, [None, 2, 'nearest']]
- [[-1, 3], 1, Concat, [1]] # cat backbone P2
- [-1, 3, C2f, [256]] # 19 (P2/4-xsmall)
- [-1, 1, Conv, [256, 3, 2]]
- [[-1, 16], 1, Concat, [1]] # cat head P3
- [-1, 3, C2f, [512]] # 22 (P3/8-xsmall)
- [-1, 1, Conv, [256, 3, 2]]
- [[-1, 13], 1, Concat, [1]] # cat head P4
- [-1, 3, C2f, [512]] # 25 (P4/16-medium)
- [-1, 1, Conv, [512, 3, 2]]
- [[-1, 10], 1, Concat, [1]] # cat head P5
- [-1, 3, C2f, [1024]] # 28 (P5/32-large)
- [[19, 22, 25, 28], 1, Detect, [nc]] # Detect(P3, P4, P5)
4.2 P2中期融合
# Ultralytics YOLO 🚀, AGPL-3.0 license
# YOLOv8 object detection model with P3-P5 outputs. For Usage examples see https://docs.ultralytics.com/tasks/detect
# Parameters
ch: 6
nc: 1 # number of classes
scales: # model compound scaling constants, i.e. 'model=yolov8n.yaml' will call yolov8.yaml with scale 'n'
# [depth, width, max_channels]
n: [0.33, 0.25, 1024] # YOLOv8n summary: 225 layers, 3157200 parameters, 3157184 gradients, 8.9 GFLOPs
s: [0.33, 0.50, 1024] # YOLOv8s summary: 225 layers, 11166560 parameters, 11166544 gradients, 28.8 GFLOPs
m: [0.67, 0.75, 768] # YOLOv8m summary: 295 layers, 25902640 parameters, 25902624 gradients, 79.3 GFLOPs
l: [1.00, 1.00, 512] # YOLOv8l summary: 365 layers, 43691520 parameters, 43691504 gradients, 165.7 GFLOPs
x: [1.00, 1.25, 512] # YOLOv8x summary: 365 layers, 68229648 parameters, 68229632 gradients, 258.5 GFLOPs
# YOLOv8.0n backbone
backbone:
# [from, repeats, module, args]
- [-1, 1, IN, []] # 0
- [-1, 1, Multiin, [1]] # 1
- [-2, 1, Multiin, [2]] # 2
- [1, 1, Conv, [64, 3, 2]] # 3-P1/2
- [-1, 1, Conv, [128, 3, 2]] # 4-P2/4
- [-1, 3, C2f, [128, True]]
- [-1, 1, Conv, [256, 3, 2]] # 6-P3/8
- [-1, 6, C2f, [256, True]]
- [-1, 1, Conv, [512, 3, 2]] # 8-P4/16
- [-1, 6, C2f, [512, True]]
- [-1, 1, Conv, [1024, 3, 2]] # 10-P5/32
- [-1, 3, C2f, [1024, True]]
- [2, 1, Conv, [64, 3, 2]] # 12-P1/2
- [-1, 1, Conv, [128, 3, 2]] # 13-P2/4
- [-1, 3, C2f, [128, True]]
- [-1, 1, Conv, [256, 3, 2]] # 15-P3/8
- [-1, 6, C2f, [256, True]]
- [-1, 1, Conv, [512, 3, 2]] # 17-P4/16
- [-1, 6, C2f, [512, True]]
- [-1, 1, Conv, [1024, 3, 2]] # 19-P5/32
- [-1, 3, C2f, [1024, True]]
- [[5, 14], 1, Concat, [1]] # 21 cat backbone P2
- [[7, 16], 1, Concat, [1]] # 22 cat backbone P3
- [[9, 18], 1, Concat, [1]] # 23 cat backbone P4
- [[11, 20], 1, Concat, [1]] # 24 cat backbone P5
- [-1, 1, SPPF, [1024, 5]] # 25
# YOLOv8.0n head
head:
- [-1, 1, nn.Upsample, [None, 2, 'nearest']]
- [[-1, 23], 1, Concat, [1]] # cat backbone P4
- [-1, 3, C2f, [512]] # 28
- [-1, 1, nn.Upsample, [None, 2, 'nearest']]
- [[-1, 22], 1, Concat, [1]] # cat backbone P3
- [-1, 3, C2f, [256]] # 31
- [-1, 1, nn.Upsample, [None, 2, 'nearest']]
- [[-1, 21], 1, Concat, [1]] # cat backbone P2
- [-1, 3, C2f, [128]] # 34 (P2/4-xsmall)
- [-1, 1, Conv, [128, 3, 2]]
- [[-1, 31], 1, Concat, [1]] # cat head P3
- [-1, 3, C2f, [256]] # 37 (P3/16-medium)
- [-1, 1, Conv, [256, 3, 2]]
- [[-1, 28], 1, Concat, [1]] # cat head P4
- [-1, 3, C2f, [512]] # 40 (P4/16-medium)
- [-1, 1, Conv, [512, 3, 2]]
- [[-1, 25], 1, Concat, [1]] # cat head P5
- [-1, 3, C2f, [1024]] # 43 (P5/32-large)
- [[34, 37, 40, 43], 1, Detect, [nc]] # Detect(P3, P4, P5)
4.3 P2中-后期融合
# Ultralytics YOLO 🚀, AGPL-3.0 license
# YOLOv8 object detection model with P3-P5 outputs. For Usage examples see https://docs.ultralytics.com/tasks/detect
# Parameters
ch: 6
nc: 1 # number of classes
scales: # model compound scaling constants, i.e. 'model=yolov8n.yaml' will call yolov8.yaml with scale 'n'
# [depth, width, max_channels]
n: [0.33, 0.25, 1024] # YOLOv8n summary: 225 layers, 3157200 parameters, 3157184 gradients, 8.9 GFLOPs
s: [0.33, 0.50, 1024] # YOLOv8s summary: 225 layers, 11166560 parameters, 11166544 gradients, 28.8 GFLOPs
m: [0.67, 0.75, 768] # YOLOv8m summary: 295 layers, 25902640 parameters, 25902624 gradients, 79.3 GFLOPs
l: [1.00, 1.00, 512] # YOLOv8l summary: 365 layers, 43691520 parameters, 43691504 gradients, 165.7 GFLOPs
x: [1.00, 1.25, 512] # YOLOv8x summary: 365 layers, 68229648 parameters, 68229632 gradients, 258.5 GFLOPs
# YOLOv8.0n backbone
backbone:
# [from, repeats, module, args]
- [-1, 1, IN, []] # 0
- [-1, 1, Multiin, [1]] # 1
- [-2, 1, Multiin, [2]] # 2
- [1, 1, Conv, [64, 3, 2]] # 3-P1/2
- [-1, 1, Conv, [128, 3, 2]] # 4-P2/4
- [-1, 3, C2f, [128, True]]
- [-1, 1, Conv, [256, 3, 2]] # 6-P3/8
- [-1, 6, C2f, [256, True]]
- [-1, 1, Conv, [512, 3, 2]] # 8-P4/16
- [-1, 6, C2f, [512, True]]
- [-1, 1, Conv, [1024, 3, 2]] # 10-P5/32
- [-1, 3, C2f, [1024, True]]
- [-1, 1, SPPF, [1024, 5]] # 12
- [2, 1, Conv, [64, 3, 2]] # 13-P1/2
- [-1, 1, Conv, [128, 3, 2]] # 14-P2/4
- [-1, 3, C2f, [128, True]]
- [-1, 1, Conv, [256, 3, 2]] # 16-P3/8
- [-1, 6, C2f, [256, True]]
- [-1, 1, Conv, [512, 3, 2]] # 18-P4/16
- [-1, 6, C2f, [512, True]]
- [-1, 1, Conv, [1024, 3, 2]] # 20-P5/32
- [-1, 3, C2f, [1024, True]]
- [-1, 1, SPPF, [1024, 5]] # 22
# YOLOv8.0n head
head:
- [12, 1, nn.Upsample, [None, 2, 'nearest']]
- [[-1, 9], 1, Concat, [1]] # cat backbone P4
- [-1, 3, C2f, [512]] # 25
- [-1, 1, nn.Upsample, [None, 2, 'nearest']]
- [[-1, 7], 1, Concat, [1]] # cat backbone P3
- [-1, 3, C2f, [256]] # 28 (P3/8-small)
- [-1, 1, nn.Upsample, [None, 2, 'nearest']]
- [[-1, 5], 1, Concat, [1]] # cat backbone P3
- [-1, 3, C2f, [128]] # 31 (P3/8-small)
- [22, 1, nn.Upsample, [None, 2, 'nearest']]
- [[-1, 19], 1, Concat, [1]] # cat backbone P4
- [-1, 3, C2f, [512]] # 34
- [-1, 1, nn.Upsample, [None, 2, 'nearest']]
- [[-1, 17], 1, Concat, [1]] # cat backbone P3
- [-1, 3, C2f, [256]] # 37 (P3/8-small)
- [-1, 1, nn.Upsample, [None, 2, 'nearest']]
- [[-1, 15], 1, Concat, [1]] # cat backbone P3
- [-1, 3, C2f, [128]] # 40 (P3/8-small)
- [ [ 12, 22 ], 1, Concat, [ 1 ] ] # cat head P2 41
- [ [ 25, 34 ], 1, Concat, [ 1 ] ] # cat head P3 42
- [ [ 28, 37 ], 1, Concat, [ 1 ] ] # cat head P4 43
- [ [ 31, 40 ], 1, Concat, [ 1 ] ] # cat head P5 44
- [-1, 1, Conv, [128, 3, 2]]
- [[-1, 43], 1, Concat, [1]] # cat head P4
- [-1, 3, C2f, [256]] # 47 (P4/16-medium)
- [-1, 1, Conv, [256, 3, 2]]
- [[-1, 42], 1, Concat, [1]] # cat head P4
- [-1, 3, C2f, [512]] # 50 (P4/16-medium)
- [-1, 1, Conv, [512, 3, 2]]
- [[-1, 41], 1, Concat, [1]] # cat head P5
- [-1, 3, C2f, [1024]] # 53 (P5/32-large)
- [[44, 47, 50, 53], 1, Detect, [nc]] # Detect(P3, P4, P5)
4.4 P2后期融合
# Ultralytics YOLO 🚀, AGPL-3.0 license
# YOLOv8 object detection model with P3-P5 outputs. For Usage examples see https://docs.ultralytics.com/tasks/detect
# Parameters
ch: 6
nc: 1 # number of classes
scales: # model compound scaling constants, i.e. 'model=yolov8n.yaml' will call yolov8.yaml with scale 'n'
# [depth, width, max_channels]
n: [0.33, 0.25, 1024] # YOLOv8n summary: 225 layers, 3157200 parameters, 3157184 gradients, 8.9 GFLOPs
s: [0.33, 0.50, 1024] # YOLOv8s summary: 225 layers, 11166560 parameters, 11166544 gradients, 28.8 GFLOPs
m: [0.67, 0.75, 768] # YOLOv8m summary: 295 layers, 25902640 parameters, 25902624 gradients, 79.3 GFLOPs
l: [1.00, 1.00, 512] # YOLOv8l summary: 365 layers, 43691520 parameters, 43691504 gradients, 165.7 GFLOPs
x: [1.00, 1.25, 512] # YOLOv8x summary: 365 layers, 68229648 parameters, 68229632 gradients, 258.5 GFLOPs
# YOLOv8.0n backbone
backbone:
# [from, repeats, module, args]
- [-1, 1, IN, []] # 0
- [-1, 1, Multiin, [1]] # 1
- [-2, 1, Multiin, [2]] # 2
- [1, 1, Conv, [64, 3, 2]] # 3-P1/2
- [-1, 1, Conv, [128, 3, 2]] # 4-P2/4
- [-1, 3, C2f, [128, True]]
- [-1, 1, Conv, [256, 3, 2]] # 6-P3/8
- [-1, 6, C2f, [256, True]]
- [-1, 1, Conv, [512, 3, 2]] # 8-P4/16
- [-1, 6, C2f, [512, True]]
- [-1, 1, Conv, [1024, 3, 2]] # 10-P5/32
- [-1, 3, C2f, [1024, True]]
- [-1, 1, SPPF, [1024, 5]] # 12
- [2, 1, Conv, [64, 3, 2]] # 13-P1/2
- [-1, 1, Conv, [128, 3, 2]] # 14-P2/4
- [-1, 3, C2f, [128, True]]
- [-1, 1, Conv, [256, 3, 2]] # 16-P3/8
- [-1, 6, C2f, [256, True]]
- [-1, 1, Conv, [512, 3, 2]] # 18-P4/16
- [-1, 6, C2f, [512, True]]
- [-1, 1, Conv, [1024, 3, 2]] # 20-P5/32
- [-1, 3, C2f, [1024, True]]
- [-1, 1, SPPF, [1024, 5]] # 22
# YOLOv8.0n head
head:
- [12, 1, nn.Upsample, [None, 2, 'nearest']]
- [[-1, 9], 1, Concat, [1] ] # cat backbone P4
- [-1, 3, C2f, [512]] # 25
- [-1, 1, nn.Upsample, [None, 2, 'nearest']]
- [[ -1, 7], 1, Concat, [1]] # cat backbone P3
- [-1, 3, C2f, [256]] # 28
- [-1, 1, nn.Upsample, [None, 2, 'nearest']]
- [[ -1, 5], 1, Concat, [1]] # cat backbone P3
- [-1, 3, C2f, [128]] # 31 (P2/4-xsmall)
- [-1, 1, Conv, [128, 3, 2]]
- [[-1, 28], 1, Concat, [1]] # cat head P4
- [-1, 3, C2f, [256]] # 34 (P3/8-small)
- [-1, 1, Conv, [256, 3, 2]]
- [[-1, 25], 1, Concat, [1]] # cat head P4
- [-1, 3, C2f, [512]] # 37 (P4/16-medium)
- [-1, 1, Conv, [512, 3, 2]]
- [[-1, 12], 1, Concat, [1]] # cat head P5
- [-1, 3, C2f, [1024]] # 40 (P5/32-large)
- [22, 1, nn.Upsample, [None, 2, 'nearest']]
- [[-1, 19], 1, Concat, [1]] # cat backbone P4
- [-1, 3, C2f, [512]] # 43
- [-1, 1, nn.Upsample, [None, 2, 'nearest']]
- [[-1, 17], 1, Concat, [1]] # cat backbone P3
- [-1, 3, C2f, [256]] # 46
- [-1, 1, nn.Upsample, [None, 2, 'nearest']]
- [[-1, 15], 1, Concat, [1]] # cat backbone P3
- [-1, 3, C2f, [256]] # 49 (P2/4-xsmall)
- [-1, 1, Conv, [256, 3, 2]]
- [[-1, 46], 1, Concat, [1]] # cat head P4
- [-1, 3, C2f, [512]] # 52 (P3/8-small)
- [-1, 1, Conv, [256, 3, 2]]
- [[-1, 43], 1, Concat, [1]] # cat head P4
- [-1, 3, C2f, [512]] # 55 (P4/16-medium)
- [-1, 1, Conv, [512, 3, 2]]
- [[-1, 22], 1, Concat, [1]] # cat head P5
- [-1, 3, C2f, [1024]] # 58 (P5/32-large)
- [[31, 49], 1, Concat, [1]] # cat head P2 59
- [[34, 52], 1, Concat, [1]] # cat head P3 60
- [[37, 55], 1, Concat, [1]] # cat head P4 61
- [[40, 58], 1, Concat, [1]] # cat head P5 62
- [[59, 60, 61, 62], 1, Detect, [nc]] # Detect(P3, P4, P5)
五、成功运行结果
前期融合结果： 可以看到输入的通道数为6，表明可见光图像和红外图像均输入到了模型中进行融合训练。
YOLOv8-early-p2 summary: 322 layers, 3,042,892 parameters, 3,042,876 gradients, 16.5 GFLOPs
from n params module arguments
0 -1 1 472 ultralytics.nn.AddModules.multimodal.MF [6, 16]
1 -1 1 2336 ultralytics.nn.modules.conv.Conv [16, 16, 3, 2]
2 -1 1 4672 ultralytics.nn.modules.conv.Conv [16, 32, 3, 2]
3 -1 1 7360 ultralytics.nn.modules.block.C2f [32, 32, 1, True]
4 -1 1 18560 ultralytics.nn.modules.conv.Conv [32, 64, 3, 2]
5 -1 2 49664 ultralytics.nn.modules.block.C2f [64, 64, 2, True]
6 -1 1 73984 ultralytics.nn.modules.conv.Conv [64, 128, 3, 2]
7 -1 2 197632 ultralytics.nn.modules.block.C2f [128, 128, 2, True]
8 -1 1 295424 ultralytics.nn.modules.conv.Conv [128, 256, 3, 2]
9 -1 1 460288 ultralytics.nn.modules.block.C2f [256, 256, 1, True]
10 -1 1 164608 ultralytics.nn.modules.block.SPPF [256, 256, 5]
11 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
12 [-1, 7] 1 0 ultralytics.nn.modules.conv.Concat [1]
13 -1 1 148224 ultralytics.nn.modules.block.C2f [384, 128, 1]
14 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
15 [-1, 5] 1 0 ultralytics.nn.modules.conv.Concat [1]
16 -1 1 37248 ultralytics.nn.modules.block.C2f [192, 64, 1]
17 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
18 [-1, 3] 1 0 ultralytics.nn.modules.conv.Concat [1]
19 -1 1 31104 ultralytics.nn.modules.block.C2f [96, 64, 1]
20 -1 1 36992 ultralytics.nn.modules.conv.Conv [64, 64, 3, 2]
21 [-1, 16] 1 0 ultralytics.nn.modules.conv.Concat [1]
22 -1 1 115456 ultralytics.nn.modules.block.C2f [128, 128, 1]
23 -1 1 73856 ultralytics.nn.modules.conv.Conv [128, 64, 3, 2]
24 [-1, 13] 1 0 ultralytics.nn.modules.conv.Concat [1]
25 -1 1 123648 ultralytics.nn.modules.block.C2f [192, 128, 1]
26 -1 1 147712 ultralytics.nn.modules.conv.Conv [128, 128, 3, 2]
27 [-1, 10] 1 0 ultralytics.nn.modules.conv.Concat [1]
28 -1 1 493056 ultralytics.nn.modules.block.C2f [384, 256, 1]
29 [19, 22, 25, 28] 1 560596 ultralytics.nn.modules.head.Detect [1, [64, 128, 128, 256]]
YOLOv8-early-p2 summary: 322 layers, 3,042,892 parameters, 3,042,876 gradients, 16.5 GFLOPs
中期融合结果：
YOLOv8-mid-p2 summary: 405 layers, 4,136,916 parameters, 4,136,900 gradients, 14.5 GFLOPs
from n params module arguments
0 -1 1 0 ultralytics.nn.AddModules.multimodal.IN []
1 -1 1 0 ultralytics.nn.AddModules.multimodal.Multiin [1]
2 -2 1 0 ultralytics.nn.AddModules.multimodal.Multiin [2]
3 1 1 464 ultralytics.nn.modules.conv.Conv [3, 16, 3, 2]
4 -1 1 4672 ultralytics.nn.modules.conv.Conv [16, 32, 3, 2]
5 -1 1 7360 ultralytics.nn.modules.block.C2f [32, 32, 1, True]
6 -1 1 18560 ultralytics.nn.modules.conv.Conv [32, 64, 3, 2]
7 -1 2 49664 ultralytics.nn.modules.block.C2f [64, 64, 2, True]
8 -1 1 73984 ultralytics.nn.modules.conv.Conv [64, 128, 3, 2]
9 -1 2 197632 ultralytics.nn.modules.block.C2f [128, 128, 2, True]
10 -1 1 295424 ultralytics.nn.modules.conv.Conv [128, 256, 3, 2]
11 -1 1 460288 ultralytics.nn.modules.block.C2f [256, 256, 1, True]
12 2 1 464 ultralytics.nn.modules.conv.Conv [3, 16, 3, 2]
13 -1 1 4672 ultralytics.nn.modules.conv.Conv [16, 32, 3, 2]
14 -1 1 7360 ultralytics.nn.modules.block.C2f [32, 32, 1, True]
15 -1 1 18560 ultralytics.nn.modules.conv.Conv [32, 64, 3, 2]
16 -1 2 49664 ultralytics.nn.modules.block.C2f [64, 64, 2, True]
17 -1 1 73984 ultralytics.nn.modules.conv.Conv [64, 128, 3, 2]
18 -1 2 197632 ultralytics.nn.modules.block.C2f [128, 128, 2, True]
19 -1 1 295424 ultralytics.nn.modules.conv.Conv [128, 256, 3, 2]
20 -1 1 460288 ultralytics.nn.modules.block.C2f [256, 256, 1, True]
21 [5, 14] 1 0 ultralytics.nn.modules.conv.Concat [1]
22 [7, 16] 1 0 ultralytics.nn.modules.conv.Concat [1]
23 [9, 18] 1 0 ultralytics.nn.modules.conv.Concat [1]
24 [11, 20] 1 0 ultralytics.nn.modules.conv.Concat [1]
25 -1 1 394240 ultralytics.nn.modules.block.SPPF [512, 256, 5]
26 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
27 [-1, 23] 1 0 ultralytics.nn.modules.conv.Concat [1]
28 -1 1 164608 ultralytics.nn.modules.block.C2f [512, 128, 1]
29 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
30 [-1, 22] 1 0 ultralytics.nn.modules.conv.Concat [1]
31 -1 1 41344 ultralytics.nn.modules.block.C2f [256, 64, 1]
32 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
33 [-1, 21] 1 0 ultralytics.nn.modules.conv.Concat [1]
34 -1 1 10432 ultralytics.nn.modules.block.C2f [128, 32, 1]
35 -1 1 9280 ultralytics.nn.modules.conv.Conv [32, 32, 3, 2]
36 [-1, 31] 1 0 ultralytics.nn.modules.conv.Concat [1]
37 -1 1 31104 ultralytics.nn.modules.block.C2f [96, 64, 1]
38 -1 1 36992 ultralytics.nn.modules.conv.Conv [64, 64, 3, 2]
39 [-1, 28] 1 0 ultralytics.nn.modules.conv.Concat [1]
40 -1 1 123648 ultralytics.nn.modules.block.C2f [192, 128, 1]
41 -1 1 147712 ultralytics.nn.modules.conv.Conv [128, 128, 3, 2]
42 [-1, 25] 1 0 ultralytics.nn.modules.conv.Concat [1]
43 -1 1 493056 ultralytics.nn.modules.block.C2f [384, 256, 1]
44 [34, 37, 40, 43] 1 468404 ultralytics.nn.modules.head.Detect [1, [32, 64, 128, 256]]
YOLOv8-mid-p2 summary: 405 layers, 4,136,916 parameters, 4,136,900 gradients, 14.5 GFLOPs
中-后期融合结果：
YOLOv8-mid-to-late-p2 summary: 464 layers, 4,391,028 parameters, 4,391,012 gradients, 17.5 GFLOPs
from n params module arguments
0 -1 1 0 ultralytics.nn.AddModules.multimodal.IN []
1 -1 1 0 ultralytics.nn.AddModules.multimodal.Multiin [1]
2 -2 1 0 ultralytics.nn.AddModules.multimodal.Multiin [2]
3 1 1 464 ultralytics.nn.modules.conv.Conv [3, 16, 3, 2]
4 -1 1 4672 ultralytics.nn.modules.conv.Conv [16, 32, 3, 2]
5 -1 1 7360 ultralytics.nn.modules.block.C2f [32, 32, 1, True]
6 -1 1 18560 ultralytics.nn.modules.conv.Conv [32, 64, 3, 2]
7 -1 2 49664 ultralytics.nn.modules.block.C2f [64, 64, 2, True]
8 -1 1 73984 ultralytics.nn.modules.conv.Conv [64, 128, 3, 2]
9 -1 2 197632 ultralytics.nn.modules.block.C2f [128, 128, 2, True]
10 -1 1 295424 ultralytics.nn.modules.conv.Conv [128, 256, 3, 2]
11 -1 1 460288 ultralytics.nn.modules.block.C2f [256, 256, 1, True]
12 -1 1 164608 ultralytics.nn.modules.block.SPPF [256, 256, 5]
13 2 1 464 ultralytics.nn.modules.conv.Conv [3, 16, 3, 2]
14 -1 1 4672 ultralytics.nn.modules.conv.Conv [16, 32, 3, 2]
15 -1 1 7360 ultralytics.nn.modules.block.C2f [32, 32, 1, True]
16 -1 1 18560 ultralytics.nn.modules.conv.Conv [32, 64, 3, 2]
17 -1 2 49664 ultralytics.nn.modules.block.C2f [64, 64, 2, True]
18 -1 1 73984 ultralytics.nn.modules.conv.Conv [64, 128, 3, 2]
19 -1 2 197632 ultralytics.nn.modules.block.C2f [128, 128, 2, True]
20 -1 1 295424 ultralytics.nn.modules.conv.Conv [128, 256, 3, 2]
21 -1 1 460288 ultralytics.nn.modules.block.C2f [256, 256, 1, True]
22 -1 1 164608 ultralytics.nn.modules.block.SPPF [256, 256, 5]
23 12 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
24 [-1, 9] 1 0 ultralytics.nn.modules.conv.Concat [1]
25 -1 1 148224 ultralytics.nn.modules.block.C2f [384, 128, 1]
26 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
27 [-1, 7] 1 0 ultralytics.nn.modules.conv.Concat [1]
28 -1 1 37248 ultralytics.nn.modules.block.C2f [192, 64, 1]
29 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
30 [-1, 5] 1 0 ultralytics.nn.modules.conv.Concat [1]
31 -1 1 9408 ultralytics.nn.modules.block.C2f [96, 32, 1]
32 22 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
33 [-1, 19] 1 0 ultralytics.nn.modules.conv.Concat [1]
34 -1 1 148224 ultralytics.nn.modules.block.C2f [384, 128, 1]
35 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
36 [-1, 17] 1 0 ultralytics.nn.modules.conv.Concat [1]
37 -1 1 37248 ultralytics.nn.modules.block.C2f [192, 64, 1]
38 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
39 [-1, 15] 1 0 ultralytics.nn.modules.conv.Concat [1]
40 -1 1 9408 ultralytics.nn.modules.block.C2f [96, 32, 1]
41 [12, 22] 1 0 ultralytics.nn.modules.conv.Concat [1]
42 [25, 34] 1 0 ultralytics.nn.modules.conv.Concat [1]
43 [28, 37] 1 0 ultralytics.nn.modules.conv.Concat [1]
44 [31, 40] 1 0 ultralytics.nn.modules.conv.Concat [1]
45 -1 1 18496 ultralytics.nn.modules.conv.Conv [64, 32, 3, 2]
46 [-1, 43] 1 0 ultralytics.nn.modules.conv.Concat [1]
47 -1 1 35200 ultralytics.nn.modules.block.C2f [160, 64, 1]
48 -1 1 36992 ultralytics.nn.modules.conv.Conv [64, 64, 3, 2]
49 [-1, 42] 1 0 ultralytics.nn.modules.conv.Concat [1]
50 -1 1 140032 ultralytics.nn.modules.block.C2f [320, 128, 1]
51 -1 1 147712 ultralytics.nn.modules.conv.Conv [128, 128, 3, 2]
52 [-1, 41] 1 0 ultralytics.nn.modules.conv.Concat [1]
53 -1 1 558592 ultralytics.nn.modules.block.C2f [640, 256, 1]
54 [44, 47, 50, 53] 1 518932 ultralytics.nn.modules.head.Detect [1, [64, 64, 128, 256]]
YOLOv8-mid-to-late-p2 summary: 464 layers, 4,391,028 parameters, 4,391,012 gradients, 17.5 GFLOPs
后期融合结果：
YOLOv8-late-p2 summary: 521 layers, 5,718,676 parameters, 5,718,660 gradients, 24.6 GFLOPs
from n params module arguments
0 -1 1 0 ultralytics.nn.AddModules.multimodal.IN []
1 -1 1 0 ultralytics.nn.AddModules.multimodal.Multiin [1]
2 -2 1 0 ultralytics.nn.AddModules.multimodal.Multiin [2]
3 1 1 464 ultralytics.nn.modules.conv.Conv [3, 16, 3, 2]
4 -1 1 4672 ultralytics.nn.modules.conv.Conv [16, 32, 3, 2]
5 -1 1 7360 ultralytics.nn.modules.block.C2f [32, 32, 1, True]
6 -1 1 18560 ultralytics.nn.modules.conv.Conv [32, 64, 3, 2]
7 -1 2 49664 ultralytics.nn.modules.block.C2f [64, 64, 2, True]
8 -1 1 73984 ultralytics.nn.modules.conv.Conv [64, 128, 3, 2]
9 -1 2 197632 ultralytics.nn.modules.block.C2f [128, 128, 2, True]
10 -1 1 295424 ultralytics.nn.modules.conv.Conv [128, 256, 3, 2]
11 -1 1 460288 ultralytics.nn.modules.block.C2f [256, 256, 1, True]
12 -1 1 164608 ultralytics.nn.modules.block.SPPF [256, 256, 5]
13 2 1 464 ultralytics.nn.modules.conv.Conv [3, 16, 3, 2]
14 -1 1 4672 ultralytics.nn.modules.conv.Conv [16, 32, 3, 2]
15 -1 1 7360 ultralytics.nn.modules.block.C2f [32, 32, 1, True]
16 -1 1 18560 ultralytics.nn.modules.conv.Conv [32, 64, 3, 2]
17 -1 2 49664 ultralytics.nn.modules.block.C2f [64, 64, 2, True]
18 -1 1 73984 ultralytics.nn.modules.conv.Conv [64, 128, 3, 2]
19 -1 2 197632 ultralytics.nn.modules.block.C2f [128, 128, 2, True]
20 -1 1 295424 ultralytics.nn.modules.conv.Conv [128, 256, 3, 2]
21 -1 1 460288 ultralytics.nn.modules.block.C2f [256, 256, 1, True]
22 -1 1 164608 ultralytics.nn.modules.block.SPPF [256, 256, 5]
23 12 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
24 [-1, 9] 1 0 ultralytics.nn.modules.conv.Concat [1]
25 -1 1 148224 ultralytics.nn.modules.block.C2f [384, 128, 1]
26 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
27 [-1, 7] 1 0 ultralytics.nn.modules.conv.Concat [1]
28 -1 1 37248 ultralytics.nn.modules.block.C2f [192, 64, 1]
29 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
30 [-1, 5] 1 0 ultralytics.nn.modules.conv.Concat [1]
31 -1 1 9408 ultralytics.nn.modules.block.C2f [96, 32, 1]
32 -1 1 9280 ultralytics.nn.modules.conv.Conv [32, 32, 3, 2]
33 [-1, 28] 1 0 ultralytics.nn.modules.conv.Concat [1]
34 -1 1 31104 ultralytics.nn.modules.block.C2f [96, 64, 1]
35 -1 1 36992 ultralytics.nn.modules.conv.Conv [64, 64, 3, 2]
36 [-1, 25] 1 0 ultralytics.nn.modules.conv.Concat [1]
37 -1 1 123648 ultralytics.nn.modules.block.C2f [192, 128, 1]
38 -1 1 147712 ultralytics.nn.modules.conv.Conv [128, 128, 3, 2]
39 [-1, 12] 1 0 ultralytics.nn.modules.conv.Concat [1]
40 -1 1 493056 ultralytics.nn.modules.block.C2f [384, 256, 1]
41 22 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
42 [-1, 19] 1 0 ultralytics.nn.modules.conv.Concat [1]
43 -1 1 148224 ultralytics.nn.modules.block.C2f [384, 128, 1]
44 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
45 [-1, 17] 1 0 ultralytics.nn.modules.conv.Concat [1]
46 -1 1 37248 ultralytics.nn.modules.block.C2f [192, 64, 1]
47 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
48 [-1, 15] 1 0 ultralytics.nn.modules.conv.Concat [1]
49 -1 1 31104 ultralytics.nn.modules.block.C2f [96, 64, 1]
50 -1 1 36992 ultralytics.nn.modules.conv.Conv [64, 64, 3, 2]
51 [-1, 46] 1 0 ultralytics.nn.modules.conv.Concat [1]
52 -1 1 115456 ultralytics.nn.modules.block.C2f [128, 128, 1]
53 -1 1 73856 ultralytics.nn.modules.conv.Conv [128, 64, 3, 2]
54 [-1, 43] 1 0 ultralytics.nn.modules.conv.Concat [1]
55 -1 1 123648 ultralytics.nn.modules.block.C2f [192, 128, 1]
56 -1 1 147712 ultralytics.nn.modules.conv.Conv [128, 128, 3, 2]
57 [-1, 22] 1 0 ultralytics.nn.modules.conv.Concat [1]
58 -1 1 493056 ultralytics.nn.modules.block.C2f [384, 256, 1]
59 [31, 49] 1 0 ultralytics.nn.modules.conv.Concat [1]
60 [34, 52] 1 0 ultralytics.nn.modules.conv.Concat [1]
61 [37, 55] 1 0 ultralytics.nn.modules.conv.Concat [1]
62 [40, 58] 1 0 ultralytics.nn.modules.conv.Concat [1]
63 [59, 60, 61, 62] 1 929396 ultralytics.nn.modules.head.Detect [1, [96, 192, 256, 512]]
YOLOv8-late-p2 summary: 521 layers, 5,718,676 parameters, 5,718,660 gradients, 24.6 GFLOPs