【YOLOv11多模态融合改进】| SDFM 表层细节融合模块，利用通道-空间注意力机制，实现跨模态特征融合，抑制噪声干扰
一、本文介绍
本文记录的是利用 SDFM 模块改进 YOLOv11 的多模态融合部分 。
SDFM模块（Surface Detail Fusion Module，表层细节融合模块） 通过在特征提取网络的浅层引入 通道-空间注意力机制，动态生成跨模态特征融合权重 。该模块可自适应 保留不同模态中的独特信息，抑制背景噪声与光照干扰 ， 实现低层细节的精准对齐与互补增强 ，为后续检测提供 高保真度 的底层特征表示，从而提升模型在复杂场景下的目标检测鲁棒性与定位准确性。
二、SDFM模块介绍
Rethinking the necessity of image fusion in high-level vision tasks: A practical infrared and visible image fusion network based on progressive semantic injection and scene fidelity
2.1 出发点
- 解决浅层特征融合的局限性 ：传统像素级融合（如直接叠加）易导致细节冗余或丢失，而SDFM通过 通道-空间注意力机制 动态调整特征权重，实现“结构保留”与“噪声抑制”的平衡。
- 为高层语义融合奠定基础 ：确保浅层特征中的目标位置（如红外热目标）与可见光场景结构（如建筑物轮廓）准确对齐，避免高层语义注入时出现位置偏差。
2.2 结构原理：基于通道-空间注意力的特征调制
2.2.1 核心组件与流程
SDFM模块的架构如图所示，主要包含以下步骤：
-
特征增强 ：
- 将红外与可见光的浅层特征在通道维度拼接（ C ( F i r i , F v i i ) C(\mathcal{F}_{ir}^{i}, \mathcal{F}_{vi}^{i}) C ( F i r i , F v i i ) ），通过 全局平均池化（GAP） 和 逐点卷积（Pw-Conv） 生成通道注意力权重（ δ ( P w − C o n v n ( G A P ( ⋅ ) ) ) \delta(Pw-Conv^{n}(GAP(\cdot))) δ ( Pw − C o n v n ( G A P ( ⋅ ))) ）。
-
权重通过元素乘法作用于原始特征，并与另一分支的特征相加，实现跨模态特征增强：
F ^ i r i = F i r i ⊕ ( F v i i ⊗ δ ( ⋅ ) ) , F ^ v i i = F v i i ⊕ ( F i r i ⊗ δ ( ⋅ ) ) \hat{\mathcal{F}}_{ir}^{i} = \mathcal{F}_{ir}^{i} \oplus \left(\mathcal{F}_{vi}^{i} \otimes \delta(\cdot)\right), \quad \hat{\mathcal{F}}_{vi}^{i} = \mathcal{F}_{vi}^{i} \oplus \left(\mathcal{F}_{ir}^{i} \otimes \delta(\cdot)\right) F ^ i r i = F i r i ⊕ ( F v i i ⊗ δ ( ⋅ ) ) , F ^ v i i = F v i i ⊕ ( F i r i ⊗ δ ( ⋅ ) )
（其中 ⊕ \oplus ⊕ 为元素相加， ⊗ \otimes ⊗ 为元素相乘， δ \delta δ 为Sigmoid激活函数）
-
融合权重生成 ：
- 将增强后的特征再次拼接，分别输入 通道注意力模块 和 空间注意力模块 ，生成通道权重（ A C i \mathcal{A}_{C}^{i} A C i ）和空间权重（ A S i \mathcal{A}_{S}^{i} A S i ）。
-
融合权重为两者的元素乘积，经Sigmoid激活后得到最终权重（
W
i
\mathcal{W}^{i}
W
i
）：
W i = δ ( A C i ⊗ A S i ) \mathcal{W}^{i} = \delta\left(\mathcal{A}_{C}^{i} \otimes \mathcal{A}_{S}^{i}\right) W i = δ ( A C i ⊗ A S i )
-
特征融合 ：
-
根据融合权重动态分配红外与可见光特征的贡献：
F f u i = ( W i ⊗ F ^ i r i ) ⊕ ( ( 1 − W i ) ⊗ F ^ v i i ) \mathcal{F}_{fu}^{i} = \left(\mathcal{W}^{i} \otimes \hat{\mathcal{F}}_{ir}^{i}\right) \oplus \left(\left(1-\mathcal{W}^{i}\right) \otimes \hat{\mathcal{F}}_{vi}^{i}\right) F f u i = ( W i ⊗ F ^ i r i ) ⊕ ( ( 1 − W i ) ⊗ F ^ v i i )
（在结构清晰区域（如可见光边缘）保留可见光特征，在热目标区域（如红外热力图）增强红外特征）。
-
根据融合权重动态分配红外与可见光特征的贡献：
2.3 关键技术特点
- 轻量化设计 ：仅通过逐点卷积（1×1卷积）和池化操作实现注意力机制，计算量小，适用于浅层特征的实时处理。
- 跨模态交互 ：通过通道-空间双重注意力，迫使模型关注红外与可见光特征的互补区域（如红外中的目标位置+可见光中的纹理细节），抑制无关噪声（如红外背景噪声或可见光低光照噪点）。
论文： https://www.sciencedirect.com/science/article/abs/pii/S1566253523001860
源码： https://github.com/Linfeng-Tang/PSFusion
三、SDFM的实现代码
SDFM
的实现代码如下：
import math
import torch.nn as nn
import torch
def autopad(k, p=None, d=1): # kernel, padding, dilation
"""Pad to 'same' shape outputs."""
if d > 1:
k = d * (k - 1) + 1 if isinstance(k, int) else [d * (x - 1) + 1 for x in k] # actual kernel-size
if p is None:
p = k // 2 if isinstance(k, int) else [x // 2 for x in k] # auto-pad
return p
class Conv(nn.Module):
"""Standard convolution with args(ch_in, ch_out, kernel, stride, padding, groups, dilation, activation)."""
default_act = nn.SiLU() # default activation
def __init__(self, c1, c2, k=1, s=1, p=None, g=1, d=1, act=True):
"""Initialize Conv layer with given arguments including activation."""
super().__init__()
self.conv = nn.Conv2d(c1, c2, k, s, autopad(k, p, d), groups=g, dilation=d, bias=False)
self.act = self.default_act if act is True else act if isinstance(act, nn.Module) else nn.Identity()
def bn(self, x):
"""Apply layer normalization to the input tensor."""
return nn.LayerNorm(x.shape[1:]).to(x.device)(x)
def forward(self, x):
"""Apply convolution, batch normalization and activation to input tensor."""
return self.act(self.bn(self.conv(x)))
def forward_fuse(self, x):
"""Perform transposed convolution of 2D data."""
return self.act(self.conv(x))
class SDFM(nn.Module):
'''
superficial detail fusion module
'''
def __init__(self, channels=64, r=4):
super(SDFM, self).__init__()
inter_channels = int(channels // r)
self.Recalibrate = nn.Sequential(
nn.AdaptiveAvgPool2d(1),
Conv(2 * channels, 2 * inter_channels),
Conv(2 * inter_channels, 2 * channels, act=nn.Sigmoid()),
)
self.channel_agg = Conv(2 * channels, channels)
self.local_att = nn.Sequential(
Conv(channels, inter_channels, 1),
Conv(inter_channels, channels, 1, act=False),
)
self.global_att = nn.Sequential(
nn.AdaptiveAvgPool2d(1),
Conv(channels, inter_channels, 1),
Conv(inter_channels, channels, 1),
)
self.sigmoid = nn.Sigmoid()
# 确保模型在初始化时就被移到 CUDA 上（如果可用）
if torch.cuda.is_available():
self.to('cuda')
def forward(self, data):
x1, x2 = data
# 将输入数据移动到与模型相同的设备上
device = next(self.parameters()).device
x1 = x1.to(device)
x2 = x2.to(device)
_, c, _, _ = x1.shape
input = torch.cat([x1, x2], dim=1)
recal_w = self.Recalibrate(input)
recal_input = recal_w * input ## 先对特征进行一步自校正
recal_input = recal_input + input
x1, x2 = torch.split(recal_input, c, dim=1)
agg_input = self.channel_agg(recal_input) ## 进行特征压缩 因为只计算一个特征的权重
local_w = self.local_att(agg_input) ## 局部注意力 即spatial attention
global_w = self.global_att(agg_input) ## 全局注意力 即channel attention
w = self.sigmoid(local_w * global_w) ## 计算特征x1的权重
xo = w * x1 + (1 - w) * x2 ## fusion results ## 特征聚合
return xo
四、融合步骤
4.1 修改一
① 在
ultralytics/nn/
目录下新建
AddModules
文件夹用于存放模块代码
② 在
AddModules
文件夹下新建
SDFM.py
，将
第三节
中的代码粘贴到此处
4.2 修改二
在
AddModules
文件夹下新建
__init__.py
（已有则不用新建），在文件内导入模块：
from .SDFM import *
4.3 修改三
在
ultralytics/nn/modules/tasks.py
文件中，需要在两处位置添加各模块类名称。
首先：导入模块
其次：在
parse_model函数
中注册
SDFM
模块
elif m in {SDFM}:
c2 = ch[f[0]]
args = [c2]
在
DetectionModel
类下，添加如下代码
try:
m.stride = torch.tensor([s / x.shape[-2] for x in _forward(torch.zeros(1, ch, s, s))]) # forward on CPU
except RuntimeError:
try:
self.model.to(torch.device('cuda'))
m.stride = torch.tensor([s / x.shape[-2] for x in _forward(
torch.zeros(1, ch, s, s).to(torch.device('cuda')))]) # forward on CUDA
except RuntimeError as error:
raise error
并注释这一行
# m.stride = torch.tensor([s / x.shape[-2] for x in _forward(torch.zeros(1, ch, s, s))]) # forward
五、yaml模型文件
5.1 中期融合⭐
📌 此模型的修方法是将原本的中期融合中的Concat融合部分换成SDFM，融合骨干部分的多模态信息。
# Ultralytics YOLO 🚀, AGPL-3.0 license
# YOLO11 object detection model with P3-P5 outputs. For Usage examples see https://docs.ultralytics.com/tasks/detect
# Parameters
ch: 6
nc: 80 # number of classes
scales: # model compound scaling constants, i.e. 'model=yolo11n.yaml' will call yolo11.yaml with scale 'n'
# [depth, width, max_channels]
n: [0.50, 0.25, 1024] # summary: 319 layers, 2624080 parameters, 2624064 gradients, 6.6 GFLOPs
s: [0.50, 0.50, 1024] # summary: 319 layers, 9458752 parameters, 9458736 gradients, 21.7 GFLOPs
m: [0.50, 1.00, 512] # summary: 409 layers, 20114688 parameters, 20114672 gradients, 68.5 GFLOPs
l: [1.00, 1.00, 512] # summary: 631 layers, 25372160 parameters, 25372144 gradients, 87.6 GFLOPs
x: [1.00, 1.50, 512] # summary: 631 layers, 56966176 parameters, 56966160 gradients, 196.0 GFLOPs
# YOLO11n backbone
backbone:
# [from, repeats, module, args]
- [-1, 1, IN, []] # 0
- [-1, 1, Multiin, [1]] # 1
- [-2, 1, Multiin, [2]] # 2
- [1, 1, Conv, [64, 3, 2]] # 3-P1/2
- [-1, 1, Conv, [128, 3, 2]] # 4-P2/4
- [-1, 2, C3k2, [256, False, 0.25]]
- [-1, 1, Conv, [256, 3, 2]] # 6-P3/8
- [-1, 2, C3k2, [512, False, 0.25]]
- [-1, 1, Conv, [512, 3, 2]] # 8-P4/16
- [-1, 2, C3k2, [512, True]]
- [-1, 1, Conv, [1024, 3, 2]] # 10-P5/32
- [-1, 2, C3k2, [1024, True]]
- [2, 1, Conv, [64, 3, 2]] # 12-P1/2
- [-1, 1, Conv, [128, 3, 2]] # 13-P2/4
- [-1, 2, C3k2, [256, False, 0.25]]
- [-1, 1, Conv, [256, 3, 2]] # 15-P3/8
- [-1, 2, C3k2, [512, False, 0.25]]
- [-1, 1, Conv, [512, 3, 2]] # 17-P4/16
- [-1, 2, C3k2, [512, True]]
- [-1, 1, Conv, [1024, 3, 2]] # 19-P5/32
- [-1, 2, C3k2, [1024, True]]
- [[7, 16], 1, SDFM, []] # 21 cat backbone P3
- [[9, 18], 1, SDFM, []] # 22 cat backbone P4
- [[11, 20], 1, SDFM, []] # 23 cat backbone P5
- [-1, 1, SPPF, [1024, 5]] # 24
- [-1, 2, C2PSA, [1024]] # 25
# YOLO11n head
head:
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [[-1, 22], 1, Concat, [1]] # cat backbone P4
- [-1, 2, C3k2, [512, False]] # 28
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [[-1, 21], 1, Concat, [1]] # cat backbone P3
- [-1, 2, C3k2, [256, False]] # 31 (P3/8-small)
- [-1, 1, Conv, [256, 3, 2]]
- [[-1, 28], 1, Concat, [1]] # cat head P4
- [-1, 2, C3k2, [512, False]] # 34 (P4/16-medium)
- [-1, 1, Conv, [512, 3, 2]]
- [[-1, 25], 1, Concat, [1]] # cat head P5
- [-1, 2, C3k2, [1024, True]] # 37 (P5/32-large)
- [[31, 34, 37], 1, Detect, [nc]] # Detect(P3, P4, P5)
5.2 中-后期融合⭐
📌 此模型的修方法是将原本的中-后期融合中的Concat融合部分换成SDFM，融合FPN部分的多模态信息。
# Ultralytics YOLO 🚀, AGPL-3.0 license
# YOLO11 object detection model with P3-P5 outputs. For Usage examples see https://docs.ultralytics.com/tasks/detect
# Parameters
ch: 6
nc: 80 # number of classes
scales: # model compound scaling constants, i.e. 'model=yolo11n.yaml' will call yolo11.yaml with scale 'n'
# [depth, width, max_channels]
n: [0.50, 0.25, 1024] # summary: 319 layers, 2624080 parameters, 2624064 gradients, 6.6 GFLOPs
s: [0.50, 0.50, 1024] # summary: 319 layers, 9458752 parameters, 9458736 gradients, 21.7 GFLOPs
m: [0.50, 1.00, 512] # summary: 409 layers, 20114688 parameters, 20114672 gradients, 68.5 GFLOPs
l: [1.00, 1.00, 512] # summary: 631 layers, 25372160 parameters, 25372144 gradients, 87.6 GFLOPs
x: [1.00, 1.50, 512] # summary: 631 layers, 56966176 parameters, 56966160 gradients, 196.0 GFLOPs
# YOLO11n backbone
backbone:
# [from, repeats, module, args]
- [-1, 1, IN, []] # 0
- [-1, 1, Multiin, [1]] # 1
- [-2, 1, Multiin, [2]] # 2
- [1, 1, Conv, [64, 3, 2]] # 3-P1/2
- [-1, 1, Conv, [128, 3, 2]] # 4-P2/4
- [-1, 2, C3k2, [256, False, 0.25]]
- [-1, 1, Conv, [256, 3, 2]] # 6-P3/8
- [-1, 2, C3k2, [512, False, 0.25]]
- [-1, 1, Conv, [512, 3, 2]] # 8-P4/16
- [-1, 2, C3k2, [512, True]]
- [-1, 1, Conv, [1024, 3, 2]] # 10-P5/32
- [-1, 2, C3k2, [1024, True]]
- [-1, 1, SPPF, [1024, 5]] # 12
- [-1, 2, C2PSA, [1024]] # 13
- [2, 1, Conv, [64, 3, 2]] # 14-P1/2
- [-1, 1, Conv, [128, 3, 2]] # 15-P2/4
- [-1, 2, C3k2, [256, False, 0.25]]
- [-1, 1, Conv, [256, 3, 2]] # 17-P3/8
- [-1, 2, C3k2, [512, False, 0.25]]
- [-1, 1, Conv, [512, 3, 2]] # 19-P4/16
- [-1, 2, C3k2, [512, True]]
- [-1, 1, Conv, [1024, 3, 2]] # 21-P5/32
- [-1, 2, C3k2, [1024, True]]
- [-1, 1, SPPF, [1024, 5]] # 23
- [-1, 2, C2PSA, [1024]] # 24
# YOLO11n head
head:
- [13, 1, nn.Upsample, [None, 2, "nearest"]]
- [[-1, 9], 1, Concat, [1]] # cat backbone P4
- [-1, 2, C3k2, [512, False]] # 27
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [[-1, 7], 1, Concat, [1]] # cat backbone P3
- [-1, 2, C3k2, [256, False]] # 30 (P3/8-small)
- [24, 1, nn.Upsample, [None, 2, "nearest"]]
- [[-1, 20], 1, Concat, [1]] # cat backbone P4
- [-1, 2, C3k2, [512, False]] # 33
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [[-1, 18], 1, Concat, [1]] # cat backbone P3
- [-1, 2, C3k2, [256, False]] # 36 (P3/8-small)
- [[13, 24], 1, SDFM, []] # 37 cat backbone P3
- [[27, 33], 1, SDFM, []] # 38 cat backbone P4
- [[30, 36], 1, SDFM, []] # 39 cat backbone P5
- [-1, 1, Conv, [256, 3, 2]]
- [[-1, 38], 1, Concat, [1]] # cat head P4
- [-1, 2, C3k2, [512, False]] # 42 (P4/16-medium)
- [-1, 1, Conv, [512, 3, 2]]
- [[-1, 37], 1, Concat, [1]] # cat head P5
- [-1, 2, C3k2, [1024, True]] # 45 (P5/32-large)
- [[39, 42, 45], 1, Detect, [nc]] # Detect(P3, P4, P5)
5.3 后期融合⭐
📌 此模型的修方法是将原本的后期融合中的Concat融合部分换成SDFM，融合颈部部分的多模态信息。
# Ultralytics YOLO 🚀, AGPL-3.0 license
# YOLO11 object detection model with P3-P5 outputs. For Usage examples see https://docs.ultralytics.com/tasks/detect
# Parameters
ch: 6
nc: 80 # number of classes
scales: # model compound scaling constants, i.e. 'model=yolo11n.yaml' will call yolo11.yaml with scale 'n'
# [depth, width, max_channels]
n: [0.50, 0.25, 1024] # summary: 319 layers, 2624080 parameters, 2624064 gradients, 6.6 GFLOPs
s: [0.50, 0.50, 1024] # summary: 319 layers, 9458752 parameters, 9458736 gradients, 21.7 GFLOPs
m: [0.50, 1.00, 512] # summary: 409 layers, 20114688 parameters, 20114672 gradients, 68.5 GFLOPs
l: [1.00, 1.00, 512] # summary: 631 layers, 25372160 parameters, 25372144 gradients, 87.6 GFLOPs
x: [1.00, 1.50, 512] # summary: 631 layers, 56966176 parameters, 56966160 gradients, 196.0 GFLOPs
# YOLO11n backbone
backbone:
# [from, repeats, module, args]
- [-1, 1, IN, []] # 0
- [-1, 1, Multiin, [1]] # 1
- [-2, 1, Multiin, [2]] # 2
- [1, 1, Conv, [64, 3, 2]] # 3-P1/2
- [-1, 1, Conv, [128, 3, 2]] # 4-P2/4
- [-1, 2, C3k2, [256, False, 0.25]]
- [-1, 1, Conv, [256, 3, 2]] # 6-P3/8
- [-1, 2, C3k2, [512, False, 0.25]]
- [-1, 1, Conv, [512, 3, 2]] # 8-P4/16
- [-1, 2, C3k2, [512, True]]
- [-1, 1, Conv, [1024, 3, 2]] # 10-P5/32
- [-1, 2, C3k2, [1024, True]]
- [-1, 1, SPPF, [1024, 5]] # 12
- [-1, 2, C2PSA, [1024]] # 13
- [2, 1, Conv, [64, 3, 2]] # 14-P1/2
- [-1, 1, Conv, [128, 3, 2]] # 15-P2/4
- [-1, 2, C3k2, [256, False, 0.25]]
- [-1, 1, Conv, [256, 3, 2]] # 17-P3/8
- [-1, 2, C3k2, [512, False, 0.25]]
- [-1, 1, Conv, [512, 3, 2]] # 19-P4/16
- [-1, 2, C3k2, [512, True]]
- [-1, 1, Conv, [1024, 3, 2]] # 21-P5/32
- [-1, 2, C3k2, [1024, True]]
- [-1, 1, SPPF, [1024, 5]] # 23
- [-1, 2, C2PSA, [1024]] # 24
# YOLO11n head
head:
- [13, 1, nn.Upsample, [None, 2, "nearest"]]
- [[-1, 9], 1, Concat, [1]] # cat backbone P4
- [-1, 2, C3k2, [512, False]] # 27
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [[-1, 7], 1, Concat, [1]] # cat backbone P3
- [-1, 2, C3k2, [256, False]] # 30 (P3/8-small)
- [-1, 1, Conv, [256, 3, 2]]
- [[-1, 27], 1, Concat, [1]] # cat head P4
- [-1, 2, C3k2, [512, False]] # 33 (P4/16-medium)
- [-1, 1, Conv, [512, 3, 2]]
- [[-1, 13], 1, Concat, [1]] # cat head P5
- [-1, 2, C3k2, [1024, True]] # 36 (P5/32-large)
- [24, 1, nn.Upsample, [None, 2, "nearest"]]
- [[-1, 20], 1, Concat, [1]] # cat backbone P4
- [-1, 2, C3k2, [512, False]] # 39
- [-1, 1, nn.Upsample, [None, 2, "nearest"]]
- [[-1, 18], 1, Concat, [1]] # cat backbone P3
- [-1, 2, C3k2, [256, False]] # 42 (P3/8-small)
- [-1, 1, Conv, [256, 3, 2]]
- [[-1, 39], 1, Concat, [1]] # cat head P4
- [-1, 2, C3k2, [512, False]] # 45 (P4/16-medium)
- [-1, 1, Conv, [512, 3, 2]]
- [[-1, 24], 1, Concat, [1]] # cat head P5
- [-1, 2, C3k2, [1024, True]] # 48 (P5/32-large)
- [[30, 42], 1, SDFM, []] # cat head P5 49
- [[33, 45], 1, SDFM, []] # cat head P5 50
- [[36, 48], 1, SDFM, []] # cat head P5 51
- [[49, 50, 51], 1, Detect, [nc]] # Detect(P3, P4, P5)
六、成功运行结果
打印网络模型可以看到不同的融合层已经加入到模型中，并可以进行训练了。
YOLOv11-mid-SDFM ：
YOLO11-mid-SDFM summary: 503 layers, 4,032,691 parameters, 4,032,675 gradients, 10.2 GFLOPs
from n params module arguments
0 -1 1 0 ultralytics.nn.AddModules.multimodal.IN []
1 -1 1 0 ultralytics.nn.AddModules.multimodal.Multiin [1]
2 -2 1 0 ultralytics.nn.AddModules.multimodal.Multiin [2]
3 1 1 464 ultralytics.nn.modules.conv.Conv [3, 16, 3, 2]
4 -1 1 4672 ultralytics.nn.modules.conv.Conv [16, 32, 3, 2]
5 -1 1 6640 ultralytics.nn.modules.block.C3k2 [32, 64, 1, False, 0.25]
6 -1 1 36992 ultralytics.nn.modules.conv.Conv [64, 64, 3, 2]
7 -1 1 26080 ultralytics.nn.modules.block.C3k2 [64, 128, 1, False, 0.25]
8 -1 1 147712 ultralytics.nn.modules.conv.Conv [128, 128, 3, 2]
9 -1 1 87040 ultralytics.nn.modules.block.C3k2 [128, 128, 1, True]
10 -1 1 295424 ultralytics.nn.modules.conv.Conv [128, 256, 3, 2]
11 -1 1 346112 ultralytics.nn.modules.block.C3k2 [256, 256, 1, True]
12 2 1 464 ultralytics.nn.modules.conv.Conv [3, 16, 3, 2]
13 -1 1 4672 ultralytics.nn.modules.conv.Conv [16, 32, 3, 2]
14 -1 1 6640 ultralytics.nn.modules.block.C3k2 [32, 64, 1, False, 0.25]
15 -1 1 36992 ultralytics.nn.modules.conv.Conv [64, 64, 3, 2]
16 -1 1 26080 ultralytics.nn.modules.block.C3k2 [64, 128, 1, False, 0.25]
17 -1 1 147712 ultralytics.nn.modules.conv.Conv [128, 128, 3, 2]
18 -1 1 87040 ultralytics.nn.modules.block.C3k2 [128, 128, 1, True]
19 -1 1 295424 ultralytics.nn.modules.conv.Conv [128, 256, 3, 2]
20 -1 1 346112 ultralytics.nn.modules.block.C3k2 [256, 256, 1, True]
21 [7, 16] 1 81920 ultralytics.nn.AddModules.SDFM.SDFM [128]
22 [9, 18] 1 81920 ultralytics.nn.AddModules.SDFM.SDFM [128]
23 [11, 20] 1 327680 ultralytics.nn.AddModules.SDFM.SDFM [256]
24 -1 1 164608 ultralytics.nn.modules.block.SPPF [256, 256, 5]
25 -1 1 249728 ultralytics.nn.modules.block.C2PSA [256, 256, 1]
26 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
27 [-1, 22] 1 0 ultralytics.nn.modules.conv.Concat [1]
28 -1 1 111296 ultralytics.nn.modules.block.C3k2 [384, 128, 1, False]
29 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
30 [-1, 21] 1 0 ultralytics.nn.modules.conv.Concat [1]
31 -1 1 32096 ultralytics.nn.modules.block.C3k2 [256, 64, 1, False]
32 -1 1 36992 ultralytics.nn.modules.conv.Conv [64, 64, 3, 2]
33 [-1, 28] 1 0 ultralytics.nn.modules.conv.Concat [1]
34 -1 1 86720 ultralytics.nn.modules.block.C3k2 [192, 128, 1, False]
35 -1 1 147712 ultralytics.nn.modules.conv.Conv [128, 128, 3, 2]
36 [-1, 25] 1 0 ultralytics.nn.modules.conv.Concat [1]
37 -1 1 378880 ultralytics.nn.modules.block.C3k2 [384, 256, 1, True]
38 [31, 34, 37] 1 430867 ultralytics.nn.modules.head.Detect [1, [64, 128, 256]]
YOLO11-mid-SDFM summary: 503 layers, 4,032,691 parameters, 4,032,675 gradients, 10.2 GFLOPs
YOLOv11-mid-to-late-SDFM ：
YOLO11-mid-to-late-SDFM summary: 575 layers, 4,528,979 parameters, 4,528,963 gradients, 10.9 GFLOPs
from n params module arguments
0 -1 1 0 ultralytics.nn.AddModules.multimodal.IN []
1 -1 1 0 ultralytics.nn.AddModules.multimodal.Multiin [1]
2 -2 1 0 ultralytics.nn.AddModules.multimodal.Multiin [2]
3 1 1 464 ultralytics.nn.modules.conv.Conv [3, 16, 3, 2]
4 -1 1 4672 ultralytics.nn.modules.conv.Conv [16, 32, 3, 2]
5 -1 1 6640 ultralytics.nn.modules.block.C3k2 [32, 64, 1, False, 0.25]
6 -1 1 36992 ultralytics.nn.modules.conv.Conv [64, 64, 3, 2]
7 -1 1 26080 ultralytics.nn.modules.block.C3k2 [64, 128, 1, False, 0.25]
8 -1 1 147712 ultralytics.nn.modules.conv.Conv [128, 128, 3, 2]
9 -1 1 87040 ultralytics.nn.modules.block.C3k2 [128, 128, 1, True]
10 -1 1 295424 ultralytics.nn.modules.conv.Conv [128, 256, 3, 2]
11 -1 1 346112 ultralytics.nn.modules.block.C3k2 [256, 256, 1, True]
12 -1 1 164608 ultralytics.nn.modules.block.SPPF [256, 256, 5]
13 -1 1 249728 ultralytics.nn.modules.block.C2PSA [256, 256, 1]
14 2 1 464 ultralytics.nn.modules.conv.Conv [3, 16, 3, 2]
15 -1 1 4672 ultralytics.nn.modules.conv.Conv [16, 32, 3, 2]
16 -1 1 6640 ultralytics.nn.modules.block.C3k2 [32, 64, 1, False, 0.25]
17 -1 1 36992 ultralytics.nn.modules.conv.Conv [64, 64, 3, 2]
18 -1 1 26080 ultralytics.nn.modules.block.C3k2 [64, 128, 1, False, 0.25]
19 -1 1 147712 ultralytics.nn.modules.conv.Conv [128, 128, 3, 2]
20 -1 1 87040 ultralytics.nn.modules.block.C3k2 [128, 128, 1, True]
21 -1 1 295424 ultralytics.nn.modules.conv.Conv [128, 256, 3, 2]
22 -1 1 346112 ultralytics.nn.modules.block.C3k2 [256, 256, 1, True]
23 -1 1 164608 ultralytics.nn.modules.block.SPPF [256, 256, 5]
24 -1 1 249728 ultralytics.nn.modules.block.C2PSA [256, 256, 1]
25 13 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
26 [-1, 9] 1 0 ultralytics.nn.modules.conv.Concat [1]
27 -1 1 111296 ultralytics.nn.modules.block.C3k2 [384, 128, 1, False]
28 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
29 [-1, 7] 1 0 ultralytics.nn.modules.conv.Concat [1]
30 -1 1 32096 ultralytics.nn.modules.block.C3k2 [256, 64, 1, False]
31 24 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
32 [-1, 20] 1 0 ultralytics.nn.modules.conv.Concat [1]
33 -1 1 111296 ultralytics.nn.modules.block.C3k2 [384, 128, 1, False]
34 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
35 [-1, 18] 1 0 ultralytics.nn.modules.conv.Concat [1]
36 -1 1 32096 ultralytics.nn.modules.block.C3k2 [256, 64, 1, False]
37 [13, 24] 1 327680 ultralytics.nn.AddModules.SDFM.SDFM [256]
38 [27, 33] 1 81920 ultralytics.nn.AddModules.SDFM.SDFM [128]
39 [30, 36] 1 20480 ultralytics.nn.AddModules.SDFM.SDFM [64]
40 -1 1 36992 ultralytics.nn.modules.conv.Conv [64, 64, 3, 2]
41 [-1, 38] 1 0 ultralytics.nn.modules.conv.Concat [1]
42 -1 1 86720 ultralytics.nn.modules.block.C3k2 [192, 128, 1, False]
43 -1 1 147712 ultralytics.nn.modules.conv.Conv [128, 128, 3, 2]
44 [-1, 37] 1 0 ultralytics.nn.modules.conv.Concat [1]
45 -1 1 378880 ultralytics.nn.modules.block.C3k2 [384, 256, 1, True]
46 [39, 42, 45] 1 430867 ultralytics.nn.modules.head.Detect [1, [64, 128, 256]]
YOLO11-mid-to-late-SDFM summary: 575 layers, 4,528,979 parameters, 4,528,963 gradients, 10.9 GFLOPs
YOLOv11-late-SDFM ：
YOLO11-late-SDFM summary: 631 layers, 5,179,283 parameters, 5,179,267 gradients, 11.7 GFLOPs
from n params module arguments
0 -1 1 0 ultralytics.nn.AddModules.multimodal.IN []
1 -1 1 0 ultralytics.nn.AddModules.multimodal.Multiin [1]
2 -2 1 0 ultralytics.nn.AddModules.multimodal.Multiin [2]
3 1 1 464 ultralytics.nn.modules.conv.Conv [3, 16, 3, 2]
4 -1 1 4672 ultralytics.nn.modules.conv.Conv [16, 32, 3, 2]
5 -1 1 6640 ultralytics.nn.modules.block.C3k2 [32, 64, 1, False, 0.25]
6 -1 1 36992 ultralytics.nn.modules.conv.Conv [64, 64, 3, 2]
7 -1 1 26080 ultralytics.nn.modules.block.C3k2 [64, 128, 1, False, 0.25]
8 -1 1 147712 ultralytics.nn.modules.conv.Conv [128, 128, 3, 2]
9 -1 1 87040 ultralytics.nn.modules.block.C3k2 [128, 128, 1, True]
10 -1 1 295424 ultralytics.nn.modules.conv.Conv [128, 256, 3, 2]
11 -1 1 346112 ultralytics.nn.modules.block.C3k2 [256, 256, 1, True]
12 -1 1 164608 ultralytics.nn.modules.block.SPPF [256, 256, 5]
13 -1 1 249728 ultralytics.nn.modules.block.C2PSA [256, 256, 1]
14 2 1 464 ultralytics.nn.modules.conv.Conv [3, 16, 3, 2]
15 -1 1 4672 ultralytics.nn.modules.conv.Conv [16, 32, 3, 2]
16 -1 1 6640 ultralytics.nn.modules.block.C3k2 [32, 64, 1, False, 0.25]
17 -1 1 36992 ultralytics.nn.modules.conv.Conv [64, 64, 3, 2]
18 -1 1 26080 ultralytics.nn.modules.block.C3k2 [64, 128, 1, False, 0.25]
19 -1 1 147712 ultralytics.nn.modules.conv.Conv [128, 128, 3, 2]
20 -1 1 87040 ultralytics.nn.modules.block.C3k2 [128, 128, 1, True]
21 -1 1 295424 ultralytics.nn.modules.conv.Conv [128, 256, 3, 2]
22 -1 1 346112 ultralytics.nn.modules.block.C3k2 [256, 256, 1, True]
23 -1 1 164608 ultralytics.nn.modules.block.SPPF [256, 256, 5]
24 -1 1 249728 ultralytics.nn.modules.block.C2PSA [256, 256, 1]
25 13 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
26 [-1, 9] 1 0 ultralytics.nn.modules.conv.Concat [1]
27 -1 1 111296 ultralytics.nn.modules.block.C3k2 [384, 128, 1, False]
28 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
29 [-1, 7] 1 0 ultralytics.nn.modules.conv.Concat [1]
30 -1 1 32096 ultralytics.nn.modules.block.C3k2 [256, 64, 1, False]
31 -1 1 36992 ultralytics.nn.modules.conv.Conv [64, 64, 3, 2]
32 [-1, 27] 1 0 ultralytics.nn.modules.conv.Concat [1]
33 -1 1 86720 ultralytics.nn.modules.block.C3k2 [192, 128, 1, False]
34 -1 1 147712 ultralytics.nn.modules.conv.Conv [128, 128, 3, 2]
35 [-1, 13] 1 0 ultralytics.nn.modules.conv.Concat [1]
36 -1 1 378880 ultralytics.nn.modules.block.C3k2 [384, 256, 1, True]
37 24 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
38 [-1, 20] 1 0 ultralytics.nn.modules.conv.Concat [1]
39 -1 1 111296 ultralytics.nn.modules.block.C3k2 [384, 128, 1, False]
40 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
41 [-1, 18] 1 0 ultralytics.nn.modules.conv.Concat [1]
42 -1 1 32096 ultralytics.nn.modules.block.C3k2 [256, 64, 1, False]
43 -1 1 36992 ultralytics.nn.modules.conv.Conv [64, 64, 3, 2]
44 [-1, 39] 1 0 ultralytics.nn.modules.conv.Concat [1]
45 -1 1 86720 ultralytics.nn.modules.block.C3k2 [192, 128, 1, False]
46 -1 1 147712 ultralytics.nn.modules.conv.Conv [128, 128, 3, 2]
47 [-1, 24] 1 0 ultralytics.nn.modules.conv.Concat [1]
48 -1 1 378880 ultralytics.nn.modules.block.C3k2 [384, 256, 1, True]
49 [30, 42] 1 20480 ultralytics.nn.AddModules.SDFM.SDFM [64]
50 [33, 45] 1 81920 ultralytics.nn.AddModules.SDFM.SDFM [128]
51 [36, 48] 1 327680 ultralytics.nn.AddModules.SDFM.SDFM [256]
52 [49, 50, 51] 1 430867 ultralytics.nn.modules.head.Detect [1, [64, 128, 256]]
YOLO11-late-SDFM summary: 631 layers, 5,179,283 parameters, 5,179,267 gradients, 11.7 GFLOPs