| 基于OpenPose关键点检测和卷积神经网络驱动的虚拟试衣方法研究 |
| A virtual fitting method based on OpenPose key‑point detection and convolutional neural networks |
| |
| DOI： |
| 中文关键词: 虚拟试衣；OpenPose；关键点检测；卷积神经网络；端到端训练 |
| 英文关键词:virtual fitting； OpenPose； key-point detection； convolutional neural network （CNN）； end-to-end training |
| 基金项目:福建省教育厅中青年教师教育科研项目（社科类）（JAS24194）、福建省教育厅中青年教师教育科研项目（科技类）（JT180830）和闽南科技学院教研项目（MKJG-2024-008）资助项目 |
| 作者 | 单位 | | 唐成俏 | 闽南科技学院 艺术设计学院，福建 泉州 362332 | | 董志学 | 中央财经大学 统计与数学学院，北京 100081 | | 朱振杰 | 山东大学 机械工程学院，山东 济南 250061 | | 彭青梅 | 闽南科技学院 计算机信息学院，福建 泉州 362332 |
|
| 摘要点击次数: 157 |
| 全文下载次数: 259 |
| 中文摘要: |
| 针对线上服装购物缺乏真实试穿，服装与人体姿态难以精确匹配等问题，提出一种基于
OpenPose关键点检测和卷积神经网络驱动的虚拟试衣方法。该方法以单张人物图像和目标图像
为输入，首先利用 OpenPose提取人体关键点并构造姿态热力图，随后通过服装分割与仿射/TPS变
换实现服装区域的几何预对齐，最后将人物图像、姿态表示和对齐服装在编码-解码式卷积网络中
进行特征融合，并结合重建损失、感知损失和掩膜一致性损失进行端到端训练。实验在 VITON-HD、DressCode 以及 DeepFashion2 子集 3 个公开数据集上开展。结果表明，在 SSIM 和 LPIPS 指标
上，所提方法在多种姿态场景下均取得更优或具有竞争力的性能。消融实验进一步验证了 OpenPose姿态约束、服装几何对齐以及感知损失和掩膜损失是关键组成部分。 |
| 英文摘要: |
| To address the challenges of online clothing shopping， such as the lack of real-world try-on
experience and the difficulty in accurately matching clothing with human poses， this paper proposes a
virtual fitting method based on OpenPose key-point detection and convolutional neural networks. This
method takes a single image of a person and the target clothing as input. First， OpenPose is used to
extract key-points and construct a pose heatmap. Second， clothing segmentation and affine/TPS
transformation are deployed to achieve geometric pre-alignment of the clothing region. Finally， the person image， pose representation， and aligned clothing are fused in an encoder-decoder convolutional network，
and end-to-end training is performed using reconstruction loss， perceptual loss， and mask consistency
loss. Experiments are conducted on three public datasets： VITON-HD， DressCode， and a subset of
DeepFashion2. Results show that the proposed method achieves superior or competitive performance on
SSIM and LPIPS metrics across various clothing categories and pose scenarios. Ablation experiments
further validate that the OpenPose pose constraints， clothing geometric alignment， and perceptual and
mask losses are key components. |
| 查看全文 查看/发表评论 附件 |