首页|An Evaluation Framework for Virtual Articulatory Movements Based on Medical Video

An Evaluation Framework for Virtual Articulatory Movements Based on Medical Video

扫码查看
原文链接
  • NETL
  • NSTL
  • 万方数据
  • 维普
An important aspect of a Speech tutoring aimed talking-head system (STTS) is the accuracy of produced articulatory movements. Little work has been done for the Articulatory movements' accuracy (AMA) evaluation in STTSs. Although subjective evaluation is reliable, it is time consuming and inconvenient. The traditional objective evaluation is comparing the motion of several points on the surface of the synthetic articulator to the Electromagnetic articulography (EMA) data which describes the motion of corresponding points on the articulatory surface of a speaker. The EMA information is too limited to describe the whole shape changing of deformable articulators for a speech process. To solve this problem, we propose a substantially different objective evaluation method based on a separately recorded medical video. The synthetic articulatory shapes in a speech process are compared to the corresponding shapes tracked from the medical video. This method is translation, rotation, and scaling invariant which allows the comparison of the shapes from the synthetic tongue and the medical images. The time difference problem of synthesis results and medical video is solved by introducing Dynamic time warping (DTW) to the proposed method. Experimental results demonstrate that our method has the ability to evaluate the deformation shape accuracy from an entire articulation process. The comparison results suggest that our method is more accurate than the traditional method especially for deformable articulators.

Accuracy evaluationMedical videoArticulatory animationSpeech tutoring aimed talking-head system

LI Rui、YU Jun、LI Xian、FANG Peng、WANG Zengfu

展开 >

Department of Automation, University of Science and Technology of China, Hefei 230027, China

Institute of Intelligent Machines, Chinese Academy of Sciences, Hefei 230031, China

National Natural Science Foundation of ChinaNational Natural Science Foundation of ChinaAnhui Provincial Natural Science FoundationFundamental Research Funds for the Central Universities

U1736123615724501708085QF138WK2350000002

2019

10.1049/cje.2018.09.019

中国电子杂志(英文版)

中国电子杂志(英文版)

CSTPCDCSCDSCIEI
ISSN：1022-4653
年,卷(期)：2019.28(3)
  • 25