Chaturvedi, Iti, Pandelea, Vlad, Cambria, Erik, Welsch, Roy, and Datta, Bithin (2024) Barrier Function to Skin Elasticity in Talking Head. Cognitive Computation. (In Press)

Preview

PDF (Publisher Accepted Version) - Published Version Available under License Creative Commons Attribution. Download (905kB) | Preview

Abstract

In this paper we target the problem of generating facial expressions from a piece of audio. This is challenging since both audio and video have inherent characteristics that are distinct from the other. Some words may have identical lip movements and speech impediments may prevent lip-reading in some individuals. Previous approaches to generating such a talking head suffered from stiff expressions. This is because they focused only on lip movements and the facial landmarks did not contain the information flow from the audio. Hence, in this work we employ spatio-temporal independent component analysis to accurately sync the audio with the corresponding face video. Proper word formation also requires control over the face muscles that can be captured using a barrier function. We first validated the approach on diffusion of salt water in coastal areas using a synthetic finite element simulation. Next, we applied it to 3D facial expressions in toddlers for which training data is difficult to capture. Prior knowledge in the form of rules is specified using Fuzzy logic and multi-objective optimization is used to collectively learn a set of rules. We observed significantly higher F-measure on three real world problems.

Item ID:	83472
Item Type:	Article (Research - C1)
ISSN:	1866-9964
Copyright Information:	This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Date Deposited:	27 Aug 2024 00:24
FoR Codes:	46 INFORMATION AND COMPUTING SCIENCES > 4603 Computer vision and multimedia computation > 460306 Image processing @ 50% 46 INFORMATION AND COMPUTING SCIENCES > 4602 Artificial intelligence > 460208 Natural language processing @ 25% 46 INFORMATION AND COMPUTING SCIENCES > 4602 Artificial intelligence > 460299 Artificial intelligence not elsewhere classified @ 25%
SEO Codes:	22 INFORMATION AND COMMUNICATION SERVICES > 2204 Information systems, technologies and services > 220403 Artificial intelligence @ 50% 28 EXPANDING KNOWLEDGE > 2801 Expanding knowledge > 280115 Expanding knowledge in the information and computing sciences @ 20% 22 INFORMATION AND COMMUNICATION SERVICES > 2204 Information systems, technologies and services > 220408 Information systems @ 30%
Downloads:	Total: 11 Last 12 Months: 7
	More Statistics