Statistical modelling of lip movement in the clinical context

An exploration of adolescent facial shape changes with age via multilevel partial least squares regression

BACKGROUND AND OBJECTIVES

Multilevel statistical models represent the existence of hierarchies or clustering within populations of subjects (or shapes in this work). This is a distinct advantage over single-level methods that do not. Multilevel partial-least squares regression (mPLSR) is used here to study facial shape changes with age during adolescence in Welsh and Finnish samples comprising males and females.

METHODS

3D facial images were obtained for Welsh and Finnish male and female subjects at multiple ages from 12 to 17 years old. 1000 3D points were defined regularly for each shape by using "meshmonk" software. A three-level model was used here, including level 1 (sex/ethnicity); level 2, all "subject" variations excluding sex, ethnicity, and age; and level 3, age. The mathematical formalism of mPLSR is given in an Appendix.

RESULTS

Differences in facial shape between the ages of 12 and 17 predicted by mPLSR agree well with previous results of multilevel principal components analysis (mPCA); buccal fat is reduced with increasing age and features such as the nose, brow, and chin become larger and more distinct. Differences due to ethnicity and sex are also observed. Plausible simulated faces are predicted from the model for different ages, sexes and ethnicities. Our models provide good representations of the shape data by consideration of appropriate measures of model fit (RMSE and R2).

CONCLUSIONS

Repeat measures in our dataset for the same subject at different ages can only be modelled indirectly at the lowest level of the model at discrete ages via mPCA. By contrast, mPLSR models age explicitly as a continuous covariate, which is a strong advantage of mPLSR over mPCA. These investigations demonstrate that multivariate multilevel methods such as mPLSR can be used to describe such age-related changes for dense 3D point data. mPLSR might be of much use in future for the prediction of facial shapes for missing persons at specific ages or for simulating shapes for syndromes that affect facial shape in new subject populations.

The effect of resting morphological lip shape during lip movement: A three-dimensional motion analysis study

Purpose

The aim of this study was to determine the influence of different morphological lip shape during lip movement.

Method

A sample of 80 individuals with three-dimensional facial images at rest and during speech were recorded. Subjects were asked to pronounce four bilabial words in a relaxed manner and scanned using the 3dMDFace™ Dynamic System at 48 frames per second. Six lip landmarks were identified at rest and the landmark displacement vectors for the frame of maximal lip movement for all six visemes were recorded. Principal component analysis was applied to isolate relationship between lip traits and their registered coordinates. Eight specific resting morphological lip traits were identified for each individual. The principal component (PC) scores for each viseme were labelled by lip morphological trait and were graphically visualized as ellipses to discriminate any differences in lip movement.

Results

The first five PCs accounted for up to 95% of the total variance in lip shape during movement, with PC1 accounting for at least 38%. There was no clear discrimination between PC1, PC2 and PC3 for any of the resting morphological lip traits.

Conclusion

Lip shapes during movement are more uniform between individuals and resting morphological lip shape does not influence movement of the lips.

Multilevel principal components analysis of three-dimensional facial growth in adolescents

BACKGROUND AND OBJECTIVES

The study of age-related facial shape changes across different populations and sexes requires new multivariate tools to disentangle different sources of variations present in 3D facial images. Here we wish to use a multivariate technique called multilevel principal components analysis (mPCA) to study three-dimensional facial growth in adolescents.

METHODS

These facial shapes were captured for Welsh and Finnish subjects (both male and female) at multiple ages from 12 to 17 years old (i.e., repeated-measures data). 1000 "dense" 3D points were defined regularly for each shape by using a deformable template via "meshmonk" software. A three-level model was used here, namely: level 1 (sex/ethnicity); level 2, all "subject" variations excluding sex, ethnicity, and age; and level 3, age. The technicalities underpinning the mPCA method are presented in Appendices.

RESULTS

Eigenvalues via mPCA predicted that: level 1 (ethnicity/sex) contained 7.9% of variation; level 2 contained 71.5%; and level 3 (age) contained 20.6%. The results for the eigenvalues via mPCA followed a similar pattern to those results of single-level PCA. Results for modes of variation made sense, where effects due to ethnicity, sex, and age were reflected in modes at appropriate levels of the model. Standardised scores at level 1 via mPCA showed much stronger differentiation between sex and ethnicity groups than results of single-level PCA. Results for standardised scores from both single-level PCA and mPCA at level 3 indicated that females had different average "trajectories" with respect to these scores than males, which suggests that facial shape matures in different ways for males and females. No strong evidence of differences in growth patterns between Finnish and Welsh subjects was observed.

CONCLUSIONS

mPCA results agree with existing research relating to the general process of facial changes in adolescents with respect to age quoted in the literature. They support previous evidence that suggests that males demonstrate larger changes and for a longer period of time compared to females, especially in the lower third of the face. These calculations are therefore an excellent initial test that multivariate multilevel methods such as mPCA can be used to describe such age-related changes for "dense" 3D point data.

What's in a Smile? Initial Analyses of Dynamic Changes in Facial Shape and Appearance

Single-level principal component analysis (PCA) and multi-level PCA (mPCA) methods are applied here to a set of (2D frontal) facial images from a group of 80 Finnish subjects (34 male; 46 female) with two different facial expressions (smiling and neutral) per subject. Inspection of eigenvalues gives insight into the importance of different factors affecting shapes, including: biological sex, facial expression (neutral versus smiling), and all other variations. Biological sex and facial expression are shown to be reflected in those components at appropriate levels of the mPCA model. Dynamic 3D shape data for all phases of a smile made up a second dataset sampled from 60 adult British subjects (31 male; 29 female). Modes of variation reflected the act of smiling at the correct level of the mPCA model. Seven phases of the dynamic smiles are identified: rest pre-smile, onset 1 (acceleration), onset 2 (deceleration), apex, offset 1 (acceleration), offset 2 (deceleration), and rest post-smile. A clear cycle is observed in standardized scores at an appropriate level for mPCA and in single-level PCA. mPCA can be used to study static shapes and images, as well as dynamic changes in shape. It gave us much insight into the question “what’s in a smile?”.

Sexual dimorphism in human facial expressions by 3D surface processing

Human face is a dynamic system where facial expressions can rapidly modify geometry of facial features. Facial expressions are believed to be universal across world populations, but only a few studies have explored whether grimacing is sexually dimorphic and if so to what extent. The present paper explores inter- and intra-individual variation of human facial expressions with respect to individual's sex based on a set of neutral and expression-varying 3D facial scans. The study sample composed of 20 individuals (10 males and 10 females) for whom 120 scans featuring grimaces associated with disgust, surprise, "u" sound, smile and wide smile were collected by an optical scanner Vectra XT. In order to quantify the dissimilarity among 3D images, surface comparison approach based on aligned 3D meshes and closest point-to-point distances was carried out in Fidentis Analyst application. The study revealed that sexual dimorphism was indeed one of the factors which determined the extent and characteristics of facial deformations recorded for the studied expressions. In order to produce a grimace, males showed a tendency towards extending their facial movements while females were generally more restrained. Furthermore, the facial movements linked to the wide smile and "u" sound were revealed as the most extensive relative to the other expressions, while the smile and surprise were shown indistinguishable from the neutral face.

Speech production, intelligibility and oromotor function in seven individuals with Möbius sequence

PURPOSE

Möbius sequence is a rare disease characterized by congenital facial and abducent nerve palsy. Other cranial nerves may be affected. Cleft palate, intellectual disability and neuropsychiatric disorders are associated with the diagnosis. The aim was to explore speech production, intelligibility and oromotor function in a group of individuals with Möbius sequence.

METHOD

Three children (5-11 years) and four adults (26-54 years) were recruited to the study via the Swedish Möbius syndrome association. In addition to cranial nerve dysfunction, two had a hearing impairment, one Asperger syndrome and one a cleft palate. Perceptual assessments included an evaluation of intelligibility in single words and spontaneous speech, the percentage of phonemes correct (PPC) and screening of orofacial functions (NOT-S). Objective measurements were used for the evaluation of nasality, lip force and tongue force.

RESULT

Three individuals had severely impaired intelligibility, two slightly impaired and two had fully intelligible speech. The PPC varied between 59.3-100%. Five individuals had bilateral facial palsy, two unilateral facial palsy and six tongue impairment. One had a slightly increased nasalance score. Compensatory strategies were being effectively used.

CONCLUSION

This case series contributes more in-depth knowledge of speech production, intelligibility and oromotor function in this rare condition.

The clinical application of three-dimensional motion capture (4D): a novel approach to quantify the dynamics of facial animations

The aim of this pilot study was to evaluate the feasibility of measuring the change in magnitude, speed, and motion similarity of facial animations in head and neck oncology patients, before and after lip split mandibulotomy. Seven subjects (four males, three females) aged 42-80 years were recruited. The subjects were asked to perform four facial animations (maximal smile, lip purse, cheek puff, and grimace) from rest to maximal position. The animations were captured using a Di4D motion capture system, which recorded 60 frames/s. Nine facial soft tissue landmarks were manually digitized on the first frame of the three-dimensional image of each animation by the same operator and were tracked automatically for the sequential frames. The intra-operator digitization error was within 0.4mm. Lip purse and maximal smile animations showed the least amount of change in magnitude (0.2mm) following surgery; speed difference was least for smile animation (-0.1mm/s). Motion similarity was found to be highest for lip purse animation (0.78). This pilot study confirmed that surgery did influence the dynamics of facial animations, and the Di4D capture system can be regarded as a feasible objective tool for assessing the impact of surgical interventions on facial soft tissue movements.

A geometric morphometric approach to the analysis of lip shape during speech: development of a clinical outcome measure

Objective assessments of lip movement can be beneficial in many disciplines including visual speech recognition, for surgical outcome assessment in patients with cleft lip and for the rehabilitation of patients with facial nerve impairments. The aim of this study was to develop an outcome measure for lip shape during speech using statistical shape analysis techniques. Lip movements during speech were captured from a sample of adult subjects considered as average using a three-dimensional motion capture system. Geometric Morphometrics was employed to extract three-dimensional coordinate data for lip shape during four spoken words decomposed into seven visemes (which included the resting lip shape). Canonical variate analysis was carried out in an attempt to statistically discriminate the seven visemes. The results showed that the second canonical variate discriminated the resting lip shape from articulation of the utterances and accounted for 17.2% of the total variance of the model. The first canonical variate was significant in discriminating between the utterances and accounted for 72.8% of the total variance of the model. The outcome measure was created using the 95% confidence intervals of the canonical variate scores for each subject plotted as ellipses for each viseme. The method and outcome model is proposed as reference to compare lip movement during speech in similar population groups.

Determining normal and abnormal lip shapes at border positions for use as a longitudinal surgical outcome measure

Objective measures of facial movement are important for interventions where surgical repositioning of facial structures can influence soft tissue mobility and include the management of patients with cleft lip, facial nerve palsy and orthognathic surgery. As such, the aim of this study is to present a method for determining the outcome of surgical procedures on lip shape during speech. A control group (CG) of 115 average subjects and 30 patients with a Class 3 malocclusion requiring bimaxillary surgery performed four reproducible verbal utterances during image capture using a non-invasive, three-dimensional (3D) motion scanner (3dMDFace™ Dynamic System). Landmark coordinates around the lips of the 3D facial shells were extracted and subjected to discriminant analysis and principal component analysis to statistically differentiate lip shapes between the CG and the patient group (PG) pre- and post-surgery. Pre-surgically, the PG showed statistically significant differences in lip shape during speech in the lateral and vertical dimensions, preferring a wider, shorter lip shape when compared with the CG for all the utterances. The shape differences normalised towards the CG post-surgery. The method presented utilises pre-existing statistical shape analyses and can be reproduced in the clinical setting to provide a diagnostic and functional outcome tool. In this example, correction of the Class 3 skeletal disproportions appeared to normalise lip shape during speech.