Facial Appearance Estimation for Orthognathic Surgeries using Diffusion Model

Introduction: Craniomaxillofacial deformities necessitate orthognathic surgery, which involves repositioning the bone structure to its normal position [1, 2]. Computer-aided surgical planning (CASS) [3, 4] has been introduced to assist clinicians, among which soft-tissue driven surgical planning [5] has been proposed. This approach predicts the patient's optimal face first and then suggests a corresponding bone plan, eliminating the need for an iterative process and aligning better with real clinical settings.
The initial step involves predicting the optimal face. Existing studies on predicting the optimal face often rely on postoperative results [6], which are not always accurate due to residual deformities, and they use normal templates [7] to replace the prediction area, leading to several limitations. Furthermore, most face simulation studies [8, 9] predict displacement vectors rather than the face itself, which degrades the realism of the results. The limited dataset further exacerbates this issue.
To address these limitations, this study adopts the following approaches. Firstly, Denoising Diffusion Probabilistic Model (DDPM) [10] is utilized to predict faces, aiming to produce more realistic results. Additionally, considering that the upper face remains unchanged after surgery, coarse face specific to each patient was constructed using the upper face. Coarse face is employed as conditional input to the DDPM to support the patient-specific information. Lastly, a public dataset of 3D facial scans was used to train the DDPM, ensuring more stable training and better representation of the actual facial data distribution.

Materials and

Methods: Dense landmarks were extracted from 3D facial scans using pretrained model (MediaPipe) [11]. Given that the upper face remains unchanged after surgery, coarse landmarks for each patient were derived by aligning the normal template’s upper landmarks with the patient’s upper landmarks (Fig.1). These coarse landmarks provide a structural approximation of the patient’s postoperative appearance.
The Dense Soft-tissue Landmark Refinement (DSLR) network converts coarse landmarks into refined landmarks (Fig.2). The coarse landmark network extracts feature from the coarse landmarks, which are then passed to the denoising network. The denoising network iteratively refines the noisy point cloud to generate the final facial landmarks.
Since the DSLR network removes noise rather than directly predicting landmarks, the upper landmarks might not remain fixed. To address this, RePaint [12] was employed during inference (Fig.3). This method fixes the conditional input during the reverse process, injecting appropriate noise at each step to extract the upper landmarks from the coarse landmarks, while the DSLR network predicts the lower landmarks.
Experiments were conducted on 60 orthognathic surgery patients and 48 normal subjects. Datasets were randomly split into a 4:1 ratio for the train/test sets. To see the effect of the number of datasets, we also used public 3D facial scans datasets “FACESCAPE” [13] for training. For evaluation, TPS interpolation was applied between preoperative and predicted dense landmarks to obtain the displacement vector, which was then applied to the preoperative facial surface to predict the postoperative facial surface. This predicted surface was compared with the actual postoperative facial surface.

Results, Conclusions, and Discussions:

Results:
Table.1 summarizes the results according to the different training dataset composition. The baseline model was trained using an internal dataset (N=108). Public dataset (FACESCAPE) comprised a total of 847 subjects. For the "Age-controlled public dataset", a subset of FACESCAPE was selected to match the age distribution of the internal dataset, resulting in 490 subjects. When utilizing the public dataset, there was an improvement in predictions for the lip area compared to the baseline. However, the performance in other regions was comparable or inferior. In contrast, employing the age-controlled public dataset resulted in a significant performance enhancement across all regions. Fig.4 illustrates the color-coded surface deviation errors for two randomly selected patients.

Discussions:
In this study, we aimed to estimate postoperative facial appearance using diffusion model. Evaluation results indicated that the error in each region was within 4mm under our optimal settings. When the internal dataset was supplemented with the public dataset, the performance was similar or tended to decline. This decline can be attributed to the diverse demographic information (e.g., gender, race, age) present in the public dataset, which may counteract the characteristics of the internal dataset, thereby reducing generalizability.
However, when only the age-controlled group was used, a significant performance improvement was observed despite this group comprising only half of the total public dataset. This can be understood as the impact of age on facial structural changes. These findings justify the need to incorporate demographic information when predicting facial outcomes. Future research should focus on utilizing such information to enhance the reliability of patient-specific optimal face predictions.
The limited performance observed may be due to the discrepancy between the data distribution of the dataset assumed to be normal and the actual postoperative patient distribution. Therefore, further research is required to explore surgically achievable facial outcomes.

Conclusion:
This study demonstrates the effectiveness of using diffusion model to estimate postoperative facial appearance, achieving errors within 4mm. Incorporating age-controlled demographic data significantly enhances prediction accuracy, highlighting the importance of tailored demographic information in surgical outcome predictions. Future research should continue to refine these models to improve reliability and generalizability.

Acknowledgements (Optional):