Journal of Aesthetic & Reconstructive Surgery

Introduction

The importance of replacing 2D photography with 3D imaging in aesthetic procedures may not be as apparent as it should be. The importance goes beyond a better way to visualize a patient’s appearance.

The limitations and complexities of 2D photography are well known by professional and amateur photographers. Some understanding of this can also be readily comprehended by the lay-person reviewing images from a professional photo shoot. Weddings, and other auspicious occasions often involve the employment of professional photographers who proceed to take multiple shots of persons in identical poses. Yet upon review, each image has identifiable differences, albeit subjective and subtle. No two shots, even taken almost simultaneously, are identical and the factors causing the differences are not always attributed to the subject. Following a family photo-shoot, there will be a selection process to determine the ’best shot’ to display as the framed memory following critical review of similar alternatives of the same subject.

It would appear that even professionals, with sophisticated equipment, including lighting and lens configurations continue to be hampered in the quest to achieve the ultimate authentic image with a single-shot of the camera shutter.

Similarly, in aesthetic procedures, there is a tendency to select the best shots we have of the post procedure image and the worst of the previous appearance as evidence of good work in aesthetic medicine. These selected outcomes are often the most proudly displayed to colleagues and patients.

The literature cites criticisms of the use of before and after two dimensional photographs [1]. Such images in aesthetic medicine are often shown at conferences or published in journal articles and advertising, where there are clear differences in exposure, ambient lighting, clothing and distance from object between the presented images.

This conduct is probably not significantly dissimilar to selecting trial data that proves our hypothesis, and avoid inclusion of data that does not. As scientists we should be using an imaging system that consistently reduces the use of evidence that essentially is a selective and artificial representation of outcomes.

There have been dedicated efforts to standardize background, expression and positioning [2]. Introduction of standardized photography reduces some of these errors. Even in the most ideal circumstances though, a single shot photograph will still only represent the lowest specificity and sensitivity because of the very small amount of data that a single still-shot can capture. Worst case scenarios exist when no standardized photographic principles were followed which result in reproductions that can attract criticism from the profession, the public and litigators.

In any case, it is no surprise that the factors of inadequacies of office-based photography of aesthetic patients are often exponentially higher than for the professional photographer for weddings and other social gatherings. The existence of limitations in office-based photography is especially true because there is a significant time interval between initial assessment and capture of the settled, end result when the patient returns for review. Often there will be multiple variables to account for occurring during the intervening time between initial presentation and intervention such as day-to-day variation of ambient lighting, angle, and depth-of-field, to name a few.

There are a number of advantages of present-day 3D systems, such as used in our research, over conventional 2D photographs.

1. Distance to subject can be within a range anywhere between 40 cm-80 cm (variations between before and after within this range does not alter specificity);

2. Ambient light variations do not impact on image capture;

3. High frequency of frames per second and triangulation density produces exponentially more data than a single open and close camera shutter;

4. Minor movements of the subject are accounted for by artificial intelligence of software;

5. Images can be rotated through the X, Y and Z axes such that the patient may view perspectives not previously appreciated;

6. Images are more ’life-like’ to the patient;

7. High degree of repeatable comparative detail (no comparative selection of the ’best shot’).

Three dimensional imaging is not new in other disciplines of surgery and medicine. Just two established examples of 3D applications are to facilitate the manufacture of orthopaedic prostheses [3] and recording of pre-existing breast morphology prior to mastectomy [4].

In cosmetic medicine, three-dimensional scanning of the face to classify wrinkles was reported by Akazakis [5] as early as 2002 using a fixed tripod 3D device with narrow-slit aperture to record finite areas (6.4 mm × 6.4 mm) in 101 patients to record depth of wrinkles.

Hatzis [6] introduced profilometry to develop 3D analysis of the facial features which requires the application of an impression similar to den-tistry. Hatzis refers to quantitative assessment of wrinkle tissue reservoir volume (WRTV) and wrinkle-to-wrinkle distance. WRTV appears a possible tool to estimate a volume of dermal filler that might be required to replace lost volume.

Rzany et al. [7], Flynn et al. [8], and Carruthers et al. [9] all employed the use of the Canfield Vectra CR10 tripod photographic system to validate facial assessment scales began in 2009. No specification was available for the resolution of the Canfield device used except that it employs stereo photogrammetry which records high resolution in terms of pixels, and hence accurate color reproduction. Stereo photogrammetry is less accurate recording geometry and is most beneficial in patients who have good skin texture with few features such as rhytides.

Non-photographic qualitative classifications of facial features have attempted to demonstrate efficacy in proprietary models submitted for clearance by therapeutic approval agencies, and elsewhere [9-12], however none are in common use in aesthetic practices. Much of the historical works attempting to create a universal language to compare patients’ facial morphology from the point just prior to treatment, and then presumably to evaluate interventions, has been directed towards attempting to produce and validate assessment scales. In addition, many of these historical works precede much of the modern understanding of the importance of volumetric losses and the contemporary advanced techniques to reduce the appearance of aging using dermal fillers and other agents.

The rate of acceptance of 3D technologies in industry continues to grow exponentially and extend into such fields as reverse engineering, 3d printing, including the manufacture of specialized equipment in space that would otherwise be readily unobtainable from earth.

In the aesthetic field, the existing 3D technology manufacturers are reviewed in 2015 by Chang, Small, Choi, Sharp concluding:

“Advancements must be made in the imaging software to be more user-friendly, efficient, and accurate to engage the utility of plastic surgeons.”

The pace of scientific development of practitioners’ expertise, therapeutic goods and devices in cosmetic medicine has not been matched by progress in devising quantitative imaging of the outcomes using these advances. Qualitative assessment is therefore to be regarded as the weak link in the otherwise scientific chain of proof that the technical advances purport to bring patients who undergo cosmetic medicine.

Research Project

The main challenges in the research and development of optimal 3D systems are:

1. Operator-dependent contingencies of image capture devices;

2. Time investment involved in capturing and processing images;

3. Production of repeatable, accurate and reliable results;

4. Cost of equipment.

A successful application to tackle these difficulties, by the Computer Science and Software Engineering faculty of the University of Western Australia, was approved by the Australian Research Council for funding in 2013 to develop a turn-key, user-friendly,high definition 3D system that quantitatively measures subtle changes in morphology and surface skin changes.

Included in the funding application was an initial, small, unpublished, prospective, controlled, double-blind trial of patients attending a dedicated cosmetic medical clinic for volumetric facial rejuvenation to investigate the potential of the project.

This preliminary study aimed to determine if 3D scanning can consistently reproduce a single, reliable comparative mapped image by over-laying of the pre and post procedure appearances. The study was designed only for feasibility of potential development beyond existing 3d systems used in aesthetic procedures.

Hypothesis

The study hypothesis was that where volumetric support had been increased in key support points of the face using dermal fillers, the degree of change at the injection site, could be accurately measured.

Study Methods and Materials

Quantitative assessment was achieved using a mid-price, portable, non-laser guided, three dimensional structured-light, video-scanning device with engineering software to scan patients receiving facial injections, before and after treatment as shown in Figure 1.

Figure 1: 2D face images (First row: Frontal views; Second row: Side views): (a) Pre treatment; (b) Post treatment. Red circles indicate treatment area on 3D face images.

Results

There was consistent correlation of significant perpendicular height changes corresponding with the areas of treatments as recognized and recorded by the equipment used in the study. However, what was also discovered was that effects could also be seen and measured at sites distant from the injection site (Figure 2).

Figure 2: Quantitative analysis of volumetric facial rejuvenation.

It can therefore be concluded that the quantitative analysis of volumetric facial rejuvenation, both at the site of injection and distant from it, is measurable using engineering-quality threedimensional structure light video scanning.

Discussion

The collection, processing and comparative analysis was time consuming using the methods and materials assigned. Controlling the hand-held camera such that the capture of optimal data had been recorded took practice, and proved to require an operator to have reasonable dexterity and concentration. Unlike existing 3D systems used in aesthetic medicine, the camera used was capable of recording several million triangulations per second. At the time of the study, all other systems in clinical use employed a two-camera static device, taking a single stereoscopic photograph. The camera used in the study however was, a video 3D device, collecting multiple frames per second with 3D resolution of 0.1 mm and 3D point accuracy of 0.05 mm.

In general, 3D technology, once the subject’s physical presence has been captured, a process known as ’registration’ is performed by the accompanying soft-ware, which also averages data to fill small scale deficits, or ’holes’. Primarily the resulting single color image is a reflection of surface morphology (also known as ’relief’). The single-color image is known as a mesh file and it is this image that is used in the alignment process of the pre and post morphology. ’Alignment’ is a complex process of over-laying two mesh files to determine differences in perpendicular height by way of a single image that demonstrates changes in the relief seen on the surface of the face. Alignment methods vary some using ’point cloud’ technology, others use ’triangulation’, or polygonal best-fit. In lay terms, all methods are akin to breaking up two completed near-identical jig-saw puzzles and then recognizing what pieces align to which puzzle and to each other to compare the resulting corresponding differences between the two re-completed jig-saws. (Just as some jig-saws have bigger or smaller pieces, low or high number of pieces, there exist considerable variations in the number of points in a point cloud or triangulations depending upon the various software and capture devices).

Single mesh-files, can be further processed by ’texturing’, which is a process that produces a coloured image reflecting conditions seen on the surface of the subject, which in this case relates to skin complexion, skin tone and colour. However, at this point, the algorithms to measure changes of texture are not well understood and are not presently subject to our research.

Progress of the Research Project

The first task for the project was to find computer algorithms that would be better suited to the human face and form. For example, the software used in the study was drawn from an engineering background, namely for production-line quality control comparing products to templates.

While the process of registration using the ’off-the-shelf’ engineering system required a short learning-curve, the average time to process alignment of mesh files was found to be seventeen minutes and involved multiple manual steps. Nevertheless the refined incremental intervals (+/- 0.05 mm) produced a single image that detailed perpendicular height changes. The resulting comparative image takes the form of contoured landscapes delineating areas of where change occurred, up and down, as a result of intervention.

This is illustrated in Figure 2. The green areas represent where no change has occurred. Warmers colours illustrate a rise in perpendicular height, and cooler colours indicate a fall in perpendicular height. In essence, in this example, dermal filler has been injected in mid-lateral face. This is reflected by the increasing warm colours. The impact of this can also be seen at the jawline where perpendicular height has reduced, indicating an overall anterio-superior elevation of the mid and lower face.

The task to shorten and automate the alignment process, while maintaining sensitivity has taken a team of PhD students’ considerable time and has proved to be challenging. Presently, we have developed an automatic alignment algorithm that has reduced alignment time to less than ten (10) seconds, but with significant wider incremental accuracy and refinement of this is the present focus of our research. The completion of this task will satisfy the first aim of the project.

So far, the number of cases we have been able to access, consent has been acquired from patients, has been limited to less than one hundred (100). What is needed is a larger number of cases to apply the developing algorithms. The ideal number of cases exceeds one thousand (1,000). This requires more participating contributors. The authors would be very pleased to accept offers to participate in the project.

Results Analysis

n=72, mean perpendicular height change in the treatment areas=1.020 mm, Standard Deviation=0.566 mm (P=0.05)

Mean number of treatment areas considered n=3.61, participation rate=69.39 or 95.05%, CI=4.98.

Applying the statistical significance of the study, the null hypothesis appears disproved.

Areas in green in the screen (Figure 2) captures represent no change in height between scans before or after treatment, within a tolerance of 0.500mm. Yellow to red zones represent positive perpendicular height changes of greater than 0.500mm above the prior state. Blue colours represent negative perpendicular height changes of less than -0.500 mm compared to the prior state. Results where perpendicular height change is less than -0.500 mm are also significant, meaning that the perpendicular height is showing as being lowered, even though volume has been injected there. Subject-02 failed to have any correlation between treatment and rescanning and is shown as demonstrating a negative result (as significant perpendicular height increase) in the frontalis region where no treatment was pro-vided. Subject-02,who received only 0.6ml of volume replacement distributed over multiple sites, found the scanning process distracting and was unable to re-strain flinching. In subject 05, the patient required additional injections of local anaesthesia at the right infra-orbital nerve site which is illustrated by asymmetrical perpendicular height change. Subject 01 who was treated only with Botulinum Toxin A appeared to have slight height changes in the lower face, probably rep-resenting slight expression changes as identified in Figure 1. Subject-25 shows an unexplained negative result in the left oral commissure region with perpendicular height lowering where no treatment was provided. However the extent of this change is not significant being more than -0.500 mm.

There were 63/72 (87.5%) results that demonstrated significant (more than 0.500mm) positive perpendicular height changes consistent with the treatment to those areas and 8.3% of the areas treated record positive results that were not significant, however also not negative either. Although it was anticipated that Botulinum Toxin-A treatments would be unlikely to show significant positive change due to fast absorption, the methodology was sensitive enough to determine volume changes of 0.025ml-0.1ml as a result of the saline diluents injected. For ex-ample, Subject-25 received 6U to the upper frontalis muscle in multiple injection sites (actual number not recorded). If 6 sites were injected this would represent 1.U (0.025 ml per site, dilution was 100U: 2.5 ml diluent saline). No result was recorded in the frontalis muscle of Subject-25 suggesting that this volume did not register as a significant perpendicular height change, qualifying as a control. The reliability of the control is probably compromised in results where more than 2U (0.05 ml) is used per injection site. If we accept that an injection of 2U of Botulinum Toxin A (0.05 ml) produces a temporary focal volumetric increase and subsequent localized perpendicular height change, and 1U (0.025 ml) does not, we avoid a Type I error. Thus the correlation of recording positive perpendicular height changes in areas where volumetric supplementation has occurred appears to favour the hypothesis that quantitative measurement is able to be sought as evidence of volumetric replacement treatment using the technology in the study. A number of subjects who demonstrate significant perpendicular height differences between the pre and post treatment scans in the neck, where it is shown. No treatments were performed on the neck. The variations are most likely due to differences in postural positioning of the head, forward or backward, yet without appreciable difference in facial expression differences between the SAFV-scanning before and after treatment.

A serendipity discovery of a separate powerful quality control was that subtle changes of expression appear identifiable with the technology used. This potential confounder was identified from color-coded changes of contouring of contiguous areas (Figure 1). Expression differences between the before and after scans of the subject is probably less likely to be noticed in present 2D photographic results. This study does not take into account the inter-variability of the technique or variation of skill set between clinicians. To some degree the intra-variability of the treating practitioner may also act as a confounder. Larger, multicenter trials with pooled data using a standard approach would help identify if this is a significant factor in the results.

Summary

Aesthetic medicine should be real medicine, but presently continues to lack appreciable credibility for a number of reasons, not the least of which is the perception of apparent absence of appetite to produce quantifiable evidence of results of intervention. Empirically, the majority of aesthetic practitioners appear satisfied with subjective evaluation, which in some cases may amount to little more than expressing approval of the King’s new clothes. There is a reliance on patient approval as a measure of a successful outcome which is not helpful in continual professional development of the procedures we perform. As medical practitioners we owe a duty to science, and this includes constantly evaluating performance with the aim to improve efficacy and to properly trial the effects of emerging aesthetic technologies.

In the field, as a component of our research, an unexpected side benefit of using the project’s 3D imaging system is high patient-approval. Relying only on the pre-treatment textured capture, patients reflect a significant appreciation of their preprocedure image. Lengthy explanations to patients regarding the various available treatment options are seldom required. As their 3D image is rotated through all axes, patients are able to view life-like images from angles and perspectives not previously appreciated. Patients also appear to gain more confidence in the follow-up period that changes have occurred. This is especially important to aesthetic patients who have concerns that the effects of intervention should rep-resent subtle, natural features. Prior to one of our author’s universal application of 3D imaging, patients often expressed uncertainty as to what changes had occurred. Taking into consideration the high relative cost of some interventions, it is important to be able to demonstrate and quantify interventional change as a representation of the patient’s value of investment and 3D imaging appears to provide significant reassurance to the patient that money has been well spent.

As a result of these side benefits, revenue per patient from treatment programs has increased significantly, readily returning the investment in the 3D system capital outlay. This has created a win-win situation for practitioners and patients alike. The practitioner no longer requires to adopt the techniques of a persuasive sales person, since appearance is self-evident in the detail of three-dimensions and not a mere reflection in a mirror, or as an unconvincing, low-data and unreliable two-dimensional photograph.

The practitioner can concentrate purely on the acquisition of informed consent and the various components of the duty of care, including the performance of the procedure itself. Finally, universal adoption of any sincere effort to validate our field with science can only provide authentication to aesthetic medicine, providing a significant point of difference from pseudo-scientific service-providers with whom we are often wrongly included.

Acknowledgments

This research is supported by the Australian Research Council (ARC) linkage grant LP130100138.

References

1. 1.Anderson J, Hallam MJ, Nduka C, Osorio D (2015) The challenge of objective scar colour assessment in a clinical setting: using digital photography. Journal Wound Care 24: 379-387.
2. Becker DG, Tardy ME Jr (1998) Standardized photography in facial plastic surgery: pearls and pitfalls. Facial Plastic Surgery 15: 93-99.
3. Hlináková P, Dostálová T, Dank J,Nedoma J, Hlavácek I (2010)Temporo-mandibular joint and its two-dimensional and three-dimensional modeling. Journal Mathematics and Computers in Simulation80: 1256-1268.
4. Yip JM, Mouratova N, Jeffery RM, Veitch DE, Woodman RJ, et al. (2012) Accurate assessment of breast volume: a study comparing the volumetric gold standard (direct water displacement measurement of mastectomy specimen) with a 3d laser scanning technique.Ann PlastSurg 68: 135-141.
5. Akazaki S, Nakagawa H, Kazama H, Osanai O, Kawai M, et al. (2002) Age-related changes in skin wrinkles assessed by a novel three-dimensional morphometric analysis. British Journal of Dermatology 147: 689-695.
6. 6.Hatzis J (2004) The wrinkle and its measurement: A skin surface profilometric method. Micron 35: 201-219.
7. Carruthers J, Flynn TC, Geister TL, Gortelmeyer R, Hardas B, et al. (2012) Validated assessment scales for the mid face. Dermatologic Surgery 38: 320-332.
8. Flynn TC, Carruthers A, Carruthers J, Geister TL, Gortelmeyer R, et al. (2012) Vali-dated assessment scales for the upper face. Dermatologic Surgery 38: 309-319.
9. Carruthers A, Carruthers J (2010) A validated facial grading scale: the future of facial ageing measurement tools? Journal of Cosmetic and Laser Therapy 12: 235-241.
10. Lemperle G, Holmes RE, Lemperle SSM (2001) A classification of facial wrinkles. Plastic and Reconstructive Surgery 108: 1735-1750.
11. Buchner L, Vamvakias G, Rom D (2010) Validation of a photonumeric wrinkle assessment scale for assessing nasolabial fold wrinkles. Plastic and Reconstructive Surgery 126: 596-601.
12. Narins RS, Brandt F, Leyden J, Lorenc ZP, Rubin M, et al. (2003) A randomized, double-blind, multicenter comparison of the efficacy and tolerability of restylane versus zyplast for the correction of nasolabial folds. Dermatologic Surgery 29: 588-595.