Cell-by-Cell Alignment of Repeated Specular Microscopy Images from the Same Eye

Cell-by-Cell Alignment of Repeated Specular Microscopy Images from the Same Eye Cell-by-CellAlignmentofRepeatedSpecularMicroscopy ImagesfromtheSameEye DanielBo?hringer*,StefanLang,ThomasReinhard UniversityEyeHospital,CorneaReadingCenter,Freiburg,Germany Abstract Purpose:Modernspecularmicroscopes(SM)robustlydepictthesamecentralareaofthecornealendotheliumatdifferent time points through a built-in fixation light. However, repeated image acquisitions slightly shift and rotate because of minute changes in head position in the chin and forehead rest. This prevents the manual retrieval of individual corneal endothelial cells (CECs) in repeated measurements because SM images usually lack obvious landmarks. We devised and validatedanimageregistrationalgorithmthatalignsSMimagesfromthesameeyetomakecorrespondingCECscoincide. Methods:Weretrospectivelyselected27imagepairsforthepresenceofsignificantimageoverlap.Eachimagepairhad beenrecordedonthesamedayandofthesameeye.Weappliedourregistrationmethodineachimagepair.Twoobservers independently validated, by means of alternation flicker, that the image pairs had been correctly aligned. We also repeatedlyappliedourregistrationmethodonunrelatedimagepairsbyrandomlydrawingimagesandmakingcertainthat theimagesdidnotoriginatefromthesameeye.Thiswasdonetoassessthespecifityofourmethod. Results:Allautomatedregistrationsofthesame-dayandsame-eyeimagepairswereaccurate.However,onesingleimage incorrectly failed to trigger the non-match diagnosis twice in 81 registration attempts between unrelated images. As it turnedout,thisparticularimagedepictedonly73CECs.TheaveragenumberofCECswas253(range73–393). Conclusion:Repeatednon-contactSMimagescanbeautomatedlyalignedsothatthecorrespondingCECscoincide.Any successfulalignmentcanbeconsideredasproofoftheretrievalofidenticalCECsassoonasatleast100CECcentroidshave beenidentified.Webelieveourmethodisthefirsttorobustlyconfirmendothelialstabilityinindividualeyes. Citation:Bo?hringerD,LangS,ReinhardT(2013)Cell-by-CellAlignmentofRepeatedSpecularMicroscopyImagesfromtheSameEye.PLoSONE8(3):e59261. doi:10.1371/journal.pone.0059261 Editor:JonathanAColes,GlasgowUniversity,UnitedKingdom ReceivedNovember8,2012;AcceptedFebruary12,2013;PublishedMarch14,2013 Copyright: ß 2013 Bo?hringer et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricteduse,distribution,andreproductioninanymedium,providedtheoriginalauthorandsourcearecredited. Funding:Theauthorshavenosupportorfundingtoreport. CompetingInterests:Theauthorshavedeclaredthatnocompetinginterestsexist. *E-mail:daniel.boehringer@uniklinik-freiburg.de coincideautomatically.Thisisofcourseonlypossiblewhenboth Introduction imagesoverlaptosomedegree.Fortunately,thisimageoverlapis The corneal endothelial cells (CECs) tightly regulate the regulated with a fixation light in all modern SMs. We herein hydration of the corneal stroma. Significant cell loss can result describeourimageregistrationalgorithmandtheassessmentofits in bullous keratopathy, a painful state of corneal edema which sensitivityandspecifityinasmalltrial. usually requires transplantation to restore vision since the CECs donotregeneratesufficiently[1].CECpreservationisthereforea Results keysafetyparameterinmanyclinicaltrialsinvolvingtheanterior Our automated method consistently identified all overlapping segmentoftheeye[2]. Traditionally,endothelialstabilityisassessedbymeansofCEC regionsinthetrialimagepairs.Thiscorrespondstoasensitivityof 100%. Two observers independently confirmed the alignments density [2]. However, it is not currently possible to confirm endothelialstabilityinindividualeyesbeforeandafterexposureto withthehelpofalternationflicker.Here,theperceptionofcellular movements during flicker would readily reveal any erroneous a potentially detrimental trial intervention. This is because CEC densityestimationsarepronetosamplingerrors[3].Weproposed alignment.However,allCECsintheoverlappingareasremained to eliminate sampling errors by comparing only identical CECs in place during flicker as judged unanimously by both investiga- beforeandaftertreatment[4].Thiswouldrequirethealignment tors.Figure1depictsthemethodinaparadigmaticimagepairto demonstrate that proper image alignment is actually based on of identical CECs which is currently unfeasible because SM images have always shifted and rotated slightly due to variable pointsetregistrationoftheCECcentroids. We also made 81 registration attempts between randomly- headpositionsinthechinandforeheadrest[4].Furthermore,the assignedimagepairsoriginatingnotfromthesameeye.Here,our manualretrievalofidenticalCECsisvirtuallyimpossiblebecause the SM images usually lack obvoius landmarks. We devised andmethodincorrectlyfailedtoreportthenon -matchonlytwice.Asit turned out, the same single image matched incorrectly with two validated an automated image registration algorithm that aligns SM images from the same eye in order to make identical CECsdifferent images originating from another eye. Interestingly, this PLOSONE | www.plosone.org 1 March2013 | Volume 8 | Issue 3 | e59261 AlignmentofSame-EyeSpecularMicroscopyImages Figure1.Paradimaticregistrationresult.A:Centroidextractionfromthescannedvideoprints.B:Centroidpointsetsfromimage1(green)and image 2 (red) after proper alignment. C: Stripe-wise image comparison of image 1 (green stripes) and image 2 (red stripes) after corresponding alignmentofimage2.Notethatthecellbordersarecompletelycontinuousbetweenthestripecrossings. doi:10.1371/journal.pone.0059261.g001 erroneously-matchingimagedepictedonly73CECs.Thiswasthe 380000 CECs a thousand times. We reliably (95% probability) imagewiththelowestnumberofCECsinourtrial.Thenumber detected a CEC damage percentage as low as 1%. A second of CEC centroids averaged 253 (range 73 to 393). The low sample from a different location would lower the detection number of extracted CEC centroids in this particular image wasthreshold to only 0.5% damage. Most importantly, however, we theresultofinhomogeneousilluminationandnotduetoverylow canconfidentlyclaimCECstabiliyinasinglepatientforthefirst CEC density. Furthermore, that image did not display higher time. Cell density comparisons, by contrast, work only in cohorts degrees of pleomorphism and polymegathism. Closer inspection revealedthatouralgorithmhadextensivelyscaledandskewedthe because they are based on statistical distributions. Statistical source image with 73 CECs against substantially larger target comparisonsofCECdensitiescanonlyproveCEClossasamatter imagesuntilacompletecoincidencewasachievedwithasubsetof ofprinciple.IfthereisnostatisticallysignificantCECloss,wehave thedestinationCECs.Thiscannothappenwhensourceandtarget toassessthestatisticalpowerwhendiscussingthechancesofCEC imagesareroughlythesamesize,asisusuallythecase.However, stability.Tosafelyruleout(95%probability)aCEClossof1%on both erroneous matches would have been readily spotted upon thebasisofanon-significantt-test,atotalofatleast5000patients manual review because of grossly unrealistic transformation wouldhavetobeanalyzed(ascalculatedwiththefunctionunder parameters. the assumption of a standard deviation of 200 cells per square millimeter). This is of course not feasible when the corneal endothelium is only a safety parameter. For this reason, small Discussion degreesofCECdamagebasedonCECdensitycomparisonsare We herein demonstrate for the first time the feasibility of currently undetectable in many clinical trials. However, an aligning two non-contact SM images in order to coincide the intraoculardevicethatwouldinduce1%CEClossoveramonth corresponding CECs. We designed our method to assess maywellinducebullouskeratopathywithin2years,dependingon endothelialstabilityinclinicaltrials.Inthiscontext,weproposed theinitialCECdensity.Thishasactuallyhappenedafteranterior comparing the baseline SM image to the image taken after chamber lens implantations [6]. One option to slightly alleviate exposuretothepotentiallydamagingtrialinterventioncell-by-cell thisdilemmawouldbetotakeadditionalSMimagesatdifferent [4].IftheCECsinbothSMimagesareinfactidentical,wecan locations (e.g. superior limbal vs central). However, such data obviously rule out any CEC damage in that area because CECs wouldrequiresophisticatedlinearregressionmodelsbecausethese ultimately shiftor enlarge withinhours in theproximity ofCEC additional observations are not fully independent additional damage[5].Weperformedastatisticalsimulationexperimentto samples. assess the powerto detect any randomly-distributed CEC lossin Inthepresentstudy,weanalyzedonlysame-daymeasurements, thewholecorneabasedonasampleofthe300CECswetypically whichiswhywecannotaddresshowthismethodperformsincase getwithourmethodintheprospectivesetting.Theentirecorneal ofactualCECloss.However,ourmethodisrobustagainstsmaller endothelium probably consists of approximately 380000 CECs. degrees of CEC loss, as evident from figure 1B. Here, several Thisnumbercomesfrom(arguablysimplistic)geometricassump- centroids are not perfectly superimposed [e.g. at coordinates tionsofaperfectlysphericalcornea,awhitetowhitedistanceof11 410,140, we see two green dots around a red one (the result of millimeters and uniform CEC density of 2000 cells per square imperfectcentroidextractionratherthanrealCECloss,asevident millimeter.Inoursimulation,werandomlydrewsamplesfromthe fromfigure1C)].Actually,ourmethodhasindeedbeenappliedto PLOSONE | www.plosone.org 2 March2013 | Volume 8 | Issue 3 | e59261 AlignmentofSame-EyeSpecularMicroscopyImages Figure2.Flowchartofouralgorithm.Weiterativelyalignsourceanddestinationcentroidsascloselyaspossible.Westartbyperformingonly translations.We thenpairsourceanddestinationcentroidsaccording to thenearestneighborprinciple. Thesepairs are multivariately correlated throughalinearregressionmodel.Weaffinelytransformthedestinationimagepointsetwiththehelpofparametersfromthisregressionmodel. Thesestepsareiterativelyrepeatedinanestedfashion. doi:10.1371/journal.pone.0059261.g002 PLOSONE | www.plosone.org 3 March2013 | Volume 8 | Issue 3 | e59261 AlignmentofSame-EyeSpecularMicroscopyImages Figure3.Thetwochainedstepsofourpointsetregistrationmethod.A:SuperimpositionoftheCECcentroidsasextractedfromthesource image(o-shaped)andthedestinationimage(x-shaped).B:SuperpositionofsourceanddestinationCECcentroidsaftertranslationofthedestination imageformaximumcorrespondence.C:SuperimpositionofsourceanddestinationCECcentroidsafteradditionalscalingandrotation. doi:10.1371/journal.pone.0059261.g003 post-keratoplasty eyes, e.g. limbo-keratoplasty [4] and Descemet enhancedthroughrepeatedimageacquisitionsandaveragingafter MembraneEndothelialKeratoplasty(unpublishedobservation).In proper alignment. This could even be done transparently within both instances, the corresponding CECs were retrieved after 11 theSMmicroscope.Surgicaleyetrackingbasedontheendothelial monthsandonemonth,respectively.Ifourmethodfailstomatch mosaic would be independent of the iris and limbus: both twoSMimagesdespitecarefulattemptstofixatethefixationlight structures are potentially altered during surgery, and are thus a andsufficientimagequality,wecanassumethoroughremodelling poor source of landmarks for general-purpose surgical eye ofthemosaicaftersubstantialCECloss.However,thiswouldbe tracking. substantiatedbyaconsiderabledropinCECdensity. The manual comparison of two SM images is a strenous and MaterialsandMethods error-proneprocedurebecauseSMimagestypicallylackobvious Imageregistrationalgorithm landmarks.Wehaveimplementedanalgorithmthatmakesuseof As the endothelial mosaic can be reconstructed from CEC the inconspicuous variations in size of the CECs to solve this centroids [7], they provide perfect landmarks for SM image problem.Ifcellsizescouldbemeasuredperfectly,thenapatchof registration.Furthermore,centroidscanbeautomaticallyextract- only15differently-sizedCECswouldoccuronlyonceininstances ed from SMimages with minimal error[8]. Whenthecentroids forcombinatoryreasons.Interestingly,even73CECsturnedout are aligned to match eatch other, the underlying images can be insufficienttoachieveperfectspecificityinourmethod.Thiswas, aligned with exactly the same transformation. The result of this however,theresultof‘‘overfitting’’thatcanonlyhappenwhena projectionisaproperly-alignedimagepairformanualcell-by-cell small image is matched against a larger one. This error is easily comparisons,whichcanbedoneviaalternationflicker(seebelow). spotted upon manual review on the basis of a grossly irrealistic Our point-set registration algorithm is outlined in Fig. 2. We projection.Accordingtooursmallvalidationstudy,anysuccessful repeatedly transform the source-image centroids until the maxi- alignmentcanbeconsideredasproofoftheretrievalofidentical mum number of centroids overlaps with the destination-image CECs,giventhataminimumofroughly100CECcentroidshad centroids.Thishappensiterativelyintwonestedloops.Briefly,we been extracted and the image projection withstands manual align source and destination centroids as closely as possible by review. translation only (Fig. 3b). Thereafter, we pair corresponding Insummary,wedevelopedacomputerprogramthatoperates (nearest neighbor) centroids of the source and destination ontwograyscaleSMimageswithoutmanualinput.Oursoftware pointsets, respectively. These paired CECs’ coordinates are either emits the stacked and aligned images, or reports a non- eventually multivariately correlated through a linear regression match diagnosis. While we consider our software quite mature, model. We affinely transform the destination centroid pointset there is still much room for improvement from the image- based on the parameters from this model (Fig. 3c). These acquisitionperspective.Ifwecouldmovethefixationlightinsteps procedures are iteratively repeated. In summary, our approach slightlysmallerthantheimageframewidth,wecould‘‘stich’’the is somehow related to the random sample consensus (RANSAC) imagestogetherandthusdepictawidefieldofthecentralcorneal principle [9]. If no convergence is reached after all scheduled endothelium. This would allow for higher degrees of certainty iterations, the algorithm reports a no-match diagnosis. Our when diagnosing endothelial stability in future clinical trials. procedure is implemented in the R programming language. On Further applications of our algorithm could be image quality enhancement and surgical eye tracking. Image quality could be PLOSONE | www.plosone.org 4 March2013 | Volume 8 | Issue 3 | e59261 AlignmentofSame-EyeSpecularMicroscopyImages modernhardware,asinglematchingruntakesapproximately60 by means of alternation flicker. The images were presented on seconds. modern27’’LCDdisplays.Theimagesspannedadiagonalsizeof approximately 20 cm and were viewed at adistance of40 to 60 centimeters. Alternation flicker is a proven and tested means of Validationstudy We retrospectively selected 27 image pairs for the presence ofdetecting glaucomatous changes in the optic disc [11] or brain significantimageoverlap.Theseimageshadbeenrecordedonthe tumor progression [12]. We also repeatedly applied ourregistra- same day. We obtained written informed consent from the tion method on unrelated image pairs. Here, we randomlydrew participants.TheethicscommitteeoftheAlbert-Ludwigs-Univer- imagesandmadesurethattheimagesdidnotoriginatefromthe sity ofFreiburg expressed afavorable opinion andapproved our sameeye.Thiswasdonetoassessthespecifityofourmethod. study.Theunderlyingdatasethasbeendescribedelsewhere[10]: briefly, we acquired multiple SM images in corneal outpatients Acknowledgments and healthy volunteers with two different Topcon SP-3000P WethankGu?ntherSchlunck(UniversityEyeHospitalFreiburg,Germany) microscopes.Therepeatedrecordingshadbeentakenwithina10- and Stefan Bo?hringer (Dept of Medical Statistics and Bioinformatics, minute timeframe. All images were printed with video printers. Leiden University Medical Centre) for review of the manuscript and for Using custom image analysis software, we digitized these prints their helpful comments. We also thank Carole Cu?rten (University Eye and extracted the CEC centroids. The average number of CEC HospitalFreiburg,Germany)forproofreadingthemanuscript. centroids was 253 (range 73–393). The differences in CEC numbersontheimageresultedfromdifferencesincelldensitybut AuthorContributions also on the extent of inhomogeneous illumination or insufficient Performed the experiments: DB SL. Analyzed the data: DB SL TR. imagequality. Contributedreagents/materials/analysis tools:DB.Wrotethe paper:DB We applied our registration method to each image pair, each SLTR. comprising two images from the same eye and same day. Two observers independently validated these automated registrations References 1. JoyceNC(2012)Proliferativecapacityofcornealendothelialcells.ExpEyeRes 7. BuchtC,SoderbergP,MannebergG(2011)Simulationofspecularmicroscopy 95:16–23. imagesofcornealendothelium,atoolforcontrolofmeasurementerrors.Acta 2. Ophthalmol89:e242–50. McCarey BE, Edelhauser HF, Lynn MJ (2008) Review of corneal endothelial Vincent L (1993) Morphological grayscale reconstruction in image analysis: specular microscopy for fda clinical trials of refractive procedures, surgical 8. applicationsandefficientalgorithms.IEEETransImageProcess2:176–201. devices,andnewintraoculardrugsandsolutions.Cornea27:1–16. 9. Fischler MA, Bolles RC (1981) Random sample consensus: A paradigm for 3. Abib FC, Holzchuh R, Schaefer A, Schaefer T, Godois R (2012) The model fitting with applications to image analysis and automated cartography. endothelial sample analysis corneal specular microscopy clinical size in CommunicationsoftheACM24:381–395. examinations.Cornea31:546–550. Bohringer D, Hettich L, Maier PC, Reinhard T (2012) [data quality of 10. 4. Bohringer D, Reinhard T (2012) Morphological comparison of specular unsupervisedendothelialcellcountingvs.readingcentreanalysisinmulticentric microscopy images may be a more robust indicator for endothelial stability clinicaltrials].KlinMonblAugenheilkd229:628–631. thancelldensityestimations.Cornea:inpress. 11. Berger JW, Patel TR, Shin DS, Piltz JR, Stone RA (2000) Computerized 5. Sherrard ES (1976) The corneal endothelium in vivo: its response to mild stereochronoscopy and alternation flicker to detect optic nerve head contour trauma.ExpEyeRes22:347–357. change.Ophthalmology107:1316–1320. 6. Mamalis N (2000) Explantation of intraocular lenses. Curr Opin Ophthalmol 12. Erickson BJ, Wood CP, Kaufmann TJ, Patriarche JW, Mandrekar J (2011) 11:289–295. Optimal presentation modes for detecting brain tumor progression. Am J Neuroradiol32:1652–1657. PLOSONE | www.plosone.org 5 March2013 | Volume 8 | Issue 3 | e59261