下载

1下载券

加入VIP
  • 专属下载特权
  • 现金文档折扣购买
  • VIP免费专区
  • 千万文档免费下载

上传资料

关闭

关闭

关闭

封号提示

内容

首页 视频去抖动资料

视频去抖动资料.pdf

视频去抖动资料

落叶
2012-06-26 0人阅读 举报 0 0 暂无简介

简介:本文档为《视频去抖动资料pdf》,可适用于IT/计算机领域

ToappearintheACMSIGGRAPHconferenceproceedingsContentPreservingWarpsforDVideoStabilizationFengLiuMichaelGleicherUniversityofWisconsinMadisonHailinJinAseemAgarwalaAdobeSystems,IncAbstractWedescribeatechniquethattransformsavideofromahandheldvideocamerasothatitappearsasifitweretakenwithadirectedcameramotionOurmethodadjuststhevideotoappearasifitweretakenfromnearbyviewpoints,allowingDcameramovementstobesimulatedByaimingonlyforperceptualplausibility,ratherthanaccuratereconstruction,weareabletodevelopalgorithmsthatcaneffectivelyrecreatedynamicscenesfromasinglesourcevideoOurtechniquefirstrecoverstheoriginalDcameramotionandasparsesetofD,staticscenepointsusinganofftheshelfstructurefrommotionsystemThen,adesiredcamerapathiscomputedeitherautomatically(eg,byfittingalinearorquadraticpath)orinteractivelyFinally,ourtechniqueperformsaleastsquaresoptimizationthatcomputesaspatiallyvaryingwarpfromeachinputvideoframeintoanoutputframeThewarpiscomputedtobothfollowthesparsedisplacementssuggestedbytherecoveredDstructure,andavoiddeformingthecontentinthevideoframeOurexperimentsonstabilizingchallengingvideosofdynamicscenesdemonstratetheeffectivenessofourtechniqueIntroductionWhiledigitalstillphotographyhasprogressedtothepointwheremostamateurscaneasilycapturehighqualityimages,thequalitygapbetweenprofessionalandamateurlevelvideoremainsremarkablywideOneofthebiggestcomponentsofthisgapiscameramotionMostcameramotionsincasualvideoareshothandheld,yieldingvideosthataredifficulttowatch,evenifvideostabilizationisusedtoremovehighfrequencyjittersIncontrast,someofthemoststrikingcameramovementsinprofessionalproductionsare“trackingshots”Kawin,wherecamerasaremovedalongsmooth,simplepathsProfessionalsachievesuchmotionwithsophisticatedequipment,suchascamerasmountedonrailsorsteadicams,thataretoocumbersomeorexpensiveforamateursInthispaper,wedescribeatechniquethatallowsausertotransformtheirhandheldvideostohavetheappearanceofanidealizedcameramotion,suchasatrackingshot,asapostprocessingstepGivenavideosequencefromasinglevideocamera,ouralgorithmcansimulateanycameramotionthatisreasonablyclosetothecapturedoneWefocusoncreatingcanonicalcameramotions,suchaslinearorparabolicpaths,becausesuchpathshaveastrikingeffectandaredifficulttocreatewithoutextensiveequipmentOurmethodcanalsoperformstabilizationusinglowpassfilteringoftheoriginalcameramotiontogivetheappearanceofasteadicamGivenadesiredoutputcamerapath,ourmethodthenautomaticallywarpstheinputsequencesothatitappearstohavebeencapturedalongthespecifiedpathhttp:wwwcswiscedugraphicsGalleryWarpForDStabilizationWhileexistingvideostabilizationalgorithmsaresuccessfulatremovingsmallcamerajitters,theytypicallycannotproducethemoreaggressivechangesrequiredtosynthesizeidealizedcameramotionsMostexistingmethodsoperatepurelyinDtheyapplyfullframeDwarps(eg,affineorprojective)toeachimagethatbestremovejitterfromthetrajectoryoffeaturesinthevideoTheseDmethodsarefundamentallylimitedintwoways:first,afullframewarpcannotmodeltheparallaxthatisinducedbyatranslationalshiftinviewpointsecond,thereisnoconnectionbetweentheDwarpandaDcameramotion,makingitimpossibletodescribedesiredcamerapathsinDWethereforeconsideraDapproachImagebasedrenderingmethodscanbeusedtoperformvideostabilizationinDbyrenderingwhatacamerawouldhaveseenalongthedesiredcamerapathBuehleretalaHowever,thesetechniquesarecurrentlylimitedtostaticscenes,sincetheyrenderanovelviewpointbycombiningcontentfrommultiplevideoframes,andthereforemultiplemomentsintimeOurworkisthefirsttechniquethatcanperformDvideostabilizationfordynamicscenesInourmethod,dynamiccontentandothertemporalpropertiesofvideoarepreservedbecauseeachoutputframeisrenderedasawarpofasingleinputframeThisconstraintimpliesthatwemustperformaccuratenovelviewinterpolationfromasingleimage,whichisextremelychallengingHoiemetalPerformingthistaskforanonrigiddynamicscenecapturedbyasinglecamerawhilemaintainingtemporalcoherenceisevenharderinfact,tothebestofourknowledgeithasneverbeenattemptedAnaccuratesolutionwouldrequiresolvingseveralchallengingcomputervisionproblems,suchasvideolayerseparationChuangetal,nonrigidDtrackingTorresanietal,andvideoholefillingWexleretalInthispaperweprovideatechniquefornovelviewinterpolationthatavoidsthesechallengingvisionproblemsbyrelaxingtheconstraintofaphysicallycorrectreconstructionForourapplication,aperceptuallyplausibleresultissufficient:wesimplywanttoprovidetheillusionthatthecameramovesalonganewbutnearbypathInpractice,wefindourtechniqueiseffectiveforvideostabilizationeventhoughournovelviewsarenotphysicallyaccurateandwouldnotmatchthegroundtruthOurmethodtakesadvantageofrecentadvancesintwoareasofresearch:shapepreservingimagedeformationIgarashietal,whichdeformsimagesaccordingtouserspecifiedhandleswhileminimizingthedistortionoflocalshapeandcontentawareimageresizingAvidanandShamirWolfetal,whichchangesthesizeofimageswhilepreservingsalientimagecontentBothofthesemethodsminimizeperceivableimagedistortionbyoptimallydistributingthedeformationinducedbyusercontrollededitsacrosstheDdomainWeapplythissameprincipletoimagewarpsforDvideostabilization,thoughinourcaseweoptimallydistributethedistortioninducedbyaDviewpointchangeratherthanusercontrolleddeformationSincethechangeinviewpointrequiredbyvideostabilizationistypicallysmall,wehavefoundthatthisnotphysicallycorrectapproachtonovelviewinterpolationissufficientevenforchallengingvideosofdynamicscenesOurmethodconsistsofthreestagesFirst,itrecoverstheDcameramotionandasparsesetofD,staticscenepointsusinganofftheshelfstructurefrommotion(SFM)systemSecond,theuserinteractivelyspecifiesadesiredcamerapath,orchoosesoneofthreeToappearintheACMSIGGRAPHconferenceproceedingscamerapathoptions:linear,parabolic,orasmoothedversionoftheoriginalouralgorithmthenautomaticallyfitsacamerapathtotheinputFinally,ourtechniqueperformsaleastsquaresoptimizationthatcomputesaspatiallyvaryingwarpfromeachinputvideoframeintoanoutputframeThewarpiscomputedtobothfollowthesparsedisplacementssuggestedbytherecoveredDstructure,andminimizedistortionoflocalshapeandcontentinthevideoframesTheresultisnotaccurate,inthesensethatitwillnotrevealthedisocclusionsornonLambertianeffectsthatanactualviewpointshiftshouldyieldhowever,forthepurposesofvideostabilization,wehavefoundthattheseinaccuraciesaredifficulttonoticeincasualviewingAsweshowinourresults,ourmethodisabletoconvincinglyrenderarangeofoutputcamerapathsthatarereasonablyclosetotheinputpath,evenforhighlydynamicscenesRelatedWorkTwodimensionalvideostabilizationtechniqueshavereachedalevelofmaturitythattheyarecommonlyimplementedinoncamerahardwareandruninrealtimeMorimotoandChellappaThisapproachcanbesufficientiftheuseronlywishestodampundesiredcamerashake,iftheinputcameramotionconsistsmostlyofrotationwithverylittletranslation,orifthesceneisplanarorverydistantHowever,inthecommoncaseofacameramovingthroughathreedimensionalscene,thereistypicallyalargegapbetweenDvideostabilizationandprofessionalqualitycamerapathsTheideaoftransforminghandheldvideostoappearasiftheyweretakenasapropertrackingshotwasfirstrealizedbyGleicherandLiuTheirapproachsegmentsvideosandappliesidealizedcameramovementstoeachHowever,thisapproachisbasedonfullframeDwarping,andthereforesuffers(asallDapproaches)fromtwofundamentallimitations:itcannotreasonaboutthemovementofthephysicalcamerainD,anditislimitedintheamountofviewpointchangeforsceneswithnontrivialdepthcomplexityTheDapproachtovideostabilizationwasfirstdescribedbyBuehleretalaInDvideostabilization,theDcameramotionistrackedusingstructurefrommotionHartleyandZisserman,andadesiredDcamerapathisfittothehandheldinputpathWiththissetup,videostabilizationcanbereducedtotheclassicimagebasedrendering(IBR)problemofnovelviewinterpolation:givenacollectionofinputvideoframes,synthesizetheimageswhichwouldhavebeenseenfromviewpointsalongthedesiredcamerapathThoughthenovelviewpointinterpolationproblemischallengingandillposed,recentsophisticatedtechniqueshavedemonstratedhighqualityvideostabilizationresultsFitzgibbonetalBhatetalHowever,thelimitationtostaticscenesrenderstheseapproachesimpractical,sincemostofusshootvideoofdynamiccontent,eg,peopleImagewarpinganddeformationtechniqueshavealonghistoryGomesetalRecenteffortshavefocusedondeformationcontrolledbyauserwhopullsonvarioushandlesIgarashietalSchaeferetalwhileminimizingdistortionoflocalshape,asmeasuredbythelocaldeviationfromconformalorrigidtransformationsThesemethods,whichbuildonearlierworkinasrigidaspossibleshapeinterpolationAlexaetal,areabletominimizeperceivabledistortionmuchmoreeffectivelythantraditionalspacewarpmethodsBeierandNeelyorstandardscattereddatainterpolationBooksteinOurmethodappliesthisprincipleincomputingspatiallyvaryingwarpsinducedbytherecoveredDscenestructureArelatedimagedeformationproblemistochangethesizeoraspectratioofanimagewithoutdistortingsalientimagestructureSeamCarvingAvidanandShamirexploitedthefactthatlessperceptuallysalientregionsinanimagecanbedeformedmorefreelythansalientregions,andwaslaterextendedtovideoRubinsteinetalHowever,thediscretealgorithmbehindSeamCarvingrequiresremovingonepixelfromeachimageroworcolumn,whichlimitsitsapplicationtogeneralimagewarpingOthershaveexploredmorecontinuousformulationsGaletalWolfetalWangetal,whichdeformaquadmeshplacedontheimageaccordingtothesalience(orusermarkedimportance)foundwithineachquadwetakethisapproachindesigningourdeformationtechniqueAlimitationofourapproachisthatitrequiressuccessfulcomputationofvideostructurefrommotionHowever,thisstephasbecomecommonplaceinthevisualeffectsindustry,andcommercialDcameratrackerslikeBoujouandSyntheyesarewidelyusedWeusethefreeandpubliclyavailableVoodoocameratracker,whichhasbeenusedinanumberofrecentresearchsystemsvandenHengeletalThorma¨hlenandSeidelFinally,thereareanumberoforthogonalissuesinvideostabilizationthatwedonotaddressMatsushitaetal,suchasremovingmotionblur,andfullframevideostabilizationthatavoidsthelossofinformationatthevideoboundariesviaholefilling(wesimplycropouroutput)ThesetechniquescouldbecombinedwithourmethodtoyieldacompletevideostabilizationsolutionTraditionalvideostabilizationWebeginbydescribingthecurrenttechnicalapproachestovideostabilizationinmoredetail,andshowingtheirresultsontheexamplesequenceinVideoFigure(sincemanyoftheissueswediscusscanonlybeunderstoodinananimatedform,wewillrefertoasetofvideofiguresthatareincludedassupplementalmaterialsandontheprojectwebsite)DstabilizationTraditionalDvideostabilizationproceedsinthreestepsFirst,aDmotionmodel,suchasanaffineorprojectivetransformation,isestimatedbetweenconsecutiveframesSecond,theparametersofthismotionmodelarelowpassfilteredacrosstimeThird,fullframewarpscomputedbetweentheoriginalandfilteredmotionmodelsareappliedtoremovehighfrequencycamerashakeVideoFiguresandshowtworesultsofthisapproach,createdusingourimplementationofMatsushitaetal(wedonotperformtheinpaintingordeblurringsteps,andthetwovideoscontaindifferentdegreesofmotionsmoothing)WhileDstabilizationcansignificantlyreducecamerashake,itcannotsimulateanidealizedcamerapathsimilartowhatcanbefoundinprofessionaltrackingshotsSincetheDmethodhasnoknowledgeoftheDtrajectoryoftheinputcamera,itcannotreasoninDaboutwhattheoutputcamerapathshouldbe,andwhatthescenewouldhavelookedlikefromthispathInstead,itmustmakedowithfittingprojectivetransformations(whicharepoorapproximationsformotionthroughaDscene)andlowpassfilteringthemStronglowpassfiltering(VideoFigure)canleadtovisibledistortionsofthevideocontent,whileweakfiltering(VideoFigure)onlydampsshakeneithercansimulatedirectedcameramotionsDstabilizationTheDapproachtovideostabilizationismorepowerful,thoughalsomorecomputationallycomplexHere,theactualDtrajectoryoftheoriginalcameraisfirstestimatedusingstandardstructurefrommotionHartleyandZissermanthisstepalsoresultsinhttp:wwwdcomhttp:ssontechcomhttp:wwwdigilabunihannoverdeToappearintheACMSIGGRAPHconferenceproceedingsFigure:AcropofavideoframecreatedusingnovelviewinterpolationWhilethestaticportionsofthesceneappearnormal,themovingpeoplesufferfromghostingFigure:AcropofavideoframecreatedusinggenericsparsedatainterpolationTheresultdoesnotcontainghosting,butdistortsstructuressuchasthewindowandpolehighlightedwithredarrowsasparseDpointclouddescribingtheDgeometryofthesceneSecond,adesiredcamerapathisfittotheoriginaltrajectory(wedescribeseveralapproachestocomputingsuchapathinSection)Finally,anoutputvideoiscreatedbyrenderingthesceneasitwouldhavebeenseenfromthenew,desiredcameratrajectoryThereareanumberoftechniquesforrenderingnovelviewsofasceneinVideoFigureweshowavideostabilizationresultcreatedusingthewellknownunstructuredlumigraphrenderingalgorithmBuehleretalbTheresultisremarkablystableHowever,likeallnovelviewinterpolationalgorithms,eachoutputframeisrenderedasablendofmultipleinputvideoframesTherefore,dynamicscenecontentsuffersfromghosting(weshowastillframeexampleofthisghostinginFigure)Oneapproachtohandlingdynamicscenecontentwouldbetoidentifythedynamicobjects,mattethemout,usenovelviewinterpolationtosynthesizethebackground,recomposite,andfillanyremainingholesHowever,eachofthesestepsisachallengingproblem,andtheprobabilitythatallwouldcompletesuccessfullyislowTherefore,inthenextsectionweintroducetheconstraintthateachoutputvideoframeberenderedonlyfromthecontentinitscorrespondinginputvideoframeOurapproachOurapproachbeginssimilarlytotheDstabilizationtechniquejustdescribedwerecovertheoriginalDcameramotionandsparseDpointcloudusingstructurefrommotion,andspecifyadesiredoutputcameramotioninD(inthissectionweassumetheoutputpathisgivenourapproachforcomputingoneisdescribedinSection)Then,ratherthansynthesizenovelviewsusingmultipleinputvideoframes,weuseboththesparseDpointcloudandthecontentofthevideoframesasaguideinwarpingeachinputvideoframeintoitscorrespondingoutputvideoframeMorespecifically,wecomputeanoutputvideosequencefromtheinputvideosuchthateachoutputvideoframeItisawarpofitscorrespondinginputframeIˆt(sincewetreateachframeindependently,wewillomitthetsubscriptfromnowon)AsguidancewehaveasparseDpointcloudwhichwecanprojectintoboththeinputandoutputcameras,yieldingtwosetsofcorrespondingDpoints:Pintheoutput,andPˆintheinputEachk’thpairofprojectedpointsyieldsaDdisplacementPk−PˆkthatcanguidethewarpfrominputtooutputTheproblemremainingistocreateadensewarpguidedbythissparsesetofdisplacementsThiswarp,whichcanusethedisplacementsaseithersoftorhardconstraints,shouldmaintaintheillusionofanaturalvideobymaintainingtemporalcoherenceandnotdistortingscenecontentWefirstconsidertwosimplewarpingsolutions,thefirstofwhichisnotsuccessful,andthesecondofwhichismoderatelysuccessfulThefirstsolutionistousegenericsparsedatainterpolationtoyieldadensewarpfromthesparseinputInVideoFigureweshowaresultcomputedbysimplytriangulatingthesparsepointsandusingbarycentriccoordinatestointerpolatethedisplacementsinsidethetrianglesthedisplacementsarethereforetreatedashardconstraintsTheresulthasanumberofproblemsMostsignificantly,importantscenestructuresaredistorted(weshowastillexampleinFigure)Thesedistortionstypicallyoccurnearocclusions,whicharethemostchallengingareasfornovelviewinterpolationAlso,problemsoccurneartheframeboundariesbecauseextrapolationoutsidethehullofthepointsischallenging(forthisexample,wedonotperformextrapolation)Finally,treatingthedisplacementsashardconstraintsleadstotemporalincoherencesincethereconstructedDpointsarenottypicallyvisiblefortheentirevideoPoppingandjitteringoccurwhenthecorrespondingdisplacementsappearanddisappearovertimeInthisexample,weuseaveryshortsegmentofvideoandonlyincludepointsthatlastovertheentiredurationofthevideohowever,theproblemisunavoidableinlongersequencesOurapproachforpreservingtemporalcoherence,whichisonlyapplicableifdisplacementsareusedassoftconstraints,isdescribedinSectionThesecondalternativeistofitafullframewarptothesparsedisplacements,suchasahomography(therebytreatingthedisplacementsasasoftconstraint)WeshowasomewhatsuccessfulresultofthistechniqueinVideoFigureThismethodcanachievegoodresultsifthedepthvariationinthesceneisnotlarge,orifthedesiredcamerapathisveryclosetotheoriginalWeshowalesssuccessfulresultofthistechniqueinVideoFigureInthegeneralcase,ahomographyistooconstrainedamodeltosufficientlyfitthedesireddisplacementsThisdeficiencycanresultinundesireddistortion(weshowanindividualframeexampleinFigure),andtemporalwobblingHowever,thisnovelapproachisthebestofthealternativeswehaveconsidereduptonowThefirstsolutiondescribedaboveistooflexibleitexactlysatisfiesthesparsedisplacements,butdoesn

用户评价(0)

关闭

新课改视野下建构高中语文教学实验成果报告(32KB)

抱歉,积分不足下载失败,请稍后再试!

提示

试读已结束,如需要继续阅读或者下载,敬请购买!

文档小程序码

使用微信“扫一扫”扫码寻找文档

1

打开微信

2

扫描小程序码

3

发布寻找信息

4

等待寻找结果

我知道了
评分:

/9

视频去抖动资料

VIP

在线
客服

免费
邮箱

爱问共享资料服务号

扫描关注领取更多福利