下载
加入VIP
  • 专属下载特权
  • 现金文档折扣购买
  • VIP免费专区
  • 千万文档免费下载

上传资料

关闭

关闭

关闭

封号提示

内容

首页 3D1-2

3D1-2.pdf

3D1-2

liujunwei
2012-11-02 0人阅读 举报 0 0 暂无简介

简介:本文档为《3D1-2pdf》,可适用于高等教育领域

DTV:AretwoimagesenoughHowdepthmapscanenhancetheDexperienceCarlosVázquez,WaJamesTamandFilippoSperanzaCommunicationsResearchCenterCanada,CarlingAvenue,Ottawa,ON,KHS,CanadaABSTRACTForDTVviewingthetwoimagesprovidedbystereoscopicimagingsystemsofferverylittlecontrolontheperceivedDexperienceHowever,depthmapsallowforimproveddepthvisualization,throughdisparitycustomizationandnewviewpointgenerationInthiscontributionweexplorethemainproblemsassociatedwithstereoscopicvisualizationandhowtheuseofdepthmapscanhelpsolvesomeofthemTheextractionofthedepthinformationandthegenerationofnewviewstoadjustthedepthofthescenetotheviewingconditionsaretwoofthetopicsthatwillbecoveredWeshowhowtheknowledgeofthedepthinformationcanbeusedtoenhancetheviewerexperiencewithrespecttostereoscopicvisualizationINTRODUCTIONOverthelastcoupleofyearswehavewitnessedanexplosioninstereoscopicDsystemsandcontentgenerationforentertainmentapplicationsThissharpincreaseinstereoscopicDhasbeenmainlydrivenbythedeploymentofdigitalcinemascreensandtheadditionofstereoscopicprojectioncapabilitiesThereleaseofanumberoffeaturefilmsinstereoscopicformatisservingthepurposeoffeedingthesestereoscopicscreensandinconsequenceadvancingthedevelopmentofthestereoscopicDindustryThenaturalnextstepinthedevelopmentofthestereoscopicDindustryistheintroductionofDTeleVision(DTV),withthedevelopmentofDcapabletelevisionsetsandthedistributionofDcontenttothehomeandtomobiledevicescapableofreceivingTVsignalsThereareanumberofcompaniesworkingonthedevelopmentofDTVsets,withsomeofthemalreadybeingcommercializedThereisalsoaconcertedeffortfromseveralstandardbodiestobuildasetofstandardsforthetransmissionandstorageofDcontenttothehomeandmobiledevicesDTVvsDcinemaThereareseveraldifferencesbetweentheDcinemaexperienceandtheDTVexperiencethatmakeitdifficulttodirectlyusethesystemsandcontentcurrentlyusedforDcinemainaTVenvironmentWecanmentionsomeofthesedifferences:Thecinemaisaonetimeevent,whiletheTVviewingismorelikeacontinuouseventwithpeoplenormallysharingthetimeinfrontoftheTVwithotheractivitiesTherelativesizebetweentheviewer'smovementsandthescreensizeareverydifferent:Inacinemaenvironment,thescreensizeisseveraltimeslargerthantheviewer,makinganysmallheadmotionshavingalimitedeffectontheDexperienceoftheviewerThisisnotthecasewiththemuchsmallerscreenandviewingdistanceinthehomeTVprogrammingisalsoavailableformobiledeviceswithsmallscreens,soanyTVserviceshouldconsiderthedeliverytosuchsmallscreenswherethepositionofthescreenwithrespecttotheviewerisconstantlychangingCinemascreensizesarequiteuniform,whileTVscreenssizesvaryfromafewtoaroundahundredinchesTVviewingissometimesasocialevent,withpeoplesharingexperiencesandcommentingonthecontentThesedifferencesinfluencetheDexperienceofTVviewerswhenusingstandardtwoviewstereoscopicsystemsbasedonglassesInthispaperweanalyzetheprocessingpossibilitiesopenedbytheknowledgeofthedepthstructureofthesceneintheformofdepthmapsDepthmapsareakeytoolintheoptimizationofstereoscopicDcontentforvisualizationinDTVdevicesFromtheadjustmentofthemaximumandminimumparallaxallowedforagivenTV,thatwillhelpavoidextremeconditionsthatcouldleadtohealthissues,tothegenerationofmultipleimagesthatwillallowviewerstomovewithoutexperiencingadegradationoftheDexperience,depthmapscanbeveryusefulintheenhancementofDexperienceforDTVapplicationsWewillexaminetheextraction,transmissionandusageofdepthmapsasatoolfortheoptimizationofDcontentsforvisualizationinDTVenvironmentsWHYTWOIMAGESARENOTENOUGHFORDTVThereareseveralreasonsthatexplainwhythestereoscopicDsystemsbasedonglassesandjusttwoimagesarenotthebestfitforapplicationsinDTVenvironmentsThefirstonecitedistheusageofglassesitselfInacinemaenvironmenttheDglassesareaminorannoyance,butathomeoronthego,wearingglassestoseeTVcontentcouldbecomeamajorobstaclebecausetelevisionviewingisoftensharedwithotheractivitiesButglassesarenottheonlyreasonTherigidityassociatedwiththetwoviewsystemisalsoamajorobstacleinthequalityoftheDexperienceLetstakealookatsomeofthemajorissueswithstereoscopicsystemsMotionparallaxWithatwoviewsystem,astheonedepictedinFigure,thepositionoftheviewerwithrespecttothesceneisfixedand,therefore,viewer'smovementsarerestrictedMinormotionsoftheheadfromsidetosideresultinthewholescenetoappeartoswivelfromsidetosidebecauseofthesamesetofimagesbeingusedfromadifferentviewpointFigureshowsageneralschemeofastereoscopicsystemwithtwoviewingpositionsInthisdrawingDrepresentsthedistancetothedisplay,ZistheDInvitedIDW’depthaxis,Btheinteroculardistanceandp=XRXLthedisparityinthescreenplaneAswecanseefromthedrawing,foraviewingpositiondefinedbyL’andR’,thepointP’ismovedtoanewpositionInacinemaenvironmentanymotionoftheheadisineffectivebecauseitissmallwithrespecttothesizeofthescreen,butthisisnotalwaysthecaseforTVandmobiledeviceswherethemagnitudeofheadmovementsarerelativelylargecomparedtothesizeofthescreenFigureGeneralschemeofstereoscopicsystembasedontwoviewsContentcreationTheDstereoscopiccontentgeneratedforDcinemaisoptimizedforprojectionontolargecinemascreensThedirectprojectionofthiscontentintosmallerscreensforTVormobileapplicationswoulddistortthedepthdimensionofthesceneThesedistortionscouldreducetheDexperienceandcouldmakeviewinguncomfortableTheperceiveddepthzPofpointPdependsonthescreendisparity,p,andtheviewingdistanceD,ascanbeseenfromEq:−=pBDzp()WhenthesizeofthedisplayisreducedfromcinematoTV,specialattentionshouldbepaidtothescalingofthedepthdimension,whichisdependentonthescalingofthehorizontaldimension(disparity)andtheviewingdistanceWeareusedtokeepthehorizontalvertical(hv)aspectratioofimageswhenscalingInasimilarwayitisdesirabletokeepthehvzaspectratiooftheDscenewhenscalingaDimageforanewscreensizeandviewingdistanceThisisnotalwayspossiblewhenweonlyhaveaccesstothedisparityandnottotherealdepthSincetherelationbetweenthedisparityandthedepthisnonlinear,alinearscalingindisparitywillproduceanonlinearscalingindepth,resultinginadeformationofobjectsinthedepthdimensionwhenevertheamountofdepthissignificantwithrespecttotheviewingdistanceDisplaysizevariationsAchangeinthesizeofthescreenisasourceofdistortionthatcanevenbecomeahealthissueAstereoscopiccontentpreparedforasmallscreenthatisprojectedintoalargerscreenwillseethedisparitiesincreasedtovaluesthatcouldgobeyondwhatisacceptablefortheaverageviewerForexampleveryfarobjectswithdisparitiesclosetotheinteroculardistancewilldivergeifthedisparityisincreasedbecauseofthesizeofthedisplaybeinglargerthantheoriginaldisplayVaryingviewerpreferencesThereisnowaytochangetheviewingpositioninthecontentthatistransmittedinatwoviewfixedsystemTheonlyadjustmentthatisallowedistheshiftofthewholescenetothebackortothefrontofthescreen,butthisishardlyenoughtoaccountforthedifferentpreferencesofviewerswithrespecttothedepthperceptionSomepeoplearemoretoleranttolargedisparitieswhileothershavemoredifficultiesfusingstereoscopicpairsSummaryScalingimagesfordifferentscreensizesislimitedwhenonlytwoviewsareavailableforstereoscopicviewingFurthermore,verylittlecustomizationisavailableOnlythezeroparallaxplanecanbechangedandthiswilldistortthedepthofthescene,degradingtheDexperienceofthefinalviewerHOWTOIMPROVETHEDEXPERIENCEATHOMETHENEEDFORMOREIMAGESThemainissuewithtwoviewsystemsispreciselythatthereareonlytwoviewsavailableToaccompanyachangeinheadorviewingpositionortomodifytheamountofperceiveddepthinaprojection,aDTVsystemrequiresthegenerationofnewimages,astheywouldbeseenfromdifferentpointsofviewTogeneratethesenewviewsatransformationrelatedtothedepthstructureofthesceneisrequiredThus,knowledgeofthedepthinformationofthedepictedsceneisessentialforimprovingtheDvisualexperienceByhavingdirectaccesstothedepthwehavetheabilitytolinearlymanipulatethedepthand,togeneratenewvirtualviewsthroughdepthimagebasedrendering(DIBR)Thesenewviewscanbegeneratedtoadjustthemaximumandminimumparallaxinsuchawaythattheycorrespondtotheviewingconditions,givennotonlybythesizeandviewingdistanceofthescreen,butalsobyviewers’preferencesVirtualviewscanalsoservetoprovideaconsistentmotionparallaxsensationbyusingdifferentimageswhentheviewermovesFurthermore,virtualviewscanfeedmultipleviewautostereoscopicmonitorstoallowglassesfreeviewingofDcontentIngeneral,theavailabilityofthedepthinformationoffersthepossibilityofgeneratingnewviewsthatservetoimprovetherealismandvisualcomfortoftheperceiveddepthandasaconsequenceimprovetheDvisualexperienceofviewersGENERATINGNEWVIRTUALVIEWSTogeneratenewvirtualviewstwocomponentsareneeded:knowledgeofthedepthstructureofthesceneandaDepthImageBasedrendering(DIBR)algorithm,forthegenerationofnewvirtualviewsTheR’L’RLDXLXRpZBScreenPPIDW’IntermediateViewReconstruction(IVR)fromstereoscopicorMultiViewimagescanbeseenasageneralizationofDIBRwithmorethanoneimageavailableGettingthedepthinformationThefirststepinthegenerationofnewviewsistheextractionofdepthmapsfromtheavailablecontentbeingitmultiview,stereoormonoscopicimagesequencesIfmorethanoneviewisavailable,thedepthinformationiscontainedinthevideoandcanbeestimatedbyanalyzingthedisparitiesbetweendifferentviewsIfthereisjustoneviewavailablethentheextractionofthedepthinformationhastobebasedonotherdepthcuessuchasmotion,shadows,perspectiveorblurOneinterestingresultthatwehaveshownisthatthedepthmapsdonothavetobedenseorhighlyaccurateinordertogenerateimageswithnewviewpointsthatofferanenhancedDperceptionDtoDconversion:ColordepthingRecently,weintroducedtheconceptof“surrogatedepthmaps”anddemonstratedhowtheycanbeappliedtoenhancethedepthsensationoforiginalDcontentsfordisplayonDscreens“Surrogatedepthmaps”canbegeneratedfromacolorcomponentoftheoriginalDimagesfortheconversionofDmaterialtoDOurnovelcolordepthingmethodusesthe“Cr”chromacomponentofaDcolorimageintheYCbCrcolorspaceasaninitialsurrogatedepthmapandthencorrectsverybrightredregionstoincreasethedepth(movethemfartherfromtheviewer)witharedreductionstrategyAsafirststeptheverybrightredregionsthatarenotidentifiedasskintonesareidentified:)GT(B)TG(R)T(RMSGR>∧>∧>=()whereMdefinestheverybrightredregions,(R,G,B)identifythethreecolorchannelsandTR,TGandTSarerespectivelyared,greenandskintonethresholdsThedepthvaluesattheselectedbrightredregionsisthenincreasedtopreventredregionsfromappearingtopopoutofthescreen:))(()()(mDMDMDMD−=α()ThefactorcontrolstheamountofreductionthatisappliedtotheredcontentFigureshowsanexampleofourchromadepthingmethodappliedtoasingleimageFigureChromadepthingresultsDepthfrommultiviewcontentWhenmorethanoneviewisavailablethedepthinformationcanbeextractedwithhighconfidencefromtheavailablecontentThisisknownasthestereoproblemfortwoviewsandiswelldocumentedintheliteratureForthemultiviewcase,wehavedevelopedamethodforextractingthedepthinformationinordertouseitforthegenerationofvirtualviewsOurproposedmethodisbasedonanexhaustivesearchstrategyandTotalVariationregularizationthatleadstoanaccurateestimationofthedepthandocclusionregionsFigureshowstheresultingdepthmapcreatedwithourproposedmethodforthewellknowntestimage“Cones”FigureDepthestimatedfromamultiviewsourceRenderingnewvirtualviewsNewvirtualviewsaregeneratedusingDIBRmethodsDIBRtechniquesarebasedontheshiftingofindividualpixelsbasedonitscorrespondingdepthconvertedintodisparityforagivencamerapositionAlthoughthisseemslikeasimplegeometricaloperationthereareanumberofissuesthatneedtobetakenintoaccountAmongthemwecountthetreatmentofedgesinthecoloranddepthimagesandtheexistenceofoccludedandnewlyexposedregionsThe“filling”ofholescreatedbythedisoccludedregionsisoneofthemostchallengingproblemsinthegenerationofnewvirtualviewsfromDDepthcontentOcclusionsandnewlyexposedregionsOcclusionsappearwhenaforegroundobjecthappenstobeinfrontofamoredistantobjectinthenewviewThestandardsolutionforthissituationistorenderpixelsinagivenordertoreplacemoredistantpixelswiththeonesthatarecloserThetreatmentofdisoccludedregionsismorechallengingsincethisinformationisnotalwaysavailableandhastobeextractedfromotherimagesorotherpartsoftheimageinordertofilltheholescreatedbytherenderingprocessWhengeneratingnewviewsfromstereoormultiviewcontentthedisocclusioninformationcanbe,atleastpartially,recoveredfromtheavailableimagesOntheotherhand,whengeneratingnewimagesfromjustoneviewandthedepthmap,theinformationforthedisocclusionsisnotavailableInthiscasesomekindof“inpainting”hastobeusedtofilltheholesgeneratedbytheDIBRprocessInourpreviousstudyofdifferentholefillingmethodswefoundthatthedepthinformationcanbeusedtoavoidforegroundobjectsfrom“leaking”intothebackgroundandhelpenhancetheDexperienceoftheviewerbyimprovingthequalityoftherenderedimageTransmissionofdepthandotherinformationThefinalstageintheusageofdepthmapsforenhancingthevisualexperienceofviewersishowtomakethisinformationavailableatthedisplaysideAlthoughdepthmapscanbelocallygeneratedfrommonoscopicorstereoscopiccontentatthereceptionside,theycanalsobegeneratedatthetransmissionsideandthentransmittedtothedisplayendInthiscase,transmissionofthedepthmapscanbeaccompaniedbyotherinformationthatcanhelpimprovethequalityoftheIDW’generatednewvirtualviewsandenhancetheDviewingexperienceTheextrainformationmightconsistofinformationoftheoccludedregionsorthetransparencyinformationWehaveproposedamethodforthecodingofthedisocclusioninformationbasedoninterpolatingwaveletsOurmethodallowsfortheefficienttransmissionofthedisocclusioninformationtohelpfillthenewlyexposedregionswhenrenderingnewvirtualviewsFigureshowsanexampleofdisocclusioncodingbasedontheproposedalgorithmThedisocclusionsarecodedbyusinganinterpolatingwaveletstrategythatreducestheamountofinformationthatneedtobetransmittedAscanbeseenfromFiguree),thedistributionofvaluesforthewaveletcoefficientsintheresultingrepresentationofthedisocclusionbeingpackedaroundzero,makingitmoreefficienttotransmitaDimagebDepthmapcDisocclusionmaskdDisoccludedregioneWaveletcoefficientsFigureExampleofdisocclusioncodingStandardsforDTVManystandardbodiesarestudyingdifferentaspectsoftheDTVindustrywiththeobjectiveofdevelopingasetofstandardstofacilitatethedeploymentofDTVsystemsAmongthem,theMPEGgroupisstudyingwaystoensurethecodingofstereoandmultiviewmaterialfortransmissionovercurrentandfuturecommunicationchannelsTheJMVC(JointMultiViewVideoCoding)isthereferencesoftwarefortheMultiViewcodingstandardbeingdevelopedbytheMPEGgroupAdditionally,theMPEGgrouphasestablishedastandardforthetransmissionofauxiliaryvideoandsupplementalinformationthatisintendedtosupportthetransmissionofdepthandothernecessaryinformationalongsideMPEGvideostreamsProblemsassociatedtodepthmapsandwaystoovercomethemThemaindisadvantageassociatedtodepthmapsforDTVapplicationsistheneedtogenerateorestimatethatinformationfromtheavailablevideocontentButthisisnottheonlyone,thereareotherimportantlimitationsassociatedwiththeDDepthformatOneofthemostcitedistheabsenceofocclusion,transparencyandreflectioninformationThiskindoflimitationcanbeovercomewithsupplementalinformation,asproposedinourDDepthDisocclusioncodingmethodInthesameline,transparenciescanbeaddedassupplementalinformationashasbeenpreviouslyproposedAsimilarapproachcanbeappliedforreflections,althoughadeeperstudyisneededinordertodefinetheimportanceofsmallreflectionsintheDexperienceCONCLUSIONSWehaveshownhowknowledgeofthedepthstructureofascenecanbehelpfulinenhancingtheDvisualexperienceforDTVapplicationsTwoviewstereoscopicsystemsaredifficulttoadaptfortheparticularconditionsofviewingthatareattachedtoTV,whiletheavailabilityofthedepthinformationwillallowtherenderingofnewvirtualviewstoadjustthevisualexperiencebytakingintoaccountviewerpreferencesandviewingconditionsathomeoronthegoREFERENCESWJTam,CVázquez,andFSperanza,“ThreedimensionalTV:Anovelmethodforgeneratingsurrogatedepthmapsusingcolourinformation”,inStereoscopicDisplaysandApplicationsXX,SanJosé,CA,Jan,ppCVázquezandWJTam,“D–TV:CodingofdisocclusionsforDDepthrepresentationofmulti–viewimages,”inComputerGraphicsandImageProcessing,Innsbruck,Austria,Feb,IASTEDCVázquez,WJTam,andFSperanza,“StereoscopicImaging:FillingDisoccludedAreasinImagebasedRendering”,inInformationTechnologyandComm,vol,Boston,MA,

用户评价(0)

关闭

新课改视野下建构高中语文教学实验成果报告(32KB)

抱歉,积分不足下载失败,请稍后再试!

提示

试读已结束,如需要继续阅读或者下载,敬请购买!

文档小程序码

使用微信“扫一扫”扫码寻找文档

1

打开微信

2

扫描小程序码

3

发布寻找信息

4

等待寻找结果

我知道了
评分:

/4

3D1-2

VIP

在线
客服

免费
邮箱

爱问共享资料服务号

扫描关注领取更多福利