关闭

关闭

关闭

封号提示

内容

首页 Foundations of Statistical Natural Language Proc…

Foundations of Statistical Natural Language Processing.pdf

Foundations of Statistical Natu…

albertium.land
2012-09-09 0人阅读 0 0 0 暂无简介 举报

简介:本文档为《Foundations of Statistical Natural Language Processingpdf》,可适用于高等教育领域

FoundationsofStatisticalNaturalLanguageProcessingEChristopherDManningHinrichSchiitzeTheMITPressCambridge,MassachusettsLondon,EnglandSecondprinting,MassachusettsInstituteofTechnologySecondprintingwithcorrections,AllrightsreservedNopartofthisbookmaybereproducedinanyformbyanyelectronicormechanicalmeans(includingphotocopying,recording,orinformationstorageandretrieval)withoutpermissioninwritingfromthepublisherTypesetinloLucidaBrightbytheauthorsusingETPXEPrintedandboundintheUnitedStatesofAmericaLibraryofCongressCataloginginPublicationInformationManning,ChristopherDFoundationsofstatisticalnaturallanguageprocessingChristopherDManning,HinrichSchutzepcmIncludesbibliographicalreferences(p)andindexISBNlComputationallinguisticsStatisticalmethodsISchutze,HinrichIITitlePSM’dcCIPBriefContentsIPreliminariesIntroductionMathematicalFoundationsLinguisticEssentialsCorpusBasedWorkIIWordsCollocationsStatisticalInference:ngramModelsoverSparseDataWordSenseDisambiguationLexicalAcquisitionIIIGrammarMarkovModelsPartofSpeechTaggingProbabilisticContextFreeGrammarsProbabilisticParsingIvApplicationsandTechniquesStatisticalAlignmentandMachineTranslationClusteringTopicsinInformationRetrievalTextCategorizationContentsListofTablesxvListofFiguresxxiTableofNotationsxxvPrefacerodxRoadMapmxvIPreliminariesIntroductionRationalistandEmpiricistApproachestoLanguageScientificContentQuestionsthatlinguisticsshouldanswerNoncategoricalphenomenainlanguageLanguageandcognitionasprobabilisticphenomenaTheAmbiguityofLanguage:WhyNLPIsDifficultDirtyHandsLexicalresourcesWordcountsZipf’slawsCollocationsConcordancesFurtherReadingVlllContentsExercisesMathematicalFoundationsElementaryProbabilityTheoryProbabilityspacesConditionalprobabilityandindependenceBayes’theoremRandomvariablesExpectationandvarianceNotationJointandconditionaldistributionsDeterminingPStandarddistributionsBayesianstatisticsExercisesEssentialInformationTheoryEntropyJointentropyandconditionalentropyMutualinformationThenoisychannelmodelRelativeentropyorKullbackLeiblerdivergenceTherelationtolanguage:CrossentropyTheentropyofEnglishPerplexityExercisesFurtherReadingLinguisticEssentialsPartsofSpeechandMorphologyNounsandpronounsWordsthataccompanynouns:DeterminersandadjectivesVerbsOtherpartsofspeechPhraseStructurePhrasestructuregrammarsDependency:ArgumentsandadjunctsX’theoryPhrasestructureambiguityContentsixSemanticsandPragmaticsOtherAreasFurtherReadingExercisesCorpusBasedWorkGettingSetUpComputersCorporaSoftwareLookingatTextLowlevelformattingissuesTokenization:WhatisawordMorphologySentencesMarkedupDataMarkupschemesGrammaticaltaggingFurtherReadingExercisesIIWordsCollocationsFrequencyMeanandVarianceHypothesisTestingThettestHypothesistestingofdifferencesPearson’schisquaretestLikelihoodratiosMutualInformationTheNotionofCollocationFurtherReadingStatisticalInference:ngramModelsoverSparseDataBins:FormingEquivalenceClassesReliabilityvsdiscriminationmodelsngramContentsBuildingmodelsngramStatisticalEstimatorsMaximumLikelihoodEstimation(MLE)Laplace’slaw,Lidstone’slawandtheJeffreysPerkslawHeldoutestimationCrossvalidation(deletedestimation)GoodTuringestimationBrieflynotedCombiningEstimatorsSimplelinearinterpolationKatz’sbackingoffGenerallinearinterpolationBrieflynotedLanguagemodelsforAustenConclusionsFurtherReadingExercisesWordSenseDisambiguationMethodologicalPreliminariesSupervisedandunsupervisedlearningPseudowordsUpperandlowerboundsonperformanceSupervisedDisambiguationBayesianclassificationAninformationtheoreticapproachDictionaryBasedDisambiguationDisambiguationbasedonsensedefinitionsThesaurusbaseddisambiguationDisambiguationbasedontranslationsinasecondlanguagecorpusOnesenseperdiscourse,onesensepercollocationUnsupervisedDisambiguationWhatIsaWordSenseFurtherReadingExercisesContentsxiLexicalAcquisitionEvaluationMeasuresVerbSubcategorizationAttachmentAmbiguityHindleandRooth()GeneralremarksonPPattachmentSelectionalPreferencesSemanticSimilarityVectormeasuresspaceProbabilisticmeasuresTheRoleofLexicalAcquisitioninStatisticalNLPFurtherReadingIIIGrammarMarkovModelsMarkovModelsHiddenMarkovModelsWhyuseHMMsGeneralformofanHMMTheThreeFundamentalQuestionsforHMMsFindingtheprobabilityofanobservationFindingthebeststatesequenceThethirdproblem:ParameterestimationHMMs:Implementation,Properties,andVariantsImplementationVariantsMultipleinputobservationsInitializationofparametervaluesFurtherReadingPartofSpeechTaggingTheInformationSourcesinTaggingMarkovModelTaggersTheprobabilisticmodelTheViterbialgorithmVariationsHiddenMarkovModelTaggersxiiContentsApplyingHMMstoPOStaggingTheeffectofinitializationonHMMtrainingTransformationBasedLearningofTagsTransformationsThelearningalgorithmRelationtoothermodelsAutomataSummaryOtherMethods,OtherLanguagesOtherapproachestotaggingLanguagesotherthanEnglishTaggingAccuracyandUsesofTaggersTaggingaccuracyApplicationsoftaggingFurtherReadingExercisesProbabilisticContextFreeGrammarsSomeFeaturesofPCFGsQuestionsforPCFGsTheProbabilityofaStringUsinginsideprobabilitiesUsingoutsideprobabilitiesFindingthemostlikelyparseforasentenceTrainingaPCFGProblemswiththeInsideOutsideAlgorithmFurtherReadingExercisesProbabilisticParsingSomeConceptsParsingfordisambiguationTreebanksParsingmodelsvslanguagemodelsWeakeningtheindependenceassumptionsofPCFGsTreeprobabilitiesandderivationalprobabilitiesThere’smorethanonewaytodoitContentsXlPhrasestructuregrammarsanddependencygrammarsEvaluationEquivalentmodelsBuildingSearchmethodsparsers:UseofthegeometricmeanSomeApproachesNonlexicalizedtreebankgrammarsLexicalizedmodelsusingderivationalhistoriesDependencybasedmodelsDiscussionFurtherReadingExercisesIVApplicationsandTechniquesStatisticalAlignmentandMachineTranslationTextAlignmentAligningsentencesandparagraphsLengthbasedmethodsOffsetalignmentbysignalprocessingtechniquesLexicalmethodsofsentencealignmentSummaryExercisesWordAlignmentStatisticalMachineTranslationFurtherReadingClusteringHierarchicalClusteringSinglelinkandcompletelinkclusteringGroupaverageagglomerativeclusteringAnapplication:ImprovingalanguagemodelTopdownclusteringNonHierarchicalClusteringKmeansTheEMalgorithmFurtherReadingxivContentsExercisesTopicsinInformationRetrievalSomeBackgroundonInformationRetrievalCommondesignfeaturesofIRsystemsEvaluationmeasuresTheprobabilityrankingprinciple(PRP)TheVectorSpaceModelVectorsimilarityTermweightingTermDistributionModelsThePoissondistributionThetwoPoissonmodelTheKmixtureInversedocumentfrequencyResidualinversedocumentfrequencyUsageoftermdistributionmodelsLatentSemanticIndexingLeastsquaresmethodsSingularValueDecompositionLatentSemanticIndexinginIRDiscourseSegmentationTextTilingFurtherReadingExercisesTextCategorizationDecisionTreesMaximumEntropyModelingGeneralizediterativescalingApplicationtotextcategorizationPerceptronskNearestNeighborClassificationFurtherReadingTinyStatisticalTablesBibliographyIndexListofTablesCommonwordsinTomSawyerFrequencyoffrequenciesofwordtypesinTomSawyerEmpiricalevaluationofZipf’slawonTomSawyerCommonestbigramcollocationsintheNewYorkTimesFrequentbigramsafterfilteringLikelihoodratiosbetweentwotheoriesStatisticalNLPproblemsasdecodingproblemsCommoninflectionsofnounsPronounformsinEnglishFeaturescommonlymarkedonverbsMajorsuppliersofelectroniccorporawithcontactURLsDifferentformatsfortelephonenumbersappearinginanissueofTheEconomistSentencelengthsinnewswiretextSizesofvarioustagsetsComparisonofdifferenttagsets:adjective,adverb,conjunction,determiner,noun,andpronountagsComparisonofdifferenttagsets:Verb,preposition,punctuationandsymboltagsFindingCollocations:RawFrequencyPartofspeechtagpatternsforcollocationfilteringFindingCollocations:JustesonandKatz’partofspeechfilterxviListofTablesThenounswoccurringmostofteninthepatterns‘strongw’and‘powerfulw’FindingcollocationsbasedonmeanandvarianceFindingcollocations:ThettestappliedtobigramsthatoccurwithfrequencyWordsthatoccursignificantlymoreoftenwithpowerful(thefirsttenwords)andstrong(thelasttenwords)AbytableshowingthedependenceofoccurrencesofnewandcompaniesCorrespondenceofvacheandcowinanalignedcorpusTestingfortheindependenceofwordsindifferentcorporausingxHowtocomputeDunning’slikelihoodratiotestBigramsofpowerfulwiththehighestscoresaccordingtoDunning’slikelihoodratiotestDamerau’sfrequencyratiotestFindingcollocations:Tenbigramsthatoccurwithfrequency,rankedaccordingtomutualinformationCorrespondenceofchambreandhouseandcommuneSandhouseinthealignedHansardcorpusProblemsforMutualInformationfromdatasparsenessDifferentdefinitionsofmutualinformationin(CoverandThomas)and(Fano)CollocationsintheBBICombinatoryDictionaryofEnglishforthewordsstrengthandpowerGrowthinnumberofparametersforngrammodelsNotationforthestatisticalestimationchapterProbabilitiesofeachsuccessivewordforaclausefromPersuasionEstimatedfrequenciesfortheAPdatafromChurchandGale(a)ExpectedLikelihoodEstimationestimatesforthewordfollowingwasUsingthettestforcomparingtheperformanceoftwosystemsExtractsfromthefrequenciesoffrequenciesdistributionforbigramsandtrigramsintheAustencorpusListofTablesxviiGoodTuringestimatesforbigrams:AdjustedfrequenciesandprobabilitiesGoodTuringbigramfrequencyestimatesfortheclausefromPersuasionBackofflanguagemodelswithGoodTuringestimationtestedonPersuasionProbabilityestimatesofthetestclauseaccordingtovariouslanguagemodelsNotationalconventionsusedinthischapterCluesfortwosensesofdrugusedbyaBayesianclassifierHighlyinformativeindicatorsforthreeambiguousFrenchwordsTwosensesofashDisambiguationofashwithLesk’salgorithmSomeresultsofthesaurusbaseddisambiguationHowtodisambiguateinterestusingasecondlanguagecorpusExamplesoftheonesenseperdiscourseconstraintSomeresultsofunsuperviseddisambiguationTheFmeasureandaccuracyaredifferentobjectivefunctionsSomesubcategorizationframeswithexampleverbsandsentencesSomesubcategorizationframeslearnedbyManning’ssystemAnexamplewherethesimplemodelforresolvingPPattachmentambiguityfailsSelectionalPreferenceStrength(SPS)Associationstrengthdistinguishesaverb’splausibleandimplausibleobjectsSimilaritymeasuresforbinaryvectorsThecosineasameasureofsemanticsimilarityMeasuresof(dis)similaritybetweenprobabilitydistributionsTypesofwordsoccurringintheLOBcorpusthatwerenotcoveredbytheOALDdictionaryNotationusedintheHMMchapterVariablecalculationsfor=(lem,icet,cola)SomepartofspeechtagsfrequentlyusedfortaggingEnglishxvlllListofTablesNotationalconventionsfortaggingIdealizedcountsofsometagtransitionsintheBrownCorpusIdealizedcountsoftagsthatsomewordsoccurwithintheBrownCorpusTableofprobabilitiesfordealingwithunknownwordsintaggingInitializationoftheparametersofanHMMTriggeringenvironmentsinBrill’stransformationbasedtaggerExamplesofsometransformationslearnedintransformationbasedtaggingExamplesoffrequenterrorsofprobabilistictaggersAportionofaconfusionmatrixforpartofspeechtaggingNotationforthePCFGchapterAsimpleProbabilisticContextFreeGrammar(PCFG)CalculationofinsideprobabilitiesAbbreviationsforphrasalcategoriesinthePennTreebankFrequencyofcommonsubcategorizationframes(localtreesexpandingVP)forselectedverbsSelectedcommonexpansionsofNPasSubjectvsObject,orderedbylogoddsratioSelectedcommonexpansionsofNPasfirstandsecondobjectinsideVPPrecisionandrecallevaluationresultsforPPattachmenterrorsfordifferentstylesofphrasestructureComparisonofsomestatisticalparsingsystemsSentencealignmentpapersAsummaryoftheattributesofdifferentclusteringalgorithmsSymbolsusedintheclusteringchapterSimilarityfunctionsusedinclusteringAnexampleofKmeansclusteringAnexampleofaGaussianmixtureAsmallstoplistforEnglishAnexampleoftheevaluationofrankingsListofTablesThreequantitiesthatarecommonlyusedintermweightingininformationretrievalTermanddocumentfrequenciesoftwowordsinanexamplecorpusComponentsoftfidfweightingschemesDocumentfrequency(df)andcollectionfrequency(cf)forwordsintheNewYorkTimescorpusActualandestimatednumberofdocumentswithkoccurrencesforsixtermsExampleforexploitingcooccurrenceincomputingcontentsimilarityThematrixofdocumentcorrelationsBTBSomeexamplesofclassificationtasksinNLPContingencytableforevaluatingabinaryclassifierTherepresentationofdocument,showninfigureAnexampleofinformationgainasasplittingcriterionContingencytableforadecisiontreefortheReuterscategory“earnings”Anexampleofamaximumentropydistributionintheformofequation()AnempiricaldistributionwhosecorrespondingmaximumentropydistributionistheoneintableFeatureweightsinmaximumentropymodelingforthecategory“earnings”inReutersClassificationresultsforthedistributioncorrespondingtotableonthetestsetPerceptronforthe“earnings”categoryClassificationresultsfortheperceptronintableonthetestsetClassificationresultsforanNNcategorizerforthe“earnings”categoryxixListofFiguresThenoisychannelmodelAbinarysymmetricchannelThenoisychannelmodelinlinguisticsAnexampleofrecursivephrasestructureexpansionAnexampleofaprepositionalphraseattachmentambiguityHeuristicsentenceboundarydetectionalgorithmAsentenceastaggedaccordingtoseveraldifferenttagsetsZipf’slawMandelbrot’sformulaKeyWordInContext(KWIC)displayforthewordshowedSyntacticframesforshowedinTomSawyerAdiagramillustratingthecalculationofconditionalprobabilityP(AJB)ArandomvariableXforthesumoftwodiceTwoexamplesofbinomialdistributions:b(r,)andb(r,Ol)Examplenormaldistributioncurves:n(x,l)andn(x,)TheentropyofaweightedcoinTherelationshipbetweenmutualinformationIandentropyHUsingathreewordcollocationalwindowtocapturebigramsatadistanceListofFiguresHistogramsofthepositionofstrongrelativetothreewordsBayesiandisambiguationTheFlipFlopalgorithmappliedtofindingindicatorsfordisambiguationLesk’sdictionarybaseddisambiguationalgorithmThesaurusbaseddisambiguationAdaptivethesaurusbaseddisambiguationDisambiguationbasedonasecondlanguagecorpusDisambiguationbasedon“onesensepercollocation”and“onesenseperdiscourse”AnEMalg

用户评价(0)

关闭

新课改视野下建构高中语文教学实验成果报告(32KB)

抱歉,积分不足下载失败,请稍后再试!

提示

试读已结束,如需要继续阅读或者下载,敬请购买!

评分:

/49

0下载券

立即下载

扫码寻找文档

送下载券

加入VIP

意见
反馈

立即扫码关注

爱问共享资料微信公众号

返回
顶部

举报
资料