关闭

关闭

关闭

封号提示

内容

首页 Foundations of Statistical Natural Language Proc…

Foundations of Statistical Natural Language Processing.pdf

Foundations of Statistical Natur…

albertium.land 2012-09-09 评分 0 浏览量 0 0 0 0 暂无简介 简介 举报

简介:本文档为《Foundations of Statistical Natural Language Processingpdf》,可适用于高等教育领域,主题内容包含FoundationsofStatisticalNaturalLanguageProcessingEChristopherDManningHinri符等。

FoundationsofStatisticalNaturalLanguageProcessingEChristopherDManningHinrichSchiitzeTheMITPressCambridge,MassachusettsLondon,EnglandSecondprinting,MassachusettsInstituteofTechnologySecondprintingwithcorrections,AllrightsreservedNopartofthisbookmaybereproducedinanyformbyanyelectronicormechanicalmeans(includingphotocopying,recording,orinformationstorageandretrieval)withoutpermissioninwritingfromthepublisherTypesetinloLucidaBrightbytheauthorsusingETPXEPrintedandboundintheUnitedStatesofAmericaLibraryofCongressCataloginginPublicationInformationManning,ChristopherDFoundationsofstatisticalnaturallanguageprocessingChristopherDManning,HinrichSchutzepcmIncludesbibliographicalreferences(p)andindexISBNlComputationallinguisticsStatisticalmethodsISchutze,HinrichIITitlePSM’dcCIPBriefContentsIPreliminariesIntroductionMathematicalFoundationsLinguisticEssentialsCorpusBasedWorkIIWordsCollocationsStatisticalInference:ngramModelsoverSparseDataWordSenseDisambiguationLexicalAcquisitionIIIGrammarMarkovModelsPartofSpeechTaggingProbabilisticContextFreeGrammarsProbabilisticParsingIvApplicationsandTechniquesStatisticalAlignmentandMachineTranslationClusteringTopicsinInformationRetrievalTextCategorizationContentsListofTablesxvListofFiguresxxiTableofNotationsxxvPrefacerodxRoadMapmxvIPreliminariesIntroductionRationalistandEmpiricistApproachestoLanguageScientificContentQuestionsthatlinguisticsshouldanswerNoncategoricalphenomenainlanguageLanguageandcognitionasprobabilisticphenomenaTheAmbiguityofLanguage:WhyNLPIsDifficultDirtyHandsLexicalresourcesWordcountsZipf’slawsCollocationsConcordancesFurtherReadingVlllContentsExercisesMathematicalFoundationsElementaryProbabilityTheoryProbabilityspacesConditionalprobabilityandindependenceBayes’theoremRandomvariablesExpectationandvarianceNotationJointandconditionaldistributionsDeterminingPStandarddistributionsBayesianstatisticsExercisesEssentialInformationTheoryEntropyJointentropyandconditionalentropyMutualinformationThenoisychannelmodelRelativeentropyorKullbackLeiblerdivergenceTherelationtolanguage:CrossentropyTheentropyofEnglishPerplexityExercisesFurtherReadingLinguisticEssentialsPartsofSpeechandMorphologyNounsandpronounsWordsthataccompanynouns:DeterminersandadjectivesVerbsOtherpartsofspeechPhraseStructurePhrasestructuregrammarsDependency:ArgumentsandadjunctsX’theoryPhrasestructureambiguityContentsixSemanticsandPragmaticsOtherAreasFurtherReadingExercisesCorpusBasedWorkGettingSetUpComputersCorporaSoftwareLookingatTextLowlevelformattingissuesTokenization:WhatisawordMorphologySentencesMarkedupDataMarkupschemesGrammaticaltaggingFurtherReadingExercisesIIWordsCollocationsFrequencyMeanandVarianceHypothesisTestingThettestHypothesistestingofdifferencesPearson’schisquaretestLikelihoodratiosMutualInformationTheNotionofCollocationFurtherReadingStatisticalInference:ngramModelsoverSparseDataBins:FormingEquivalenceClassesReliabilityvsdiscriminationmodelsngramContentsBuildingmodelsngramStatisticalEstimatorsMaximumLikelihoodEstimation(MLE)Laplace’slaw,Lidstone’slawandtheJeffreysPerkslawHeldoutestimationCrossvalidation(deletedestimation)GoodTuringestimationBrieflynotedCombiningEstimatorsSimplelinearinterpolationKatz’sbackingoffGenerallinearinterpolationBrieflynotedLanguagemodelsforAustenConclusionsFurtherReadingExercisesWordSenseDisambiguationMethodologicalPreliminariesSupervisedandunsupervisedlearningPseudowordsUpperandlowerboundsonperformanceSupervisedDisambiguationBayesianclassificationAninformationtheoreticapproachDictionaryBasedDisambiguationDisambiguationbasedonsensedefinitionsThesaurusbaseddisambiguationDisambiguationbasedontranslationsinasecondlanguagecorpusOnesenseperdiscourse,onesensepercollocationUnsupervisedDisambiguationWhatIsaWordSenseFurtherReadingExercisesContentsxiLexicalAcquisitionEvaluationMeasuresVerbSubcategorizationAttachmentAmbiguityHindleandRooth()GeneralremarksonPPattachmentSelectionalPreferencesSemanticSimilarityVectormeasuresspaceProbabilisticmeasuresTheRoleofLexicalAcquisitioninStatisticalNLPFurtherReadingIIIGrammarMarkovModelsMarkovModelsHiddenMarkovModelsWhyuseHMMsGeneralformofanHMMTheThreeFundamentalQuestionsforHMMsFindingtheprobabilityofanobservationFindingthebeststatesequenceThethirdproblem:ParameterestimationHMMs:Implementation,Properties,andVariantsImplementationVariantsMultipleinputobservationsInitializationofparametervaluesFurtherReadingPartofSpeechTaggingTheInformationSourcesinTaggingMarkovModelTaggersTheprobabilisticmodelTheViterbialgorithmVariationsHiddenMarkovModelTaggersxiiContentsApplyingHMMstoPOStaggingTheeffectofinitializationonHMMtrainingTransformationBasedLearningofTagsTransformationsThelearningalgorithmRelationtoothermodelsAutomataSummaryOtherMethods,OtherLanguagesOtherapproachestotaggingLanguagesotherthanEnglishTaggingAccuracyandUsesofTaggersTaggingaccuracyApplicationsoftaggingFurtherReadingExercisesProbabilisticContextFreeGrammarsSomeFeaturesofPCFGsQuestionsforPCFGsTheProbabilityofaStringUsinginsideprobabilitiesUsingoutsideprobabilitiesFindingthemostlikelyparseforasentenceTrainingaPCFGProblemswiththeInsideOutsideAlgorithmFurtherReadingExercisesProbabilisticParsingSomeConceptsParsingfordisambiguationTreebanksParsingmodelsvslanguagemodelsWeakeningtheindependenceassumptionsofPCFGsTreeprobabilitiesandderivationalprobabilitiesThere’smorethanonewaytodoitContentsXlPhrasestructuregrammarsanddependencygrammarsEvaluationEquivalentmodelsBuildingSearchmethodsparsers:UseofthegeometricmeanSomeApproachesNonlexicalizedtreebankgrammarsLexicalizedmodelsusingderivationalhistoriesDependencybasedmodelsDiscussionFurtherReadingExercisesIVApplicationsandTechniquesStatisticalAlignmentandMachineTranslationTextAlignmentAligningsentencesandparagraphsLengthbasedmethodsOffsetalignmentbysignalprocessingtechniquesLexicalmethodsofsentencealignmentSummaryExercisesWordAlignmentStatisticalMachineTranslationFurtherReadingClusteringHierarchicalClusteringSinglelinkandcompletelinkclusteringGroupaverageagglomerativeclusteringAnapplication:ImprovingalanguagemodelTopdownclusteringNonHierarchicalClusteringKmeansTheEMalgorithmFurtherReadingxivContentsExercisesTopicsinInformationRetrievalSomeBackgroundonInformationRetrievalCommondesignfeaturesofIRsystemsEvaluationmeasuresTheprobabilityrankingprinciple(PRP)TheVectorSpaceModelVectorsimilarityTermweightingTermDistributionModelsThePoissondistributionThetwoPoissonmodelTheKmixtureInversedocumentfrequencyResidualinversedocumentfrequencyUsageoftermdistributionmodelsLatentSemanticIndexingLeastsquaresmethodsSingularValueDecompositionLatentSemanticIndexinginIRDiscourseSegmentationTextTilingFurtherReadingExercisesTextCategorizationDecisionTreesMaximumEntropyModelingGeneralizediterativescalingApplicationtotextcategorizationPerceptronskNearestNeighborClassificationFurtherReadingTinyStatisticalTablesBibliographyIndexListofTablesCommonwordsinTomSawyerFrequencyoffrequenciesofwordtypesinTomSawyerEmpiricalevaluationofZipf’slawonTomSawyerCommonestbigramcollocationsintheNewYorkTimesFrequentbigramsafterfilteringLikelihoodratiosbetweentwotheoriesStatisticalNLPproblemsasdecodingproblemsCommoninflectionsofnounsPronounformsinEnglishFeaturescommonlymarkedonverbsMajorsuppliersofelectroniccorporawithcontactURLsDifferentformatsfortelephonenumbersappearinginanissueofTheEconomistSentencelengthsinnewswiretextSizesofvarioustagsetsComparisonofdifferenttagsets:adjective,adverb,conjunction,determiner,noun,andpronountagsComparisonofdifferenttagsets:Verb,preposition,punctuationandsymboltagsFindingCollocations:RawFrequencyPartofspeechtagpatternsforcollocationfilteringFindingCollocations:JustesonandKatz’partofspeechfilterxviListofTablesThenounswoccurringmostofteninthepatterns‘strongw’and‘powerfulw’FindingcollocationsbasedonmeanandvarianceFindingcollocations:ThettestappliedtobigramsthatoccurwithfrequencyWordsthatoccursignificantlymoreoftenwithpowerful(thefirsttenwords)andstrong(thelasttenwords)AbytableshowingthedependenceofoccurrencesofnewandcompaniesCorrespondenceofvacheandcowinanalignedcorpusTestingfortheindependenceofwordsindifferentcorporausingxHowtocomputeDunning’slikelihoodratiotestBigramsofpowerfulwiththehighestscoresaccordingtoDunning’slikelihoodratiotestDamerau’sfrequencyratiotestFindingcollocations:Tenbigramsthatoccurwithfrequency,rankedaccordingtomutualinformationCorrespondenceofchambreandhouseandcommuneSandhouseinthealignedHansardcorpusProblemsforMutualInformationfromdatasparsenessDifferentdefinitionsofmutualinformationin(CoverandThomas)and(Fano)CollocationsintheBBICombinatoryDictionaryofEnglishforthewordsstrengthandpowerGrowthinnumberofparametersforngrammodelsNotationforthestatisticalestimationchapterProbabilitiesofeachsuccessivewordforaclausefromPersuasionEstimatedfrequenciesfortheAPdatafromChurchandGale(a)ExpectedLikelihoodEstimationestimatesforthewordfollowingwasUsingthettestforcomparingtheperformanceoftwosystemsExtractsfromthefrequenciesoffrequenciesdistributionforbigramsandtrigramsintheAustencorpusListofTablesxviiGoodTuringestimatesforbigrams:AdjustedfrequenciesandprobabilitiesGoodTuringbigramfrequencyestimatesfortheclausefromPersuasionBackofflanguagemodelswithGoodTuringestimationtestedonPersuasionProbabilityestimatesofthetestclauseaccordingtovariouslanguagemodelsNotationalconventionsusedinthischapterCluesfortwosensesofdrugusedbyaBayesianclassifierHighlyinformativeindicatorsforthreeambiguousFrenchwordsTwosensesofashDisambiguationofashwithLesk’salgorithmSomeresultsofthesaurusbaseddisambiguationHowtodisambiguateinterestusingasecondlanguagecorpusExamplesoftheonesenseperdiscourseconstraintSomeresultsofunsuperviseddisambiguationTheFmeasureandaccuracyaredifferentobjectivefunctionsSomesubcategorizationframeswithexampleverbsandsentencesSomesubcategorizationframeslearnedbyManning’ssystemAnexamplewherethesimplemodelforresolvingPPattachmentambiguityfailsSelectionalPreferenceStrength(SPS)Associationstrengthdistinguishesaverb’splausibleandimplausibleobjectsSimilaritymeasuresforbinaryvectorsThecosineasameasureofsemanticsimilarityMeasuresof(dis)similaritybetweenprobabilitydistributionsTypesofwordsoccurringintheLOBcorpusthatwerenotcoveredbytheOALDdictionaryNotationusedintheHMMchapterVariablecalculationsfor=(lem,icet,cola)SomepartofspeechtagsfrequentlyusedfortaggingEnglishxvlllListofTablesNotationalconventionsfortaggingIdealizedcountsofsometagtransitionsintheBrownCorpusIdealizedcountsoftagsthatsomewordsoccurwithintheBrownCorpusTableofprobabilitiesfordealingwithunknownwordsintaggingInitializationoftheparametersofanHMMTriggeringenvironmentsinBrill’stransformationbasedtaggerExamplesofsometransformationslearnedintransformationbasedtaggingExamplesoffrequenterrorsofprobabilistictaggersAportionofaconfusionmatrixforpartofspeechtaggingNotationforthePCFGchapterAsimpleProbabilisticContextFreeGrammar(PCFG)CalculationofinsideprobabilitiesAbbreviationsforphrasalcategoriesinthePennTreebankFrequencyofcommonsubcategorizationframes(localtreesexpandingVP)forselectedverbsSelectedcommonexpansionsofNPasSubjectvsObject,orderedbylogoddsratioSelectedcommonexpansionsofNPasfirstandsecondobjectinsideVPPrecisionandrecallevaluationresultsforPPattachmenterrorsfordifferentstylesofphrasestructureComparisonofsomestatisticalparsingsystemsSentencealignmentpapersAsummaryoftheattributesofdifferentclusteringalgorithmsSymbolsusedintheclusteringchapterSimilarityfunctionsusedinclusteringAnexampleofKmeansclusteringAnexampleofaGaussianmixtureAsmallstoplistforEnglishAnexampleoftheevaluationofrankingsListofTablesThreequantitiesthatarecommonlyusedintermweightingininformationretrievalTermanddocumentfrequenciesoftwowordsinanexamplecorpusComponentsoftfidfweightingschemesDocumentfrequency(df)andcollectionfrequency(cf)forwordsintheNewYorkTimescorpusActualandestimatednumberofdocumentswithkoccurrencesforsixtermsExampleforexploitingcooccurrenceincomputingcontentsimilarityThematrixofdocumentcorrelationsBTBSomeexamplesofclassificationtasksinNLPContingencytableforevaluatingabinaryclassifierTherepresentationofdocument,showninfigureAnexampleofinformationgainasasplittingcriterionContingencytableforadecisiontreefortheReuterscategory“earnings”Anexampleofamaximumentropydistributionintheformofequation()AnempiricaldistributionwhosecorrespondingmaximumentropydistributionistheoneintableFeatureweightsinmaximumentropymodelingforthecategory“earnings”inReutersClassificationresultsforthedistributioncorrespondingtotableonthetestsetPerceptronforthe“earnings”categoryClassificationresultsfortheperceptronintableonthetestsetClassificationresultsforanNNcategorizerforthe“earnings”categoryxixListofFiguresThenoisychannelmodelAbinarysymmetricchannelThenoisychannelmodelinlinguisticsAnexampleofrecursivephrasestructureexpansionAnexampleofaprepositionalphraseattachmentambiguityHeuristicsentenceboundarydetectionalgorithmAsentenceastaggedaccordingtoseveraldifferenttagsetsZipf’slawMandelbrot’sformulaKeyWordInContext(KWIC)displayforthewordshowedSyntacticframesforshowedinTomSawyerAdiagramillustratingthecalculationofconditionalprobabilityP(AJB)ArandomvariableXforthesumoftwodiceTwoexamplesofbinomialdistributions:b(r,)andb(r,Ol)Examplenormaldistributioncurves:n(x,l)andn(x,)TheentropyofaweightedcoinTherelationshipbetweenmutualinformationIandentropyHUsingathreewordcollocationalwindowtocapturebigramsatadistanceListofFiguresHistogramsofthepositionofstrongrelativetothreewordsBayesiandisambiguationTheFlipFlopalgorithmappliedtofindingindicatorsfordisambiguationLesk’sdictionarybaseddisambiguationalgorithmThesaurusbaseddisambiguationAdaptivethesaurusbaseddisambiguationDisambiguationbasedonasecondlanguagecorpusDisambiguationbasedon“onesensepercollocation”and“onesenseperdiscourse”AnEMalg

用户评论(0)

0/200

精彩专题

上传我的资料

每篇奖励 +1积分

资料评分:

/49
仅支持在线阅读

意见
反馈

立即扫码关注

爱问共享资料微信公众号

返回
顶部

举报
资料