Logistic classification model (logit or logistic regression)
文章推薦指數: 80 %
The logistic model (or logit) is a classification model used to predict variables that can take only two values. StatLect Index>Fundamentalsof statistics Logisticclassificationmodel(logitorlogisticregression) byMarcoTaboga,PhD Thelogisticmodel(orlogit)isa classification modelusedtopredictvariablesthatcantakeonlytwo values. Tableofcontents OverviewInterpretationofthepredictedoutputClassificationvsregressionSampleConditionalprobabilitiesThelogisticfunctionExplanationAlternativesThelogitmodelasalatentvariablemodelEstimationbymaximumlikelihoodHypothesistesting Overview Thelogisticclassificationmodelhasthefollowingcharacteristics: theoutputvariable canbeequaltoeither0or1; thepredictedoutput isanumberbetween0and1; asinlinearregression,weuseavectorofestimatedcoefficients tocompute , alinearcombinationoftheinputvariables ; unlikeinlinearregression,wetransform usinganonlinearfunction , tomakesurethatthepredictions are between0and1. Interpretationofthepredictedoutput Inalogitmodel,thepredictedoutput hastwointerpretations: theestimatedprobabilitythat willbeequalto1; ourbestguessofthevalueoftheoutputvariable . Classificationvsregression Alogitmodelisoftencalledlogisticregressionmodel. However,weprefertosticktotheconvention(widespreadinthemachine learningcommunity)ofusingthetermregressiononlyformodelsinwhichthe outputvariableiscontinuous. Therefore,weusethetermclassificationherebecauseinalogitmodelthe outputisdiscrete. Sample Supposethatweobserveasampleofdata for . Eachobservationhas: anoutputvariabledenotedby ; a vectorofinputs,denotedby . Conditionalprobabilities Theoutput cantakeonlytwovalues,either0or1(itisa Bernoullirandom variable). Theprobabilitythattheoutput isequalto1, conditional ontheinputs , isassumedto bewhere is thelogisticfunctionand isa vectorofcoefficients. Theprobabilitythat isequalto0 is Thelogisticfunction Itisimmediatetoseethatthelogisticfunction isalwayspositive. Furthermore,itisincreasingand so thatit satisfies Thus, isawell-definedprobabilitybecauseitliesbetween0and1. Explanation Whyisthelogisticclassificationmodelspecifiedinthismanner? Whyisthelogisticfunctionusedtotransformthelinearcombinationof inputs ? Thesimpleansweristhatwewouldliketodosomethingsimilartowhatwedo inalinear regressionmodel:usealinearcombinationoftheinputsasourprediction oftheoutput. However,ourpredictionneedstobeaprobabilityandthereisnoguarantee thatthelinearcombination isbetween0and1. Thus,weusethelogisticfunctionbecauseitprovidesaconvenientwayof transforming andforcingittolieintheintervalbetween0and1. Alternatives Wecouldhaveusedotherfunctionsthatenjoypropertiessimilartothe logisticfunction. Asamatteroffact,otherpopularclassificationmodelscanbeobtainedby simplysubstitutingthelogisticfunctionwithanotherfunctionandleaving everythingelseinthemodelunchanged. Forexample,bysubstitutingthelogitfunctionwiththecumulative distributionfunctionofastandardnormaldistribution,weobtainthe so-called probit model. Thelogitmodelasalatentvariablemodel Anotherwayofthinkingaboutthelogitmodelistodefinealatentvariable (i.e.,anunobserved variable)where isarandomerrortermthataddsnoisetotherelationshipbetweentheinputs andthevariable . Thelatentvariable isthenassumedtodeterminetheoutput as follows: Fromtheseassumptionsandtheadditionalassumptionthat hasasymmetricdistributionaround , itfollows thatwhere isthecumulativedistribution functionoftheerror . Itturnsoutthatthelogisticfunctionusedtodefinethelogitmodelisthe cumulativedistributionfunctionofasymmetricprobabilitydistribution calledstandardlogisticdistribution. Therefore,thelogitmodelcanbewrittenasalatentvariablemodel, specifiedbyequations(1)and(2)above,inwhichtheerror hasalogisticdistribution. Bychoosingdifferentdistributionsfortheerror , weobtainotherbinaryclassificationmodels. Forexample,ifweassumethat hasastandardnormaldistribution,thenweobtaintheprobitmodel. Estimationbymaximumlikelihood Thevectorofcoefficients isoftenestimatedby maximum likelihoodmethods. Assumethattheobservations inthesampleareIIDanddenotethe vectorofalloutputsby andthe matrixofallinputsby . Thelatterisassumedtohavefull rank. Itispossibletoprove(seethelectureon Maximum likelihoodestimationofthelogitmodel)thatthemaximumlikelihood estimator (whenitexists)canbeobtainedbyperformingsimple Newton-Raphson iterationsasfollows: startfromaguess (e.g., ); recursivelyupdatethe guess:where:and isan diagonalmatrix(i.e.,havingalloff-diagonalentriesequalto ) suchthattheelementsonitsdiagonalare stopwhennumericalconvergenceisachieved,thatis,whenthedifference between and issosmallastobenegligible; setthemaximumlikelihoodestimator equaltothelastupdate (denotethelastiterationby ). Theasymptoticcovariancematrixofthemaximumlikelihoodestimator canbeconsistentlyestimatedby so thatthedistributionoftheestimator isapproximatelynormalwithmeanequalto andcovariance matrix . Hypothesistesting Ifthelogitmodelisestimatedwiththemaximumlikelihoodprocedure illustratedabove,anyoneoftheclassical tests basedonmaximumlikelihoodprocedures(e.g., Wald, Likelihood Ratio,Lagrange Multiplier)canbeusedto testan hypothesisaboutthevectorofcoefficients . Othertestscanbeconstructedbyexploitingtheasymptoticnormalityofthe maximumlikelihoodestimator. Forexample,wecanperformaztesttotestthe nullhypothesis where isthe -th entryofthevectorofcoefficients and . Theteststatistic iswhere isthe -th entryof and isthe -th entryonthediagonalofthematrix . Asthesamplesize increases, convergesindistributiontoa standardnormal distribution.Thelatterdistributioncanbeusedto derivecriticalvaluesandperformthe test. Proof We haveBy theasymptoticnormalityofthemaximumlikelihoodestimator,thenumerator convergesin distributiontoanormalrandomvariablewithmean . Furthermore,theconsistencyofourestimatoroftheasymptoticcovariance matriximplies thatwhere denotesconvergence inprobability.Bythe ContinuousMapping theorem, and, bySlutsky'stheorem, convergesindistributiontoastandardnormalrandomvariable. Howtocite Pleaseciteas: Taboga,Marco(2021)."Logisticclassificationmodel(logitorlogisticregression)",Lecturesonprobabilitytheoryandmathematicalstatistics.KindleDirectPublishing.Onlineappendix.https://www.statlect.com/fundamentals-of-statistics/logistic-classification-model. Thebooks Mostofthelearningmaterialsfoundonthiswebsitearenowavailableinatraditionaltextbookformat. ProbabilityandstatisticsMatrixalgebra Featuredpages Hypothesistesting Meansquareconvergence Bernoullidistribution Gammadistribution Likelihoodratiotest Pointestimation Explore Gammafunction Multivariatenormaldistribution Waldtest Mainsections Mathematicaltools Fundamentalsofprobability Probabilitydistributions Asymptotictheory Fundamentalsofstatistics Glossary About AboutStatlect Contacts Cookies,privacyandtermsofuse Glossaryentries Continuousmappingtheorem Nullhypothesis TypeIIerror Meansquarederror Distributionfunction Probabilitydensityfunction Share Toenhanceyourprivacy, weremovedthesocialbuttons, butdon'tforgettoshare.
延伸文章資訊
- 1What is Logistic regression? | IBM
This type of statistical model (also known as logit model) is often used for classification and p...
- 2Logit Models for Binary Data
Logit Models for Binary. Data. We now turn our attention to regression models for dichotomous dat...
- 3羅吉斯迴歸分析(Logistic regression, logit model) - 永析統計
羅吉斯迴歸分析(Logistic regression, logit model)-統計說明與SPSS操作 · 1. 模式係數的Omnibus測試:相當於線性迴歸裡的ANOVA-F檢定,探討羅吉...
- 4Logit Regression | R Data Analysis Examples
Logistic regression, also called a logit model, is used to model dichotomous outcome variables. I...
- 5邏輯迴歸- 維基百科,自由的百科全書
二類評定模型(Binary Logit Model)編輯 ; categorical, percentage, Chi-square test, 邏輯迴歸 ; persontime, KM es...