6.2. The Multinomial Logit Model - Germán Rodríguez

Perhaps the simplest approach to multinomial data is to nominate one of the response categories as a baseline or reference cell, calculate log-odds for all ... / Courses GLMs Multilevel Survival Demography Software Stata R Markstat / GLMs Multilevel Survival Demography Stata R Markstat GermánRodríguez GeneralizedLinearModels PrincetonUniversity Home LectureNotes StataLogs RLogs Datasets ProblemSets   LectureNotes Home LectureNotes StataLogs RLogs Datasets ProblemSets 6.2TheMultinomialLogitModel Wenowconsidermodelsfortheprobabilities\(\pi_{ij}\). Inparticular,wewouldliketoconsidermodelswhere theseprobabilitiesdependonavector\(\boldsymbol{x}_i\)ofcovariates associatedwiththe\(i\)-thindividualorgroup. Intermsofourexample,wewouldliketomodelhowthe probabilitiesofbeingsterilized,usinganothermethodor usingnomethodatalldependonthewoman’sage. 6.2.1MultinomialLogits Perhapsthesimplestapproachtomultinomialdataisto nominateoneoftheresponsecategoriesasabaselineor referencecell,calculatelog-oddsforallother categoriesrelativetothebaseline,andthenletthe log-oddsbealinearfunctionofthepredictors. Typicallywepickthelastcategoryasabaseline andcalculatetheoddsthatamemberofgroup\(i\)fallsin category\(j\)asopposedtothebaselineas \(\pi_{i1}/\pi_{iJ}\).Inourexamplewecouldlook attheoddsofbeingsterilizedratherthanusingnomethod, andtheoddsofusinganothermethodratherthannomethod. Forwomenaged45–49theseoddsare91:183(orroughly1to2) and10:183(or1to18). Figure6.2Log-OddsofSterilizationvs.NoMethodandOtherMethodvs.NoMethod,byAge Figure6.2showstheempiricallog-oddsofsterilizationand othermethod(usingnomethodasthereferencecategory) plottedagainstthemid-pointsoftheagegroups. (Ignorefornowthesolidlines.) Notehowthelog-oddsofsterilizationincreaserapidlywithage toreachamaximumat30–34andthendeclineslightly. Thelog-oddsofusingothermethodsrisegentlyuptoage25–29and thendeclinerapidly. 6.2.2ModelingtheLogits Inthemultinomiallogitmodelweassumethatthe log-oddsofeachresponsefollowalinearmodel \[\tag{6.3}\eta_{ij}=\log\frac{\pi_{ij}}{\pi_{iJ}}=\alpha_j+\boldsymbol{x}_i'\boldsymbol{\beta}_j,\] where\(\alpha_j\)isaconstantand\(\boldsymbol{\beta}_j\)isavectorofregression coefficients,for\(j=1,2,\ldots,J-1\). Notethatwehavewrittentheconstantexplicitly, sowewillassumehenceforththatthe modelmatrix\(\boldsymbol{X}\)doesnotincludeacolumnofones. Thismodelisanalogoustoalogisticregressionmodel,except thattheprobabilitydistributionoftheresponseismultinomial insteadofbinomialandwehave\(J-1\)equationsinsteadofone. The\(J-1\)multinomiallogitequationscontrasteachofcategories \(1,2,\ldotsJ-1\)withcategory\(J\),whereasthesinglelogistic regressionequationisacontrastbetweensuccessesandfailures. If\(J=2\)themultinomiallogitmodelreducestotheusual logisticregressionmodel. Notethatweneedonly\(J-1\)equationstodescribeavariable with\(J\)responsecategoriesandthatitreallymakesnodifference whichcategorywepickasthereferencecell,becausewecan alwaysconvertfromoneformulationtoanother. Inourexamplewith\(J=3\)categorieswecontrastcategories1versus3 and2versus3.Themissingcontrastbetweencategories1and2 caneasilybeobtainedintermsoftheothertwo,since \(\log(\pi_{i1}/\pi_{i2})= \log(\pi_{i1}/\pi_{i3})-\log(\pi_{i2}/\pi_{i3})\). LookingatFigure6.2,itwouldappearthatthelogitsare aquadraticfunctionofage.Wewillthereforeentertainthemodel \[\tag{6.4}\eta_{ij}=\alpha_j+\beta_ja_i+\gamma_ja_i^2,\] where\(a_i\)isthemidpointofthe\(i\)-thagegroup and\(j=1,2\)forsterilizationandothermethod,respectively. 6.2.3ModelingtheProbabilities Themultinomiallogitmodelmayalsobewrittenintermsofthe originalprobabilities\(\pi_{ij}\)ratherthanthelog-odds. Startingfromr<>q:mlogit> andadoptingtheconventionthat\(\eta_{iJ}=0\),wecanwrite \[\tag{6.5}\pi_{ij}=\frac{\exp\{\eta_{ij}\}} {\sum_{k=1}^J\exp\{\eta_{ik}\}}.\] for\(j=1,\ldots,J\). ToverifythisresultexponentiateEquation6.3 toobtain \(\pi_{ij}=\pi_{iJ}\exp\{\eta_{ij}\}\),andnotethatthe convention\(\eta_{iJ}=0\)makesthisformulavalidforall\(j\). Nextsumover\(j\)andusethefactthat\(\sum_j\pi_{ij}=1\) toobtain\(\pi_{iJ}=1/\sum_j\exp\{\eta_{ij}\}\). Finally,usethisresultontheformulafor\(\pi_{ij}\). NotethatEquation6.5willautomaticallyyieldprobabilities thatadduptooneforeach\(i\). 6.2.4MaximumLikelihoodEstimation Estimationoftheparametersofthismodelbymaximumlikelihood proceedsbymaximizationofthemultinomiallikelihood(6.2) withtheprobabilities\(\pi_{ij}\)viewedasfunctionsofthe \(\alpha_j\)and\(\boldsymbol{\beta}_j\)parametersinEquation6.3. Thisusuallyrequiresnumericalprocedures, andFisherscoringorNewton-Raphsonoftenworkratherwell. Moststatisticalpackagesincludeamultinomiallogitprocedure. Intermsofourexample,fittingthequadraticmultinomiallogit modelofEquation6.4leadstoadevianceof20.5on8d.f. TheassociatedP-valueis0.009,sowehavesignificant lackoffit. Thequadraticageeffecthasanassociatedlikelihood-ratio \(\chi^2\)of500.6onfourd.f. (\(521.1-20.5=500.6\)and\(12-8=4\)), andishighlysignificant.Notethatwehaveaccountedfor 96%oftheassociationbetweenageandmethodchoice (\(500.6/521.1=0.96\))usingonlyfourparameters. Table6.2.ParameterEstimatesforMultinomialLogitModelFittedtoContraceptiveUseData ParameterContrast Ster.vs.NoneOthervs.None Constant-12.62-4.552 Linear0.70970.2641 Quadratic-0.009733-0.004758 Table6.2 showstheparameterestimatesforthetwomultinomiallogitequations. Iusedthesevaluestocalculatefittedlogitsforeachage from17.5to47.5,andplottedthesetogetherwiththeempirical logitsinFigure6.2. Thefiguresuggeststhatthelackoffit, thoughsignificant,isnotaseriousproblem, exceptpossiblyforthe15–19agegroup,wherewe overestimatetheprobabilityofsterilization. Underthesecircumstances,Iwouldprobablystickwiththequadratic modelbecauseitdoesareasonablejobusingveryfewparameters. However,Iurgeyoutogotheextramileandtryacubicterm. Themodelshouldpassthegoodnessoffittest.Arethefitted valuesreasonable? 6.2.5TheEquivalentLog-LinearModel* Multinomiallogitmodelsmayalsobefitbymaximumlikelihood workingwithanequivalentlog-linearmodelandthePoisson likelihood.(Thissectionwillonlybeofinteresttoreaders interestedintheequivalencebetweenthesemodelsandmay beomittedatfirstreading.) Specifically,wetreattherandomcounts\(Y_{ij}\) asPoissonrandomvariableswithmeans\(\mu_{ij}\) satisfyingthefollowinglog-linearmodel \[\tag{6.6}\log\mu_{ij}=\eta+\theta_i+\alpha^*_j+\boldsymbol{x}_i'\boldsymbol{\beta}^*_j,\] wheretheparameterssatisfytheusualconstraintsforidentifiability. Therearethreeimportantfeaturesofthismodel: First,themodelincludesaseparateparameter\(\theta_i\) foreachmultinomialobservation,i.e.eachindividualor group.Thisassuresexactreproductionofthemultinomial denominators\(n_{i}\).Notethatthesedenominatorsare fixedknownquantitiesinthemultinomiallikelihood, butaretreatedasrandominthePoissonlikelihood. Makingsurewegetthemrightmakestheissue ofconditioningmoot. Second,themodelincludesaseparateparameter\(\alpha^*_j\) foreachresponsecategory.Thisallowsthecountstovary byresponsecategory,permittingnon-uniformmargins. Third,themodelusesinteractionterms\(\boldsymbol{x}_i'\boldsymbol{\beta}^*_j\)to representtheeffectsofthecovariates\(\boldsymbol{x}_i\)onthe log-oddsofresponse\(j\). Onceagainwehavea‘step-up’situation, wheremaineffectsinalogisticmodelbecomeinteractions intheequivalentlog-linearmodel. Thelog-oddsthatobservation\(i\)willfallinresponsecategory\(j\) relativetothelastresponsecategory\(J\)canbecalculatedfrom Equation6.6as \[\tag{6.7}\log(\mu_{ij}/\mu_{iJ})=(\alpha^*_j-\alpha^*_J)+ \boldsymbol{x}_i'(\boldsymbol{\beta}^*_j-\boldsymbol{\beta}^*_J).\] Thisequationisidenticalto themultinomiallogitEquation6.3 with\(\alpha_j=\alpha^*_j-\alpha^*_J\) and\(\boldsymbol{\beta}_j=\boldsymbol{\beta}^*_j-\boldsymbol{\beta}^*_J\). Thus,theparametersinthemultinomiallogitmodel maybeobtainedasdifferencesbetweentheparametersinthe correspondinglog-linearmodel. Notethatthe\(\theta_i\)cancelout,andtherestrictions neededforidentification,namely\(\eta_{iJ}=0\),aresatisfied automatically. Intermsofourexample,wecantreatthecountsintheoriginal \(7\times3\)tableas21independentPoissonobservations, andfitalog-linearmodelincludingthemaineffectofage (treatedasafactor),themaineffectofcontraceptiveuse (treatedasafactor)andtheinteractionsbetweencontraceptive use(afactor)andthelinearandquadraticcomponentsofage: \[\tag{6.8}\log\mu_{ij}=\eta+\theta_i+\alpha^*_j+\beta^*_ja_i +\gamma^*_ja_i^2\] Inpracticaltermsthisrequiresincludingsixdummyvariables representingtheagegroups,twodummyvariablesrepresenting themethodchoicecategories,andatotaloffourinteractionterms, obtainedastheproductsofthemethodchoicedummiesbythe mid-point\(a_i\)andthesquareofthemid-point\(a_i^2\)ofeach agegroup.Detailsareleftasanexercise. (ButseetheStatanotes.) Mathrenderedby