Enterprise-Wide Model Risk Management for Deposit-Taking ...

ICAAP – Guideline E-19 provides OSFI's expectations for institutions around internal capital adequacy assessment processes. Skiptomaincontent Skiptosecondarymenu Enterprise-WideModelRiskManagementforDeposit-TakingInstitutions PageContent​​AlternativeFormats PDF,497KBAccompanyingDocuments Letter GuidelineImpactAnalysisStatement ​​​DocumentProperties TypeofPublication:Guideline Category:SoundBusinessandFinancialPractices No:E-23 Date:September2017ThisGuidelineoutlinesOSFI'sexpectationsaroundinstitutions'establishmentofsoundpoliciesandpracticesforanenterprise-widemodelriskmanagementframework.Itappliestobanks,bankholdingcompanies,federallyregulatedtrustandloancompaniesandcooperativeretailassociations,andcollectivelyreferredtoas'institutions'.TableofContents 1.Introduction 2.Definitions 3.ScopeandKeycharacteristics 4.ModelRiskFramework 4.1Modelriskmateriality 5.Themodelmanagementcycle 5.1Rationaleformodeling 5.2Modeldevelopment 5.3Independentreview(vetting) 5.4Approval 5.5Ongoingmonitoringandreview(validation) 5.6Modificationsanddecommission 6.Vendorproducts 7.ModelsofForeignBanksSubsidiaries 8.InternalAudit 9.Modelinventory AppendixA.ModelriskmanagementrequirementsreferencedinOSFIGuidelines1.IntroductionThisGuidelineoutlinesOSFI'sexpectationsfortheestablishmentofanenterprise-widemodelriskmanagementframeworkatinstitutions.Takinganenterprise-wideviewofriskimpliesthattheseprinciplesshouldbeappropriatelyappliedacrosstheentirespectrumFootnote1ofmodelsusedbyinstitutionsforriskmanagementpurposes.Itisaninstitution'sresponsibilitytodevelopaconsistentsetofpoliciesand/orproceduresinordertoidentify,assess,manageandcontroltherisksinherentinanymodel–asdefinedinSection2Footnote2.OSFIrecognizesthatlargecomplexinstitutionswithinternalmodelsapprovedforregulatorycapitalpurposesalreadyhavethenecessaryriskcontrolinfrastructureinplace,whereasothersmightapplycontrolsonlyinmateriallyrelevantareas.Asaresult,thisGuidelineshouldbeinterpretedinthecontextofaproportionalityprinciplewherebyapplicabilityiscommensuratewiththenature,size,complexity,andriskprofileoftheinstitution.OSFIwilldistinguishbetweeninternalmodelsapprovedinstitutions(IMAIs)andotherstandardizedinstitutions(SIs).ThedividinglinewillbecontingentonwhetherornottheinstitutionhasreceivedOSFIapprovaltouseaninternalmodelforregulatorycapitalpurposes.IfapprovalhasbeengrantedthentheinstitutionisconsideredtobeamongtheIMAIcohort;otherwiseitwillbetreatedasanSI. IMAIsshouldcomplywithallcomponentsofthisGuidelineincludingrelevantsectionsofAppendixA,whichprovidesreferencestoadditionalmodelriskmanagementrequirementsforcreditrisk,operationalrisk,marketriskand'PillarIIcomponents'(i.e.,InternalCapitalAdequacyAssessmentProcesses(ICAAP)andinterestrateriskinthebankingbook(IRRBB))andMarginRequirements. SIsshouldstrivetocomplywiththisGuidelinebutOSFI'sminimumexpectationsarethattheinstitution:ProvideaninventorytoOSFIofallmodels(notjustcapitalmodels)thatareinuse(asdescribedinSection9),Identifyandassessthemostmaterialmodelsfromamodelriskperspective(asdescribedinSection4.1)ComplywithgovernanceandcontrolrequirementsnotedinAppendixAwithrespecttothePillarIIcomponentsandmarginrequirements(referencedinSection5.5butbroadlyapplicabletoallfacetsofSection5andtheuseofvendormodelsinSection6.).2.Definitions Model–Amodelgenerallyreferstoamethodology,system,and/orapproachthatappliestheoreticaland(expert)judgmentalassumptionsandstatisticaltechniquestoprocessinputdatainordertogeneratequantitativeestimates.Amodelhasthreedistinctcomponents:i)adatainputcomponentthatmayalsoincluderelevantassumptions;ii)aprocessingcomponentthattranslatestheinputsintoestimates;andiii)aresultcomponentthatpresentstheseestimatesinaformatthatisusefulandmeaningfultobusinesslinesandcontrolfunctions. Modelrisk–Theriskofadversefinancial(e.g.,capital,losses,revenue)andreputationalconsequencesarisingfromthedesign,development,implementationand/oruseofamodel.Itcanoriginatefrom,amongotherthings,inappropriatespecification;incorrectparameterestimates;flawedhypothesesand/orassumptions;mathematicalcomputationerrors;inaccurate,inappropriateorincompletedata;inappropriate,improperorunintendedusage;andinadequatemonitoringand/orcontrols. Modeluser–Theunit(s)/individual(s)thatreliesonthemodel'soutputsasabasisformakingbusinessdecisions.Whilemodelusersmaybeinvolvedintheearlystagesofmodeldevelopmentandongoingmonitoringactivities,thisinvolvementisnosubstitutionforindependentandobjectivereview. Modeldeveloper–Theunit(s)/individual(s)responsiblefordesigning,developing,evaluatinganddocumentingmodelswhichmayalsoperformongoingmonitoringandoutcomesanalysisaswellasperiodicreassessmentonceamodelisinuse. Modelowner–Theunit(s)/individual(s)responsibleforthemodelselection,coordinatingmodeldevelopment,initialtesting,ongoingmonitoring,outcomesanalysis,administeringchangesanddocumentation.Themodelownercouldalsobethemodeldeveloperoruser. Modelreviewer–Theindependentunit(s)responsibleformodelvetting,validation,Footnote3,Footnote4andreportingitsfindingsandrecommendationstothemodelapprover.Otherresponsibilitiesmightincludeprovidingthemodeldeveloperanduserwithguidanceontheappropriatenessofmodelsfordefinedpurpose. Modelapprover–Theindividual(s)and/orcommittee(s)responsibleforassessingthemodelreviewer'sfindingsandrecommendationsandapprovingtheuseand/orlimitationofuseofanynewmodelorchangestopre-existingmodels.Dependingonthesizeandcomplexityoftheinstitution,alongwiththematerialityofthemodelbeingreviewed,itmaybeacceptablefortherolesofmodelreviewerandapprovertobecombinedaslongasthereisnopotentialconflictofinterestandindependenceismaintained.Forthepurposesofthisguideline,theterms'modelriskcommittee'and'modelapprover'areusedinterchangeably.3.ScopeandKeyCharacteristicsThismodelriskguidelinetakesanenterprise-wideviewthatdescribescommonpoliciesandprocessesapplicabletoanymodelthatcouldmateriallyimpacttheriskprofileofaninstitution.Atminimum,thisincludesallmodelsapprovedforuseinthecalculationofinputstoregulatorycapitalmodelsusedforinstitutions'internalassessmentsofrisk.Institutionsshouldhavemodelriskmanagementframeworksthatexhibiteachofthefollowingkeycharacteristics,whichareelaboratedoninthesubsequentsections:Appropriateandcommensurategovernancesystemsovermodelusage.Modelmaterialityclassificationsandlimitations,whereappropriate,overtheuseofindividualmodels.Policiesandprocessesaroundmodelselectionanddevelopment.Independentvettingandongoingvalidation/reviewprocessesthatcontinuallyassessthemodel'sperformanceandsuitability.Footnote5Changecontrolprocessesgoverningeachstageofthemodel'slifecycle.Internalauditfunctionstoindependentlyassessthemodelriskmanagementgovernanceandcomplianceframework.Amodelinventorythatcataloguesthetype,classificationandperformanceofallmodelsinuse,orthathavebeendeveloped,andapprovedforuse,orrecentlydecommissionedthatcouldactasabenchmarkornecessarysubstituteforamodelinuse.4.ModelRiskFrameworkGiventheimportanceofmodels,thegovernancestructuresurroundingtheiruseshouldbealigned,asappropriate,withinaninstitution'sbroadercorporategovernanceframework.Footnote6Tothisend,modelriskmanagementbeginswithasoundgovernanceframeworkconsistingofpoliciesandproceduresthatidentifyallrelevantstakeholdersalongwiththenecessaryprocessesandcontrolstoensurecompliancewiththisguideline.Thesepoliciesand/orproceduresshouldencompasseachstageofamodel'slifecycle,including:modeldevelopment,validation,approval,review,modificationanddecommission.Thesepoliciesand/orproceduresshouldidentifyrelevantmodelstakeholdersandarticulatetheirroles,responsibilitiesandauthoritiesinidentifying,assessing,managingandreportingmodelriskwithintheinstitution.Enterprise-widemodelriskmanagementpoliciesshouldbedevelopedandoperationalizedbySeniorManagement.Institutionsshouldclearlyspecifywhichindividualsareresponsibleforthispolicyanddescribetheirrequirementstodevelopandpromoteit.Ownershipofeachin-usemodelshouldbeclearlyassignedtoanindividualand/orteam,whichmaybeeitherabusinesslineorhousedwithinariskmanagementfunction.Institutionsshoulddevelopaprocessthatenablesamodelownertomanagethemodel'sevolutionfrominceptiontodecommission,inanincreasinglyrigorousmannercommensuratewithitsmateriality.Modelownersshouldberesponsibleformaintainingthoroughdocumentation–thatiseasilyaccessibletoallmodelstakeholders–ateachstageofamodel'slifecycle.Thisdocumentationshouldbeitemizedinthemodelinventorymaintainedbytheinstitution(seeexampleinSection9).ForIMAIs,anindependentFootnote7modelreviewershouldberesponsiblefortheinitialmodelvettingandongoingvalidation.Inaddition,thereviewermightproviderecommendationsonapprovals,enhancements,andanyotherlimitationsonusage.Themodelreviewershouldhavetherequisitequantitativeskillstoconductamodelreviewbut,ideally,thedemonstratedexpertiseinthebusinessareaforwhichthemodelisbeingdesigned.Duringpre-approvalvettingorpost-approvalvalidationprocesses,themodelreviewercanprovidefeedbacktothemodelownerbutitshouldseparatelyreportitsfindingsandrecommendationsdirectlytoamodelapproverfunction.ForSIsthemodelreviewercouldalsoactasthemodelapproverprovideditremainsindependentfromthemodelowners/users.Inordertoensureeffectivecontrolovermodelriskitisimportantthatthegovernancestructurevestsinternalapprovalandoversightauthorityprimarilywithpartieswhoareindependentfromindividualswithadirectstake(suchasrevenuegeneratingfunctionsorbusinesslinemanagement)inthemodel'sapproval.4.1ModelriskmaterialitySeniorManagementshouldimplementanappropriatemodelriskmaterialityclassificationschemeapplicabletotherelevantmodels.Itsdesignandapprovalshouldbeintegratedwiththegovernancestructureformodelapproval.Sizeandcomplexityofmodelinventoriesmayrequire,asappropriate,separategovernancestructures.ForIMAIs,thisclassificationschemeshouldbeconsistentwithmodelmodificationmaterialitydefinitionsasdescribedinSection5.6.Ideally,aninstitutionshoulddesignasystemthatiscapableofrankingthelevelofriskposedbyeachofthemodelsused,whichprovidesthebasisforprioritizingmodelreviewsandschedulingongoingvalidationwork.Dependingonthesophisticationoftheinstitution,thissystemshouldincludebothquantitativeandqualitativeaspects.Footnote8ForSIs,modelriskmaterialityassessmentsshould,ataminimum,identifymaterialmodelsthatposethemostsignificantriskstotheinstitution.Modelriskmaterialityassessmentsshouldbeperiodicallyreviewedbyallinstitutionsandupdatedasappropriatebasedonexperience.Inthelattercase,institutionsshouldestablishtriggersthatmandatethere-assessmentofamodel'smaterialityand/ortheimpositionoflimitationsonusage.Footnote95.ThemodelmanagementcycleIncreasinglysophisticatedinstitutionsshouldbestrivingtowardsaconsistententerprise-wideprocessforchoosing,developing,reviewing,approvingandmonitoringeachofitsmaterialmodels.OSFIbelievesthisresemblesalifecycleprocess(i.e.,aperpetualactivitythatiscontinuallyrefurbishedandupdatedasthemodelevolveswiththepassageoftime)thatconsidersaseriesofphasessuchasi)therationaleformodeling;ii)modeldevelopment;iii)independentreview(vetting);iv)modelapproval;v)ongoingmonitoringandreview(validation);andvi)modification/decommission.Institutionsshouldensurethateachstageofamodel'slifecycleisaddressedbythemodelriskpolicyandadequatelydocumented.OSFIexpectsIMAIstoadheretoallelementsofthemodellife-cyclewhileSIsmustuseappropriatemeasures,assuggestedinSection5.5,todeterminetheirmostmaterialmodelsandbroadlyapply,whererelevant,toPillarIIcomponentsandmarginrequirements.Additionalconsiderationsthatshouldbetakenintoaccountineachphaseareprovidedbelow.5.1RationaleformodelingPriortomodeldevelopment,therelevantfirstlineofdefencebusinessarea(e.g.,modelusers)shouldidentifyaneconomicorbusinessrationalefordevelopinganewmodeland/ortheneedtochangeanexistingmodel.Institutionsshouldensurethatmodelownersthathavebeenassignedtheproductiontaskhavethenecessarytrainingand/orexperienceintherelevantareasunderconsiderationformodelconstruction.Modelownersshouldidentifyandunderstandtheproposedpurposeofthemodelandensurethatmodellingchoicesaredocumentedandevidenceisprovidedonthesuitabilityoftheselectionfortheproposedpurpose.Thisproofshouldinclude,forexample,acomparisonwithothercandidatemodelswhereappropriate,includingonespreviouslyconsideredoralreadyusedforthesamepurpose.5.2ModeldevelopmentIMAIsshouldhavedevelopmentprocessesformodelownerstofollowoncethemodelchoicehasbeenmade.Theintentistoimplementamodelthatcanaccuratelyquantifythedesiredmeasuresandreportthembacktothemodelusers.Thedevelopmentprocessisafirstlineofdefenceactivityandshouldconsideritemssuchas:thedeterminationofsuitabledata,andofcriticalassumptionsandthequantificationofkeyparameters;thedeterminationofthemethodologyusedtoarriveatdesiredoutputsandpreliminarymeasuresofperformance;programmingofthenecessarycodeformeasurement;anddevisingtheformattingofoutputsinamannersothatmodeluserscaneffectivelymakeproperbusinessdecisionsandmodelownerscanmonitorongoingmodelperformance.Documentationisanecessaryingredientinthemodeldevelopmentprocess.Itensuresthatotherpartiescanunderstandthemodel,implementitandconstructsuitablebenchmarksforcomparison.Inaddition,itmakesthemodelriskmanagementprocessmoretransparenttothirdpartyreviewers.Finally,itensuresthepreservationofknowledgeattheinstitutionasmodelusersandownerschangeovertime,whichisessentialforbusinesscontinuity.Documentationrelatedtothemodeldevelopmentprocessshouldbecomprehensiveandaddressthemodellingtechniquesadopted,anyassumptionsandapproximationsemployed(includingjustificationsand/orreasonabilityassessmentsforallkeyassumptions,coveringbothjudgmentalandqualitativeaspects),thedatasourcesanddataproxiesutilized,andanyrelevantmodelweaknessesandlimitations.5.3Independentreview(vetting)Independentreviewisacriticalcomponentofthemodellifecycle.IMAIsshouldhaveindependentmodelvettingprocessesinplaceasasecondlineofdefencetocheckwhethermodelsaresoundandfitfortheirintendedpurpose.Independentreviewshouldinclude,ataminimum,thefollowingfeatures: Verificationandassessment:thisincludescheckingthatalldocumentationisuptodateandavailableforthirdpartyreview;reviewingthemodelowner'smodelselectiondecisionrelativetootherpossiblecandidates;and,evaluatingthethreecomponents–inputs,computationprocesses,andreportingprocessesFootnote10–ofthedevelopmentprocess. Secondaryreview:thisincludesanappraisalofconceptualsoundnessandmodelperformanceagainstcriteriaforsuccessthatisreflectiveofmodelpurposeandproductscope.Theprocessmayinvolveevaluationagainstalternativebenchmarkmodels,whereappropriate,thatassesstheaccuracyofthemodel;and,sensitivitytestinginordertoassessthemodel'spredictivecapacityoverarangeofassumptionstoidentifyweaknessesandlimitations.Theresultsfromthismodelreview/vettingprocessshouldbedocumented,madeavailabletoallmodelstakeholders,andshouldformthebasisforapprovalrecommendationsalongwithanyconditionsonusagethatreflectidentifiedmodelweaknessesandlimitations.Inthisprocess,themodelreviewershouldremainobjectiveandwellinformedbutshouldnotdirectorengageinmodeldevelopmentsoastoensuretheprincipleofindependenteffectivechallengeispreserved.Whereappropriate,internalauditmaybeinvolvedinthereviewofthevettingworkasathirdlineofdefenceandprovideapositiveopinionpriortoapproval.5.4ApprovalModelsusedforregulatorycapitalinputsorinternalriskassessmentandcontrolandrelatedvaluationsshouldnot,unlessanexceptionhasbeengrantedanddocumented,beapprovedforoperationalusewithoutfirstundergoinganindependentreview.IMAIsshouldhaveadedicatedmodelriskcommittee(s)oramodelapprover(s)forthepurposeofapprovingforusenewmodelsalongwithanymaterialmodelmodifications.Themodelreviewerisresponsibleforprovidingtothemodelriskcommitteeorapproveritsvetting,validationandreviewreportalongwithaninitialrecommendationonthemodelapprovalpetition,asapplicable.Ifthereareidentifiedweaknessesorlimitations,themodelcouldberecommendedforconditionalapprovalprovidedthatcompensatingmitigationsareinplace.Thatbeingsaid,conservatisminassumptionsshouldnotbeasubstituteforfundamentalanalysisandshouldbebalancedagainstmodelaccuracywhereappropriate.Forinstance,pricingandprovisioningmodelsshouldprioritizeaccuracy.Institutionsshouldhavepoliciesthatarticulatetheiruseofconservatisminmodelsand,whereappropriate,overlaysonmodeloutputs.5.5Ongoingmonitoringandreview(validation)Oncethemodelhasbeenapproved,ongoingmonitoringbecomesajointresponsibilityofmodelusers,ownersandvalidators.Modelowners,users,orwhereappropriate,otherfirstlinestakeholdershavetheinitialresponsibilityformonitoring.Modelsshouldbesubjecttoaperiodicreviewwithafrequencythatisconsistentwiththeirmodelriskmaterialityassessments.Thisisanimportantpartofmodelriskmitigationsince,withthepassageoftime,multipleaspects(e.g.,changestomarkets;regulations;theoreticaladvancements;andfinancialinstitutionpolicies)canaltertheinherentlevelofmodelrisk.Ataminimum,reviewsshouldoccurannuallyformodelsthatexhibitthehighestdegreeofmodelrisk(seeSection4.1).Alternativelyareviewmightbeinitiatedininstanceswheretherehasbeenamaterialchangeinamodel'sscope,assumptions,methodology,andexposureorexceptionhistory.Modelowners(andotherfirstlineprocessstakeholders)andvalidation(asasecondlineprocess)should,duringmodelreview,considerare-assessmentofthequalityofthemodeldesignanditsconstruction.Thisprocessshouldconsider,butisnotlimitedto,itemssuchas:reaffirmingthecompletenessofexistingdocumentation;revisitingtheassumptionsanddatachosenaswellastheeffectsofanymodifications;benchmarkinganalysis,andre-examininganynotedmodellimitationsordocumentedweaknesses.Formodels,whosedeficiencieshavethegreatestpotentialtogenerateimmediateandmateriallosses,bothIMAIsandSIs(inthecontextofPillarIIcomponentsandmarginrequirements)areexpectedtousevariousmeasures,suchasbacktesting,discriminatoryanalysis,stress-testing,sensitivityanalysis,etc.,intheirongoingvalidationandreviewprocess.Institutionsshouldhaveclearguidelinesfordeterminingamaximumtoleranceonperformanceexceptionswherebyoncethatthresholdisbreached,anexceptioneventisconsideredtohaveoccurred,whichshouldtriggeranescalationprocess.ExceptionsandEscalationsFormodelsthatposemateriallevelsofmodelrisk,institutionsshouldhavepoliciesandprocessesinplacetomanagemodelexceptions.Footnote11Further,institutionsshouldhaveescalationprocessesinplacesothatthemodelriskcommitteeand/orSeniorManagementarepromptlymadeawareofamodelexception.Policiesand/orproceduresshouldspecifynotificationandreportingresponsibilitiesofthemodelownerandreviewerinanexceptionevent.Uponescalation,theoversightauthorityshouldhavethepowertoimposerestrictionsonthemodel'susage.Institutionsshouldhaveanestablishedpolicydictatingcircumstancesthatmerittheremovalofthemodelorimposingconditionsthatlimitsmodelusage.Underlimitedcircumstances,andforalimitedtimeperiod,themodelcouldcontinuetobeusedprovidedthemodelownerhasadocumentedandapprovedplantoremedytheexceptionsituationwithclearlydefinedconstraintsonuse,proposedactionsandassessmentmilestonesintheinterim.Internalauditshouldmaintainanongoingreviewoftheexceptionandescalationprocessandperformancetoensureitisbeingconductedinamannerthatisconsistentwithestablishedpolicy.5.6ModificationsanddecommissionTheprocessaroundmodelmodificationsandthedeactivationofmodelsduetopoorperformanceorobsolescenceisanimportantcomponentofaninstitution'smodelriskpolicy.AllinstitutionsshouldmaintainaholisticprocessthatarticulateswhatconstitutesamaterialFootnote12modelmodification.Footnote13,Footnote14Whensuchamodificationisundertaken,institutionsshouldapplythesamelevelofrigourinvettingandvalidationasisinvolvedwithanewmodelapproval.Ifthemodelmodificationisnotconsideredtobematerial,themodelownershouldstilldocumentthescopeanddetailsofthechangealongwithanyimplicationsforthemodel'sperformance.Noindividualsshouldhavetheauthoritytochangeamodelormodelusewithoutre-approvalofthechangedmodeloruse,whichmaybebyasummaryprocessforimmaterialchanges.Institutionsshouldestablishaprocessformanaginganddocumentingmaterialmodelmodifications.Thisprocessshouldconsider,forexample:aseriesofcontrolsgoverningauthorizationstochangemodelcomponents;arecordofvalidationsign-offssincemodelinceptiononago-forwardbasis;andarecordofempiricaltestresultstoassesswhetherornotmodelresultshavechanged.Suchaprocessshouldidentifythepersonnelorauthoritythatcanchangeandormodifythemodel.Thechangecontrolfunctionandvalidationrecordpreventsadivergencebetweentheapprovedmodelandtheoneusedinoperation.Thisprovidesanefficientmechanismforprioritizingongoingvalidationwork,whetherex-anteorex-postdependingonmateriality,aftereventssuchassystemsupgrades,whichtendtoaffectnumerousmodelssimultaneously.Thedecommissioningofamodelshouldnotbeconsideredtheendofthemodelrisklifecycleasthereisanexpectationthatanewmodelwillreplacethedecommissionedone.Adecommissionedmodelcould,however,actasabenchmarkormightneedtobere-commissionedifthenewmodelfailstobeimplementedproperlyorperformuptominimumrisktolerances.Institutionsshouldhavepoliciesand/orproceduresinplacefordecommissioningmodels,whichincludesnotifyingrelevantstakeholdersoftheupcomingevent.Inaddition,thereshouldbetransitionalarrangementsavailabletogovernsituationswhenthereisatiminggapbetweentheinceptionofthenewmodelandexpirationoftheold.Institutionsshouldhaveapolicyforthelengthoftimeitwillcontinuetomaintaintheoldmodel'sinformationinitsmodelinventorysystemandarecordofwhichmodelitisreplacedby.6.VendorproductsInstitutionsmightwishtorelyonthird-partyvendorsourcesformodelsordata,whereitisunderstoodthisinformationmightbeproprietary.Asidefromoutsourcingthemodeldevelopmentphase,adoptingavendorproductdoesnoteliminatetheneedtoapplyasimilarprocessforvetting,approval,ongoingvalidation,decommissioningandoveralldocumentation,aswouldbeconductedforin-housedevelopedmodelsanddatasources.InstitutionsshouldhaveultimateaccountabilityforalloutsourcedactivitiesFootnote14andshouldseekaccessfromthevendortoadequatetechnicaldocumentationrelatedtothemodeltounderstandhowthemodelisdesigned,calibratedandoperating,aswouldbeexpectedforaninternallydevelopedmodel.IMAIsmustdemonstratetoOSFI'ssatisfactionthatthereareconditionssurroundingthevendor'sproprietaryintellectualpropertythatinhibittheiraccesstodocumentation.Institutionsshouldestablishpoliciesand/orprocedures,withclearlyspecifiedauthorizations,aroundtheselection,ongoingmonitoringandretentionofvendormodels.Institutionsshoulddevelopcontingencyplansformaterialmodelsthatconsidertheeventwherethevendorproductisinadequatelysupported.7.ModelsofForeignBankSubsidiariesOSFIexpectsallforeignbanksubsidiariestocomplywiththescoperequirementsestablishedinSection1.Thatis,allshouldmaintainaninventoryofmodels,identifyandassessmaterialsourcesofmodelrisk,andcomplywithPillarIIcomponentsandmarginrequirementsnotedinAppendixA.Ifaforeignbanksubsidiaryreliesonmodelswhichareapprovedforusebytheirparentinstitution,itmustdemonstratethat,wherematerial,suchmodelsarefitforintendedpurposewithintheirmodelriskmanagementprocesses.Thelevelofprocessrequiredshouldbeinterpretedinthecontextofaproportionalityprinciplewherebythelevelexpectediscommensuratewiththenature,size,complexity,andriskprofileoftheforeignbanksubsidiaryinCanada,subjecttotheminimumexpectationsnotedabove.Theconceptofmaterialityofamodelwilldifferwhenappliedtotheforeignbanksubsidiaryonastandalonebasis.Thisimpliesthatmaterialmodels,includingthosebeyondregulatorycapitalusage,mayrequireasmuchrigouratallstagesbeyondthemodeldevelopmentstageasmodelsdevelopedlocallyandin-house.Aforeignbanksubsidiaryshouldhaveaccesstotechnicaldocumentationfromitsparentinordertoassessandmanagemodelriskuniquetoitsriskprofile.Beforereceivingpermissiontousetheirparents'modelsforregulatorycapitalpurposes,IMAIforeignbanksubsidiariesshouldfirstdemonstratetoOSFIaminimumlevelofcompetenceandcompliancewiththerelevantrequirementsoutlinedinAppendixA.8.InternalAuditInternalaudit,asthethirdlineofdefence,shouldassesstheoveralleffectivenessandadequacyofthemodelriskpolicy,ingeneral,anddeterminecompliancebythevariousstakeholderswiththatpolicy.Thisassessmentshouldbeundertakenbyindividualsthatareindependentofmodeldevelopment,validationoruse.Forallinstitutions,internalauditshoulddetermine: Policyexistence:confirmtherearemodelapproval,modificationanddecommissionprocessesandthereisanadequateprocessaroundmodelriskmateriality;and,thatauthorizationsaroundthemodelchangecontrolprocessareclearlyspecified. Policyadherence:assesswhethervalidationworkconductedbymodelreviewersissufficientlyindependentandoccurringonschedule;andconfirmthattheexceptionandescalationrecordareconsistentwithstatedpoliciesand/orprocedures. Documentation:performacheckforconsistencyandcompletenessindocumentationandreportingincludingthemodelinventoryrecords.Institutionsareultimatelyresponsibletomanagemodelriskbutcan,whenappropriateandnecessary,useanindependentexternalserviceasaresourcewiththeexpertiseandobjectivitytoassessthemodelriskmanagementprocessincludingthereviewbyinternalaudit.Thismayincludeoutsourcingofvalidationworkandperformingaspectsoftheinternalauditfunctiononmodelriskassessment,providedtheinstitution'sgovernancecontrolsandprocessesareinplaceandeffective.9.ModelinventoryInstitutionsshouldmaintainacatalogueofmodelssinceinceptioninordertobeabletoidentify,understandandtracktheperformance,risksandlimitationsassociatedwitheachandaffirmthatamodelisusedforitsoriginalpurpose.Assuch,OSFIexpectsIMAIsandSIstomaintainanup-to-dateinventoryofallmodelsinuseandrecentlydecommissioned.Allinstitutionsshouldestablishalistofindividualswithintheinstitutionthathavethesoleauthoritytocontrolandmaintainacentralizedmodelinventory.ThemodelinventoryshouldbemadeavailableuponrequestbyOSFIandshouldhavethefollowingcomponentsonamodel-by-modelbasis,whereapplicable:Modelnameanddescriptionofkeyfeatures.Modelriskrankingandmaterialityassessment.Identificationofthemodelownerand/ordeveloper.Referencestothetypeandsourcesofdatausedbythemodel.Whichproductsandbusinesslinesthatthemodelisapprovedforuse.Referencestovettingandvalidationreportsincludinganitemizationofdeficienciesandlimitations.Dateofinception,approvalforuseandexceptionhistory.Detailedsummaryofmaterialmodelmodifications.Referencestooutcomesanalysis(e.g.,backtestingresults).Referencestointernalauditfindingsastheypertaintothemodel.AppendixA.ModelriskmanagementrequirementsreferencedinOSFIGuidelinesOSFIexpectsinstitutionstocomplywithmodelriskguidanceexpectationsreferencedinotherOSFIguidelines.ThisAppendixprovidesinstitutionswithasetofreferencestoOSFIguidelines,brokenoutintothosespecifictoparticularPillarIandPillarIIinternalcapitalmodelsandthoserelatedtomarginrequirementsandaccountingmodels.NotethatthePillarIIinternalcapitalmodelpertainstobothIMAIsandSIs.CreditriskInternalRatingsBasedapproaches(IRB)CapitalAdequacyRequirements(CAR)Chapter6ImplementationNote:ApprovalofRegulatoryCapitalModelsforDTIs(i.e.materialityconsiderations)ImplementationNote:RiskQuantificationatIRBinstitutionsImplementationNote:ValidatingRiskRatingSystemsatIRBBanks(i.e.validationissues)SupervisoryFormulaApproach(SFA)forsecuritisations(CARChapter7)CounterpartycreditriskandCreditValuationAdjustments(CVA)(CARChapter4–SettlementandCounterpartyRisk)Operationalrisk(CARChapter8)Generalrequirementforrigorousproceduresforoperationalriskmodeldevelopmentandindependentmodelvalidation(paragraph48)Requirementsfortheuseofinternaldata,andrequiredvalidation(paragraphs58,59,60,61)Requirementsfortheuseofexternaldata,andrequiredvalidation(paragraph62)Expectationsrelatedtotheuseofscenarioanalysis(paragraph63)Expectationsrelatedtotheuseofbusinessenvironmentandinternalcontrolfactors(paragraph64)Marketrisk(CARChapter9)Standardized–OptionsScenariobasedapproach(paragraph172statesthatinstitutionsusingthescenariomethodshouldmeettheappropriatequalitativestandardssetforthinthesectionontheinternalmodelsapproach.).VaRandstressedVaR–Section9.11.2summarizesminimumqualitativestandardsthatOSFIexpectsinstitutionstomeetbeforetheyarepermittedtouseamodels-basedapproach.Section9.11.3providesguidanceinspecifyingaminimumsetofriskfactorsforinternalmodels.Section9.11.4specifiestheminimumquantitativestandardsforinternalmodelsandstressedVaR.Section9.11.7describesOSFI'sexpectationsforstresstestingprogramsforinternalVaRmodels.Section9.11.8describestheexpectationsaroundvalidationprocessesfortheinternalVaRmodel.SpecificriskVaR–Section9.11.5.1describesminimumrequirementsofspecificriskVaRmodels.Section9.11.6specifiesthebacktestingrequirementsforspecificriskVaRmodels.Incrementalriskcharge(IRC)–Appendix9-9describestheIRCcharge.SectionII.BoutlinesthekeysupervisoryparametersforcomputingtheIRC.SectionIIIspecifiesvalidationrequirements.Comprehensiveriskcharge–Section9.11.5.2describestheminimummodelrequirementsforacomprehensiveriskmeasuremodelforcorrelationtradingportfolios.Appendix9-10providesstresstestingguidanceforcorrelationtradingportfoliocomprehensiveriskmeasures.Interestrateriskinthebankingbook–GuidelineB-12onInterestRateRiskManagementstatesthat"OSFIsupportstheprinciplesoutlinedintheBaselCommittee'sJuly2004document."Institutionsshouldrefertothisdocumentand,inparticular,Principles6to9onriskmeasurement,monitoringandcontrolfunctions,andPrinciple10oninternalcontrols.ICAAP–GuidelineE-19providesOSFI'sexpectationsforinstitutionsaroundinternalcapitaladequacyassessmentprocesses.KeyconsiderationspertainingtomodelrisksaredescribedthroughoutSectionsIthroughVIhoweverparticularattentionshouldbemadeto'PillarIIcomponents'whensuchrisksarequantifiedviamodels.Marginrequirementsfornon-centrallyclearedderivatives–GuidelineE-22providesexpectationaroundinstitutionsuseofinternalmodelsforcalculatinginitialmarginrequirements.ParticularattentionshouldbepaidtoSection3.2forinstitutionsusinganinternalmodelforinitialmargin.OSFI'sGuideline IFRS9FinancialInstrumentsandDisclosures–Section2Impairment Footnotes Footnote1Thisspectrumincludesregulatorycapitalmodels,internalriskmanagementmodels,valuation/pricingmodels(includingthoseusedforaccountingpurposes),businessdecision-makingmodelsforriskmanagement(suchascreditadjudicationandscoringmodels),andstresstestingmodels. Returntofootnote1Footnote2ForeignbankbranchesarenotinscopeforthisGuideline;howeverOSFIexpectsBranchManagementtobeaccountableforensuringthereareappropriatecontrolsovermodelrisk,wherematerial,asdescribedinOSFI's GuidelineE-4. Returntofootnote2Footnote3Theterms"validation"and"vetting"areoftenusedinterchangeably.However,forthepurposesofthisdocument,validationisdistinguishedfromvetting.Vettingisadiscreteactivity,occurringonlyatsomepre-definedeventortiming(e.g.,initialmodelapproval).Bycontrast,validationisacontinuousactivity(e.g.,ongoingmodelperformanceassessments).Thegeneralphrase"review"willbeusedinthedocumentwhereverthereisaneedtoaddressbothactivities. Returntofootnote3Footnote4IndependenceofthevettingandvalidationfunctionisexpectedamongIMAIs,whileSIscanhousethevettingandvalidationfunctionwithinanoverallriskmanagementunitand/orrelyonexternalauditorsasdescribedinSection7.Regardlessofthegovernancestructureusedbyaninstitution,OSFIexpectsthatanoverridingprincipleof'effectiveoversightovertheuseofmodels'bemaintained. Returntofootnote4Footnote5Ibid. Returntofootnote5Footnote6OSFI's CorporateGovernanceGuidelinearticulatesOSFI'sprinciplesandexpectationswithrespecttocorporategovernanceofinstitutions. Returntofootnote6Footnote7Independenceimpliesthatthemodelreviewerhasnostakeintheapprovalofthemodel,whichenhancesthecredibilityofthemodelriskcontrolparadigmviaeffectivechallengeonthemodel'sappropriateness. Returntofootnote7Footnote8Forinstance,institutionscouldconsiderquantitativefactorssuchasthesizeandgrowthoftheportfoliothatthemodelcoversinadditiontoitscapitalimpact(e.g.,VaR).Qualitativefactorssuchasmodelage,complexity,purposeandstrategicimportancemayalsobeconsidered,whererelevant,aslongasthisisdoneinaconsistentfashion. Returntofootnote8Footnote9Examplesoftriggersinclude,amongothers:changesinunderlyingbusinessenvironment;increasesinthesizeorscopeofabusinessline;deteriorationinmodelperformanceandmaterialmodelmodifications. Returntofootnote9Footnote10Theevaluationofinputscouldincludeanassessmentoftherationalebehindkeyassumptionchoicesanddataqualityvetting.Theevaluationofcomputationprocessescouldincludecheckingforcomputercodeerrorsandthequalityoftheprogrammingandtheory.TheevaluationofreportingprocessescouldincludeevaluatingthecommunicationofoutcomesanalysisandresultstoSeniorManagement. Returntofootnote10Footnote11Exceptionscanoccur,forexamplebutnotlimitedto,when:modelsnotapprovedforusagebytheappropriateoversightentityarebeingused;avalidatedmodelisusedoutsideitsintendedpurpose;amodelthatdisplayspersistentbreachofperformancemetricscontinuestobeused;orbacktestingsuggeststhemodelresultsareinconsistentwithactualoutcomes. Returntofootnote11Footnote12Forgreatercontextonthenotionofmodelmateriality,institutionscanrefertotheOSFI'sImplementationNote:ApprovalofRegulatoryCapitalModelsforDeposit-TakingInstitutionspg.13. Returntofootnote12Footnote13Modificationscouldinclude,butarenotlimitedto:theintroductionofanewdatasource;achangeinthetechnology/infrastructureusedtosupplythedataordetermineoutputs;achangeintheunderlyingmethodology;orachangeinthemodel'soperatingenvironment. Returntofootnote13Footnote14ThisisnecessaryforSIsinordertobeabletoidentifyandrankthematerialityofvariousmodelsinuse. Returntofootnote14Footnote15InstitutionsrelyingonvendormodelsshouldalsorefertoGuidelineB-10OutsourcingofBusinessActivities,FunctionsandProcesses. Returntofootnote15​ ModifiedDate: 2021-12-23 Secondarymenu RegulationandGuidance ActsandRegulations Guidance TableofGuidelines TableofAdvisories ApprovalsandPrecedents ApplicationandApprovalGuides RegulatoryandLegislativeAdvisories LegislativeRulings CapitalRulings NameRequest RiskAssessmentandIntervention SupervisoryPractices GuidetoIntervention TrustAgreements RegulatoryData FilingCorporateReturns FilingFinancialReturns ViewingFinancialData Anti-moneyLaunderingandCompliance Anti-moneyLaundering Anti-terrorismFinancing Sanctions Messages IndustryNotices WebTools WhoWeRegulate