Face Recognition | Electronic Frontier Foundation

Face recognition is a method of identifying or verifying the identity of an individual using their face. Face recognition systems can be used to identify ... Skiptomaincontent Emailupdatesonnews,actions, andeventsinyourarea. JoinEFFLists ElectronicFrontierFoundation Donate FaceRecognition PAGE STREET-LEVELSURVEILLANCE Street-LevelSurveillance FaceRecognition Facerecognitionisamethodofidentifyingorverifyingtheidentityofanindividualusingtheirface.Facerecognitionsystemscanbeusedtoidentifypeopleinphotos,video,orinreal-time.Lawenforcementmayalsousemobiledevicestoidentifypeopleduringpolicestops.  Butfacerecognitiondatacanbepronetoerror,whichcanimplicatepeopleforcrimestheyhaven’tcommitted.FacialrecognitionsoftwareisparticularlybadatrecognizingAfricanAmericansandotherethnicminorities,women,andyoungpeople,oftenmisidentifyingorfailingtoidentifythem,disparatelyimpactingcertaingroups. Additionally,facerecognitionhasbeenusedtotargetpeopleengaginginprotectedspeech.Inthenearfuture,facerecognitiontechnologywilllikelybecomemoreubiquitous.Itmaybeusedtotrackindividuals’movementsoutintheworldlikeautomatedlicenseplatereaderstrackvehiclesbyplatenumbers.Real-timefacerecognitionisalreadybeingusedinothercountriesandevenatsportingeventsintheUnitedStates.  HowFaceRecognitionWorks   Source:IowaDepartmentofTransportation Facerecognitionsystemsusecomputeralgorithmstopickoutspecific,distinctivedetailsaboutaperson’sface.Thesedetails,suchasdistancebetweentheeyesorshapeofthechin,arethenconvertedintoamathematicalrepresentationandcomparedtodataonotherfacescollectedinafacerecognitiondatabase.Thedataaboutaparticularfaceisoftencalledafacetemplateandisdistinctfromaphotographbecauseit’sdesignedtoonlyincludecertaindetailsthatcanbeusedtodistinguishonefacefromanother.  Somefacerecognitionsystems,insteadofpositivelyidentifyinganunknownperson,aredesignedtocalculateaprobabilitymatchscorebetweentheunknownpersonandspecificfacetemplatesstoredinthedatabase.Thesesystemswillofferupseveralpotentialmatches,rankedinorderoflikelihoodofcorrectidentification,insteadofjustreturningasingleresult.  Facerecognitionsystemsvaryintheirabilitytoidentifypeopleunderchallengingconditionssuchaspoorlighting,lowqualityimageresolution,andsuboptimalangleofview(suchasinaphotographtakenfromabovelookingdownonanunknownperson). Whenitcomestoerrors,therearetwokeyconceptstounderstand:  A“falsenegative”iswhenthefacerecognitionsystemfailstomatchaperson’sfacetoanimagethatis,infact,containedinadatabase.Inotherwords,thesystemwillerroneouslyreturnzeroresultsinresponsetoaquery. A“falsepositive”iswhenthefacerecognitionsystemdoesmatchaperson’sfacetoanimageinadatabase,butthatmatchisactuallyincorrect.Thisiswhenapoliceofficersubmitsanimageof“Joe,”butthesystemerroneouslytellstheofficerthatthephotoisof“Jack.”  Whenresearchingafacerecognitionsystem,itisimportanttolookcloselyatthe“falsepositive”rateandthe“falsenegative”rate,sincethereisalmostalwaysatrade-off.Forexample,ifyouareusingfacerecognitiontounlockyourphone,itisbetterifthesystemfailstoidentifyyouafewtimes(falsenegative)thanitisforthesystemtomisidentifyotherpeopleasyouandletsthosepeopleunlockyourphone(falsepositive).Iftheresultofamisidentificationisthataninnocentpersongoestojail(likeamisidentificationinamugshotdatabase),thenthesystemshouldbedesignedtohaveasfewfalsepositivesaspossible.  HowLawEnforcementUsesFaceRecognition  Source:ArizonaDepartmentofTransportation Lawenforcementagenciesareusingfacerecognitionmoreandmorefrequentlyinroutinepolicing.Policecollectmugshotsfromarresteesandcomparethemagainstlocal,state,andfederalfacerecognitiondatabases.Onceanarrestee’sphotohasbeentaken,themugshotwillliveoninoneormoredatabasestobescannedeverytimethepolicedoanothercriminalsearch.  Lawenforcementcanthenquerythesevastmugshotdatabasestoidentifypeopleinphotostakenfromsocialmedia,CCTV,trafficcameras,orevenphotographsthey’vetakenthemselvesinthefield.Facesmayalsobecomparedinreal-timeagainst“hotlists”ofpeoplesuspectedofillegalactivity.  Mobilefacerecognitionallowsofficerstousesmartphones,tabletsorotherportabledevicestotakeaphotoofadriverorpedestrianinthefieldandimmediatelycomparethatphotoagainstoneormorefacerecognitiondatabasestoattemptanidentification. Facerecognitionhasbeenusedinairports,atbordercrossings,andduringeventssuchastheOlympicGames.Facerecognitionmayalsobeusedinprivatespaceslikestoresandsportsstadiums,butdifferentrulesmayapplytoprivatesectorfacerecognition.  Supportingtheseusesoffacereconitionarescoresofdatabasesatthelocal,stateandfederallevel.Estimatesindicatethat25%ormoreofallstateandlocallawenforcementagenciesintheU.S.canrunfacerecognitionsearchesontheirowndatabasesorthoseofanotheragency. AccordingtoGoverningmagazine,asof2015,atleast39statesusedfacerecogntionsoftwarewiththeirDepartmentofMotorVehicles(DMV)databasestodetectfraud. TheWashingtonPostreportedin2013that26ofthesestatesallowlawenforcementtosearchorrequestsearchesofdriverlicensedatabases,howeveritislikelythisnumberhasincreasedovertime. Databasesarealsofoundatthelocallevel,andthesedatabasescanbeverylarge.Forexample,thePinellasCountySheriff’sOfficeinFloridamayhaveoneofthelargestlocalfaceanalysisdatabases.AccordingtoresearchfromGeorgetownUniversity,thedatabaseissearchedabout8,000timesamonthbymorethan240agencies.  Thefederalgovernmenthasseveralfacerecognitionsystems,butthedatabasemostrelevantforlawenforcementisFBI’sNextGenerationIdentificationdatabasewhichcontainsmorethan30-millionfacerecognitionrecords.FBIallowsstateandlocalagencies“lightsout”accesstothisdatabase,whichmeansnohumanatthefederallevelchecksupontheindividualsearches.Inturn,statesallowFBIaccesstotheirowncriminalfacerecognitiondatabases. FBIalsohasateamofemployeesdedicatedjusttofacerecognitionsearchescalledFacialAnalysis,ComparisonandEvaluation(“FACE”)Services.TheFBIcanaccessover400-millionnon-criminalphotosfromstateDMVsandtheStateDepartment,and16U.S.statesallowFACEaccesstodriver’slicenseandIDphotos.  GiventhelargenumberofDMVdatabasesusingfacerecognitionandthenumberofAmericanswhosephotosareintheStateDepartment’sdatabaseofpassportandU.S.visaholders,GeorgetownUniversityhasestimatedclosetohalfofallAmericanadultshavebeenenteredintoatleastoneifnotmorefacerecognitiondatabases.  WhoSellsFaceRecognition MorphoTrust,asubsidiaryofIdemia(formerlyknownasOT-MorphoorSafran),isoneofthelargestvendorsoffacerecognitionandotherbiometricidentificationtechnologyintheUnitedStates.IthasdesignedsystemsforstateDMVs,federalandstatelawenforcementagencies,bordercontrolandairports(includingTSAPreCheck),andthestatedepartment.Othercommonvendorsinclude3M,Cognitec,DataWorksPlus,DynamicImagingSystems,FaceFirst,andNECGlobal. ThreatsPosedByFaceRecognition Facerecognitiondataiseasyforlawenforcementtocollectandhardformembersofthepublictoavoid.Facesareinpublicallofthetime,butunlikepasswords,peoplecan’teasilychangetheirfaces.Weareseeingincreasedinformation-sharingamongagencies.Camerasaregettingmorepowerfulandtechnologyisrapidlyimproving. Facerecognitiondataisoftenderivedfrommugshotimages,whicharetakenuponarrest,beforeajudgeeverhasachancetodetermineguiltorinnocence.Mugshotphotosareoftenneverremovedfromthedatabase,evenifthearresteehasneverhadchargesbroughtagainstthem. Inspiteoffacerecognition’subiquityandtheimprovementintechnology,facerecognitiondataispronetoerror.Infact,theFBIadmittedinitsprivacyimpactassessmentthatitssystem“maynotbesufficientlyreliabletoaccuratelylocateotherphotosofthesameidentity,resultinginanincreasedpercentageofmisidentifications.”AlthoughtheFBIpurportsitssystemcanfindthetruecandidateinthetop50profiles85%ofthetime,that’sonlythecasewhenthetruecandidateexistsinthegallery.Ifthecandidateisnotinthegallery,itisquitepossiblethesystemwillstillproduceoneormorepotentialmatches,creatingfalsepositiveresults.Thesepeople—whoaren’tthecandidate—couldthenbecomesuspectsforcrimestheydidn’tcommit.Aninaccuratesystemlikethisshiftsthetraditionalburdenofproofawayfromthegovernmentandforcespeopletotrytoprovetheirinnocence. Facerecognitiongetsworseasthenumberofpeopleinthedatabaseincreases.Thisisbecausesomanypeopleintheworldlookalike.Asthelikelihoodofsimilarfacesincreases,matchingaccuracydecreases.  FacerecognitionsoftwareisespeciallybadatrecognizingAfricanAmericans.A2012study [.pdf]co-authoredbytheFBIshowedthataccuracyratesforAfricanAmericanswerelowerthanforotherdemographics.Facerecognitionsoftwarealsomisidentifiesotherethnicminorities,youngpeople,andwomenathigherrates.CriminaldatabasesincludeadisproportionatenumberofAfricanAmericans,Latinos,andimmigrants,dueinparttoraciallybiasedpolicepractices.Thereforetheuseoffacerecognitiontechnologyhasadisparateimpactonpeopleofcolor. Somearguethathumanbackupidentification(apersonwhoverifiesthecomputer’sidentification)cancounteractfalsepositives.However,researchshowsthat,ifpeoplelackspecializedtraining,theymakethewrongdecisionsaboutwhetheracandidatephotoisamatchabouthalfthetime.Unfortunately,fewsystemshavespecializedpersonnelreviewandnarrowdownpotentialmatches. Facerecognitioncanbeusedtotargetpeopleengaginginprotectedspeech.Forexample,duringprotestssurroundingthedeathofFreddieGray,theBaltimorePoliceDepartmentransocialmediaphotosthroughfacerecognitiontoidentifyprotestersandarrestthem.Ofthe52agenciesanalyzedinareportbyGeorgetownCenteronPrivacyandTechnology,onlyoneagency,theOhioBureauofCriminalInvestigation,hasafacerecognitionpolicyexpresslyprohibitingtheuseofthetechnologytotrackindividualsengagedinprotectedfreespeech.  Fewfacerecognitionsystemsareauditedformisuse.Of52agenciessurveyedbyGeorgetownthatacknowledgedusingfacerecognition,lessthan10%hadapubliclyavailableusepolicy.Onlytwoagencies(theSanFranciscoPoliceDepartmentandtheSeattleregion’sSouthSound911)restrictthepurchaseoftechnologytothosethatmeetcertainaccuracythresholds.Onlyone—MichiganStatePolice—providesdocumentationofitsauditprocess. TherearefewmeasuresinplacetoprotecteverydayAmericansfromthemisuseoffacerecognitiontechnology.Ingeneral,agenciesdonotrequirewarrants,andmanydonotevenrequirelawenforcementtosuspectsomeoneofcommittingacrimebeforeusingfacerecognitiontoidentifythem.  TheIllinoisBiometricInformationPrivacyActrequiresnoticeandconsentbeforetheprivateuseoffacerecognitiontech.However,thisonlyappliestocompaniesandnottolawenforcementagencies.  EFF'sWorkonFaceRecognition   %3Ciframe%20allowfullscreen%3D%22%22%20src%3D%22https%3A%2F%2Fwww.youtube-nocookie.com%2Fembed%2FK_2Uww_gZio%3Frel%3D0%26autoplay%3D1%26mute%3D1%22%20width%3D%22560%22%20height%3D%22315%22%20frameborder%3D%220%22%20allow%3D%22autoplay%22%3E%3C%2Fiframe%3E Privacyinfo. Thisembedwillservecontentfromyoutube-nocookie.com   Wesupportmeaningfulrestrictionsonfacerecognitionusebothbygovernmentandprivatecompanies.WetestifiedaboutfacerecognitiontechnologybeforetheSenateSubcommitteeonPrivacy,Technology,andtheLaw,aswellastheHouseCommitteeonOversightandGovernmentReformHearingonLawEnforcement’sUseofFacialRecognitionTechnology. WealsoparticipatedintheNTIAfacerecognitionmultistakeholderprocessbutwalkedout,alongwithotherNGOs,whencompaniescouldn’tcommittomeaningfulrestrictionsonfacerecognitionuse. Wehaveconsistentlyfiledpublicrecordsrequeststoobtainpreviouslysecretinformationonfacerecognitionsystems.WeevensuedtheFBIforaccesstoitsfacerecognitionrecords.  In2015,EFFandMuckRocklaunchedacrowdsourcingcampaigntorequestinformationonvariousmobilebiometrictechnologiesacquiredbylawenforcementaroundthecountry.Wefiledanamicusbrief,alongwiththeACLUofMinnesota, demandingthereleaseofemailsregardingtheHennepinCountySheriffOffice’sfacerecognitionprogramthatwererequestedbyonelocalparticipantintheproject. EFFLegalCases EFFv.U.S.DepartmentofJustice TonyWebsterv.HennepinCountyandHennepinCountySheriff'sOffice ForMoreInformation ThePerpetualLine-Up (GeorgetownLawCenteronPrivacyandTechnology) FaceRecognitionTechnology:FBIShouldBetterEnsurePrivacyandAccuracy (GovernmentAccountabilityOffice) CaliforniaCopsAreUsingTheseBiometricGadgetsintheField (EFF) FaceRecognitionPerformanceRoleofDemographicInformation (IEEE) PrivacyImpactAssessmentfortheFacialAnalysis,Comparison,andEvaluation(FACE)ServicesUnit (FBI) MostrecentlyupdatedOctober24,2017 Backtotop FollowEFF: Contact About Issues Updates Press Donate JavaScriptlicenseinformation