How can I draw and project a video from an HTML5 player ...

Here is an example code to setup a webGL object that can receive your video pixels. Hope its useful to you. Home Public Questions Tags Users Companies Collectives ExploreCollectives Teams StackOverflowforTeams –Startcollaboratingandsharingorganizationalknowledge. CreateafreeTeam WhyTeams? Teams CreatefreeTeam Collectives™onStackOverflow Findcentralized,trustedcontentandcollaboratearoundthetechnologiesyouusemost. Learnmore Teams Q&Aforwork Connectandshareknowledgewithinasinglelocationthatisstructuredandeasytosearch. Learnmore HowcanIdrawandprojectavideofromanHTML5playerontoa3DplaneinWebGL? AskQuestion Asked 1year,1monthago Modified 1year,1monthago Viewed 2ktimes 2 I'musingUnitytobuildmyapptoWebGL,andI'vefoundmyselfneedingtograbvideofromanHTMLplayeranddrawitonaplaneinthe3Dspace. IknowyoucancalldrawImage()onaCanvasRenderingContext2Dandpassinareferencetothevideoplayer,andthecurrentframewillbedrawntothecanvaswhenthefunction'srun. Theclosest3DequivalentI'vebeenabletofindforthatfunctionisWebGL2RenderingContext.texImage3D().However,Idon'tentirelyunderstandhowitworks,andwhenItriedtestingit,Ireceivedthefollowingexception:UncaughtDOMException:Theoperationisinsecure.Iusedmyown,localvideofile,soitcan'tbeCORS,butIdon'tknowwhatitis. YoucanseethetestprojectinthisGitHubrepo. Ifoundasimilarquestionhere,butunfortunately,theanswersshowhowtodrawwhatIassumearepre-loadedtextures.Idon'tknowhowtograbthosefromthevideoplayerandpassthem,orifthat'sfastenoughtobedoneeveryframe. Togivesomecontext,I'mtryingtoshowanHLSlive-streaminsidemyUnity/WebGLapp.Icoulddownloadthe.ts(MPEG-2TransportStream)videosegmentsandqueuethemupintoacoherentvideostream,butUnity'sbuilt-invideoplayerdoesn'tsupportthisformat. Asasolution,IthoughtImightgrabthevideoinanHTML5player(usinghls.jsifnecessary)andinjectthetextureintotheWebGLapp,usingJavaScript,everyframe. UnityletsyourunJavaScriptcodefrominsideitsC#scripts,sotimingprobablywon'tbeanissue,norgettingtheworldscale/locationofthetargetplane.IjustneedtowritetheJavaScriptfunctiontosomehowdrawthetexture. Hereismycurrentcode:
Capture

unity3dvideowebglhttp-live-streamingunity-webgl Share Improvethisquestion Follow editedJun18,2021at15:34 VC.One 13.5k44goldbadges2323silverbadges5252bronzebadges askedJun17,2021at12:03 verified_tinkerverified_tinker 50333silverbadges1212bronzebadges 4 Inthetestprojectcode,sinceyouputthehtmlandvideoinsamelocationyoushouldsetyour"source"assrc="test.mp4"notassrc="./test.mp4".Itworkedformeandyourcodeisfine(testedonChrome). – VC.One Jun17,2021at14:51 1 @VC.OneWeird.Ichangeditasyousaid,butitstilldoesn'twork.Chromegivesamoreverboseerrormessage:DOMException:Failedtoexecute'texImage3D'on'WebGL2RenderingContext':Thevideoelementcontainscross-origindata,andmaynotbeloaded.Butitobviouslydoesn't!Edit:Hostingitonlocalhostfixedthatissue.Stilldoesn'twork,though:(index):30WebGL:INVALID_OPERATION:texImage3D:notextureboundtotargetYoudidn'tgetthisissue,either? – verified_tinker Jun18,2021at7:12 Yeahmybad.Iwassleepyandrememberchangingthesource.IthinkIsawthevideotagoutput&noGPUcompiler'salert/pop-up&sleepilyassumed"Noproblemhere,casesolved".Sorry.I'vemadeituptoyouwithacomplete&testableexample(editedfromoneofmyprojects) – VC.One Jun18,2021at15:31 PPS:TestcodeinlocalhostoruseonlineMP4linktoavoidsecurityissueswhendrawing. – VC.One Jun18,2021at16:29 Addacomment  |  1Answer 1 Sortedby: Resettodefault Highestscore(default) Trending(recentvotescountmore) Datemodified(newestfirst) Datecreated(oldestfirst) 1 HereisanexamplecodetosetupawebGLobjectthatcanreceiveyourvideopixels.Hopeitsusefultoyou. Basicallyyoucreateabox/rectangleshapeusingtwotrianglesandthenprojectthevideoontothatrectangle. 0-------1 || 3-------2 //#twosetsof...connected3-pointsofatriangle varvertexIndices=[0,1,2,0,2,3,]; TheexamplecodebelowalsocreatessomerequiredGPUshadersandprograms.Experimentwithit IfanycodersjustwanttodoGPUpixeleffectsthenwriteyoureffectscodeintheFragmentshader. (seecodepartat://#exampleofbasiccoloureffect).
Capture

// //#thesetwovarswillaccess varyingmediumpvec2vDirection; uniformsampler2DuSampler; voidmain(void) { //#getcurrentvideopixel'scolor(noFOR-loopsneededlikeinJSCanvas) gl_FragColor=texture2D(uSampler,vec2(vDirection.x*0.5+0.5,vDirection.y*0.5+0.5)); /* //#exampleofbasiccoloureffect gl_FragColor.r=(gl_FragColor.r*1.15); gl_FragColor.g=(gl_FragColor.g*0.8); gl_FragColor.b=(gl_FragColor.b*0.45); */ } attributemediumpvec2aVertexPosition; varyingmediumpvec2vDirection; voidmain(void) { gl_Position=vec4(aVertexPosition,1.0,1.0)*2.0; vDirection=aVertexPosition; }

Share Improvethisanswer Follow answeredJun18,2021at15:21 VC.OneVC.One 13.5k44goldbadges2323silverbadges5252bronzebadges 4 Wow,that'sawesome.AlotmorethanIexpected.Howwouldthistranslatetoprojectinginto3Dspace?I'vebeenmessingaroundwiththepositionsvariable—addingathirddimension—andadjustingtheshadercodeaccordingly,butallthatdidwasdraw1triangleinsteadof2.Also,online130,didyouperhapsmeantowriteuniforms[uniformName]insteadofattributes[uniformName]? – verified_tinker Jun21,2021at7:35 Doyoumean,forexample,youwantthevideoprojectedontoacube?I'venevertriedthat.Myprojectjustuseda2DboxbecauseIwasdoingheavypixeleffectsonavideo(inCanvasittook4secondspereachframe,butonGPUit'ssmoothplayback).Seeifthistutoriallinkgivesyousomehintsabouttexturingin3Dspace.Foranon-cubeshapeseetheirothertutorial.PS:Explain"projectinginto3Dspace"andI'llseewhatcanbetried. – VC.One Jun21,2021at14:22 Thanks,I'llcheckthemout.AsIsaid,IhaveanexistingWebGLappexportedbyUnity,thegameengine.ButUnity'svideoplayerdoesn'tsupportthekindoffiletypeIneed,soIthoughtImightusethebrowser'splayertodothat,grabthetexturedata,andpassittotheappsomehow.IfiguredImightcreateaquadintheapp,plasteredonawall,fromwithintheJScodeandstreamthevideotexturetoit.Or,maybe,ifpossible,Icouldmaybecreateanaccessiblefield/variableintheappmemory,ifWebGLsupportsthat,andwritetoitfromthebrowserandreadfromtheapp/game. – verified_tinker Jun21,2021at14:42 "IfiguredImightcreateaquadintheapp,plasteredonawall,fromwithintheJScodeandstreamthevideotexturetoit."Betterexplanation:imagineaTVmountedonthewall,ina3Droom.IneedtoplaythevideoonthatTV.Theroomalreadyexists,asdoestheTV.ButIneedtousethebrowser'splayertomakesenseofthedatainthevideofile,andthensomehowstreamthetexture,everyframeofthevideo,tothedisplayontheTV.Sincetheroomis3D,thedisplay—thequad—needstohaveaworld-spacelocation,scale,andorientation. – verified_tinker Jun21,2021at14:54 Addacomment  |  YourAnswer ThanksforcontributingananswertoStackOverflow!Pleasebesuretoanswerthequestion.Providedetailsandshareyourresearch!Butavoid…Askingforhelp,clarification,orrespondingtootheranswers.Makingstatementsbasedonopinion;backthemupwithreferencesorpersonalexperience.Tolearnmore,seeourtipsonwritinggreatanswers. Draftsaved Draftdiscarded Signuporlogin SignupusingGoogle SignupusingFacebook SignupusingEmailandPassword Submit Postasaguest Name Email Required,butnevershown PostYourAnswer Discard Byclicking“PostYourAnswer”,youagreetoourtermsofservice,privacypolicyandcookiepolicy Nottheansweryou'relookingfor?Browseotherquestionstaggedunity3dvideowebglhttp-live-streamingunity-webgloraskyourownquestion. TheOverflowBlog Codecompletionisn’tmagic;itjustfeelsthatway(Ep.464) HowAPIscantakethepainoutoflegacysystemheadaches(Ep.465) FeaturedonMeta AnnouncingtheStacksEditorBetarelease! Trending:Anewanswersortingoption The[options]tagisbeingburninated Linked 12 DrawinganimageusingWebGL 1 three.jsvideotexturecolorissue Related 1 HowdoIAutoplay