IRC Log Viewer » #firebreath » 2013-03-19

IRC Nick Time (GMT-7) Message
Aktau 11:03 Hey guys
Been researching a project I have to rescue. It's sort of a kiosk browser player that alternates between flash presentations, html5 presentations and h264 videos
Thing is, the (massively bought) hardware platform is insufficient for 1080p which now appears to be a hard requirement
So I got to work and compiled an mplayer that could use VAAPI and it plays wonderfully without noticeably CPU usage
The mplayer-gecko plugin seems to crash much more than I'd like
So I thought about making my own plugin
And before I start on this crazy ride, I would like to ask: just how stupid is this?
taxilian 11:03 relatively stupid =] but probably doable if you're really serious about it
it's not really a simple task, though
what platform?
Aktau 11:03 linux
Debian wheezy to be more specific
taxilian 11:03 ahh, that makes it a bit more stupid =]
Aktau 11:03 Browser could be either chrome, chromium or firefox. I get to choose
taxilian 11:03 I can't help you much
main problem there is drawing in a hardware accelerated manner
there are many who have done it
but I don't know how myselkf
Aktau 11:03 Would you know of any fire breath plugins that are 1) open source 2) hw accelerated 3) work on linux?
taxilian 11:03 AFAIK no such thing exists
though jshanab has been working on one
he doesn't seem to be here right now, though
Aktau 11:03 Ok
Maybe he'd be willing to share a little
Thanks for your insights taxilian, they're much appreciated
taxilian 11:03 you're welcome
jshanab 16:03 Aktau. I have been working on a plugin for this for quite some time. We have versions in the field but stability is an issue for large camera and plugin counts.
Aktau 16:03 jshanab: what do you mean with camera exactly?
I was referring more to hw-accelerated video display inside of any plugin supporting browser
jshanab 16:03 I use live555 to stream, or curl depending on source. I use libavcodec to decode and frome there I have tried SDL,SFML,GLFW(modified),straight DirectX and straight OPENGL
I do security camera video. 1 plugin per camera and that causes additional isses with accelrated video.
Onlinix and mac opengl is the way, for sure.
Aktau 17:03 Ah right I see what you mean
SDL, SFML and GLFW are all just context-creating toolkits
So how is it different from straight OpenGL
jshanab 17:03 My latest OpenGL back-end is actually more like my version for the iphone an android. Since I do live and recorded playback direct from camera and from restreamer/archiver, it gets interesting.
Aktau 17:03 You mean you talk to glx directly?
jshanab 17:03 gl In windows I use a few wgl functions to get the context. I am compileing a yuv shader and an ortho matrix so I can have accelerated color space conversion and scaling/zoom.
SDL works well on XP but I have event issues with all of them. Here is the jest. They all were written with the assumption of a single main thread in a desktop window. You CANNOT block the main thread on a browser, it is not yours and if you suceed the browser turns white and crashes. The window is handed to you (or a drawing context for windowless)
The window events come thru the browser framework. (Unless you allow people to do things like tear it off then the events are handled by the app!)
SDL chooses directX on windows and opengl elsewhere. SFML is nice c++ library. opengl only but I ran into crashed browser issue. SDL and SFML wer slow to respond to these bugs. (months)
I tried modifiing GLFW to allow for an embedded window and that seemed to wrok well on linux, and windows, but I did run into the same event bugs
My newest effort is just OpenGL. Firebreath creates a folder for each OS so I can easily have a version for each one.
And then there is chrome.....chrome crashes on scroll or resize, it rips the opengl context away. grrrrr
Firefox is by far the easiest and most stable
Aktau 17:03 Oh wow
That actually sounds like a minefield
And I was thinking of quickly getting some hardware accelerated display somewhere
Still, there must be some way that gecko-mplayer is doing it
I've tried it and when it doesn't crash the CPU usage is pretty low
jshanab 17:03 The actual HW acceleration part is simple
In my case it is complicated by all the features and the need for 16 videos not all from the same source. In retrospect a single canvas split and drawn on in a single plugin would be better for my case.
Get a context. Load,compile a vertex and a frament shader. take a decoded frame in yuv format and update the pixels on 3 textures; one for Y one for Y and one for Y. call swapbuffers and the shader will convert and display the RGB.
The single biggest trick is draw on the same thread that creates the context,textures etc. This means you have a check in the draw loop and if not initialized, initialize() All changes are flags set from the UI thread, I use boost atomics for them
OMG "one for Y one for U and one for V"
Shaders are super simple I use the same one on my iphone,android and windows PC (but I found a little massaging was needed to get ATI and NVIDIA to accept the same code, they are each a bit picky about different things http://iphonedevsdk.com/forum/iphone-sdk-development/99753-opengl-es-render-video-yuv-to-rgb-by-shaders.html
Aktau 17:03 I didn't even know the iPhone browsers took plugins
I would also require many different viewports to many different sources, unfortunately
taxilian 17:03 iphone browsers don't; he's using an app
jshanab 17:03 iPhone and android do not, but they have a UI and a way of calling C++ code, so it turns out an app can be quickly created in their gui tools and then connected up to a c++ plugin like code. Much Much Much easier and faster on iPhone than android, but replace the concept of a browser giving you a window with the UI toolkit and the javascript with a wrapper to their toolkit and it is similar
My iPhone code is mixed more. the android code is very much like my plugin code.
Soomeday when I have the time I would like to make skeletons of an iphone and android app that could be incorporated into FireBreath!
taxilian 17:03 that would be pretty cool
I could probably help with that
jshanab 17:03 The multiple viewports into many sources is really just render to a texture and put the texture anywhere you like. viewports in the traditional sense would be differnt views of the same sceen
I have just been dealt a new reality in work to employee ratio unfortunantly.
As a matter of fact. I wrote a test app for the iphone and an iphone guy wrote a gui around it. I just supply the two views, live and archive. Same with the android, I ported my test app and the plugin code and then he wrote a gui around it
Another project i have been thinking of writing in FireBreath is called flavor. Use librtmp to pull flash video from youtube and the like and then do NOTHING except display video. Simplest opengl only. No adds no redirects, etc. I do not know if that would render it useless but flash on 64 bit linux sucks
reichi 17:03 hmm
you can actually get pretty much any youtube video via http
jshanab 17:03 There are so many non youtube videos and I turned that feature on once and found even on youtube, it limits the selection.
reichi 18:03 that's just a questions of "magic" ;)
it's all there
the mobiles need it
jshanab 18:03 I really was thinking of it as the vehicle for the FireBreath OpenGL abstraction and the IPhone/Android modules
reichi 18:03 yeah
it hink it would be a great example
jshanab 18:03 I have kind of worked on all the pieces to make such a project fly
reichi 18:03 and also a cool tool
i'm just a little well... branded, when it comes to rtmp
jshanab 18:03 I have glgears running in html tables on 3 OS's hehe
reichi 18:03 :D
especially rtmpe...
jshanab 18:03 which is sorta secure
reichi 18:03 adobe likes DMCAing everything that's capable of doing it...
like rtmdump in binary form...
well
it "was" secure
it's pretty much completely reversed
jshanab 18:03 I am back and forth in front of the computer ( I am moving my drums from the basement to a spare room) I may be spotty on answers
reichi 18:03 I'l go to bed soon anyways ;)
jshanab 18:03 reichi so not in the US?
reichi 18:03 nope, Germany
(on the Border to Austria)
jshanab 18:03 Cool. Nice area. Been a few years since I was there
reichi 18:03 well
i'm coming over in september
doing the "California Circle" or whatever you may call it
LA, Grand Canyon, Vegas, Frisco (and what's between)
jshanab 18:03 I lived in CA up until 2 years ago. I do not know what the california circle is LOL