Solving Different Problems

The most common apples ‘n oranges-style tech comparison I hear these days takes place between the Adobe Flash Player and pick-your-favorite-dedicated-video-player program. “Why does Flash Player take more CPU to play a video than my favorite video player?” It’s a reasonable question and the answer can be distilled quite simply as, “The Flash Player solves a different problem than your favorite video player.”

This is the core thesis of coworker Tinic’s “On Performance” blog post from some time ago which addressed complaints comparing the performance of AJAX, SVG, and even static HTML to Flash. I’ve wanted to write this post for awhile but always hesitated since previous attempts at technical exposition largely fell on deaf ears. However, with the recent developments that minority browsers have made with supporting the HTML5 video tag, coupled with the recent announcements of such video sites as YouTube and Vimeo offering streams via these technologies, more people are recognizing that the issues surrounding video playback in the browser are in no way unique to the Flash Player. For those who wish to learn, here goes (for those who don’t, feel free to skip straight to the comment form and get your flame on).

Video flow

Here is the general flow that a dedicated media player has to worry about:

video-flow.png

The compressed video data traverses through the video decoder which outputs YUV data to be splatted on the screen.

Here is what the Flash Player needs to do in order to play back video:

flash-flow.png

The Flash Player has to do a little bit more. In addition to decoding the data, it has to convert YUV data to the RGB colorspace and combine the image with other Flash elements. Then it has to cooperate with another application (web browser) to present the video to the user.

So the dedicated media player solves a problem: Generally, it plays linear media files from start to finish while allowing user interaction in the form of random seeking along the timeline. That’s the most basic, trained monkey-type of labor in video playback. At most, the player might handle DVD menus.

Flash Player solves a different problem: It plays linear media files from start to finish while combining the video with a wide array of graphical and interactive elements (buttons, bitmaps, vector graphics, filters), as well as providing network, webcam, and microphone facilities, all programmable via a full-featured scripting language, and all easily accessible via a web browser using a plugin that most of the browsing population already has installed.

“But if I download the media file and play it in my dedicated media player, it takes less CPU.”

Perhaps, but that’s solving a different problem (playing a file using a dedicated, single-purpose application). The Flash Player works to solve the problem of making video accessible via the web browsing environment.

“But minority browsers don’t use as much CPU while playing HTML5 video tag data.”

Are you sure about that? As of this writing, browsers that support the video tag are likely to use a tremendous amount of CPU in order to play video. Why? It’s likely because they have to deal with the exact same performance penalties that Flash does; specifically, that YUV video data needs to be transformed into RGB data to be plopped in the browser window. This is why I thought this might be a good time to write this article– I have been seeing more and more comments in online forums observing that the video tag seems quite likely to peg CPUs.

“But at least with HTML5 video tag data, I can download the video and play it offline using a dedicated player.”

Try to keep up here. That’s not the problem that Flash Player is chartered to solve (same goes for HTML5 video). I know you’re smart enough to download videos and watch them offline, but try to think on a macro level. Try to think of non-tech-savvy people of your acquaintence. (Please tell me you know people who aren’t as tech savvy as you. If you don’t, get to know some in order to maintain proper perspective.) Have you ever tried to tutor such an individual (remotely) in downloading and installing a separate video player? I have and it rarely goes well.

But I digress…

What can be done to improve performance?

Flash Player has an impressive software H.264 decoder, competitive with anything else you’re likely to find out there. But there’s going to be that YUV->RGB conversion penalty, plus blending the other SWF elements on the video, and possibly scaling to a much larger rectangle. We’re pushing the software algorithms as hard as they will go. So what are the hardware options?

Dedicated decoding coprocessor

We’re beginning to see video decoding coprocessors packaged in computers, particularly low-end netbooks. In this situation, the green box labeled [Video decoder] in the diagrams above is replaced by a piece of hardware. Everything else about those diagrams remains the same (including Flash’s need to convert YUV to RGB). Flash Player 10.1 has support for Broadcom’s video decoding coprocessors (according to the release notes, the BCM70012 “LINK” chip). This is currently supported in Windows. Support for these chips in Mac and Linux would be nice but the drivers aren’t ready yet.

Video card acceleration

Assorted video cards from Intel, ATI and nVidia are also able to assist in video decoding as of Flash Player 10.1. Again, see the release notes for more detail on which models and drivers are required. Again, this is presently only available in Windows.

“Why aren’t these card supported in Linux? My free software media player support VA-API, VDPAU, and XvBA (the various hardware acceleration APIs).”

Again, say it with me: Flash Player solves a different problem.

In the above depiction of Flash Player’s workflow, the video card stands in for the green [Video decoder] box. The key point here is that the decoded video frames need to be accessible by the Player which needs to do its thing before the data can be presented to the user. As of this writing, none of these drivers in Linux allow retrieval of the decoded video data. Their counterpart Windows drivers do allow this which is why this feature is supported in Windows.

That’s for Linux. What about Mac? I’m not sure but my Mac colleagues have mentioned something about Apple not making their hardware decoding APIs available to applications (if the APIs exist at all, which I’m not sure they do).

Video card scaling

After Flash Player has decoded a video frame, overlaid everything on top and is ready to present it in fullscreen mode, the hardware can assist in doing the scaling to fullscreen. This is a feature that Flash Player has had for a few releases now. Having to scale an RGB bitmap from 400×300 -> 1920×1200 is something that can easily be pawned off on hardware assuming that the right APIs are present.

“Why doesn’t the Linux Flash Player use Xv for fullscreen scaling?”

We’re not really going through this again, are we? I hope I have cogently presented the reasons between this post and this other one.

Summary

In case it’s unclear, let me re-iterate: Flash Player is not the same as your standalone multimedia program. Flash Player solves a different set of problems than your standalone multimedia program. You might claim that your favorite multimedia program solves video playback problems to your satisfaction. However, the internet at large has unanimously declared that it appreciates the Flash Player’s solution to the video problem.

Regarding the myriad video acceleration APIs in Linux, we are watching developments closely and look forward to implementing support when the drivers support the features conducive to Flash Player’s operational model.

119 Responses to Solving Different Problems

  1. Anonymous says:

    Thank you. You can’t do anything about the video playback at the moment, fair enough.I’d like to hear about your progress on issues that is solvable, though. There’s massive slowdowns in fullscreen mode (tied to the mouse cursor update, I believe), up to several seconds sound lag for anything except video, random crashes, and more. These problems all disappear when running the Windows version of Flash in Wine, so they can’t be issues with Linux itself. Machinarium was unplayable in Linux due to this, and VVVVVV recently had to abandon its Linux port for apparently the same reasons.I do appreciate the work you’ve done on the Linux player so far, but it’s very frustrating that it’s still so far from usable.

  2. raylu says:

    Then perhaps Flash has solved the wrong problem…

  3. dixon says:

    Thanks for the blog post. I think you’ve covered the basic problems. I’m just curious:1) How come gnash is working with VAAPI?2) Are you contributing at least with ideas to VAAPI design?3) Windows flash player was performing much better since I can remember (also without some acceleration API). Is the linux flash codebase less optimized?

  4. jonathan says:

    Well played..

  5. Leif says:

    I would hazard a guess that the primary ‘problem’ typical users want to solve is just streaming video. Yes lots of games use Flash’s “buttons, bitmaps, vector graphics, filters” as you say. So hey, they should keep using Flash (or maybe migrate to HTML5 canvas, but I digress). But these “different problems” that Flash solves really aren’t the problems MOST users care about. If Flash wants to keep focussing on those cases while it’s market share for streaming video applications shifts to HTML5 video (and presumably open codecs) then keep on trucking.

  6. mike russell says:

    Why can’t flash player do its RGB-YUV on the GPU with a shader (I have done this, its quite easy) and its blending on the GPU? 2 OpenGL textures? Surely most of flash should be GPU accelerated? [ It’s YUV to RGB conversion, and we also need the data back from the GPU’s memory so we can do other stuff with it. -Mike M. ]

  7. Kipio says:

    This post starts off well enough. A little defensive, but perhaps that’s been warranted. By the end however, it is condescending and rude. The post seems to be in reaction to some ongoing conversation or dialog, but those of us who are just visiting this blog see only an employee of Adobe posting on a corporate blog in a rude and condescending way. [ If you think the post is rude, just wait until you read the responses. -Mike M. ] It is totally fair for an engineer working on a widely used product to explain publicly why that product doesn’t always perform at the level users would like. But that message should be combined with more information about what the manufacturer is doing to rectify the issue, even if it is not directly in their hands. (eg, explain to us how you’ve been working with video driver vendors to assure support for features that Flash needs in Linux.)Finally, you use “non tech-savvy people” as part of your plea for standalone players not being a solution. I agree with this, but I think it indicates a further need to be upfront about the path to solutions to these problems. The non tech-savvy people I know may not be able to install a standalone video player; nor do they understand or care why Flash video is slow. They just know that it is and that they don’t like it.

  8. Ernst says:

    Why doesn’t the Linux Flash Player use OpenGL for fullscreen scaling when using an open-source driver? That’s the case, if I remember correctly? And do you still do ps -a | grep compiz and disable OpenGL accelleration? I think it’s time to stop doing that and let opensource devs fix any bugs that may be left. [ Read about the override switch: http://blogs.adobe.com/penguin.swf/2008/05/flash_uses_the_gpu.html -Mike M ]

  9. mike russell says:

    According to Steven Warren from NVIDIA: The VDPAU presentation queue can render to X pixmaps.Maybe Adobe should talk to NVIDIA and get this sorted.

  10. Fran says:

    Summary: Linux sucks, coding for Linux sucks, apis on Linux sucks, Windows is perfect and Flash too of course … but you are not able to make a 64 bits version yet and instead of supporting the Linux community you cry and blame everyone else.You say “minority browsers” today, tag video hurds, isn’t?, let’s see what happens in the future.Oh!!! and take care about Silverlight, maybe Windows will sucks too in the end 😉

  11. Daniel says:

    This post came across as pretty smug and bitter. It’s good that you’re addressing the criticism, but the tone isn’t helping any.Also, I’m curious as to how much of CPU time is spent doing the linear YUV -> RGB conversion. I would have thought it would be pretty quick with SIMD.

  12. Anonymous says:

    Again, say it with me…

    I’ve been reading this blog since the beginning. I always thought you were a little condescending, but you seemed knowledgeable enough and I thought you were doing good work.However, you can say this with me: Eat shit.

  13. yves says:

    Minority browsers you say? Liek Firefox/Safari/Chrome/OperaThey together hold between 20 and 50% of the browser market share.Where I am from that is not a minority… :-9 [ The unimpeachable source of information — Wikipedia ( http://en.wikipedia.org/wiki/Usage_share_of_web_browsers ) — estimates IE at 59% as of last month. That makes everything else, combined, “minority”. -Mike M. ]

  14. Maik says:

    However, the internet at large has unanimously declared that it appreciates the Flash Player’s solution to the video problem.

    I believe you’re wrong about that.If I wanted to deploy video on the web today, I would use Flash. Granted.However, that does not in the least imply that I think Flash is a good way of doing video on the web. In fact, it sucks as a video player. A Lot. This is mostly because it isn’t a video player, even if it happens to be a piece of software that can play videos – you’re right about that part.What Flash has going for it is that it is, as of right now, the only reliable way to do video on the web in a way that is basically functional on all relevant platforms. Again, that doesn’t imply that it’s a good way, or that anyone thinks it’s a good way.I think “the internet at large” uses Flash for video for the exact same reason they still make their pages Internet Explorer-compatible: because it’s necessary to make stuff work, not because they love it. And I think they’ll stop the second a better alternative comes along, which the video element might become, but isn’t yet.

  15. metellius says:

    Nice and informative post.I’m very curious to hear if you have any data on how much the various hardware accelerated stages of the rendering actually makes a difference. How much time is spent on scaling vs yuv-rgb conversion? How much faster are they respectively once hardware takes over?And on a more futuristic note, what if there was a way to do drawing in YUV? If a large site like youtube enabled such a feature and required all of it’s interface and ads to be YUV-encoded, wouldn’t this remove the RGB-YUV encoding from the picture? Or am I missing a point here on how YUV works? [ Yeah, it doesn’t quite work like that. Hit up the Wikipedia links for YUV and RGB. -Mike M. ]

  16. Gautier says:

    So basically, playing video in flash is just plain stupid?

  17. Anonymous says:

    The jack-of-all-trades is the master of none

  18. Jeffrey W. Baker says:

    So why don’t you put the video and the other flash content in separate visuals and let the windowing system composite them? It’s hard to buy your argument when I can e.g. float a translucent scrolling xterm on top of my mplayer window (which incidentally also has on-screen controls) [ Different problems; read the post again. -Mike M. ]

  19. Gwenole Beauchesne says:

    Mike, this statement is wrong: “As of this writing, none of these drivers in Linux allow retrieval of the decoded video data”. Actually, all Linux drivers but IEGD do implement that. The GMA500 “psb” driver does that, likewise for “nvidia” and “fglrx”. BTW, retrieving decoded frames is slow and even worse on GMA500. There are means to avoid that: overlays/layers or OpenGL.

  20. Jay says:

    “But minority browsers don’t use as much CPU while playing HTML5 video tag data.Are you sure about that?”Safari 4.0.4 running in 32-bit mode under Mac OS X 10.6.2, on an iMac with a 2.16 Ghz Intel Core 2 Duo and 3GB of RAM.Using http://www.youtube.com/watch?v=1AwvuirSEAA and viewing the 720p version sized to 960 pixels wide (the larger size in the browser window), streaming.Using Flash Plugin, 100-115% CPU (105% typical). (!!!)Using Quicktime Plugin (that’s in the browser via Click2Flash), 30% – 40% CPU (35% typical).Unfortunately YouTube’s HTML5 beta doesn’t appear to work with anyhing but standard def videos (I can’t get it to work with that video anyway). So, with http://www.youtube.com/watch?v=mEIZ_FFmjnwUsing Flash Plugin, 65-70% CPU.Using Quicktime Plugin, 15-20% CPU.Using HTML5, 15-20% CPU.Pesky facts.j.

  21. jbus says:

    I see a pattern here and it doesn’t give me much hope that Adobe is putting much effort into their development on the Linux platform.BTW, Does Adobe approve of you taking this tone with Adobe customers? Mike, You sound overwhelmed and unhappy. I suggest you move on and let someone else take the wheel for Linux development. Perhaps you’ll fit in better on the Windows dev team.

  22. btmorex says:

    I still don’t understand how a standalone player overlaying various controls (maybe even transparent) is fundamentally different than what flash is doing. If flash needs the RGB data to do this, then the standalone players should be in the same boat and consequently they shouldn’t be able to take advantage of hardware acceleration either… but they can and do.Beside that, your post convinces me that for plain old streaming video, flash is not a good technology (webcams, games, anything more fancy, sure flash is great).

  23. STrRedWolf says:

    Nice… but outdated. Let’s go back to the graph in the previous post.First, if NVidia says “it renders to a X11 pixmap if you tell it to” that means that VDPAU decoding goes two ways. H.264 in, RGB out.Second, even if you get YUV out, can’t you work in YUV space for the rest of the elements (silently converting RGB to YUV) and letting the video card render the whole thing YUV? That’ll drop the requirements all the way down to needing X Video support, and you don’t need a heavy YUV-to-RGB conversion.This reminds me of a paper called “News Need Not Be Slow,” located here: http://www.starcomsoftware.com/proj/usenet/doc/c-news.pdf It has some advise, including:* If you walk the same road repeatedly, consider moving to the other end.* Storerepeatedly-used information in a form that avoids expensive conversions.Time to bone up on some history, I take it.

  24. Deanjo says:

    “Flash Player 10.1 has support for Broadcom’s video decoding coprocessors (according to the release notes, the BCM70012 “LINK” chip). This is currently supported in Windows. Support for these chips in Mac and Linux would be nice but the drivers aren’t ready yet.”What?http://www.broadcom.com/docs/support/crystalhd/crystalhd_linux_20091229.zipBTW you might want to catch the opinion of the freedesktop devs on which API to use.http://article.gmane.org/gmane.comp.freedesktop.xorg/40358“Of those, only VDPAU is worth implementing”Seriously do some research before blaming others for your inability to code.

  25. Gravis says:

    There is a good solution for the YUV to RGB conversion process but you may not like it: assembly language. yes, using assembly language is not portable, so i propose you write it in C or whatever and then use the preprocessor to use different sections of assembly language for different architectures (x86, x86-64, ARM, etc).the good news is that this already code exists and is extremely optimized. the bad news is that it’s part of chrome, so you have to abide by the license. if you open sourced a tiny section of code, you could use this sweet sweet code.

  26. Penguis says:

    yuv to rgb conversion, blending and scaling can all be done with OpenGL. There’s no excuse. The reason this hasn’t been done is because Adobe management only cares about Windows. You’re probably the only person at Adobe working on Linux.

  27. Matt says:

    “However, the internet at large has unanimously declared that it appreciates the Flash Player’s solution to the video problem.”If that were the case, then I wouldn’t think the HTML5 video standard would be getting all the attention it is right now. Even Microsoft has said they’re going to add it in their next browser. It’s true that it’s the only viable solution for now, but I don’t think the future of video in Flash is looking very bright. You’ve even explained why yourself – Flash just wasn’t designed to be a video player, and although it can be abused to become one it doesn’t do the job that well in comparison.Regarding the details of grabbing decoded data – others have pretty convincingly said you are wrong. Have you double checked your info, or are they just spewing nonsense. I did think i heard vdpau could output to pixmaps, is there some reason you can’t use those?On a final note, after reading this I still don’t get your last post here, complaining about a “thicket” of api’s. Now you’re saying that none of them do what you need, so that’s not exactly an overabundance of choice, is it? Rather the opposite. And as others have noted, you can basically just pick VAAPI and have everything be covered (if it worked the way you needed, of course). Windows certainly has a huge number of abstraction layers where you can choose high or low level api’s, and you don’t typically hear too many complaints about that.Really, the reason you face so much hostility in these comments is because Flash just seems to be standing still, with no progress being made. You can pretty much name a feature, and it either doesn’t work or works poorly under linux and OSX. Even on Windows, the Netscape plugin is terrible performance-wise compared to the IE plugin, and there doesn’t seem to be any interest at all from Adobe in fixing it, or any indication that the complaints are even being heard.I wish you luck in dealing with all of this, I know you’re in a difficult position and don’t envy it. But to be honest, I don’t really expect anything to ever get any better from Flash, and that’s really your problem in a nutshell.

  28. pem says:

    Hi, I’m a bit confused, you say that the linux drivers don’t allow retrieval decode data, and you also say that is a driver functionality, so why do you write a post about the libraries that exist about hardware video decoding?So from my point of view you should talk to hardware vendors to implement that functions on his drivers.[As of this writing, none of these drivers in Linux allow retrieval of the decoded video data. -Mike M.]

  29. Jack says:

    Thanks for the post. I’ll take the language and tone with a grain of salt. If the current API’s don’t exist, I think what most Linux users would like to see is for Adobe to be proactive and talk to the folks at NVIDIA (and other teams) to get the necessary ingredients into VDPAU/VA-API. The NVIDIA driver team seems quite active and responsive. Some communication between yourself and that team (which hopefully is already going on) would likely be quite profitable.Now, go have a beer; sounds like you are having a rough day. Cheers.

  30. noisymime says:

    This is an honest question, not trying to troll or anything.yes your standard video playing app (vlc, mplayer etc) doesn’t have to worry about complex onscreen displays, but what about media centres? Boxee for example can play HW accel’d HD h.264 smoothly whilst overlaying an alpha blended OSD without any problems (on linux).Additionally, I’ve played with the Clutter framework using the gstreamer API. It does a YUV->RGB conversion and can overlay arbitrarily complex objects without affecting playback. And that’s with a software video decoder too.Like I said, not trying to troll, but at least on the surface, it seems like there are already apps doing what you say can’t be done on linux.

  31. schlock says:

    How about an apples-to-apples comparison then? On OS X, Flash takes 2-3 times more CPU than either Chrome or Safari does on the exact same Youtube page (flash set to 360p, dunno what res the HTML5 beta uses)

  32. Marcus says:

    I still don’t really think that explains everything. I clearly see the difference between playing a downloaded file and playing an embedded video at a website. But with a greasemonkey script and mplayer plugin you can play youtube videos using the same cpu as a downloaded file.Can’t say i care THAT much about specifically performance. Proper support for 64bit however, would be nice. That’s pretty much the only setback with 64bit linux right now.

  33. Oli Warner says:

    Few points.1) As another commenter said, perhaps you’re solving the wrong problem.Why do you need to convert the video into RGB and munge the video with other elements? Dedicated video players (I’m thinking mplayer but I can think of dozens more) can overlay elements on the screen without destroying their VA.I don’t know how they do this but I’d suggest it relies on some form of composting that allows the video to go straight to screen and the overlay elements (your buttons, etc) are just that – a transparent control overlay over the video. Why can’t you separate the Flash/AS/etc from the video in this way?2) You accuse many of us of not having the perspective of a non-technical user but have you ever used Flash on Windows?It uses less CPU for me to play a video in Windows on VirtualBox (without acceleration) than it does to play a video in native Firefox.And it’s not just video – it’s Flash as a whole. It’s a hundred times slower on Linux than its Windows counterpart.Perhaps if you solve this issue, complaints about video acceleration will diminish as they’re less relevant to most people.I don’t mean to go over other people’s points in part but you *seem* to be ignoring the meat of the issues and replying “Different problem; read the post again”. I think we’d like to know *why* traditional video overlays don’t work in Flash or why Flash for Windows is so much faster… Or even why 64bit Flash for Linux is slower than 32bit Flash for Linux.

  34. LGB says:

    “The Flash Player has to do a little bit more. In addition to decoding the data, it has to convert YUV data to the RGB colorspace and combine the image with other Flash elements. Then it has to cooperate with another application (web browser) to present the video to the user.” Ok I get it, but still I don’t care about other “flash elements”! I want to see the video I don’t care about other stuffs. I only use flash, because this is (was?) the standard to see videos on youtube for example. I am simply not even interested in flash banners, or other tricks, I even block flash elements from my browser, expect youtube play areas. I guess most people (but I can be wrong here) are interested in flash (as end user, not web developer!) to see videos (ok, sometimes – I should admit – there are flash based little games and so on too, though I never played with those). So for me, it’s more natural to switch html5 video element if it will be used by major video sites. It does not require binary blob like flash is (which often crashes the browser and resource hungry, also it’s not open source software, while many browsers are with video playing capability byhtml5/video tag – so in theory I can even hack the source code to suits me better – I can’t do this with flash, for example there was some locking issue in case of pulseaudio some time ago, and nobody could fix it since open source developers couldn’t see the source code of flash plugin and it’s really the power of open source that you can fix problems …). Anyway, just I would like tell, I am not “angry” with flash, I don’t want to blame it here, what I wanted to say, that some people (like me) needs it mainly for video. And if there is another possibility to do this (video tag), I (and others) will move towards, since flash can’t compete with it in long term, it seems because of various problems. Still, please note, that I must thank, that flash is available for Linux, it would be an even more hopeless situation without it 🙂

  35. cosku says:

    Mike, you didn’t address the “windows vs linux” performance difference.If everything is as you said in this post, performance should suck in windows too but I think we all agree that flash video in windows beats the shit out of flash video in linux.

  36. Yaniv says:

    “As of this writing, none of these drivers in Linux allow retrieval of the decoded video data.” – then contribute code that does that. It’s open source, isn’t it?

  37. Grzes says:

    Hi,I appreciate your comments and I am sure that all the technical issues you mentioned are valid (I am not an expert myself). However it is a fact that the Windows version runs much faster than the Linux version (and not only videos, this is a valid comment for any content).Reading between the lines I think the real problem is that the Linux development is massively underfunded at Adobe. From technical perspective it shouldn’t be a problem to contribute to some of the open source projects you mentioned and add the missing functionality.I agree with Jay: if you are annoyed with the state of affairs in the company, just move on.

  38. someone says:

    I’m not sure your explanation completely adds up.E.g. “mplayer -vo x11 -osdlevel 3” will do conversion to RGB and overlay an elpha-blended OSD.Admittedly it does take shortcuts, but I still don’t think it fully explains the differences.Is it possible that maybe a lot of performance is lost getting the data “into the browser” (a step you haven’t in your diagram)?Also, a lot of people mostly complain about fullscreen, maybe the scaling algorithm is “too good” for its purpose?Also I think all hardware acceleration APIs allow reading back the decoded pixels, I think VDPAU will even allow you to read back the RGB-converted, scaled, filtered, deinterlaced etc. frames (though I do have my doubts if they are really practical for flash to use), and VDPAU does/will support rendering directly into OpenGL textures as well.

  39. Sea says:

    Indeed conversion from yuv two rgb is expensive. SSE can make a big difference here.However the claim that flash has a different problem to solve than your video player is simply not true. mplayer can do OSD and can do this in YUV. If the RGB elements were converted (once!) into RGB and then combined with the YUV output from the video decoder, then non of this conversion would be necessary and flash would not be such a CPU hog.Just do all your processing in YUV.

  40. James Neave says:

    So, why don’t you render all your flash elements in YUV instead? It’s just a different colour encoding after all.Then you wouldn’t have to convert the video.So:DecodeBlendGive to browserAs for GPU leverage, wouldn’t it be possible to accelerate all of this on Windows/OSX/Linux using OpenCL and DirectCompute? They definitely let you get the data back off of the GPU.J1M.

  41. Anonymous says:

    last time i checked i can play 720p videos with mplayer -vo x11 (eg. software yuv->rbg + software scaling) without any problems. flash is choking on youtube videos …

  42. have_to_use_flash says:

    If you don’t like something in VA-API, why not just sit down with VA-API developers and resolve problems. This is how it is supposed to work in open source world…

  43. This has cleared a few things up, but raises more questions again :-)If the CPU killer is compositing Flash elements onto the accelerated video, why can’t that be done in an alternative way (i.e. ask the window manager to do it, and have the WM float the Flash content over the accelerated video).KDE (as of v4) doesn’t even use Compiz to provide it’s compositing any more…

  44. Licaon_Kter says:

    Adobe 10.1 release notes: “Supports GPU-acceleration for smoother online HD videos with the new Adobe Flash 10.1 beta.”nVidia Windows drivers release notes: “Supports GPU-acceleration for smoother online HD videos with the new Adobe Flash 10.1 beta.”So why isn’t Adobe talking to the nVidia Linux driver devs too, like they did with the Windows ones ?

  45. buccaneer says:

    “As of this writing, none of these drivers in Linux allow retrieval of the decoded video data.”Err…VdpVideoSurfaceGetBitsYCbCr: “Copy image data from a VdpVideoSurface to application memory in a specified YCbCr format.” *Since Adobe works closely with Nvidia, I would assume you’d know the capabilities of an API they created.Also, while I haven’t checked, I’m quite sure VA-API supports something like that too.* http://http.download.nvidia.com/XFree86/vdpau/doxygen/html/group___vdp_video_surface.html#ge3550dc62e4e4c390c1874b37f96e7c0

  46. Tet says:

    Interesting post. How can I meaningfully report a bug in Flash player? There are several, some quite significant. But whenever I report them via Adobe’s bug reporting page, they disappear into a black hole. I can’t even see my own bug reports, and nothing I’ve reported has been fixed in subsequent releases. Is there any way to report bugs that will actually result in a better Flash player?

  47. clast says:

    Thanks for you insights in flash player development.When I read your posts i always get the impression that you’re just developing the linux version because you were told to do so. It seems like you don’t actually use linux and therefor don’t care if it works properly or not.The video accelerations APIs are open source, meaning everyone can contribute and make them better, even Adobe. Hell, AMD, Intel and Nvidia are doing it…Again, it seems like a linux Flash Player is something you just “have to work on” not something anybody at Adobe is really passionate about or else we would certainly see some improvements to the underlying layers in the stack.I still very much appreciate the existence of the (linux) flash player as it is today, since it enables us to do lots of cool stuff on the web. Thanks!PS: Nevertheless, i’m looking forward to move to more open standards

  48. Ferdinand says:

    Calling the best 4 browsers in the world minority browsers is a bit weird. People that use IE are people that don’t even know that they have a choice.[ It’s not weird; it’s straight math. Less than 50% = minority. -Mike M. ] It is also weird that you post your comments in other peoples comments.[ You mean, like this? -Mike M. ] The main reason why flash is installed in most browsers is because it solved a lot of problems like video, flashy menus and games. Video is almost solved with HTML5. Flashy menus have been possible for a few years with javascript. That leaves games. So people that don’t play flash games will be able to not install flash in the future.It seems that flash and the users have a hate-hate relationship.

  49. iive says:

    Sorry Mike, but your post is total BS.Flash player is times slower than mplayer using software yuv->rgb transformation (and ssa/ass subtitles).The real problem is that Flash player for linux doesn’t use any assembler and SIMD optimizations, probably because they are written for MSVC inline asm, and it is not trivial to port it to gcc/yasm one.The solution: use ffmpeg libswscale dynamically (SIMD is GPL).You know how to do it, you are (still) FFmpeg developer after all!

  50. ceski says:

    Thanks for this post.Did I understand correctly that the transformation YUV->RGB is imposed by the process of embedding the “Flash image” into the browser? So, assuming that someone would invent a browser capable of embedding YUV “image components”, then you could do the “blend flash elements” part in the YUV domain, skip both linear transformations, and then apply hardware acceleration (possibly even fullscreen scaling) to the YUV data?Does this line of reasoning fail only because there is no browser supporting embedding of YUV “image components”, or does it fail for further reasons?

  51. Peter says:

    I guess this is what happens when you put a Windows developer to code Linux applications. No excitement at all, only excuses.

  52. Jon says:

    What about a video codec that stores RGB instead of YUV? I have no idea if such a thing exists, but if it did would that help speed up flash video?

  53. Stephen Warren says:

    Mike,This post seems to rest on two basic assertions:1) Flash must have CPU-access to the decoded video surfaces.2) Flash can’t obtain CPU-access to the decoded video surfaces.I believe that both of these assertions are wrong, at the very least for VDPAU, and probably for other video APIs as Gwenole claims above; he should know.Taking the points in reverse order:VDPAU currently has two different ways of obtaining CPU access to the surfaces. One can use APIs such as VdpVideoSurfaceGetBitsYCbCr or VdpOutputSurfaceGetBitsNative to download the data to the CPU, and then act on the data in any way. I don’t believe any media players currently do this, because there’s very little point. Alternatively, one can use the VDPAU presentation queue to present the data to an X pixmap. One can then use X APIs, or OpenGL’s GLX_EXT_texture_from_pixmap to composite this data with application UI elements. At least XBMC uses this method (specifically GLX_EXT_tfp) very successfully today, even on low-end platforms; it is a very well tested path.Finally, more mechanisms will be made available in the near future.On to your second point:I imagine that the only reason Flash requires CPU-access to the frames is to render/blend the UI on the CPU. I don’t think this is the correct approach; GPU acceleration should be used for the UI rendering (or at least upload and blending). VDPAU itself has various rendering/blending/scaling operations built in specifically for this purpose. Alternatively, you could get the video into OpenGL and then use OpenGL’s rendering/blending/scaling operations, as XBMC does. Do also note that VDPAU’s VdpVideoMixer fully performs the YUV->RGB conversions you mentioned on the GPU, and if Flash really needs, it can download the resultant RGB data after that step with almost no effort.I also take issue with your point that Flash is somehow fundamentally different to other media players. Specifically, MPlayer renders UI/OSD, subtitles, etc. on top of the video using VDPAU features. XBMC renders a potentially complex and pretty UI over the video using OpenGL. I believe both of these applictions, and others, can also perform network streaming at the same time for example. It sounds like they’re both doing the exact same thing that Flash needs to.Please note that I haven’t yet read your “flash uses the GPU” post, or at least note recently. I’ll go read it now. Still, I doubt that will change my mind that widely available APIs (across platforms and vendors) such as OpenGL are the correct way to accelerate graphics-oriented applications such as Flash.In summary: If you have any issues understanding or using VDPAU, please feel free to contact NVIDIA. We’d be very happy to help you.

  54. Spencer says:

    Mike M.:I see several sarcastic and biting responses to the comments here, but they seem to disappear with Gwenole’s comment.Can we get responses to Gwenole and Jay, please?Even if it’s just a “oh well shut up” or something equally non-substantial (as I expect,) simple acknowledgement that you’ve seen these facts would be useful.I agree with jbus: perhaps you should move on. You’re clearly not a Linux developer.And if this is the tone Adobe uses to deal with its userbase, I welcome HTML5 as a replacement.While 41% (according to the not-very-credible source Wikipedia) is technically a “minority,” what company would willingly alienate 41% of its users? Furthermore, have you heard of user-agent spoofing? Many applications mask themselves as IE for compatibility reasons with poorly-designed web applications.

  55. John says:

    When will you realize that Flash isn’t a killer app anymore? Flash is the modern equiavlent of GIF-laden GeoCities web sites from the 90’s. As much as you gripe about developing Flash for Linux, when will you realize that the Linux community doesn’t need you? There are other alternatives out there, some of which are progressing rather nicely.

  56. bug says:

    Great stuff dude. Please don’t support anything but 32bit Windows in the future.

  57. Stuart Ladd says:

    While I’m sure most of the technical arguments you put forward for what flash has to do are valid, as others have noted: as far as the user is concerned, it should be Browser+Video Site = Smooth Video. I don’t particularly care how the controls appear (though presumably content providers want their ad overlays), I just want smooth video playback. Clearly the alternatives (mplayer, vlc, HTML5) while solving a ‘different problem’ are doing this simple thing well.That said I sympathise that some of the bottlenecks caused by the linux a/v driver situation, but it’s kind of ridiculous that windows flash under wine is better than native linux (something I’ll admit I have yet to try myself…but I will now since I just can’t get BBC iPlayer to play HD smoothly via flash or the adobe download player on linux).If flash is incapable of solving these ‘different problems’ while HTML5 et al solve MY problems, then I only hope HTML5 usage spreads quicker.

  58. John Dowdell says:

    (I’m sure it’s some Internet rule someplace that when you make personal comments about someone else or take objection to their “tone”, you need to hide your own persona while doing so…. ;-)Thanks for the info, Mike, greatly appreciated.jd/adobe

  59. Apopas says:

    Minority browsers? You must be joking or you aim to insult?1 out of 3 people use Firefox while if you separate the browsers by version instead of name then Firefox-3.5 is maybe the most popular version of browser out there and you refer to it as minority?That’s outrageous![ Indeed, math is outrageous. -Mike M. ]

  60. mike russell says:

    funny how my comments about vdpau providing access to the decoded frames are not approved…

  61. Remi says:

    It seems that when it comes to video and composition, everyone in the Linux community knows better than the people actually working on the problem. I guess they’ve implemented plenty of flash-like plugins themselves!No, really people, mind your manners. If you don’t understand the problem, don’t rave like a lunatic, it is very unprofessional.

  62. Daniel J Blueman says:

    Sawyer in Lost: “Zeek, we ain’t done”YUV->RGB conversion in system memory is cheap on modern processors, particularly at 30Hz – the processor is good at prefetching with linear read patterns, hiding much of the stalling. Multiply with add instructions are cheap at 3 clock cycles, or you use the lookup tables which fit nicely in L1 cache – better yet with SSE2 (use the CPUID instruction to fall back to compatible code); missing the point.Modifying the H.264 decode libraries to directly write RGB is far better than the read-modify-write cycle (RGB->YUV conversion is commutative, so doesn’t even need to be done in one pass); still missing the point.Perhaps I’m missing something, but GLX_EXT_texture_from_pixmap and the textured video Xv interface will allow the GPU to read the unscaled YUV data from system memory (optionally sharpen the Y channel), convert, scale and DMA the data back to the destination pixmap and set a flag when done, without any host processor overhead. Triple buffered, you draw your graphics over this (or have the GPU render them to the pixmap) before before setting the font buffer pointer. Proprietary nvidia/fglrx and open intel, radeon and nouveau drivers all support these interfaces.

  63. compn says:

    adobe flash is to playing video what adober reader is to viewing pdfs. which is to say that it sucks badly.of course its not mikes fault. so dont blame him.i’ve heard somewhere that vlc uses vdpau to decode/encode video, e.g. it returns video back to vlc. can you confirm/deny this mike?

  64. mdias says:

    Ohh yeah, those 3D games with moving folliage, taking care of AI and so on, in realtime on my full-hd monitor, surely don’t have to take care of so much stuff as you do…Uhhh… YUV->RGB used to be costly, yes, but we’re in 2010 now, and hey! we have multi-core CPUs! And don’t even get me started on GPU shaders!Hardware scaling? yeah, it’s really difficult to blit a bitmap to vram and render that full-size… Oh, it shatters your vectors? simple; do that only for bitmap data.Let’s get real, flash performance problem doesn’t come only from video rendering… There’s so much stuff that could be accelerated through our GPUs…Complex to code? yes, but that’s what you guys are paid for.

  65. gogothebee says:

    Mike, I will skip the comments about your language because I know how upset a person can be. Let’s say I’ve had my negative experience with X. Don’t get me wrong. Linux is perfectly fine until X kicks in.Let’s focus.Your blog post is informative and fully understandable. As I am the developer of the Direct3D MPlayer video renderer for Windows, I see your point. However as other people have already said, you could accomplish the task very easily from my point of view. I don’t know if this is possible in OpenGL, of course. I have little experience with Direct3D only.Why don’t you render the video in YUV surface directly through your favorite API and blend the other content (RGB based buttons and so on) in a RGB surface above the YUV based one?This will give you exactly the same unprecedented speed of all standalone players and the added functionality of flash-based elements on top of the video!The current flash video pipeline you pictured is extremely inefficient. I understand the poor performance as it represents the WORST possible scenario. Well, at least you can scale up RGB surfaces by hardware.Please, try to compare what is done differently in Windows. Converting by hardware YUV->RGB is fast, but reading the content back from the video memory is slow. If this is your rationale of not using the video card to do it, please elaborate why the same technique works in Windows? When the limitation is not hardware based, where is it? As I internally understand, if you say that the Windows API you use does allow you to convert by hardware YUV->RGB and read back the converted RGB frame, this doesn’t make sense. This involves reading back the RGB frame from the video memory (slow).Even if this is the case (which I highly doubt), as someone said, VDPAU can render to a X pixmap. Why don’t you use the HW acceleration and render with hardware YUV->RGB acceleration to a X pixmap and then blend you controls?Really – focus on the case where everything works fine (Windows) and compare it to the case where is doesn’t (X).I’m sure that this is just a slight limitation, which can be patched with a few lines of code. Fundamentally the hardware is the same, so the reason for the poor performance under X are plain software-based.In the end this whole discussion seems unrelated to the real problem. When you say that you need to do other stuff with the rendered frame (the whole reason to get back the RGB colorspace converted frame), I don’t really understand what this stuff is! All you need to do is to blend other controls OVER the rendered frame. You don’t modify the frame itself. Please, it would be good to explain why this approach is impossible!However – if you feel unsatisfied with your duties, please move to the Windows development team. I really stand behind this alternative as programming such stuff for Windows is easier and everything tends to work out of the box.Comparing the graphics subsystem of X and Windows have always been irrelevant. Especially since WDDM came to existence.

  66. Why can’t Flash have the option of acting as a dedicated video player? That’s what it is used as the majority of the time. Most people don’t care about adding elements.

  67. Anonymous says:

    You are somewhat right, but there’s a lot of room for improvement. Some examples:* Do *everything* in hardware. This is possible and is the direction most browsers are going, including IE9. The performance you get out of this is exhilarating, and there are demos which work *right now* (supporting all of HTML and SVG) at insane speeds. The downside is that you’re going to need pretty decent OS and driver support, so right now this would work for maybe 10% of all users.* Hybrid approach, easy version. If the video doesn’t have anything over it draw it with hardware accelerated YUV->RGB + resizing. The moment something is draw over, fall back to an all-software path. This would work well since most video players have separate controls and hide them while fullscreen. Challenges include making the switch seamless. It might have robustness problems.* Hybrid approach, hardcore version. Divide the scene in three zones: under the video, video, over the video. Update and render them separately, compositing them in hardware as three textures (the top one RGBA with transparency, the video YUV hardware-assisted, the bottom one just opaque). The surfaces which aren’t the video will seldom be updated (and when they do it’ll be small rectangles, upload these selectively). Requires half-decent drivers (not too bad though, probably anything from the last 5 years) and half-decent fillrate.Also it doesn’t help Flash any that people don’t know how to use it and just copy-paste crap code (wmode=transparent is prevalent and few video players do hardware-assisted fullscreen resizing).P.S.: It’s perfectly possible to overlay RGB over YUV via software (at a slight quality cost for the RGB stuff), converting on the fly. Many players do this for subtitles and the like on older cards.

  68. dacer says:

    Your last two post sounds like you are unable to solve the problems!!!Ask for help!!!, do it GPL and sure you get a lot of help now.But do it soon, maybe HTML5 kill Flash Player.Bye

  69. Jo User says:

    you sound bitter, and you should.can you tell me why Apple isn’t offering Flash support on its iPad ?you know, we know and i am SO happy that HTML5 is finally here.adobe was first to the market with browser video support. you had your time.move on and invent something new…

  70. FlashScope says:

    “Summary: Linux sucks, coding for Linux sucks, apis on Linux sucks, Windows is perfect and Flash too of course”I’ll put this into memories ROFL

  71. Paul says:

    Your post is a tacit admission that Flash is not the best way to simply play video on a web page. So why are we supposed to use it to do that again?

  72. Joe says:

    Some of you Linux guys are good folks. Bright and reasonable…But I find it comical that the rest of the free-tards are lecturing us on what is and isn’t a “success”. Seriously? How’s that Linux penetration going anyway?Sorry Mike, I know this doesn’t help but I couldn’t resist!

  73. About the RGB>YUV stuff.. Bear with my ignorance, but is YUV specific to video or common to all screen graphics? If it’s a video-specific color space, for the proposed “display all elements in YUV and let the video card convert” approach, wouldn’t you need to do near constant conversion of runtime-generated sRGB art into YUV and then back into RGB on the card? How is that more efficient?Flash is not a video player; It is a frontend development suite that happens to have a video object.Again, i’m assuming YUV is video specific. If it’s not, ignore this comment and forgive me 😉

  74. 2eurocents says:

    Mike,you’re being US-centric in calling Firefox et al. “minority browsers”. There’s a world outside the USA, and that’s where MSIE is the minority browser:http://www.webmasterpro.de/portal/webanalyse-aktuell.htmlAlso, the German Federal Agency for IT security (Bundesamt für Sicherheit in der Informationstechnik) actively warns against the use of Internet Explorer.https://www.bsi.bund.de/cln_174/ContentBSI/presse/Pressemitteilungen/Sicherheitsluecke_IE_150110.htmlMy conclusion from all this: Including video playback in Flash was a stupid idea right from the start. It filled a hole for a while, but YouTube, DailyMotion, Apple etc. pushing for HTML5 video are making it very clear that the end for Flash as the primary video playback technology on the web is right around the corner. What’s left then? A few browser-based games. And those will be moving to SVG and Canvas.As others stated before, you’re trying to solve a very special problem with Flash, but it’s one that nobody really has.

  75. Thomas says:

    Mike, your post just makes you look stupid. I read nothing but excuses. The fact that you are lame enough to only stress that less than 50% is a minority and don’t reply to technical comments makes you look even more stupid, like you have no idea of what you are actually writing about. The facts _are_ against what you’re saying. Except, of course, that 40% is a minority. Watch it grow and see Flash die, eventually.

  76. cosku says:

    can you at least respond to some of the comments?

  77. Uri says:

    – The problem Flash solves is “Filling the gap in standards for creating dynamic websites”. This gap is quickly closing. Flash was a stopgap measure. Yes, it’s widely used today; in 3 years, it will be a bad memory.- If Flash has to be bloated and slow by design, as you say, it has no chance on resource-constrained mobile platforms, which is where consumers are going. This is not theory: it’s losing there badly right this moment.- Saying Flash is “appreciated” is Orwellian. Flash is universally reviled by users. It’s one of the unfortunate facts of life, like cell phone contracts. Many (like non-Windows or mobile users) hate Flash with a passion; others may only find it annoying. There’s a reason users are begging YouTube and others to drop Flash as quickly as possible. Also read the comments on Adrian’s post on the Flash Platform Blog, about the lack of Flash on iPad; the sentiment in the many, many comments is an unequivocal “Good Riddance”.

  78. David says:

    Even ignore the numerous fallacies already outlined, there is no excuse for a flash video on pause to burn 100% CPU. At the end of the day, the problem here isn’t so much that flash is fundamentally different, but that flash is fundamentally poorly written.

  79. Gert says:

    But WHY is Flash on linux so UNBELIEVABLY crappy compared to the Windows version even without hw acceleration? I don’t compare apples and oranges, I compare the same product on different platforms, and I can clearly see that one of them SUCKS compared to the other. So why this huge difference? Because Adobe don’t give a shit about the linux version, is why. And I hear the same goes for the OSX one.

  80. Emm says:

    I hope html5 video becomes a reality soon in order to uninstall the adobe’s closed source plugin. I only have 2 cpu cores and it doesn’t seem enough for flash.

  81. foo says:

    I’m amazed: twice in the last few eeks I posted questions related to flash on Linux, the comments NEVER showed up, while flames just go on and on…WTF is wrong with so-called moderators on this blog?! Just what this blog is for: justify that flash on Linux does not live up users’ expectations? Shouldn’t it be a means for Adobe developers to do more than dumb one-sided communication? [ This isn’t an appropriate place for specific bug reports. I delete such comments. Strange, I know. Try bugs.adobe.com -Mike M. ]

  82. Doctor says:

    Mike, have you considered to apply for a different job? Clearly you hate wrestling with Linux. Please consider your health also!

  83. Jack says:

    Mike. Please ignore the flame comments and read the comment from Stephen Warren (of NVidia) above. All I, and most Linux users, want/expect of you is to work with NVidia and other vendors/developers to get the ingredients you need into their APIs, drivers etc…

  84. Kevin Newman says:

    I still wonder why you don’t just open the source for the binary you give away for free (or at least the shell/renderer, and other parts Adobe owns – you can keep your copyrights, so that can’t be it). I never get an answer to this query either.

  85. toto_de_france says:

    >> I still wonder why you don’t just open the sourceThere must be some patent inside. Something like “an algorithmic way to contribute to global warming”.

  86. Mike G. says:

    Mike,I was wondering if you could write an article discussing the non-video performance improvements in Flash 10.1. As others mentioned above, certain websites bring the browser to a screeching halt in Linux — even with the GPU acceleration forced on. I was surprised to find out that these same websites work very well in Windows. I am curious if Flash 10.1 will help close this gap, which is most apparent on netbooks.I’m interested in learning more about anything BUT video acceleration, which seems to start a flame war each time it’s mentioned.My apologies if this was covered elsewhere in the blog.

  87. Steven B. says:

    If you open source the flash player plugin, I’m sure within months there will already be a flash player fork with all the features that will _fix_ flash for the Linux desktop, things that you have been trying to solve.You won’t have to do a thing except sign patches.Everyone is eager to fix the stability and security of this plugin once and for all, so why not?

  88. heheno says:

    “the internet at large has unanimously declared that it appreciates the Flash Player’s solution to the video problem.”except then the iPad, iPhone, all linux users before a few years.o and well all the effort we put into html5 because we want to get rid of this bugridden binary security disaster.you should really try to get over yourselves. Because any market dominance you guys had is slipping through your fingers.

  89. Chad Rodrigue says:

    Your solution to the problem (of being an online video player), in comparison to other solutions, is clearly substandard – doubly so in Linux. This article does a great job of explaining why that is.However, Flash DOES offer *a* solution to the problem, rendering the statement “Flash solves a different problem” inaccurate.Further, the repeated deflections about Flash’s “charter” and what its really designed to accomplish do nothing but completely ignore the people for whom the software is supposed to be engineered for. You (and Adobe) may choose to bury your collective heads in the sand and stick your proverbial fingers in your ears while shouting “la la la” all you want to, but it certainly does not change the fact that Flash-as-Video-Player is, far and away, head and shoulders above, THE primary use for the software in modern computing.And if you, Adobe, and Flash don’t stop being almost completely unwilling to solve the problem, roll with the usage patterns, and re-purpose the software to be more in line with what people actually need to use it for, then other solutions (Silverlight? HTML5 – web video without that whole messy plugin business?) that *are* willing to listen are going to come in and steal your market share, as has already begun to happen.And, for what its worth, I think you’re simply in denial about HTML5. At least on my Linux box here, it *does* draw *less* CPU than the equivalent Flash solution, at least on YouTube beta. The difference is not a lot, but it does favor HTML5 – and HTML5’s solution is admittedly unoptimized, preview-quality stuff.Stop it, seriously. Stop being on the defensive, stop dodging the issue with weak deflections about Flash’s “purpose” and what problem its actually engineered to solve vs. what problems the user base actually *need* solved, and please stop pretending you know what we want and need more than *we* do.There’s a reason so many of us consider FlashBlock and its ilk absolutely necessary, and it isn’t because Flash is merely busy “solving a different problem.”

  90. jza says:

    “I hope html5 video becomes a reality soon in order to uninstall the adobe’s closed source plugin. I only have 2 cpu cores and it doesn’t seem enough for flash.”Yeah, but as Mike says, HTML5 won’t do any good if it has the same YUV->RGB overhead…

  91. Wolfgang says:

    explains exactly nothing… apart from youtube, google and others obviously being idiots for choosing flash for video playback – because flash is not a video player…it also explains very well the big performance difference in windows and other OS…so let’s say it all together – bla bla bla

  92. Patrick says:

    Mike,I doubt you are still reading comments on this post, but in the unlikely event you are, I’d like to say thank you.Linux without flash player is useless for desktop purposes. Anyone who is stupid enough to think that all flash is good for is video streaming really has no idea of what has been going on on the web for the past 10+ years. Go to any site on good/creative web design and you will find tons, absolutely TONS of major design award winning sites that actually make browsing the internet fun and interesting all written in flash/flex. All the open standards you want in HTML5 and canvas and the video tag, and WebGL — this stuff is all good but it’s really just attempting to get to where the flash player was at version 5.Thank you for your work, please try as best as you can, to let this mindless, inane criticism “roll off your shoulders”. One of the biggest problems with Linux is always that the users are all know-it-alls who think they have it completely figured out and could do everyone’s job better than them anyway. In my experience most of these guys are just morons (read some slashdot comments sometimes if you don’t believe me — they’re all idiots).Really. Anybody that’s criticizing you and what you are doing here does not know how good we have it that Adobe is putting the resources here at all. Gnash?! HA! Right!Please, keep up the good fight, don’t give up – despite the protests of a bunch of whiny readers.I guarantee you most of your readers are like me — long time readers but non-commenters. Usually I don’t even read the comments ’cause they’re so annoying. We know the value of what you are doing, and appreciate it.Thanks again,Patrick

  93. Ben says:

    Maybe it’s been mentioned in a previous comment… couldn’t you just send all image data (video and controls/interactive stuff included) to the graphics card instead of sending it to the card to be decoded, retrieving it, converting it from YUV to RGB and sending it back?I know that’s not always possible (like when there’s no hardware acceleration available), but that would seem like the best choice when it is available. And when it’s not, then all the image data would be in RGB already, no?Just a few mind-prodding ideas; feel free to digest, dispose, or comment on as appropriate. ^.^

  94. Anonymous says:

    Patrick, you’re describing Windows flash there. Linux flash is quite a different beast.Out of interest, have you tried aviary.com’s flash image editor, Phoenix, in Linux? Scribble a little, and it won’t be long before the lag makes it unusable. Now try it in Windows (or Wine!) on the same machine. Quite a difference, isn’t it?

  95. Steve says:

    If you spent as much time working as you do bitching this issue would be solved.Using the GPU or not, the flashplayer still produces bad flickery video even on my Core2Quad.

  96. Craig Kelley says:

    Most Xv drivers offer an RGB port — although you are correct, some do not. The Gnash accelerator blits the output of the renderer to an Xv RGB port, or it does a YUV -> RGB conversion if none is available.Thanks for the great post. I’ve been wrestling with these issues for a long time. OpenGL seems like the best solution long-term. Unfortunately, it does not work on low-end machines of today, which is where it is needed most.The real problem is that the Flash stage is all based on RGB; owing to its legacy of not being a video tool. If the stage could work in different color spaces, it would solve the problem.

  97. Carey says:

    Good post, thanks! The detailed information was a good read. Good to see people at Adobe are working/thinking about this issue.Reading the post, it seems that a major problem is that the Linux video drivers for these cards are not “allow[ing] the retrieval of the decoded video data”. However, this appears to be wrong if the posts by Jeffrey W. Baker and Licaon_Kter are anything to go by.So Mike, assuming this support in Linux is indeed all ready-to-go, then perhaps we can expect full nVidia and ATI hardware support in Linux just as in Windows with the final release of Flash 10.1? Maybe? Fingers crossed? :)In any case, I encourage you to get cracking’ on this support guys and doing whatever you can to improve Linux Flash performance, as we are all horribly sick-to-death of slow Linux Flash ruining our browsing experiences! 🙂 Especially anyone like me with a low-end processor/machine such as an Intel Atom!I am also curious: Do you know of anyone at Adobe that has tried using any heavy-Flash sites on a low-end processor Linux machine? If you haven’t, please do. Steps: Just (1) grab yourself an Intel Atom netbook running Linux with Adobe Flash and (2) browse a website like mcdonalds.com.au. The flash will be so slow that it will cause you to have a fit of frustration and have evil thoughts about Flash and Adobe all-at-once (people walking outside office window: please watch out for a flying netbook) :-).

  98. Ben says:

    As a follow-up to my previous comment, I’d like to propose something like this (png, 24.4KB) for handling video in Flash. I would think just drawing the flash parts after the video and on the same buffer would work, if you can’t use textures as others have proposed.An explanation about why my proposed method wouldn’t work would be nice, but I’ll accept a simple “no”, if you don’t have time.[ No, because I can’t access your server. -Mike M. ] Note that the image may be removed eventually, so feel free to make a copy. There’s also an xcf, in case anyone wants it.

  99. Paul says:

    Bashing minority browsers on a company blog about how they promote their products to minority operating systems where the _only_ choice is using said minority browsers is just… insensitive to the audience, don’t you think?

  100. z says:

    Ugh. I’m so sick of excuses from you clowns at Adobe. You can’t for the life of you, port your software to any platform in a reasonable manner. You cannot seem to properly build your software on a 64-bit target OS.All you do is offer excuses for your crappy software failing to work. Flash is a disease on the web, and while you continue to offer excuses, everyone else hopes that HTML5 sneaks in and forces your product into obsolescence.Ugh. It’s disgusting. Go a head and keep offering excuses for your software’s failure to properly work. The rest of the world can leave you behind, for all we care.

  101. bdeonline says:

    Nice write up really helps to better understand the problem. Flash 10.1 has been a good update allowing my Asus eee pc running Ubuntu 9.10 to run the Hulu player at 480i with a viewable frame rate & decent performance. Great considering the Hulu player ask for a minimum Core 2 Duo.No one has stated they “like” the fact IE is still the majority. At least IE 6 is finally under 7 & 8 in usage now. Go Firefox…

  102. This is an excellent post and I’d like to thank you for posting not only this one, but the others where you expose the mess you have to put up dealing with linux.I would, however, like to pose a couple of questions.Firstly, this post answers an interesting question, but I do believe that the question that anyone who uses flash on linux asks is “Why is this faster on Windows?” and it’s surprisingly difficult to find an answer.I’m still confused on this. On a first read I was under the impression that flash did not use graphics acceleration“Video card acceleration[…] Assorted video cards from Intel, ATI and nVidia are also able to assist in video decoding […] Again, this is presently only available in Windows.”So the post “Flash Uses The GPU” is refering to other elements besides video decoding?What am I missing?Still on the matter of performance, video is indeed a major issue, but performance is poor on pretty much all fronts. Scrolling a webpage on firefox with embedded flash content is slow, animations, etc.My question is: Where can we point the finger? Is it the video drivers? Is it firefox? Is it some flawed internal flash architecture?A small comment or pointing in the direction of where I can find this information would be much appreciated. Thank you for reading.

  103. Nigel Jones says:

    Thanks for the explanation.However today many sites are using flash for basic video streaming, and the user experience under linux is very poor.The only option is to avoid those sites or compaign for a better technology choice that does delivery a good user experience.Jerky video with a hot computer struggling to do sw decoding is NOT a good user experience.Most of the time these sites are not using the additional flash capabilities, but rather just as a video streaming technology

  104. HM says:

    Thanks for the explanation. Thanks for your work, flash player for Linux is usable for me.Please ignore trolls.

  105. DJ München says:

    Thanks for this useful informations. I do not really understand the problems of some other users. You’re doing a great job – keep on working! 🙂

  106. Goeland says:

    So I understand your point about the YUV to RGB conversion… But lemme ask you this: when your dedicated media player is in full screen playback, and you have overlays appearing (movie scroll bar, volume control, playback buttons…) are those overlays in YUV?I think you could possibly consider doing dual-output and layer your overlays in RGB on top of the YUV feed.Hell, consider even making a plugin where the video will popup in a separate window if you can’t do the YUV/RGB overlay in the browser… I’m sure most people wouldn’t care as long as they get to watch a turtle humping a shoe in full screen without lag.And is it possible (just maybe) to consider converting the overlays from RGB to YUV and keep the whole display YUV? Then you get to use HW acceleration, interactivity and make everyone happy. Since the button overlays usually hide, converting those elements is probably less troublesome than converting the entire video frame from YUV to RGB.Just my $0.02 worth. Otherwise I have to admit flash works pretty good on linux, just a few tweaks that need to occur.

  107. Chris J says:

    I have read most of this blog as well as the “Flash Uses the GPU” one but there’s one thing I don’t understand. I have a Pentium 3 laptop with Windows 2000 and a couple of Linux distributions. The Neomagic GPU in it doesn’t do any acceleration on flash in either platform yet I can watch Youtube videos in Windows. In Ubuntu, the flash videos are horribly slow to the point of being unwatchable. So what’s the difference here? And why can’t Adobe give us some options like reducing post processing, etc. to speed things up on slow computers or reduce CPU usage on faster ones?

  108. Anonymous says:

    It has now been YEARS the Flash for Linux has been dramatically underperforming. I, for one, look forward to the day HTML5 arrives just so I won’t have to read the endless stream of excuses from Adobe.

  109. D says:

    Easy Solutions:Integrate different modes for flash:- many people just want video, then give them just a video player that works in all browsers (don’t provide for additional element blending in that mode)-> accept that for many people flash is really just a video player.- if there are issues with the browser API, fix them where it’s open source, the others will follow. Work with Firefox, Chrome and Opera, they will happily fix APIs to get better performance.- if there are issues with the Video APIs, contact the developers and fix them. I’m sure if you tell Intel you’ll integrate VAAPI in Linux Flash they will help you to get the frames where you need them. Of course the first point might apply here: if you implement a video decoding mode, you need not worry about getting the data back.

  110. Avuton Olri says:

    Reason don’t care: HTML5 will /always/ work on my 64-bit browser.

  111. “D” cut right to the chase of the matter. Nicely explained!

  112. Livio says:

    You just suck and want to die in agony.

  113. Xunil says:

    http://www.phoronix.com/scan.php?page=news_item&px=ODUzOQ

    “While Adobe supports GPU-assisted video decoding in their official Flash Player for Windows and Mac OS X platforms, the Gnash project has beat Adobe into supporting GPU video decoding under Linux. Adobe hasn’t implemented VA-API support nor VDPAU but rather they have just ranted about the Linux video situation at length. Nice job to Splitted Desktop Systems and the Gnash project in supporting GPU video acceleration prior to Adobe’s official Flash Player for Linux. Gnash also builds fine on 64-bit Linux platforms, which as of late has been shafted by Adobe with their proprietary Flash Player.”

    Adobe, LAZY, INCOMPETENT company who just embarrassed by the open source community.

  114. TonyMac says:

    Flash has been more than underperforming on linux, it’s been crippling to many systems.

    Any respect I may have had for your dev team has been lost with this demeaning and rude blog entry.

    Say it with me: “We’re breaking the fundamental rule of customer service”

  115. thecapsaicinkid says:

    “Any respect I may have had for your dev team has been lost with this demeaning and rude blog entry. ”

    Likewise.

  116. nanonyme says:

    How about adding OpenVG support to the Linux client? You already know how to use it since you do it in the mobile Flash. Opensource drivers in Linux are doing OpenVG 1.1 nowadays.

  117. B-Con says:

    I understand that Flash “solves a different problem”. However, when it comes to performance gripes, that is the question, *not the answer*. This article misses that point.

    Why are we using something for multimedia playback that isn’t designed for multimedia playback? This article’s claim that in order to support HTML5 we’re going to have to spend almost as much CPU power is nonsense, the video codecs won’t come close to needing the same CPU throughput. This article claims that other users can’t figure out how to watch videos without flash because downloading them is too hard, but when HTML5 is natively supported universally that won’t be an issue.

    Yes, we all understand that Flash is at a disadvantage when it comes to efficient performance for multimedia playback. But the fact that we’re stuck with that is that we’re using the wrong tool for the job, not that this is the only viable option.

    Finally, this article claims that the web has “clearly spoken” and said that it prefers Flash to a multimedia-playback dedicated platform. This is completely wrong. There hasn’t been such a universal platform to use. Flash wasn’t used because people loved it that much, it was used because there was no dedicated alternative that actually did the right job. The Internet fell into using the wrong tool for the job because there was no other option, and now that there is light at the end of the tunnel for providing the right tool for the job, I hope Flash gets ditched in that capacity, because, as you point out, Flash is a tool for a different job.

  118. Afief says:

    Thanks for the throughput explanation, I enjoyed reading it very much.

    I have one question(perhaps it was answered in another post, I haven’t been keeping up): What does windows have that makes flash play videos, render elements…etc without a noticable CPU penalty(yes it’s still higher than a standalone player, but nowhere as high as the linux version of the flash player)

    Is there an API windows provides that has no equivalent in Linux-land? Is the codebase for windows just more optimized? Are there issues with other parts of Linux(e.g. the X server)? I would very much appreciate the answer to this question so people can start working towards a solution.

    Thank you very much.