API Review
What follows is a list of the APIs that the current Linux Flash Player 9 development version is using. I debated whether it was appropriate to publish this information. Then I remembered that anyone who knows what they're doing should be able to figure this stuff out by themselves anyway once the final Player is released.
- General graphics: X11
- GUI elements (dialog boxes): GTK
- Audio I/O: ALSA
- Camera input: Video4Linux, API version 1
- Threads: POSIX threads
- non-HTTP Networking: BSD sockets
- SSL: OpenSSL
- IME: you know what? I don't think we've settled on this one yet...
There are undoubtedly others that I'm missing... dlsym() and friends for dynamically loading shared library functions to avoid static linking of LGPL libraries.
Now that I think about it, how come there are so many layered methods for doing basic A/V operations but one for doing network communication (BSD sockets)? I have searched for higher level general libraries on top of the sockets. I know that some criticize raw ALSA programming as being comparable to ASM for audio operations. BSD sockets strike me as the ASM of network programming.
I would now like to open this thread for readers to voice their concerns over API choices.
Comments
I believe you've made good choices. I'm sure you'll get a lot of complaints for the use of GTK, but the reality of it is that it's a good choice. Besides, I assume that a large percentage of KDE users have GTK installed anyway and it's not difficult for those who don't to install it.
Posted by: mark pinto | July 24, 2006 01:08 PM
All those things sound like good targets to me. The only hangups may be the use of V4L v.1 instead of v.2 but if Adobe decides to make more frequent releases of Flash player than they have in the past, it may not be an issue (I hope we don't have to wait years for a new version again).
Alsa will probably give the BSD guys a headache, but it's the better choice for Linux.
Do you know if you guys will be able to offer versions with GTK statically compiled to keep the QT/KDE guys happy?
Posted by: James C | July 24, 2006 01:20 PM
Can I suggest 'ARTS' Sound Architecture?
The linux xine player is a good successful case for this.
Posted by: Fernando | July 24, 2006 01:25 PM
> "GUI elements (dialog boxes): GTK"
Do you mean "GTK 1" or "GTK 2" ?
> Fernando :
ARTS is dead, so it's better to stick with Alsa (or another multimedia framework, why not)
Posted by: Jonathan Ballet | July 24, 2006 01:31 PM
All I can say, GTK+ oh my god! Good luck with_that_weird_long_api . Qt anyone? Adobe owns Qt licenses did you know that? ;)
Posted by: cartman | July 24, 2006 01:40 PM
I'm a bit curious as to why you picked ALSA over GStreamer, considering that you went you also went with GTK
Posted by: mark pinto | July 24, 2006 01:53 PM
I think V4L1 may have been formerly deprecated. You should check that - the player may be partially non-functional in a years time.
For higher-level networking, I think glib (the gtk support library) has a network library - gnet. Not sure how standardised it is yet, though.
gnet seems to have windows support too, so there is now *one* standard GUI and network library for both Linux and Windows (GTK+GNET) - that could save you some money in the future.
Posted by: Tristan Wibberley | July 24, 2006 02:04 PM
glib, which you are already using through GTK+ contains a socket abstraction if I understand the API docs correctly.
Since most of the APIs on Unix are in C, there is often no abtraction needed, while there is usually always an abstraction API or binding for other languages like C++, Python and so on.
Posted by: Kevin Krammer | July 24, 2006 02:11 PM
Sounds like a bunch of good choices to me too.
For finding out which libraries are used, ldd (for the dynamically) and nm (for the statically linked ones) have told me quite a lot. On Mac OS X, it's otool's job. They also have an strace equivalent, ktrace, but I think strace is much better.
Posted by: Ondra Hosek | July 24, 2006 02:28 PM
There are APIs that sit on top of the BSD sockets, but they can be a little hard to find because they're usually a part of a package.
If you haven't written too much networking code yet, you might wish to check out SDL_net.
Posted by: Rene Horn | July 24, 2006 02:29 PM
I think that these are very good choices. And in this case I would also say that GTK2 (I sure hope it is not GTK1) is a better choice since most people will use Flash player in Firefox browser which is also GTK2 and it will feel more integrated that way. Other then that I found Qt to be a lot easier and nicer to work with, when I tried to learn one of the Linux GUI toolkits. But then again if programmers at Adobe working on Flash already know GTK well I guess it would only delay things if they needed to learn Qt in addition to all work.
Posted by: Jure Repinc | July 24, 2006 02:47 PM
Sounds good to me, assuming you mean GTK2. Personally I'm a KDE user and prefer QT from an integration\eye candy\end user standpoint, but since Firefox is GTK2 you're better off with that toolkit for flash. Definitely looking forward to the ALSA sound sync goodness! Don't listen to the guys asking for aRts and GStreamer - aRts isn't that great and I think development has largely stopped, and I for one think GStreamer is another set of unnecessary libraries on top of the same old thing and don't have it installed. You've made good choices for the widest compatibility, IMHO (although of course the best way to make it as compatible as possible would be to release the source ;)
Posted by: Arren Lex | July 24, 2006 03:07 PM
Both GTK1.2 and GTK2.x are supported, you can select specific .so you want through an environment variable (FLASH_GTK_LIBRARY).
Otherwise, by default it will look for 'libgtk-1.2.so.0'. If that fails it will look for 'libgtk-x11-2.0.so.0' and then finally for 'libgtk.so'.
Posted by: Tinic Uro | July 24, 2006 03:20 PM
For GUI elements, I hope you are using GTK2, as people are now finally getting rid of the original GTK. For camera input, V4L will be out of the kernel likely before the release of Flash, or close to it. A better idea so you don't have to tie yourselves to Linux and V4L2 is to just use GStreamer. Same for audio with ALSA, as some people want arts, some want OSS, some want pulse audio, etc... using GStreamer for the audio output will have their system automatically use the correct audio output system. As others have mentioned, GLib also provides network abstraction, but BSD sockets also work fine for it. I hope you guys choose GStreamer and make it easier for everyone to have Flash ported to other platforms like BSD and Solaris, without you guys having to do it yourselves, without having to make anything open source if you don't want to.
Posted by: DannyB | July 24, 2006 03:30 PM
Three comments:
1. V4L version one is not very useful. According to Linux's feature removal schedule, it is just about to be removed from the kernel entirely.
2. Please use GTK 2.0 in preference to GTK 1.2!
3. What's with the dlopening? I'm not sure why you don't just dynamically link to the libraries in question. Using dlopen will not affect the extent to which the Flash plugin binary will be considered a derived work of the libraries you use; Is there a technical advantage that I am not seeing which outweighs the additional complexity and fragility of doing all your library linking by hand, at runtime?
Posted by: Sam Morris | July 24, 2006 04:18 PM
@Sam Morris:
We go the dlsym() route because, per my experience, if you rely on the OS to do it for you, and a particular library or function does not exist on the system, the dependent module will not load. Thus, if the user does not have ALSA installed, the plugin will simply -- and silently -- fail to load. It gets to be a philosophical matter-- is it better for the plugin to silently fail, or to proceed but not play any sound? There will be tech support issues in either case, but I think the latter option will be easier to diagnose.
Posted by: Mike Melanson | July 24, 2006 04:22 PM
why would you try to load gtk1 before gtk2? gtk1 = legacy apps only. if you can use gtk2, go for it.
Posted by: jimbo | July 24, 2006 04:29 PM
@jimbo:
Actually, Tinic was a bit off. The policy, after checking for that env var, is to query the library loader for GTK's major version number and load the appropriate library, 2.0 or 1.2, as indicated.
Posted by: Mike Melanson | July 24, 2006 04:45 PM
I still beleave that Adobe would help GNASH instead of create a flash player for linux. It will won nothing with that non-free flash player.
Posted by: llp | July 24, 2006 04:58 PM
My only "concern" is, as stated above, video for linux 1, which is being almost completely removed from the kernel, also stated above. Also, go with GTK2 for sure.
The rest of the choices look good. Most "modern" distributions come with both GTK and Qt nowadays anyways, with GTK being favored as more of the standard.
The same for the sound, most modern distributions use ALSA here instead of OSS. As far as I know, ALSA and OSS are the only two sound layers on the same level. The rest (GStreamer/Arts/ESD/etc.) Sit on top of either ALSA or OSS, and are mainly in existence to make multimedia processing easier.
As always, keep up the good work, I look forward to seeing the release.
Posted by: Chester | July 24, 2006 05:18 PM
I don't know a whole lot about this stuff, but why not use OpenGL for the graphics? If I'm being stupid, let me know, but it seems like that would be a better choice because it supports hardware acceleration where possible and mesa where not.
Posted by: Justin | July 24, 2006 05:24 PM
I'd say nice choices. Except for V4L version 1. That is getting a bit old and if it's flagged for removal, shouldn't probably be used. On the other hand, more conservative distros may not have V4L version 2 compiled in yet.
And if glib has network support, I think that perhaps it would be a good idea to check it out as it would bring consistency to the whole thing. More glib and gtk yay!
Posted by: Bloody | July 24, 2006 05:27 PM
no complaints at all, aslong as we get it and it's possible to get it to work properly (without MAJOR bugs, like the current player's A/V desync) we will figure it out. And there will be tutorials all over the web on how to get it working.
Posted by: Anders Aagaard | July 24, 2006 05:33 PM
- You might consider GStreamer for the multimedia parts (Audio/Camera/Video?). It is an upcoming standard, not bound to a desktop, abstracts stuff away for you and enables people to use present or future backends of their choice (v4l, v4l2, alsa, oss, sunaudio...) without you having to bother about it. Roughly think of it as the "DirectX" (/slapmyself) counterpart. The Linux "Desktop" was missing a decent unified multimedia framework for quite some time...
- Graphics: X11
While there might be ideas to use cairo or derivates for the vector rendering, Flash's range of graphical requirements prolly applies best onto a(/the already present?) custom renderer. However sure something to revalidate as soon as the F9 player is done. I guess it's a general question for Flash to finally use some acceleration on each platform (OpenGL, Cairo on glitz, ...) and not bump the CPU graph. ;)
- GUI: GTK2
Guess it is the best for both GNOME/KDE worlds and blends fine with Firefox ;)
- Audio: GStreamer
GST Gives people the choice to choose their audio backend and integrates with modern linux desktops.
- Camera: GStreamer
Same as with audio; people can choose the backend of their choice and you don't have to update stuff nor care for any kernel removal of v4l(1). Also seems correct in the longterm.
- IME: Clueless... Anyone? Something in GTK?
... rest looks ok. If you stick to your current setup just try to use v4l2 and check glib/gtk/gdk for other stuff you might need.
Posted by: FunkyM | July 24, 2006 06:02 PM
Very good choices, excepting, as said, v4l1. ;) Please use v4l2 and gtk2.
About the sound library, it's best to use ALSA. If you use GStreamer, it's one more (really big) dependancy you would put in the package.
I found interessanting the commentary by Mike Melanson. I had no knowledge of the existance of gnash. It would be great if Adobe put his developers to help gnash instead of writing a closed plugin. But I know it is out of control of you developers.
Maybe us users could create an undersigned for convincing Adobe that an opensource plugin solution would be most lucrative for him (turning easier porting and spreading swf to new platforms). :)
Posted by: Paulo Matias | July 24, 2006 06:07 PM
The choices of API seem fine, apart from the V4L which is meant to be deprecating soon.
However, I know your working on the new flash player and such, and thats great, but, if it is meant to be release some time next year wouldn't it make sense to just upgrade the current flash player to fix the crippling A/V sync problem right now? I'm not sure how hard it would be to fix, but it gets damn annoying having to right click, and then pause and play.
It sucks not being able to see alot of the new flash content out there, but for a crutch till the player is out and ready, having A/V sync would be great.
Also, using GTK+2 would be the gui wrapper to use I believe, as it is the lightest compared to QT and its LGPL. Just saying. It is possible to create a simple dialog application in under 20 lines tops.
Good work so far.
Chris
Posted by: C. Sawczuk | July 24, 2006 06:13 PM
Makes sense except for the V4L1 choice over V4L2.
Posted by: EnviroTO | July 24, 2006 06:27 PM
@Justin
OpenGL is not a good option right now for browsers. Imagine every small .SWF in a HTML page using up a full OpenGL context. Every single OpenGL context uses up a lot of resources and requires some time to start up. So every time you would go to a new page you would have to wait a second for it to initialize or wait several frames for an existing context to be updated for the new location. Talk about a bad user experience.
In addition you end up with scrolling synchronization issues (since the rest of the web page is usually straight X11 and OpenGL does not work together well with the X11 scrolling APIs).
It would also be a nightmare to fix your system if something is wrong with your browser or OpenGL installation. OpenGL still works best for dedicated full window or full screen applications.
Ideally browsers should provide native OpenGL support before the use of it makes sense for something like the Flash Player. For now we try to stay as compatible as possible with browsers out there which still assume that plugins only do X11.
Obviously our long term goal is to made advances on this front. We understand that OpenGL offers performance benfits in specific areas.
Currently we use XShmPutImage as the main blit function, with XPutImage as a fallback if the Shm extension is not available.
As for using OpenGL to render the vectors, there are not too many cards out there who can currently handle that in an acceptable fashion. It'll work for simple movies (a few squares and circles maybe), but as soon as you add loads of vector text the software renderer is still faster. We have done extensive tests on OSX to show this. In addition designers are very particular about how their content renders. Every pixel counts for them, so we have to get as close as possible to what they expect. This is something you can't rely on with the current generation of cards which people have out there. We can't ask everyone to buy new graphics cards just for Flash.
Posted by: Tinic Uro | July 24, 2006 06:32 PM
What about Cairo? It draws vectors with antialiasing etc. Perhaps Gtk2 uses it for widget drawing, and it uses OpenGL acceleration when available, via Glitz, otherwise falls back to software rendering.
Posted by: Paulo Matias | July 24, 2006 08:09 PM
For an IME I would just like to put my vote for SCIM.
Posted by: Anonymous | July 24, 2006 08:32 PM
As I know, Ubuntu, Fedora ... use SCIM as there default multi language input.
SCIM support so many languge via its plugin module style. I heard that It use XIM protocol.
Care about XIM, SCIM for multilanguage input.
Posted by: Mait | July 24, 2006 08:53 PM
@Paulo
Cairo is not fully compatible with the Flash rendering engine. As I said designers want pixel level accuracy, nothing less, including ALL the rendering bugs Flash has as strange as that sounds (one famous example: http://www.misuseit.com/bitmapbug/). We version check EVERY change in the renderer so that old SWFs will not look different (we got into big trouble many times for ever so slightly changes we made, even if they fixed a bug). While it would work great for simple animations, we would run into countless of issues with current content out there as users really fiddle with every single pixel. 95% cross platform compatibility is what matters for us, that's the main value of Flash IMO.
Flash 8 makes the situation even worse because of new rendering features cairo does not have (filters like blur, drop shadow etc, focal point radial gradients, stroke pixel hinting, SVG like blend modes etc. etc.). All of this would have to be redone in some form or another.
I believe cairo uses a painters model for rendering meaning shapes are drawn on top of other shapes (never looked at the source though) vs. Flash which is using a scanline based renderer, e.g. no scanline is ever touched twice during the rendering process. This has implications, making content look vastly different in some cases (did you know that Flash has a 12 layer limit per scanline span f.ex. and drops any additional ones?). The devil is in the details, not the high level APIs.
The software rendering performance of cairo is not what Flash can do on the desktop last time I checked (at least on Windows and MacIntel. FP7 on Linux was absolutely terrible in that regard, I admit. It essentially had NO optimizations, even the gcc options were not ideal for this code. :-(
I am really not trying to dismiss cairo here, it's a wonderful library solving a lot of cross platform issues. Cairo works perfectly well for a _lot_ of purposes and I believe any SVG implementation should use it.
Posted by: Tinic Uro | July 24, 2006 09:13 PM
Using gtk2, never worry about input method. gtk2 itself support multiple input modules that is changable at runtime. Mentions about SCIM or whatever are out of coverage. It's up to each end user.
If with gtk1, don't mind too. gtk1 only support XIM. (And please don't use gtk1 as possible :p)
Posted by: kz | July 24, 2006 09:51 PM
V4L version two please. It's pointless to code to a deprecated API. V4L v2 already works just fine ... I have a PVR card and 1.3 megapixel webcam using it.
If GTK*, absolutely positively use GTK v2... or FLASH WILL NOT WORK ON ANY SYSTEM WITH XGL INSTALLED (b/c GTK v1 does not to transparency). You'd have to use crazy launch tricks to get Flash working in XGL on GTK 1.
* From my brief poking around, QT seems alot easier to learn and code ... you a masochist? It's like the Python of GUIs... easy yet powerful.
Posted by: Tux | July 24, 2006 10:34 PM
Great news!
However, personally, I'm a QT fan so its disappointing to see GTK used over QT. Hopefully it will be static linked.
Posted by: Greg Loscombe | July 25, 2006 04:26 AM
How about SDL?
Posted by: Apreche | July 25, 2006 04:33 AM
Nothing mentioned about 64-bit version of flash player 9 yet. Is it gonna happen or not? :)
Posted by: Kulin | July 25, 2006 04:38 AM
Just want to ask if this will be available for 64bits linux. Will you make a 64 bit version?
I can´t say anything about your API choices since I don´t understand too much about it, sorry.
Posted by: Sebastian Bassi | July 25, 2006 04:44 AM
Using GTK is quite a strange option, given that Adobe uses quite a lot QT. What's more, given that QT>4.2 integrates very well under KDE and GTK based DEs, this would be much better. Using GTK means it will not fit at all under KDE, and will be very weird under konqueror.
Posted by: lezard | July 25, 2006 04:56 AM
All good with the exception to ALSA. Use Gstreamer. There will always be some odd-numbered person who needs to use a certain back-end for whatever reason and every user should be considered to be as important as every user you have.
I can see quite clearly nothing there is processor dependant. So there sure as hell be some PPC PPC64 and SPARC ports.
Posted by: Jon | July 25, 2006 05:15 AM
Agreed with the people that say "go for GStreamer". Amarok, BMPx, Totem, etc., almost all of the audio/video players out there have a GStreamer back-end as an option or are working on implementing one. Please consider this option very carefully. The rest looks great IMO!
And don't worry about the people complaining that the player should be Qt, they can always use the QTK-Qt theme engine. ;-)
http://www.freedesktop.org/wiki/Software/gtk-qt
Posted by: 1c3d0g | July 25, 2006 05:31 AM
Why not use Qt? It can help you with some of the network stuff as well ;)
Posted by: Jesper T. | July 25, 2006 05:57 AM
* GTK is a good choice considering firefox uses GTK2. though using GTK1 instead of GTK2 would be shooting yourself in the foot. it would be like buying a brand new car... from 1987.
* for IME use GDK (part of GTK). you are already getting your hands dirty with GTK, so use it's input methods. with GDK you can have any kind of input there is and it's easy. support for keyboards, joysticks, tablets and rocks, very small rocks.
*** did i mention that GTK/GDK is cross platform? ***
* ALSA for audio was a _GOOD_ choice. i think the jillions of comments to use ALSA in the original post got through. horray!
* the camera support seems a trivial feature to me but V4Lv2 is a much better choice than V4Lv1. i would make this a very low priority as ive never come across flash that needed a webcam.
Posted by: Anonymous | July 25, 2006 05:57 AM
Please be aware, that posts on Digg are user created and driven. Changing your title wont make any difference. Its like just Slashdot but not edited. When a user likes a story they "Digg" it. This will draw more traffic. If they dont, the "Bury" it, this will bury the story. Its a great site, but I can understand if you have been on the wrong side of it. Not very much like fun, not for a serious site like this one.
Posted by: Super Carrot | July 25, 2006 06:01 AM
Well I would like Gstreamer for input/output as this is where the future seems to go for Linux desktops in general. As long as you use ALSA and sync is good though, it's ok. ;-)
Otherwise I don't have much comments, ppl have already warned you about v4l.
Looking forward to this, when can we look forward to an alpha to test (please)?
I'm currently running Flash 9 via CrossOver Office, but that doesn't work at places like Youtube that tries to autodetect with flawed methods (the player works just fine).
I really look forward to a native version, and while I would much prefer it to be open, I respect that this is not your descision.
Posted by: Stoffe | July 25, 2006 06:10 AM
For video out, audio, network you could have just used SDL. Also, the nice thing about SDL, is it ports to almost all platforms...
Posted by: Jacob Brown | July 25, 2006 06:18 AM
I've setup Japanese input support on a number of distro's (Mandrake, Slackware, Gentoo) using SCIM and whatever it was that was recommended before that came out. In my experience XIM has always been the fallback option, as it is part of the X protocol. IIIMF [www.openi18n.org/subgroups/im/IIIMF/]
seems to be writing a replacement, but it looks doubtful that it will be included in the 7.0 series of X.org releases. (According to a PDF in Googles cache, Sun seems to be pushing IIIMF).
Posted by: Levi Durham | July 25, 2006 06:24 AM