Baseline

Tinic thought that some readers might find development process details to be interesting. With that, let me state enthusiastically that I (heart) Autotools!

The Autotools are a suite of utilities including automake, autoconf, and libtool that have become the de facto standard for facilitating the building and distribution of open source software. When you get a package, unpack it, type ‘./configure; make; make install’, you are seeing the Autotools’ work in action (usually; some projects fake it). When the Linux Flash Player team inherited the Flash Player 7 code base, the build system was comprised of a series of ad-hoc Makefiles. There was room for improvement but I didn’t want to autotool the build system at first. It seemed like overkill. Plus, when I think of Autotools, I usually think of a system that facilitates source code distribution. Not necessary since Adobe isn’t open sourcing the Player at this time.

However, after some research regarding how to do some things that I really, really wanted (like proper automated dependency generation), I realized that the Autotools take care of common build system headaches. Certainly, some contend that the Autotools constitute an even larger headache. Fortunately, between the Autobook, the official manuals, and thousands of existing projects already using the tools, comprehension is within reach.

So I autotool’d the Flash Player source to facilitate building on Linux. Then I got my wish of proper dependencies which means, for better or worse, the build system knows that most source files, one way or another, depend on that one particular header file that I just modified.

It’s important to acknowledge that the Autotools really, honestly do know how to just “Do The Right Thing”. After autotool’ing the project, a few strange build idiosyncrasies and minor bugs (and at least one major bug) disappeared, having apparently been artifacts of the legacy build system. Overall, we have much more confidence in the built binaries thanks to the Autotools.

Don’t be afraid of the Autotools.

11 Responses to Baseline

  1. TTimo says:

    Autotools doesn’t take care of build system headaches, it is the build system headache.It’s sad to see so many people still thinking autocrap is the holy grail because they’ve never used anything better like jam or scons.PS: The blog backend here is flacky. Regularly getting mysql errors after posting comments. A lot of times the last posts ( not comments, blogger posts ) just disappear from the page or change order. It’s bad.

  2. TTimo: The blog flakiness has to do with Movable Type’s date handling and my manual corrections thereof. I like WordPress better, but this is Adobe’s blog system.About build systems, it’s easy to say that something you don’t like sucks. Instead, I’d like to hear you defend an alternative on a technical level.

  3. Ondra Hosek says:

    Well, since TTimo is (at least partly) responsible for the building of GtkRadiant, the premier multiplatform mapping tool for a lot of games based on the Quake engine, I would definitely not dismiss him as a tiny-codebase programmer with no experience. Indeed, I checked out the GtkRadiant code a few times and it’s no simple 100-lines-of-code utility, and if he has learned to like Jam or SCons more for keeping his dependencies intact better, then I guess it is not an irrelevant opinion.I think the autotools generate a lot of fatty code. The “configure” shell scripts are looooooong and I also have my implications about M4, the macro language used through and through. Oh, and I find Python nice and clean, which is why I also prefer SCons to the autotools.

  4. Alex Dupre says:

    Mike, please, give us only one definitely answer to this simple question: there will be a FreeBSD release of Flash 9? If the answer is ‘no’, why are you going to ignore thousands of people who requested it for so long?

  5. Sterling Christensen says:

    Can you tell us anything about how easy or difficult it looks like the project will be? What would you say the biggest challenge is?

  6. M-A Loyer says:

    That’s an interresting subject, I’ve been in the same situation lately. I had to change the build system of a big linux project ( thousands files to compile ) and needed a nice, felxible and human readable build system.Until then, autotools was doing the job. Everything seemed to work, but the automake scripts where driving us insane (lots of workaround, ugly fixes and strange automake magic). The whole thing was unreadable, unmaintanable and was not keeping track of dependency while building.After studying the other builds system we choosed SCons.The transition was a bit painfull as autotools have some “undocumented features” that create files and do some thing without being asked and without showing it !Now that the build system is done, I found SCons to be flexible mainly because it use Python for it’s buildscripts. All the automake magic has been replaced by some SCons builder or pyhton code.Ok, now here’s the interreting part for you, the pro and cons :Pros:- Flexible build system- Syntaxe easy to learn (python syntaxe)- Keep track of dependency- Have a lot of featuresCons:- Can be slow for big/very big projects- Documentation is not complete for some advanced featuresThat’s about it, so my recomandation : SCons 🙂

  7. bloody says:

    1. Yes autotools is nice. But when someone else’s ./configure script is buggy and you don’t want to fix it, you just want the darn app to compile then you say damn the autotools, I’ll just fix the generated makefiles by hand.2. And like Alex Dupre said, we’d like to know about platform support. But, as for myself, I’m asking about linux-amd64 instead of FreeBSD.

  8. TTimo says:

    Damn .. I have a friend working at sixapart and I find myself complaining about their blogging products :-(Anyway. I tend to rant against Autotools any occasion I get. I’ve had too many frustrations with it.If you are writing software and are not already fluent in Autotools ( or you want a change ), SCons is a great alternative. It’s very flexible and powerful. After using it for a bit, you’ll realize it’s *clever*, simple as that.Then someday you’ll find yourself staring with incredulity at an autotools build system, the multi-layered approach ( .am, automake -> .in, autoconf -> makefiles ), the obscure macro language. It does the job, but it’s nasty.

  9. I appreciate the feedback on build systems and I have taken some time to look at alternatives. Internally, of course, we will stick with the Autotools since they are working very well for us. But I will keep SCons in mind in case I need to kick off another project in the future.

  10. Jakob Petsovits says:

    Just for the record, KDE (the whole project) has recently completed their switch away from autotools, which was a huge undertaking and wouldn’t have been done if autotools hadn’t been a major headache in every respect. Until last year, it seemed like SCons would be the build system, but there were some problems with it (I don’t know the exact details).So now KDE4 will be building on CMake which seems to be quite cool, relatively easy and very flexible. It has support for all major platforms (including Windows) and can generate makefiles, VS.Net projects or KDevelop projects.With KDE4 building on CMake, it will for sure get much exposure soon.

  11. Christopher Dunn says:

    I have used scons quite a bit. Recently, I looked into some other build systems because I wanted some simplification. Here is a summary:* omake+ Starts with standard Makefile syntax, and adds functions and scoping.+ Written in Ocaml, and therefore runs fast.- Too domain-specific. It’s not Ocaml; it’s a special language.- Missing lots of useful features.* rake+ Pure Ruby! Learn rake and you’ve learned Ruby.- Missing too many advanced features.* rant+ Pure Ruby!+ Picks up where rake left off. Look at the rake v. rant comparison page.- Does not do caching.- Does not easily handle side-effects.* scons+ Easy to get something simple working.- Hard to get something moderately complex working.+ For really hard things, it shines!- Can be very slow (mainly because all MD5 checksums are calculated and compared everytime).I keep coming back to scons, for a variety of reasons. First off, it doesn’t just allow MD5 checksums, it actually uses them for something: caching! The caching system is wonderful. It looks into everything necessary to build a file, including special flags and dependencies, so it’s very reliable. It’s not as efficient as cachecc or ccache, but it’s very helpful for re-building. Several build systems allow MD5 checksums for dependencies, but few support caching.Second, with a bit of work, it can handle amazingly complex operations. For example, SWIG produces extra files (.py and .pm) which most build systems ignore. In fact, the default SCons rule for SWIG ignores these. But, once you figure out how, you can write your own elegant “emitter”. Then you can write your own “scanner” for the .i files. It’s not trivial, but once its done, it’s all part of the caching system. All other build systems that I’ve tried have trouble with these complex, multiple-target tasks, even without caching.Third, it handles multiple architectures and multiple directories better than anything else. I want all builds in separate directories, and I want the project directories to be oblivious to the build directories. SCons handles this by loading sub-directory SConscript files repeatedly, once for each architecture and set of build flags.The main problem with SCons is that its concept of a Builder is not general enough. It should be possible to chain Builders by passing one to another. Instead, SCons supports chaining with a sort of hack, a special parameter to the Builder constructor, which never does quite what I want. What I want to do is to tell the SWIG builder my .i interface file, and then to let it figure out dependencies, produce the .cc files, figure out the dependencies on that, produce the .o file, and finally produce the .so file, with -rpath expansion of .so dependencies (on Linux). What I end up doing is specifying a target for a custom Builder, and passing that target as the source to another custom Builder. It’s not automatic, so it’s annoying. However, once it’s all specified, caching and dependency-checking are automatic.