How does video work?

When you open up a web browser and type in an URL, how do you then see a video in the webpage?

There are a couple of different levels of requirements, but it’s not hard to understand how they stack together. You’ve got to have a computer of some type, of course, as well as an operating system. This lets you load applications like WWW browsers.

The HTML markup that a browser processes could be written in any of several ways, but at some point will call up a locally-installed “codec” (short for “encoder/decoder”) to process the incoming video stream, and display it within the browser window.

Most of the time the markup asks the browser to ask Adobe Flash Player to render the video, because Player is already on (nearly) all the world’s machines, and Adobe has paid the licensing costs to distribute a good set of codecs. Sometimes the markup asks browsers to use other cross-browser plugins, such as QuickTime or Windows Media or Real Video. Sometimes the markup asks the browser to call a codec directly, as with the VIDEO tag or iPhone.

But that’s the gist of it — a given video file requires a corresponding local video decoder. It’s a pretty simple thing.

Of course, there are ways to make the story more complicated, by adding more details…. ;-)

For instance, each codec has different strong points, different angles to appreciate. Some of the older codecs have been donated by their creators for any type of redistribution. The more efficient codecs usually charge a license for redistribution to recoup their development costs. Flamewars among codec fans have historically been amusing, but are a separate topic from understanding what a codec is and does.

The server can introduce additional complicating details too. For instance, some servers can deliver different video streams to different client requests, depending on what codecs and transfer rates the clients have. Other servers can confirm that the request comes from a paid-up subscriber (“DRM”) or offer playback amenities such as scrubbing and variable-speed playback. Still other video servers can make use of both the encoder and decoder on the playback machine, for videochat and such.

And, of course, the above is talking about only the most basic type of video, the linear playback stream that we’re used to seeing on movies and TV. Usually you’d also expect, at minimum, some type of playback interface in the webpage (stop/pause/play etc), and there are other interactive features such as captioning, bookmarking, branching, realtime editing, the list goes on. The codec handles just the linear video stream, which is usually only one part of the overall video experience.

But those are the basics: you’ve got one or more video files on the server, and the markup asks the browser to find an appropriate local decoder. That’s how your video gets viewed.

(My apologies if you already knew the above info, and I ended up wasting your reading time… I’m hoping it will be read and understood by more of the weblogs listed on Techmeme, so that in the future all of us readers will end up wasting less time reading soap-opera dramas atop a “web video” storyline.)