[Time code];

Fanaticism consists in redoubling your effort when you have forgotten your aim.

George Santayana, Life of Reason (1905) vol. 1, Introduction

A few weeks back, I asked the Twitterverse “When did user-facing Linux become completely and officially irrelevant? Seems like a fait accompli, doesn’t it?”. With the OSS community’s reaction to the iPad, I’m now sure of it.

And by reaction, I’m of course thinking of the response by the FSF’s John Sullivan to Steve Jobs’ much-discussed Thoughts on Flash. Sullivan’s piece is a straightforward “pox on both your houses” take, damning Apple and Adobe alike for not adopting the One True Faith of Open Source Software.

I had allowed myself to hope that the OSS community would take the hugely successful launch of the iPad and the flight of developers to the iPhone OS platform, even with Apple’s cravenly self-serving App Store policies, as an opportunity to take a good, long look in the mirror. With iPad sales now already estimated at over a million, you have to wonder how soon use of this nascent platform will exceed that of desktop Linux, if it hasn’t already. What should the open source community think of this profound preference for the unfree over the free? I would think they’d see it as a very public, and very obvious repudiation of the party line, suggestive of an urgent need for some soul-searching and time to re-examine how the OSS community is engaging the world at large.

Sullivan’s piece, sadly, shows no such self-awareness. It’s the usual total detach from real people that we’ve come to expect from True Believers. Indeed, the last third of it is the all-too-familiar zealous denunciation of all proprietary software, calling on readers to adopt Linux and Ogg Theora.

Seriously, Ogg? They’re still clinging to the “everything should be in Ogg” kick? OK, let’s ask the people who should know on this one, content professionals, to see if they buy in at all. DV magazine’s website logs 0 hits in searches for Theora and for Ogg, StreamingMedia.com’s 11 hits (versus 248 for H.264) are largely related to codec politics and not mentions in training articles or product reviews, and legendary retailer B & H assumes the search term theora is a misspelling of “theory”. No matter how many essays and insults the zealots spew forth, they seem unable to generate any inkling to know or care about Theora among video professionals.

In effect, the FSF’s position is to tell the world “screw you for not wanting the stuff we produce”, oblivious to the fact that the marketplace of ideas is very clearly telling them “screw you for not producing the stuff we want.”

The quote at the top of this article was often cited by Chuck Jones in describing the character of Wile E. Coyote, who was never afforded a moment of clarity to reconsider his hunting regimen, the inherent dangers therein, and the prospects for a way of life that didn’t involve falling off cliffs every 30 seconds. I fear the OSS community is so wrapped up in their own self-righteousness, they are similarly immune to equally-needed moments of humility, clarity, and perspective.

Mike Shaver’s blog about Mozilla forgoing H.264 support in the HTML <video> tag is getting a lot of play, predictably from the /. crowd.

It’s a shame, because it’s a childish and facile argument. Here’s the gist of it.

For Mozilla, H.264 is not currently a suitable technology choice. In many countries, it is a patented technology, meaning that it is illegal to use without paying license fees to the MPEG-LA. Without such a license, it is not legal to use or distribute software that produces or consumes H.264-encoded content.

In short, the very idea that something is patented and requires licensing is prima facie proof that it is intolerable. Going on:

These license fees affect not only browser developers and distributors, but also represent a toll booth on anyone who wishes to produce video content. And if H.264 becomes an accepted part of the standardized web, those fees are a barrier to entry for developers of new browsers, those bringing the web to new devices or platforms, and those who would build tools to help content and application development.

Yeah, can you imagine if any other technology were encumbered by patents? They’d have to pass on the costs to customers too! Imagine if television were patented, or if automobiles involved 100,000 patents… surely those products could never exist and never be affordable.

There is a case to be made against patents and intellectual property as a whole, but this blog doesn’t make it. Instead, it blithely refuses to acknowledge that we do live in a world of IP, decrying its costs as if they are out of the ordinary or unjust. Ultimately, it flees back to the intellectual dead-end of “everything should be in Ogg”, a stance so untenable that even ESR conceded it was a non-starter, four years ago.

A final irony: by refusing to support H.264, Mozilla bolsters the primary alternative for video on the web: the Flash plug-in, which is not just patent-encumbered but proprietary, available only from Adobe and only in those environments where it serves the company’s strategic ends. Shaver, to be fair, admits that proprietary plug-ins are bad too, but declines to say they’re far more bad than patent-encumbered standards. Instead, he holds out for a pollyannaish vision of completely free web video technology, naming Ogg as his moral standard-bearer, without acknowledging Ogg’s lamentable but obvious feet of clay.

Well, another go ’round with this: HTML5 won’t mandate Ogg as universally-supported codecs, and the freetards are on a tear. I was going to follow up on a JavaPosse thread about this, but I hurled enough abuse onto their list last week.

It’s abundantly clear in this blog that I don’t think Ogg is the solution that its supporters want it to be: I have a whole tag for all the posts where I dismiss Vorbis, Theora, and friends. Among these reasons:

• I don’t think it’s technically competitive.

• It certainly isn’t competitive in terms of expertise and mindshare, which is vitally important in media codecs: there’s a much deeper pool of shared knowledge about the MPEG codecs, which leads to chip-level support, competition among encoders, compressionists who understand the formats and how to get the most out of them, etc.

• Its IP status remains unclear. With even MPEG-4, following a lengthy and formal patent pooling process, attacked by AT&T’s claim of a submarine patent, I have no reason to think that Ogg wouldn’t face similar claims, legitimate or not, if there was any money behind it, which there isn’t.

• If I go to my former colleagues at CNN or in Hollywood and say “you guys should use Ogg because…”, there are no words in the English language that plausibly complete the sentence and appeal to the rational self-interest of the other party.

On this last point, I’ve got an ugly analogy: just as proponents of “Intelligent Design” are people who don’t really care about biology beyond the point at which it intrudes on their religious belief, so too do I think Ogg advocates generally don’t know much about media, but have become interested because the success of patent-encumbered formats and codecs is an affront to their open-source religion.

Ogg’s value is in its compatibility with the open source religion. It has little to offer beyond that, so it’s no surprise that it has zero traction outside of the Linux zealot community. Even ESR realized that continually shouting “everything should be in Ogg” was a losing strategy, and he said that three years ago.

I think the open source community would like to use HTML5 to force Ogg on the web community, but it’s not going to work. As others have pointed out, there’s little reason to think that IE will ever support HTML5. Even if they do, the <video> tag is not going to replace Flash or Silverlight plug-ins for video. Despite my initial enthusiasm for the <video> tag commoditizing video, I see nothing in the spec that would support DRM, and it’s hard to imagine Big Content putting their stuff on web pages without DRM anytime soon. And while you can put multiple media files in a <video> tag easily enough, having to encode/transcode to multiple formats is one reason that Big Content moved away from the Real/WMP/QuickTime switch to the relative simplicity of works-for-everyone Flash.

I’m tired of being lectured by computer people about media; it’s as ludicrous as being lectured about computers by my old boss at Headlines. Just because you use YouTube, doesn’t make you an expert, any more than my knowing how to use a username and password means I understand security (seriously, I don’t, and doubt I ever will). Kirill Grouchnikov pretty much nailed what computer people think good video is with this tweet. I’ll add this: there are probably a thousand people at Sun who understand 3D transformations in OpenGL, and maybe five who know what an Edit Decision List is. So they go with what they know.

A couple years back, I gave a JavaOne BoF in which I made a renewed call for a Java Media library which would support sample-level access and some level of editing, arguing that enabling users to take control of their own media was a manifest requirement of future media frameworks. By a show of hands, most of the developers in the audience thought it would be “enough” to just support playback of some modern codecs. JavaFX now provides exactly that. Happy now?

People who actually work in media don’t mind paying for stuff, and don’t mind not owning/sharing the IP. Video production professionals are so accustomed to standardizing on commercial products, many of them become generic nouns in industry jargon: “chyron” for character generators, “grass valley” for switchers, “teleprompters”, “betacam” tape, etc. Non-free is not a problem here. And if your argument for open-source is “you’re free to fix it if it doesn’t do what you want it to,” the person who has 48 shows a day to produce is going to rightly ask “why would I use something that doesn’t work right on day one?”

The open source community doesn’t get media. Moreover, it doesn’t get that it doesn’t get media. The Ogg codecs placate the true believers, and that’s the extent of their value.

Surfin’ Safari notes initial support for the HTML5 <video> and <audio> tags in their latest nightly builds.

Indeed, if you have a browser that supports the video tag, then you (hopefully) can see an autoplaying video here:

There’s been a little bit of controversy over the fact that calls for inclusion of the Ogg formats have been removed in more recent versions of the spec. Section 3.14.7.1, “Video and audio codecs for video elements”, currently reads:

It would be helpful for interoperability if all browsers could support the same codecs. However, there are no known codecs that satisfy all the current players: we need a codec that is known to not require per-unit or per-distributor licensing, that is compatible with the open source development model, that is of sufficient quality as to be usable, and that is not an additional submarine patent risk for large companies. This is an ongoing issue and this section will be updated once more information is available.

That more or less matches my take on Ogg, which is that it poses an unknown patent liability risk: the /. mob insists it’s patent-free, but how the hell do they know? They don’t; they just want it to be so, because it suits their worldview. And Ogg may indeed be patent-free, but I don’t think anybody knows for sure, and even so, proving it would be expensive. To top it all off, Ogg just isn’t that popular or useful outside the warm bubble of Linux zealotry.

Still, there’s a huge need for at least one video and audio codec to be available more less everywhere, or at least for one class of devices: i.e., one codec you can expect all desktops to have, one for all phones, etc. To just dump out to “whatever QuickTime supports on the Mac, whatever Windows Media supports on Windows, etc.” ends up moving the problem, either to the web author (who has to sniff the OS from the user-agent and write the tag on the fly… to say nothing of hosting multiple encodings of every clip) or to the end-user.

It’s funny, because while the HTML5 <video> tag should displace Flash as the only practical option for web video — something that’s become screamingly obvious in the two years or so since YouTube launched — it might not, if it gets tangled up in codec hassles. The remarkable thing about Flash Video isn’t that it’s good (it’s not), but that it’s consistent and available on all Flash-enabled desktops.

That’s turned out to be a much bigger deal than the quality of competitors like H.264 and WMV, or the fact that other approaches could support many more codecs. Flash doesn’t try to support every codec under the sun, or even offer extension points for third parties to do so, but it doesn’t matter — with a known-viable video codec, content providers can just push their content with the package-deal of FLV and the Flash plug-in. Sure, the QuickTime plugin, a QuickTime for Java applet (or even a JMF applet, fercryinoutloud) could support more formats and codecs than Flash, but the typical use is not a general-purpose “play arbitrary content” application; the web-embedded player is usually meant to play the content from a single content provider, who’s perfectly happy to use a single, sub-optimal format, if the alternative is having to encode everything a dozen ways from Sunday to support the various OSes and devices.

Which makes me think that Flash’s ubiquity as a web-embedded video player won’t be threatened by HTML5, so long as there is neither a de jure nor de facto ubiquitous video codec for HTML5. Ironically, while H.264 might be the best candidate for that, Flash is already supporting it too.

Which leads me to an idea: if you were writing a browser on a platform without H.264 support from the native multimedia library, but you had Flash available, could you just pull the Flash player into service on the fly and have it play the H.264?

A couple of interesting comments from the first half day of Mobile & Embedded Developer Days:

• Two speakers were hesitant to even use the word “android” at this Sun-hosted event, demurring with apologies like “I know that’s a dirty word around here”. Here’s the thing: both of these speakers were with outside companies. None of the Sun speakers have mentioned Android at all. All of which leads me to think the enmity towards Android among Java’s old guard is even worse than is generally known.

• All the Sun people I’ve talked with about Java Media… and remember, I’m trying to retire from this topic… have referred over and over again to what they call “the codec problem.” Remember, I think the issue is one of capability, but we’ve been over this (the Java types generally think playback-only is “good enough”), but that’s neither here nor there. People I know at Sun, even those in no way associated with Java Media, talk about “the codec problem” or “the codec question”, implying that what Sun is wrestling with is what video (and audio?) codecs to license.

  So James Gosling gave a brief keynote and brought up a slide on the Java FX platform. One line item was a pair of key features: media and a scene graph. He noted that the scene graph is looking great (and it is; it’s already open-sourced on java.net), but on media, he made only a very fleeting reference to “dealing with the nightmare of MPEG-LA”. Which tells us two interesting things:

  1. They’re looking at some MPEG-LA-licensed codec, probably H.264, and
  2. it’s not going well

  Is it safe to assume that the price is too dear for Sun? Maybe. Sun did pull MP3 out of the Java Media Framework years ago when the license proved unaffordable (it came back later, but now playback-only). Still, all the other companies that are serious about media — Apple, Sony, and now Adobe — have decided it was worth it.

  What about the alternatives? You have to wonder if they’re at least thinking about VC-1, especially given the new cozy relationship with Microsoft. The hoi polloi will probably demand Ogg, though I think that would be crazy.

I was just in a chat on the #javaposse IRC channel, and I mistakenly walked into the whole Java Media topic, of which I have said far too much already (1, 2, 3, 4, 5, 6), and which I’m really not interested in discussing further, as the Java Community has given my basic argument — that Web 2.0 will call for a read/write, cross-platform, web-embeddable media library, and if Java doesn’t step up, then it will be Flash — a fair consideration, and has rejected it, saying that “all most people need is playback.”

Fine, I say, but all the consulting work I’ve taken on in the last few years has involved editing, capture, or both, so if Java Media’s big comeback is going to be playback only, then I need to do something else, either single-platform (QuickTime), or Flash.  So, see ya.

And at this point, the playback-only crowd gets into the whole “how do we support all the patent-encumbered codecs out there” snit — overlooking the fact that Flash succeeded with a small number of not-particularly-good codecs — and as software developers, they all decide that what they really need to support is Ogg Vorbis and Ogg Theora.  In fact, the feature request  to add the Ogg codecs to the long-abandoned Java Media Framework is consistently in the top 5 Java RFEs.

And it never seems to occur to anyone that while everything else Sun does with client-side/user-facing Java is a “me too” effort, that nobody else in the industry seems to have embraced Ogg in a big bet-the-company kind of way.  Despite its lack of patent encumbrances and its open-source implementation.

Let me posit that these traits are not features, they’re actually bugs.

Any company that embraced Ogg and put dollars behind it (or used it in products or services that involved serious dollars) would need to do some due diligence to prove to itself that Ogg is, indeed, not infringing on any known patents.  Given Ogg’s nature as an open-source project, this would likely be an expensive and tedious process, if even practical at all.  But failing to do such an examination would be fiscally irresponsible, as in “could get sued for not doing it” irresponsible.  Because in the end, all you’ve got is the community’s assurances that there is no patent-encumbered IP in the code base.  Are they telling the truth?  Can they even know what’s in there and where it came from?  A proper examination would demand proof, and that would be tedious and expensive.

And, even if Sun (or whoever) did such an exam, there’s no preventing some patent troll from going after Ogg anyways if it gets attached to a successful product or service.  So many of the media codecs are so similar — the various DCT-based codecs share common ancestors and many of the same ideas (page 77 of Damien Stolarz’ Mastering Internet Video has a remarkable “family tree” of video codecs) — that it’s easy to imagine a sleazy legal team convincing a jury in Marshall, Texas that those similarities are prima facie proof of infringement.

And anyone fighting such a suit over Ogg stands alone.  Sign up for VC1 and you get Microsoft in your corner, with more money than god.  Even the H.264 camp, currently implicated in a presumably-bogus suit from SBC AT&T, has deep-pocketed partners who would fight an infringement finding.  With Flash adopting H.264, and YouTube being bought by Google, it’s likely that Google ultimately has a dog in this fight.

Ogg doesn’t magically solve the codec problem.  And it’s the wrong problem to solve  anyways, since playback alone won’t be interesting by the time Sun can bring a product to market.