So, Apple announced yesterday that they’ll stream today’s special event live, and everyone immediately assumed the load would crash the stream, if not the whole internet, myself included. But then I got thinking: they wouldn’t even try it if they weren’t pretty damn sure it would work. So what makes them think this will work?
HTTP Live Streaming, that’s why. I banged out a series of tweets (1, 2, 3, 4, 5, 6, 7, 8, 9) spelling out why the nature of HTTP Live Streaming (which I worked with briefly on a fix-up job last year) makes it highly plausible for such a use.
To summarize the spec: a client retrieves a playlist (an .m3u8, which is basically a UTF-8′ed version of the old WinAmp playlist format) that lists segments of the stream as flat files (often .m4a’s for audio, and .ts for video, which is an MPEG-2 transport stream, though Apple’s payload is presumably H.264/AAC). The client downloads these flat files and sends them to its local media player, and refreshes the playlist periodically to see if there are new files to fetch. The sizing and timing is configurable, but I think the defaults are like a 60-second refresh cycle on the playlist, and segments of about 10 seconds each.
This can scale for a live broadcast by using edge servers, which Apple has long depended on Akamai (and others?) for. Apple vends you a playlist URL at a local edge server, and its contents are all on the edge server, so the millions of viewers don’t pound Apple with requests — the load is pushed out to the edge of the internet, and largely stays off the backbone. Also, all the local clients will be asking for the same handful of segment files at the same time, so these could be in in-memory caches on the edge servers (since they’re only 10 seconds of video each). All these are good things.
I do wonder if local 3G cells will be a point of failure, if the bandwidth on a cell gets saturated by iPhone clients receiving the files. But for wired internet and wifi LANs, I suspect this is highly viable.
One interesting point brought up by TUAW is the dearth of clients that can handle HTTP Live Streaming. So far, it’s iOS devices, and Macs with QuickTime X (i.e., running Snow Leopard). The windows version of QuickTime doesn’t support HTTP Live Streaming (being based on the “old” 32-bit QuickTime on Mac, it may effectively be in maintenance mode). Open standard or not, there are no handy HTTP Live Streaming clients for other OS’s, though MacRumors’ VNC-based workaround (which requires you to manually download the .m3u8 playlist and do the refresh yourself), suggests it would be pretty easy to get it running elsewhere, since you already have the ability to play a playlist of segments and just need to automate the playlist refresh.
Dan Leehr tweeted back that Apple has talked a good game on HTTP Live Streaming, but hasn’t really showed much. Maybe this event is meant to change that. Moreover, you can’t complain about the adoption — last December, the App Store terms added a new fiat that any streaming video app must use HTTP Live Streaming (although a February post seems to ratchet this back to apps that stream for more than 10 minutes over the cellular network), so any app you see with a video streaming feature almost certainly uses HLS. At WWDC, Apple boasted about the MLB app using HLS, and it’s a safe bet that most/all other iOS video streaming apps (Netflix, Crunchyroll, etc.) use it too.
And one more thing to think about… MLB and Netflix aren’t going to stream without DRM, right? That’s the other piece that nobody ever talks about with HTTP Live Streaming: the protocol allows for encrypting of the media files. See section 5 of the spec. As much as Apple and its fanboys talk up HTML5 as a rival to and replacement for Flash, this is the thing that should really worry Adobe: commoditizing DRM’ed video streaming.
I’ve been away from the blog on a pretty grueling day-job contract. It’s actually been positive for my work on the Core Audio book: nothing encourages a side project like disliking your day job.
Squirreling away some morning hours between 5 and 9 AM over the last two weeks, I’ve managed to get first drafts of the first two chapters done (admittedly with a helpful start from the fragments left from Mike and Kevin’s initial work on the book). I actually surprised myself by finding a concretization of the Nyquist theorem, which (basically) says that to reproduce a frequency, you have to sample at at least double that rate. In chapter 2′s digital audio intro, I added an example to get readers looking at raw samples, writing out their own waves as PCM samples. An inner loop uses a samplesPerWave value, and it occurred to me that something interesting happens at the Nyquist point. If you’re sampling at 44100 Hz, then for a 22050 Hz frequency, samplesPerWave is 2. That’s the minimum for a perceivable, repeatable pattern: at a higher frequency, you have less than 2 samplesPerWave and therefore no more repeating pattern, no wave.
Not a big deal, but it was a nice little “aha” moment to stumble across.
I’m enjoying having momentum on Core Audio, and since I wrote that never-published series of articles (yep, the publisher is still wedged) on low-latency use of units and OpenAL, I’ve already worked through some of the conceptually-hardest material, so there are fewer unknowns looming ahead than normal. Which I wouldn’t have expected for Core Audio, surely the hardest topic I’ve ever written about.
I’ve found myself wondering what would be a good topic to pitch and hopefully write about in late 2010, time and budget permitting (writing for anyone other than the Prags is a financial indulgence… it most certainly does not pay the bills).
Here are a few things I’m thinking about.
- HTTP Live Streaming – The only choice for streaming video to an iPhone OS device, and a compelling choice for streaming media in general. Would probably cover setting up your own server, working with CDNs that support it, writing your own client (if QTX or MediaPlayer.framework doesn’t work for you), encryption and DRM, etc.
- Core Services – all the C functions that none of the introductory Cocoa books tell you you’re going to need for a serious iPhone or Mac application. Core Foundation, CFNetwork, Keychain, System Configuration, Reachability, stuff like that.
- C for iPhone – I’ve mentioned this before, how I wrote about 100 pages of this for the Prags, and everything was fine until it wasn’t. With all the people getting into iPhone OS development without a C background and the problems of applying the very dated K&R to modern C development (especially for beginners, who won’t follow its Unixisms or its analogies to Fortran and Pascal), I still think we badly need a C intro that can serve as a prerequisite to all the iPhone books (including ours) that assume C experience. Plus, I find it a perversely fun topic to speak about (see the slides of my CodeMash session).
- OpenAL – There’s, like, next to nothing to get beginners started with this audio technology, and what’s out there is frequently wrong or outdated (for example, some “getting started” type blog entries use the deprecated
loadWAVFilethat isn’t even present on iPhone OS). I’m actually thinking of a radically different format for this book, but don’t want to give it away just yet.
I was going to make this a poll, but I don’t think I have enough readers to make that viable, and all the poll plugins for WordPress suck anyways. So if you’d care to vote for one of these options, please post a comment. Besides, I’d love to have new registered users who aren’t offshored spammers.
There will likely be an update to iPhone SDK Development as well, when SDK changes warrant, but that hasn’t happened yet. Let’s see when we get a 4.0 and what’s in it.
Thinking about the iPad and the appealing idea of watching video on this device, the usual objections come up about being limited to the Apple iTunes ecosystem, and being shut out from great stuff, just like the AppleTV is.
But wait a second, there’s a second way to get video on your iPad, just like on an iPhone/iPod touch: apps can stream video. In fact, there are a bunch of these already. In many cases, the apps scratch the itch of certain niches. Apple continues to trot out the MLB app to show off crowd-pleasing baseball video, but I notice that I have downloaded at least four apps for streaming anime: Crunchyroll, Babelgum, Joost, and The Anime Network. Looking on the App Store, I see a few others, including one app that exists just as a container for the first volume of the charming His and Her Circumstances.
Let’s think here. Some of these services (Crunchyroll and Anime Network) offer Flash-based viewers on the web, but they’re not in the Flash business, they’re in the content business, so they use a different technology to get their content to iPhone OS viewers.
And what is that technology? By Apple fiat, it is HTTP Live Streaming, which Apple recently declared as the only permitted technology for streaming video to iPhone OS devices. Technically, it seems like you could also use HTML5 <video> tags in a UIWebView, but there are lots of nice reasons to use HTTP Live Streaming (content providers are probably happy to see DRM in the spec, even if most of us consumers aren’t).
So here’s what I’m wondering. If content providers are going to have to use HTTP Live Streaming to support all the iPhone OS devices, is this eventually going to put pressure on Flash? All that has to happen is for HTTP Live Streaming to become more viable on desktops, and then you’ll have a video distribution solution that skips the Adobe tax and the extra step of Flash encoding. HLS is supported by QuickTime X on Snow Leopard, but I don’t believe that the Windows version of QuickTime handles it yet. Pity.
Still, if I were Adobe, I’d be pretty concerned about HTTP Live Streaming right now.
It’s easy enough to get hits with bad tech journalism: just rile the Apple fanboys. The Mobile Technology Weblog’s Microsoft Announces iPhone Killer deserves accolades for not involving any kind of actual announcement, nor in any way substantiating how its topic might be capable of “kill”ing the iPhone.
Android is a far more credible competitor to the iPhone, and as I said earlier, I’m interested to see if it can disprove some of Apple’s assertions about the way it conducts the iPhone ecosystem, such as the tightly-restricted review process or the prohibition on background apps, or if it will fail and thereby vindicate Apple. Still, whatever success Android might achieve, it’s hyperbole to posit it as an iPhone killer, at least at this stage.
Honestly, the two biggest threats I see to the iPhone come from the iPhone ecosystem itself.
In the worst case, one leads to mere tragedy, the other to catastrophe. Not that I’m predicting such a thing; I’m not a Cassandra (except when playing Soul Calibur, but that’s another story). But I do think these are issues that have the potential to cause great harm to Apple, its users, and its developers.
1. iPhone development is a bubble economy, already severely overinflated
Rilo Kiley, “The Execution of All Things“:
Soldiers come quickly, I feel the earth beneath my feet.
I’m feeling badly, it’s not an attempt at decency.
Surely only the most foolish and greedy developers at this point think they’re going to write a simple game and make a million dollars, like Steve Demeter, who pulled $250,000 in two months with the indie game Trism. The easy, early money has been made. Now there are 100,000 apps to compete with for attention, and only the top 2,000 have any significant usership.
But still, lots of developers are picking up Objective-C, learning from books like ours (thank you!), and putting their own time and money into writing their own indie apps. Even if they don’t expect to become iPhone millionaires, they’d at least like a shot at being, as Dan Grigsby aptly put it, warm, clothed, and fed with the proceeds of their iPhone work.
It’s possible the door has already largely closed to this too.
When I say that iPhone development is a bubble, I welcome you to interpret that in very literal economic terms. A developer who puts time and money into learning iPhone development does so in hopes of realizing a significant return.
And if you’re well off, well then I’m happy some for you.
But I’d rather not celebrate my defeat and humiliation here with you.
Personal case study: I put about two months of full time work into RoadTip… actually, three when you account for the month spent on the nightmare of implementing in-app purchase. That was time that I lived off a credit card, meaning I used a high-interest loan from Capital One to fund my development. I also put fixed costs, like artwork for the icon (from a former CNN colleague who used to work in the graphics department) and trademark registration on my card. So this is serious skin in the game.
After two weeks of sales, it is clear that I will never make back this investment. Not even close. I may never even make back the fixed costs, to say nothing of paying myself.
OK, there are reasons for this, not the least of which is the fact that there were no exit-finder apps when I started, and at least four others now (none of which seem to have bothered licensing their map data… would love to know how they’re not violating their providers’ terms of service). In the hit-driven App Store, you fall off your section’s “recently released” page after just a day or two, and unless you make the top 20 or have something else that makes you findable (a brand name, a well-travelled incoming link), you’re already dead.
Someone come quickly, this place was built for moving out.
Leave behind buildings, the city planners got mapped out.
Bring with you history, and make your hard earned feast.
Then we’ll go to Omaha to work and exploit the booming music scene, and humility.
If this is a hit-driven business, then the winners may be the ones who best understand marketing. And that’s the scenario by which we get lots of name-brand big-company apps, where there’s enough money to get the app in front of lots of people. The indie developer, who knows more about code than marketing, is at a severe disadvantage.
And if the risk of failure is not paying the mortgage, then it’s probably time for a lot of indies to bail out.
And this is still classic economics: too much money is in the iPhone market, chasing too little reward, and eventually there’ll be a pullback.
The question is if anyone will notice. Will the average user really miss 200 different Twitter clients or unit converters? Or is there some special, unique, topic-specific niche app that some tiny group of users would dearly love, and now will never get, because the developer who could write it is instead going to go take a contract to write the “American Idol Sing-A-Long Fun Kit”™ or god fucking knows what else?
Who knows? But much as I continue to think the world needs a touch-driven “IDE for podcasts”, something that might be extraordinary on a hypothetical OS X-powered tablet, I’m not going to risk any more of my own precious lucre developing such a thing. At this point, I expect my iPhone development to be largely for-hire works.
And we’ve been talking all night….
So that’s just sad, this one is dangerous:
2. Some of Apple’s iPhone development policies could violate anti-trust law
Anti-trust?! You should think I’m nuts. You’re probably thinking: Apple’s just a small player in a very competitive market, prone to lose their position at any time to strong competitors. You want to talk monopolies, let’s talk Microsoft.
No, I don’t want to talk monopolies. That’s the Sherman Act. I want to talk restraint of trade… the Clayton Act.
Consider this: the crux of the 90′s antitrust case against Microsoft — successfully prosecuted, if you’ll recall — was that Microsoft abused its dominant position by bundling its Internet Explorer web browser with Windows, to the disadvantage of Netscape Navigator, which had to be installed separately. Microsoft was branded as evil incarnate for the act of simply including a browser.
On the iPhone, Apple not only includes a browser, it prohibits developers from using any web-rendering technology other than WebKit.
Oh god come quickly, the execution of all things.
Let’s start with the bears and the air and mountains, rivers, and streams.
Then we’ll murder what matters to you and move on to your neighbors and kids.
Crush all hopes of happiness with disease ‘cause of what you did.
But that’s just the beginning. Consider some other technological and commercial restraints put on developers:
- You may not use any interpreted language or a virtual machine in an app.
- You must use Apple’s in-app purchase API, and pay Apple a 30% cut, for any post-sale commerce in your app.
- Effective December 2009, streaming video to an iPhone can only be delivered with HTTP Live Streaming
There’s more, but that’s enough to get us started talking about tying, the practice of compelling a customer to purchase one product as a condition of buying another. The crux of my argument is: could this be applied to Apple’s App Store policies?
Maybe, if a court is sufficiently inventive. It’s been 18 years since I took Prof. Litman’s telecomm economics class in grad school, but he was particularly interested in antitrust law, and I recall the significant issues of that area of case law. In one case — and I’m sorry, I forget if it was FTC vs. Proctor & Gamble (the “Clorox Case”) or U.S. vs. Alcoa — the court was basically willing to invent a market within the defendant company’s operations, in order to claim that the market was closed to competition.
Let’s play with this line of reasoning. Can we imagine parties that might have a complaint against Apple for shutting off a market to them?
- Sun (Java) and Adobe (Flash) for code-execution environments (i.e., interpreters and virtual machines)
- PayPal and other payment processors for selling apps and handling in-app purchases
- Adobe (Flash) and Microsoft (Silverlight) for video playback and streaming technologies
- Opera and the Mozilla Foundation for web rendering technologies
All of these companies have services that they could sell to iPhone developers, but for the fact that Apple forbids their use on iPhone by fiat. Of course developers contractually agree to that, but they don’t really have a choice: it’s Apple’s way or the highway. And you may agree that it’s better that way; that’s entirely reasonable.
But if you successfully make the argument that things like in-app purchase processing or video streaming are competitive markets, then you might have a case that Apple’s App Store terms violate the Clayton Act.
Granted, I’m not a lawyer, and if it were such a slam dunk, then it’s fair to ask: why haven’t Adobe, Sun, PayPal and the rest sued on exactly this basis? Fair enough.
Still, if we woke up next week to an antitrust investigation of Apple’s App Store practices, it wouldn’t surprise me in the least.
And lastly, you’re all alone with nothing left but sleep.
But sleep never comes to you, it’s just the guilt and forever wakefulness of the weak.
It’s just you and me….The execution of all things.
The execution of all things.
The execution of all things.
One-time self-described “World’s Greatest Compressionist” Ben Waggoner posts a pointed question to the quicktime-api list:
http://www.apple.com/macosx/what-is-macosx/quicktime.html
What I’d like to know is if QuickTime X is going to be available for Windows and older versions of Mac OS X.
It’s an important issue, because despite iTunes’ insistence on installing QuickTime on Windows, the future of that product seems completely unknown. For years, every question I’ve seen about the future of QuickTime on Windows has been met with absolute silence from Apple. Yeah, I know, “Apple does not comment on unannounced products,” and all… Still, Apple has left this technology in limbo for a remarkably long time. I recall asking ADC reps about QuickTime for Windows back at Leopard Tech Day Atlanta in 2006, as I was considering calling it from Java with JNI, and (as previously noted), I got no reply at all. And every other public question I’ve seen about the future of QuickTime on Windows has gone similarly unanswered, for years.
Smell that? That’s the scent of Abandoned Code Rot. We got that from QuickTime for Java for a few years before they managed to finally deprecate it (though they apparently haven’t gotten the message out).
It wouldn’t be too surprising to see QT for Windows fall by the wayside… Apple probably cares more about the popularity of its favorite formats and codecs (AAC and H.264) than of the QuickTime APIs and QuickTime’s interactive features like Wired Sprites that have been clearly and unequivocally beaten by Flash.
But if that’s true of Windows, is it also true on the Mac? QuickTime developers are right to be a little worried. The old C-based QuickTime API remains a 32-bit only option, intended to be replaced by the Objective-C QTKit. But in the four years since its introduction in Tiger, QTKit has only taken on part of the capabilities of the old QuickTime API. With Leopard, you could finally do capture and some significant editing (e.g., inserting segments at the movie or track levels), but raw sample level data was unavailable for any track type other than video, and some of the more interesting track types (like effects and especially tweens, useful for fading an audio track’s volume between specific times) are effectively useless in QTKit.
With Snow Leopard, the big news isn’t a more capable QTKit API, it’s QuickTime X. And as Apple’s QuickTime X page points out, QTX is all about a highly-optimized playback path (using decode hardware if available) and polished presentation. Great news if you’re playing 1080p movies on your computer or living room PC, not so much if you want to edit them: if you want to edit anything, you’re back in the old 32-bit QuickTime (and the code is probably still written in C against the old APIs, given QTKit’s numerous limitations). You don’t see a 64-bit Final Cut Pro, now do you? (BTW, here’s a nice blog on that topic.)
When you all install Snow Leopard tomorrow and run the QTX-based QuickTime Player, you’ll immediately understand why the $30 QuickTime Pro (which bought you editing and exporting from the Player app and the plug-in) is gone. Follow up in the comments tomorrow (after the NDA drops) and we’ll discuss further.
If I were starting a major new multimedia project that wasn’t solely playback-based — imagine, say, a podcast studio that would combine the editing, exporting, and publishing tasks that you might currently perform with Garage Band, iTunes, and FTP — I would be very confused as to which technology to adopt. QuickTime’s cross-platform story seems to be finished (QTJ deprecated, QTW rotting away), and everything we hear on the Mac side is about playback. Would it be safer to assume that QuickTime doesn’t have a future as a media creation framework, and drop down to the engine level (Core Audio and Core Video)? And if not QuickTime… then what?
Oh, and as for the first question from the quicktime-api thread:
… How about Apple throwing us a bone as to what QuickTime X will offer those of us that use QT and QTSS?
From what I can tell, Apple has all but ditched QTSS in favor of HTTP Live Streaming, supported by QuickTime X and iPhone 3.0.
