So for everyone who’s been pining for the Core Audio, I’m please to announce that the Rough Cut is now available on Safari Books Online. If you’re a member, you now have access to the first draft of the first six (of an anticipated 12) chapters.
The first three chapters are set-up, but they’re not idle chatfests. The first introduces the framework and its major conventions. Chapter 2 discusses digital audio processing and how it works. Then chapter 3 puts the two together, showing how Core Audio models and works with digital audio. All three of these introductory chapters have example projects… it’s a very code-heavy book. Heck, you’re already generating raw PCM samples in chapter 2.
Part II has two chapters on Audio Queues: the first for recording, the second for playback. Chapter 6 gets into the Audio Converter and ExtAudioFile frameworks, for converting between encoded formats and PCM. There’s surprisingly little information out there on using the Audio Converter directly (and not that much on ExtAudioFile), so we’re breaking a little new ground here.
That’s where the current Rough Cut ends. Right now, I’m working on one hell of a chapter about Audio Units, the heart and soul of Core Audio. We have four examples in this chapter: a file player, a speech synthesizer, a sine-wave generator (which introduces render callbacks), and a play-through example (which covers input callbacks, audio devices, the CARingBuffer, and mixing). This chapter might end up being as long as all of Part II, which in turn might argue for a chapter split if we can find a good place to do it. Nevertheless, do not fear for depth in this book: this chapter in particular is going crazy deep. I hope to get it off to Chuck this week.
After that, it’s on to OpenAL, which I worked on for an unpublished ADC article, which in turn led to an earlier brain dump. My plan here is to cover both plain ol’ single-shot ALBuffers and streaming to an ALSource.
There’s still a ways to go, but I think getting past Audio Units will give me a break from “hard parts”, at least until the last chapter when we get into custom units.
And yes, the estimated date has slipped again. Clearly, if I’m on chapter 7 of 12, printed copies will not be ready by Christmas. Sorry about that. Please bear with us, as we try to get this one right. It’s too important a topic to do a slapdash rip-off-the-ADC-docs kind of job.
Last weekend was the Voices That Matter: iPhone Developers Conference in Seattle, put on by Pearson, who’s publishing our Core Audio book. However, co-author Kevin Avila handled the Core Audio talk for this conference, so I took on two topics that I had fairly deep knowledge of, thanks to my work on Road Tip.
Core Location and Map Kit: Bringing Your Own Maps
This talk starts with a basic tour of Core Location’s more or less straightforward means of getting information about your current location, getting updates as the data gets better (i.e., GPS kicks in) or you move. Then we got into Map Kit and how to get map images of any given location. The sample app for this takes a list of Apple Store locations, which I converted from CSV to a plist and stuck in the app bundle, and shows you the nearest store to a given start location. Each time you hit a + button in the nav bar, it drops the next closest store as a pin on the map and re-scales the map so that all pins are visible.
This is where it gets good. Starting from a canned location at my house, the closest Apple Store is here in Grand Rapids. The next two that come up are outside Detroit (Ann Arbor and Novi). The fourth closest store is in Milwaukee. The comedy is when you see this on the map — Milwaukee is close only if you ignore the fact that going there would involve driving 100 miles straight across Lake Michigan. Since ferries across the lake are slow and expensive, and run only in Summer, you would probably drive through or around Chicago to get to Milwaukee… and go right past 7 Apple Stores in the process.
This is a common mistake to make — I forgot to show in my slides that the apple.com retail store finder does the same thing. Still, as-the-crow-flies distance calculations, paired with map images, can be a problem. Map Kit doesn’t know anything about roads, bodies of water, geographic features, political borders, etc… all it does is serve up images, and provide a touch UI to interact with them.
The last third of the talk is about “bringing your own maps”, meaning integration with third party map data providers to get some navigational intelligence into your app. The final code sample uses the MapQuest directions web service to get actual driving distances to the first few hits, and to keep the list of locations sorted in order by driving distance. This not only keeps me from going to Milwaukee, it even smartens up the earlier results: being right on I-96, the Novi store is now my second closest result, and as it turns out, even Indianapolis is a shorter drive than going around the lake to Milwaukee.
- Slides (on slideshare.com)
- SimpleCoreLocationDemo.zip
- AppleStorePOIMapper.zip
- MQAppleStorePOIMapper.zip
In-App Purchase: Best/Worst Thing Ever
My second talk was on In-App Purchase. In many ways, it covered the same hard-earned experience that I covered in An In-App Purchase Brain Dump. I spent a little time on both the iPhone Provisioning Portal (to set up an AppID, authorize it for I-AP, and provide enough of an app submission to create purchase objects, without actually going into the review queue), and on iTunes Connect (creating purchase products). The sample app simulated an online social game, minus the game, in which users might be able to purchase digital goods like virtual clothes for their avatar. The example offered a tabbed UI with a blank view for the game, a table of purchased objects, and a table for the store. When you tap “store”, the app collects the available products with a call to Store Kit, and tapping one of them kicks off the purchase. If the purchase goes through, the item is added to the inventory page.
Of course, this is easier and demos better because it uses non-consumable products, which have always been the best-understood and best-supported class of I-AP products. I did weigh in against the still-broken subscriptions, and how they have to be restored to a user’s many devices, even though restoreCompletedTransactions doesn’t support subscriptions, and nothing in Store Kit gives you a key you can associate with the user to save the purchase on your own server.
Erica Sadun talked with me before this session and mentioned an interesting workaround. Get your user to purchase a non-consumable item, and persist its transactionId. When they purchase subscriptions, log the purchase and this other transactionId on your server. Then, when they restore on another device, they’ll get this non-consumable back and its transactionId, which you can then use to query your server for subscription purchases. Depending on the nature of your app, and how well you hide the clunkiness from the user, this could be a very viable workaround.
That said, I still think I-AP subscriptions suck and hope that Apple deprecates them soon.
The last part of the talk covers of doing commerce without I-AP, which is more viable than you might think. If you’re already selling digital goods from a web store, you can continue to do so, and make your iPhone app a rich client to your web-vended content. In this case, you’re already doing everything that I-AP offers, so there’s no reason to give Apple a 30% cut. For example, the various e-bookstores within Stanza don’t use I-AP — they go out to websites for O’Reilly, All Romance Books, etc., to complete the purchase. This might not be allowed for an iPhone-only app, but where the goods are available in multiple forms, it only makes sense for Apple to allow an iPhone app to be a rich client to such a store’s goods.
My final example is streaming anime: Crunchyroll sells premium content subscriptions on their website, and their free iPhone app lets you use the same login to get the same content. By contrast, The Anime Network has premium subscriptions on their website and via I-AP in their paid iPhone app, and credentials aren’t shared between the two. Worse, the iPhone app offers fewer videos for the same price. Their use of I-AP is bad for them (they’re paying Apple to process payments they’re already capable of handling), and bad for their users. WTF?
- Slides (on slideshare.com)
- VTMPurchaseDemo.zip
Coming next: my 360iDev slides, and the much-anticipated cleaned-up Core Audio sample code
My editor on the Core Audio book passed along my name as a potential speaker for the Voices That Matter: iPhone 2010 conference… but not on Core Audio. My co-author, Kevin Avila, has that covered. And I’ll be covering CA two weeks prior at 360iDev, so why would I want to put all that work into a talk and use it twice, when I can put lots more work into two talks. That would be too easy.
Oh, did I say two talks? I meant three. Chuck suggested I do a location/mapping talk based on my experiences developing Road Tip, and after agreeing, I noticed the schedule had a spot for an In-App Purchase talk. Foolhardy me, I said to the organizers, “hey, if you don’t have a speaker for that topic, I can take it.” So now I’ve got two talks for VTM:i.
The tricky thing is, the VTM talks are going to be pretty easy to put together, given that I’ve already written extensively about mapping and in-app purchase here on [T:c];. Heck, I did I-AP last week for Ann Arbor CocoaHeads, and I think I only swore three times.
The Core Audio talk, which comes two weeks earlier at 360iDev, will actually be the hardest to put together because the proposal and topic are still pretty nebulous — “advanced Core Audio” — and because I’ll want to pull, you know, actual advanced stuff into it. On the other hand, it’s great that there’s a “Three Core Audio Hello Worlds” earlier in the conference, covering AVAudioPlayer, OpenAL, and Audio Units to some degree, so I can elide some of the basics and get right into the hard-core.
Thing is, I now have, what, three weeks to put the hard-core together. Alas, another crunch.
Oh, and here’s a promo code for VTM:i: PHASPKR. I don’t have a code for 360iDev, but if you follow 360idev on Twitter, you might catch one flying by.
On a few of my books, most notably QuickTime for Java: A Developer’s Notebook, I reached a point where I’d google for background knowledge, alternative opinions, or just help on some API and only find stuff I’d written and posted myself, or find nothing at all.
On one hand, it’s bad news, because it means I have to tough it out myself. On the other hand, it also means that the book is going to be able to deliver real value, since it’s going to offer knowledge that isn’t already readily available. The worst kind of books are the ones that just rehash official documentation; material like that is as miserable to write as it is useless to buy and read.
Anyways, in this case, the query in question (ooh, avoid awkward alliteration always…) is the constant kAudioFileGlobalInfo_AvailableStreamDescriptionsForFormat, which appears to have 200 Google hits, but actually only has about two pages and a total of 16 hits once all the duplicates of posts to coreaudio-api are removed.
And what is this constant for? There’s an interesting function in Audio File Services, AudioFileGetGlobalInfo(), that instead of dealing with a specific file, returns properties of Core Audio’s file handling in general. You can use it to figure out what filename extensions, MIME types, and HFS type codes go with various formats. In chapter 3, after discussing container formats, audio data formats, and the AudioStreamBasicDescription, we can use the kAudioFileGlobalInfo_AvailableStreamDescriptionsForFormat property to make some discoveries about CA-supported formats. Quoting from the property’s documentation:
Returns an array of audio stream basic description structures, which contain all the formats for a particular file type and format ID. The audio stream basic description structures have the following fields filled in: mFormatID, mFormatFlags, and mBitsPerChannel for writing new files.
In other words, we pass in a AudioFileTypeAndFormatID struct, setting the mFileType to kAudioFileAIFFType and the mFormatID to kAudioFormatLinearPCM and it will pass back ASBDs that tell us some interesting things about the limits of AIFF files: that they can’t handle more than 32 bits per sample, and that they only support big-endian integer samples. Change the mFileType to kAudioFileCAFType and you’ll find you have a lot more freedom (because you can put audio in pretty much any CA-supported format into a CAF file).
The Core Audio book can’t help but be hard; it’s the nature of the source material. But I think we’re doing pretty well in the intro chapters to demystify the core concepts and provide the tools for finding your way around the many APIs.
I’ve been away from the blog on a pretty grueling day-job contract. It’s actually been positive for my work on the Core Audio book: nothing encourages a side project like disliking your day job.
Squirreling away some morning hours between 5 and 9 AM over the last two weeks, I’ve managed to get first drafts of the first two chapters done (admittedly with a helpful start from the fragments left from Mike and Kevin’s initial work on the book). I actually surprised myself by finding a concretization of the Nyquist theorem, which (basically) says that to reproduce a frequency, you have to sample at at least double that rate. In chapter 2′s digital audio intro, I added an example to get readers looking at raw samples, writing out their own waves as PCM samples. An inner loop uses a samplesPerWave value, and it occurred to me that something interesting happens at the Nyquist point. If you’re sampling at 44100 Hz, then for a 22050 Hz frequency, samplesPerWave is 2. That’s the minimum for a perceivable, repeatable pattern: at a higher frequency, you have less than 2 samplesPerWave and therefore no more repeating pattern, no wave.
Not a big deal, but it was a nice little “aha” moment to stumble across.
I’m enjoying having momentum on Core Audio, and since I wrote that never-published series of articles (yep, the publisher is still wedged) on low-latency use of units and OpenAL, I’ve already worked through some of the conceptually-hardest material, so there are fewer unknowns looming ahead than normal. Which I wouldn’t have expected for Core Audio, surely the hardest topic I’ve ever written about.
I’ve found myself wondering what would be a good topic to pitch and hopefully write about in late 2010, time and budget permitting (writing for anyone other than the Prags is a financial indulgence… it most certainly does not pay the bills).
Here are a few things I’m thinking about.
- HTTP Live Streaming – The only choice for streaming video to an iPhone OS device, and a compelling choice for streaming media in general. Would probably cover setting up your own server, working with CDNs that support it, writing your own client (if QTX or MediaPlayer.framework doesn’t work for you), encryption and DRM, etc.
- Core Services – all the C functions that none of the introductory Cocoa books tell you you’re going to need for a serious iPhone or Mac application. Core Foundation, CFNetwork, Keychain, System Configuration, Reachability, stuff like that.
- C for iPhone – I’ve mentioned this before, how I wrote about 100 pages of this for the Prags, and everything was fine until it wasn’t. With all the people getting into iPhone OS development without a C background and the problems of applying the very dated K&R to modern C development (especially for beginners, who won’t follow its Unixisms or its analogies to Fortran and Pascal), I still think we badly need a C intro that can serve as a prerequisite to all the iPhone books (including ours) that assume C experience. Plus, I find it a perversely fun topic to speak about (see the slides of my CodeMash session).
- OpenAL – There’s, like, next to nothing to get beginners started with this audio technology, and what’s out there is frequently wrong or outdated (for example, some “getting started” type blog entries use the deprecated
loadWAVFilethat isn’t even present on iPhone OS). I’m actually thinking of a radically different format for this book, but don’t want to give it away just yet.
I was going to make this a poll, but I don’t think I have enough readers to make that viable, and all the poll plugins for WordPress suck anyways. So if you’d care to vote for one of these options, please post a comment. Besides, I’d love to have new registered users who aren’t offshored spammers.
There will likely be an update to iPhone SDK Development as well, when SDK changes warrant, but that hasn’t happened yet. Let’s see when we get a 4.0 and what’s in it.
Answering the recurring call for a proper third-party book on Core Audio, one that doesn’t throw the reader to the wilds of picking out side-effects and ephemera from sample code and mailing lists, I’ve got an announcement…
Oh, wait, freakin’ Amazon beat me to it.
Well, the announcement is that I’m joining Addison-Wesley’s Core Audio book as a co-author to Kevin Avila.
This has been in the works for a little while, but in the last week we made it official: sending around contracts, setting up a Subversion repository, and going over the first batch of stuff that Kevin has already gotten started, with the assistance of legendary Mac programmer Mike Lee. Our editor is Chuck Toporek, who tried so hard to get me to write a real Mac book for him at O’Reilly, and I’m pleased to have the chance to work with him for the first time.
I don’t know if we can replace subscribing to coreaudio-api as a practical requirement of aspiring Mac and iPhone audio developers, but hopefully, we’ll be able to keep a few people from going nuts trying to figure out WTF error 1718449215 means (it’s kAudioFormatUnsupportedDataFormatError, and I’ll bet you tried to use floating point samples on the iPhone, didn’t you?).
