Miscellaneous updates:
CocoaConf Chicago early bird pricing ends this weekend. I’ll be there, doing the Core Audio on iOS 5 talk (with MIDI and Audio Unit effects), and a new “Core What?” talk about neat stuff I’ve found over the years in the various C frameworks. A big piece of it will actually center on the CGPDF… stuff in Core Graphics, after all the PDF work I did last fall. Drawing into a PDF and drawing a PDF page into your view are common enough tasks, and the parsing of a PDF is another one of those tasks where the API makes absolutely no damn sense until you understand the problem domain behind it. Beyond that… CFUUID, CF-only collections, other curiosities and oddities.
Oh, and there’s a Fry’s Electronics in Downer’s Grove, about 20 minutes from the conference hotel. I have at least $500 worth of stuff on my shopping list (NAS, FCPX-compatible video card, Rock Band drums or Pro Guitar, out-of-print anime…). So that will add to the fun.
Speaking of CocoaConf, Saul Mora (who recorded our panel at CocoaConf Raleigh) had me on as a guest for NSBrief episode #33, in which we discuss audio theory, Core Audio, and AV Foundation for an hour. Plus, I lay out the plan for the “Reverse Q&A” we’re doing at CocoaConf Chicago, inspired by the Harmonix Reverse Q&A Panel at PAX East 2011. I wondered aloud about the idea of doing this in an earlier blog, and I’m glad we’ll have a chance to give it a shot. Hopefully, this will prove to be a good way to shake up the regular old panel format, and be fun and insightful for audience and speakers alike.
Learning Core Audio continues to work its way through Pearson’s production process. This week we signed off on author bios and cover blurbs. Copy-edit was a few weeks ago… admittedly a bit of a disappointment as I discovered all the third-person sentences had been somewhat mechanically rewritten to second-person (i.e., “we” becomes “you”). Sometimes it works, sometimes it really doesn’t, and I was too tired to fix all the cases of the latter. Still, it would have been nice to have been told about this house style two years ago when we started the damn thing.
There’s nothing like losing your editor — to Apple, no less — to ratchet up the heat to finish a book that has been too long on the burner. But with Chuck doing exactly that at the end of December, Kevin and I had the motivation to push aside our clients and other commitments long enough to finally finish the Learning Core Audio book (yes, the title is new), and send it off to the production process.
In our final push, we went through the tech review comments and reported errata — Lion broke a lot of our example code — and ended up rewriting every example in the book as Xcode 4 projects, moving the base SDK to Snow Leopard, which allowed us to ditch all the old Component Manager dependencies. For the iOS chapter, we rev’ed up to iOS 4 as a baseline and tested against iOS 5.
One of the advantages of Xcode 4 is that the source directory is cleaner for source control (no more ever-changing build folder that you have to avoid committing), while also offering a pretty simple way to get to the “derived data” folder with the build result, which was important for us because we have a number of command-line examples that create audio files relative to the executable, and Xcode 4 makes them easy to find.
In our final push, we also managed to get in an exercise with the AUSampler, the MIDI instrument that pitch-shifts the audio file of your choice, into the wrap-up chapter. So that should keep things nice and fresh. Thanks to Apple for finally providing public guidance on how to get the .aupreset file to load into the audio unit, and how the unit deals with absolute paths to the sample audio in the app bundle case.
Updated code is available from my Dropbox: learning-core-audio-xcode4-projects-jan-03-2012.zip
From here, Pearson will take a few months to layout the book, run it by us for proofing and fixes, and send it off to the printer. So… paper copies probably sometime in Spring. The whole book — minus this last round of corrections — is already on Safari Books Online, and will be updated as it goes through Pearson’s production process.
It seems like there’s been a big uptake in Core Audio in the last year or two. More importantly, we’ve moved from people struggling through the basics of Audio Queues for simple file playback (back in iPhone OS 2.0 when Core Audio was the only game in town for media) and now we see a lot of questions about pushing into interesting uses of mixing, effects, digital signal processing, etc. Enough people have gotten sufficiently unblocked that there’s neat stuff going on in this area, and we’re fortunate to be part of it.
I’m surprised how fast iOS conference slides go old. I re-used some AV Foundation material that was less than a year old at August’s CocoaConf, some of it already seemed crusty, not the least of which was a performSelectorOnMainThread: call on a slide where any decent 2011 coder would use dispatch_async() and a block. So it’s probably just as well that I did two completely new talks for last weekend’s Voices That Matter: iOS Developers Conference in Boston.
OK, long-time readers — ooh, do I have long-time readers? — know how these posts work: I do links to the slides on Slideshare and code examples on my Dropbox. Those are are at the bottom, so skip there if that’s what you need. Also, I’ve put URLs of the sample code under each of the “Demo” slides.
The AV Foundation talk is completely limited to capture, since my last few talks have gone so deep into the woods on editing (and I’m still unsatisfied with my mess of sample code on that topic that I put together for VTM:iPhone Seattle in the Spring… maybe someday I’ll have time for a do-over). I re-used an earlier “capture to file and playback” example, and the ZXing barcode reader as an example of setting up an AVCaptureVideoDataOutput, so the new thing in this talk was a straightforward face-finder using the new-to-iOS Core Image CIDetector. Apple’s WWDC has a more ambitious example of this API, so go check that out if you want to go deeper.
The Core Audio talk was the one I was most jazzed about, given that Audio Units are far more interesting in iOS 5 with the addition of music, generator, and a dozen effects units. That demo builds up an AUGraph that takes mic input, a file-player looping a drum track, and an AUSampler allowing for MIDI input from a physical keyboard (the Rock Band 3 keyboard, in fact) to play two-second synth sample that I cropped from one of the Soundtrack loops, all mixed by an AUMultichannelMixer and then fed through two effects (distortion and low pass filter) before going out to hardware through AURemoteIO. Oh, and with a simple detail view that lets you adjust the input levels into the mixer and to bypass the effects.
The process of setting up a .aupreset and getting that into an AUSampler at runtime is quite convoluted. There are lots of screenshots from AULab in the slides, but I might just shoot a screencast and post to YouTube. For now, combine WWDC 2011 session #411 with Technical Note TN2283 and you have as much a fighting chance as I did.
I’ll be doing these talks again at CocoaConf in Raleigh, NC on Dec. 1-2, with a few fix-ups and polishing. The face-finder has a stupid bug where it creates a new CIDetector on each callback from the camera, which is grievously wasteful. For the Core Audio AUGraph, I realized in the AU property docs that the mixer has pre-/post- peak/average meters, so it looks like it would be easy to add level meters to the UI. So those versions of the talks will be a little more polished. Hey, it was a tough crunch getting enough time away from client work to get the sample code done at all.
Speaking of preparation, the other thing notable about these talks is that I was able to do the slides for both talks entirely on the iPad, while on the road, using Keynote, OmniGraffle, and Textastic. Consumption-only device, my ass.
- Capturing Stills, Sounds, and Scenes with AV Foundation
- Core Audio Cranks It Up
Sometime last week, the archives at lists.apple.com went dark, replaced by the ambiguous message:
Archives are currently disabled. If you require urgent access to the Archives, please contact postmaster@lists.apple.com with the subject “ARCHIVES”.
Apple’s developer mailing lists have been around for at least a decade, and contain a wealth of real-world knowledge, particularly on the many APIs that pre-date the establishment of Apple’s web-based Dev Forums a few years ago. While the lists themselves continue to operate and distribute new messages, the archives have been out since at least late last week.
For Core Audio, the loss of the archives is absolutely devastating. Everyone complains about the lack of documentation for Core Audio, but that’s not quite the problem. It’s that the nature of Core Audio is that the syntax is simple and the semantics are where the real work gets done. It’s very easy to create a AudioUnitSetProperty() call that compiles, and very difficult to create one that doesn’t crash and instead does something useful. That’s the good and the bad of an API whose favorite idiom is getting and setting void* properties; compare to AV Foundation, whose far more limited abilities can reasonably be gleaned by reading the docs.
When the nature of the framework is knowing what you can and can’t do with the various properties, what the audio engines will and won’t accept, and why the hell you’re getting OSStatus -50 (paramErr), the best resource you could hope for is 10 years of people doing the same thing and sharing their results. And now it’s gone.
What’s worse is that Apple, as ever, has provided far too little information for their partners to know where they stand or what to do. Are the archives down because of prolonged maintenance, like taking the tapes from Cupertino to North Carolina and hosting them there? Was there a database crash that is now being recovered from? Or was there a business or political decision to just shut down the archives indefinitely (imagine if, say, a legal rival was getting evidence against Apple from the listserv archives, or if Charlie Miller’s App Store security hack was based on information gleaned from the archives). The thing is, we don’t know whether or not to expect the archives to return, and as is so often the case, Apple isn’t being clear with us.
For now, we can complain. And file bugs against the apple.com site. Feel free to dupe mine from Open Radar.
Real life intervenes again (parsing PDF, whee!) and I have to cut short a planned epic iDevBlogADay entry. But I do want to bang out a few quick notes on various topics of interest.
The first is Core Audio in iOS 5, which we can now talk about publicly. If we go through the iOS 4.3 to iOS 5.0 API Differences document, we see that Audio Units accounts for a large number of diffs. This comes from the addition of a suite of units that finally make programming at the unit level a lot more interesting. Whereas we used to get a single effects unit (AUiPodEQ), we now get high and low pass filters, the varispeed unit, a distortion box, and a parametric EQ that lets us play with sliders instead of the “canned” EQ settings like “Bass Booster” and “Spoken Word”. Even more useful, we get the AUFilePlayer, meaning you can now put audio from a file at the front of an AUGraph, instead of the sheer pain of having to decode your own samples and pass them to the AUGraph through a CARingBuffer.
iOS also gets the AUSampler unit introduced in Lion, which provides a MIDI-controlled virtual instrument whose output is pitch-shifted from a source sample. This was shown off at WWDC, although providing the source to the unit by means of an .aupreset is still a dark (undocumented) art. This is the first actual MIDI audio unit in iOS, which makes the presence of Core MIDI more useful on the platform.
Core Audio giveth, but Core Audio also taketh away: iOS 5 removes (not deprecates) VoiceIOFarEndVersionInfo. This struct, and its related constants (specifically kVoiceIOFarEndAUVersion_ThirdParty), were documented as interoperating with a hypothetical “3rd-party device following open FaceTime standards”, something I took note of last May as possibly meaning that FaceTime was still ostensibly being opened up. With the removal of these APIs, I think that closes the book on Apple having any intention to live up to its vow to publish FaceTime as an open standard.
There’s lots more to talk about, but I’m already over my allotted blogging time, and work beckons. Perhaps you’d like to hear me speaking about this stuff and demo’ing it? I’m doing an all-new-for-iOS-5 Core Audio talk at two upcoming conferences:
- Voices That Matter: iOS Developers Conference, Nov. 12-13 in Boston, and $150 off with discount code
BSSPKR5. - CocoaConf, Dec. 2-3 in Raleigh, NC, which has early-bird pricing available through Oct. 31.
I’ll also be doing a talk about AV Foundation capture at these conferences. And back on audio, I just heard from my editor that the last three chapters of Core Audio should be in the Rough Cut on Safari Online Books in the next week or so, although I still have some work to do to clean up bits that are broken on Lion (read the sidebar on Audio Components if you’re having a problem creating audio units with the old Component Manager calls) and to clear out forward references to stuff that didn’t end up making the final cut for the book.
Yuna:book cadamson$ svn commit
Sending book/ch11/Ch11.txt
Sending book/ch12/Ch12.txt
Transmitting file data ..
Committed revision 237.
Yuna:book cadamson$ Folks, that’s the svn commit that marks the completion of the first draft of Core Audio.
How long has it taken to get here? Subversion’s got that too:
Yuna:coreaudiobook cadamson$ svn log -r1 .
----------------------------------------------------
r1 | invalidname | 2010-01-06 17:18:06 -0500 (Wed, 06 Jan 2010) | 2 lines
Created the usual trunk, tags, branches.
----------------------------------------------------Yeah, almost two years from when I took over after original author-fade… probably will actually be two years by the time we handle rewrites and comments from reviewers, Lion fixes (AudioComponentInstance instead of ComponentInstance, etc.), and get through the production process of copy-edit, layout, and printing.
Oh, and will you be getting your money’s worth?
Yuna:book cadamson$ wc ch??/Ch??.txt
208 3972 24817 ch01/ch01.txt
295 5440 32883 ch02/Ch02.txt
371 5344 34641 ch03/Ch03.txt
623 6674 44278 ch04/Ch04.txt
318 4052 26261 ch05/Ch05.txt
515 5161 37540 ch06/Ch06.txt
884 10837 72025 ch07/Ch07.txt
740 9156 60837 ch08/Ch08.txt
738 8070 54086 ch09/Ch09.txt
910 9949 68737 ch10/Ch10.txt
498 6233 40981 ch11/Ch11.txt
77 2080 12570 ch12/Ch12.txt
6177 76968 509656 total
Yuna:book cadamson$ At a standard estimate of 250 words per page, this 80,000-word book should clock in around 320 pages. We have a few figures, so maybe it’ll be more like 350. We’ll see. Enough to cover what we thought was crucial, not so big that it could stop a charging bison.
More to do, but nice to have finally reached this milestone. Thanks to everyone who’s waited so patiently for it.
