[Time code];

As mentioned before, I’m not a big fan of panels at developer conferences, and whenever I’m in one, I deliberately look for points of contention or disagreement, so that we don’t end up as a bunch of nodding heads who all toe a party line. Still, at last week’s CocoaConf in Raleigh, I may have outdone myself.

When an iOS developer in the audience asked if he should get into Mac development, most of the panel said “go for it”, but I said “don’t bother: the Mac is going to be gone in 5-10 years. And what’s going to kill it is iOS.”

This is wrong, but not for the reason I’ll initally be accused of.

First, am I overstating it? Not in the least. Just looking at things where they stand today, and the general trends in computing, I can’t see the Mac disappearing in less than five years, but I also can’t imagine it prevailing more than another 10.

This isn’t the first time I’ve put an unpopular prediction out there: in 2005, back when I was with O’Reilly and right after the Intel transition announcement, I I predicted that Mac OS X 10.6 would come out in 2010 and be Intel-only. This was called “questionable”, “dumb”, and “ridiculous advice”. Readers said I had fooled myself, claimed I was recommending never upgrading because something better is always coming (ie, a straw man argument), argued Intel wouldn’t be that big a deal, and predicted that PPC machines would still be at least as common as Intel when 10.6 came out. For the record, 10.6 came out in August, 2009 and was indeed Intel-only. Also, Intel Macs had already displaced PowerPC in the installed base by 2008.

So, before you tell me I’m an idiot, keep in mind that I’ve been told that lots of times before.

Still, punditry has been in the “Mac is doomed” prediction business for 20 years now… so why am I getting on board now?

Well, the fact that I’m writing this blog on my iPad is a start. And the fact that I never take my MacBook when I travel anymore, just the iPad. And the issue that Lion sucks and is ruining so much of what’s valuable about the Mac. But that’s all subjective. Let’s look at facts and trends.

iOS Devices are Massively Outselling Mac OS X

One easy place to go for numbers is Apple’s latest financial reports. For the quarter that ended Sept. 30, 2011 (4Q by Apple’s financial calendar), the company sold 4.89 million Macs, 11.12 million iPads, and 17.07 million iPhones. That means the iPad is outselling all models of Mac by a factor of more than 2 to 1.

The numbers also mean that iOS devices are outselling Mac OS X devices by a ratio of at least 5.7 to 1… and it must be much more, since Apple also sold 6.62 million iPods (since these aren’t broken down by model, we don’t know which are iOS devices and which aren’t).

While the Mac is growing, iOS is growing much faster, so this gap is only going to continue to grow.

Goin’ Mobile

Let’s also think about just which Macs are selling. The answer is obvious: laptops. The Mac Unit Sales chart in this April 2011 MacWorld article shows Apple laptops outselling desktops by a factor of nearly 3-to-1 for the last few quarters. Things are so cold for desktops that there is open speculation about whether the Mac Pro will ever be updated, or if it is destined to go the way of the Xserve.

Now consider: Apple has a whole OS built around mobility, and it’s not Mac OS X. If the market is stating a clear preference for mobile devices, iOS suits that better than the Mac, and the number of buyers who prefer a traditional desktop (and thus the traditional desktop OS) are dwindling.

Replace That Laptop!

Despite the fact that it’s only about 28% of Apple laptop sales, I would argue the definitive Mac laptop at this point is the MacBook Air. It’s the most affordable MacBook, and has gotten more promotion this year than any other Mac (when was the last time you saw an iMac ad?). It also epitomizes Apple’s focus on thinness, lightness, and long battery life.

But on any of these points, does it come out ahead of an iPad 2? It does not. And it costs twice as much. The primary technical advantage of the Air over the iPad 2 are CPU power, RAM, and storage.

Is there $500 worth of win in having bigger SSD options or a faster CPU?

Few People Need Trucks…

“But”, you’re saying, “I can do real work on a Mac”. Definitely true… but how much of it can you really not do on an iPad? For last month’s Voices That Matter conference, I wrote slides for two talks while on the road, using Keynote, OmniGraffle, and Textastic… the same as I would have used on my Mac Pro, except for using Xcode to work with code instead of Textastic. I also delivered the talk via Keynote off the iPad with the VGA cable, and ran the demos via the default mirroring over the cable.

I’ve gone three straight trips without the laptop and really haven’t missed it. And I’m not the only one. Technologizer’s Harry McCracken posted a piece the other week on How the iPad 2 Became My Favorite Computer.

Personally, the only thing that I think I’m really missing on my iPad is Xcode, so that I could develop on the road. I suspect we’ll actually see an Xcode for iPad someday… the Xcode 4 UI changes and its single-window model made the app much more compatible with iPad UI conventions (the hide-and-show panes could become popovers, for example). And lest you argue the iPad isn’t up to the challenge, keep in mind that when Xcode 1.0 was first released in Fall, 2003, a top-of-the-line Power Mac G5 had a dual-core 2.0GHz CPU and 512 MB RAM, whereas the current iPad 2 has a dual-core A5 running at 1.0GHz and 512 MB RAM, meaning iPad is already in the ballpark.

…and Nobody Needs a Bad Truck

“But Mac OS X is a real OS,” someone argues. Sure, but two things:

• How much does that matter – a “real OS” means what? Access to a file system instead of having everything adjudicated by apps? I’d say that’s a defining difference, maybe the most important one. But it matters a lot less than we might think. Most files are only used in the context of one application, and many applications make their persistence mechanism opaque. If your mail was stored in flat files or a database, how would you know? How often do you really need or want the Finder? With a few exceptions, the idea of apps as the focus of our attention has worked quite well.

  What exceptions? Well, think of tasks where you need to combine information from multiple sources. Building a web page involves managing HTML, CSS, JavaScript, and graphic files: seemingly a good job for a desktop OS and bad job for iOS. But maybe it argues for a task-specific app: there’s one thing you want to do (build a web page), and the app can manage all the needed files.

• For how long will Mac OS X be a “real OS”? – Anyone who follows me on Twitter knows I don’t like Lion. And much of what I don’t like about it is the ham-handed way iOS concepts have been shoehorned into the Mac, making for a good marketing message but a lousy user experience. Some of it is just misguided, like the impractical and forgettable LaunchPad.

  But there’s a lot of concern about the Mac App Store, and the limitations being put on apps sold through the MAS. This starts with sandboxing, which makes it difficult or impossible for applications to damage one another or the system as a whole. As Andy Ihnatko pointed out, this utterly emasculates AppleScript and Automator. Daniel Steinberg, in his “Mac for iOS Programmers” talk at CocoaConf, also wondered aloud if inter-application communication via the NSDistributedNotificationCenter will be the next thing to go. And plenty of developers fear that in time, Apple will prohibit third-party software installs outside of the MAS.

Steve Jobs once made a useful analogy that many people need cars and only a few need trucks, implying that the traditional PC as we’ve known it is a “truck”, useful to those with advanced or specific needs. And that would be fine… if they were willing to let the Mac continue to be the best truck. But instead, the creep of inappropriate iPad-isms and the iOS-like limitations being put on Mac apps are encroaching the advanced abilities that make the truck worth owning in the first place.

The Weight of the Evidence

Summarizing these arguments:

• iOS devices are far outselling Mac OS X, and the gulf is growing
• The iPad and the MacBook (the only Mac that matters) are converging on the same place on the product diagram: an ultra-light portable computing device with long battery life
• iOS is already capable of doing nearly anything that Mac OS X can do, and keeps getting better.
• Mac OS X shows signs of becoming less capable, through deliberate crippling of applications by the OS.

Taken together, the trends seem to me like they argue against the future of Mac OS X. Of the things that matter to most people, iOS generally does them better, and the few things the Mac does better seem like they’re being actively subverted.

Some people say the two platforms will merge. That’s certainly an interesting possibility. Imagine, say, an iOS laptop that’s just an iPad in a clamshell with a hardware keyboard, likely still cheaper than the Air, and the case for the MacBook gets weaker still.

Given all this, I think OS X becomes less necessary to Apple — and Apple users — with each passing year. When does it reach a point where OS X doesn’t make sense to Apple anymore? That’s what I’m mentally pencilling in: 5-10 years, maybe after one or two more releases of OS X.

Wrong

But in the beginning, I said I was wrong to say that an iOS developer shouldn’t get into Mac programming because the Mac is doomed. And here’s why that’s wrong: you shouldn’t let popularity determine what you choose to work with. Chasing a platform or a language just because it’s popular is always a sucker’s bet. For one thing, times change, and the new hotness becomes old news quickly. Imagine jumping into Go, which in 2009 grew fast enough to become the “language of the year” on the TIOBE programming language index. It’s currently in 34th, just behind Prolog, ahead of Visual Basic .NET, and well behind FORTRAN (seriously!).

Moreover, making yourself like something just because it’s popular doesn’t work. As Desktop Java faded into irrelevance, I studied C++ and Flash as possible areas of focus, and found that I really didn’t like either of them. Only when the iPhone OS came along did I find a platform with ideas that appealed to me.

Similarly, I greatly value the time I spent years ago studying Jini, Sun’s mis-marketed and overblown self-networking technology. Even though few applications were ever shipped with it — Jini made the fatal mistake of assuming a Java ubiquity that did not actually exist outside of Sun’s labs — the ideas it represented were profound. Up to that point, I had never seen an API that presented such a realistic view of networking, one that kept in mind the eight fallacies of distributed computing and made them managable, largely by treating failure as a normal state, rather than an exceptional one. In terms of thinking about hard problems and how stuff works (or doesn’t work) in the real world, learning about Jini was hugely valuable to me at the time I encountered it.

And had I known Jini was doomed, would I have been better off studying something more popular, like Java Message Service (which my colleagues preferred, since it “guaranteed” message delivery rather than making developers think about failure)? I don’t think so. And so, even if I don’t think the Mac has a particularly bright future, that’s no reason for me to throw cold water on someone’s interest in learning about the platform. There are more than 20 years of really good ideas in the Mac, and if some of them enlighten and empower you, why not go for it?

I’m surprised how fast iOS conference slides go old. I re-used some AV Foundation material that was less than a year old at August’s CocoaConf, some of it already seemed crusty, not the least of which was a performSelectorOnMainThread: call on a slide where any decent 2011 coder would use dispatch_async() and a block. So it’s probably just as well that I did two completely new talks for last weekend’s Voices That Matter: iOS Developers Conference in Boston.

OK, long-time readers — ooh, do I have long-time readers? — know how these posts work: I do links to the slides on Slideshare and code examples on my Dropbox. Those are are at the bottom, so skip there if that’s what you need. Also, I’ve put URLs of the sample code under each of the “Demo” slides.

The AV Foundation talk is completely limited to capture, since my last few talks have gone so deep into the woods on editing (and I’m still unsatisfied with my mess of sample code on that topic that I put together for VTM:iPhone Seattle in the Spring… maybe someday I’ll have time for a do-over). I re-used an earlier “capture to file and playback” example, and the ZXing barcode reader as an example of setting up an AVCaptureVideoDataOutput, so the new thing in this talk was a straightforward face-finder using the new-to-iOS Core Image CIDetector. Apple’s WWDC has a more ambitious example of this API, so go check that out if you want to go deeper.

The Core Audio talk was the one I was most jazzed about, given that Audio Units are far more interesting in iOS 5 with the addition of music, generator, and a dozen effects units. That demo builds up an AUGraph that takes mic input, a file-player looping a drum track, and an AUSampler allowing for MIDI input from a physical keyboard (the Rock Band 3 keyboard, in fact) to play two-second synth sample that I cropped from one of the Soundtrack loops, all mixed by an AUMultichannelMixer and then fed through two effects (distortion and low pass filter) before going out to hardware through AURemoteIO. Oh, and with a simple detail view that lets you adjust the input levels into the mixer and to bypass the effects.

The process of setting up a .aupreset and getting that into an AUSampler at runtime is quite convoluted. There are lots of screenshots from AULab in the slides, but I might just shoot a screencast and post to YouTube. For now, combine WWDC 2011 session #411 with Technical Note TN2283 and you have as much a fighting chance as I did.

I’ll be doing these talks again at CocoaConf in Raleigh, NC on Dec. 1-2, with a few fix-ups and polishing. The face-finder has a stupid bug where it creates a new CIDetector on each callback from the camera, which is grievously wasteful. For the Core Audio AUGraph, I realized in the AU property docs that the mixer has pre-/post- peak/average meters, so it looks like it would be easy to add level meters to the UI. So those versions of the talks will be a little more polished. Hey, it was a tough crunch getting enough time away from client work to get the sample code done at all.

Speaking of preparation, the other thing notable about these talks is that I was able to do the slides for both talks entirely on the iPad, while on the road, using Keynote, OmniGraffle, and Textastic. Consumption-only device, my ass.

Slides and code

• Capturing Stills, Sounds, and Scenes with AV Foundation
  • Slides (on slideshare.net)
  • Code: VTM_AVRecPlay.zip, VTMFaceFinder.zip
• Core Audio Cranks It Up
  • Slides (on slideshare.net)
  • Code: VTMAUGraphDemo.zip

Dennis Ritchie, a co-creator of Unix and C, passed away a few weeks ago, and was honored with many online tributes this weekend for a Dennis Ritchie Day advocated by Tim O’Reilly.

It should hardly be necessary to state the importance of Ritchie’s work. C is the #2 language in use today according to the TIOBE rankings (which, while criticized in some quarters, are at least the best system we currently have for gauging such things). In fact, TIOBE’s preface to the October 2011 rankings predicted that a slow but consistent decline in Java will likely make C the #1 language when this month’s rankings come out.

Keep in mind that C was developed between 1969 and 1973, making it nearly 40 years old. I make this point often, but I can’t help saying it again, when Paul Graham considered the possible traits of The Hundred-Year Language, the one we might be using 100 years from now, he overlooked the fact that C had already made an exceptionally good start on a century-long reign.

And yet, despite being so widely used and so important, C is widely disparaged. It is easy, and popular, and eminently tolerated, to bitch and complain about C’s primitiveness.

I’ve already had my say about this, in the PragPub article Punk Rock Languages, in which I praised C’s lack of artifice and abstraction, its directness, and its ruthlessness. I shouldn’t repeat the major points of that article — buried as they are under a somewhat affected style — so instead, let me get personal.

As an 80′s kid, my first languages were various flavors of BASIC for whatever computers the school had: first Commodore PETs, later Apple IIs. Then came Pascal for the AP CS class, as well as a variety of languages that were part of the ACSL contests (including LISP, which reminds me I should offer due respect to the recent passing of its renowned creator, John McCarthy). I had a TI-99 computer at home (hey, it’s what was on sale at K-Mart) and its BASIC was godawful slow, so I ended up learning assembly for that platform, just so I could write programs that I could stand to run.

C was the language of second-year Computer Science at Stanford, and I would come back to it throughout college for various classes (along with LISP and a ruinous misadventure in Prolog), and some Summer jobs. The funny thing is that at the time, C was considered a high-level language. At that time, abstracting away the CPU was sufficient to count as “high-level”; granted, at the time we also drew a distinction between “assembly language” and “machine language”, presumably because there was still someone somewhere without an assembler and was thus forced to provide the actual opcodes. Today, C is considered a low-level language. In my CodeMash 2010 talk on C, I postulated that a high-level language is now expected to abstract away not only the CPU, but memory as well. In Beyond Java, Bruce Tate predicted we’d never see another mainstream language that doesn’t run in a VM and offer the usual benefits of that environment, like memory protection and garbage collection, and I suspect he’s right.

But does malloc() make C “primitive”? I sure didn’t think so in 1986. In fact, it did a lot more than the languages at the time. Dynamic memory allocation was not actually common at that time — all the flavors of BASIC of that time have stack variables only, no heap. To have, say, a variable number of enemies in your BASIC game, you probably needed to do something like creating arrays to some maximum size, and use some or all of those arrays. And of course relative to assembly language, where you’re directly exposed to the CPU and RAM, C’s abstractions are profound. If you haven’t had that experience, you don’t appreciate that a = b + c involves loading b and c into CPU registers, invoking an “add” opcode, and then copying the result from a register out to memory. One line of C, many lines of assembly.

There is a great blog from a few years ago assessing the Speed, Size, and Dependability of Programming Languages. It represents the relationship between code size and performance as a 2-D plot, where an ideal language has high performance with little code, and an obsolete language demands lots of work and is still slow. These two factors are a classic trade-off, and the other two quadrants are named after the traditional categorization: slow but expressive languages are “script”, fast but wordy are “system”. Go look up gcc – it’s clearly the fastest, but its wordiness is really not that bad.

Perhaps the reason C has stuck around so long is that its bang for the buck really is historically remarkable, and unlikely to be duplicated. For all the advantages over assembly, it maintains furious performance, and the abstractions then built atop C (with the arguable exception of Java, whose primary sin is being a memory pig) sacrifice performance for expressiveness. We’ve always known this of course, but it takes a certain level of intellecutual honesty to really acknowledge how many extra CPU cycles we burn by writing code in something like Ruby or Scala. If I’m going to run that slow, I think I’d at least want to get out of curly-brace / function-call hell and adopt a different style of thinking, like LISP.

I was away from C for many years… after college, I went on a different path and wrote for a living, not coming back to programming until the late 90′s. At that point, I learned Java, building on my knowledge of C and other programming languages. But it wasn’t until the mid-2000′s that I revisted C, when I tired of the dead-end that was Java media and tried writing some JNI calls to QuickTime and QTKit (the lloyd and keaton projects). I never got very far with these, as my C was dreadfully rusty, and furthermore I didn’t understand the conventions of Apple’s C-based frameworks, such as QuickTime and Core Foundation.

It’s only in immersing myself in iOS and Mac since 2008 that I’ve really gotten good with calling C in anger again, because on these platforms, C is a first-class language. At the lower levels — including any framework with “Core” in its name — C is the only language.

And at the Core level, I’m sometimes glad to only have C. For doing something like signal processing in a Core Audio callback, handing me a void* is just fine. In the higher level media frameworks, we have to pass around samples and frame buffers and such as full-blown objects, and sometimes it feels heavier than it needs to. If you’re a Java/Swing programmer, have you ever had to deal with a big heavy BufferedImage and had to go look through the Raster object or whatever and do some conversions or lookups, when what you really want is to just get at the damn pixels already? Seems to happen a lot with media APIs written in high-level languages. I’m still not convinced that Apple’s AV Foundation is going to work out, and I gag at having to look through the docs for three different classes with 50-character names when I know I could do everything I want with QuickTime’s old GetMediaNextInterestingTime() if only it were still available to me.

C is underappreciated as an application programming language. Granted, there’s definitely a knack to writing C effectively, but it’s not just the language. Actually, it’s more the idioms of the various C libraries out there. OpenGL code is quite unlike Core Graphics / Quartz, just like OpenAL is unlike Core Audio. And that’s to say nothing of the classic BSD and other open-source libraries, some of which I still can’t crack. Much as I loathe NSXMLParser, my attempt to switch to libxml for the sake of a no-fuss DOM tree ended after about an hour. So maybe it’s always going to be a learning process.

But honestly, I don’t mind learning. In fact, it’s why I like this field. And the fact that a 40-year old language can still be so clever, so austere and elegant, and so damn fast, is something to be celebrated and appreciated.

So, thanks Dennis Ritchie. All these years later, I’m still enjoying the hell out of C.

Real life intervenes again (parsing PDF, whee!) and I have to cut short a planned epic iDevBlogADay entry. But I do want to bang out a few quick notes on various topics of interest.

The first is Core Audio in iOS 5, which we can now talk about publicly. If we go through the iOS 4.3 to iOS 5.0 API Differences document, we see that Audio Units accounts for a large number of diffs. This comes from the addition of a suite of units that finally make programming at the unit level a lot more interesting. Whereas we used to get a single effects unit (AUiPodEQ), we now get high and low pass filters, the varispeed unit, a distortion box, and a parametric EQ that lets us play with sliders instead of the “canned” EQ settings like “Bass Booster” and “Spoken Word”. Even more useful, we get the AUFilePlayer, meaning you can now put audio from a file at the front of an AUGraph, instead of the sheer pain of having to decode your own samples and pass them to the AUGraph through a CARingBuffer.

iOS also gets the AUSampler unit introduced in Lion, which provides a MIDI-controlled virtual instrument whose output is pitch-shifted from a source sample. This was shown off at WWDC, although providing the source to the unit by means of an .aupreset is still a dark (undocumented) art. This is the first actual MIDI audio unit in iOS, which makes the presence of Core MIDI more useful on the platform.

Core Audio giveth, but Core Audio also taketh away: iOS 5 removes (not deprecates) VoiceIOFarEndVersionInfo. This struct, and its related constants (specifically kVoiceIOFarEndAUVersion_ThirdParty), were documented as interoperating with a hypothetical “3rd-party device following open FaceTime standards”, something I took note of last May as possibly meaning that FaceTime was still ostensibly being opened up. With the removal of these APIs, I think that closes the book on Apple having any intention to live up to its vow to publish FaceTime as an open standard.

There’s lots more to talk about, but I’m already over my allotted blogging time, and work beckons. Perhaps you’d like to hear me speaking about this stuff and demo’ing it? I’m doing an all-new-for-iOS-5 Core Audio talk at two upcoming conferences:

• Voices That Matter: iOS Developers Conference, Nov. 12-13 in Boston, and $150 off with discount code BSSPKR5.
• CocoaConf, Dec. 2-3 in Raleigh, NC, which has early-bird pricing available through Oct. 31.

I’ll also be doing a talk about AV Foundation capture at these conferences. And back on audio, I just heard from my editor that the last three chapters of Core Audio should be in the Rough Cut on Safari Online Books in the next week or so, although I still have some work to do to clean up bits that are broken on Lion (read the sidebar on Audio Components if you’re having a problem creating audio units with the old Component Manager calls) and to clear out forward references to stuff that didn’t end up making the final cut for the book.

I’m a half-day late with my iDevBlogADay post… sorry.

So I was thinking about conference panels recently, something I don’t often attend or participate in. Panels to me seem like something that should work better than they usually do. You have smart, interesting people, but unless they know to “play ball”, to go out of their way to find ways to dig deeper or draw out conflicts and differences between each other, you tend to end up with a lot of head-nodding and personal pet theories that the rest of the panel doesn’t really have a stake in.

It’s not clear that the audience gets a lot out of it either. At Cocoaconf, I was on an iOS developer panel and the first question we got was the hopelessly played out “how do I get my app noticed” one. Ugh. You don’t need a panel for that, we’ve all been griping about that for three damn years now, and if we don’t have good answers yet, we’re never going to. Moreover, I’m not sure that attendees have a good sense of the potential of panels and how they can draw that out.

So here’s a solution. It comes to us by way of the fine folks at Harmonix, makers of Rock Band, Dance Central, the new iOS novelty VidRhythm, the rare iPod nano/Classic game Phase, etc. At their last two panels at PAX, they did a “Reverse Q&A”, which works like this: the panelists either ask big poll-type questions of the room, ask followup questions and get shouted-out responses from the crowd, or they ask “man on the street” style questions to whoever is at the front of the line for the mic. Either way, the topic is then followed up by the panelists and whoever from the crowd happens to be at the front of the line for the mics.

It still seems like a work-in-progress on the Harmonix podcasts, but there is a gem of a great idea here. Anyone who’s working in iOS and attending conferences has something interesting to say, and probably some unique real-world perspectives that wouldn’t necessarily be obvious to the kind of people that get picked for panels. We’re all self-employed hipster indies and authors, so we likely have little if any idea how iOS is playing out in big enterprises, how well or poorly it rubs shoulders with other technologies, etc. So in a Reverse Q&A Panel, I could ask these kinds of questions of whoever is first at the mic: “what do you use iOS for… how’s that working out… what’s missing that you think should be there…”

The responses we would get from the attendees would drive panel discussion, and in a sense, the person at the front of the line for the mic becomes a temporary member of the panel. In this, it’s a lot like the “open chair panel” that I’ve seen pulled off only once (at the Java Mobility conference in Jamuary 2008, where I saw the last gasp of the old world prior to the iPhone SDK announcement a few weeks later).

And I still like the format of both the Reverse Q&A and the Open Chair Panel more than I like straight-up open spaces, which at the end of the day are just chats, and chatting is best done over food and drink, like at the end of Cocoaconf where Bill Dudney, Scott Ruth and I grabbed two guys from Ohio U. that Bill had met and headed down to Ted’s for some bison burgers. That’s chattting. If you’re going to schedule a time and a room, it’s already more formal, and a structure helps set expectations.

I’m inclined to talk up Reverse Q&A as a format to the Cocoaconf and CodeMash organizers… would like to give this a try in the next few months.

And speaking of which, let’s practice. Here are some questions I’d like to ask of Reverse Q&A attendees. Feel free to answer any of them in the comments. I’d like to know what you guys and girls are thinking:

• Do you learn new platforms, languages, and frameworks from books, blogs, official docs, or what? (I want to know so I can figure out whether I should bother writing books anymore… signs point to no)
• What do other platforms do better than iOS?
• What’s the one App Store policy that pisses you off the most?
• Do you sell your own apps, write apps for someone else (employer, contract clients, etc.) or something else? Which of these do you think makes the most sense for you?
• Do you want more or less webapps in your life?

OK, you guys and girls talk for a while…

I hopped in on the MIDI chapter of the nearly-finished Core Audio book because what we’ve got now is a little obscure, and really needs to address the most obvious questions, like “how do I hook up my MIDI hardware and work with it in code?” I haven’t taken MIDI really seriously in the past, so this was a good chance to catch up.

To keep our focus on iOS for this blog, let’s talk about MIDI support in iOS. iOS 4.2 added CoreMIDI, which is responsible for connecting to MIDI devices via physical cables (through the dock connector) or wifi (on OSX… don’t know if it works on iOS).

Actually getting the connection to work can be touchy. Start with the Camera Connection Kit‘s USB connector. While Apple reps are typically quick to tell you that this is not a general-purpose USB adapter, it’s well-known to support USB-to-MIDI adapters, something officially blessed (with slides!) in Session 411 (“Music in iOS and Lion”) at WWDC 2011.

The catch is that the iPad supplies a tiny amount of power out the dock connector, not necessarily enough to power a given adapter. iOS MIDI keeps an updated list of known-good and known-bad adapters. Price is not a good guide here: a $60 cable from Best Buy didn’t work for me, but the $5 HDE cable works like a charm. The key really is power draw: powered USB devices shouldn’t need to draw from the iPad and will tend to work, while stand-alone cables will work if and only if they eschew pretty lights and other fancy power-draws. The other factor to consider is drivers: iOS doesn’t have them, so compatible devices need to be “USB MIDI Class”, meaning they need to follow the USB spec for driver-less MIDI devices. Again, the iOS MIDI Devices List linked above is going to help you out.

For keys, I used the Rock Band 3 keyboard, half off at Best Buy as they clear out their music game inventory (man, I need to get Wii drums cheap before they become collector’s items). This is only an input device, not an actual synthesizer, so it has only one MIDI port.

Once you’ve got device, cable, and camera connection kit, try playing your keys in GarageBand to make sure everything works.

If things are cool, let’s turn our attention to the Core MIDI API. There’s not a ton of sample code for it, but if you’ve installed Xcode enough times, you likely have Examples/CoreAudio/MIDI/SampleTools/Echo.cpp, which has a simple example of discovering connected MIDI devices. That’s where I started for my example (zip at the bottom of this blog).

You set up a MIDI session with MIDIClientCreate(), and make your app an input device with MIDIInputPortCreate(). Both of these offer callback functions that you set up with a function pointer and a user-info / context that is passed back to your function in the callbacks. You can, of course, provide an Obj-C object for this, though those of you in NDA-land working with iOS 5 and ARC will have extra work to do (the term __bridge void* should not be unfamiliar to you at this point). The first callback will let you know when devices connect, disconnect, or change, while the second delivers the MIDI packets themselves.

You can then discover the number of MIDI sources with MIDIGetNumberOfSources(), get them as MIDIEndpointRef‘s with MIDIGetSource(), and connect to them with MIDIPortConnectSource(). This connects your input port (from the previous graf) to the MIDI endpoint, meaning the callback function specified for the input port will get called with packets from the device.

MIDIPackets are tiny things. The struct only includes a time-stamp, length, and byte array of data. The semantics fall outside of CoreMIDI’s responsibilities; they’re summarized in the MIDI Messages spec. For basic channel voice messages, data is 2 or 3 bytes long. The first byte, “status”, has a high nybble with the command, and a low nybble indicating which MIDI channel (0-16) sent the event. The remaining bytes depend on the status and the length. For my example, I’m interested in the NOTE-ON message (status 0x9n, where n is the channel). For this message, the next two bytes are called “data 1″ and “data 2″ and represent the rest of the message. The bottom 7 bits of data 1 identify the note as a number (the high bit is always 0), while the bottom 7 bits of data 2 represent velocity, i.e., how hard the key was hit.

So, a suitable callback that only cares about NOTE-ON might look like this:

static void MyMIDIReadProc (const MIDIPacketList *pktlist,
                           void *refCon,
                           void *connRefCon)
{
   MIDIPacket *packet = (MIDIPacket *)pktlist->packet;
   Byte midiCommand = packet->data[0] >> 4;
   // is it a note-on
   if (midiCommand == 0x09) {
      Byte note = packet->data[1] & 0x7F;
      Byte veolocity = packet->data[2] & 0x7F;
      // do stuff now...

So what do we do with the data we parse from MIDI packets? There’s nothing in Core MIDI that actually generates sounds. On OSX, we can use instrument units (kAudioUnitType_MusicDevice), which are audio units that generate synthesized sounds in response to MIDI commands. You put the units in an AUGraph and customize them as you see fit (maybe pairing them with effect units downstream), then send commands to the instrument units via the Music Device API, which provides functions like MusicDeviceMIDICommand, and takes the unit, and the status, data1 and data2 bytes from the MIDI packet, along with a timing parameter. Music Device isn’t actually in Xcode’s documentation, but there are adequate documentation comments in MusicDevice.h. On OSX, the PlaySoftMIDI example shows how to play notes in code, so it’d be straight-forward to combine this with CoreMIDI and play through from MIDI device to MIDI instrument: get the NOTE-ON events and send them to the instrument unit of your choice.

On iOS, we don’t currently have instrument units, so we need to do something else with the incoming MIDI events. What I decided to do for my example was to just call System Sounds with various iLife sound effects (which should be at the same location on everyone’s Macs, so the paths in the project are absolute). The example uses 4 of these, starting at middle C (MIDI note 60) and going up by half-steps.

To run the example, you’ll actually have to run it twice: first to put the app on your iPad, then stop, plug in your keyboard, and run again. It might just be easier to watch this demo:

Anyways, that’s a brief intro to CoreMIDI on iOS. The book will probably skew a little more OSX, simply because there’s more stuff to play with, but we’ll make sure both are covered. I’m also going to be getting into this stuff at CocoaHeads Ann Arbor on Thursday night.

• Sample code:CoreMidiToSystemSoundExample.zip