[Time code];

Great question from Ross on the last post:

Is QTJ not dead? I bought your book today out of interest, but it appears QTJ is a long time without updates and looking a little abandoned.
Would you still use it on a new project?

OK, first, thanks for buying the book. I hope you enjoy it.

Would I use QTJ today… it really depends. If a client asked me, my first two questions would be “what kind of app do you want to use it for”, and “how long does this app need to be viable?”

I’d need to know that so I could assess the project against what the facts currently are with QuickTime for Java. First, Apple stopped adding features to QTJ a few years ago. QuickTime 7 was the first time that the QT API picked up significant new functionality, without any attempt or plan to expose that functionality to QTJ. Among QT 7′s most important adds are support for frame-reordering codecs, metadata, and an audio API that basically makes it easier to to Core Audio stuff from a higher level in the call stack. The frame-reordering stuff was a profound change, necessary to support H.264, which replaces many sample-related functions: AddMediaSample became AddMediaSample2, and MediaTimeToSampleNum became either MediaDecodeTimeToSampleNum or MediaDisplayTimeToSampleNum (depending on what you were doing, since frame-reordering codecs require you to stop assuming that frames are decoded in the order they’ll be displayed and vice versa). Without this stuff exposed to QTJ, the practical upshot is that lower-level stuff doesn’t work if you try to use H.264, as I found with a capture app with an all-Java onscreen preview (more on that later, BTW), which choked on the frame decompress if you tried to compress the capture data to H.264 on the fly.

So, for some things, QTJ is already unacceptable. But what’s the future, and what are the alternatives? The future is that QTJ will go away. For one thing, they’ve obviously stopped putting new features into it. At an Atlanta Leopard Tech Day in 2006, some Apple reps asked me to not encourage people to start into QTJ projects, as the technology was not on their road map going forward. They will fix bugs and handle ADC support incidents, but it’s not getting any new work.

Of course, even if QTJ weren’t specifically on the “stay away” list, it’s based on some technologies that are already deprecated (notably QuickDraw), and the whole thing sits atop the main QuickTime API. Since we know that QuickTime, like all Carbon APIs, won’t be upgraded to 64-bit , it will eventually be deprecated as well (John Siracusa’s Leopard Review makes ths point better than I can). So even if Apple were maintaining QTJ, it’s a dead-end, because its underpinnings are destined for obsolescence.

But when? Apple is serious about getting rid of old stuff, and they usually follow three steps:

1. Informally tell developers to stop using a technology
2. Formally deprecate it
3. Remove it from the OS

Right now, Apple is telling developers to stop using Carbon. It’s still available in Leopard, but its days are numbered. So maybe it will be deprecated in 10.6 and actually be absent in 10.7, but that’s pretty aggressive and possibly a worst-case scenario (and assumes that Apple can get Adobe and Microsoft, with their large Carbon code-bases, to go along). With two years between Mac OS X releases, that sets a minimum four-year viability on apps that require Carbon, including anything that uses the old QuickTime, including QTJ.

But what’s the alternative? The Apple guys said that in general, they were advocating QTKit. I thought at the time that that was because it’s object-oriented and thus more friendly to QTJ converts. But now we know that QTKit is the only 64-bit option, and long term, is presumably the future of QuickTime.

Well, the future of QuickTime on the Mac. Because after all, it’s single-platform.

So this gets me back to answering the question of would I recommend QuickTime for Java today. Despite Apple’s warnings, there are a few cases where I would. I haven’t fully broken this down yet — maybe it could be flow-charted — but here are the various factors I think go into a consideration:

• Do you need to run on Mac and Windows? Go ahead and use QTJ, it’s more practical than writing straight-C QuickTime for both platforms, and the alternative would be to use QTKit on the Mac and the little-understood QuickTime COM on Windows. You still might be screwed long-term, but if Carbon QuickTime is going away, that leaves a big open question of what Apple will do with QuickTime for Windows (they didn’t reply when I asked last year)
• Do you need to do editing or access low-level functions? QTKit remains an 80/20 proposition, so if you’re in the 80% that just wants to play media or maybe support cut/copy/paste editing, you’re good. If you want to support things like effects, tweens, or write samples to a new movie, I’d have to do some prototyping before I was comfortable recommending it… those kinds of hardcore features are the 20% that may not be in there. In cases like this, QTJ or old QuickTime may be a better choice
• How long does this code have to be viable? For green-fields development that you hope is going to be in use for years, QTJ and QuickTime are probably a bad choice, given that we’ve been explicitly told to move off of them. For short term stuff, including demos, experiments, and prototypes, they both work today and should be viable for a while yet. I also think hobbyists can go ahead and use QTJ because it’s probably a more practical way to introduce yourself to QuickTime’s concepts than using the straight-C API, and there’s no definitive book on QTKit yet (and even QTKit often assumes you get some of QuickTime’s deeper concepts, which doesn’t help the beginner).

So, does that answer your question? I’m figuring not, because I’ve just burned about a thousand words to say it depends

This’ll burn someone eventually — according to a small note at the bottom of an e-mail from Apple to developers about QuickTime updates, “for QT Java developers, Java applets are now required to be “signed” in order to access QuickTime for Java and take advantage of QTJava.”

You used to have to sign to get access to the file system or the SequenceGrabber; now you have to be signed for any QTJava applet. The fact that QTJ has been the attack vector in a number of security exploits, including the infamous one about accessing your camera and using it without your permission, may have led Apple to decide the entire framework was untrustworthy. Probably a good decision, but I bet someone’s got QTJ applets and will get a nasty surprise when their users upgrade to Leopard and the applets stop working.

Cooper asked for OpenID support, and I thought it was a good suggestion, so I tried to install the WP-OpenID+ plugin, but it kicks up a parse error when activated. Not a big surprise, considering it’s supported up to WordPress 2.1.2 and I’m now running 2.3.1.

Anybody know of a working OpenID plugin for WordPress?

The big news in QTKit for Leopard is capture, but I was wondering what else had changed. After all, according to the Leopard Graphics and Media Overview, “another key benefit of QTKit in Leopard is the ability to run in 64-bit mode. In fact, it’s the only way to access QuickTime in 64-bit applications [...] The current C-based QuickTime API will only be supported in 32-bit applications.” Given that 64-bit is the future, you have to therefore conclude that QTKit is also the future of application-level multimedia on the Mac platform.

Which makes me wonder: how quickly is it evolving as the destined replacement for the straight-C QuickTime API?

So, I diff‘ed the header files. Tiger has 10, while Leopard has 28:

Obviously, the new classes are overwhelmingly about capture. In fact, there are more classes relating to capture, than to all other QTKit functionality combined. It’s a little bit more granular and subclassy than I’ve seen elsewhere in Cocoa, but that might just be me getting hung up on Hillegass’ comments in his book about how Cocoa really isn’t about heavy use of subclassing (as opposed to, say, Java).

So, given that that’s new functionality, what else has changed? Here are some highlights from the diffs of classes that exist in both Tiger and Leopard QTKit:

• Several classes #if around importing QuickTime.h if they’re compiling for 64-bit.

• Because of that, QTKitDefines.h defines a number of constants (mostly OSTypes/4CC’s) for constants that would otherwise have been picked up from Movies.h, QuickTimeComponents.h and ImageCompression.h. These include file types, media types, codec types, codec quality constants, and graphics modes.

• QTMovie picks up a number of methods, including chapter-related methods, thread-safety methods that resemble QuickTime’s EnterMoviesOnThread, segment-level editing (yay), some new methods for initializing empty movies with writable outputs (i.e., for capture), and stuff related to two new categories (QTMovieVisualContext and QTMovieVisualSupport) that I haven’t really looked at.

• QTMovieView exposes getters and setters for the visibility of specialty controller widgets: zoom, step, translate, volume, hotspot (in the old QTVR sense?), etc.

• QTTime works with SMPTE times (yay!)

• QTTimeRange offers some new convenience methods for calculating the intersection or union of time ranges, plus QTSTrings to represent them.

• QTTrack has some “aperture mode” methods similar to the QTMovieVisualSupport stuff in QTMovie.

So interesting, but I’m not sure how complete a replacement it is for QuickTime yet. I still don’t see, for example, how I create things like effects tracks or tweens, which would be essential for serious editing apps (rubber-banding the volume in a Garage Band type app is a pretty straightforward application of a volume tween, for example). The key may be the new QTSampleBuffer class, defined by Apple thusly (boy, it’ll be nice when the ADC website has this stuff):

This class provides format information, timing information, and metadata on media sample buffers. QTSampleBuffer objects contain data from media samples as well as metadata about those samples, including format information, timing information, and other attributes. Some extended information can be accessed via a QTSampleBuffer’s attributeForKey: and sampleBufferAttributes methods, using the keys described in the Constants section. In addition to these explicit methods, applications can use key-value coding to get extended attributes. For an object that supports a given attribute, valueForKey: will be functionally identical to attributeForKey:. Applications wishing to observe changes for a given attribute can add a key-value observer where the key path is the attribute key.

So, assuming it’s possible to get a sample and somehow decompress it, you should be able to do, say, waveforms and thumbnails in Final Cut-like editing GUIs. If you can build samples one by one and add them to the media, then you can generate whatever kinds of movies and tracks suit you. To me, this is the way interesting stuff, and I’ll be trying to figure out just how far QTKit can get you without dropping into QuickTime.

Because some day in the bright, shiny, 64-bit-only future, dropping into QuickTime isn’t going to be an option.

My muddled-through QTKit capture application is beginning to work:

Click for full-size image

Actually, I wasn’t too far off in getting the preview up. There’s a QTCaptureSession object that I’d alloc‘ed but cluelessly failed to init (yeah, Obj-C is definitely a second language to me, still), and I’d overlooked the need to do a [captureSession startRunning];, which looks a lot like starting up an idler thread for the old SequenceGrabber, just much less ugly.

What unblocked me was five minutes with the “Building a Simple QTKit Capture Application” tutorial… I don’t see it on Apple’s website, but it’s part of the Leopard dev docs, under QuickTime > Conceptual > QTKitCaptureProgrammingGuide.

Oh, none of those buttons do anything yet – right now it’s just previewing the default device. What I aim to figure out is how to achieve QTKit’s huge advantage over old QuickTime capture: the ability to use multiple capture devices simulatanously. What I’m going to try is to have each window owning a QTCaptureSession, which is naturally associated with the QTCaptureView in the window. My concerns are how I’m going to do the device selection GUI (I just bought Aaron Hillegass’ Cocoa Programming for Mac OS X because I saw it covers sheets), and how to manage multiple windows (Interface Builder gives me the first window for free… later instances presumably need to be instantiated and made visible in code).

Still, not bad for banging my head cluelessly against the class documentation. Might be a pleasure to use once I know what I’m actually doing. If I really build this example out, it might be nice to pair an audio device with each window and do an animated level meter (assuming that QTKit has a method to do that, otherwise maybe there’s a way to use the straight-C QuickTime approach… except that’s for movies and not capture… still, it’s doable, I had capture-time level-metering in the QTJ book after all, just have to assume that QTKit capture will let you escape to regular QT if you really need to, like the movie playback/editing parts of QTKit do).

Anyways, seeing the preview work made my weekend. And after all the hackery and misery of juggling Java and native threads in Keaton (one reason I rarely hack on it), coding Obj-C qua Obj-C was really pleasant this time.

Apple doesn’t provide an SDK for Windows apps in Leopard.

I just wanted to make sure everyone knew that.

Also, they don’t have an SDK for Wii games. Or DoCoMo iMode mobile apps. Or for the Zune. Or the N-Gage.

In fact, it looks like Apple’s SDKs are primarily focused on writing software for… Apple’s Mac platform! Imagine that!

I’m talking to you, Java colleagues.

I’ve really been sticking my foot in it today, jumping into the “no JDK 6 on Leopard is an outrage” threads on The Java Posse’s Google Group and Javalobby. Honestly, I mean well: I think the Java community is coming off as shrill and obnoxious, and I would like us to not make complete fools of ourselves.

It’s not going well.

See, here’s the position I’m staking out: by and large, Java is not a language for developing Mac applications:

• Double-clickable Java apps are few and very far between. And they’re sometimes single-platform anyways (every project I worked on at full-time jobs was Windows-only by management decree)
• Flash is wiping out the last of the applets
• Apple is not a significant server-side player. Since the overwhelming majority of Java developers write server-side code, it follows that what they’re writing, even if they write it on a Mac, will almost certainly not be running on a Mac in production.

In other words, nearly everyone using Macs to write Java apps is not actually writing Mac apps. Obvious question: why is it Apple’s duty to support this?

This argument is getting absolutely nowhere. What I’m seeing instead is a lot of the same foolish statements, over and over again:

Steve Jobs hates Java

Really? Then why is there still a Mac Java team, that’s patiently handling all the hate on the java-dev list? If he “hates” Java, he could just disband that team and reassign them to some other nefarious purpose, such as the “iTunes Britney Spears Career Revival Task Force”, or whatever other evil you’d care to ascribe to him.

It has to be an issue of inadequate resources

Apple had JDK 6 b88 running on Tiger last Fall. They were pretty much done. Really think there were more changes between b88 and 1.0, or between Tiger and Leopard, that the team couldn’t manage to handle it all in a year?

I don’t know what’s going on. There are interesting hypotheses: maybe Apple’s JDK didn’t pass the JCK yet. Or maybe there’s a licensing or legal dispute that hasn’t been resolved. But I don’t know, and neither does anyone else who can talk about it.

Apple needs to make some kind of public statement!

“Apple does not comment on unannounced products.” What, you hadn’t heard that before? Are you new here?

Yeah, well, I’m gonna switch to Linux, and so are all the other Java developers

Right. Because every Linux distro welcomes Java with open arms. Except Debian, and all the others like it that only ship FOSS software, which Java is not (as of this writing, though OpenJDK will fix that in time).

And if every Mac Java developer switched, literally how big a deal would that be? Estimates of the size of the Java developer population range from 3 million to 6 million. Let’s be generous and assume 5 million. What number of them use Macs? Sure, you see a lot at conferences, but those are the guys who get to pick their own machines (or buy them themselves because they’re independent), and don’t have Windows forced on them by the Stalinist IP department. How ’bout 10%, which is significantly higher than the Mac’s estimated market share worldwide. So, that means about 500,000 Mac-based Java developers… which I still think is way high, but I’m giving the other side the benefit of numbers for this argument.

Because you see, Apple sold 2.1 million Macs last quarter (Q4 2007 by their fiscal year), up 34% from Q4 2006, and half of the buyers in the Apple Stores were switchers. With that trend, they’ll sell more than 10 million Macs in fiscal 2008, and if half the buyers are switchers, then the hypothetical 500,000 departing Java developers will be replaced by 5,000,000 switchers.

In the big picture, Java developers just aren’t as important as we tend to think we are.

But Apple has to have up-to-date Java to be a player in the enterprise

As of today, AAPL‘s market cap is US$161.1B. Greater than Intel, greater than IBM, about two and a half times Dell‘s market cap, and about eight times Sun‘s. So enterprise strategy or not, I think they’ll do OK.

And with the iPod, Apple TV, and iPhone, Apple has as much in common with the consumer electronics companies as with the computer companies, and nobody complains about Sony or Samsung not having an enterprise computing strategy.

Still, Steve Jobs said at JavaOne that the Mac would be the best Java development platform available

Yep. And I’m sure this is the only time in recorded history that a company has not lived up to its trade show keynote hyperbole.

But they should still support their developers!

They do support their developers: the ones writing Cocoa, Carbon, and even POSIX apps. If they’re not in a hurry to help you write a webapp that runs on Linux or Windows servers, and may well block Mac user-agents, well, I don’t really blame them.

Apple’s always hated Java anyways

Right. Paying Sun a license fee for the privilege of then hiring engineers to port the JDK to Apple’s operating system — something that was much harder on the non-Unixy Classic Mac OS — and then including Java by default on every copy of Mac OS X since 10.0, supporting multiple JVM versions and CPU architectures. A lot of languages should love to be so “hated”.

Well, still, supporting Java developers should be Apple’s job, not Sun’s

As I’m arguing, there are so few Java apps actually run on the Mac, that they shouldn’t be considered a form of Mac app anymore, so you’re basically saying that Apple should support development for other companies’ platforms. And I can’t think of any other case in which they, or anyone else, enthusiastically does this.

Consider by the way, the strange case of ME development. Sun has always only provided developer tools for Windows, only just starting to pull together a Linux version of the Wireless Tookit (WTK) this year. And with every Java API that required native code, like JMF or JavaTV, they’ve consistently supported Windows and not much else. The implication is clear: Sun thinks Java developers should be using Windows. Maybe Apple’s just coming around to Sun’s way of thinking.

But I bought my Mac assuming that JDK 6 would be in Leopard

Then you ignored Adamson’s First Law: all software is vapor until it ships

Well, me and all my friends are going to switch to Linux anyways, and Mac OS will die

As Fake Steve might say, don’t let the genuine Swiss crystal, laser-etched door hit you on the way out.