Ben Thompson’s Stratechery blog had a recent series of posts on sustainability in App Store development, and the third part of the series focused particularly on productivity apps and how the nature of the App Store ecosystem has caused that category to implode.
I wrote a really long reply to him with some of my thoughts on the matter, something I’ve tweeted and blogged about before — and about part-way through the e-mail I realized it would be a pretty good blog on its own.
So I’m pasting it below in its e-mail form, with a few links and formatting added here and there.
Read the rest of this entry…
MacWorld ran a story last week to remind readers that A $5 App Isn’t Expensive, and imploring readers to stop being such cheapskates for the sake of the App Store economy.
Earth to MacWorld: It’s already too late. The market has spoken, and it refuses to pay for apps, even when the toxic side-effects of that are manifest.
MacWorld’s piece comes in part as a response to Michael Jurewitz’s five-part series on app pricing, posted on the eve of his return to Apple (and, presumably, a lot more circumspection about his future blogging). Jury sees the app pricing race to the bottom as a self-inflicted wound and urges developers to charge what their apps are worth.
Great advice… for anyone still around to take it.
Read the rest of this entry…
Sorry for another anime/manga-related post, but a thread on Twitter reminded me of some Apple misdeeds that need rectifying. It started with a pair of tweets, first from Zac Bertschy of Anime News Network:
I’m sure this has been asked a million times, but why are there so many goddamn bootleg manga apps on the iOS store?
@ANNZac I’ve tried to write Apple/Google about the links to bootleg sites. Neither has a reasonable way for reasonable people to complain
So let me back up a second… what we’re talking about are dedicated apps that read “scanlations”, which are comics (usually Japanese manga) that have been scanned, translated by fans into English, and posted for free to various websites or made available through channels like BitTorrent.
Zac righly calls this “bootlegging” because there is no question that copyright violation is involved. Entire works are being digitally redistributed with zero compensation to the original authors or publishers. What can make this a gray area is a question of whether or not any actual harm is done: if the work is unavailable in English, nor likely to ever be, then how can a scanlation eliminate a sale that could never be made? This is a fairly bogus defense because (as we’ll see), the untranslated works are just a minor part of the story. Moreover, we could apply the established tests of “Fair Use” under US copyright law, such as:
- Is the new work “transformative”? In other words, are we using the original to create a fundamentally different thing?
- How much of the original is being used?
- Does the copying impede future sale of the original work? Does it harm the creator?
Guidelines like this permit use like, say, presenting few pages of a comic in the context of a critical review or an academic paper: fundamentally new work, small amount of copying, doesn’t replace the original (and might actually drive new interest and sales). And obviously, a scanlation fails every one of these tests: it’s a full-on copy that changes only the language, and fully replaces a translation the original publisher might provide. It’s also been pointed out that scanlations are harming the development of a legal digital manga industry in the US. Scanlations would have zero chance of surviving a legal challenge.
So why the hell is Apple in the business of distributing them on iOS?
manga on the App Store and you’ll get dozens of hits. Most of them are apps for downloading and reading scanlations on your iPad or iPhone. For the purpose of this blog, I tried the free versions of:
- Manga Storm
- Manga Rock
- Manga Rock 2
- Manga Browser, Downloader, and Reader
- MangaX3 Lite – didn’t use, crashes when loading title database
- eManga – didn’t use, hangs on sync
- Komik Connect – didn’t use, since it requires a login to do anything
Note: these are not affiliate links. I wouldn’t want a cut of their sales, since I consider them illegal and illicit.
Most of these apps get their contents from three scanlation websites: MangaFox, Mangareader.net, and MangaEden. Some of these sites play at supporting the source of their titles by slapping in pseudo-legal disclaimers and vague admonishments to somehow support artists as seen on this page of The Rose of Versailles:
This image is hosted at mangafox.com. We take no credit for the creation or editing of this image. All rights belong tot he original publisher and mangaka. While we hosted this for free at mangafox.com, please don’t forget to support the mangaka in any way that you can once his/her work becomes available for retail sale in your region!
Some of these sites also adhere to an ethic that they don’t host scanlations of titles that have been licensed in the US. In this screenshot, Manga BDR (which awkwardly makes you browse MangaFox rather than scraping its index) shows an notice that Fullmetal Alchemist is unavailable from MangaFox because it has been licensed in the US:
Does this mean there’s honor among thieves? Hardly. The sites are still violating the original Japanese copyright of the titles they do offer. And they’re not living up to the implicit promise to make obscure titles available to a wider audience — the Rose of Versailles manga cited above has not been completely translated, despite being more than 30 years old. And wherever Manga Rock gets its data from, it has no compunction about offering up titles that have US publishers. Here’s Manga Rock 2 offering Fullmetal Alchemist in its entirety:
Not only is this stuff illicit bootlegs, these apps are popular because they allow access to pirated manga. Every single one of these apps advertises itself on the App Store with screenshots of browsing popular titles that have US publishers: Manga Storm shows Fairy Tail, Manga Rock shows Fairy Tail, Air Gear, and Negima!, and Komik Connect shows Bleach and Naruto. And the users use these apps precisely because of their illegal nature: the one-star reviews on Manga Storm don’t complain about it ripping off artists, but because it lacks US-licensed titles (due to its dependence on MangaFox and friends), and because it’s a paid app.
And speaking of the paid versions…
Apple gets a 30% cut of every sale of the full versions of these apps. That makes Apple a direct beneficiary of copyright piracy.
Everyone who stood up to say Apple does more to support creators than Google and its cavalier attitude towards IP rights, you can sit down now. So long as these apps are available on the App Store, Apple is complicit in piracy.
It’s fair game to criticize Apple for these, when the company has such a stringent review process. When it’s so careful to consider what it will and won’t sell, approval of an app has to be considered an explicit endorsement, particularly considering Apple gets a cut of the sales.
And that’s what makes it all the more galling:
- Apple has started rejecting apps for largely hypothetical privacy concerns, but is OK with pirating manga.
- Apple won’t allow BitTorrent clients, despite significant non-infringing uses, but is OK with pirating manga.
- Apple won’t let third parties sell gay- or lesbian-themed comics, but is OK with pirating manga.
The last of these may be the most galling. Erica Friedman again:
I went on a rant about why is it okay with the those of you who like shiny things that Apple just told DMP to take their BL off the iPad app? WHY?!? If the TV hardware manufacturers told you what TV stations you could receive, you’d be enraged. When your work blocks sites, you find ways around it. So why the hell is it okay will all you Apple fans that Apple censors content? I cannot understand why you are not screaming at all, much less loudly? APPLE CENSORS CONTENT. Especially LGBTQ content. Why are you still giving money to a company like that? People boycott BP and Chik-Fil-A and Target…but are absolute sheep about Apple’s censorship of content. ARGGGGGHHHH.
It’s as if Apple is saying “we won’t let anyone sell you gay manga for your iPad, but we will sell you tools to help you steal the stuff.”
This has to stop.
If nothing else, these apps are in obvious violation of section 22.4 of the iOS App Store Review Guidelines:
22.4 Apps that enable illegal file sharing will be rejected
Apple apparently won’t listen to third-party criticism (people have been calling attention to these bootlegging apps since at least 2010: 1, 2, 3), but there are channels that aggrieved parties can use. Viz and Yen Press have legitimate iOS apps for their manga titles. Since Manga Rock 2 makes bootlegs of those titles available (I saw Viz’s Fullmetal Alchemist and Yen’s High School of the Dead), these companies could use Apple’s dispute policies to at least have Manga Rock 2 taken down.
Beyond this, it’s hard to see what will work. Via Twitter, Erica noted yesterday that most US manga publishers are too small and operating on margins too thin to follow up with DMCA takedowns, and Apple may be technically in the clear on DMCA because they’re not themselves hosting the offending content.
However, since Apple’s making money off the sale of the apps used to pirate this content — in clear and obvious violation of their own policy — another option is that the Japanese publishers might want to sue Apple directly. They would presumably have more legal resources to stick with a lawsuit, and with Apple deaf to criticism, maybe it would take a few subpoenas to call their attention to the fact that making money off piracy is an awfully dirty business for one of the world’s largest and most prestigious companies to be involved in.
For the sake of Apple and the creative community, these apps need to disappear forever.
I’m getting a lot of hits on Sunday’s App Rejections Are A Lousy Way to Communicate Policy Changes, courtesy of a link from Daring Fireball, which is pretty magnanimous considering I kind of slam Gruber halfway through. That blog argued that suddenly rejecting apps for accessing the UDID is a terrible way to handle a policy change, that Apple’s usually far better about controlling the message, and that developers shouldn’t hear about this stuff from Twitter and TechCrunch.
Unfortunately, a lot of the conversation here and elsewhere has been hijacked by a focus on the UDID specifically and privacy issues in general, largely by uninformed readers who are quick to blame everyone who didn’t see this coming.
I’m going to use this blog to explain what the UDID is and the problems around it, from the POV of a developer who’s both chosen to use it and inherited code that uses it.
Take this with the grain of salt that I Am Not A Security Expert (IANASE), or perhaps I Am Not Graham Lee (IANGL).
What Is the UDID?
The Unique Device Identifier is exactly what it says on the tin: a long string of characters that uniquely identifies one iOS device. It only identifies the device, not the person using it (I believe it stays the same after a device wipe, but I don’t feel like wiping my iPad to test that). Any user can find his or her UDID by connecting their device to iTunes and option-clicking the Serial Number.
Since the first public SDK in iPhone OS 2.0, the UDID has been available to developers by calling
-[UIDevice uniqueIdentifier]. The reason you would want to do so is typically to identify one instance of your app running in the wild, or perhaps as a stub to create other unique IDs (for example, a document-creating app that IDs each document as UDID plus the time it was created).
Imagine you want to know how often your users use your app. You could set up a URL to log startups, and then hit it from your app. But if you got 30 hits, you wouldn’t know if that was 30 devices running your app once, or one device running it 30 times. If each call sends the UDID, then you’d be able to tell.
So What’s The Problem
Most of us would agree that this scenario is pretty benign. And in fact, many apps gather a lot more metrics than this — what features are used most, which ones are used together, etc.
Let’s set aside for the moment of whether gathering usage metrics is OK (with or without the user’s permission). So far, this doesn’t seem bad. In fact, we still don’t know who’s running the app — no matter how much information we collect on a given UDID, all we know is that one device exists that is being used in that way.
Let’s say we have 9 apps that collect metrics like this, each phoning home metrics to the developer or some third-party’s server. Now what if a 10th app does the same thing, but it for some reason is able to collect personal information (maybe it’s subscription based, maybe it uses a Facebook login, whatever). That app is clearly able to correlate metrics to a given user. But since it shares the same UDID with the other 9 apps, we can now associate the activities in those apps with a specific individual.
Can you say “unintended consequences”? This is, as I understand it, the problem with the UDID, and why Apple has deprecated its use.
They Should Have Known!
No, the issue is that no one should have thought it was ok to use information that could identify or otherwise compromise another person’s privacy. It should have been obvious that once this got out Apple would need to do something about it. It should have been obvious that you should not have been doing it at all.
Icouldseeitcoming, commenting on App Rejections Are A Lousy Way To Communicate Policy Changes
My point in the above story is that the potential for misuse of the UDID does not come from the string itself. Remember, it conveys no personally identifying information. The problem has arisen over time, in an unexpected way, based on many apps working together (not necessarily with their explicit intent or permission).
Was Apple negligent in offering access to the string in the first place? All indications are that they actually gave privacy some serious thought as they developed the iPhone SDK. Consider the Address Book. No, not the Path debacle (we’ll get to that). Over on Mac OS X, there is a method called
-[ABAddressBook me] (also the C function
ABGetMe()) that returns the user’s address book card. In the C-based iPhone Address Book API, there is no equivalent call to get the user’s record. Clearly, Apple did put some thought into what third-party developers should and shouldn’t have access to, and decided that being able to personally identify the user of the phone was a bad idea. Despite benign uses this prevents, many of us would consider this a good decision.
To that end, let me tell you another story…
Apple’s guidance in the iOS 5 deprecation statement for the
-[UIDevice uniqueIdentifier] call is to generate a
CFUUID and persist that in the
NSUserDefaults. I’d be inclined to use the Keychain so it survives app deletion and reinstalls, but same difference. The upshot is, the one app that just needs to log app launches still has what it needs: a unique identifier of one instance of the app. When each app creates its own CFUUID, the 10 apps in our above example phone home with 10 different unique identifiers, so one can’t be used to compromise the identity of another. So far, so good.
So what is a
CFUUID? It’s Apple’s C API for working with Universally Unique Identifiers (UUIDs). From Apple’s docs:
UUIDs (Universally Unique Identifiers), also known as GUIDs (Globally Unique Identifiers) or IIDs (Interface Identifiers), are 128-bit values guaranteed to be unique. A UUID is made unique over both space and time by combining a value unique to the computer on which it was generated—usually the Ethernet hardware address—and a value representing the number of 100-nanosecond intervals since October 15, 1582 at 00:00:00.
So that’s good, we can’t trace the IDs back to the… hey, wait a minute, what was that? Combining the network hardware address (aka, the MAC address) with the time the UUID was created? Doesn’t that mean that a group of UUIDs created on the same device would all have the same MAC address? So if you can get that MAC address out of the UUID, then even though the 10 apps that phone home 10 different UUIDs, we could get the MAC address that’s common to all of them, identify the user and we’re right back where we started! What the hell? Rabble! Rabble!
Well, as it turns out, Apple’s documentation is wrong. Look back at the Wikipedia entry again and notice that UUIDs have versions. The description above is for version 1, which was abandoned for exactly this reason:
This scheme has been criticized in that it is not sufficiently “opaque”; it reveals both the identity of the computer that generated the UUID and the time at which it did so.
Read in and we find that there is a version digit in the UUID, the character after the second hyphen. Now click to enlarge the screenshot below, from a sample app I showed at CocoaConf Chicago, and which generates hundreds or thousands of UUIDs a second in a (futile) search for duplicates:
Notice that the digit after the hyphen is always a 4, meaning these are version 4 UUIDs, which are just really huge pseudo-random numbers, and therefore don’t reveal anything about who or what created them. Problem solved.
The reason I bring this up is that it was not at all “obvious” that putting the MAC address in the UUID was a bad idea, at least not so bad that it held up ratification of the standard (or, alternately, it wasn’t a problem until UUIDs started being used for purposes where revealing the creator’s identity was an issue). Lots of smart people worked on this stuff; it wasn’t thought to be a problem until later.
Like bugs, security problems are not things people create on purpose, and it’s insulting to insinuate that. They show up later, after reconsideration, after systems evolve, after third-party attacks.
You Want Evil? I’ll Show You Evil!
Yeah, you guys are right. How could anyone have expected that something called a “unique device identifier” might be used to track people.
Icouldseeitcoming, commenting on App Rejections Are A Lousy Way To Communicate Policy Changes
The UDID is neither necessary nor sufficient to track users. An app could use a hand-rolled UUID to track a device, as is Apple’s recommendation, and is in a position to log every tap and keystroke and phone it home to the analytics server. Heck, I had an AV Foundation demo a few years ago that encoded screen-grabs to a QuickTime movie — it wouldn’t take much more to provide a live video stream of a user’s interaction with my app back to an IP address of my choice, all without the user’s knowledge or assent.
That’s assuming I’m being evil on purpose of course. What removing the UDID hopes to improve is cases where apps inadvertently provide a way to correlate activity in different apps, which in turn could be linked to a real person if any of the apps capture personally-identifying information. Even now, there are other ways this could be done: apps could share hand-rolled UUIDs via URLs, document exchange, or the Keychain (if the apps were all signed with the same credentials and share a bundle id stub), though this requires a far greater degree of deliberate cooperation than the inadvertent UDID case.
There are other APIs with privacy implications too, such as free access to the Address Book (as epitomized by the Path case a few weeks back, and Facebook years earlier). An app that incorporates a
So to make a big deal out of the UDID as this obvious privacy problem seems badly ignorant. Its problematic aspects come from unintended consequences, not the nature of what it is.
And since any app can itself collect any information on your interactions with it, UDID or not, and phone it home with a network connection, the only perfect way to avoid being tracked at all is to go to the Settings application, turn on Airplane Mode, and never turn it off.
Following Twitter reports from earlier in the week, MacRumors and TechCrunch now report that Apple is rejecting apps that use the unique device identifier (UDID), by means of the
-[UIDevice uniqueIdentifier] call.
This in itself is not surprising: iOS 5 deprecates the method, adding “Special Considerations” that advise developers to create a one-off CFUUID to identify a single install of the app on a single device.
So far, so good. I even had a section of my talk at last week’s CocoaConf Chicago that talked about CFUUIDs and the need to migrate any UDID dependencies to them now.
The problem is the timing. Apple’s established pattern has been to deprecate a function or method in one major version and, at the speediest, remove the call in the next major version. Many developers, myself included, expected that we had until iOS 6 to get off of the UDID.
But instead, without warning, the app review process is being used as an immediate death-penalty for the
This is a problem because we’ve all had about six months to get off of UDID, and while that’s surely enough to get a simple app migrated — indeed, I have cases where switching it out is a 5-line fix — it is not necessarily the case that everyone can be expected to have already done this.
The real problem isn’t with developers; it’s with whom we develop apps for. Our clients don’t know or care what a UDID is, nor are they aware of a single line in Apple’s documentation saying “stop using this”. Sure, it’s our job to be on top of it. But let’s imagine apps with long development cycles — big apps, or academic apps that rev for the new school year in the Fall and are largely dormant the rest of the year. It’s entirely plausible and reasonable that developers of these apps have “get off UDID” as a high-priority item in their bug trackers, but are waiting for budget and approval to start working. And what if it’s not a simple process? What if an app has some deep dependency on access to the UDID, both in the app and on a server somewhere, meaning that two different teams are going to need to deal with losing access to
uniqueIdentifier, and will need to come up with a plan to migrate user records over to a new id scheme?
Well, they just lost their chance.
Captain Hindsight is in full effect in the MacRumors forums, loudly asserting that developers should have known this was coming, have had plenty of time, etc. I get that it’s the natural defensiveness about Apple, but it gets worse… because this isn’t the only case of Apple using app rejections to carry out policy changes.
Thanks perhaps to my many posts about in-app purchase, I recently heard from a group of developers who’d gotten a galling rejection. They have an app with a subscription model, and used the new “auto-renewing subscription” product. This product is far superior to the original subscriptions that I have repeatedly described as broken, as they cannot restore between a user’s devices, and do not carry state to indicate what was subscribed to and when. Auto-renewing subscriptions fix these problems, and the In-App Purchase and iTunes Connect guides had (seemingly until a couple weeks ago), clearly disparaged use of the old subscriptions in favor of the new auto-renewing subscriptions.
So imagine the surprise of my colleagues when their app was rejected for using auto-renewing subscriptions. The reason given was that they were using it for a different business plan like a data-plan model, and auto-renewing subscriptions are, according to the reviewer, reserved only for content subscriptions like Newsstand magazines. I have never seen anything to this effect in any Apple I-AP documentation. Nevertheless, the developers had to switch to the shitty, broken, old subscriptions.
In both of these cases, we see Apple breaking with their own documentation or with long-established practice with no warning, and instead using app rejections as a tool to communicate and carry out new policies. This is wretched for developers, who get caught scrambling to fix problems they didn’t know they had (or didn’t expect just yet).
It’s also terrible for Apple, because the aggrieved developers initially control the message as they flock to blogs and Twitter, leaving it to loyalist commenters and bloggers like Gruber and the Macalope to mount a rear-guard gainsaying defense. To see Apple — of all companies! — not controlling the message is astounding.
All it takes is clarity. If they’re going to make such a major change, they’ve already got our attention via e-mails, the developer portal, and many other channels. They could and should clearly state the what, why, when, and how of policy changes. “We’re getting rid of UDIDs, because they constitute a privacy risk. We’ll reject any app that calls
-[UIDevice uniqueIdentifier] as of March 23, 2012.” Not that hard. They’ve done it before: a few years back, Apple required streaming media apps to use HTTP Live Streaming if they streamed more than 10MB of data — this was communicated via announcements on the developer portal a month or so before its implementation, and nobody got caught by surprise, nobody complained.
Apple has developed a reputation for capriciousness in its app review process, and a cavalier attitude towards its developer partners. It’s not undeserved. As Erica Sadun once cleverly put it, “Apple is our abusive boyfriend.”