Category Archives: Mac OS X

Log Littering

Apple has dramatically revamped its standard logging mechanism. Unified Logging, available in macOS 10.12 and iOS 10, replaces various file-based logging approaches with a centralized, database-backed repository for log information of all levels of interest.

The big win, both for developers and for users, is performance. Where the simple act of logging some debugging information used to have enough of a cost to dissuade rampant use, the new system makes it so cheap that developers inside Apple and out are being encouraged to “log like the dickens” in the name of generating more evidence for posthumous debugging. The new system also offer impressive data mining abilities, making it far easier to filter for what you’re looking for and to track logging information across specific activities, even if the activity spans multiple processes.

The two big losses, in my opinion, are that the sheer size, number, and variety of logging messages makes it impractical for users to skim the console for “real problems,” and that the resulting logging archives are so large that it’s impractical to casually include them with bug reports to Apple or 3rd party developers.

The WWDC session on the topic goes into detail on these points, but perhaps most useful for framing my complaints here are Apple’s stated goals for the revamp:

  • One common, efficient logging mechanism for both user and kernel mode
  • Maximize information collected while minimizing observer effect [the phenomenon that enabling or adding logging changes the behavior of the system you are trying to understand]
  • We want as much logging on all the time as possible
  • Design privacy into the system

Two of the stated goals, “maximizing information collection,” and “as much logging on all the time as possible,” exacerbate the problems of there being so much log data that it’s both difficult to skim, and cumbersome to share.

Imagine a society in which all packaging has been transformed into fast-composting, biodegradable materials. Your soda bottles, snack wrappers, cigarette packages, etc., all biodegrade to dirt within 48 hours of use. What a boon for the world: the major, global problem of trash accumulating in our towns and environment would be gone. Poof!

Or would it? When “littering has no cost,” I suspect that we’d face a new problem: far more people would litter. Why bother finding a place to throw out that bottle, when nature will take care of it in within 48 hours? Multiply this attitude out over even a small portion of the world’s billions of people, and we’d be guaranteed to be buried in trash. All trash that will be gone in 48 hours, mind you, but constantly replenished with a fresh supply.

I think this metaphor points to a similar problem with unified logging: the sudden onslaught of “low cost logging” has left our developer society unprepared in some specific ways to deal with the consequences of the new reality.

So, how can we, and Apple fix this? We need specific solutions that make skimming for problems, and sharing pertinent log data easier, yet don’t impact the very positive goals outlined by Apple above. Here is my advice:

  1. Don’t litter the logs. Just because you can log it, and just because doing so is cheap, doesn’t mean there isn’t a cost. Some of the recurring log messages littering my Console arrive at a rate of thousands per minute. There may be good arguments that certain subsystems, in certain states, should log this extensively if the resulting data will justify easier diagnosis of problems, but much of the junk I see are redundant reiterations of relatively useless information:
Not switching as we're not in ~/Library/Keychains...
...
CSSM Exception:...
...
Etc. Etc. Etc.

Some of these seem to be well and truly noise, but some of them, even if they seem highly redundant and noisy to me, could in fact represent useful logging if you happened to, for example, be tracking down a subtle bug in Apple’s security libraries. That’s fine, I’ll accept that some amount of log diarrhea in the name of better software, but only if the following accommodation is made…

  • Annotate high-frequency logs for easier filtering. The logging system offers a variety of tools for annotating log messages, but even internal Apple groups do not seem to use these extensively or appropriately. The system supports the notion of three levels of log message: default, info, and debug. Only the “default” level messages are displayed by default in the Console app, yet all of the above-described garbage is displayed in that default mode. The Console will become significantly more useful when the worst offenders have marked their messages for severity, subsystem, and category so that they can be effectively omitted from the default view, and so that they can be mined later for use by folks who can actually make sense of them.

  • Generate context-specific sysdiagnoses. One of the worst practical outcomes of unified logging is that capturing a “sysdiagnose” on my Mac now generates a compressed archive that is still larger than 400MB. These archives are typically requested by most groups at Apple in response to bug reports, regardless of how severe the bug is or how readily reproducable it appears to be. In the world of Apple bug reports, sysdiagnoses are treated like lightweight bits of information that should be appended to every issue, except they’re now anything but lightweight.

    All the annotation work advised above should really pay off when Apple provides a streamlined mechanism for capturing sysdiagnose information pertinent to a specific subsystem or product. The company already offers product-specific advice for capturing log information, sometimes requiring the installation of custom profiles and other shenanigans. The lightweight unified logging system empowers Apple to both require that internal groups properly annotate their log messages and to facilitate smarter gathering of that data.

    Currently if you want to capture a sysdiagnose at any time a Mac, you simply press Ctrl-Opt-Cmd-Shift-Period. On an iOS device it’s done by holding both volume keys and the power key, but you have to enable logging first with a custom profile. This shortcut will generate one of those mondo 400MB style sysdiagnose archives, which you can upload to attach to your bug reports in your copious spare time.

    I envision a prompt that appears after invoking the existing shortcut, or else a new shortcut for “interactive sysdiagnose” where you could specify the category of bug you are reporting. The prompt would list categories correlating to groups at Apple who had done the work of providing streamlined log filtering data that will effectively strip out all the useless (to that group) noise from your log data.

    In fact, sysdiagnose already takes a “process” parameter when invoked from the command line, and my understanding is that this enables it to capture data that is pertinent to the target process. The unified logging system seems to provide the infrastructure for even smarter capturing along these lines.

    I realize that in this time of flux, where much log data is not properly annotated, there is still an incentive for many groups to capture as much as possible and sort it out on Apple’s end. These groups should know, however, that the larger the sysdiagnose archive you ask users to upload, the lower the chances they will bother actually following through on filing the bug or on providing the pertinent information you have requested.

  • Annotate specifically for user-concerning issues. Power users on the Mac have, for years, counted on being able to skim the console log for information that might explain a slow Mac, persistent crashes, or other inexplicable behavior. Given the huge increase in number of log message and the lackluster annotation of these messages by Apple, the Console app is effectively useless to users for this purpose.

    On top of getting the annotation right in general, I think a new special level of log message should be created, indicating particular concern to end-users. These would identify log messages that developers specifically think users should take notice of and follow up about. I mentioned there are three basic levels of log message: default, info, and debug. On top of that, there are two special levels called “fault” and “error”, which can be filtered on in the Console, and which receive special treatment from the logging system.

    These special levels seem close to what users might find interest in, but they’re not quite right either.

    A “user interest” level of logging message would facilitate the kind of high level skimming that seems impossible in Sierra today. These log messages would convey information that a user might gain some insight from. They would stand in stark contrast to the noise of messages like these, from the filecoordinationd process:

    Claim D32EDEBE-C702-4638-800C-E4BAB9B767F3 granted in server

    Nobody knows what a claim is, or cares whether it was “granted in server” … unless they are actively debugging filecoordinationd. On the other hand, a message indicating the failure to grant a claim “because the disk is full,” could be very useful to users indeed.

    I realize the line could be difficult to draw in some circumstances, but a huge number of messages currently being logged are obviously not of user interest. Perhaps then, the opposite approach could be taken, and log messages could be annotated as being specifically useful for posthumous, internal debugging by developers.

I tell you, things are bleak. We’re swimming in a morass of useless console noise, and there is little we can do about it. There is room for optimism however, as the logging infrastructure is good, and seems to support many thoughtful measures that could attenuate the problems and make the system more useful and less cumbersome to developers, end-users, and well-meaning bug reporters.

The console is littered with trash. Sure, it will biodegrade in 48 hours, but that doesn’t mean we shouldn’t be concerned with cleaning it up. A combination of more thoughtful logging, proper annotation, and appropriate filtering will get us out of this mess.

Bad Preference Gatekeeper

With the release of OS X 10.11.4, developers of standalone preference panes face a new challenge with respect to users installing their software.

Apparently, the validation process that Apple applies to downloaded software, Gatekeeper, fails to validate OS X preference panes, even if they are signed with a legitimate Developer ID code signature.

The upshot of this is when users download a bona fide 3rd party preference pane such as Noodlesoft’s excellent Hazel, instead of having the software install as expected, a scary warning is displayed indicating the purported untrustworthiness of the software.

According to Paul Kim of Noodlesoft, the problem affects every preference pane he’s tested, including a freshly built, completely plain preference pane built with Apple’s latest tools. I put this to the test in Xcode 7.3, running on 10.11.4, by creating a new Preference Pane project from Apple’s template, setting it to sign with my Developer ID, and creating a release build of the project.

Running Apple’s “spctl” tool on a binary is a reasonably approximate way of determining whether Gatekeeper would reject the binary after downloading it from the web. Here’s the result for all the affected preference panes:

% spctl -av ./TestPanel.prefPane
./TestPanel.prefPane: rejected
source=obsolete resource envelope

Ah, that pesky “obsolete resource enveloped” message. Those of us who survived the transition from Version 1 to Version 2 code signing remember it well. But it’s not an accurate assessment in this case:

 % codesign -dv ./TestPanel.prefPane              
Executable=/Volumes/Data/daniel/Desktop/TestPanel 2016-03-31 14-03-51/Products/
[...]
Sealed Resources version=2 rules=12 files=2
Internal requirements count=1 size=220

The “version=2” indicates we are using an appropriate version for the signed resources. It would be hard, perhaps impossible, to do otherwise on a modern system with a modern Xcode toolchain.

The “spctl” tools supports a command line option to ask for more and more verbose results by adding “v”s to the command line. Unfortunately “spctl -avvvvvvv” doesn’t yield anything more informative than the seemingly inaccurate “obsolete resource envelope.”

I wondered if there was some magic flag that preference panes must now exhibit, or some new requirement that internals be signed in a different way than before. Surely, if anybody could get this right, it would be Apple! Their “Network Link Conditioner” is the only downloadable preference pane I could think of, and what do you know, it was updated as part of the Hardware I/O Tools for Xcode 7.3 download package, released on March 20. I downloaded a fresh copy to be sure I had the best that Apple could offer, located the preference pane, and double-clicked it.

Untrusted

You know it’s bad when even Apple’s own downloads are portrayed as untrustworthy.

This is a minor annoyance for folks trying to install an obscure development tool, but it’s a major issue for developers like Noodlesoft whose entire livelihood is built on the distribution of software packaged as a preference pane. The scary wording in the dialog casts doubt on the reputation of the developer, and for the more savvy, on the reputation of Apple’s ability to properly assess the trustworthiness of software that we download.

Let’s hope Apple can address this problem soon. Although it doesn’t pose a security risk, it seems appropriate that they could include this in a security update. After all, it has everything to do with preserving trust between users, developers, and Apple.

Update: According to Paul Kim, it’s not just preference panes that are affected, but any standalone non-app code bundle. So, for example, color palettes, screensaver modules, and, if anybody ever used them anymore, Dashboard widgets, are all affected. Pretty, pretty, pretty, pretty bad.

(Radar #25468728)

Selectively Smart Typography

OS X offers a system-wide feature to “Use smart quotes and dashes” when typing text. The option can be enabled or disabled in the System Preferences “Keyboard” pane, inside the “Text” tab:

Keyboard

This essentially ensures that when you rattle off a bit of prose, any quotation marks will be “smartened” so that they curl inwards towards the quoted phrase, and sequences of double-hyphens will be converted to typographic dashes.

It’s a good thing.

However, the feature can be vexing when typing in contexts where typographic beauty is clearly not the priority. For example the behavior would not make sense in a code editor like Xcode or BBEdit. These apps are more often used to write text that will please a programming language compiler or interpreter, where a string for example must be enclosed by "straight quotes", and “smart quotes” would only lead to a syntax error.

Apple actually makes it possible for developers to control the behavior of this smart typography on a very fine-grained basis. For example, my own app MarsEdit features an HTML text editor in which it makes sense to support smart quotes and dashes, but not within markup tags. Because the app is aware of when the user is typing within a tag or not, it simply disables the functionality as appropriate for the context.

Depending on the apps you use, and what you use them for, you might find yourself vexed by the unwanted conversion of quotes and dashes. You could turn the feature off at the System Preferences level, but what if you want to keep it in all the other apps where it works just as you want it to?

Thanks to the flexibility of Apple’s “user defaults” system for registering preferences on a system-wide and app-specific basis, you can impose override preferences for these options on the specific apps of your choosing. I was inspired to write this article by Jesse Atkinson, who complained on Twitter that the behavior was meddling with his code snippets in Slack:

So, here’s a general recipe for overriding the settings in any app:

  1. Quit the app.
  2. Determine the “bundle identifier” for the app. Slack’s, for example, is “com.tinyspeck.slackmacgap”.
  3. Open the OS X Terminal app.
  4. Type the following commands to impose overriding defaults for the pertinent features:
    defaults write com.tinyspeck.slackmacgap NSAutomaticDashSubstitutionEnabled -bool NO
defaults write com.tinyspeck.slackmacgap NSAutomaticQuoteSubstitutionEnabled -bool NO
  5. Reopen the app.

You should now find that this specific app behaves as though the system-wide setting for these features were turned off.

If you want to undo your handiwork, just delete the overrides you put into place:

  1. Quit the app.
  2. Open the OS X Terminal app.
  3. Type the following commands to delete the overriding defaults:
    defaults delete com.tinyspeck.slackmacgap NSAutomaticDashSubstitutionEnabled
defaults delete com.tinyspeck.slackmacgap NSAutomaticQuoteSubstitutionEnabled
  4. Reopen the app.

If you’re a Slack user who has also been annoyed by the smart quotes and dashes functionality, the above may prove helpful to you. If you’re not, maybe a similar situation will arise where you can use the power of OS X’s default registration system to fine-tune the behavior of your favorite apps.

Update: My old friend Chris DeSalvo, who also happens to work at Slack on the Mac Slack app points out that there is a much simpler solution for Slack in particular:

I just confirmed that this seems to work well with Slack! So was this whole article for naught? In my experience, toggling the Substitutions flags as Chris describes only changes the setting temporarily. This is because those menu items within an app are usually configured to communicate directly with the text field you are focused on. A developer has to go out of her way to intercept and persist such a toggle, to ensure that it “sticks” the next time the app launches.

So! If you have this problem with Slack, just toggle the setting in the Edit -> Substitutions menu. With most other apps? Perhaps the technique outlined above will help.

Lazy Password Storage

When you run an app on your Mac that connects to a secure web service, how confident are you that the password will be treated with care, and protected from prying eyes?

As a rule, Mac developers are pretty responsible about storing passwords and other private data in the OS X system keychain but, of course, there are exceptions.

I found a handy trick for uncovering passwords stored insecurely by applications directly to their preferences storage. The trick takes advantage of a cool functionality of the OS X “defaults” command line tool, which you can run from the “Terminal” app:

'defaults' [-currentHost | -host ] followed by one of the following:
  [...]
  find <word>     lists all entries containing word

How convenient: a simple command line tool to search the entirety of all the preferences stored by all of your apps. So, a good first step would be to simply search for “password”:

defaults find password

On my Mac, this yields an overwhelming number of matches that includes a lot of false positives such as, for example, the preferences pertaining to 1Password, preferences pertaining to apps’ password dialog windows, and other innocuous uses of the term.

It occurred to me that most developers storing passwords insecurely in preferences would probably store the value either under the key “password,” or some variation such as “twitterPassword”. So I tweaked the command line to try to filter out these results. The “defaults find” command doesn’t take any options, but I can winnow the results using grep:

defaults find password | grep -i -E "password\"? ="

This grep invocation searches for case insensitive matches for “password”, optionally followed by a quotation mark, then a space and an equal sign. In other words, examples where a key that ends in “password” is being assigned a value.

This actually did reveal some problematic password storage on my Mac, but the grep is so good at filtering out the results, I can’t see which app to blame. I need to match ALL the lines that pinpoint the app, and all the lines that looks like they store a value into a password. Add an | (or) case to the grep expression to match for the tell-tale signs of the lines that summarize findings per-app:

defaults find password | grep -i -E "password\"? =|keys in domain"

Here I find a neat summary of potentially problematic password storages. Some of them remain false positives, but the list is now small enough to easily interpret. Any example where the app is something I plan to use again, I’ll be in touch with the developer to encourage them to improve the password storage security. Any example where the app is nothing I’ll ever run again?

defaults delete com.example.lazyapp

And the insecurely stored password is obliterated from my preferences.

Obviously this trick won’t match all the careless password storage that apps on your Mac may be committing, but I suspect it will root out a good number of them. Experiment with the grep commands to filter out based on different, less restrictive matches. You might also have some luck searching for examples of apps that store other sensitive information such as credit card numbers, secret questions and answers, etc.

Familiar Spell Checking

Even if you’re a great speller, automatic spell checking provides a valuable safety net, often preventing us from sharing with the world our own individual ignorances of the languages we communicate with.

For years, spell checkers of all kinds have relied upon word databases to determine that a word has been misspelled. I’m sure it’s at least a little bit more complicated than this, but in a nutshell: if the word you’ve just typed is not in this enormous list of words, then it’s probably a misspelling, and is marked as such.

Thus as you type your master works, when you either flub your typing or can’t recall the correct spelling, most modern editors will flag the word in question, for example by showing a red squiggle below it.

One challenge, though, is that for any individual, a number of words that are spelled correctly may nonetheless be absent from spell checker’s enormous database. One great example of this is the variety of proper names we deal in that may not happen to be included. For example, if I were a Boston Red Sox fan I might type Dustin Pedroia’s name often, and at least OS X’s built-in spell checker would mark it as incorrect. The workaround for these situations is to control-click the word and, from the popup menu, select “Learn Spelling” to avoid future flagging of the item.

The problem can be especially annoying when the OS repeatedly flags the names of people in your own family. For example, my own last name, “Jalkut,” is incredibly uncommon and would be flagged as a misspelling by most spell checkers in the world. However, on my Mac it is not, even though I’ve never asked the system to “learn” the spelling. In fact, none of the names of my family, friends, or business associates are marked as misspellings. How does this work?

Apple’s engineers realized that they could spare you the hassle of “learning” these spellings, since they already have access to a massive database of proper names that matter to you. Your Contacts database. If you’re on a Mac right now, open up the Terminal application and paste in the following line:

/System/Library/Services/AppleSpell.service/Contents/MacOS/findNames

The result that should come flying down the terminal window are the proper names of everybody and everything in your Contacts database. Specifically, Apple consults every entry in your database for the following attributes:

  • First Name
  • Middle Name
  • Last Name
  • Nickname
  • Maiden Name
  • Organization
  • Address
  • City

Each of these words, if present, is used to augment the already-massive list of correctly spelled words. So if you happen to know somebody named Frenk Xssl, who lives in Cwmystwyth and works for Infinitea, you can write all about them and their work, and never be bothered once by the tell-tale red squiggle of a word misspelled.

SiriScript

Today I faced a long list of alarms on my iPhone, and decided that I wanted to clean them out. The typical iOS “Edit” interface puts a red “delete” button next to each item, and upon tapping it you must then confirm it by tapping the explicit word “delete” at the other end of the item. Suffice to say: for a list of any significant size, this is very tedious.

On a whim, I decided to give Siri a shot at simplifying the process. I long-pressed the home button, and uttered: “delete all my alarms.”

NewImage

Well, isn’t that nice?

I realize that when I am faced with a problem on iOS or watchOS, for which I wish there were an automated, “power user” mechanism to simplify it, I reach for Siri and hope for the best.

In this respect Siri fills the gap that is left by the omission of an automating service such as AppleScript. On a Mac, when I’m faced with a problem like this, I look to Script Editor, and hope that the app is scriptable enough to get the job done. For example, if alarms were a service provided by my Mac (and why aren’t they!?), then I would expect a script like this to complete the task at hand:

tell application "Clock" to delete all alarms

Having AppleScript at my disposal is great, but I am also frustrated that Siri doesn’t live on my Mac. I should be able to invoke the dictation UI (fn-fn keystroke, by default) and ask Siri to “add to my reminders,” or to perform any of the other common tasks I can perform with my phone or watch.

Siri is more limiting than AppleScript, because it can only carry out tasks that Apple’s engineers have predicted that I will want to perform. But it’s also much easier than opening up Script Editor, scrutinizing a scripting dictionary, spending 10 years learning AppleScript, and then writing and running a script.

Ideally, I’d like to have the best of both worlds: the ease of asking Siri to perform complex procedures and the option of extending the catalog of procedures that it knows how to perform. And I’d like this perfect combination of ease and extensibility on all of Apple’s platforms.

Since the debut of the iPhone, people have speculated about whether we would ever see an official solution for automation, along the lines of AppleScript or Automator. I think we’ve actually been seeing it since Apple acquired and integrated Siri. I hope that one day this will culminate into the consistent, extensible solution for automation that I have imagined here.

Six In One

Craig Hockenberry’s “Half-Assed” calls out the disparity between Apple’s Mac and iOS App Stores with respect to app analytics, limiting customer reviews from beta OS releases, and support for beta testing with TestFlight:

Mac developers have never had access to TestFlight, either internally or externally. It’s “coming soon”, and until that day comes, there’s no way to test apps that use the iCloud servers. Which sucks for both the developer and the customer.

For years, I’ve harbored my own resentments about the way Apple seems to treat the Mac App Store as a second-class citizen. One point that has nagged at me since the Mac App Store launched five years ago is the lack of effort Apple makes in promoting these products through social networks. For years, the @AppStore account has been extremely active promoting iOS apps to its huge (now 3.8 Million people!) audience. It’s obviously a priority for Apple to promote the iOS App Store, and their Twitter account does a great job of it.

But there is no Twitter account for the Mac App Store, and the @AppStore account has nearly never mentioned Mac software. (Go ahead, do an advanced Twitter search for “from:AppStore mac“). Around four years ago at WWDC, I asked a group of App Store affiliated engineers what they made of this. They all looked at me blankly, paused, looked at each other blankly, paused, and then shrugged as if to say “Huh, I wonder why we don’t promote the Mac App Store?” I don’t know either! But they were all in a much better position than I to find out or push for a change. Evidently, that hasn’t happened.

The neglect makes sense on one hand: iOS and now watchOS are the new hotness, the platforms that represent the fastest and most culturally significant growth for Apple. They sell the most devices, generate the most revenue, and create the biggest headlines, worldwide. It still makes sense for Apple to prioritize the iOS App Store, but it’s a shame they haven’t done more over the past five years to integrate the systems that power the stores so that benefits to developers for one platform’s store would automatically benefit the developers for the other.

On the other hand, let’s acknowledge that the disparity is not all roses for iOS and weeds for the Mac. iOS developers continue to face obstacles for which we have long enjoyed simple solutions for on the Mac. As Craig points out, we can’t ship a useful beta testing app that fully exercises all of Apple’s locked down services such as iCloud, but what can we do? We can ship arbitrary binaries to an unlimited number of people, who can install and run those apps on whatever devices they choose. How do you like them, ahem, Apples?

And what about the absurd steps iOS developers must take in order to stay up to date with beta releases of the OS? Either they commit themselves to running guaranteed buggy and possibly unusable versions of the OS on their machine “machines,” or else they spend extra money on devices whose only purpose will be to act as testbeds for the risky new OS. Want to maintain backwards compatibility? Better keep multiple iPhones, iPod touches, and iPads, of varying generations handy, but be careful not to update them to a newer OS, as there’s no going back.

On the Mac, we enjoy the ability to erase and reinstall whatever version of the OS we choose, whenever we choose. Furthermore, we can maintain parallel installations of the same or different versions of the OS, on easily switchable partitions of the same disk. Even in the context of a single OS install, we enjoy the ability to test varying user scenarios by configuring test accounts and “Fast-User Switching” between them from the comfort of our Mac’s menu bar. Heck, we can even virtualize whole Macs thanks to VMware, Parallels and VirtualBox. When it comes to testing various configurations affordably and expeditiously, it’s the iOS developers who should be crying to Apple that their priorities are skewed.

I guess it’s lucky the Mac was around for so many years before the dawn of iOS, it’s had time to accumulate many developer-empowering features. Although Apple’s priorities with respect to development resources and marketing seem to be focused on iOS today, we enjoy many privileges on the Mac that I doubt iOS developers will ever see. And to top this off with a bit of true optimism for the future: the weird thing is, Apple keeps improving OS X. I’m sometimes surprised by the amount of attention Apple continues to give to the OS given its apparent relative lack of importance. But I guess Apple is well and truly stocked with a bunch of Mac softies, after all.

There’s a ton of totally vexing behavior that seems to be ill-spirited towards Mac developers, but also a ton that seems to hold iOS developers in low regard. I think this speaks to the likely truth that Apple is, more than anything, under-staffed and not well situated to deploy solutions to both platforms in tandem. It truly is a case of six in one, a half-dozen in the other. But wouldn’t it be great if iOS and Mac developers could each enjoy the benefits of all twelve? I’ll hold out hope that one day Apple will unlock the secret of organizing their efforts so that developers on all their platforms can benefit more or less equally from the technologies the company provides.

Mobile Is A Fad

Christopher Mims of the Wall Street Journal published a story this week with the sensational headline “Why Apple Should Kill Off the Mac.” In it, Mims argues that Apple doesn’t have the resources to focus on the Mac, which is clearly less important to the company than its growing lineup of mobile devices.

Long-time, Mac-savvy journalist Glenn Fleishman couldn’t resist rebutting the argument for Macworld. His take boils down to the Mac remaining too vital a part of Apple’s spiritual core, and it being important to Apple to control the devices used for the very development work that makes the other devices so valuable.

John Gruber unleashed his coveted “Jackass of the Week” award for Sims, dismissing the argument as nonsensical, and proclaiming “the end of the Mac is not in sight.”

The assumption that mobile devices will rise to the occasion of obsoleting desktop computers has run rampant for years. I responded to the sentiment nearly five years ago in a Macworld article: “Why I’m Sticking with the Mac.” The Mac is a general-purpose computer, which makes it a little boring in some respects, but its role as the hub for many customers’ computing needs remains valuable. “Apple’s strength on the desktop permits it to take risks with other products,” I wrote in 2010, and I believe that remains true today.

One sensational headline deserves another, so let’s get to mine: “Mobile is a Fad.” Why have people been so convinced, for years, that mobile is the future, while desktop computers are on the way out? What if mobile devices only fill the variety of temporary needs that arise from how we live today?

Product genres proliferate because they meet a need. In the 1970s when oil supplies were scarce, tiny economy cars began to dominate the market. It probably seemed reasonable at the time to assume that gas-guzzling trucks and off-road vehicles were on the way out, yet as soon as the world became flush with cheap oil again, car sizes ballooned and fuel economy plummeted. For many the demand for utility and power, even in the absence of justifiable need, outshines the specialized economy of smaller cars.

Why are people so excited about mobile? Because they go places. What if people stopped going places? Mobile devices are a godsend for people who are expected to wake up every morning, commute to work, run a variety of errands, and possibly stop for a drink with friends on the way home, expecting the battery life on their pocket and wrist-bound devices to stay at least above 20%.

In the future, your errands will all be handled by a low-paid army of Postmates and Taskrabbits. Your friends, geographically distributed around the world, will be selected from among the group of loyal comrades that chooses to live in your virtual reality: Facebook or Google. And you won’t ever be found rushing out the door at dawn, unless you’ve just woken from a nightmare flashback to the days before everybody worked from home. And now that we’re all at home most of the time, we might as well use powerful, multi-purpose computers with full-sized tactile keyboards. We deserve it.

Hopefully that’s all far-fetched, but I indulge in the dystopian fantasy to draw focus on my contention that neither the proliferation of mobile devices nor the extinction of desktop computers is a foregone conclusion. Each of the computing devices Apple in particular sells today, from the tank-like Mac Pro to the butterfly-light Apple Watch, has its own unique set of advantages and compromises. Most likely, the vast majority of us will find uses for several or all of these classes of products, as we engage in the aspects of our lives that favor either mobility or stability.

Years from now, I suppose it’s possible desktop computers will become obsolete, supplanted by mobile devices. But given the unpredictability of progress, I suspect we’ll look back at both classes of computing devices and chuckle about how silly it all seemed, before the advent of ______. On a long enough time scale, everything is a fad.

OS X El Capitan

Thanks to a tweet from Michael Steeber, I learned that I have the dubious honor of having been the first person to tweet the literal phrase “OS X El Capitan”, which happens to be the name of of Apple’s forthcoming OS X 10.11 operating system.

I did some research of my own and believe this claim is true. I don’t think this is some great victory or proof of my insightfulness, but I do admit to thinking it’s at least moderately cool.

Out of curiosity, I used Twitter’s now-awesome advanced search to zero in on the day when I first tweeted the phrase, June 19, 2013. This must have been in the aftermath of WWDC 2013, shortly after Apple had debuted its new OS naming strategy, after places that are distinctly Californian. People were joking about future code names like OS X Sacramento, OS X Carmel, or gasp!, OS X San Jose. I thought these were all poor choices because they would not be evocative of the spirit of California in the same visceral way that Mavericks was:

I quickly followed with a quip about using OS X El Capitan:

Though to be honest I don’t remember exactly what grammatical pedantry I was alluding to. What’s (not really) important is that I was the first to use the phrase, two years before Apple announced the name of the OS.

And while I’m patting myself on the back, let me give a shout out to Mark Hachman, who while similarly throwing out possible names for future OS releases, managed to make the first ever reference to “OS X Yosemite”:

Those who can, do. Those who can’t, teach. Those who can’t teach, tweet early and often.

Push Notification Traps

Recently Marco Arment bemoaned Apple’s use of push notifications for promotional purposes. Apple sent a notification promoting their project (RED) products for sale in the App Store, which Marco judged as user-hostile and in poor taste, even if it can be argued it was “for a good cause.” I tend to agree with Marco on this point.

In the latest episode of the Accidental Tech Podcast, Marco, along with co-hosts John Siracusa and Casey Liss, talked more about the problem of notification spam in general and the difficulty of enforcing it at app review time. They seemed to be in agreement that the only realistic tool at Apple’s disposal is to devise a crowd-sourced flagging system for inappropriate notifications, and to use that collective information to pinpoint the worst offenders, and then to use that information to impose consequences upon them.

They went on to lament that Apple is not very good at these kinds of crowd-sourcing solutions, and that in all probability the vast majority of iOS users are not concerned or aware that they should be concerned about notification spam. The lack of consumer awareness about the nature of the problem could itself be a limiting factor in any crowd-sourced solution.

But I propose that Apple does have tools at its disposal that could help flag the worst offenders immediately, without the cooperation of the public, and without violating any user’s privacy.

All remote push notifications are delivered from an app’s developer to an end-user’s device via the Apple Push Notification service. This is good, because it puts Apple in a position to intercept and e.g. immediately shut down a bad actor from delivering notifications to any of its intended recipients. However, the content of all these notifications passing through Apple’s service is encrypted. This is good, even required, because it protects developer and company data from being eavesdropped. But it’s bad from an enforcement sense because it thwarts possible solutions such as using a Bayesian filter on content to flag spam, similarly to the way an app like SpamSieve works on the Mac.

So Apple has complete control over the distribution mechanism, but zero ability (apart from metadata including the originating company and the target device) to examine the content passing through. Game over? I don’t think so.

Apple can still use its unique role as the center of all things iOS to devise a system through which they would themselves be virtually subscribed to all unremarkable notifications from a particular app’s developer. Think about the worst notification spam you’ve seen. In my experience it’s not super-personalized. In fact, it’s liable to be an inducement to keep using the app, to advance in a game, to become more engaged, etc. I think Apple would collect a ton of useful information about spammy developers if they simply arranged that every app on the App Store that is capable of sending push notifications included, among its list of registered devices, a “pseudo-device” in Cupertino whose sole purpose was to receive notifications, scan them for spammy keywords, apply Bayesian filters, and flag questionable developers.

Because Apple controls the namespace for device IDs, has access to the executables for all the apps in the store, and is technically equipped to run these apps in contrived environments, they could coax applications to perceive themselves as having been installed and run on a device with ID of Apple’s choosing. In fact, it’s probably simplest if this very thing happens while App Store reviewers are evaluating apps. It’s true that they won’t see the spammy notifications during review, but the mechanics of triggering an app’s registration for future notifications would ensure delivery to a “trap device,” actually a giant database against which arbitrary research could be conducted.

This would not be a violation of anybody’s privacy, because only the artificial App Store review team’s data (if any) would be involved. Most likely, it would not capture most bona fide useful notifications, because reviewers wouldn’t use the app to the extent that such notifications are generated. But it would capture all the “send a notice to everybody whose every launched the app” and “send a notice to folks who haven’t launched lately” type spam. That seems like a pretty big deal.

At the very least, such a system could serve as a baseline mechanism for flagging developers, and in the event that some future crowd-sourced solution was unveiled, it would layer nicely on a system in which Apple was already collecting massive amounts of data about the most humdrum, spammy notifications that developers send.

Insecure Keyboard Entry

If you use a passphrase to control access to your computer, as you probably should, then it has no doubt become second nature to type it quickly when you sit down to get to work. If you’ve set an aggressive lock-screen timeout, as you probably also should, then you have become blazingly efficient at typing this password. Perhaps too blazing, perhaps too efficient.

If this sounds like you so far, perhaps I can complete the picture by describing the heart-stopping horror of sitting down to your computer after a short time away, methodically typing your password in to unlock it, only to realize the computer wasn’t locked at all, and you just typed it into a chat window, or worse, posted it to Twitter?

I set out recently to address this problem on my computer by writing my own nefarious little tool, which would act as a global keystroke sniffer, looking for any indication that I am typing my password, at which point it puts up a helpful reminder:

Panel reminding me not to type my password in plain text fields.

The beauty of this tool is it catches me at the moment I type my password (actually just a prefix of it, but that’s a technicality), and by nature of putting up a modal dialog that jumps in my face, absorbs any muscle-memory-driven effort to complete the password and press return in whatever insecure text field I might have been typing into.

You may wonder whether this prevents the legitimate entry of my password, e.g. into fields such as the system presents when asking me to confirm an administrator task? The answer is no, because part of the beauty of those standardized password fields is that Apple has taken care to enable a secure keyboard entry mode while these fields is active. While a standard password field is focused, none of your typing is (trivially) available to other processes on the system. So my tool, along with any other keyboard loggers that may be installed on the system, are at least prevented from seeing passwords being typed.

I’ve been running my tool for a few weeks, confident in the knowledge that it will prevent me from accidentally typing my password into a public place. But its aggressive nature has also revealed to me a couple areas that I expected to be secure, but which are not.

Insecure Input Fields

The first insecure input area I noticed was the Terminal. As a power-user, it is not terribly uncommon for me to invoke super-user powers in order to e.g. clean up a system-owned cache folder, install additional system packages, kill system-owned processes that are flying out of control, or simply poke around at parts of the system that are normally off-limits. For example, sometimes I edit the system hosts file to force a specific hostname to map to an artificial IP address:

sudo vi /etc/hosts
Password:•

The nice “•” is new to Yosemite, I believe. Previously tools such as sudo just blocked typing, leaving a blank space. But in Yosemite I notice the same “secure style” bullet is displayed in both sudo and ssh, when prompting for a password. To me this implies a sense of enhanced security: clearly, the Terminal knows that I am inputting a password here, so I would assume it applies the same care that the rest of the system does when I’m entering text into a secure field. But it doesn’t. When I type my password to sudo something in the Terminal, my little utility barks at me. There’s no way around it: it saw me typing my password. I confirmed that it sees my typing when entering an ssh password, as well.

The other app I noticed a problem with is Apple’s own Screen Sharing app. While logged in to another Mac on my network, I happened to want to connect back, via AppleShare, to the Mac I was connecting from. To do this, I had to authenticate and enter my password. Zing! Up comes my utility, warning me of the transgression. Just because the remote system is securely accepting my virtual keystrokes, doesn’t mean the local system is doing anything special with them!

What Should You Do?

If you do type sensitive passwords into Terminal or Screen Sharing, what should you do to limit your exposure? Terminal in particular makes it easy to enable the same secure keyboard entry mode that standard password fields employ, but to leave it active the entire time you are in Terminal. To activate this, just choose Terminal -> Secure Keyboard Entry. I have confirmed that when this option is checked, my tool is not able to see the typing of passwords.

Why doesn’t Apple enable this option in Terminal by default? The main drawback here is that my tool, or other tools like it, can’t see any of your typing. This sounds like a good thing, except if you take advantage of very handy utilities such as TextExpander, which rely upon having respectful, trusted access to the content of your typing in order to provide a real value. Furthermore, if you rely upon assistive software such as VoiceOver, enabling Secure Keyboard Entry could impact the functionality of that software. In short: turning on secure mode shuts down a broad variety of software solutions that may very well be beneficial to users.

As for Screen Sharing, I’m not sure there is anyway to protect your typing while using it. As a “raw portal” to another machine, it knows nothing about the context of what you’re doing, so as far as it’s concerned your typing into a password field on the other machine is no different from typing into a word processor. Unfortunately, Screen Sharing does not offer a similar option to Terminal’s application-wide “Secure Keyboard Entry.”

What Should Apple Do?

Call me an idealist, but every time that tell-tale • appears in Terminal, the system should be protecting my keystrokes from snooping processes. I don’t know the specifics of how or why for example both ssh and sudo receive the same treatment at the command-line, but I suspect it has to do with them using a standard UNIX mechanism for requesting passwords, such as the function “getpass()” or “pam_prompt()”. Knowing little about the infrastructure here, I’m not going to argue that it’s trivial for Apple to make this work as expected, but being in charge of all the moving parts, they should make it a priority to handle this sensitive data as common sense would dictate.

For Screen Sharing, I would argue that Apple should offer a similar option to Terminal’s “Secure Keyboard Entry” mode, except that perhaps with Screen Sharing, it should be enabled by default. The sense of separation and abstraction from the “current machine” is so great with Screen Sharing, that I’m not sure it’s valuable or expected that keyboard events should be intercepted by processes running on the local machine.

What Should Other Developers Do?

Apple makes a big deal in a technical note about secure input, that developers should “use secure input fairly.” By this they mean to stress that any developer who opts to enable secure input mode (the way Terminal does) should do so in a limited fashion and be very conscientious that it be turned back off again when it’s no longer needed. This means that ideally it should be disabled within the developer’s own app except for those moments when e.g. a password is being entered, and that it should absolutely be enabled again when another app is taking control of the user’s typing focus.

Despite the strong language from Apple, it makes sense to me that some applications should nonetheless take a stronger stance in enabling secure input mode when it makes sense for the app. For example, I think other screen sharing apps such as Screens should probably offer a similar (possibly on by default) option to secure all typing to an open session. I would see a similar argument for virtualization software such as VMware Fusion. It’s arguable that virtualized environments tend to contain less secure data, but it seems dangerous to make that assumption, and I think it does not serve the user’s expectations for security that whole classes of application permit what appears to be secure typing (e.g. in a secure field in the host operating system) that is nonetheless visible to processes running on the system that is running the virtualization.

What Should I Do?

Well, apart from writing this friendly notice to let you know what you’re all up against, I should certainly file at least two bugs. And I have:

  • Radar #19189911 – “Standard” password input in the Terminal should activate secure input
  • Radar #19189946 – Screen Sharing should offer support for securing keyboard input

Hopefully the information I have shared here helps you to have a better understanding of the exposure Terminal, Screen Sharing, and other apps may be subjecting you to with respect to what you might have assumed was secure keyboard input.