Category Archives: Mac OS X

Disciplined Innovation

Apple’s macOS 13 “Ventura” beta features a major redesign of the System Preferences application. In addition to renaming it “System Settings,” Apple revamped the interface with an organization style that is far more reminiscent of the iOS “table view” organization than of the macOS status quo:

Screenshot of the macOS 13 Ventura System Settings Appearance tab

At first blush, there are apparent advantages to this design. Perhaps most significantly, the resemblance to iOS in both appearance and function may make the interface more navigable, and easier to understand, for users who are more familiar with iOS than with the Mac. The familiar “stack based” approach to delving deeper into the details of a particular preference pane has some advantages to the typical Mac approach of presenting detail as modal panels, which sometimes beg to be awkwardly nested in a pile of secondary or tertiary panels.

On second, third, and many blushes beyond, however, the design of System Settings appears to represent a major regression in overall usability and aesthetics. It has been taking public jabs since the first deveoloper beta, prompting a challenging question from John Gruber in his interview with Apple’s Greg Joswiak and Craig Federighi. Federighi replied with a compelling argument in favor of unifying the design experience across platforms, and removing historic cruft from the legacy macOS designs. Federighi further complained that Apple was being “judged for its betas,” implying that the UI would see significant improvements over the course of the summer.

We have seen improvements, but by the judgement of most Mac faithful (or at least the loudest among them), these improvements are not enough. Recently, a trending Twitter thread by Niki Tonsky reignited criticism, while many people pointed out that as the expected ship date for macOS Ventura draws closer, we are less likely to see dramatic improvements.

John Gruber returned to the subject in a piece yesterday, in which he asserts “the basic fit and finish of Ventura’s new System Settings is just bad.” He lays much of the blame for this on SwiftUI, which he further asserts that with SwiftUI “so many little layout details are apparently hard to get right.”

I have heard several people acknowledge that the successful transition to Apple Silicon was accomplished in part by focusing on the underlying architectural change while having the discipline to keep the hardware the same. The thinking is that with such a dramatic change in the underlying technology of the Mac, it would have been reckless to attempt other major hardware changes at the same time. In short: the Apple Silicon transition can be judged as successful on the basis that anybody using such a Mac might not even know they were using a fundamentally different computer design.

With Apple Silicon, Apple was able pull the proverbial table cloth out from under the exquisite place settings of the Mac, comprising its beloved hardware and software features, while leaving everything standing exactly as it was. That’s quite an achievement.

I think that SwiftUI would be judged as a more successful transition if Apple had pulled off a similar stunt. What if they had approached the challenge by making sure, first and foremost, that every Mac and iOS UI component behaved exactly the same as before? Then, as with the Apple Silicon changes, they could leverage the advantages of the new technhology to expand and improve upon the status quo, rather than attempting to replace it.

By choosing to change the underlying technology while also introducing new UI designs and behaviors, Apple has effectively attempted to pull the tablecloth out from under the place settings, while also pouring tea, cutting the cake, and serving the sandwiches. Messes will get made, some dishes are bound to get broken, and they’ll take a long time to put back together.

Purgeable Mac Apps

For months now, I have been scratching my head over a small but persistent number of “crash reports” affecting a few of my apps. The issue is most prevalent in MarsEdit, where I have a handful of users who run into the issue multiple times per day. Luckily, one of these users is my good friend and colleague, Manton Reece. I’ve been peppering him with questions about the issue for weeks, while he stoicly puts up with the behavior.

Even with the assistance of a highly technical friend who can reproduce the issue at will, I had thrown my arms up in despair several times. I put “crash reports” in quotes above, because although my in-app crash reporter notices the app abruptly terminates, the system doesn’t create any obvious artifacts. No crash or hang reports. No “Quit Unexpectedly” dialog. The app is just … gone. I wrote a question in the Apple Developer Forums, which turned into a kind of de facto diary as I pursued the issue.

When I started to feel bad about asking Manton to try this, that, and the other thing, I finally asked if he could send me a “sysdiagnose” report. If you’re curious, the easiest way to grab one of these on any Mac is to simply press the Control, Option, Command, Shift, and “.” (period) keys at once. You’ll see the screen flash, an indication the system is starting to collect the reports. A few minutes later the report will be revealed in the Finder: a probably quite large zip archive. Open it up and see the wealth of information about nearly every aspect of the system.

Yet even with this wealth of information, I was stymied. It wasn’t until I chanced upon the delightfully pertinent nuggets of information in “/var/log/com.apple.xpc.launchd/launchd.log” that I got my first whiff of a clue:

2022-05-03 09:15:22.088718 (gui/501/application.com.red-sweater.marsedit4.384452971.384452977 [13840]) : exited with exit reason (namespace: 15 code: 0xbaddd15c) - OS_REASON_RUNNINGBOARD | <RBSTerminateContext| code:0xBADDD15C explanation:CacheDeleteAppContainerCaches requesting termination assertion for com.red-sweater.marsedit4

Here we have a message asserting that MarsEdit was terminated, on purpose, and better still, it includes an explanation! As far as explanations go, “CacheDeleteAppContainerCaches” is not much of one, but it did give me something to go on. Searching for the term yielded pertinent results like this post about Apple Mail and Safari “suddenly quitting.” Unfortunately, they all seem to be scratching their heads as much as I am.

The other thing that jumped out at me from the log was the term “OS_REASON_RUNNINGBOARD”. Searching for this results in only a few scant links, all related to Apple’s open source Darwin kernel. However, Searching instead for just “RunnningBoard” offered a glimmer of hope. A post on Howard Oakley’s blog, “RunningBoard: a new subsystem in Catalina to detect errors“, includes a particularly succinct description of the eponymous OS subsystem (emphasis mine):

Catalina brings several new internal features, a few of which have been documented, but others seem to have slipped past silently. Among the latter is an active subsystem to replace an old service assertiond, which can cause apps to unexpectedly terminate – to you and me, crash – in both macOS 10.15 and iOS 13: RunningBoard.

Unexpected termination. Yep. To you and me? Crashing. At this point in the story I’m going to elide several hours of long, tedious, and yet still somehow fun work, wherein I disabled System Integrity Protection on my Mac, so that I could attach to the pertitent system daemons and try to make sense of how, and when, they might decide to unilaterally terminate an app like MarsEdit. While digging deeper into the issue, I remembered that “explanation” from the log, CacheDeleteAppContainerCaches, and it reminded me of system maintenance software like CleanMyMac. I normally shy away from these kinds of apps because they are historically known to be overly-aggressive in what they decide to delete. In the name of science, however, I decided to run it, with care, on my Mac.

Boom! After running CleanMyMac once, MarsEdit, along with Numbers, were suddenly not running anymore. I had finally reproduced the issue on my own Mac for the first time. Anybody who has fixed software bugs, either for a living or as a passion, knows this is the critical first step to really addressing an issue. With some tinkering, I was able to narrow down the reproduction steps to running the “Free Up Purgeable Space” action. It turns out this is invokes a system API responsible for trying to delete caches, etc., from a Mac. Normally the system only does this when disk space is critically low, but CleanMyMac gives you the option to exercise the behavior at any time.

That single log line quoted above turns out to hold another gem of information. The “code:0xBADDD15C” looks like it could be an arbitrary hexadecimal value, but it’s an example of an error code designed to both uniquely identify and suggest a mnemonic clue to the underlying issue. Apple documents many of these codes, which include 0xc00010ff (cool off), 0xdead10cc (deadlock), and 0xbaadca11 (bad call). I searched the system frameworks for this code and found it in the disassembly of “/System/Library/PrivateFrameworks/CacheDelete.framework”. Particularly, in an internal function called “assert_group_cache_deletion”. It was only after exploring the issue in the forums, did Quinn explain that the code in this scenario is a mnemonic for “bad disk”. I guess it was easier to spell out than trying to represent “full disk”.

Equipped with all this new information, what can we do about the unexpected terminations? Well, nothing. I do wish Apple’s framework would try asking nicely if the app would quit, before summarily terminating it, but I guess the thinking is that this functionality should typically only be reached in extenuating circumstances. After learning more about the issue, I confirmed with Manton that his Mac did have low disk space, so I guess it was just the system trying its best to free up space that caused the issue for him.

The one thing I plan and hope to do as followup is to amend my built-in crash reporter so that it will not prompt the user or report a crash when the app terminates for this reason. I think it should be possible to detect the codes alluded to above, and simply let “0xBADDD15C” terminations happen without fanfare.

NetNewsWire 6.1 for Mac: Subscribe to Feed

Nearly ten years ago, inspired by Apple’s removal of the “RSS” button from Safari 6, I released a standalone Safari extension called Subscribe to Feed. The extension simply replaced the functionality of having an easy way to subscribe to the RSS feed of any web site you might be viewing.

This worked great for many years, until Apple’s ever-increasing security controls changed the behavior of Safari extensions such that they couldn’t call out to desktop apps (such as a news reader) without requiring approval from the user each and every time. This made the process of subscribing to a feed more cumbersome, albeit still easier than manually searching for the feed URL, copying, and pasting it into an RSS reader app.

When Brent Simmons resurrected NetNewsWire, I saw it as a perfect opportunity to carry on the mission of the Subscribe to Feed extension. Native applications on the Mac can offer Safari app extensions that extend the browser in ways that are similar to traditional web browser extensions, but which are not subject to the same tedious security restrictions. The idea is that if the user knowingly installs an app and then enables the corresponding extension, they are probably happy to have its functionality performed without warning.

I implemented a simple version of Subscribe to Feed within NetNewsWire a few years ago, but held off on “announcing it” per se, because I was waiting for some pieces to fall together so that I could endorse it as a fully-functional replacement for the old Subscribe to Feed extension. One of my main criteria was that the extension should be useful to people even if they don’t use NetNewsWire.

With the release of NetNewsWire 6.1, the app now has a preference for the Safari extensions so you can choose whether to open feeds in NetNewsWire itself, or in the default news reader.

So if you are a fan of another RSS app that doesn’t have its own Safari extension, you can simply install NetNewsWire 6.1, configure the Safari extension to open feeds in your default app, and enjoy the full functionality of the “Subscribe to Feed” extension without ever launching NetNewsWire again. Though you might as well give it another look since it’s there …

Catalina’s Custom Keyboard Viewers

Long-time Mac users will remember an app called “Key Caps”, which later become “Keyboard Viewer”, a feature of the Mac that is now accessible via the menu bar’s “Input Methods” item. If you’ve never played with this, I encourage you to enable it and check it out. Apple has detailed instructions for configuring the menu and these options.

I don’t use the Keyboard Viewer often, but when I do, it’s a real life-saver. I brought it up recently while I was debugging an issue with keyboard shortcuts in FastScripts, my scripting utility app. The Keyboard Viewer not only reflects every bona fide keystroke you make on a hardware keyboard, but also allows you to simulate keystrokes by tapping on the keys of the on-screen keyboard.

On macOS 10.15 Catalina, Apple has evidently dramatically overhauled the Keyboard Viewer. I don’t see any hint of this on the Apple marketing sheet for the OS, but this is what the Keyboard Viewer looks like on my Mac now:

NewImage

Well, isn’t that spiffy? But what I really want to talk about is that little Gear Button in the upper right corner of the window. Click it, and this what you get:

Popup menu with various options for customizing keyboards

A whole slew of options for tweaking the behavior of the virtual keyboard, and an enticing “Customize…” item at the bottom. When you select it, a dedicated application called “Panel Editor” opens up. It’s essentialy a construction set for building virtual keyboard layouts:

Custom keyboard editor with silly bitsplitting.org keyboard layout

This example is obviously comical, but the point is you can create and layout tappable regions that correspond to whatever keystrokes you desire. The options for configuring these keys even include options to perform multiple keystrokes, open apps, run scripts, etc. It’s a powerhouse of utility superpowers.

How did they possibly find time to add all this great functionality in one OS upgrade? They didn’t. Folks who are familiar with Apple’s Accessibility Keyboard have no-doubt recognized my screenshots as being familiar from past OS releases. I personally had never seen it before, but it’s been hiding in the System Preferences Accessibility tab. What happened in macOS 10.15 Catalina is that Apple has evidently recognized its superiority in all ways to “Keyboard Viewer” and allowed the Accessibility Keyboard to simply take its place.

This is an excellent example of software being designed to assist people with specific needs, yet actually being useful to everybody. That is the heart of accessible software design, and I think we’ll see more and more “accessible” software released from the relative obscurity of the Accessibility tab as we move forward.

Terminal Security Profiles

In macOS Mojave, Apple introduced a number of new security features that impact the day-to-day use of the computer. Activities such as running scripts, or using apps that access private information, are altered now such that users are prompted with one-time permission-granting requests.

One consequence of these changes is that you can no longer access certain parts of your home directory from the Terminal. Don’t believe me? Try opening Applications > Utilities > Terminal, and run the following command:

ls ~/Library/Mail

In all previous macOS releases, this would list the contents of Apple’s internal Mail files. As a privacy enhancement, access to these files is now restricted unless apps have requested or been proactively granted access.

If you really wanted to regain access to these files via the Terminal, you have to grant the app “Full Disk Access.” This is a new section of the Security & Privacy pane in System Preferences.

Well, that’s fine. Now you can “ls” anything in your home folder, but absolutely every other thing you run in Terminal can as well. To grant myself the ability to list files in ~/Library/Mail, am I willing to grant the same access to every single thing I’ll ever run in Terminal?

This isn’t earth-shattering: it’s been the case forever that tools you run in the Terminal have access to “all your files.” But the new restrictions in macOS Mojave shine a light on a problem: the bluntness of security restrictions and relaxations with regard to Terminal.

I’ve run into a variation of this problem in the past. I use the excellent TripMode to limit bandwidth usage when I’m traveling, and tethered to my phone. A consequence of this is that, unless I grant unlimited network access to Terminal, I can’t perform routine tasks such as pushing git changes to a server.

Ideally these permission grants would be applicable at the tool level, rather than at the application level. It would be better if I could say “let ls access my Mail” rather than “let anything I run from Terminal access my Mail.”

I don’t completely understand the limitations there, but I suspect that because commands in the Terminal are running as subprocesses of Terminal, there is some technical challenge to making the permissions apply at such a fine-grained level.

As an alternative, I wonder if Apple could introduce some kind of “Security Profiles” feature for Terminal so that individual windows within the app could be run when different permissions? This could build on Terminal’s existing support for “Profiles” which already support varying Terminal settings dramatically on a per-window basis.

With Security Profiles, a user would be configure an arbitrary number of named profiles, and security privileges acquired by Terminal would be stored separately for the active profile. Each profile would be considered by the system effectively as a different app. For example, given my uses of Terminal, I might set up a few profiles for the types of work I regularly do:

  • Personal: Everyday productivity tasks including running scripts, editing files in my home directory, etc.
  • Administrative: Tasks that pertain to the overall maintenance of my Mac: examining system logs, delving into configuration files, etc.
  • Collaborative: Tasks that involve installing and running third-party tools that I trust, committing to shared source repositories, etc.
  • Experimental: Tasks that involve installing or running third-party tools that I am not familiar with and do not have a high degree of faith in.

These are off the top of my head, and just to give an idea of the kinds of profiles that might make sense here. Switching between these modes would also switch the system’s active list of entitlements for Terminal. If I run a script that accesses my Calendar items from the “Personal” profile, the system would prompt me once to ask my permission, but never prompt me again in that profile. When I switch to “Experimental” and run some unfamiliar third-party tool that tries to access my calendar, it would ask permission again for that profile.

I filed Radar #45042684: “Support a finer-grained permissions model for Terminal”, requesting access for this or something like it.

More on Mojave’s Automation Sandbox

I wrote last month about macOS Mojave’s restrictions on automation, and how users can reset the database that controls them. In that post, I cited Felix Schwarz’s excellent article on the subject.

In recent weeks, Apple has made changes to the behavior of macOS Mojave, and added some API calls to help developers better handle the restrictions of the system. Felix is back with an updated post, describing the changes, and what he thinks can still be improved.

Reauthorizing Automation in Mojave

The macOS Mojave betas include a significant enhancement to user control over which applications can perform automation tasks. When we talk about automation on the Mac, we usually think of AppleScript or Automator, but with a broader view automation can be seen as any communication from one application to another.

One ubiquitous example of such an automation is the prevalence of “Reveal in Finder” type functionality. For example if you right-click a song file in iTunes, an option in the contextual menu allows you to reveal the file in the Finder. This is a very basic automation accomplished by sending an “Apple Event” from iTunes to the Finder.

In the macOS Mojave betas, you’ll notice that invoking such a command in an application will most likely lead to a panel asking permission from the user. The terminology used is along the lines of:

“WhateverApp” would like to control the application “Finder”.

If the user selects “OK”, the application sending the command will be thereafter whitelisted, and allowed to send arbitrary events (not just the one that prompted the alert) to the Finder. If you’re running macOS Mojave you can see a list of applications you’ve already permitted in System Preferences, under “Security and Privacy,” “Privacy,” “Automation”.

These alerts are a bit annoying, but I can get behind the motivation to give users more authority over which applications are allowed to control other applications. Unfortunately, there are a number of usability issues and practical pitfalls that come as side-effects of this change. Felix Schwarz made a great analysis of many of the problems on his blog.

I ran into another usability challenge that Felix didn’t itemize: the problem of denying authorization to an application and then living to regret it. I guess at some point I must have hastily denied permission for Xcode (Apple’s software development app) to control the Finder. This resulted in a seemingly permanent impairment to Xcode’s “Show in Finder” feature. I’m often using this feature to quickly navigate from Xcode’s interface to the Finder’s view on the same files. After denying access once, the feature has the unfortunate behavior of succeeding in activating the Finder (I guess that one is whitelisted), but failing silently when it comes to revealing the file.

OK, that’s fine. I messed up. But how do I undo it? Unfortunately, the list of applications in the Security and Privacy preference pane is only of those that I have clicked “OK” for. There’s no list of the ones that I’ve denied, and no apparent option to drag in or add applications explicitly. For this high level problem, I filed Radar #42081464: “TCC needs user-facing mechanism for allowing previously denied privileges.”

What’s TCC? I’ll be darned, I don’t know what it stands for. But it’s the name of the system Apple uses for managing the system’s so-called “privacy database.” This is where these and other permissions, granted by the user, are saved. For instance, in macOS 10.13 when the system asks whether to grant access to your Address Book or Contacts, the permission is saved, and managed thereafter, by TCC.

Resetting TCC Privileges

I knew from past experience testing Contacts privileges in my own apps, that Apple supports a mechanism for resetting privileges. Unfortunately, it’s pretty crude: if you want to change the authorization setting for an application you’ve previously weighed in on, you have to universally wipe out all the privileges for all apps using a particular service. For Contacts, for example:

tccutil reset AddressBook

This completely removes the list of apps authorized to access Contacts. (The AddressBook naming is a vestige of the app’s former user-facing name.) In fact, if you type “man tccutil” from the Terminal, you’ll find that AddressBook is the only service explicitly documented by the tool. Fixing my Xcode problem is not going to happen by resetting AddressBook privileges. So what do I reset? I tried the most obvious choice, “Automation,” results in an error: “tccutil: Failed to reset database”.

What’s the service called, and does tccutil even support resetting it? After a crude search of the private TCC.framework’s binary, I discovered I was looking for “AppleEvents”:

tccutil reset AppleEvents

After running this, I quit and reopened Xcode (the TCC privileges seem to be cached), and selected “Show in Finder” on a file. Voila! The Finder was activated and I was again asked if I wanted to permit the behavior. This time, I made sure to say “OK.”

You can get a sense for the variety of services tccutil apparently supports resetting by dumping the pertinent strings from the framework:

strings /System/Library/PrivateFrameworks/TCC.framework/TCC | grep kTCCService

The list of matching strings includes names like AppleEvents and AddressBook, as well other names for things I don’t recognize, and a seemingly useful “All,” which can presumably be used to wipe out all authorizations across all services.

Because the tccutil is far more useful than is advertised, and because users are undoubtedly going to end up needing to reset services more than ever in Mojave, I also filed Radar #42081070: “Documentation and command-line help for tccutil should enumerate services.” There are some items in the dumped list that appear likely to be private to Apple, but anything genuinely useful to customers (or more likely, the consultants who fix their Macs) should be listed in the manual.

Lighten Up, Eh?

While I support the technical and user-facing changes suggested by Felix Schwarz in the previously linked blog post, some issues would be avoided by simply giving apps the benefit of the doubt for widely used, innocuous forms of automation.

I mentioned earlier that the Apple Event sent by Xcode to “activate the Finder,” was apparently whitelisted by the system. Evidently Apple saw wisdom in the thinking that simply causing another application to become active is unlikely to be widely abused. I think the same argument holds for asking the Finder to reveal a file. I filed Radar #42081629: “TCC could whitelist certain widely used, innocuous Apple Events.”

I mentioned before that I can support Apple’s effort to put more power into users’ hands with this feature, but one side-effect of requiring the authorization even for innocuous events like “Show in Finder” is that apps that do not otherwise offer automation functionality to users will nonetheless require that users grant that power.

If the merit in the feature is to allow users to limit what kinds of automation apps can perform, then supporting a “Show in Finder” feature for an application should not require me to simultaneous permit it to do whatever kind of Finder automation it chooses to. For example, an application so-authorized is now empowered, presumably, to send automation commands to the Finder that modify or delete arbitrary user files.

These days Apple always seems to be pushing the privacy and security envelope, and in many ways that is great for their users and for their platforms. With a little common-sense and some extra engineering (“It should be easy” — Hah!), we can get the best of the protection these features offer, while suffering the fewest of the downsides.

Unified Swift Playgrounds

Usually I write about “programmer stuff” on Indie Stack, and “other tech stuff” here.

Today I had it in mind to write about my belief that Apple should get rid of Xcode Playgrounds and instead focus on adapting the iOS Swift Playgrounds app to the Mac.

I thought at first I would post it here because it’s not strictly about programming, but I ended up posting it there instead:

Unified Swift Playgrounds

May be of some interest even to folks not strictly interested in programming!

Accessible Resistance

Accessibility in software refers to the noble ambition of ensuring that software is usable by as diverse a user base as possible. To that end, software is made more accessible by adapting to a variety of physical or cognitive impairments that may affect any individual user.

In the United States and other countries, there is an ugly trend towards supporting politicians who don’t believe that people from diverse backgrounds, or with specific impairments, should be accommodated by society as a whole.

Many developers are looking for concrete ways to fight these politicians who don’t value diversity and inclusion. One small thing we can all do to push back, to resist, is to ensure our own apps are as accessible as possible.

During my many years as an indie Mac developer, I have often prioritized accessibility in my apps. I have heard from many MarsEdit users, particularly those with vision difficulties, who tell me its accessibility makes it a better alternative to many other blogging solutions.

I am gratified to hear about the ways I have gotten accessibility right, but I am still not satisfied that I have done enough. There are nuances of MarsEdit’s accessibility that can yet be improved, while some of my other apps, such as Black Ink, are still hardly accessible at all.

If you are a Mac or iOS developer who is committed to improving the accessibility of your app, a great place to start is with the WWDC 2016 What’s New In Accessibility session. Apple is always enhancing the variety of accessible features that are built in to iOS, macOS, tvOS, and yes!, even watchOS.

Spend a half hour watching this video, and start getting up to speed with how you will enhance the accessibility of your app. No matter where you live in the world, you can be a strident voice for inclusion by declaring, through your actions in Xcode, that your software is designed to be used by everyone.

Bitter iCloud Truth

A few days ago, I set about tidying up my completely unruly Contacts database. Over the years I have accumulated 2,953 distinct contact cards. Thousands are duplicates, hundreds were added automatically by Mail or macOS, some are for people whom I no longer know, and a very few are for folks I no longer want to know.

The first thing to do was obviously to take care of those duplicates. Contacts on the Mac features a couple seemingly handy menu items for dealing with this problem: “Look for Duplicates” and “Merge Selected Cards.” I cannot recommend using either of these features.

When I invoked “Look for Duplicates” on my 3000 cards, of which I could tell by visually scanning that at least 1000 were duplicates, Contacts came back at me with a stunningly unhelpful offer to fix 9 of them:

NiceTry

Seeing that I would have to identify the clones myself, I committed to selecting groups of two or three clearly duplicated identities, and selecting Card -> Merge Selected Cards. I generally trust Apple software to do the right thing, but recognizing the heartache that would come with losing any information from these cards, I decided to proactively export my Contacts database so that I would be safe should anything go wrong. Selecting File -> Export… -> Contacts Archive from the menu bar produced a 60MB bundle that seemed certain to contain all of the contact information on my Mac.

I tried “Merge Selected Cards” on a few items, and it seemed to do the trick. Where data from the two cards was identical, it merged them cleanly. Where something was in conflict, for example if a Company Name had changed from “Google” to “Apple” at some point, the conflict was resolved by picking one and appending the other to the merged card’s “Note”.

After painstakingly merging cards in this manner for an hour or so (!), I had stopped paying close attention to whether conflicting data was being persisted well or not. At one point I stumbled upon the realization that I lacked the phone number for a contact whom I had sent an SMS message just within the past week. Other contacts were missing key data, too. An outdated email address here, a missing mailing address there. Whoops! Abort mission! Time to recover from that backup file.

I double-clicked the backup file in the Finder, and agreed when Contacts asked if I was sure I wanted to “replace my Contacts data” with the contents of the archive.

Screenshot 1 3 17 1 14 PM

You’re damned right, I am sure! Give me back my data.

After restoring from Backup, I found that I didn’t have any duplicate entries in Contacts. In fact, I only had around 130 cards. Somehow I had lost the vast majority of cards in this process. What’s going on here?

I think what happened is that exporting from Contacts on my Mac is creating an archive that, when imported, possibly reflects the state of my Contacts database from years ago, before I ever agreed to sync Contacts with iCloud. No amount of futzing with local data on my Mac, including disabling iCloud syncing, and restoring the contents of Contacts’s data folder (~/Library/Application Support/AddressBook) would coerce Contacts into showing me all of my Contacts. I began to panic. Had I actually lost all of my Contacts data? In spite of dutifully backing up my local data and making a proactive archive from Contacts, the app had a very different idea of what my Contacts “truth” was.

I was relieved to discover an iCloud.com feature for recovering Contacts to the state they were in on a previous date. You get to it by navigating to iCloud Settings -> Advanced, and clicking the “Restore Contacts” link.

NewImage

After restoring Contacts through iCloud.com, and syncing to iCloud from my Mac, my Contacts were finally restored to their absurdly broken, but data-intact, state.

The overarching lesson here is that when it comes to iCloud-synced data, you cannot count on local data, and worse: you can’t count on local archives. If you care to archive your data, navigate to iCloud.com and use the pertinent export feature from the web site. For example, from iCloud.com’s Contacts interface, click on a contact, then select “Select All”, and then “Export vCard…”.

ExportVcard

Amusingly, downloading this file will probably cause Contacts on your Mac to immediately attempt to import all of the contacts, which of course you will want to decline to do. Apple has a reference page, with detailed instructions for archiving various kinds of iCloud data.

The other lesson of course is you can’t trust Contacts to do a good or even acceptable job merging Contacts. I’ll be looking for another solution on this front. On Twitter, a f waller suggested Smart Merge Pro for iOS, but I tried that and it unfortunately is also blind to the duplicate nature of my cards! I guess something special is making my cards appear non-duplicated to both Contacts and this third-party app.

Update: I figured out why “Look for Duplicates” behaved so poorly: it wasn’t even considering my iCloud-based cards. So when I invoke it in Contacts on my Mac, it evaluates only the ~140 cards that are considered local to my Mac, and completely ignores the ~3000 cards that are synced via iCloud. Unfortunately there is no “Look for Duplicates” on iCloud.com, so what is a Mac-based user, for whom Smart Merge Pro doesn’t work, to do?

I think the way it works is it only considered “On my Mac” cards, if there are any local cards. If all your cards are iCloud based, then the “Look for Duplicates” command seems to work as expected.

Touch Bar Everywhere

Apple announced on Thursday that many of their new MacBook Pros will ship with a “Touch Bar,” a narrow, high resolution touch screen in place of the Mac keyboard’s traditional Function Key row.

Many people immediately wondered whether we can expect Apple to release an external keyboard with the Touch Bar. This would bring the technology to the much wider audience of Mac users who are not ready to update to the latest MacBooks, or who prefer desktop Macs, or who prefer the flexibility of using their MacBook in a desktop-style configuration.

The question was discussed on the latest Accidental Tech Podcast, in which, if memory serves, John Siracusa and Marco Arment argued different angles of the “no” argument, citing the hardware cost, the extent to which a Touch Bar keyboard would complicate the accessories lineup, and perhaps most significantly, that Tim Cook does not care enough about the Mac to prioritize pushing any such technology.

Casey Liss watched football. Zing! Actually, Casey pushed back against the cynicism, suggesting that Apple’s apparent lack of enthusiasm for the Mac does not reflect a lack of commitment to improving it. I, on the other hand, take exception with the very suggestion that Apple lacks enthusiasm or is not investing heavily in the Mac. Or, at least in this feature.

I think Apple intends to push the Touch Bar as as widely as it possibly can. The current MacBook Pro lineup is the most practical computer to debut the feature, but as it becomes possible to bundle it with external keyboards, and on notebook computers at every price point, they will do so.

Why am I so assured of Apple’s big plans for the Touch Bar? Because while many people assert that Apple is not investing seriously in the Mac, the Touch Bar’s hardware and software support appear to have been a major priority for the company in the year or two leading up to its release.

A massive amount of design work must have gone into the Touch Bar’s physical hardware, structuring the information it represents, and deciding how users will most usefully interact with it. I suspect that the Touch Bar merited an amount of design effort perhaps less than, but not completely incomparable to a standalone product like the Watch.

Thanks to the Touch Bar simulator in Xcode 8.1, we can also already take stock of the sheer amount of engineering effort, across many disparate groups in the company, that went into supporting the Touch Bar from a wide range of different apps and modes in macOS. Leave the simulator running while you go about your work, and prepare to be repeatedly surprised by the variety of novel use cases that have already been identified and implemented.

I find it impossible to believe that Apple would go to all this work, both on the Touch Bar itself, and across the entire range of its own apps and OS features, unless it had a grand vision for the Touch Bar that extends way beyond the internal keyboard of its premium notebook computers.

Instead, I think Apple sees the Touch Bar as a long-term, distinguishing aspect of using a Mac. Users will always be able to get by without one, just as they do for example when a multi-touch trackpad is not available. But macOS, and nearly every app that users run on it will work better with a Touch Bar. One day we’ll expect to always have access to one, and will feel that something is missing if we don’t.

It’s easy to see why Apple couldn’t come charging out of the gate with this vision fully realized. The Touch Bar hardware is no doubt expensive, and there are probably practical considerations with respect to the security of Touch ID, bandwidth between the device and the Mac, and managing its power needs in a user-friendly manner.

I say give Apple time. They’ve made a huge investment in Touch Bar, and all indications are they are prepared to continue prioritizing support for it down the road. We’re only on the brink of entering the early adopter phase, but in years to come I do think the Touch Bar will be everywhere.