Author Archives: Daniel Jalkut

Three Podcasts and a Blog

I’ve been wanting to create my own crossword puzzles since I was a kid, but never quite got around to it. Earlier this year I decided to renew my commitment, and tweeted a bold claim:

Let’s see, June, July, August. Yup, I missed that target. But FOUR months later, I’m ready to catch up, and share the very first puzzle I’ve ever completed. I call it “Three Podcasts and a Blog” because I set out to build a puzzle around the “theme” of names of podcasts in the Apple developer/technophile scene. Only after I was almost done with the puzzle did I realize I’d included one blog that, in fact, has no podcasting counterpart. You’ll see the clue marked “oops!”

If you want to solve the puzzle, I encourage you to download and use my own Mac app, Black Ink. If you don’t have a Mac or would prefer to solve on paper, I’m also including a PDF so you can download and print it:

Download Across Lite Puzzle

Download PDF Puzzle

Seeing as this was my first foray into the art of puzzle-crafting, I’m sure there will be lots of issues with the puzzle. Hopefully it’s still fun, especially for folks who are acquainted with Apple-related podcasts and blogs. Let me know what you think!

Apple News Encourages Frequent Blogging

When Apple News debuted, I was intrigued to learn that virtually anybody can submit their own blogs for inclusion in the service. Why not allow Bitsplitting, the Red Sweater Blog, and Indie Stack to be part of this service? For reader who enjoy Apple News, it could serve as a kind of substitute RSS reader.

Apple did, in fact, accept my news sources, and for the past several years these articles have been available through the service.

I guess I’ve dropped the ball a bit as a blogger, though, because this week I received a terse email from Apple:

Dear Daniel Jalkut,

We noticed that you have not published to your Bitsplitting channel in three months or more. Your channel will be removed in one week.

Regards,
The Apple News Team

Regards, indeed. Apple will drop me in one week if I don’t publish something, or maybe even if I do; the wording is ambiguous. I’m a little annoyed at this, but I’m also a little annoyed at myself for not blogging more frequently, so I guess I’ll just say: “thanks, Apple News!”

Update: Manton Reece notes on Micro.blog that there may be a less encouraging rationale for Apple’s crackdown on inactive publications:

@danielpunkass If you hadn’t heard, Apple News dropped RSS support for new blogs, and it sounds like they rarely approve personal blogs anymore. Weeding out inactive blogs could be the first step to removing them altogether.

Catalina’s Custom Keyboard Viewers

Long-time Mac users will remember an app called “Key Caps”, which later become “Keyboard Viewer”, a feature of the Mac that is now accessible via the menu bar’s “Input Methods” item. If you’ve never played with this, I encourage you to enable it and check it out. Apple has detailed instructions for configuring the menu and these options.

I don’t use the Keyboard Viewer often, but when I do, it’s a real life-saver. I brought it up recently while I was debugging an issue with keyboard shortcuts in FastScripts, my scripting utility app. The Keyboard Viewer not only reflects every bona fide keystroke you make on a hardware keyboard, but also allows you to simulate keystrokes by tapping on the keys of the on-screen keyboard.

On macOS 10.15 Catalina, Apple has evidently dramatically overhauled the Keyboard Viewer. I don’t see any hint of this on the Apple marketing sheet for the OS, but this is what the Keyboard Viewer looks like on my Mac now:

NewImage

Well, isn’t that spiffy? But what I really want to talk about is that little Gear Button in the upper right corner of the window. Click it, and this what you get:

Popup menu with various options for customizing keyboards

A whole slew of options for tweaking the behavior of the virtual keyboard, and an enticing “Customize…” item at the bottom. When you select it, a dedicated application called “Panel Editor” opens up. It’s essentialy a construction set for building virtual keyboard layouts:

Custom keyboard editor with silly bitsplitting.org keyboard layout

This example is obviously comical, but the point is you can create and layout tappable regions that correspond to whatever keystrokes you desire. The options for configuring these keys even include options to perform multiple keystrokes, open apps, run scripts, etc. It’s a powerhouse of utility superpowers.

How did they possibly find time to add all this great functionality in one OS upgrade? They didn’t. Folks who are familiar with Apple’s Accessibility Keyboard have no-doubt recognized my screenshots as being familiar from past OS releases. I personally had never seen it before, but it’s been hiding in the System Preferences Accessibility tab. What happened in macOS 10.15 Catalina is that Apple has evidently recognized its superiority in all ways to “Keyboard Viewer” and allowed the Accessibility Keyboard to simply take its place.

This is an excellent example of software being designed to assist people with specific needs, yet actually being useful to everybody. That is the heart of accessible software design, and I think we’ll see more and more “accessible” software released from the relative obscurity of the Accessibility tab as we move forward.

A Real Gatekeeper


In the years since Apple released the iPhone, with its “locked-down-by-nature” approach to application security, the company has progressively chipped away at the freedoms Mac developers have historically had to do, more or less, whatever the heck they wanted.

With the introduction of the Mac Application Sandbox in 2012, Apple applied an iOS-like mechanism through which applications are entitled only to access their own data, and must explicitly request permission from Apple to access any resources “outside of their own sandbox.” At the time, I wrote that while the technology was promising, it left much to be desired.

Around the same time, they introduced Developer ID, a system for certifying at runtime that a given piece of software has been cryptographically signed by a developer whose identity is known to Apple. Applications that are not signed with Developer ID are allowed to run in macOS, but by default are met with a foreboding warning about the safety of doing so. The component of macOS that is responsible for limiting the launch of software from unknown developers is called “Gatekeeper.”

Last year, in 2018, Apple introduced a new notarization service, an expansion of Developer ID functionality. Developers submit their applications to Apple, where they are scanned for known malware, and have their use of specific system technologies vetted. The “notarization” on an app allows the system to verify at runtime that a given application passes a baseline safety metric for downloaded software.

Finally, in 2019, Apple announced that software signed with Developer ID certificates, that is to say all non-Mac App Store software, must also be notarized. The Catalina 10.15 public beta identifies software that has not been notarized as potentially risky because it “cannot be scanned for malware.”

In effect: developers who ship software directly to end-users are now required to notarize their apps.

While working on the notarization process for my own apps, and a company I work for, I noticed an interesting error from “altool”, the command line program that is used to submit binaries to Apple for verification:

1 package(s) were not uploaded because they had problems:
Error Messages:
To use this application, you must first sign in to 
iTunes Connect and sign the relevant contracts. (1048)

The error is easily worked around by logging in to App Store Connect and agreeing to any updates Apple has recently made to their contracts. I’m so used to more-or-less blindly agreeing to these changes, that it didn’t sink in for me at first what a potentially major change this is.

My colleague Patrick Machielse noticed right away what the larger implication is: all Mac software, inside or outside of the Mac App Store, can now be held up by unsigned contract agreements with Apple. In a rush to fix a horrible bug and get it out to customers? Better review that new contract ASAP.

For the past 35 years, any Mac developer who wanted to ship an update directly to customers could do so by recompiling a binary and distributing it. When macOS 10.15 ships this fall, the status quo will change. Mac developers must register with Apple and sign their products. They must submit their binaries to Apple for notarization. And most significantly of all, they must agree to the terms of Apple’s App Store developer contracts, even if they don’t distribute their apps through the App Store.

Unloved Patches

For a long time I have admired the WordPress project, for developing such a robust blogging platform that is ultimately open, and free, and anybody can contribute improvements to it. I encourage many of my customers to use WordPress with MarsEdit, because it seems like a “safe bet” going forward.

My admiration has diminished a bit in the past 7 months because … I haven’t succeeded in contributing to it.

For a long time, I heard reports from my customers that dates were being set wrong in posts to WordPress. The issue in summary is that if you have a draft post on WordPress, changing its status to “Published” doesn’t update the publish date from the time the draft was originally saved.

I didn’t really get a handle on this problem until it started affecting me. Sometimes I write the show notes for my podcast, Core Intuition, ahead of the time the podcast actually goes public. In these situations, the blog post has a published date corresponding to the time I first starting writing the post, and when we finally go to publish the podcast, the date remains the same.

I did the hard work of not only diagnosing the problem in WordPress’s source code, but also writing a fix, and writing unit tests to confirm the fix. I filed a bug with a patch that will fix the problem for my customers, and any other clients of the WordPress API:

#45322: Editing a draft post with wp.editPost causes its published date to be set

Shortly after filing the bug, I went to the WordPress Slack to see what I could do about having my fixes integrated. I was lucky to have a positive response from a couple members of the WordPress team, and my bug fix seemed slated for integration.

Time passed. I wondered. I didn’t want to nag the hard-working members of the team, but I also didn’t want my hard work to have been for naught. Also, my customers, as well as other clients of the WordPress API, would benefit from this.

It’s been on my TODO list for 7 months now to “check in” with the WordPress team about this. Unfortunately, every time I do, the only thing I’ve noticed is that nobody substantially responds to my inquiries. I’m in the dead zone.

I don’t think the WordPress team is bad, by any means, but I think this reflects a problem in their process. When somebody comes to your project with a well-thought-out, unit-tested fix, and is met by radio silence? The chances are high that they will never come back again. I have submitted WordPress patches in the past, but after this experience I don’t know if I will bother submitting them again. That’s a big change in my perspective on how the WordPress team works, and on how it should work.

This post is about WordPress, but I think there are lessons for every open source project. Obviously, you can’t coddle every contributor. Some submissions will be bogus, some will be contrary to the aims of the project. But mine was a clear fix to a defect that affects multiple clients of the API. If it’s not a clear fix, I’m at least owed an explanation for why it hasn’t been committed after 7 months. In. My. Humble. Opinion.

Blog a Little

Over on Twitter today, I was inspired to ask people to write “just one blog post” today:

Later, it occurred to me that after 10+ years on Twitter, I am privileged to have a substantial following. I thought I would take the opportunity to help promote some folks who don’t have as much immediate reach:

I tagged all my retweets to those responses with #LongLiveTheOpenWeb. I think it turned out to be a pretty cool cross-section of bloggers, and I sort of editorialized the kind of blogging that people were doing.

I think people neglect to write blog posts because the feedback loop is not as tangible as the onslaught of (sometimes mechanical) likes or faves that you can receive on a social network. With blogging, you need a little faith that you will gain an audience. And on the open web, you never know who might come along and expand your audience.

The Mac Open Web

These days, as the giant social networks behave more and more reprehensibly, many people are looking back to the “good old days” of the web, when self-published blogs were the primary means of sharing one’s thoughts.

Brian Warren has taken this enthusiasm, and combined it with his nostalgia for another classic resource: the links page. He’s created a new one called Mac Open Web:

A collection of open and indie Mac, iOS, and web apps that help promote the open web.

The solitary page is jam-packed with links to resources for creating and perusing content on “the open web,” that is to say “the web.” If you’re sick of Facebook and Twitter owning your experience of what is still a hugely diverse and free global network, then spend some time investing in writing and reading on the web “the way we used to do it.”

Mac Sandboxing: Privileged File Operations

At WWDC 2018, Apple announced with great fanfare that two beloved Mac apps, Transmit and BBEdit, would be returning to the Mac App Store.

Each of these apps had departed the App Store years ago, citing various reasons, but chief among them the limitations of the Mac App Sandbox, which restricts the functionality of apps in the Mac App Store.

I was curious whether Apple made any specific concessions to these developers, and whether those concessions would be opened up to “the rest of us” or not.

Today, Panic launched Transmit 5 on the Mac App Store. It’s a free download, and costs $24.99/year after an initial 7-day free trial.

I downloaded Transmit even though I own a copy of the direct-purchase version. I wanted an answer to my question, which I got, at least partially, by dumping the application binary’s “entitlements”, which represent the sandboxing exceptions that the app has received.

New to me among the entitlements is “com.apple.developer.security.privileged-file-operations”, which is a boolean value set to true for Transmit. I don’t see any Google results for this key, so I’m assuming it’s something new that was added for Panic (and maybe BBEdit), and which may or may not be documented in the future for use by other developers.

Another interesting entitlement is “com.apple.security.automation.apple-events”, which is documented by Apple, but only in the context of the new “Hardened Runtime.” This technology is aimed primarily at developers who are not developing for the Mac App Store, but who want to provide enhanced security for their customers. In that context, I believe this entitlement provides unfettered access to sending AppleEvents, excepting that in Mojave and later the app is still subject to fine-grained system alerts that require user approval for each application that is targeted.

In short: it appears that Transmit possesses at least two “official” entitlements that could be made available, or are perhaps already available, to other developers. One way to find out: add them to your app and submit it for approval!

Update: Thanks to Jeff Nadeau for alerting me to the pertinent API that correlates with the privileged file operations entitlement. NSWorkspaceAuthorization can be used to request privileged file access from the user, and Apple includes a link for requesting access to the entitlement.

Update 2: It turns out my intrigue around “com.apple.security.automation.apple-events” was ill-founded. I assumed that a sandboxed app could use this entitlement to gain unfettered access to automating other apps, but in the case of a sandboxed app it turns out to work in conjunction with the existing “com.apple.security.temporary-exception.apple-events” entitlement, which requires enumeration of specific targets. Thanks to Jeff Johnson and Paolo Andrade for talking me through my misunderstanding of the situation.

Terminal Security Profiles

In macOS Mojave, Apple introduced a number of new security features that impact the day-to-day use of the computer. Activities such as running scripts, or using apps that access private information, are altered now such that users are prompted with one-time permission-granting requests.

One consequence of these changes is that you can no longer access certain parts of your home directory from the Terminal. Don’t believe me? Try opening Applications > Utilities > Terminal, and run the following command:

ls ~/Library/Mail

In all previous macOS releases, this would list the contents of Apple’s internal Mail files. As a privacy enhancement, access to these files is now restricted unless apps have requested or been proactively granted access.

If you really wanted to regain access to these files via the Terminal, you have to grant the app “Full Disk Access.” This is a new section of the Security & Privacy pane in System Preferences.

Well, that’s fine. Now you can “ls” anything in your home folder, but absolutely every other thing you run in Terminal can as well. To grant myself the ability to list files in ~/Library/Mail, am I willing to grant the same access to every single thing I’ll ever run in Terminal?

This isn’t earth-shattering: it’s been the case forever that tools you run in the Terminal have access to “all your files.” But the new restrictions in macOS Mojave shine a light on a problem: the bluntness of security restrictions and relaxations with regard to Terminal.

I’ve run into a variation of this problem in the past. I use the excellent TripMode to limit bandwidth usage when I’m traveling, and tethered to my phone. A consequence of this is that, unless I grant unlimited network access to Terminal, I can’t perform routine tasks such as pushing git changes to a server.

Ideally these permission grants would be applicable at the tool level, rather than at the application level. It would be better if I could say “let ls access my Mail” rather than “let anything I run from Terminal access my Mail.”

I don’t completely understand the limitations there, but I suspect that because commands in the Terminal are running as subprocesses of Terminal, there is some technical challenge to making the permissions apply at such a fine-grained level.

As an alternative, I wonder if Apple could introduce some kind of “Security Profiles” feature for Terminal so that individual windows within the app could be run when different permissions? This could build on Terminal’s existing support for “Profiles” which already support varying Terminal settings dramatically on a per-window basis.

With Security Profiles, a user would be configure an arbitrary number of named profiles, and security privileges acquired by Terminal would be stored separately for the active profile. Each profile would be considered by the system effectively as a different app. For example, given my uses of Terminal, I might set up a few profiles for the types of work I regularly do:

  • Personal: Everyday productivity tasks including running scripts, editing files in my home directory, etc.
  • Administrative: Tasks that pertain to the overall maintenance of my Mac: examining system logs, delving into configuration files, etc.
  • Collaborative: Tasks that involve installing and running third-party tools that I trust, committing to shared source repositories, etc.
  • Experimental: Tasks that involve installing or running third-party tools that I am not familiar with and do not have a high degree of faith in.

These are off the top of my head, and just to give an idea of the kinds of profiles that might make sense here. Switching between these modes would also switch the system’s active list of entitlements for Terminal. If I run a script that accesses my Calendar items from the “Personal” profile, the system would prompt me once to ask my permission, but never prompt me again in that profile. When I switch to “Experimental” and run some unfamiliar third-party tool that tries to access my calendar, it would ask permission again for that profile.

I filed Radar #45042684: “Support a finer-grained permissions model for Terminal”, requesting access for this or something like it.

Saying Goodbye to NetNewsWire 3

In case you haven’t heard the news, Brent Simmons recently regained the rights to NetNewsWire, the groundbreaking Mac news reader, which also happens to be the progenitor of MarsEdit.

I have been a fan of NetNewsWire since before Brent sold it to NewsGator. Since before NewsGator sold MarsEdit to me. Before they sold NetNewsWire to Black Pixel. For a long time.

After Black Pixel took the reins, they put a lot of effort into a massive overhaul of the app, modernizing the look and feel and adding a robust, in-house syncing mechanism. When they released NetNewsWire 4 in 2015, it seemed as though the future for the app was bright.

As nice as NetNewsWire 4 was, it also differed a lot from NetNewsWire 3. They pared back the feature set a lot, in ways that made switching inconvenient to me. So I soldiered on with 3.3.2, thinking that I would update to 4.x eventually.

I never did. For whatever reason, work on NetNewsWire seemed to stall, and I never found the updated version of the app to fit my needs. NetNewsWire 3 worked just fine.

The meaning of “just fine” started to shift as macOS changed underneath the app. Subtle bugs emerged, the app’s lower-resolution graphics started to look fuzzy, and the networking infrastructure of the app is from an older era that is failing to connect to some SSL servers. In short, it’s no longer the great app that it once was. One particular bug with the size of the “Clippings” folder icon has been bugging me for years:

Screenshot of NetNewsWire showing blurry icons and a missized Clippings folder icon.

Over the years I considered other news readers such as Reeder (which is free for a limited time, by the way), but none of them scratched that NetNewsWire 3 itch. I rely upon some arcane features of the app including “scripted feeds,” which allow me for example to run Python scripts on my Mac that connect to Twitter and generate RSS feeds from search results. That’s not possible in most feed readers.

I used to fantasize about getting access to the NetNewsWire 3 source code and sprucing it up. I wondered how things might have turned out differently if, in addition to acquiring MarsEdit from NewsGator, I had acquired both? I can’t say I would have done a better job than Black Pixel, but I would have preserved the features I care about, and that Clippings folder icon would be the right size!

Because Brent and I are still close friends, we have been in conversation about NetNewsWire and the various options for moving it forward into the future. I’ve also been contributing to the NetNewsWire open source project, which is based on an entirely new code base unrelated to NetNewsWire 3.

Since I’m not the only stalwart NetNewsWire 3 user, one of the things Brent was curious about was whether he could give that version “one last hurrah,” so to speak. Fix a few of the most glaring bugs, build against a modern SDK, and not only create an artifact for history to more accurately judge the app’s virtues, but to give long-standing users something to tide them over while development continues on NetNewsWire 5.

I was honored when Brent handed me the keys to the castle, so to speak, by sending me a copy of NetNewsWire 3’s source code. To heavily paraphrase what he said, it was basically “let me know if it’s worth saving.”

I got the app building with Xcode 10 on macOS Mojave beta 9. There were some major glitches. The sidebar was pure black, fonts were rendering wrong. Probably whole subsets of functionality were not working, or working unreliably. I sent the source base back to him with a report that it builds and runs, but would probably take some work to get into shippable shape.

Brent made the pragmatic choice not to release an updated NetNewsWire 3. Putting the bugs aside, he recognized that any time invested in that old version is an investment in older technology that does not have a viable future. It’s a distraction from the New World NetNewsWire.

To be honest, the decision doesn’t sting at all. I’ve switched most of my news reading to development releases of NetNewsWire 5, and only use NetNewsWire 3 for a handful of those geeky script-based RSS feeds I am still relying on.

I was grateful to have the opportunity after all these years to take a peek at the source code to the app, and to get a feel for what it would take to salvage what’s left. I couldn’t resist fixing at least one bug before I passed it along though:

Screenshot of the NetNewsWire window with Clippings folder icon restored to normal size

If you’re curious: the Clippings icon is obtained from the Mac operating system. At one point in history it must have come from the system at just the perfect size to fit the source list in the app, but as Apple modernized and adapted to higher resolution Macs, they must have updated the icon to support drawing at much larger sizes. NetNewsWire 3.3.2 doesn’t manually set the size to the expected 16x16pt size, but 3.3.3j (for Jalkut!) does.

Goodbye, NetNewsWire 3. You were a great app, but your time has passed. Long live NetNewsWire 5.

More on Mojave’s Automation Sandbox

I wrote last month about macOS Mojave’s restrictions on automation, and how users can reset the database that controls them. In that post, I cited Felix Schwarz’s excellent article on the subject.

In recent weeks, Apple has made changes to the behavior of macOS Mojave, and added some API calls to help developers better handle the restrictions of the system. Felix is back with an updated post, describing the changes, and what he thinks can still be improved.