It’s been a while since my last update. I wrapped up my sabbatical, did a whole bunch of interviewing, and have since been working on some interesting spreadsheet-related problems at a new job.

Oh, and there’s been a global pandemic on, it’s been weird. Anyway, I kept busy over the last year and a bit:

Switched to Windows

I switched to Windows full time, after a lifetime of Macs for personal use and a decade of Windows for professional use.

This was an experiment in intentionally shaping my own interests; I knew that if I used Windows at home I’d be more interested in the minutia of making good Windows applications.

It worked; I’m just over a year in, and I’ve learned a lot. I still miss a lot of things about macOS but I’m mostly happy on Windows. The Surface devices are really neat pieces of hardware.

ReiTunes

I built my own music player. This was the culmination of something I’ve been thinking about for years; I listen to a lot of long-form music and existing music services don’t serve that niche well. iTunes is progressively less interested in catering to users who bring their own music, and the Windows client has many irritating bugs.

So I decided to build my own music library system with an offline-first native client; how hard could it be? Well, 500+ commits later…

Robin Sloan’s post about “home-cooked” code has been making the rounds, and for good reason. It’s about highly personalized programs; when you build something purely for yourself or a small audience, it’s more like home cooking than industrial production. I really love the analogy, and it’s inspired me to write about one of my own “home-cooked” tools: wat.

The Problem

I spend a lot of my day in a terminal, and command line interfaces have notoriously poor discoverability. It’s easy to remember the handful of commands+flags you use on a regular basis; it’s much harder to remember the ones you only use a couple times a year. So, like many people, I used to spend a lot of time digging through man pages when 1) I know what I want to accomplish 2) I can’t remember the exact CLI syntax to make it happen.

Unfortunately, man pages usually aren’t designed for this. The average man page is more like a lengthy novel than Cliff’s Notes; it’s thorough+exhaustive documentation when I just want a quick reference.

There are a few excellent projects that provide succinct documentation for *nix commands: tldr and cht.sh are my favourites. But they don’t provide good facilities for adding personalized documentation of my own; that’s where wat comes in.

What Wat Does

wat started out as a simple wrapper around tldr and cht.sh. The two provide similar content, but they do so in different ways; tldr is installed locally, cht.sh is usually accessed through a dead-simple web interface: curl cht.sh/command. The first iteration of wat was just a simple fish function that would call tldr if it’s installed, and otherwise fall back on cht.sh.

These days, wat is written in Python and it’s a tiny bit more sophisticated: it also displays custom Markdown notes which are synced across all my machines using Git.

Say I want to get some information about a video file, but I can’t remember the exact ffprobe syntax. I just type wat ffprobe in my terminal:

This returns a handy tldr/cht.sh summary and it renders the contents of ~/notes/ffprobe.md if it exists. In this case, it looks like I got annoyed enough by ffprobe to write down a note for the -hide_banner flag.

I’ve been maintaining command line notes for a long time, in various tools that just happened to be at hand: OneNote, Notes.app, Markdown in a repo… but wat lets me organize that information better, and view it inline with excellent community-provided documentation.

You might have already seen Jens von Bergmann’s interactive map of Vancouver which integrates a ton of different data sets. It’s really useful for looking up zoning, year of construction, tax data, and even the estimated amount of floor space in buildings. I find it essential for understanding the city.

It has also has a great feature that many people don’t know about: the ability to filter data for better visualization through the use of a query parameter in the URL. With Jens’s permission, I’ve compiled a list of ways to use this.

Basics

Say you want to see the impact of Vancouver’s early building boom, circa 1905-1914. Just add a years_1905_1914 filter to the URL like so: https://mountainmath.ca/map/assessment?filter=[years_1905_1914]:

And those are just the buildings still left standing! It’s pretty wild how Mount Pleasant, Grandview-Woodland, and Shaughnessy all sprung into existence in a decade.

If you wanted to see all buildings with an estimated floor space ratio (FSR) between 0 and 1, add a fsr_0.0_1.0 filter: https://mountainmath.ca/map/assessment?filter=[fsr_0.0_1.0]

Filters can easily be combined with a comma. If you wanted buildings that were built between 1900-1940 and are between 0 and 1 FSR, the URL would look like this: https://mountainmath.ca/map/assessment?filter=[fsr_0.0_1.0,years_1900_1940]

Bret Victor’s talk Inventing on Principle (video, transcript) changed the way I think about computing in 2019. Inventing on Principle is partly about Bret’s guiding principle:

“Creators need an immediate connection to what they create. And what I mean by that is when you’re making something, if you make a change, or you make a decision, you need to see the effect of that immediately.”

The Edit-Compile-Run Cycle

Although Bret doesn’t use the term, programmers are deeply familiar with his principle. We’ve all worked with toolchains that introduce significant delay before you can “see” the results of a change, and we know they’re painful. Everyone wants a short edit-compile-run cycle.

But until IoP, I’d assumed that slow cycles wouldn’t materially change the output – you’d eventually get to the same place. This was wrong. I also didn’t appreciate the very small time scales involved; a 5 second delay used to seem trivial to me, but it’s still meaningfully different from a response time measured in milliseconds.

Through some very impressive custom tools, Bret shows how immediate feedback enables exploration, which then gives birth to ideas which would otherwise never see the light of day. This was an epiphany for me. Since IoP I’ve constantly been looking for better ways to code, and re-evaluating my existing processes for shorter feedback cycles. The results:

Rust

My typical Rust development workflow goes something like this:

1. Write a small function that does roughly what I want
2. Write a small unit test inline to exercise the function (even if it’s a private function)
3. Iterate using cargo test until the function is correct
4. Later, “productionize” the tests if necessary

Rust’s native support for inline unit tests helps a lot here, and the excellent type system catches a lot of issues before I even run cargo test. On the other hand, Rust’s compiler is notoriously slow and that extends to IDE tooling that depends on the Rust Language Server. I’m looking forward to Cranelift for faster debug builds.

I’ve been busy for the last month, and I completely forgot to update the blog. In no particular order, here’s what’s been occupying my time lately:

Coworking

I joined a coworking space in East Vancouver with a friend, and I’m working from there 3-4 days/week. It’s like an airy spacious café with fast internet and quiet space, I love it. I find that getting out of the house helps me be more disciplined with my working hours; I’m much less likely to disappear down a Wikipedia+YouTube rabbit hole at the office. And then when I inevitably do that at home, I feel less guilty about it because I’ve accomplished so much at the office.

Node.js

I got really into Node and the modern back-end JS ecosystem.

javascript.info has been remarkably helpful (I can finally remember exactly what a closure is!), it might be the first .info website that is genuinely a great source of information.

This talk by Franziska Hinkelmann on the V8 team is a great overview of JS engine internals.

I’ve been pleasantly surprised by Node APIs a few times; the Cluster module makes it trivial to fork workers and take advantage of multiple cores, and Buffer+friends make low-level bit manipulation quite pleasant. Which leads me to:

Cryptopals

The Cryptopals Crypto Challenges by Matasano Security are a delightful introduction to practical cryptography. I decided to do them in Node to get more practice with back-end JS, my solutions are on GitHub. The problems are very well-designed. They’re small enough that you can (usually) do each one in a single sitting, and it feels great every time you decode a ciphertext.

Raspberry Pi

I bought a Raspberry Pi 4 and have been loving it. For some reason I always thought of the Pi as only relevant for education and hardware hackers, but I was wrong – it’s a remarkably capable little Linux machine. I’m currently turning mine into a private Dropbox clone using ownCloud.

Anki

I started using Anki flashcards. I have decks for Node internals, uncommon JS syntax, and infrequently-used keyboard shortcuts and CLI options. Creating cards is a bit of an initial investment, but once that’s done I find that 5 minutes of daily study is enough.