I’ve been busy for the last month, and I completely forgot to update the blog. In no particular order, here’s what’s been occupying my time lately:

Coworking

I joined a coworking space in East Vancouver with a friend, and I’m working from there 3-4 days/week. It’s like an airy spacious café with fast internet and quiet space, I love it. I find that getting out of the house helps me be more disciplined with my working hours; I’m much less likely to disappear down a Wikipedia+YouTube rabbit hole at the office. And then when I inevitably do that at home, I feel less guilty about it because I’ve accomplished so much at the office.

Node.js

I got really into Node and the modern back-end JS ecosystem.

javascript.info has been remarkably helpful (I can finally remember exactly what a closure is!), it might be the first .info website that is genuinely a great source of information.

This talk by Franziska Hinkelmann on the V8 team is a great overview of JS engine internals.

I’ve been pleasantly surprised by Node APIs a few times; the Cluster module makes it trivial to fork workers and take advantage of multiple cores, and Buffer+friends make low-level bit manipulation quite pleasant. Which leads me to:

Cryptopals

The Cryptopals Crypto Challenges by Matasano Security are a delightful introduction to practical cryptography. I decided to do them in Node to get more practice with back-end JS, my solutions are on GitHub. The problems are very well-designed. They’re small enough that you can (usually) do each one in a single sitting, and it feels great every time you decode a ciphertext.

Raspberry Pi

I bought a Raspberry Pi 4 and have been loving it. For some reason I always thought of the Pi as only relevant for education and hardware hackers, but I was wrong – it’s a remarkably capable little Linux machine. I’m currently turning mine into a private Dropbox clone using ownCloud.

Anki

I started using Anki flashcards. I have decks for Node internals, uncommon JS syntax, and infrequently-used keyboard shortcuts and CLI options. Creating cards is a bit of an initial investment, but once that’s done I find that 5 minutes of daily study is enough.

Rust has a really powerful macro system. You can use it to do great things safely… or you can have fun with it and quickly prototype coding productivity features. I chose the latter.

todo-macro is a procedural Rust macro that stops compiling after a user-specified deadline. It’s like TODO comments, but with teeth. It’s probably best explained with an example:

Using todo-macro

It’s January 1, 2020. I’m working on some Rust code that compiles, but it’s not quite ready to ship.

I want to take a break, but I know myself – I’ll probably forget about the deficiency. I could add a TODO comment, but that depends on me actively searching for TODO comments next time I open the project.

To save me from myself, I add a quick todo macro with a deadline of January 2 (in ISO 8601 format):

// Implement the timeout handling
todo!("2020-01-02")

I’ve recently been building .NET Core back-end services that run on Linux. Linux .NET development is in an interesting place; it’s clearly the future of back-end .NET, but it’s still a little rough around the edges compared to our old friend .NET-on-Windows.

Let’s dive into what you (an experienced Windows .NET developer or a .NET-curious Linux developer) need to know to start building .NET services for Linux. I’ll cover IDEs, service hosting, Linux system calls and more.

Background & Motivation

.NET development for Linux has been possible via Mono since 2004, but it was always a bit… fringe compared to Windows .NET development. That all changed when Microsoft released .NET Core in 2016 as a cross-platform .NET implementation; first-party support from Microsoft is a big deal to most .NET developers.

We’re also in a world where Linux is the lingua franca for back-end development; if you want to do anything involving cloud services, distributed systems, or containerization, Windows is typically an afterthought (if it’s supported at all). I want to skate to where the puck is going (sorry!), and it’s headed toward Linux.

Every now and then, I wonder “Is low-level programming still unpleasant?” Call it a morbid fascination, but I really did enjoy some aspects of working in C and C++; in particular, reasoning about low-level behaviour becomes a lot easier with fewer layers of abstraction on top of the metal. On the other hand: memory management and interoperability are hard, the C/C++ ecosystems have accumulated decades of cruft, and both languages are missing a lot of features that are now par for the course.

It turns out that the short answer is “No, because Rust is great”, but it’s still useful to get a feel for modern C and C++. Here are some projects and tools that I’ve found particularly indispensable for that.

Preface: Why Not Book Learnin'?

There are a lot of books out there that will teach you how to use newer C++ features. Effective Modern C++ is a popular one, but personally, that’s not what I’m looking for. I know that I learn best from hands-on experience, and so what I really wanted was to play with some decent-sized example projects using modern tooling.

CMU’s Research/Educational Databases for C++

I can’t say enough good things about CMU’s Database Group, and specifically professor Andy Pavlo’s work. CMU has 3 (yes, 3!) open source relational databases written in modern C++ on GitHub:

1. BusTub, an educational system written for the Database Systems course
2. Terrier, CMU’s current research database
3. Peloton, CMU’s older research database

BusTub and Terrier use C++17, but Peloton uses C++11. I’d recommend BusTub and Terrier since they’re both under active development. Terrier has excellent documentation for getting up and running on the wiki. They recommend using CLion, a modern C/C++ IDE from JetBrains.

CMU’s even provided some small pieces of work to dip your toes in; the projects for the Database Systems course involve implementing basic functionality in BusTub, and grading scripts for the projects will be available soon.

I recently started experimenting with building tooling for ETL systems. After many years of wrestling with ETL in industry, I had a few questions on my mind:

1. Can we make common data issues quick to resolve?
2. Can we make automated data transformations as easy to work with as spreadsheets?

I think the answer to both questions is yes, but don’t take my word for it – you can try the prototype at etlsheets.netlify.app (double-click on an issue to get started), and view the source code.

Motivation

Importing data at scale is painful. Your data providers will screw up the formatting, systems will experience connectivity issues, and your transformation logic will fail on cases you didn’t expect. What if our tools focused on helping with those failures, instead of assuming the happy path?

Speaking of transformations, how should we write them? Some systems take a code-first approach, which is great for coders and impenetrable for everyone else. Others take a GUI-driven approach, which usually becomes the stuff of nightmares. I think we can do better, by drawing inspiration from a tool that’s found in every office:

Transformations

Here’s what building a new transformation might look like, and here’s what it might look like when that transformation fails to run successfully.