It feels a bit early to be writing an update to something I wrote 1.5 months ago, but we live in interesting times. Shortly after writing that post, I started trying out Aider with Claude 3.5 Sonnet. Aider’s an open source Python CLI app that you run inside a Git repo with an OpenAI/Anthropic/whatever API key¹.

My Aider workflow

1. I direct Aider toward a file or multiple files of interest (with /add src/main.rs or similar)
2. I describe a commit-sized piece of work to do in 1 or 2 sentences
3. Aider sends some file contents and my prompt to the LLM and translates the response into a Git commit
4. I skim the commit and leave it as is, tell Aider to tweak it some more, tweak it myself, or /undo it entirely

This works shockingly well; most of the time, Aider+Claude can get it right on the first or second try. This workflow has a few properties that I really like:

1. It’s IDE-agnostic (no need to switch to something like Cursor)
2. It’s very low-friction, which encourages trying things out
   1. No need to copy code from a browser, write commit messages, etc.
   2. Undoing work is trivial (just delete the Git commit or run /undo)
3. It’s pay-as-you-go (I pay Anthropic by the token, no monthly subscription)

Prompts

Here are some examples of the prompts I do in Aider:

• Library updates should be streamed to all connected web clients over a WebSocket. Add an /updates websocket in the Rust code that broadcasts updated LibraryItems to clients (triggered by a successful call to update_handler). The JS in index.html should subscribe to the WebSocket and call table.updateData() to update the Tabulator table
• Add a new endpoint (POST or PUT) for adding new items to the library. It will create a new LibraryItemCreatedEvent, save it to the DB, apply it to the in-memory library, then broadcast the new item over the websocket
• add a nice-looking button that bookmarks the current song. don’t worry about hooking it up to anything just yet
• Add a new “test-api” command to justfile. It should curl the API exposed by add_item_handler and check that the response status code is CREATED
• Write a throwaway C# program for benchmarking the the same SQLite insert as create_item() in lib.rs

I’m still developing an intuition for how to write these, but with all of these examples I got results that were correct or able to be fixed up easily. Sometimes I am very precise about what I want, and sometimes I am not; it all depends on the task at hand and how confident I am that the LLM will do what I’m looking for.

What does all this mean?

I dunno! The world is drowning in long-winded AI thinkpieces, so I’ll spare you another one.

All I know for a fact is that if I have a commit-sized piece of work in mind, there’s a very good chance that Claude+Aider can do it for me in less than a minute — today. I’m still exploring the implications of that, but Jamie Brandon’s Speed Matters post feels very relevant. I can try out more ideas and generally be more ambitious with my software projects, which is very exciting.

I find LLMs to be pretty useful these days. I don’t consider myself to be on the frontier of LLM experimentation, but when I talk to (technical) people it sounds like my workflow is pretty uncommon, so I should probably write about it.

LLM (the command-line tool)

Simon Willison’s llm command-line tool is the primary way I use LLMs. I sometimes struggle to describe the appeal of llm to people because it’s boring. llm lets you do the following with any popular LLM (hosted or local):

1. Ask the LLM one-off questions (optionally taking stdin as context)
2. Start a chat session (optionally starting from the last ad-hoc question)

And that’s about it! It’s one of those lovely tools that does a few things well. I usually start sessions with exploratory questions/requests, sometimes piping in data:

cat xycursor.rs | llm "the end() function in this file is confusing, explain it"

And then if I need to follow up on a question, llm chat --continue drops me into an interactive chat that starts after the last question+response:

> llm chat --continue
Chatting with claude-3-5-sonnet-20240620
Type 'exit' or 'quit' to exit
Type '!multi' to enter multiple lines, then '!end' to finish
> write a comment explaining that function, using ASCII diagrams

Important things about this workflow:

1. It’s trivial to “connect” the LLM to other data+files
   1. For example, every week I used to manually rewrite the output of this script to be more readable before publishing it; now I pipe it to llm and tell it to do an initial rewrite first
2. llm makes it trivial to go from exploratory work to more focused iteration

I have llm set up to use this custom prompt, no matter what underlying LLM it’s using. I find that it helps make responses much more succinct.

GitHub Copilot

It’s good, I use it every day. It’s a lot more widely known than llm so I won’t spill too much ink over it.

Observations

I use LLMs and web search in a similar way: do a quick exploratory investigation into something, taking the initial results with a grain of salt. The skills+knowledge you need to evaluate Google results are very similar to the ones you need to evaluate LLM results!

I mostly use LLMs for computer stuff, and it’s often really easy to verify whether a programming/computing answer is any good; just try it out! LLMs are probably not quite as useful for fields where that’s not the case.

I’m happy with llm but it is, ultimately, a wrapper around a basic chat interface and we can probably do better. Claude Artifacts is very appealing in that it can offer a faster iteration cycle for web development (but is unfortunately coupled to an expensive subscription service), and Aider is interesting as a better way to give an LLM access to context from an entire code base. I’m hoping we’ll see more tools like these that extend what we can do with LLMs.

Been a while since the last update, I’ve been busy with the new job. Things have been going well!

I went to New York in March for my first week at Datadog, which was a good way to start the job; I met other new joiners, met people on my team, saw a bit of the city. And the view from the office is alright:

Datadog operates at a much larger scale than any company I’ve ever worked at, and that has some upsides and downsides (but mostly upsides). I feel lucky to work with a lot of smart, enthusiastic people.

Last week I was in New York again, for Datadog’s annual DASH conference. I was helping run a booth for my team and it was good to talk to users (and potential users) in person. I got to see a bit more of the city outside of work; the Intrepid Museum was a highlight (an aircraft carrier! a Space Shuttle! a submarine!).

Outside of work, I’ve been spending a little bit of time on Nushell (but not as much as I would like). I’ve been driving some changes to the explore interactive pager (which reminds me, I need to update that documentation). I’m trying to get it to a point where I’m happy with it for version 1.0; I’m not quite there yet but I’ve made a lot of changes under the hood.

Nonfiction I read recently

What Goes Around Comes Around… And Around… (⭐⭐⭐⭐) Michael Stonebraker’s back for another opinionated overview of databases, this time with Andy Pavlo. The whole thing is good but I particularly like their take on vector databases:

They are single-purpose DBMSs with indexes to accelerate nearest-neighbor search. RM DBMSs should soon provide native support for these data structures and search methods using their extendable type system that will render such specialized databases unnecessary

On a related note, I like Simon Willison’s point that maybe you don’t need vectors for RAG:

The more time I spend with this RAG pattern (ed: one using full-text search) the more I like it. It’s considerably easier to reason about than RAG using vector search based on embeddings, and can provide high quality results with a relatively simple implementation.

Fiction I read recently

Lonesome Dove (⭐⭐⭐⭐⭐) This was described to me as “the Western novel to read if you don’t normally read Westerns” and yeah, it was great.

Glorious Exploits (⭐⭐⭐⭐) Funny + touching story about 2 unemployed potters in 412 BC who decide to put on a play with imprisoned Athenian soldiers as the cast.

Microserfs (⭐⭐⭐⭐) Strange that I hadn’t read this before, but Douglas Coupland has a not-entirely-positive reputation in his hometown. He’s kinda known as the guy who got too many undeserved public art commissions around here. Anyway! It was a really fun read and it felt like it could have been written yesterday (surprising for a book about the tech industry written nearly 30 years ago).

A bunch of Horatio Hornblower and Richard Bolitho books (⭐⭐⭐) I was looking for something along the same lines as the excellent Aubrey-Maturin series. These weren’t quite it. Hornblower isn’t very fun as a protagonist and Bolitho is a boring one, I couldn’t make it very far into either series.

So, big news: I’m off to a new job at Datadog, working on their CoScreen screen sharing tool 😎. I’m gonna be working remotely from Vancouver, which will be interesting because I’ll be dogfooding CoScreen as I work on it.

It’ll be good to work on native software again. It feels like the vast majority of jobs today operate at a level of abstraction where the OS doesn’t matter much; when you’re writing REST services it’s rare that you actually interact with the OS itself, right? CoScreen is very different in that it’s native software that needs to do lots of interaction with the OS to provide a great UX. Which is exciting and fun and cool, at least until I get bogged down in the details of GDI or whatever 😅.

I reread Robin Sloan’s post about a “home-cooked” app, and it continues to resonate; there’s something special about writing software for very small audiences. In that vein, here’s something that my partner and I have been using since last year:

We needed a solution for shared notes that works well across Android, iOS, and the (desktop) web… so I made one. We use it a lot, mostly for things like our grocery list and cooking notes. It’s simple, fast, and it works for us.

Technology-wise, it’s just a website that works well on desktop and mobile. The back-end is written in Rust, it uses HTMX for some dynamic updates, and the note contents get saved to a SQLite database (backed up to S3 with Litestream). The whole thing compiles down to a single-file executable that gets run on a cheap VPS.

Many corners were cut during the development of this project! Proper collaborative text editing is hard, and I took a quick-and-dirty approach. Eventually I’ll get it working better with Automerge or whatever, but for now I have something good enough for a user base of 2.

And now it’s really easy to do stuff with our shared notes:

That’s a cheap 7" e-ink display with a Raspberry Pi on the back, showing a customized view of our grocery list plus enough information to answer the question “do I need an umbrella today?”. It’s like a little household dashboard that we can take a quick look at as we head out the door. Eventually I need to make a frame for it and mount it on the wall, but it’s pretty handy as-is.

Why do all this?

Shared notes are a solved problem, I’m sure I could have found an existing service for this. But:

• I like having a project to tinker on for someone I care for, in the same way that I like cooking for them
• This is permanent in a way that cloud services are not. It will never change unless we want it to, and it will will work for decades without much effort. We might be using this in a retirement home one day
• It’s trivial to tweak and extend software I wrote myself. I didn’t need to figure out an API to get the the e-ink display, I just built a new HTML view on top of existing data
• Software designed for an audience of 2 is able to be much simpler than software that anticipates the needs of a large+diverse user base

Bonus: Burninator.exe

Now that I’ve shown you something useful, here’s some much sillier home-cooked software:

A friend asked:

What is a program that I can run in the background to raise the temperature of a laptop? If my laptop gets too cold the screen starts to flicker…

I wasn’t aware of any, so I wrote one that queries WMI for the current CPU temperature and then does busy work until the temperature is high enough. Is it dumb? Yes. Does it solve this one person’s very specific problem? Also yes!