Reflections on building forum software

How has two Thursdays have passed already. Everything's a blur. Thanks, subscribers, for supporting this newsletter's ongoing shenanigans.

This week, I went and threw bsBB into the world, which is the software powering our shiny new official forums. It's intended to be a free space, much like our very fun discord, to talk about things we find interesting in this community. It's official in that It's not going away any time soon so go and make use of it by sharing something you like, either about this newsletter, or just data stuff in general.

Yes, we are a data community at heart because that's what I write about and what y'all signed up for, but as I've mentioned on multiple occasions, we're humans first, data practitioners second. That means that as humans we have tons of just stuff we find interesting and cool and want to share.

So let me make this clear. Anyone who wants to talk about something they're excited about or just in general brings them joy or fascination, I'm more than happy to find and make a space for that kind of sharing on the forum.

Otherwise, just have fun with it. I deliberately built it as a homage to the forums I used and managed in my younger days.

Reflections on building the whole forum thing

bsBB isn't the first time I've coded stuff with a lot of heavy lifting done by AI tools, but it's the first one where I was deliberately working on a pet project that I was going to put out as a public artifact for people to use. So, in some sense, it was "serious business". In another sense, it was still hairbrained weekend project spawned from pure nostalgia for the Web 1.0 days as the literal Web around us crumbles from the flood of AI slop.

There's plenty of people out there extolling the benefits, and detriments on AI-coding out there. As far as I'm concerned, the people who write code for a living can hash those answers out far better than I can.

The interesting part today is the fact that I don't code for a living. In fact, while I can write okay-ish code and go way deep into the weeds of implementation on certain things, I never liked going that deep into data structures and algorithms that is the whole foundation of Computer Science. My interests and strengths has always pulled me much along a wider sweep of domains.

So I always preferred to approach code about one or two levels of abstractions higher, where there's system diagrams and APIs and behavior contracts and problems of coordination, trust, and information. It's why I have the most fun at work working with an engineer to figure out how stuff I want can actually be created. It's also exactly why I struggled my entire life to understand frontend web development past HTML 4 when CSS was just barely becoming a thing. I couldn't keep the sheer volume of css attributes and behaviors and browser-specific nonsense in my head long enough to make sense of it.

And then come these LLM tools that promise, to very varying degrees of success, to write the low level annoying stuff that requires tons of memory and experience to put together. All I have to provide were the very things that I enjoy, the information, planning, design, and vision that lies one step above.

Or at least, that's the promise.

So the things that I do enjoy is that I really can spend my energy doing what I like best, figuring out how a system works in my head, how ideas and functional units play off each other to create a functioning piece of software. Since I spend more of my professional life poking at the edges of where software fails due to user interactions, I believe I'm reasonably good at anticipating places that problems can pop up. These of course make good LLM fodder, since they work best under highly constrained, clearly specified constraints delivered in tight chunks.

But no one can imagine a final product in all its perfection at the very beginning of the design phase of anything. Building something, be it software or furniture, is an iterative process where you find unanticipated obstacles or opportunities along the way. One fascinating thing I'm discovering in this process is that while I'm not learning anything at all in terms of languages and implementation details, I'm learning a lot about the systems that software is part of.

Here's what I mean. For bsBB, the original idea in my head is "What if simplified version of phpBB, but use Bluesky as the user". I know from past history that no one wants yet another account signup, and I don't want to be in the business of storing user data that people eventually want to breach. Even though I didn't have to engage with the ATProto specifications to figure out the exact details to allow users to authenticate, I still learned about the implications of using that as the sole authentication method. For one thing it requires bsBB have a publicly accessible domain as part of the auth process, which causes a ton of local development testing issues. It also has implications on how users on bsBB get their profile data initialized (by pulling their profile data from their Personal Data Sever) – I had to decide what info we would pull, and at what times does that sync happen.

While testing the basic markdown post functionality, I realized that I forgot a bunch of the parts of that markup language because I rarely use them (like the image syntax of ![alt_text](url) . That meant building a popup markup reminder. And if we have that as a feature, then we really should have a popup emoji picker since we could resume much of the popup logic.

Oh we need moderation tools... but then what moderation tools do we want? Bans – permanent or reversible, with or with reasons, who has the power? Moderators – what can/cant they do, where can they wield their power, is it a global role or do we have individual forum mods? Post deletion – do we allow it at all, if so who can delete what and to what extent? What about soft deletes versus hard? What if there's PII we need to nuke from all records?

I absolutely revel in working on these interesting problems, much more so than figuring out the best implementation pattern to use to solve them. For a dinky forum system that might see 1-5 active users on any given day, none of the typical and reasonable measures of "Code quality" even matter beyond "does the code faithfully do what it's supposed to do, no more, no less".

And so, I find the whole process of LLM-assisted code writing to be walking a very interesting tight-rope that stretches my system design and "imagining future issues" muscles a lot.

I've got another little side project that I have running on the side that I'll write about at another day that's showing me some interesting details of systems I've been interested in since a kid, but never stopped to pay attention enough to notice until I started coding a pet project against the interest.

Then, on the flip side, I learn all the horrifically stupid things these LLMs can do. In one experiment, I told Anthropic's Haiku model to go build out a medium complexity feature, uploading, storing, and managing small files for bsBB (for saving the favicon and maybe a few logos and things – basic infrastructure feature). Once the overall design concept was hashed out, I let it go wild while I cooked dinner. I come back and then have a couple of other LLMs, (Sonnet and Gemini) to just audit the code to see what broken things it tried to ship... and the models found all sorts of issues to fix. Then I told the same models to audit the code again – and they found new broken things to flag. It never ends. There's no magical cure-all for this besides unending vigilance (which is something us humans are famously bad at).

I take this experience with me whenever I have to use LLMs to write analysis code faster than I can churn out. Since with analysis code, there's no final guardrail of "did the code actually compile and function as intended", building that sense of "the model's likely to do something stupid by, get ahead of it" is important. While building bsBB was often a lot of spamming "1" to accept stacks of access requests to tooling, I can't get into that same complacent mode for analysis. More time watching lines of commentary scroll by with hands on the ctrl+c in case the LLM decides to use some hairbrained scheme to work around a perceived problem instead of using an actual legitimate method.

So anyways, due to the specific quirks of how I like to play with systems and their connections, I can find ways to make LLMs work to fill in gaps. But it very much exercises different skills – setting guidelines and management instead of just "Getting Stuff Coded". It's a lot like how I went from being the analyst that was building models to having to design up analysis frameworks that someone else would use. If you want to retain those "lower level" skills, you have to use them or else they will rust over.