Technical writing is too important to leave to language models

[Note from Randy: Hi everyone! This guest post from Elliot will be the last guest post for a while. If you're interested in writing a guest post, feel free to contact me. Otherwise, see everyone next week!]

There are no shortcuts to writing something worth reading

I started my technical writing career during what most would consider the most inauspicious time in history. Days after OpenAI dropped ChatGPT onto the world in November 2022, I signed a freelancing technical writing contract with my first enterprise client. I was astounded at how much I could bill per hour, a significant jump over anything else I had made in my still early tech career. I wondered how long this windfall would even last before I lost my job to a $20 USD/month LLM subscription, which I avoided even testing out for fear of learning how exactly it could outcompete me.

I didn’t worry too much at the time as I saw the gig, and others I eventually took on, as a stepping stone towards learning data engineering while I trudged through graduate school in the UK. To write about something well, you have to know a bit about what you’re talking about—and I’ve had to learn many new technical things in each article I’ve written since. And I really wanted to move away from data analytics and into becoming a data engineer or analytics engineer.

As these things go, a year passed and I had somehow obtained enough different clients to freelance full-time. With each new client, I raised my rate a little more and no one blinked. Eventually, I received an offer to go in-house as an actual, legit, full-time Technical Writer™ in enterprise data engineering. So I took the leap.

In my current role, I split my time pretty evenly between my three responsibilities: overseeing our docs, cranking out marketing content, and internal analytics engineering to keep my skills sharp. (This is a pretty unique mix for a technical writer, and it’s a credit to Datafold for carving out a space for me to have meaningful hands-on coding opportunities.) Being hands-on is essential to good technical writing, and I want to get as close to the developer experience as I can in everything I write. 

While each new announcement about the different LLMs outcompeting each other brought concerns about job security, as a good economist, I have long internalized that prices are signals wrapped in an incentive. There’s a reason why companies still find it hard to hire good technical writers and will pay them quite well to keep the ones they find, despite the proliferation of much cheaper LLMs. In mulling over the puzzle of my continued employment, I’ve since learnt enough about developer marketing and technical writing, overlapping but still distinct spaces, to understand why technical writers add more value than an LLM can or ever will. 

FWIW, I have not had to persuade clients of my utility. One said they felt a visceral “allergic” reaction when they spotted ChatGPT in marketing. So what exactly differentiates good technical writing from LLMs? The best writing that moves me to read from top to bottom has these components:  it’s opinionated, surfaces “ghost knowledge”, and makes the underlying context explicit.  

Experts have opinions

We don’t just want to learn how to do something. We want to know if it’s the better, or worse, way of doing it, and why. 

This is why content built around “best practices” performs so well. Knowing which path to take, and when to take it, is hard won through experience. LLMs can only list paths, and they’ll list as many as you want them to. 

It’s still really hard to create resources that truly add value. I know because it's something I grapple with each day at work. Technical writers must lead from their own experience, or borrow someone else’s through interviews. And there are no shortcuts to obtaining firsthand knowledge. 

The interesting thing about this is that your opinions don’t have to be right to have an impact. What matters is whether it’s authentic–and developers have finely trained bullshit detectors. Just the act of making the case for your ideas produces professional trust in that you, too, are in the trenches trying to chip away at a shared problem, and not some growth hacker trying to SEO optimize another tutorial-listicle.   

When done poorly, say if you outsource your opinion to an LLM, this comes off as generic thought leadership posts that everyone hates reading on LinkedIn, and chips away at brand credibility faster than you might realize. 

Surfacing ghost knowledge

Vicki Boykis first wrote about ghost or implicit knowledge in data, which is how a lot of useful technical knowledge is only accessible by interacting directly with experts. A good example is learning through pair-programming: the times I’ve been fortunate enough to watch an engineer at work via screen-sharing, or have them watch me fumble through a task, led to enormous improvements in my workflows. 

You not only get to see them apply knowledge new to you, but their mental models for approaching a problem and how they debug errors become yours. Learning this way has saved me hours of frustration trawling through Stack Overflow or the docs. I have not found textbooks or courses to be anywhere good substitutes for learning implicit knowledge.

In practice, this type of learning remains expensive since the developer’s time is better spent working on the product. It's also not accessible to most people, since you might be shy to ask someone to screen share what they're working on, and I can imagine that developers feel similarly anxious about others watch them make mistakes in public.

This is where good technical writing can help, because it scales much better than one-on-one learning throughout a company or a community of learners. 

There are practitioners who are slowly making this kind of knowledge more accessible in their domains: Simon Späti on data engineering, Laszlo Sragner’s Deliberate Machine Learning, Eugene Yan on machine learning systems, Chip Huyen on MLOps, Shreya Shankar on ML engineering, Randy Au on quant UX (his recent PyData talk on the tradeoffs between “doing the work and thinking about the work” is a great example of making mental models explicit), and of course, Vicki Boykis on data science. 

One of my favourite newsletters, Justin Gage’s Technically, offers engaging pieces on software engineering concepts. While the articles are geared at helping non-technical folks like designers and product managers understand things like APIs and microservices, they’re rigorous enough to have attracted a following among developers too. 

When I worked at Towards Data Science, I pitched several new columns, including the Notes from Industry, to flag content written by data science practitioners that talked about what data science actually looked like in practice. 

It’s not just blogs; I used to watch Mark Saroufim code through learning new concepts, and was a fan of a game-show style data science competition called SLICED where you could see data professionals work on their solutions live (Season 1 still lives here!). SLICED’s co-founder, Nick Wan, Director of Analytics for the Cincinnati Reds, continues to provide ways to learn ghost knowledge through his livestreams, where he talks through his thought processes as he codes solutions to Kaggle competitions. 

On the enterprise marketing side of things, Tim Castillo’s blog of how Dagster uses their own orchestration tool to run their internal analytics is a great example of teaching by showcasing ghost knowledge. (Disclosure: Dagster is a former client.) Tim drills down to details such as:

At Dagster Labs, we've found ourselves constantly writing logic to decide what database or S3 bucket to read and write from. This is why we have a set of helpers that not only figure out which environment Dagster is currently running in, but also what to do if in a specific environment.

One of my 2024 goals is to get more opportunities to learn directly from experts, whether through pair-programming sessions, code reviews, or collaborating on projects to absorb new and better ways of doing things. It’s clear that good technical writing (or livestreams, for the more courageous) remains an underrated resource that more of us in tech should consider contributing to. 

Context is scarce

Many tutorials are standalone and laser focused on achieving a very specific and narrow outcome. This is good because it makes learning new things straightforward, but really good tutorials consider how people learn, and anticipate questions or problems.

People don’t always follow a linear path when learning. We reach step three, find that a concept or tool is new, open a browser window to learn more, and then try to implement a slightly different version based on what we find to be better for our use case. 

The shorter the tutorial relative to the complexity of material presented, the worse the learning experience. For example, I’ve been working through this “GitHub Actions in 15 minute tops” tutorial from one of my favorite practitioner-led newsletters. It’s heavy on code snippets and light on explanations. I found myself asking why a lot, and considering whether each suggested tool was a best practice or if I could substitute another tool instead. In the end, I abandoned the tutorial, cobbled together my own learning path through painful trial-and-error, and wrote a more complete beginner’s guide to GitHub Actions for a dbt project in a recent post for Datafold. 

An example: SQLFluff & CI/CD

Let’s look at an example that I think beautifully ties the three points together. Recently, while trying to teach myself CI/CD by building my first GitHub Actions workflow, I ran into SQLFluff’s Github Actions README. SQLFluff is a linter (enforces code format consistency, catches errors etc), and the README had this very interesting bit:

This was more useful than a lot of the tutorials I encountered on GitHub Actions. The author took the trouble to explain the overarching concept because they knew:

• from experience (point #1)
• that it was important to make ghost knowledge accessible (point 2)
• and expanded on the repo’s purpose, which was to make it easy to setup SQLFluff in GitHub Actions via various quickstart templates, to discuss important context (point 3) about “the power of workflows in GitHub Actions” 

Towards technical writing excellence 

I hope instead to have refocused your attention on writing opportunities in abundant supply. Questions about how ChatGPT might do away with technical writing (or any other job) are certainly valid but greatly premature. Having studied the role of technological change in economic history, I wouldn’t bet on either the AI hype rhetoric or the status quo–there are too many intervening factors, known and unknown, that will shape the long-run outcome. Just ask any economic historian what they think caused the Great Divergence. 

If you’ve felt discouraged by LLMs to start your own blog or newsletter, good technical writing remains scarce. We could always use more thoughtful (and tested) tutorials, empathetic marketing copy, and documentation that speaks to a broader range of skill levels. 

I’m curious to hear from you: did this resonate? Do you disagree? My LinkedIn DMs are open, and I would love to hear about how data folks are thinking about LLMs–where it’s helpful, where it’s lacking, and where it surfaces interesting areas to work on next.

I have found it easier to write stuff with a collaborator – and I have lots of ideas on things to write about in data. If you're interested in co-writing something with me, please reach out! My email is [email protected] and I’m happy to continue the conversation there too. 

Elliot is a technical writer at Datafold where he recently helped launch the Data Quality Guide, a fun medieval-themed take on data engineering challenges. He previously worked at Plotly and Towards Data Science.