Code review for data (and non-SWE) folks

TopPost Apr 14, 2026

Yikes, this post ballooned to 2300 words in one swoop. TL;DR is, your job as a reviewer is to help the reviewee come up with better code than they initially did. The only way to do that is to approach reviews in a way that makes sure they're willing to listen and engage with what you have to say. And everything you say should be meaningful and beneficial to them.

Last week on our discord, I was mentioning doing some code review stuff at work and somehow we came to the conclusion that I should write about how I approach code reviews in general, hence we have this week's post. A lot of stuff in the software industry is shifting around so now's a good a time as any to write a modern refresher on code reviewing. I don't think I'm suggesting anything particularly novel or outlandish, but there's been shifts at the perimeters.

During our discord discussion, it became very clear that the experience of women giving code reviews is significantly different from what men experience. I'm only writing about my own experiences and preferences here, and while I'm intellectually aware that things are different, I don't have the lived experience to write about it. SO! If anyone who DOES have lived experience and would like to write a guest post about it, let me know! It can be as spicy (or not!) as you want.

My goal here isn't to write out some big manifesto of how I think code reviews should go. Professional SWEs with a lot more grounded opinions have that covered. I'm just out to share how I personally do things. It's not the most refined, nor most battle-tested way by any stretch of the imagination, but it works well enough for what I need.

Code reviewing in 2026 is somewhat different in interesting ways

So for context, at work I was working with an engineering team. They wanted to build a data processing pipeline to do something we in UX also wanted. Most importantly, they had funding and explicit executive orders to build this pipeline. Now, either we in UX could try to scrappily build our own with neither of those things and spark arguments over who wrote the correct pipeline in a high stakes meeting later, or I could team up with that team and help them make the best pipeline possible. Obviously I chose to partner up.

I got a copy of their prototype pipelining code and went to review it. Reviewing code in 2026 is, in some ways, easier than before because LLMs that are good at writing code are actually even better at laying out what existing code is doing in plain language. This is great because even though I'm familiar with Python, I'm not familiar with the call stack, with the libraries and idioms the engineer uses. It used to be that when I had to do a code review, I'd have to painstakingly trace the code line by line (after a long struggle to figure out the initial entry point), and take lots of notes along the way as I slowly made sense of what all the code did. It was hours and hours of work to get to a working mental model of what the hell is going on.

What this translates to is, you can choose to what level do you engage with code. If you want to get down into the nitty gritty of the style, you can. If you want to engage way up at the architectural design, structure, and algorithm choices, you can. You don't have to engage with everything at the same time – meaning there's a lot less cognitive load to juggle in your head while you look through code. This works nicely for me because I love working at the higher levels of figuring out how data flows and how and where we're pulling information out of data. I am significantly less interested (or capable) of nitpicking the fine minutiae of algorithm implementations.

But what about "AI"?

Elephant in the room and all that. If LLMs can translate code into plain language, and LLMs can pretty effectively write code and likely had a hand in generating the code you're reviewing, why bother doing code review instead of automating it at all?

The answer is – because LLMs make all sorts of ridiculous architectural and system design assumptions when working with code. They can spam out a code critique. Those LLM critiques can find interesting bugs and blind spots that developers might not have thought about, it's worth seeing what LLM critiques come out. BUT, most importantly, LLMs have no idea what the hell the collective group of humans that's writing the code is actually trying to do. Even if a developer writes a long essay prompt explaining what they want in minute detail, the ambiguity of natural language means there's always room for (mis)interpretation. This is literally why humans invented formal languages like programming languages to begin with!

I've had multiple loops of this exact conversation:

What is this code doing?
Is it really really doing that? – (Forcing LLMs to directly, explicitly read the relevant code again actually helps sometimes)
This is nothing like what anyone wants or intends, WTF is this?

The software engineer submitting the code has likely done a first pass "hey, LLM, critique this code" at some point in their workflow to kick the tires a bit. They're going to a human because humans are still better at this. So lean hard into your human skills – the ability to understand why we're doing all this work.

I'm here to help you, not block you

My approach to code review pretty much grew up in a vacuum because 1) I was often on a data team of size=1 and 2) data science code can very often be throwaway code, which means we don't care to have it reviewed as much even if we do have teammates.

That said, I've done code reviews of data pipelines and massive SQL queries made by various peers, I've had to help teams as they built data infrastructure and pipelines, and to top it off I've had a lot of years commenting on translations as a checker. I even act as article editor whenever someone writes a guest post to this newsletter. All those experiences melt together because they're all the same thing in my head.

During all this time, I've seen horror stories of people doing terrible code reviews. There's the infamous "Reviewer #2" behavior in academic circles who apparently didn't read the paper, demands irrelevant or redundant work simply because they didn't stop to understand. I've seen people who will nitpick every last detail to fit their specific conception of how code should be written, unwilling to admit that the same problems can be solved in multiple ways and it's not their code. I very consciously try to not be one of those people.

Over time, I've learned that when doing code reviews, if you get into a heated argument with the other person, you've completely failed. If the point is to give someone feedback and have them be receptive to any of it, taking an antagonistic stance doesn't help things move along.

In the end the stance I've settled on is that I'm playing a supporting role. The assumption is that the code is going to go out whether I do my work or not. I'm here to help the code that's going out be better than in the state I get it.

Is this [issue] even worth a comment?

Any time I look at other people's code or analysis, it always feels like I'm observing an alien landscape built out of components I recognize. It's like how everyone has their own idiosyncratic way of writing SQL, everyone's thinking and writing processes are unique to them. Reading other people's code is like peeking into their minds. It's weird. It's uncomfortable. It's hard to understand because nothing is where I expect things to be.

The worst code reviewers I know seem to take offense at this alien landscape and demand things look "how they should". These are the guys, and they're often guys, who seem to have a knack for bringing people to tears or screaming rage. Even when I review other people's work, the thing that jumps out at my attention the quickest are the "things that don't look like what I would've done". This is a clear cognitive bias going on in all of us.

The problem is that many of those things that look different don't matter in the grand scheme of things. It is literally equivalent whether someone uses leading commas or not in their SELECT statements, but some people can fly into flaming rants over it. In many situations, making comments about these details is a complete waste of everyone's time and it's best for everyone well-being if I just shut up and leave it alone.

So here are the questions I ask myself to help determine if something is actually worth commenting on:

What is the author trying to do here? e.g. answer a question, summarize some data, etc
Did they accomplish what they set out to do? e.g. yes/no/sorta
Are they making any assumptions that might not be valid? e.g. upstream data quality
Is there something going on they might not be considering? e.g. how people might be using their output

If I have a clear, articulated doubt to any of these questions, then I might be justified in making a comment about what is going on. The clearer my objections and doubts are, the stronger case I feel I have. This let's me skip over a lot of the times where I see what they're doing, I might not 100% agree with their decision but it's a reasonable decision, and they also executed well on the decision.

As an example, let's say I'm reviewing a code pipeline that is taking data from one system, processing it, and leaving it for another team to pick up. If I see they're aggregating data by local time without any clear reason why, I'd comment with something like "You're aggregating on local time here, so you're inviting daylight savings and other complicated shenanigans. Is this what you want?" If it's actually what they want to do, then at least they know what they're signing up for.

Tone matters a lot

Maybe 75% of my review comments end with a question mark. Something like "I'm seeing this, is it what you're going for?" Because I'm trying to verify one or more of those 4 questions I'm asking myself because it's not clear within the code itself. I'm also signaling that I'm not trying to "pass judgement" on code that I don't fully understand the story of. I almost never "pass judgement" anyways but people who haven't worked with me wouldn't know that.

The times when I'm making statements, they're usually of the informative sort because I think the information may change how they decide to implement their idea. "This upstream data has a 15-minute lag and all will return zero for those points. I don't think your code is accounting for it." "There's this edge case that happens quite often and is going to ruin your day here." That sort of stuff.

Depending on how important a comment is, I might lead with "nitpick:" or "ignore if you want:". I don't bother flagging things as "URGENT!" or similar because if it truly was an urgent issue, my explanation for why I'm flagging something should've made that obvious without my editorializing.

Overall, I do my best to keep the tone friendly and professional. I'm aware that some people seem to think this style of writing is... to put it politely, not assertive. And yeah, that's kind of the point. I'm trying to help someone else's work be even better work, but it's okay if they don't listen to everything I say. My way isn't guaranteed to be 100% better, either. When systems get complex enough every decision is a balance of tradeoffs and it's the author's job to pick which one they think is best. After all, they're the one who is signing their name to the code, not me.

Sometimes it becomes pair-design instead of a review

I usually try to go over my comments in person, at least for the major highlights. Since my comments are often in the form of a question, and they're usually "big questions", they inevitably become conversations that are more efficient to go over together.

Very often, those conversations about the things I find wind up turning into full blown conversations about what "the actual solution" should be. That is to say, it's more of a pair-design exercise. It's not pair-programming because I prefer to work at a more abstract level, so we often don't write code. But we put our heads together and talk about assumptions, constraints, the tech stack, and most importantly our goals. Put all that stuff together and eventually you're going to come up with a clearer idea of what we both want to write. I don't know how the person I'm working with feels, but in general I find them to be fun and productive work sessions. I consider those to be a success more than anything else.

At the least, no one usually comes away from any of these things upset, which is in and of itself a decent outcome.

Subscribe!

While browsing around the past few weeks, I found this fascinating explanation to the algorithms that go into detecting harmful CSAM material while also preserving privacy for everyone else.

Standing offer: If you created something and would like me to review or share it w/ the data community — just email me by replying to the newsletter emails.

Guest posts: If you’re interested in writing something, a data-related post to either show off work, share an experience, or want help coming up with a topic, please contact me. You don’t need any special credentials or credibility to do so.

"Data People Writing Stuff" webring: Welcomes anyone with a personal site/blog/newsletter/book/etc that is relevant to the data community.

I’m Randy Au, Quantitative UX researcher, former data analyst, and general-purpose data and tech nerd. Counting Stuff is a weekly newsletter about the less-than-sexy aspects of data science, UX research and tech. With some excursions into other fun topics.

All photos/drawings used are taken/created by Randy unless otherwise credited.

randyau.com — homepage, contact info, etc.

All Tuesday posts to Counting Stuff are always free. The newsletter is self hosted. Support from subscribers is what makes everything possible. If you love the content, consider doing any of the following ways to support the newsletter:

Consider a paid subscription – the self-hosted server/email infra is 100% funded via subscriptions, get access to the subscriber's area in the top nav of the site too
Send a one time tip (feel free to change the amount)
Share posts you like with other people!
Join the Approaching Significance Discord — where data folk hang out and can talk a bit about data, and a bit about everything else. Randy moderates the discord. We keep a chill vibe.
Get merch! If shirts and stickers are more your style — There’s a survivorship bias shirt!