We're not always "back of house"

Ever take the wrong turn in a hotel or event venue and you find yourself suddenly going from a fancy, cozy, opulent, or ultra-modern space to a maze of cinderblock walled corridors lit by hard fluorescent tubes? As a teenager, I started noticing those spaces and thought they were really odd, like peering into a different dimension that somehow existed in the same building. Why is it that pretty places don't have pretty back areas? Is everything I was experiencing and touching a lie like a TV show or movie set?

Much later, as I slowly developed my hobbies around DIY repairing of stuff, I eventually learned this pattern is everywhere. Watch any teardown video of a slick looking tool, or even a car and you'll find that in the most well-built and user friendly items, a huge amount of design work had been spent making the parts we see, touch, and experience be good. The internal bits that provide the critical functionality of a device may have to contort into extremely impractical shapes to accommodate such usability features. Watch any home repair go on and you will learn the adage of "caulking and paint make me the carpenter I ain't."

I'm sure anyone who has any experience in theater or media production, or the production and maintenance of just about anything physical, has seen and known about this for ages. I just led a relatively sheltered life as primary consumer of goods and services and not a producer, and so it took many decades to click in my head.

But today, I bring this up because this notion of a pretty, usable, friendly interface versus an internal back-end that no one really cares about its implementation is something that we deal with all the time in software. But most of us probably don't think about it in a similar way.

The front and back ends

As data scientists, the majority of us work primarily in the "back office". We write code, software, analyses that are for internal business consumption. Even if we're ostensibly creating "a product" in the form of some machine learning model or something, we are primarily creating systems that can be accessed by others via some relatively ugly interface like an API. "Making things pretty" isn't usually a major consideration for many practitioners (except perhaps the data vis folks!).

In terms of a hypothetical hotel that we're operating, we're running around the concrete corridors fixing the plumbing, washing the laundry, shipping objects around. Rarely do we think we have much part to play in the direct user experience at the front side of the house. Heck, I'd imagine a decent number of us, myself included, picked this career choice because we wouldn't have to actually speak to clients and customers directly.

Well, despite my personal choices in work styles, this pattern of "fancy front of house, rough back of house" comes up so, so, so often across so many completely unrelated contexts that it's gotta be saying something about how we humans organize ourselves in relation to one another. And so, are we missing out on anything by keeping our heads down and doing our backend work?

Resource allocation

So, to get the obvious right out of the way so we can get onto more interesting thoughts, economics plays a large part of this pattern. Given finite resources, it makes the most sense for a business and product to place resources into making features that people are willing pay money for, be it fancy finishes in a hotel room or on a laptop. Since users are not supposed to be able to see into the workings behind the facades, users don't care insofar as the product does what it is supposed to. Most of us don't care whether the wires in the wall is a nightmarish tangle of wires so long as it works and doesn't catch fire.

In a data work context, this is very much the tension we feel in wanting to get our work done quickly and efficiently using common everyday methods, and wanting to use super rigorous, bleeding edge methods. The stakeholders we're providing answers to largely don't care what we use so long as they have trust in us to provide them the most accurate information. We always try to make them care out of pride in our workmanship and work ethics, but we all know it tends to be an uphill battle.

When the facade affects the structure

Life in the back office tends to be pretty chill in that we are usually only accountable to our organization. We aren't usually dealing with irate or confused customers making impossible demands. We get to be the folks writing the data APIs or providing the models that other people will work to make into a product that customers will actually buy. The convention image of a back office data job is one of pushing things out to other people to use.

But the reality of things is that even something as far away from actual customers as possible like data science still receives plenty of effects from the front, customer-facing functions. Those demands and complaints coming from irate customers eventually filter into feedback that we have to incorporate into our work. The feedback from A/B testing our new models will force us to accept that our brilliant ideas didn't work. The product managers complain that our results are unsatisfying and leading them to dead ends. The engineers complain that our API designs are garbage. All this feedback is telling us that our own "interface" needs to be relatively userfriendly as well. We ignore our peers at our own peril.

What's my point? Well, I know many of us take pride in our work and due to our proximity to math and code, we take extra pride in achieving a certain amount of economy and elegance in our work. After all, we may need to scale our work to billions of data points. Sloppy work tends to fall flat there.

But we mustn't forget that we largely work in the ugly, painted-concrete, fluorescent corridors of "the product". Sometimes, we need to make painful, inelegant contortions with our work to get it to fit into what users actually find super useful and compelling. That super expensive table refactoring and merging of fields into a materialized view that breaks every few days for "reasons" just so users can have better display latency? Could be worth it.

Anyways. We of all people should know that the world does not cleave into two mutually exclusive categories of front and back end. Once in a while, even if we're not good at it, we should be considering how the design decisions we make can influence things in the front of the house.

Standing offer: If you created something and would like me to review or share it w/ the data community — just email me by replying to the newsletter emails.

Guest posts: If you’re interested in writing something, a data-related post to either show off work, share an experience, or want help coming up with a topic, please contact me. You don’t need any special credentials or credibility to do so.

"Data People Writing Stuff" webring: Welcomes anyone with a personal site/blog/newsletter/book/etc that is relevant to the data community.

About this newsletter

I’m Randy Au, Quantitative UX researcher, former data analyst, and general-purpose data and tech nerd. Counting Stuff is a weekly newsletter about the less-than-sexy aspects of data science, UX research and tech. With some excursions into other fun topics.

All photos/drawings used are taken/created by Randy unless otherwise credited.

• randyau.com — Curated archive of evergreen posts. Under re-construction thanks to *waves at everything

Supporting the newsletter

All Tuesday posts to Counting Stuff are always free. The newsletter is self hosted. Support from subscribers is what makes everything possible. If you love the content, consider doing any of the following ways to support the newsletter:

• Consider a paid subscription – the self-hosted server/email infra is 100% funded via subscriptions
• Send a one time tip (feel free to change the amount)
• Share posts you like with other people!
• Join the Approaching Significance Discord — where data folk hang out and can talk a bit about data, and a bit about everything else. Randy moderates the discord. We keep a chill vibe.
• Get merch! If shirts and stickers are more your style — There’s a survivorship bias shirt!