Passing down data work style

Today I was reflecting on a bunch of project work from this past year and was noting the massive variation between everyone for everything. I'm not just saying my stuff is particularly good or bad relative to anyone else's. I'm simply stating thta everyone's calibrated gauges for what is good enough for a given situation contains a huge variation. One question I always come back to asking myself is how is that knowledge transferred from one generation of workers to the next.

To start, the end result of "when do we think our work is complete" has lots of dimensions to it. How do we learn what kind of work to do in a situation, how we are supposed to do that work, and then finally at what point is our work good enough to declare "complete"? Anyone who has done any sort of project-based deliverable work, even in school, has at least some answers to all these questions. They're the answers we build up over time as we learn the ins and outs of our respective jobs and environments. The combination of all the answers to those questions is what leads us to think our work is done.

My sense is that the most common way people learn their own personal answers to these questions is through imitation and feedback. When we start out doing projects, we look to our parents, teachers, mentors, managers for guidance and feedback. This of course leads to a lot of path dependency – people who study under more demanding people are more likely going to learn to be similarly demanding of their work (for better or worse). People will see everything from the ultra-demanding perfectionist helicopter parents, to the completely uncaring supervisor who's a year from retirement and can't be made to care about anything. Through imitating and absorbing the feedback from those sources, people internalize similar quality bars. This is why you hear of people specifically studying under famous chefs, researchers, craftsmen, to learn not just their techniques but also their work ethics. This is also the reason why people transitioning from academic life with it's stronger emphasis on rigor and thoroughness need a few months to understand that industry work isn't normally done to that same standard.

But surely, environment is not destiny.

As active human agents with free will, we have the ability to see out better examples and role models for ourselves. We have the ability to decide without any prompting that a certain level of work does not meet our own internal expectations. How do we raise our internal quality bar?

The most obvious way is to actively study and imitate the work of people we admire. Just like how tracing artwork early on helps teach your hand muscles how to place lines better, doing analyses that follow templates and patterns of work we admire helps us slowly internalize a lot of subtle details you wouldn't normally notice. Imitation allows you to separate the hard work of executing an idea from the hard work of coming up with a good idea. The two are often intertwined, but it doesn't have to be so. I often credit a lot of my writing skills to spending years doing translations where I'd have to deal with the craft of putting words down effectively without having to come up with interesting words of my own.

In more practical terms, junior members of a team are often encouraged to reference templates made by more senior members of the team or through shadowing. The expectation is that by imitating these model work examples, they can learn what is acceptable without having to have everything spelled out. It also means that senior teammates are expected to create and refine these artifacts to pass on to the next wave of new hires. I've had some recent new teammates actively seek such artifacts out and was recently embarrassed to say that I didn't happen to have example resources of sufficient quality to share on demand.

Another, more passive way, is to just immerse ourselves in the work until we unconsciously figure out what we like and don't like about things. Just like how reading a thousand books will inevitably give you a better sense of good writing versus bad even if you can't articulate exactly why you feel that way. See a hundred board decks with boring line charts going up and to the right and you'll probably have an intuitive sense of why those charts aren't great.

I'm not sure if there's a standard industry practice for "drowning oneself in work until we master it"... most of us are more than happy to leave work separate from our private lives. This is more something for people who find the work fun and interesting in its own right and they do things like... write long newsletter posts about data work.

I can't really think of other alternatives aside from this one active and one passive way of learning how to do work. Going much farther than those ideas and things become more similar to a school than a work setting. And of course, given the pace of work in industry and the relatively tiny size of data teams, such training materials are considered extremely low priority relative to everything else that needs doing.

That's why I inevitably end up coming to the morbid conclusion that data folks have to find motivation to learn a lot of the ropes on their own while seeking advice from more experienced data folks outside. Having peers and folks to ask questions of on social media has been a big boon in regards to giving people who don't have access to any other mentors. This is yet another under-appreciated function of the #dataBS cluster of folks. The only thing I can say is that we're lucky that many of those who are in a position to teach and share experiences with more junior peers are usually very happy to do so. I don't know how this culture came to be, but it's great.