People who dared to measure food energy

Thanks to so, so many of you who reached out with kind words and offered connections, leads on new positions, or even just wanting a fun chat to distract me from the grind. I've got some conversations going, but I'm still looking for work, so please keep sending leads for senior-y data research/analyst/eng type roles if you have them! But luckily life is calming down from the initial shock so I'll be writing more normally from here on out!

To no one's surprise, I was a weird kid in school and still have had odd little questions bouncing around in my head from those ancient days. Today's post is me finally chasing after one of those stray loose ends – what is going on with the "Calorie" unit we use in food.

So, in basic chemistry and maybe even biology class, we learn about how food energy is often quoted in "food calories", which is the amount of heat needed to raise the temperature of one liter of water by one degree Celsius. This "large calorie", sometimes referred to as the kilocalorie (kcal) is 1000x bigger than the "small calorie" which is the amount of heat needed to raise the temperature of 1 ml of water by one degree Celsius. I remember from science class that we were supposed to use the capitalized Calorie when referring to the bigger unit, but its usage is apparently inconsistent.

Overall, the calorie is an obsolete unit that isn't part of the official SI, with the kcal being equivalent to 4.184 kJ. Since the units are just a scalar factor apart, it's not a huge deal which particular unit is in use, so long as users are familiar with the unit.

But the definition of the energy value of the food calorie isn't what confused me as a kid. The confusing part was when we curiously asked how did we come to the conclusion that a carbohydrates and proteins were worth 4 kcal/g, and fats 9 kcal/g. The answer was that scientists started by burning food up in a bomb calorimeter and then "did studies" to somehow figure out how much energy those classes of molecules were worth. They never went into detail, and so I always just pictured scientists burning foods to heat water to get a big average and somehow our entire food energy system just rested on that fact. Even as a kid, the fact that everything seemed to rest on a clearly over-estimated value of energy (after all, our bodies can't be as efficient as pure combustion of food, right?).

Either way, we owe our current system of figuring out the energy content of food to Wilbur Olin Atwater and his original method of figuring out how much energy was available in food. The Atwater system, in modified form, is still used as the basic underpinning of food energy calculations because as messy as it is, no one's really come up with a better alternative.

But first, bomb calorimeters

Let's start from the beginning. The basic theory behind Atwater's system was that the total metabolizable energy of food is equal to the gross energy in the food minus energy lost through excretions: poop, urine, gases, and other bodily secretions.

So the first step to making the formula work is to figure out the gross energy available in the food. And apparently that means "burn the food". On some level it makes sense because there is no way our bodies are going to be able to extract more energy out of a piece of bread than actively burning it in an oxygen-rich atmosphere until it becomes ash. Our bodies will oxidize food for energy similar to what fire would do, but in a much more controlled manner and to less complete end products. Burning gives us a firm upper bound on an energy process that we know little about.

The design of bomb calorimeters used to make these combustion measurements is pretty interesting. The basic idea is simple enough, you take a reinforced sealed container, put your sample in there and pressurize the container with pure oxygen so the combustion will go to completion. The whole thing is submerged under a known amount of water and the reaction is triggered using electrodes leading into the calorimeter. The combustion chamber should warm based on the amount of energy released by the combustion based on the heat capacity of the bomb, and that heats up the water surrounding the container. The devil is in the details because you need to account for every scrap of energy that is being put into the calorimeter in order to isolate just the energy content of the food being combusted. That includes the tiny bit of electricity used to start the combustion in the chamber as well as the energy contributed by the small amount of fuse wire that combusts within the chamber as the reaction happens. There's quite a bit of accounting and calibration involved.

But surely, combustion != energy we get from food

Not even Atwater himself was crazy enough to believe that we got the same amount of energy from food as we could get from burning it completely down to ash. There must be inefficiencies involved. The question was how do scientists account for all the complexity of biological processes, variations in the food we regularly eat, and even the difficulty in measuring how humans digest food and use energy.

So to get an understanding of this, I looked up this paper Energy Values of Food ... basis and derivation by Annabel Merrill and Bernice Watt from 1973 for the United States Department of Agriculture. It goes into great detail first explaining the original Atwater system and the lengthy series of experiments and studies that were used to figure out the original food energy values. Then it goes on to show a wide range of research on individual foods, their composition, and estimated energy content to improve upon the base system. Much of that data didn't exist at the time Atwater did his work.

To start with deriving how much energy we get from our food, we need the concept of "digestibility" for a given food. Basically, it is how much of the food energy-wise can be absorbed and used by the body. This plays a major role in things like plant-based foods like grains where cellulose and dietary fiber is a major component of the mass of the food but our bodies do not digest it for energy.

Apparently a lot of the original studies on digestibility involved measuring the nitrogen content of food going in via proteins, and the nitrogen content of feces coming out. Poop has a very complicated mix of not just undigested food (which is apparently a very small component), but also bacteria, fiber, dead cells from the digestive tract, etc.. Honestly measurements of digestibility are approximations given the practical difficulties of measuring what is going on in the process. But the scientists pressed on and came to some values.

The table below is a tabulation of various foods and the factors used to calculate their energy values. The values were either painstakingly derived from research done on specific foods, typically grain and common meats. Other times, the values are estimates or interpolations based on best available data – for example, butter and margarine are mostly fat but carry trace amounts of sugar and protein and those trace energy values are copied from the milk that the butter is churned from instead of being directly measured experimentally.

In the table below, the first data column shows digestibility coefficients and you can see for meats and fats, digestibility for the protein component of a food is typically 97%, but for various grains the value varies wildly between 20% for whole sorghum meal, to 86% for spaghetti. Fat digestibility is assumed to be 90%, while the carbohydrate digestibility can vary from 56% for pure wheat bran and 98% for rolled oatmeal.

Honestly, the important thing to note here is that the heat of combustion for many foods tends to be in a fairly tight range because Atwater had taken typical values from a bunch of samples. Elsewhere in the paper, there's more detailed analysis of the energy value and digestibility of various foods that show more of the variance for protein or fats derived from difference sources like beef vs pork vs poultry, but on average the variation isn't huge.

Leaning into accepting error

If there's anything to learn about this whole process is that food is complicated. There are so many variables involved that bring unique measurement challenges, not all of which are surmountable. So even when you have all sorts of refined instruments and deep studies on specific ingredients, and everything is studied to two decimal places or better, all that accuracy goes out the window because a normal human will be completely unable to give an accurate record of how much food they ate in a given day. They're going to be off by a non-trivial amount that's going to have a material affect on any energy calculation you run. It's important to get as much accuracy as possible, but despite those efforts the end results are still really approximate thanks to the errors around other terms.

So yes, we largely make assumptions that our bodies are very efficient about our food digestion and can derive the majority of the energy that is available from burning the food completely. Yes, after factoring in our (in)ability to digest the content of the common foods we eat, 4 kcal/g makes sense for carbohydrates and proteins, 9 kcal/g works for fats. Given what a lot of extremely smart and dedicate folk have done in over a century of scientific inquiry, this is the closest answer we have.

And perhaps most importantly, it's not super important on a practical level how accurate ANY of this is. So long as all foods are measured in a similar, reproducible way, so that their energy content is reliably published for consumers to see, then all a consumer needs to know is roughly how many Calories they're supposed to be eating to reach their energy and weight requirements. Most of us go through life looking at food labels and having no idea how to process the obscenely high caloric content of fast food, but we sorta stumble around through life and guess that "I probably shouldn't be eating 5000 Calories today". The whole conversation could've been with any arbitrary number and units and my understanding of the situation would not have improved one bit.

The many scientists who worked on this, doing tons of extremely complex experiments did their best to make an inherently messy concept tractable. We're lucky that their findings landed on a set of values that is surprisingly easy to remember and apply.

For us normal folks who are tasked with setting metrics and doing esoteric business measurement, it's good for us to know that some problems are just really annoying and hard. Even if you have a ton of grit and backing to do years of complicated validation studies on the problem, you're sometimes left with a giant approximation that requires a dozen asterisks to explain.

But most of us don't have the luxury(??) of having the time to do such massive validation studies. Our giant approximations aren't as grounded, but for some situations, they might not be as ugly as we tend to think.

Standing offer: If you created something and would like me to review or share it w/ the data community — just email me by replying to the newsletter emails.

Guest posts: If you’re interested in writing something, a data-related post to either show off work, share an experience, or want help coming up with a topic, please contact me. You don’t need any special credentials or credibility to do so.

"Data People Writing Stuff" webring: Welcomes anyone with a personal site/blog/newsletter/book/etc that is relevant to the data community.

About this newsletter

I’m Randy Au, Quantitative UX researcher, former data analyst, and general-purpose data and tech nerd. Counting Stuff is a weekly newsletter about the less-than-sexy aspects of data science, UX research and tech. With some excursions into other fun topics.

All photos/drawings used are taken/created by Randy unless otherwise credited.

• randyau.com — homepage, contact info, etc.

Supporting the newsletter

All Tuesday posts to Counting Stuff are always free. The newsletter is self hosted. Support from subscribers is what makes everything possible. If you love the content, consider doing any of the following ways to support the newsletter:

• Consider a paid subscription – the self-hosted server/email infra is 100% funded via subscriptions, get access to the subscriber's area in the top nav of the site too
• Send a one time tip (feel free to change the amount)
• Share posts you like with other people!
• Join the Approaching Significance Discord — where data folk hang out and can talk a bit about data, and a bit about everything else. Randy moderates the discord. We keep a chill vibe.
• Get merch! If shirts and stickers are more your style — There’s a survivorship bias shirt!