The much broader world of datetime formats
You know I seemingly can't go too long without finding some way to write about time.
Someone on the internet did something that I was always curious about but never had the mental stamina to sit down and do – compare the differences between different datetime display standards.
Specifically, this person built a site (with corresponding GitHub repo) that compares ISO 8601-1:2019, the most recent version of the ISO standard on datetimes we all love, and RFC 3339 the proposal from 2002 on how the internet community should format dates and times. ALSO, as a kind of bonus feature, the site includes a checkbox that activates the "HTML living standard" that shows the recommended guidelines for showing datetimes in HTML that mostly follow the standards.
Note, before we go into the weeds, as far as I can tell the site uses its own simplified time formatting syntax. It's largely similar to the familiar ones from JavaScript or Python (which often based on C's strftime() formatting strings). But there's simplifications like %O is used for ordinal dates, which would be %j in Python for Julian date. Various 0-padded/unpadded variations that are also omitted. For simplicity I'll copy/paste the format used on the site itself for the examples here but your specific programming language will absolutely differ.
While everyone that works with data know that "date and time notations are NOT simple", even I was very surprised at just how many valid formats existed. Many are subtle variations of each other, like for example displaying substrings like dates, or years, hours, etc.. But there's features for expressing time periods and repetition that I didn't even know existed.
%Y-%M-%DT%h:%m:%.3s => 2024-08-18T13:54:57
To start, we have the most commonly seen formats of date and time put together. The standards allow going down to microseconds (or even finer if you have add more precision to the last term). Note that you can put a capital, or lowercase T or a space in between the date and time and it's still valid. Some older standards allowed for having a space instead of a T.
%Y-%M-%DT%h:%m:%.3s%Z:%z => 2024-08-18T13:54:57.427-04:00
You can also put microseconds (up to 3 digits) after seconds using a period OR a comma. The period/comma is apparently used to express a decimal fraction of the least significant time unit (which will come up again later).
%h:%m:%.3s%Z:%z => 13:53:40.427-04:00
Then you can include the time zone offset notation that many of us rarely use but vaguely know of. A -04:00 tacked to the end of a timestamp means it's 4 hours behind UTC. If there's a Z at the end that's supposed to mean the time is from UTC.
So far, these are relatively familiar strings of time, albeit verbose and somewhat hard on the eyes. So let's get weird.
%V-W%W-%wT%h:%.1m => 2024-W33-7T14:00.7
WEEK format? With the W character, you can denote the week and weekday (and include a time if you want with the T symbol). In this scheme, Sundays are weekday #7. Using this format though means you're using a week-based calendar that can have 52 or 53 weeks in a year. That means you'd actually have to understand the rules for numbering the first and last weeks of the year. It's confusing because the first week of a year is defined as the one that has at least 4 or more days of the year in it. So Week 1, Day 1 can actually mean December 29th of the previous year if January 1 lands on Thurdsay (Thu+Fri+Sat+Sun is 4 days making that week the first week of the year).
I keep trying to find out who uses the ISO week numbering system but the best I can find is that working with week numbers is important in Europe and international logistics. But there's disagreement how the weeks are numbered. I sounds complicated and rooted in history. If someone knows more please inform me!
%Y-%OT%h:%m:%sZ => 2024-231T18:03:13Z
Hate weeks or simply can't make sense of those rules? That's fine, you can always use the Julian date which just counts days from Jan 1 instead. It's a pain for us humans to understand, but it's so much easier to work with such dates in computers. I've heard astronomers also use it.
Then on top of all these variations, there's a whole RAFT of repetitive accepted formats where the separating hyphens, colons, etc. between terms are omitted. Except the period/comma before the last significant time unit. That one stays and you can honestly apply it to anything like days, hours, minutes. Since the ISO standard is about data interchange standards, it's considered fine saving a few bytes and using fixed width fields that are machine readable.
P1DT1H => Period of 1 day, 1 hour
But there are also entire chunks of the standard that I wasn't even aware of, like all the values like P1D, P1DT1H1M – these denote "Period" (time periods, as in durations) of time. These are mostly just string statements that tell receiving computers the relevant interval information.
%Y-%M-%D/P1M => 2024-08-18/P1M == Period STARTS at 2024-08-18 for 1 Month
P1M/%Y-%M-%D => P1M/2024-08-18 == Period ENDS at 2024-08-18 after 1 Month
But once you have durations, you can express time ranges, which are more useful. These are effectively a datetime entry combined with a period. What's interesting is that you can denote the start or the end of a period depending on the ordering of components.
R10/%Y-%M-%D/P1Y => R10/2024-08-18/P1Y == Repeat period P1Y (1 year) 10x starting at 2024-08-18s
Finally, you can make time ranges even more complex by adding repetition. By specifying R<repetitions>/ you can communicate a time range that repeats multiple times. In the back of my mind, I'm sure this sort of format is useful for something, though I struggle to imagine the exact use case.
Phew.
Looking at these, I'm actually somewhat more confused as to what the standards say for the little minutiae of formatting complex things like duration or even simple things like the decimal fraction of the least significant time unit. I sorta itch to reference the standards themselves to see just what is going on there...
So I took a look into buying a copy of the standards. I also learned it would cost a LOT of money – $400 at the ANSI Store, 386 CHF (~$445 at current rates) from the ISO store once you include ISO 8601-1:2019 + amendments, and ISO 8601-2:2019. Subscriber support over past few years has built up a small cache of money that I can use to buy the occasional fun item to write about, but I'm not about to waste a huge chunk of it for a single meme post. We'll have to save such a post for when they have a sale or something.
Standing offer: If you created something and would like me to review or share it w/ the data community — just email me by replying to the newsletter emails.
Guest posts: If you’re interested in writing something a data-related post to either show off work, share an experience, or need help coming up with a topic, please contact me. You don’t need any special credentials or credibility to do so.
About this newsletter
I’m Randy Au, Quantitative UX researcher, former data analyst, and general-purpose data and tech nerd. Counting Stuff is a weekly newsletter about the less-than-sexy aspects of data science, UX research and tech. With some excursions into other fun topics.
All photos/drawings used are taken/created by Randy unless otherwise credited.
- randyau.com — Curated archive of evergreen posts. Under re-construction thanks to *waves at everything
Supporting the newsletter
All Tuesday posts to Counting Stuff are always free. The newsletter is self hosted, so support from subscribers is what makes everything possible. If you love the content, consider doing any of the following ways to support the newsletter:
- Consider a paid subscription – the self-hosted server/email infra is 100% funded via subscriptions
- Share posts you like with other people!
- Join the Approaching Significance Discord — where data folk hang out and can talk a bit about data, and a bit about everything else. Randy moderates the discord. We keep a chill vibe.
- Get merch! If shirts and stickers are more your style — There’s a survivorship bias shirt!
