Tenday Notes 11 October - 20 October 2023
Every ten days or so, I share a quick digest of what I've been working on and reading. Here's the latest. More in the series here.
Earlier this year I did a bunch of data analysis and visualization work for Possible on a project looking at the effect of SUVs on the average greenhouse gas emissions of the UK's car fleet. The results have now been published in a report titled "Tractor Attack". Here's the summary:
The tale that’s being spun is that policies to reduce petrol and diesel cars will mean that those who can only afford to drive cheap old petrol or diesel cars have to pay, while the wealthiest drive around in electric vehicles.
We crunched the numbers and found this isn’t the case. In fact, when it comes to greenhouse gas emissions, it’s the complete opposite.
Our research finds that, due to the growth in demand for bigger, heavy vehicles, it is the newer and more expensive petrol or diesel cars that are the worst for the climate.
Thus, cheaper, older cars aren’t the biggest problem - it’s the shiny SUVs that we need to worry about (and those come with a price tag much higher than a typical electric vehicle).
Who’s driving those high-carbon vehicles? You guessed it - those living in wealthy London boroughs like Westminster, Kensington & Chelsea, and Hammersmith & Fulham - where one in three new cars bought is now a large SUV, and sales of electric vehicles remain lower than those of new super-heavy-emitting new cars.
The findings were covered in the Guardian and the Independent, but I was proudest of all to see one of the UK's top transport academics say nice things about my charts. Here's one of my fav graphics from the report - really simple, but really effective:
More recently, I've been spending a lot of time with the UK National Travel Survey. This dataset, which has been collected since the 60s, is designed to help governments with long-term policymaking, but it contains a wealth of useful information on every aspect of how and why people get around the UK.
The published summary tables on GOV.UK are helpful, but to really get in deep you need to get access to the raw (anonymised) survey results through the UK Data Service, which publishes them in tsv, SPSS and Stata formats. My last few weeks have involved trying to get those back into an SQL database format, with the tables properly linked, and then trying to get them into a format where my colleagues and I can start querying the data.
That has involved leaning heavily on Datasette - a web-based tool for exploring and analysing data. After much faffing about with various hosting options, I managed to get it working on a Google Cloud Run server which is extremely cheap to run but does take about a minute to load if you haven't accessed it in a little while. It works well for our purposes.
The most painful task, though, has been converting the metadata (the information on what different values in the database mean in the real world). It's provided in rich text files in the download from the UK data service, and I need it in a specific format called "JSON". With hundreds of columns in some of the tables, this was shaping up to be a very tedious task.
Luckily, ChatGPT exists. With a bit of trial and error, I found I was able to teach it how to accept some pasted rich text and turn it into exactly the data format I needed. It was still a bit tedious - it could only handle up to about six or seven columns at a time before freaking out. But so much less tedious than it would have otherwise been. Feels crazy to think that eighteen months ago this would only have been possible by concocting some crazy complicated regex incantation.
Anyway, the upshot of all this is that now I have a lovely database, hosted on the web, where I can run queries across 3.5GB of data and get answers back that can easily be shared with other people. This is going to turbocharge Possible's ability to dig out new and interesting stories about how British people get around, and also our ability to fact-check dodgy claims from politicians. A win all round.
After missing last month due to widespread ill-health, Alli, Gabby, Will and I got back together last week to record another episode of the Elevate Dataviz Show. This one, inspired by Paul Graham, asks: "if you were going to take a break from 'serious' work to work on something just because it would be really interesting, what would you do?"
The answers might be interesting to some of you, but what I think is more useful is our discussion about various strategies for how to make time to actually do that thing that you want to do, alongside your serious work.
You can watch the full episode here, and check out the whole archive of the show here. Thanks to the members of the Elevate Dataviz Learning Community for supporting the show!
I've been having a little trouble with my bass guitar lately. Unlikely literally every other bassist on the planet, who plug their instruments into an amp, I plug mine into my modular synth and play it that way. However, the inputs on my synth are designed to get signals from other synths, and the signals that a guitar or a bass puts out are lower. So I've ended up with a situation where my bass is too quiet, and if I turn it up to the volume I want, it distorts.
I tried a bunch of approaches to solve this, but in the end this week the only remaining solution seemed to be buying my first pedal - a second-hand Darkglass Harmonic Booster. I also reached out to a local acquaintance who knows about these things, who recommended a compressor too - and it just so happened he had one lying around that he wasn't using. I arranged to buy it off him, and he then turned up with three pedals (the compressor, as well as an overdrive, a chorus pedal and even a power supply that he also wasn't using) and insisted that I took the lot. Thanks so much Erik 💚
So now, out of nowhere, I've ended up with a miniature pedalboard. Quite looking forward to playing with them. Once the move is over.
Ah yes, the move. It's seven days away now - this time next week I'll be loading boxes into a van. The very boxes that are surrounding my little desk island. As such, I'm issuing no promises that the next newsletter will arrive on time, but I can guarantee that the next time you hear from me, the words will have been typed in Malmö rather than Helsingborg. Exciting!
Catch you then.
Duncan