Trip to South Korea

We arrived back from South Korea just over a week ago and the trip was everything that we hoped it would be: fantastic food, great locations and stunning scenery. It was also absolutely exhausting. Some random stats and highlights:

  • Miles walked over 13 days: 89
  • Most useful purchase in preparation for the holiday: an eSIM (more on this later)
  • Most unusual food: sea squirt
  • Most raw garlic cloves in a single meal (each): 3
  • Number of men needed to deal with a poolside snake: 6, including one in a safety helmet.
  • Photos taken: around 600
  • Hours playing the new Zelda game: around 25
  • Constant life-saver: Google Live Translate
  • Weirdest breakfast (my wife; not on the same plate and over about an hour): Omelette with avocado and asparagus, sautéed potatoes, Brussels sprouts, rice, mapo tofu, spicy Chinese aubergine, banana milk, rasperry Danish, 2 coffees
  • Hours spent poolside debugging a TLS problem on this site on a terminal app on my phone (5 inch screen): 4. Yes, Certbot, we’re looking at you.

Camera
Barring one cheese-eating-grin LinkedIn portrait I got my wife to take of me at the start of lockdown, I haven’t used my SLR in almost 4 years, so I decided it was time to dust it down. I stumbled on the term for what I wanted: a ‘hybrid’ rucksack which, as the name suggests, is half-and-half: the padded dividers for a modest amount camera equipment, and then unstructured storage for ‘essentials’ like my Switch. I ended up getting a K&F Concept 20L bag. It’s pretty decent, although the main buckles are a bit fiddly. As well as my walkabout 24-105mm, I took my fast 50 which I never used. I thought I might do some tripod based long exposures in and around Seoul but, in all honesty, I could probably have got away with just my phone.

eSIM
Based on our own research, we’d decided that we wanted mobile network access, and had been looking at portable WiFi hotspots and physical SIMs. On the recommendation of one of the advisors from the travel company that we use for all our long haul trips, I bought an eSIM. It’s like the Internet Wild West: you end up looking at aggregation sites that resell dozens of variants. I didn’t go with the company that we were recommended: they had a KYC requirement where you had to upload a photo of your passport. While you get it photocopied or scanned routinely at hotels, there is something distinctly offputting about uploading it to hadn’t-heard-of-us-5-minutes-ago-but-trust-us.com. I ended up getting a data-only 14 day deal from a company called klook.com, who linked through to a company called frewie.com who have, I don’t actually know, some sort of peering arrangement with a South Korean provider called KT – who are big. They had shops everywhere. And I had an IP address geolocated in Hong Kong. Simple as that (!). No KYC passport info required.

I don’t know, or particularly care, how the network stuff bootstraps on a phone, but I get the distinct impression that the (physical) SIM provider gets some room to play in terms of configuration they can apply. The reason I say this is because when I got to the hotel in Seoul, jet-lagged to hell, and tried to set it up, my phone seem to be suggesting that the eSIM was going to be from my UK network provider. The instructions the eSIM company emailed me were wildly different from what I was seeing in the config menus.

As there were a couple of warnings about what I assume are fallback options between providers, I decided that the safest thing to do was to take the physical card out of the phone and then set up the eSIM. To cut a long story short, it worked really well. However, there were a couple of things I didn’t think of beforehand. First, I thought, now it’s set up and working, I’ll just double check that I have the data roaming cap set to zero with my UK company. Their app uses SMS as a secondary authentication factor.

Doh!

A few days in, my bank card was refused in a shop. It was a one-off, but I know that my bank would…

Doh!

Oh well. I’ll know for the next time, and I haven’t been hit with a £500 roaming bill. Yet.

Seoul

We flew with Asiana (thoroughly recommended), and checked into Hotel 28 in Myeongdong. We had enough energy to hit the night market, which was only a couple of blocks away for treats like tteokbokki and mandu.

Tteokbokki
Mandu

Outside the rooms in Hotel 28. Air purifiers I think. Or possibly time machines for shoes

The next day we had an all-day tour with a guide. Note for future reference: we need to cut ourselves some slack and have very little planned for the first couple of days. It was 30C by mid-morning; combined with the jet lag and our guide’s walking speed (fast!), I hit a wall and thought I was going to have to head back to the hotel.

The Palace

Insa-dong

We had most of the next day to ourselves, so hit the shops around Myongdong. I had no idea that South Korea is the centre of the cosmetics universe, so it was happy hunting for my wife. In the evening we had a tour of a large indoor market in Jongno 5-ga. We had a few firsts:

  • De-boned chickens’ feet. Not keen on the texture, but a really nice flavour.
  • Raw crab in chilli, still in the shell. That was a big nope.
  • Bindaetteok, a mung bean pancake. Delicious
  • Hotteok, which is a sweet pancake. Also delicious.
  • Makgeolli. We’d had soju in the UK but I’d never heard of this. It’s a sparkling rice wine, and we were given divergent opinions on how to drink it: shake it up first (there is a lot of expired yeast at the bottom) versus have the clear liquid at the top first and then upend the bottle. It’s like nothing I’ve ever tasted before and really good. Kind of like fizzy sweet and sour milk shake that gets you pissed.
Hotteok

You know when you’re watching a foodie show on Netflix where the host is somewhere far flung, and being shown all of the delicacies by a well-informed Instagrammer or food critic? What never happens is that a local man squeezes onto one of the benches beside them and is so completely and utterly pissed out of his brains – bear in mind it was 7pm on a Monday night – that he looks like he might soil himself, and manages to frighten off everyone in the vicinity. Maybe it happens all the time and the rest is great editing.

Army stew
Difficult to eat noodles and dumplings – but lovely (Myeongdon Kyoja)

Next day we had an early start with a trip to the DMZ. The walk back to ground level from the 3rd infiltration tunnel is one hell of a workout.

Unfit for military service 🙂

Busan
There were a couple of big protests in Seoul during our stay and one was happening on the morning that we were due to get the (zombie free) train to Busan. Combine the resulting road closures with the part of town that Hotel 28 is in which has really narrow streets, and we were starting to worry if we would make the station in time. We did, but had to improvise lunch (gimbap), rather than have a chance to orient ourselves and explore. I’d expected the trains to be something like the Shinkansen: in reality, they are a bit more routine in appearance. Although ‘routine’ is something of a disservice to topping out at 185mph and running to the second. Sorry National Rail, but you have about 75 years of progress to make up.

We didn’t try this…

We stayed in the Lotte Hotel: business oriented and absolutely enormous. It was fine: a bit characterless, but you couldn’t fault it for efficiency. It merges into a huge shopping centre which we managed to get utterly lost in on our first evening. I’m blessed with a sense of direction that you might expect a remedial toddler to have, so things got a little spicy when I started making suggestions to my wife about how to get out of the place. When we did, there was some great exploring to be done in and around Bujeon2-dong: various markets and and food stalls.

Dried fish skin. Probably. With gochujang and mayo. Fabulous with beer

We were there for 3 nights, which included one full day tour with a guide. Unfortunately the weather was pretty awful, but one of the highlights of the holiday was a trip to the fish market. Our guide was fantastic: we wouldn’t have had a clue. You pick your lunch while it’s still swimming, and 20 minutes later, it turns up on the plate. We opted for a sea bream, half sushi, half fried. We had a few seafood sides as well: scallops and prawns, which were cooked and utterly declicious, and sea squirt – raw and, well, one to put down to experience. It wasn’t gross (which the raw crab in Seoul was), but I wouldn’t go out of my way to have it again.

Fish market in Busan
We didn’t try these either…
Sea bream
The same sea bream
Food market in Busan
Food market in Busan
Yep, the same food market in Busan again.
Temple in Busan

Jeju
The boarding of the Korean Airways flight from Busan to Jeju was, for a non Korean speaker, chaotic. It transpired that it was done in groupings (‘zones’) but there was absolutely no indication of what we had to do, where to stand or when. That said, we had the same experience on each of the four airports that we went through in terms of security and baggage processing: jaw dropping efficiency. Actually there was one puzzling process for the uninitiated at the bag drop: you have to stand at a specific point and look at a screen to see if your bag is going to make it through (presumably) an initial X-ray check. We had no idea what we were supposed to do. By the time it had been explained to us, our bags were nowhere to be seen on-screen. We kind of shrugged, quietly shuffled off when the staff lost interest in us, and assumed that we weren’t in any sort of trouble.

This brings us to the Shilla, which was our hotel for the four nights we were on the island. It’s pretty swanky and, it has to be said, that the breakfasts were out of this world. But it had Kafkaesque rules about sitting by the pool that we never fully understood. On the first morning, I was told that I couldn’t stay on one of the free loungers (others you had to pay for, and it was expensive) because I wasn’t wearing swimming gear. Twice, in the space of about 5 minutes. It was also timebound: you had to vacate after 3 hours. We weren’t sure how this was enforced: cattle prods and stopwatches presumably. As the prices were eye-watering, we took a walk to a local shop to buy some snacks and booze. One of the staff at reception clocked me as reptilian for having the gall to walk into the place with a clearly visible bottle of soju in a plastic bag. Just so that we didn’t come across as total cheapskates we ate a very refined, and slightly unsatisfying meal in one of the restaurants.

Not our cup of tea, to put it mildly.

Snake By The Pool. Movie adaptation unlikely
Cutlass fish: a lot tastier than it looks

Lava tube
Fried chicken from BHC. Unbelievably good

Back to Seoul and home
Unfortunately our flight back to Seoul was delayed by about 3 hours. At least we understood what was going on during boarding. Our last hotel was the Four Seasons, where we were upgraded to a suite on the 28th floor. We’ve been lucky enough to stay at some lovely hotels down the years and this was right up there with the best of them. As well as a remote control for the blinds, there was also a remote control for the lid of the toilet. Because, you know, hygiene and effort: it’s the only way to go (in every sense).

I was joking with someone the other day that going to South Korea from the UK is like visiting the future: stuff works, really well. We both loved Seoul: it has the vibe you’d expect of a global economic powerhouse, but not the ‘you fall over, I’ll step over you’ you get in London. We would go back in a heartbeat. All we need to do is work off the calories we deposited while we were there.

Permanent programmatic access to Google Calendar

Following on from this post about setting up an e-Ink screen with calendar based content, what the Google quickstart example doesn’t call out is the significance of this setting in the OAuth Consent screen:

OAuth Consent

If you leave the ‘Publishing status’ set to ‘Testing’, the access and refresh tokens expire after 7 days. Also, the way the out-of-the-box script works, you need to delete the token file before re-running the script, which requires you to re-authenticate.

The process of publishing means that the script is open to the public, and requires a review by Google – clearly not appropriate for this project.

I tried a couple of different approaches to work around this, starting with service account impersonation – i.e., having a service account that impersonates the end user account (mine). While it seems to be possible to do this, you need to use Google Workspace features which I don’t have a subscription for.

The alternative approach that I’ve taken is simply to add the service account to have read only access in the Google Calendar interface:

Add the service account…

I’ve exported the credentials for the service account to a JSON file, and altered the original quickstart example to use it. It’s actually a lot simpler: it should (!) run without any manual intervention at all.

The details of the authentication that the service account is using is hidden by the Google Python library, but my working assumption is that it’s using the private key as a secret, and presenting a token via the Client Credential grant. I’ve added the new version of the quickstart file which uses the service account authentication to my GitHub repo.

Building an E-Ink Calendar Display

A few weeks ago I saw a link to this project on Daring Fireball and it really caught my eye, so I decided to have a go at doing something similar based around the same WaveShare 7.5 inch screen. So over the course of the last few evenings and weekends, what I’ve produced so far is this:

A future in industrial design is not beckoning….

The Enclosure
Before I get into the development, I’ll cover the blindingly obvious: the enclosure is a bit of a mess. Those are soldering iron marks across the bottom of the frame for the screen. Having never tried it before, I thought it would be an interesting sub-project to use some CAD software to design an enclosure and order up a 3D print. It turns out that ‘having a go’ at CAD is about as good an idea as ‘having a go’ at parking an aircraft carrier. Anyway, in the process of doing battle with the software on my iPad I managed to get an internal ‘y’ dimension wrong by a whopping 5mm, and ended up having to carve an extra cavity to slot the screen into. So, to cover that mini crime scene, what was intended as the top of the enclosure is now on the underside and covered with gaffer tape.

Let’s pretend none of this happened and move on…

The Screen
As covered on the original posting, there are significant limitations with this E-Ink screen. I don’t mind about the refresh: as I will go on to explain in more detail, I currently only update it once every 15 minutes. What the original author doesn’t mention and which I found much more restrictive is the 1 bit colour depth. I now know more about typefaces than I ever thought I’d need to in my life. On a ‘normal’ screen and assuming the text is black, most fonts – as a sized, weighted and rendered instance of a typeface – are going to be displayed with anti-aliasing, the shades of grey that smooth out the jagged edges of any non horizontal or vertical lines. No access to those shades means jagged edges are the order of the day. I toyed with using dithering instead, but the results were worse. While you are obviously free to use any typeface you like, the results are going to be variations on the theme of ‘awful’. While I tried a few – I found a few resources suggesting Verdana, but it looked a bit meh – I ended up using a TrueType font that comes with the sample code for the screen.

The TL;DR Version

I have two Python scripts, running on an old Raspberry Pi 3 (which I use for other Philips Hue control related stuff such as this). The first is a lightly modified Google example script which pulls down the next 10 events from a shared calendar. It writes the data out both as a pipe-separated file for some post processing, and also as a HTML table. [Update 17/04/22: all other things being equal, the example script stops working after a week. See this for more info.]

Those post processing steps are performed by a second Python script which also calls a BBC RSS feed, and then eventually renders the various content as a 1 bit bitmap for the screen to display. Both scripts are called from cron: the calendar processing twice a day, and the rendering every 15 minutes. The two main tables alternate in prominence.

Warts And All…

I’ve included some detail on elements of the implementation which I tried and didn’t work. Part of what made the project interesting was figuring out what was viable in terms of the screen’s display capability and, generally, revising downwards.

Starting Point for the Calendar Content

The initial motivation for this was a tongue in cheek attempt to convert my wife from using a wall calendar. What I thought might be an interesting approach would be to use Selenium and chromedriver to authenticate to Google Calendar, screenshot the month, crop it down to size and convert it.

Authentication

It appears that Google may block this. Regardless, I dismissed this out of hand: I’d not thought about the implications of 2FA, so that was a showstopper (just… no, don’t :). What I settled on was the Google calendar API, using this example code from Google. It requires you to configure a Cloud project and then permission an API key for OAuth2.

So: you run the script on the command line, which initiates the Authorisation Code grant flow, printing a url to standard out. You paste this into your browser of choice – I copied this from a Putty window into Chrome on my PC – and authenticate with Google. It then redirects you to a url on localhost (not running, in my case). I copied this url from the browser, put it in quotes to avoid the shell interpreting the ampersands on the path, prefix it with the wget command and then paste it back into a separate ssh window. This then populates a json file with an access and refresh token which will be used without the browser interaction going forward.

It sounds convoluted but it’s actually straightforward in practice, and it’s a common enough approach for demonstrating grant flows without full application integration. (I’ve actually written something in the same ballpark myself, although my implementation was a lot simpler.) It’s also a lot more ‘joined up’ if you are running the browser on your Pi, as the localhost redirect will resolve to the little server the code spools up for the duration.

Update: 17/04/22: unless you ‘publish‘ the client, the example code will only be allowed to haul down tokens which expire after a week. I’ve gone with a simpler alternative, discussed here. Back to the original post…

I’ve made a couple of minor tweaks to the code. One is to change from the default ‘primary’ calendar to a shared one. You just find the calendarId string by using the ‘try this method’ built into the documentation.

The second change is writing the event data out to a couple of files for onward processing. In order to make that as simple as possible, I ‘flatten out’ the dates that the API returns. There is a gotcha here, because recurring events are returned without a start time so:

try:
   formattedDate = datetime.datetime.strptime(start, '%Y-%m-%dT%H:%M:%S%z')
except ValueError:
   # recurring events like birthdays just have:
   formattedDate = datetime.datetime.strptime(start, '%Y-%m-%d')
   formattedDate = formattedDate.replace(hour=12)

I hadn’t expected this at all, particularly as recurring events are rendered at a time on every Google Calendar UI I’ve seen, and the change in date format only emerged when I started adding birthdays to the shared calendar, which broke the original parsing – without the ‘try’.

Content Display (and wasting more time with selenium)
While I’d abandoned using selenium and chromedriver for the Google calendar screen grab, I hadn’t completely given up on it for other content rendering. It’s actually quite convoluted to get working. Using pyvirtualdisplay as a way of convincing the operating system that you’ve got a display to run X on is straightforward enough, but the versions of chromedriver and chrome itself have to match up exactly. At the time of writing the former is lagging the latest chrome release, and getting a slightly older version to work required a workaround.

The thing that made me finally abandon it was that something I had working one day (browser.find_element_by_xpath to click on ‘Most Read’ on the BBC website) stopped working a day or two later. I never got to the bottom of why chromedriver was saying it couldn’t find the element, and finally realised that RSS would be an awful lot simpler.

Up until that point I’d also been using chromedriver as a way of rendering and screenshotting the html table for the calendar events. I decided to use imgkit to do this instead. It has its foibles, specifically with UTF characters, so I may swap it out at some point.

As I alluded to earlier, while with the benefit of hindsight the whole approach around selenium reads like a bad idea (or, to put it another way, utter madness) from the get-go, it reflects the fact that over the period of the couple of weeks that I was working on the project, there was a lot of trial and error with figuring out what the screen was capable of displaying. Although as 1 bit colour depth clearly suggests, the answer to that is ‘not much’.

The last significant decision was how to present the two main pieces of content: the BBC headlines and the calendar events. Because of the lack of anti-aliasing, bigger text is better. Rather than just display one at a time, what I decided on is overlapping, and then flipping them every 15 minutes. This way, you get to see the most news- or time significant information either way round. I’ve got some clunky code to read / write out a single word to file – either ‘beeb’ or ‘cal’ indicating which was the last ‘top’ item. Depending on which of the two’s turn it is, I set the frame coordinates and, most importantly, the order that I call PIL’s Image paste method.

The last part was having something useful in the opposite corners the main tiles of content leaves. The date was a no brainer. The ‘up next’ thing took a little more work and at the moment is the only consumer of the serialised event data.

Tidying up

As well as the appearance of the table, there are a few stragglers that I may get round to tidying up. I check the length of the headlines before I print them out, truncate them if they hit a character limit, and add ellipses. I have no idea if there is any kerning magically happening with the TrueType font. I don’t check any of the other lengths – and the Pil Image Draw function will display exactly what it’s given, for instance, which could get interesting with longer event titles.

Another possible enhancement is very specific to my implementation but brings up a feature of the API call: it decides whether or not to return event data based on the end time. There doesn’t seem to be a way of changing this to start time. This would have been handy for me as I import calendar data from an external source which creates all day events that aren’t actually useful after the start date has passed. As the start time is returned by the API (recurring events not withstanding), it would be trivial to post process this.

The Implementation

There’s nothing terribly exciting about the implementation but the two python scripts are here. As ever, the repo is principally intended for my own backup purposes. The various dependencies (sourced from git, pip3 and apt-get), while not documented explicitly, are all fairly self evident.