Note that all references that follow are for V1 of the Hue API.

A recent house move has required me to revise the code for a bedside Raspberry Pi with touchscreen which I use frequently to control some Hue lights.

At the time I wrote the code a couple of years ago, I did my best to make it as tidy as possible, particularly for the class I wrote for monitoring the lights’ state and switching. I’ve had to hack it in order to take account of some unanticipated new functionality: using it to control a group of bulbs, rather than an individual light.

In order to do this, I had to remind myself of some of the original logic, which includes contingency for an occupational hazard for the use of Hue bulbs at home: people switching them off at the wall. While the mobile app – at least at the time of writing – still doesn’t check for this and will merrily let you switch lights on and off with no effect, the API does facilitate it. So in order to query the current light state you can AND both of these from the JSON response from the …/lights/yourLightNumber endpoint:

isSwitchedOn = str(jsonData['state']['on']).strip()

isReachable = str(jsonData['state']['reachable']).strip()

Groups do not have an indication of reachability. What I’ve decided on as the lowest impact way of checking the state of the group is simply to check on one of the bulbs it contains – a safe enough assumption for me as all of them are on the same physical light switch in the room.

For bulb switching control, you just need to append the word ‘state’ to the end of the URL used for querying status. While the HTTP verb changes to a PUT (with payload) for the latter, my pre-groups code structure was based around this assumption on the URL structure. Now I search for the name of the bulb we are checking state on, and rewrite the URL for the reference to the group:

elif action == "switch":

   if "/6" in interMedURLString:

      interMedURLString = URL + APIKEY + "/groups/81/action"

One more change like this and I’ll have to start the code from scratch – but it works and is good enough for now. If you’ve stumbled here from Google and are looking for some code to get you started – with caveats on structure! – you can find it here.

Over the summer, I wrote up a post on my latest Raspberry Pi project, which is based around an 8 inch touchscreen and some Tkinter code.

It is heavily reliant on the after() method to periodically update the state of the various labels and buttons that comprise the UI elements. The main label cycles through RSS feeds, a shared Google calendar, and a NASA image of the day which involves some screenscraping. (I’m well aware of how bad an idea this is; in fact its reliability is something I’ll come back to in a second.) There is also a row of per-Hue light buttons that are constantly polling the bulbs’ states, and setting their colours in a meaningful way.

I continued refining the functionality after I posted the write-up and when I finally got to a point where I thought I was done, I noticed a problem. After a day or two, the buttons started to fall permanently out of sync with the bulb states. They displayed the wrong colour (for on / off / unreachable) but still worked to control the lights. For weeks, the cause was a complete mystery.

I think I’m starting to figure it out, starting with a few novice-level implementation errors. Despite the fact that I’ve been using Python for a very long time, it has generally been for fire-and-forget utility scripts. This code is fundamentally different in that it has to run indefinitely, which elevates the need for stability. At the same time, I introduced a few new moving parts (for me): using modules to try to make everything more readable and potentially reusable, and the use of the Python logging framework.

One embarrassingly simple mistake that I was also making was using a desktop shortcut to launch the GUI. While this made sense for the touchscreen, it meant I was losing anything that was being written to standard error. With the benefit of hindsight I am sure I could have redirected this using the logger but at the time, it appeared as if the functionality was failing silently no matter what logging statements I was using. But there was another aspect to this: the problem seems to have been with errors that I’d intuitively expect to be unrelated.

It appears that Tkinter is very sensitive to exceptions being thrown, and it’s worth digging into how this manifests in more detail. I have one module which is using an after() method call in a 15 second loop to poll the Hue hub for every bulb, using the Requests library. In another module, I have a randomised selection of content, all of which uses Requests in one way or another. One of these is for the NASA image of the day. The approach I use for this is pretty ridiculous: a search for a relative URL in the HTML, and then the construction of an absolute URL to haul down the image it refers to. Coming as a shock to absolutely no-one, this is occasionally unreliable.

What I wasn’t expecting was a Requests exception in one module affecting the use of the same library in another one.

While I have used threading in other languages, I haven’t in Python. I imagine that each of the after() calls I make is running in its own thread: I can’t see how this works any other way, otherwise the main event loop would be blocked. But the exception in one thread tearing other threads down? I’m not sure about this. There are plenty of hits on Google if you search for ‘Tkinter thread safety’ but, from what I can see, these get into discussions on what happens if you implement threads yourself as opposed to what’s going on under the hood.

I have – I think – eliminated most of the obviously problematic sources of errors by doing what I should have done in the first place for long-running code: surrounding the various calls to Requests, JSON parsing etc. in try / except clauses. This seems to have ironed out the stability issues. So far.

But I haven’t discounted the possibility that I’ve made a fundamental mistake with the approach I’ve taken…

Following on from this post about setting up an e-Ink screen with calendar based content, what the Google quickstart example doesn’t call out is the significance of this setting in the OAuth Consent screen:

If you leave the ‘Publishing status’ set to ‘Testing’, the access and refresh tokens expire after 7 days. Also, the way the out-of-the-box script works, you need to delete the token file before re-running the script, which requires you to re-authenticate.

The process of publishing means that the script is open to the public, and requires a review by Google – clearly not appropriate for this project.

I tried a couple of different approaches to work around this, starting with service account impersonation – i.e., having a service account that impersonates the end user account (mine). While it seems to be possible to do this, you need to use Google Workspace features which I don’t have a subscription for.

The alternative approach that I’ve taken is simply to add the service account to have read only access in the Google Calendar interface:

I’ve exported the credentials for the service account to a JSON file, and altered the original quickstart example to use it. It’s actually a lot simpler: it should (!) run without any manual intervention at all.

The details of the authentication that the service account is using is hidden by the Google Python library, but my working assumption is that it’s using the private key as a secret, and presenting a token via the Client Credential grant. I’ve added the new version of the quickstart file which uses the service account authentication to my GitHub repo.