Monitoring Humidity…

TL;DR

This boils down to:

  • A Raspberry Pi with a humidity sensor attached
  • Two Python scripts: one for a web endpoint, and another to generate push notifications running from cron.
  • A simple iPhone app to receive the push notifications and to graph the humidity data.
App Screenshot…

I have a small collection of guitars, one of which suffered damage from humidity over the winter. It was a tough lesson: I had no idea that they were so sensitive. I’ve subsequently bought another instrument which I’m now nervous about keeping out of its case.

Having looked unsuccessfully on Amazon for USB connected humidity sensors, I decided to re-purpose a sensor which I’d been using with an Arduino a couple of years back, and which had subsequently been consigned to the attic. I was reasonably satisfied with my first pass: the connection to the GPIO pins on the Raspberry Pi is a lot less faff than processing serial data over USB from the Arduino. I wrote a Python script which ran from cron every hour, and if either the temperature or humidity was over / under a threshold, it would email me. Side by side testing with a borrowed standalone sensor suggested it was reading consistently a few degrees hotter. The only male-to-female jumper wires I had lying around were pretty short, so the most obvious culprit was the heat from the Pi. I ordered some longer ones — and duly broke the sensor somehow when I reconnected it. Doh!

I ordered up a replacement [GY-BME280] which seems a lot more sensitive. A hot spell of weather suggested that something less intrusive than email would be better, so I thought I’d have a quick play with iOS push notifications. I ended up using OneSignal, who have a generous free tier. One slight constraint is that their Python library is for V2 only, and the dependencies for the sensor had already committed me to V3. Having gone through the app setup on OneSignal’s admin interface, and a quick browse of their API [simple for my purposes], I use requests to send the notification instead:

if (ALERT):
   post_body ={
      "app_id": "app-id-from-One-Signal",
      "headings": {"en": "Indoor weather warning!"},
      "contents":  {"en": CONTENT_TXT},
      "included_segments": ["Active Users"]
   }
   r = requests.post(url = pushEndpoint, headers={'Authorization': 'Basic api-key-from-OneSignal'},
 json=post_body)
 

Obviously the vendor library takes care of the exact syntax of the Authorisation header. That took a bit of testing to see exactly what was required when rolling my own API call.

I generate the CONTENT_TXT based on some if / elif statements earlier in the code. I took a different approach when I was using email: the content was additive, so if both the temperature and humidity were over the threshold, I’d get all of the info. Now, the first one to trigger wins as there isn’t a lot of real estate in the notification window. As an aside, the code above doesn’t process the response from the OneSignal endpoint. If it failed for some reason, I could resort to email, but that is a little circular for my implementation.

It’s rather one-dimensional having a boilerplate app on the phone which does nothing other than receive the push notifications, so charting seemed like a sensible addition. The state of the art has certainly moved on from when I first used charting in my weight tracking app [it was 7 years ago!]. There are a couple of components to this: the first is a very simple web service with a couple of endpoints [using Flask], the first of which returns historic data generated by a script called by cron – which I’ll come back to – and the second which calls a script to get live data from the sensor.

The cron script – the same one that generates the push messages – retains the last 24 hours of data on a rolling basis. I managed to make pretty heavy work of this, but here’s what I’ve come up with:

 with open(dataFile) as json_file:
         data = json.load(json_file)
         json_file.close()
         data['allData'].append(dataPoint)
         currentLength = len(data['allData'])
         if currentLength > 24:
             # This should always only be deleting the single, front-most element:
             delta = currentLength - 24
             trimmedData = data['allData'][delta:]
             trimmedData = {"allData":  trimmedData}
             data = trimmedData
         with open(dataFile, 'w') as outfile:
             json.dump(data, outfile)
             outfile.close()

The JSON structure is an outer dictionary with a single value, the array of ‘dataPoint’ dictionaries, where each of those is a single reading. Where I got myself into a bit of trouble was with this line:

trimmedData = data['allData'][delta:]

…which returns the ‘inner’ array reduced by the delta number of values from the front, rather than the ‘outer’ dictionary which wraps the array. There is probably a better way of doing this – not using the outer dictionary, for instance – but it works.

Programmatically, the iPhone app itself is pretty trivial. The more complicated part – relatively – is stepping through the install and config of the OneSignal SDK [plus admin processes on their web UI] and cutting the push certificate, but it’s all well documented.

I’m still clinging on to writing Objective-C which is starting to feel akin to admitting that I like to code in Latin. One aspect of Charts which took some digging into was how to put custom labels on an axis – in Objective-C. At the same time I parse out the readings into a mutable array of data entries…

[hums addObject:[[ChartDataEntry alloc] initWithX:xCounter y:humidity]];

…I also grab the timestamps – actually just the hours:minutes – into another mutable array and then…

 lineChart.xAxis.valueFormatter = [ChartIndexAxisValueFormatter withValues:timeLabels];

The Buried Bodies

Functionally, that’s pretty much it. There are a couple of extra decorations, the most significant of which is only attempting to haul the data down from the endpoints when the phone is connected to my home network. While I have the facility to carve out a DMZ on my network, this endpoint isn’t going to be the trigger for me to take the plunge. One body I’ve buried up to this point is that the Python scripts which connect to the GPIO need to run as root. There are ways round this, but it’s a utility vs effort trade-off thing. The same goes for using Flask, which is calling a script which runs as root.

Well I work in security, so it’s nice to let my hair down at weekends 🙂 .

The second minor detail of this implementation is that unfortunately Apple doesn’t allow the configuration of the push notification capability without signing up as a full developer. When I deleted my apps from the Apple Store last year, I decided to let my developer subscription expire.

While you can do quite a lot without it, having to recompile apps every 3 months or so can be a nuisance – something that was reinforced when my wife was asking me what I was doing on my Mac 10 minutes before a taxi was due to take us to the airport last week. The answer: re-compiling the iOS password manager that I wrote, and am quite reliant on when travelling. Instinctively, I knew this was the wrong answer 🙂 .

Anyway, I’ve signed up for another year.

Parsing FIDO Raw Messages

I’ve spent the last few weeks working on iOS app to use a Feitian BLE Multipass token. When I get to the final version of that I’ll do a separate posting but parsing the raw messages to and from the device turned out to be pretty challenging. Unfortunately, if you send something that doesn’t conform to the required format, rather than getting an error back, you get no response at all. This makes debugging [particularly at the start, when I really didn’t know what I was doing] tricky.

While the standards for both the FIDO BlueTooth spec and the raw message format are both well written, there are a few bits that aren’t immediately clear, and that I couldn’t find documented anywhere.

The Feitian device enforces a maximum fragment size of 20 bytes. Any arithmetic I use in the rest of this write-up will assume this, but that will vary from one device to another. Also, I will tend to use the actual number of hex characters transmitted. As the payloads are double byte, this will mean that a decimal value will be derived by converting a hex value to decimal, and then doubling it.

Sending An Enrolment Command

The first fragment you send looks like this:   

The FIDO command prefix has the value 830049. The 83 indicates a message, the 00 is a blank high data length, and 49 is the length in hex, which equates to 73 in decimal, 146 characters. This is derived from adding together the APDU command prefix [14 characters], the concatenated challenge and application parameters [128], and the a final 4 characters, all zeroes, which are the APDU Le values. 

The APDU command – the example I’m using is enrolment – has the value 00010000000040. As per section 3.1 of the raw format spec, that’s a preceding 00, then the enrolment command of 01, then a bunch of blank parameters, and ultimately the 40, which equates to the 64 bytes for the combined length of the challenge and application parameters.

For each of the subsequent chunks, you have a payload [in the Feitian’s case] of 19 double bytes, plus one more byte for the order. The last message includes the 4 bytes for the Le values.

Parsing the Response

The response is a bit trickier. First, the device may respond with a FIDO keep alive command, which is 82000101. So before I start concatenating the full response, I check that the individual chunk value doesn’t equal this. Just a quick detour into the implementation: the BlueTooth spec says that the values you get back from the token will be Big Endian double bytes. I couldn’t find a nice way of processing these using an NSUTF value, so I just convert them to text and parse out the angle brackets and spaces.

On to the processing. The first six characters I get are 830259. Per the spec, 83 is playing back the original command. 259 in hex means that the total length of the response will be 1202 characters. That’s going to vary: both the attestation certificate and the signature are variable length. For my implementation, I calculate a running total, so that I know when the final message has been sent.

So, there is no APDU header at all in the response. After the 6 character FIDO header, you get straight into the standard payload as per the spec. You have a legacy 05, then 130 characters for the public key, then a key handle length. For my device, the value is 60, which translates [96 in decimal x 2] to 192 characters. 

The next part took quite a bit of digging, which is figuring out the length of the attestation certificate. The certificate starts with the following: 30820168. While my read of DER encoding of ASN.1 [ugh!] is that the length specification can vary, for me, that 0168 is the length in hex, equating to 360 bytes. Add back in 4 bytes to include the header [30820168], that gives me an attestation cert length of 728 characters. 

What remains is 144 characters. The last 4 are 9000, which per the raw message spec is the SW_NO_ERROR value. 

And that’s it. 6 weeks of trial and error in about 700 words.

Deleting My Apps From The App Store…

Five and a bit years after I released my first app, WeighMe, I’ve just deleted it and Pin Your Pics from the App Store. Here were the total downloads:

Total App Downloads

The reason I’ve zapped them is simple: it doesn’t make sense to continue to invest serious amounts of time in Objective C, in [for Pin Your Pics in particular] what developed into pretty large and complicated codebases. Apple made an announcement quite recently that support for iPhone X screen dimensions is about to become a mandatory requirement, which was going to be quite a bit of work for me.

But the more immediate death knell for Pin Your Pics came a few weeks ago when Google announced a change to their Maps API. Up until this month you could get an API key, and it would just stop working above a fairly generous threshold. Now you have to sign up with your card details. While my app usage is unlikely to trouble the new limit, it’s a chance I’m not prepared to take.

I am a bit gutted that, when Google cut over the API, the app is going to break quite unceremoniously, which is a bit rubbish for people using it. But I hadn’t updated it in ages and it was starting to get fairly unstable anyway.

I still code up the odd utility in Objective C but I’m not going to renew my developer subscription when it expires this year: I really can’t see myself publishing any more apps.

Some interesting numbers buried in the 5k download total. More than half were in China. I have no idea why. Also, that spike last September was weird: 1.9k downloads that month alone. I did absolutely no marketing for either of the apps. Finally, to my lone ‘customers’ in each of Uzbekistan and Kazakstan, thank you for using my apps, whoever you are :).