Articles

June 18, 2015 · 10:18 pm

RESTful JWT login process

Introduction

RESTful APIs are a great way to build reliable and decoupled server/client protocols. Nearly all application require some type of login and/or session capabilities and there are numerous systems that allow users to share logins across multiple systems. Therefore, building a RESTful authentication system which can handle third party authorizations and allow a different third party access to the API is highly desirable. However, a recent search of the web and questions on StackOverflow did not provide a modern, standard, RESTful solution. Hopefully, with feedback, this outline can cover the basics of a modern, RESTful, login system.

Systems involved

In todays modern, connected world with multiple systems sharing login credentials and information it is sometimes difficult to discern which system has a particular responsibility. An example of building a Todo App will be used to augment the discussion. This process will involve the following systems:

MyTodoApp is the name of the actual application used by the user. It may be implemented in iOS, Andriod, a JS SPA, an in vehicle proprietary system, and/or something else. The MyTodoApp developers are concerned with building a great user interface and providing new ways to work with todo lists. There are not interested in new ways of storing or drawing conclusions from stored todo lists. As a side effect of not doing “big data” things, MyTodoApp also does not want to focus on keeping track of users.
BigCorpTodoServer will be the system implementing the RESTful, JWT based, login process. BigCorp is a data science shop and wants to find better ways for todo list to predict market trends, find new ways for people to share and be social with todos, and other data focused research operations. Therefore, BigCorp has build the BigCorpTodosServer. The server may be implemented in ASP.NET MVC API, RoR, Nitrogen, and/or something else. It stores todo lists from different people and provides todo list services. Unfortunately for BigCorp, few users have the BigCorp flagship social media application installed on their devices. Therefore, BigCorp has to support a variety of third party authentication methods.
SocialSystemsInc supports third party authentication and claims via OAuth. They are the current leader in social media and their extensive collection of users enjoy the convenience of having a single sign-on via the SocialSystemsInc API.

Resources used

The most critical component of a RESTful API is the resources. The resources have to accurately, and individually, represent the nouns in the story. A single resource representing two nouns works if Jack and Jill go up the hill but if there is a future version of the story where Jack stays and Jill goes there will be volatility in the resource definitions. Therefore, for the login system resources will be created for the session, user, authorization, and credentials. Please note that to the best of my knowledge there is no true standard for this process and therefore no standard resources types.

Sessions

The SessionResource represents a relationship between the application and the server. The session does not represent the actual device, application, or person. Also important to note is that the SessionResource does not represent a relationship between the server and a person using an application. The session relationship begins when an application is first launched on a device or after a new install of the application or after a logout has occurred. The session ends when the application is uninstalled or the user logs out. The SessionResource is defined, in JSON for example purposes, as follows:

The session resource has a unique id which servers to identify this particular session resource. The session ID should be generated on the client. However, responsibility for security is a server concern. Therefore, it should be obvious this is not, and need not be a “secure UUID.” Further to this point is awareness that the session ID is used anyplace a reference to this session needs to be created and therefore may be stored in numerous locations. Ultimately, the takeaway is that no security measure should rely on this ID being kept secret and there is no onus to keep this ID secret.

As stated and obviously, the responsibility for a servers security and any details of the security implementation are the responsibility of the server. While the client does have to know on some level how the server wishes to authenticate the session, the more knowledge required by the client the greater the coupling between the server and the client. Therefore, this shared knowledge is kept to a minimum. For this implementation, the client will need to know that the server wants to authenticate the session based on an HTTP header. The auth-header property specifies which header to use for authentication and to what value the aforementioned header is set.

The session determines to what resources the client has access. To expand the client’s nexus of accessible resources the session will need additional endorsements. The system has little understanding of the possible endorsements that may come in the future. Therefore, this component is loosely coupled and will be parsed with a dictionary approach. The type of endorsement is the part that is dictionary matched. The value is then used in a domain specific manner post the type match. The authority identifies who is providing the endorsement. The signature is proof that the aforementioned authority is providing said endorsement.
Users

The UserResource represents the user as they are known to BigCorpTodosServer. The UserResource has to be able to tie multiple sessions together and extends the list of resources to which a session has access. Since the user ties multiple sessions together it is also the keeper of how sessions are permitted to connected to the user. Finally, the UserResource also stores information about the user. It is important to note that the UserResource does not store any authentication information.

The UserResource is defined, in JSON for example purposes, as follows:

Other than the ID the first property of the UserResource is the metadata. This is just information about the user that may be of interest to the client. This data is stored in a key value pair format because it is volatile and therefore should be parsed with a dictionary as to avoid volatility in the resource and the accommodate a variety of client usages.

The metadata-sources property allows other systems to store and modify user data. The UserResource then becomes the aggregator of the user metadata. The order in metadata-sources may be pertinent since sources listed higher in the order would have priority items listed lower in the list if providing the same metadata. The metadata link is also dictionary parsed, with the knowledge that the token is the secure component, due to the potential variety of alternate sources and user metadata.

Authorizations simply provides a list of accounts on authorization providers that are allowed to connect a session to a user. This is highly structured data since it need only identify the provider and user within the provider. All the other complexity associated with the relationship is stored with the provider.
Credentials

The credentials resource is the challenge information provided to the authorization server. The CredentialsResource is defined, in JSON for example purposes, as follows:

Given there are a vast array of credential types, it is most logical to use a credentials resource that is dictionary parsed. The supported / provided challenge types are listed under challenges. The authority property is concerned with providing information that allows other systems know that credentials were accepted.
Authorization

The AuthorizationResource connects a session to its associated claims by optionally connecting credentials to sessions. The AuthorizationResource is defined, in JSON for example purposes, as follows:

Given a valid value for the authority property (expectantly derived from a valid credentials resource), the tokens property contains a valid token which states the claims are valid. This token is signed by the authorization server which does not have to be the credentials server.

The process

Now let’s examine the steps to create a user and login to the system.

1.) After the user first opens MyTodoApp, the app needs to create a session on the server.

1.a) Immediately, in the background the application generates an ID to represent its session, connects to the server, and performs a POST to http://api.mytodoapp.com/Sessions with the follow body:

Please note this session ID was generated on the client and may not be and does not have to be a “secure UUID.” The session ID is used anyplace a reference to this session needs to be created and therefore may be stored in numerous locations.

1.b) The server replies with the same resource and populates some additional values:

The auth-header tells the client what HTTP header value to set for future communications. This information is will be used for securely identifying the session and should be treated appropriately.

Likely, the session create is the only permitted activity without a session id. With the session header value set, the device is capable of interacting with more services on the server. However, the user-id value is not set. This indicates that no login has occurred or, put differently, there is no relationship between a person and the server for this session. However, certain functionality of the application may require the server to have a relationship with the user, not just the device.

2.) A prerequisite for connecting the session to a user is that a user must exist in the system. A user could already exist on the BigCorpTodosServer in which case all of step two can be skipped. If user creation is required, two common user creation workflows are illustrated here.

2.a.1) The user opens MyTodoApp and choses to create an account using SocialSystemsInc. This either opens the SocialSystemsInc app or SocialSystemsInc website to perform an OAuth2 authentication. The result is an unique id for the user in the SocialSystemsInc system and a token for claims to the users data stored with SocialSystemsInc.

The MyTodoApp then creates a new user on the BigCorpTodosServer by POST with the following body:

2.a.2) The BigCorpTodosServer uses the claim provided in the resource to connect to SocialSystemsInc and populates the remainder of the user information.

2.b) The user is created directly on the BigCorpTodosServer.

2.b.1) The user visits the BigCorpTodosServer website and POSTs the following form:

which creates an AuthorizationResource on BigCorpTodosServer.

2.b.2) The user then posts a form with required information and a UserResource is created on BigCorpTodosServer. Hidden in this form as a linked account is the connection to the AuthorizationResource created earlier.

Since this creates an account directly with BigCorpTodosServer, future verification will not include SocialSystemsInc.

2.c) The user creates an account with BigCorpTodosServer directly from MyTodoApp. This is done in three steps. First, credentials need to be created on the authorization server.

2.c.1) The app fetches the create CredentialsResource from the BigCorpTodos authorization server and populates it.

This step is not necessary and only needs to be performed once. It is useful for identifying how the server wants passwords to be hashed.

2.c.2) The app creates (POSTs) the CredentialsResource on BigCorpTodos authorization server.

2.c.3) The app POSTs the create user resource with a link to the authorization.

There is not a linked account in the metadata so the metadata must be populated directly. The account is created directly with BigCorpTodosServer so future verification will not include SocialSystemsInc.

3.) The client obtains a JWT token for the session. This could be done in may ways. Two common approaches are outline here.

3.a.1) The client opens the SocialSystemsInc app to perform an OAuth2 authentication. The SocialSystemsInc app returns a token which is valid for claims to SocialSystemsInc data about the user.

3.a.2) The client then creates an authorization resource on the server using the token from SocialSystemsInc.

3.a.3) The BigCorpTodosServer then uses the claim to connect to SocialSystemsInc and validate the claim and fetch the necessary information to complete the response (see step 4).

Alternative workflow using an account previously created directly on BigCorpTodosServer:

3.b.1) MyTodoApp prompts the user to enter their username and password and then performs a query for an authentication object associated with the provided username:

GET: https://auth.bigcorptodos.com/API/Auth?username=billray%40example.com

REPLY:

Note that the query is necessary to fetch the appropriate salt for that username. For security, specifically anti-phishing practices, it may be best for the server to always reply even if a valid username does not exist.

3.b.2) MyTodoApp then computes the appropriate password hash, populates the session id field, and performs a PUT to update the auth resource:

4) In both flows, the BigCorpTodosServer server deletes the credentials field from the resource and populates the auth-header field with the JWT and the appropriate HTTP header name, populates the user-id field with the ID of the authenticated user, populates the expires field, and the resource is returned.

Now the client only needs to connect the session to the user id by updating the SessionResource via a PUT:

The BigCorpTodosServer then uses the JWT sent in the token HTTP header to validate that the session is authorized to be connected to the specified user. The serve also ensures that the account that signed the JWT is a linked account for the user.

Injectable ordering

The pattern occurs when module A needs to be mutated twice, with the first mutation occurring before some action which is coupled to module B’s concerns, and a final time as a required consequence of the initial mutation.

Here’s a simple example in Swift where a collection of objects need to write their positions into a render buffer:

For starters, BallThrower needs out of band knowledge to use the renderer which is not developer friendly. This could be better articulated. That RenderBegin() and RenderEnd() need called before/after RenderBall() is a concern of the Renderer, not the BallThrower, so this is a violation of concerns. Finally, the object is not in stable state until all three calls are complete, which violates the principle of Stable State.

Better is to use Injectable Ordering:

State isn’t floated between calls which is more functional, threadsafe, etc.
It’s in class BallThrower’s concern to return nil of values that are out of it’s bounds. This is simply articulated by using the option parameter.
The ordering of the rendering (which is the Renderer’s concern) is controlled by the Renderer (image a situation in which the renderer finds it faster to z-sort the balls instead of using a depth buffer, this implementation offers more flexibility for that refactoring).

PUT POST to REST: How to properly create resources on a server

The purpose of this POST is to PUT to REST the discussion on how to properly create resources on a server with good API design using HTTP and without violating REST architectural principles.

The problem this article addresses is the design of an HTTP + REST API which enables client systems to create resources on a server. Simplified with an example: a user has created a group message in an app on their mobile device and wants to share it. Other apps are not going to read the message from the client device (it may be unavailable and they won’t even know to look there) so the app wishes to transfer hosting responsibility for this resource to the server. The server needs an API which enables this functionality should be RESTFul and use HTTP as the transport.

The decision of whether to use PUT or POST to create a resource on a server with an HTTP + REST API is based on who owns the URL structure. Having the client know, or participate in defining, the URL struct is an unnecessary coupling akin to the undesirable couplings that arose from SOA. Escaping these types of couplings is the reason REST is so popular. Therefore, the proper method to use is POST. There are exceptions to this rule and they occur when the client wishes to retain control over the location structure of the resources it deploys. This is rare and likely means something else is wrong.

At this point some people will argue that if Restful-URLs are used, the client does knows the URL of the resource and therefore a PUT is acceptable. After all, this is why canonical, normalized, Ruby on Rails, Django URLs are important…. blah blah blah. Those people need to understand there is no such thing as a Restful-URL and that Roy Fielding himself states that:

A REST API must not define fixed resource names or hierarchies (an obvious coupling of client and server). Servers must have the freedom to control their own namespace. Instead, allow servers to instruct clients on how to construct appropriate URIs, such as is done in HTML forms and URI templates, by defining those instructions within media types and link relations. [Failure here implies that clients are assuming a resource structure due to out-of band information, such as a domain-specific standard, which is the data-oriented equivalent to RPC’s functional coupling].

– http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven

The idea of a “RESTful-URL” is actually a violation of REST as the server is in charge of the URL structure and should be free to decide how to use it to avoid coupling. If this confuses you see a future article on the significance of self discovering on API design.

As a side note, some may consider having the server instruct the client on how to create the URL. This is another edge case where the server might specify, via a resource, a URL for the client to PUT a resource. However, the server should have a valid answer if a GET is performed at that place before the client does a PUT there. Additionally, having the server generate, with foresight, all the locations for clients to PUT things is kludge so stop kicking and screaming about PUT and how the client can somehow know the location. You are being dragged to use POST and do it the right way (don’t worry, client libraries will make this easier when it catches on).

Using POST to create resources comes with a design consideration because POST is not idempotent (this scares people into using PUT to create resources when they should not). This means that repeating a POST several times does not guarantee the same behavior each time. This concern is demonstrated in the following situation:

The client POST a new resource to the server.
The server processes the request and sends a response.
The client never receives the response.
The server is unaware the client has not received the response.
The client does not have a URL for the resource (therefore PUT is not an option) and repeats the POST.
POST is not idempotent and the server …

Step 6 is where people commonly get confused about what to do. However, there is no reason to create a kludge to solve this issue. Instead, HTTP can be used as specified in RFC 2616 and, in my opinion, the server should reply with:

10.4.10 409 Conflict

The request could not be completed due to a conflict with the current state of the resource. This code is only allowed in situations where it is expected that the user might be able to resolve the conflict and resubmit the request. The response body SHOULD include enough

information for the user to recognize the source of the conflict. Ideally, the response entity would include enough information for the user or user agent to fix the problem; however, that might not be possible and is not required.

Conflicts are most likely to occur in response to a PUT request. For example, if versioning were being used and the entity being PUT included changes to a resource which conflict with those made by an earlier (third-party) request, the server might use the 409 response to indicate that it can’t complete the request. In this case, the response entity would likely contain a list of the differences between the two versions in a format defined by the response Content-Type.

Replying with a status code of 409 Conflict makes a lot of sense as a generic response because: performing a POST of data which has an ID which matches a resource already in the system is “a conflict with the current state of the resource.”

Also, since the important part is for the client to understand the server has the resource and to take appropriate action. This is a “situation(s) where it is expected that the user might be able to resolve the conflict and resubmit the request.”

Finally, a response which contains the URL of the resource with the conflicting ID and the appropriate preconditions for the resource would provide “enough information for the user or user agent to fix the problem” which is the ideal case per RFC 2616.

Another option for the server might be:

10.3.4 303 See Other

The response to the request can be found under a different URI and SHOULD be retrieved using a GET method on that resource. This method exists primarily to allow the output of a POST-activated script to redirect the user agent to a selected resource. The new URI is not a substitute reference for the originally requested resource. The 303 response MUST NOT be cached, but the response to the second (redirected) request might be cacheable.

The different URI SHOULD be given by the Location field in the response. Unless the request method was HEAD, the entity of the response SHOULD contain a short hypertext note with a hyperlink to the new URI(s).

Replying to a POST method with 303 See Other is actually discussed in section 9.5 where POST is defined in RFC 2616. However, the discussion is in regard to caching:

However, the 303 (See Other) response can be used to direct the user agent to retrieve a cacheable resource.

This cache specific language is my motivation for choosing 409 Conflict as the generic response in this situation. Take note that since idempotent and caching are often coupled, what RFC 2616 should say is that:

The 303 (See Other) response can be used to direct the user agent to an idempotent resource.

Therefore, 303 (See Other) is, in my opinion, a valid response but RFC 2616 will need updated (PUT method) for this the be standard behavior (though it does not, AFAIK, forbid using 303 here). The last option is:

10.3.1 300 Multiple Choices

The requested resource corresponds to any one of a set of representations, each with its own specific location, and agent- driven negotiation information (section 12) is being provided so that the user (or user agent) can select a preferred representation and redirect its request to that location.

Unless it was a HEAD request, the response SHOULD include an entity containing a list of resource characteristics and location(s) from which the user or user agent can choose the one most appropriate. The entity format is specified by the media type given in the Content- Type header field. Depending upon the format and the capabilities of the user agent, selection of the most appropriate choice MAY be performed automatically. However, this specification does not define any standard for such automatic selection.

If the server has a preferred choice of representation, it SHOULD include the specific URI for that representation in the Location field; user agents MAY use the Location field value for automatic redirection. This response is cacheable unless indicated otherwise.

Replying to a POST method with 300 Multiple Choices is not a good candidate per the specification because this response is “cacheable unless indicated otherwise” which is in conflict with the POST method which is never cacheable. If one takes the option that a POST (along with required headers) indicates otherwise and decides 300 Multiple Choices is a valid response this could be used in cases where the server sees it most fit to accept the second POST and allow the client to select the correct version. Again though, the appropriate 409 (Conflict) response could handle this situation and adheres more closely to RFC 2616.

As a note, 304 (Not Modified) could not work because RFC 2616 specifies that it is only available for GET requests and it has no way of returning the URL for the resource.

To understand how the server knows there is a conflict, read my future article on “Resource IDs in Distributed Environments.” To understand the content which the server should reply with for the 201, 200/204, 409, 303, and 300 cases, see my future article on “An <a /> for the cloud.”

Injectable ordering

PUT POST to REST: How to properly create resources on a server

Recent Posts

Recent Comments

Archives

Categories

Meta

Articles

RESTful JWT login process

Introduction

Systems involved

Resources used

Sessions

Users

Credentials

Authorization

The process

Injectable ordering

PUT POST to REST: How to properly create resources on a server

Recent Posts

Recent Comments

Archives

Categories

Meta