OAuth Integration - Overview

What seemed like a simple endeavor -- learn how to implement OAuth to protect web services and portal applications using Java and Spring -- turned out to be much more complicated than expected. The posts in this series are targeting two groups -- developers with little background in the goals and mechanics of Oauth and more experienced developers who just need help avoiding pitfalls resulting from inconsistencies in implementations of OAuth. The content is divided into multiple posts:

OAuth Integration - Overview <---- YOU ARE HERE
OAuth Integration - Challenges
OAuth Integration - Authorization Server
OAuth Integration - Command-Line Validation
OAuth Integration - Resource Server
OAuth Integration - Service Client
OAuth Integration - User Portal Application

OAuth Use Cases

Anyone reading this material is likely interested in OAuth for one of three reasons:

1. You need to CONSUME a web service that is protected with OAuth
2. You need to CREATE a web service that is protected with OAuth
3. You need to CREATE a portal application that allows users to authenticate with existing identities in one or more external providers (such as Google, Facebook, LinkedIn, etc) and / or multiple INTERNAL identity systems within a corporate setting that expose identities via OAuth. This app may also need to consume OAuth protected web services, either internally or externally.

The developers of OAuth capabilities had a longer list of specific use cases they wanted solved by implementing OAuth. Those scenarios include:

1. allow developers in DIFFERENT organizations to re-use existing identity platforms for authentication / authorization functions for new applications
2. allow developers WITHIN an organization to re-use existing identity platforms rather than duplicating userid administration within each application (a simplified version of #1)
3. provide complete separation between AUTHENTICATION functions (which should only be performed within existing remote identity providers) and AUTHORIZATION functions (which should only convey information about specific AUTHORITIES required by the application
4. during authentication, explicity identify all AUTHORIZATIONS requested by an application and provide the user the ability to selectively approve or deny those AUTHORIZATIONS (e.g, the application wants to user authorizations for email, openid, functionX and functionY but user decides he doesn't trust the application to have access to functionY capabilities)
5. provide extension points within authentication flows to allow applicatioins to suspend state and return to it after the asynchronous remote authentication completes -- an example scenario: remembering a user surfed via bookmark to https://myapp.myfirm.com/private/somepage and had to first redirect through authentication prior to being redirected to the original requested page
6. provide extension points after authentication to map generic external authorization strings (scopes / claims) to application-specific values
7. provide support for expanded user / authorization data while using stateless tokens to limit loads on centralized authorization platforms
8. supporting encryption of tokens to limit risks of exposure of user data

Due to the number of goals pursued by the architects of OAuth, the resulting protocols and code libraries built to implement are likely more complicated than the needs of any individual developer, especially when used for basic web services. It's also possible to integrate to ONE system using OAuth and assume the larger capabilities are fully understood until integration with a second or third system is required, at which point variations in implementations will require additional code or refactoring of existing code.

OAuth System Actors

Actions performed when authentication and authorizing access using OAuth are conceptually mapped to five different actors in the ecosystem.

User	The party which wishes to invoke a resource protected by Oauth2 authentication. The user might be a human using a browser submitting AJAX queries from HTML pages or a smartphone app. The user may also be another application process -- maybe an outer proxy service that has accepted a request from a human user which now needs to invoke an internal enterprise web service.
Client App	An application which must interact with data secured by a Resource Server on behalf of a User. A client application could be a browser or smartphone application directly in the hands of a user. It could also be a backend service of a portal application sitting between a browser / smartphone client and a core enterprise web service that uses Oauth authentication / authorization.
Resource Server	A process running atop the services being protected which filters incoming HTTP requests and invokes Oauth authentication flows when required to establish an authentication token and executes filters defined by the applicaton enforcing restrictions on which endpoints can be invoked by an authenticated user given the authorizations conveyed in their token.
Authorization Server	A process implementing the core Oauth2 protocols for accepting authentication requests, querying underlying Identity Provider systems (relational databases, LDAP databases, or various commercial identity pools such as Gmail, Facebook, LinkedIn, etc.). NOTE: This term can be misleading. Technically, the "Authorization Server" name describes its primary purpose in the context of Oauth which is managing and transforming AUTHORIZATIONS. In many implemenations, the Authorization Server combines or abstracts external AUTHENTICATION functions which can generate confusion.
Identity Provider	A distinct system that houses core identities and passwords. Identity Provider functionality CAN be combined with Authorization Server functions in implementations but tends to be separated for larger companies with thousands of empoyees or service providers providing service to tens of millions of customers.

2.1.3 Access Tokens - Creation / Validation

Information reflecting user / entitlement information for ongoing access after authentication is communicated between actors using Java Web Tokens or JWTs. JWT objects are typically modeled internally as HashMaps of (key:value) pairs, serialized in Json syntax and transmitted between parties using Base64URLSafe encoding. (Base64URLSafe encoding is essentially Base64 encoding with an additional character substitution to transmit - (minus) and _ (underscore) instead of + (plus) and / (slash). Each token provides a header with information identifying the cryptographic algorithm and public key used to sign the token and a body section containing "claims" with specific user / entitlement information. A final signature section reflects the digital signature of the header + body concatenated with a period. The complete process of JWT creation is visually depicted below.

1. creates a JSON header providing information about the public key used to sign the final payload
2. creates a JSON body encoding the claims providing information about the application, user, roles and timestamps of creation / expiration of the token in "epoch seconds")
3. creates Base64URLSafe versions of the header and body JSON strings.
4. concatenates header64url + "." + body64url into a final message string
5. creates an RSA256 hash signature of the message string and encodes that signature via Base64URLSafe
6. combines message + "." + signature64url as the final bearer token string

That process is illustrated below.

After a token is first obtained by processes described in the next sections, any software component relying upon a JWT submitted by a client must validate the content of the JWT by reversing many of the steps above then verifying the signature AND comparing the expiration flag in the body against the current time. The specific validation steps look like this:

1. split the incoming token at the "." character into three pieces -- header64url, body64url, signature64url
2. decode header64url, body64url and signature64url back to the original clear-text JSON strings
3. extract the kid ("key id") value from the header and issuer from body
4. map the kid / issuer values to the public key used by that Authorization Server
5. concatenate header64url + "." body64url into "message"
6. use the publickey to verify message and signature
7. extract the iat and exp values from body and test for token expiration against Instant.now()

Claims / Authorities / Roles

Claims are the (key:value) pairs of strings provided in the body of an OAuth token that allow software using the token to map external information about the authenticated user into authorization rights within the application. The terms authorities and roles are used somewhat interchangeably in security libraries in Java (and probably other langauges as well). Perhaps the best distinction to make with these terms is that claims are external encodings of authorizations while authorities / roles are internal equivalents as required by an application. It is the application's responsibility to map the external claims to internal authorities / roles as a token is submitted with a request.

This table illustrates the type of mapping required from external claims to internal authorities / roles. It reflects a scenario where authentication will be integrated to two different providers -- mdhlabs and providerx -- who each implement two levels of authorization in their proprietary schemes which need to be mapped to the customer or admin role within the internal application.

mdhlabs Role	providerx Role	Internal Role
mdhlabsuser	providerxbasic	customer
mdhlabsfull	providerxadvanced	engineer

External parties not only use a variety of labels for authorizations but the authorizations are encoded with a wide variety of data structures sent in the Json claims, requiring significant flexibility if applications need to support authentication for users served by multiple Authorization Servers (e.g Google logins and Facebook logins).

Much of the complexity in implementing OAuth stems from business logic that must be defined and inserted at the appropriate point in token flows to ensure translation from external labels to meaningful internal labels. The layers within Spring Security performing these translations have undergone significant re-engineering by the community, resulting in many deprecated approaches and incompatibilities with other layers of functions in the OAuth libraries.

OAuth Processing Flows

The use cases supported by OAuth require a variety of interactions between different actors in the system. These actions are summarized below using a term which will be used consistently throughout this document.

1. AUTHORIZATION -- This flow creates an authorization token by redirecting a human user to a login page served by the Oauth Authorization Server, avoiding the need for the client application to ever posess the clear-text (username + password) combination. If authentication is successful, the authorization token returned contains a set of scope labels allowed by the human user to be returned to the application. These scope labels are often NOT identicial to role names used in the requesting application.
2. ACCESS -- This flow submits the AUTHORIZATION code returned by the Authorization Server to the user back to the client application which then submits the authorization code in a request for an ACCESS TOKEN to the Authorization Server. The Authorization Server validates the code submitted is still valid and returns both an ACCESS token and a REFRESH token to the application. Use of a short-lived access token (typically valid for 5 minutes) and a longer lived refresh token limits the window of time over which the access token could be mis-used if intercepted while allowing a new token to be regenerated without full human re-authentication with (username + password)
3. USERINFO -- Some applications require more user / role data to be retrieved when mapping an AUTH token into an ACCESS token which requires a secondary data structure to be fetched during authorization. The application can submit the AUTH token to the user information endpoint of the Authorization Server to return any additional information available.
4. REFRESH -- If an access token reaches its expiration, the client application can use the matching REFRESH token to obtain a new access / refresh token pair to use for another N minutes indefinitely without forcing the user to re-authenticate via username/password. Not all providers return refresh tokens with assigned access tokens. Those that do not instead typically provide VERY long lifetimes for access tokens but require an application to bounce the user back through the provider's authentication page which will detect their own cookie and redirect them back to the callback endpoint without actually challenging the user to re-authenticate.
5. REVOCATION -- For highly sensitive applications, applications can implement a Logout function in the user interface which can make a call to the revocation endpoint of the Authorization Server to explicitly invalidate a current access / refresh token pair. This will prevent the refresh token from being honored to immediately create another access token without (username + password) authentication. NOTE: The application itself must still implement logic to stop accepting the access token as valid before its expiration time.
6. REGISTRATION -- In many cases, an application developer will work with a human administrator of an Authorization Server to "onboard" a client application so it can use the Authorization Server. For scenarios where Authorization Server functionality is a retail product, additional APIs are provided to allow specially privileged userids to create other client registrations so developers can avoid human support flows and possibly automate onboarding. This is particularly useful for creating "clients" in an Authorization Server for test environments. Registration flows will NOT be covered in this material.

Access TokenRequest Flows

The OAuth2 protocol defines different types of flows for obtaining access tokens, each reflecting differences in the security relationship between the user, the client application, the resource server and the authorization server. The key flows are summarized in table form here.

authorization-grant-type	Description
authorization_code	Splits access token generation into two phases. In phase 1, the user is redirected directly to the Authorization Server to AUTHENTICATE which returns a one-time authorization code to the user. In phase 2, the user submits the one-time authorization code to the application client which submits a token request to the Authoirzation Server which then returns an access token the application client continues to use. This flow is ideal for user interfaces where an application doesn't own / control the identity of the user for AUTHENTICATION but only needs to independently confirm it and map it to AUTHORIZATIONS.
client_credentials	Obtains an access token in a single interaction between a client application and Authorization Server by submitting (client-id + client-secret + username + password) to the $token_uri endpoint of the Authorization Server. This approach is suitable for machine clients needing access to services independent of any individual human user.
password	Obtains an access token in a single interaction between a client application and Authorization Server by submitting (client-id + client-secret + username + password) to the $token_uri endpoint of the Authorization Server. While similar to client_credentials in functionality, this flow is not recommended by OAuth developers. It can also result in different access token content being returned which libraries expecting client_credential responses may not handle propertly, requiring customization.

Oauth Token Types -- JWT and Opaque

An OAuth authorization server can be configured to return one of two types of tokens to a client that successfully authenticates. A JWT token is a standard base64 encoded token that the client can decode using a public key fetched from the authorization server, allowing the client to continue using that token for its configured lifespan without re-querying any other component to decode it. An Opaque token cannot be unencrypted by the client without submitting it to another endpoint. In general, JWT tokens tend to be used for machine-to-machine authentication WITHIN an application core (e.g from the access tier to the internal enterprise web service tier). Opaque tokens tend to be used for external (human) browser clients accessing the outermost public tier of web services into a core application.

OAuth Demonstrations

Demonstrations of different OAuth integration approaches will be provided in four different areas. Each area will be covered in a different post (linked at the top of all of the posts) and structured to be independent of the others. If knowledge of all aspects of OAuth integration is desired, reading through them all in order from Authorization Server to User Portal Application will provide the most sensible stair-step introduction.

User Portal Application	This segment will illustrate additions required to a typical Java Spring MVC application written to run in Spring Boot will implement user login redirect / callback functions required for two-step authentication and authorization of users and how external roles are normalized to application capabilities. Authentication flows and authorization normalization will be illustrated for the mdhlabs and providerx Keycloak realms created in the Authorizatioin Server segment as well as Facebook, Github, Google and LinkedIn.
Service Client	This segment will create a Java web service named oauthdependsclient written to run within SpringBoot that implements OAuth client libraries to obtain access tokens from a remote Authorization Server and submit tokens as credentials when invoking a taret web service. The service client example will use credential configurations defined in a Keycloak instance in the Authorization Server example and will invoke the oauthdependsresource web service.
Resource Server	This segment will create a Java web service named oauthdependsresource written to run within Spring Boot that implements OAuth in the security filter layer. The service itself uses JdbcTemplate and a MariaDB database to access records from a PROJECTS table in a database called DEPENDS.
Authorization Server	This segment will illustrate how to download, unpack, configure and run Keycloak as a combined Authorization Server and Identity Provider platform. Two different authorization realms named mdhlabs and providerx will be created for use in the later examples. Techniques for large-scale production deployment will not be discussed but Keycloak is a very high quality implementation that makes it easy for developers to self-administer. Keycloak is also the most standard-compliant implementation of any of the providers used for demonstrations.

The relationship between these demonstration elements is shown below for reference.

---