ADR – Architecture Decision Record

In an Agile work environment, architecture is described and handled a bit differently meaning that not all decisions will be made upfront, when the projects start nor at once (actually, one the benefits of having and architecture is the possibility of deferring decisions).

As Agile methods are not opposed to documentation (only to meaningless one), we need a way of keeping track of architecture decisions taken during a project or across the whole organisation and documenting them. As nobody likes large documents (and large documents are never kept up to date), a small, modular, “agile” document format would be desired.

In a world of evolutionary architecture it is important to record certain design decisions. Whenever you are going back to a certain project to review the architecture or somebody new joins the team, there is a strong possibility the rationales for past decisions are lost, forgotten or worst, propagated by the “word of mouth”, “we always did it this way” and in this case one can only blindly accept the decisions or change it. To avoid such a behaviour, a repository of past architecture decisions and rationales behind them must be kept in an orderly manner, either linking those decisions to past project or to organisational best practices.

Definitions

An architecture decision record (ADR) is a document that captures an important architectural decision made, along with it’s context, rationale and consequences.

An architecture decision (AD) is a software design choice (or project design choice) that address a significant requirement.

An architecture decision log (ADL) is the collections of all ADRs created and maintained for a project or organisation.

Decision

We keep a collection of records for architectural decisions; those that affects the structure, the composition, non-functional characteristics, interfaces, deployment strategies, etc.

An ADR is a short text file in a format similar to RFCs. Each record describes a set of constraints and assumptions and a single decision in response.

In my particular case, we keep the ADRs in a Confluence repository, with a naming convention like ADR-NNN-Subject, where NNN is a unique number. All ADRs are numbered sequentially and numbers will not be reused. Where an ADR is deprecated, then we mark this in the file name as ADR-NNN-Subject-Deprecated, at least to have a history of those decisions. In addition, when and ADR becomes deprecated because it was superseded by another ADR, we mention this in the file content.

ADR Structure

The ADR contains, at minimum, the following sections:

  • Title
    • A meaningful name for ADR
  • Status
    • What is the status of ADR, such as proposed, accepted, deprecated, superseded
  • Context
    • What is the problem we encounter that is motivating the change or the proposal. This section describes the forces at play, technological, projects, constraints.
  • Decision
    • What is the change or action we’re proposing
  • Consequences
    • This section describes the resulting context after applying the decision. All consequences should be listed, both negative and positive as each decision will have not only positive consequences but also negative and neutral that will affect the technological landscape from now on and fore coming projects.
    • All involved parties (developers, stakeholders, PMs) can see ADRs, even if the teams composition change over time and the motivation for previous decisions is visible for everyone.

Of course, you can include also other sections in the ADR, as it is fit for your practice. In my case, I preferred to keep them as simple as possible, until we need further sections.

Examples of ADRs, simple and complex, you can find here.

We use ADRs at the organisation level, to record important decisions for our architecture, like “ADR-002-SplitDNS” which states the problem we encountered, the rationale behind the decision of implementing SplitDNS and all the consequences we can foresee. Organisational level ADRs ar related to stuff like how do we publish an API, what is the naming convention for IT, what system is Master Data System for client data, what the client identity management architecture, and so on.

At the same time, I encourage development teams to keep their own internal ADRs for each project (especially in an Agile setup), where we can have a log of all design and architecture decisions for a software project made at team level and the rationale behind them.

Same important is to also document decisions that have not been taken. In my ADRs I usually write also what other options were on the table when the decision was made and the pros and cons for each one and why they weren’t chosen. Acting like this, allow us to later review a decision and if the circumstances have changed significantly, we can take a look at what other options we had, maybe something will fit now. Or maybe is time to come out with some new options.

When to write an ADR

An ADR should be written whenever a decision with a significant impact is made; it’s up to the organisation or team to decide what “significant impact” means. An example of such a decision was that databases for non critic applications will be MariaDB.

Sometimes a decision was made in the past, or an implicit decision was mutually taken but never documented (“we always did it this way”) and it’s not clear what the decision was and what was the rationale behind it. In such a situation, writing an ADR, formalizing an unofficial decision or standard it’s a must.

Whenever we are doing a change to a system (well, not for every system, for the moment only for critical ones) we are making also an impact assessment and document the change and impact in an ADR because it has to be clear for everyone how the system evolved during various changed, how the system was extended and more.

For short, whenever a technical decision is made, which has some impact, document it in an ADR.

Resources

Using OAuth 2.0 to protect APIs published on Azure APIM (API Management Gateway)

As OAuth 2.0 offers different flows, or grant types, to cover multiple authorisation scenarios, it can also be used to protect APIs published on Azure API Management Gateway (APIM). The flow that can be used to allow OAuth 2.0 authorisation between applications is the client credentials flow.

When exposing APIs on Azure APIM, we usually have service to service communication, without any form of user interaction, where APIs are consumed by other applications. The writing below covers how to use the client credentials flow to protect the APIs, how to cofigure Azure APIM with OAuth 2.0 to also allow to the the published APIs from Developer Portal, using OAuth 2.0 authorisation and also C# sample code to connect to an API and authorize using OAuth 2.0.

Prerequsites

Protocol diagram

According to official Microsoft documentation, the following diagram represents the client credentials flow:

OAuth 2.0 client credentials flow

An app typically receives direct authorization to access a resource in one of two ways:

These two methods are the most common in Azure AD and are recommended for clients and resources that perform the client credentials flow. A resource can also choose to authorize its clients in other ways. Each resource server can choose the method that makes the most sense for its application.

Instead of using ACLs, you can use APIs to expose a set of application permissions. An application permission is granted to an application by an organization’s administrator, and can be used only to access data owned by that organization and its employees.

The current setup will contain an Azure APIM instance, a test API published (the out of the box Echo API), a policy for the API and two applications that will be created in Azure AD tenant.

First step is to deploy an Azure APIM instance, using Azure Portal. After deployment is complete, we already have a standard test API, Echo API, published as a test.

For making thing easier on the testing side, I created a simple policy in the inbound processing step to return 200 OK message if the client call reached the API.

<inbound>
        <return-response>
            <set-status code="200" />
            <set-header name="content-type" exists-action="override">
                <value>application/json</value>
            </set-header>
            <set-body>
            {
            "status": "200",
            "message": "OK"
            }
            </set-body>
        </return-response>
        <base />
</inbound>

Using VS Code and Rest Client Extension, writing the following request:

We will get the following result:

Which is perfectly fine, API is alive and responsive.

Now I had to create two application registrations in Azure AD, which enables the creation of the OAuth 2.0 authorisation with client credentials flow. Follow the instructions from Microsoft documentation here, as in the following steps I will need this setup to also enable authorisation for API from Developer Portal of Azure APIM.

I’ve created two apps registration, one representing the API Proxy (backend) and the other one the API Client.

  • ago-apiproxy-oauth-app
  • ago-apiclient-oauth-app

For both applications take note of application ID, as it will be needed later.

Now, for the API client application, a secret needs to be created.

  1. Navigate to the API client app, in my case ago-apiclient-oauth-app.
  2. Navigate to certificates and secrets and add a secret (I kept the default expiration date, as this is just a test setup)
  3. Take note of the generated secret

Now, test the setup in VS Code. In VS Code, these are the settings for the API calls.

Now we are ready to get a JWT token from the token endpoint. The request looks like below (VS Code):

The result of running the above request is a JWT token, like below:

{
  "token_type": "Bearer",
  "expires_in": 3599,
  "ext_expires_in": 3599,
  "access_token": "eyJ0eXAiOiJ........."
}

Next step, as we have the apps registered and can generate a JWT, is to configure Azure APIM to validate the JWT and its claims to see if the client is authorized to call the API.

Some differences from Microsoft documentation:

  • Granting permissions – Above mentioned documentation explains how to grant delegated permissions, which are applicable when we have a signed in user, which is not the case here, when we are dealing with app to app flow. We need to use the application permissions, applicable in app to app scenarios.
  • Enable user authentication in APIM developer portal. If we do that then a some functionalities of the flow will no longer be working for app to app scenario. In the APIM developer portal we are dealing with a signed in user. I will explain later.

Microsoft documentation suggests using a policy in the API inbound processing leg to validate the audience claim (aud)

The above definition means the audience claim should only be used be used to validate that a token was issued targeting our application. This does not imply there are any permissions granted for the caller app.

Now we can test this by adding a validate-jwt policy to the API operation we are testing.

<validate-jwt header-name="Authorization" failed-validation-httpcode="401" failed-validation-error-message="Unauthorized. Access token is missing or invalid.">
            <openid-config url="https://login.microsoftonline.com/common/v2.0/.well-known/openid-configuration" />
            <required-claims>
                <claim name="aud" match="any">
                    <value>api://4a41d081-77b5-4936-b87c-52b76b36dbc2</value>
                </claim>
            </required-claims>
        </validate-jwt>

For the test to work, when we call the OAuth 2.0 protected endpoint, we have to add the Authorization header along with the bearer token generated when we called the token endpoint. The request would look like this:

We should be getting a 200 OK response. As we have not given any permission, yet, to the client app to call the proxy (backend) app, validating only the audience claim is not enough. An approach is to also validate the azp claim to check whether the caller is authorized to call the endpoint.

Note: In https://platform.deloitte.com.au/articles/oauth2-client-credentials-flow-on-azure-api-management is it advised to validate the appid claim. However, this claim seems not to be presented in token, at least not for version 2, hence I’ve tested it for azp claim.

Policy for testing botd aud and azp claims:

<validate-jwt header-name="Authorization" failed-validation-httpcode="401" failed-validation-error-message="Unauthorized. Access token is missing or invalid.">
            <openid-config url="https://login.microsoftonline.com/common/v2.0/.well-known/openid-configuration" />
            <required-claims>
                <claim name="aud" match="any">
                    <value>api://4a41d081-77b5-4936-b87c-52b76b36dbc2</value>
                    <value>4a41d081-77b5-4936-b87c-52b76b36dbc2</value>
                </claim>
                <claim name="azp" match="any">
                    <value>f73ba7f2-5bf9-44b5-b82c-e4b191a80c41</value>
                </claim>
            </required-claims>
        </validate-jwt>

After testing this, we should get a 200 OK response.

A more advanced scenario is when we would add Application Permission for the client app, to fine grain control of what APIs and API operation the app can call. But if we want this setup to also work in Developer Portal of Azure APIM (users being able to try and test the API from the developer portal)P, then the roles claim is no longer working anymore. In an app to app scenario the token contains the roles claim, but in APIM Developer Portal we have an user already signed in (or asked to sign in) and token is generated based on this user so no more roles claim available.

How to configure APIM Developer Portal is described in details in Microsoft documentation, here. As we already created the apps, continue from “Grant permissions in Azure AD” step. Once configured, for settings to take effect in the new portal, it must be published again. The result is showing the Authorisation field in the Developer Portal when you go an try an API.

Now we have the API published on Azure APIM procted by OAuth 2.0 tokens and also, from the developer portal, we can test it with authorization. As the policies are implemented at API level, without configuring APIM with OAuth 2.0, it would not be possible anymore to use the Azure Portal, always getting an authorisation error.

Visual Studio C# code for accessing the API with token authorization

Code is pretty straight forward. To help with REST calls and JSON objects, I’ve used RestSharp and Newtonsoft.JSON packages.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using RestSharp;
using Newtonsoft.Json;
using System.Net.Http;
using System.Net;

namespace PCBQPCAPITokenAuth
{
    class Program
    {
        static void Main(string[] args)
        {
            ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12 | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls;

            Token tok=GetTokenRestSharp();
            Console.WriteLine(tok.AccessToken);

            RestClient client = new RestClient("https://agotestauth.azure-api.net/echo-clone/resource?param1=sample");
          
            RestRequest request = new RestRequest(Method.GET);
            request.AddHeader("Ocp-Apim-Subscription-Key", "your subscrition key");
            request.AddHeader("Authorization", "Bearer " + tok.AccessToken);
            request.AddHeader("cache-control", "no-cache");
            IRestResponse  response = client.Execute(request);

            Console.WriteLine(response.Content.ToString());

            Console.WriteLine("Hit ENTER to exit...");
            Console.ReadKey();
        }

        private static Token GetTokenRestSharp()
        {
            var client = new RestClient("https://login.microsoftonline.com/your azure tennat id/oauth2/v2.0/token");
            var request = new RestRequest(Method.POST);
            request.AddHeader("cache-control", "no-cache");
            request.AddHeader("content-type", "application/x-www-form-urlencoded");
            string gtype = "client_credentials";
            string cid = "f73ba7f2-5bf9-44b5-b82c-e4b191a80c41";
            string csecret = "your api app secret";
            string scope = "api://4a41d081-77b5-4936-b87c-52b76b36dbc2/.default";
            request.AddParameter("application/x-www-form-urlencoded", "grant_type=" + gtype +"&client_id=" + cid + "&client_secret=" + csecret + "&scope=" + scope, ParameterType.RequestBody);
            IRestResponse response = client.Execute(request);
            Token tok = JsonConvert.DeserializeObject<Token>(response.Content.ToString());
            return tok;
        }

        internal class Token
        {
            [JsonProperty("token_type")]
            public string TokenType { get; set; }

            [JsonProperty("expires_in")]
            public string ExpiresIn { get; set; }

            [JsonProperty("ext_expires_in")]
            public int ExtExpiresIn { get; set; }

            [JsonProperty("access_token")]
            public string AccessToken { get; set; }
        }

    }
}

Resources used:

Securing Azure Web Apps – Private Endpoint Part II

Well, everything described in Part I works perfectly fine when your VNETs are using Azure provided DNS Server.

But if you VNETs are using customs DNS servers, as in my case, things get a little tricky. You use custom DNS servers because you are living in a hybrid cloud solution and resources from Azure must also find resources from on premises data centers adn in this case, standard Azure provided DNS servers are of no help.

In this case, when resources are trying to resolve name for your web apps with private integration activated, no DNS server will know about them. And in this case, there at least two options available:

  1. Add a conditional resolver to your custom DNS server, as described here: https://docs.microsoft.com/en-us/azure/virtual-network/virtual-networks-name-resolution-for-vms-and-role-instances#:~:text=The%20Azure%20DNS%20IP%20address,129.16. . An important thing to have in mind in this case is that traffic to Azure resolver with IP 168.63.129.16 should go over an Express Route gateway or VPN gateway, otherwise if coming on the public network it will be rejected.
  2. A simpler approach (tested and working) is to add the IP of the Azure resolver (168.63.129.16) in your list of custom DNS servers for the VNET. Don’t add it on the first position, as this might bring trouble in resolving on premises resources and also not on the third position or lower because this will increase the time needed for name resolution and potentially can throw timeout errors. Adding it on the second position looks like a reasonable compromise.