Accessing the Azure Key Vault using REST and oAuth

This article describes how to access the Azure Key Vault using standard REST APIs and oAuth bearer tokens, and not through specific Microsoft-provided DLLs that are available in the .NET environment. This is important because non-.NET environments (such as Node.js) can’t directly leverage these .NET assemblies.

The material on obtaining oAuth bearer tokens is generally applicable outside the key vault. You use bearer tokens to access any of the Azure Graph APIs, so the processes for obtaining them and the interactions with Azure RBAC hold for anything you might want to do in Azure, not just relating to the key vault.

oAuth and the key vault

You can access the key vault by obtaining a bearer token through an oAuth grant flow or by leveraging a managed identity to get a bearer token, and then calling the appropriate APIs.

The example below will walk you through the creation of a key vault in an Azure AD tenant (in this case, the Corellian Engineering test tenant) and then the granting of appropriate scopes and roles to allow access to the key vault.

Obtaining an X509 certificate object from the key vault

Once you have successfully authenticated with the key vault, you may want to get the full X509 certificate object out of it. To learn how to do this, refer to Retrieving an X509 certificate from the Azure Key Vault.

Key steps

  • Access is granted by performing an oAuth grant flow against a registered application in the Azure AD tenant, just as for other authentication operations to obtain a bearer token.

  • If the calling process has been assigned a managed identity, it can gain a bearer token automatically. Refer to Using a managed identity to obtain an oAuth bearer token.

  • If the calling process is not associated with a managed identity, then it must specify a registered application on the identity provider against which the grant flow will be performed.

    This will be a non-interactive grant flow, thus, the calling process must supply either a shared secret or a signed JWT in order to authenticate. Normally, a shared secret would be preferred due to the simplicity of the solution.

    The registered application must be granted rights to the key vault. In the example, full administrator rights are granted but Least Privilege has not yet been explored to determine an appropriate more constrained set of rights.

  • The bearer token requested must request a special scope, https://vault.azure.net/.default, in order to access the key vault.

  • The registered application, as a service principal, must have been granted appropriate rights to access the key vault from within the key vault identity management (IAM) screen.

  • As discussed above, if the VM or other SaaS environment has a managed identity assigned to it, then an oAuth bearer token can be automatically obtained without going through a grant flow. Once you have the oAuth bearer token, you can use it exactly the same way as one obtained through a grant flow.

When using shared secrets rather than managed identities, you must take precautions to protect the secret if you are planning on building a production solution around this approach. This typically relies on file permissions to lock down access to the secret to only a specific account or group. Never encode secrets directly in configuration files or pass them on on command lines.

Grant flows

Normally, you would obtain a bearer token using a non-interactive grant flow, for which there are two options:

  • Using a signed JWT

  • Using a shared secret

Bearer tokens retrieved using a managed identity don’t use oAuth grant flows. Instead, a direct request is made to an IPv4 address using HTTP. This is discussed in more detail further down.

For the first of these options, we end up with a scenario where we are using a certificate to effectively gain access to other certificates. However, we can’t place the JWT signing certificate in the Azure Key Vault because we would then be in a "chicken and egg scenario": we need to access the certificate to sign the JWT to gain access to the vault.

If the calling process does have access to a local certificate store that provides security over certificate access (such as the Windows certificate store or the emulated certificate store available in .NET Core running on non-Windows platforms), then certificate-based oAuth grant flows are always more secure because access to the certificate private key can be restricted to the calling account, and no process can exfiltrate the private key.

If you are using a shared secret from a Windows endpoint, you can choose to store the secret in the Windows Credential Store (available to both .NET-based and standard Win32 API callers) or the Windows Credential Vault (available to .NET-based callers only). This ensures that the secret is stored encrypted at rest, and that only the specified account can retrieve it.

Finally, if the environment is SaaS-hosted and can be assigned a managed identity, this is the most appropriate solution since a bearer token can be automatically obtained by calling an unsecured endpoint inside the environment.

Example

In the example below, we will create and access a key vault in the Corellian Engineering tenant. If you have a Visual Studio subscription, you can use your Azure free credits to create a key vault in your Azure subscription.

We will then use a managed identity in a Visual Studio subscription to obtain an oAuth bearer token instead.

Creating a key vault

Refer to Creating an Azure Key Vault.

Note that this process discusses creating a Managed Identity and then granting that identity access to the vault. While this is appropriate for managed identity access, access via oAuth using a certificate or shared secret does not require a managed identity, although it certainly doesn’t hurt to set one up.

Vault URI

Note the Vault URI on the Overview page in the Key Vault section of the portal.

Creating or modifying a registered application

We will be requesting a bearer token from a registered application on Azure. This application must be granted additional API rights from the default.

In this example, an existing application was already set up for non-interactive authentication, and we added the new Azure Key Vault API permission user_impersonation.

This application was already set up to allow certificate-based authentication using the corellian.pfx certificate whose public key has already been uploaded.

We will initially authenticate using the Corellian certificate to get a bearer token, although this is not ideal, as stated above. We will add a shared secret later on and then demonstrate how to authenticate using that instead.

Adding the application as a service principal to the key vault

From the IAM assignment page in the key vault, click Add and select Add Role Assignment.

On the role assignment page, select Key Vault Administrator and then click Members.

This is not consistent with Least Privilege but will suffice for the example.

Click Select Members and type in part of the name of the registered application to locate as.

Click Review + assign to add the application as a service principal.

Repeat the above to add yourself to the Key Vault Administrator role. Otherwise, despite being the owner, you won’t be able to upload a certificate to it.

Uploading a certificate to key vault

Upload a PFX file to the vault. In this example, we use the Corellian.pfx certificate.

You will need the private key password of the certificate when uploading it.

Click Certificates from the key vault and then Generate/Import.

Click Import, then browse for the file and select it. Enter the private key password and click Create.

You should now see the certificate in the vault.

Obtaining a bearer token to access the key vault

We will use the PowerShell Toolkit to obtain a bearer token because it can perform oAuth grant flows for us. These are just simple REST calls, so we can abstract out the oAuth calls to use on other platforms if we wish.

You must use version 1.2.9 or later of the Toolkit, as this version supports the custom scope required for this bearer token.

Note that this version also has key vault access via managed identities and shared secrets baked in to key cmdlets (based on the techniques discussed on this page), but we are going to perform the REST calls ourselves in these examples to understand what’s happening behind the scenes.

Import the Toolkit in the normal way.

Copy 
import-module .\ps1etoolkit.psd1 -force

We are initially going to request a bearer token from the application using a signed JWT using the Corellian certificate. This is not ideal because we probably want to store all our certs in the vault and use, for example, a shared secret instead, but since most people are already familiar with this grant flow, we’ll show how to do it this way first, and we will then go back and set up a shared secret.

Obtaining the metadata endpoint

We first need to know the Azure metadata endpoint in order to get a bearer token.

Requesting a bearer token

We will store the token in a variable $t in PowerShell so we can easily get it back when we need it

The bearer token only has a one-hour lifetime, after which the grant flow needs to be repeated to get a new one. Using the 1E PowerShell Toolkit is just a convenience to demonstrate the process of accessing the key vault, and in production code you will have code to automatically get a fresh token as required.

We specify the application id of the registered application we set up earlier, and we assume we have the Corellian certificate in our local machine personal certificate store. Note that we request a non-standard scope of https://vault.azure.net/.default. Without this scope, the bearer token would not be allowed to access the key vault.

Copy 
$t = get-1ebearertoken -appid 3f752837-3303-4d0e-9e39-5757dda60aa2 -certsubject cn=corellian -metadataendpoint https://login.microsoftonline.com/81f2b2d5-56db-4d75-86d1-b64974bd35bd/v2.0/.well-known/openid-configuration -scope https://vault.azure.net/.default

Using the bearer token

We will now enumerate the certificates in the vault by making a REST GET call using the standard curl utility from a Windows command prompt.

You could also do this using PowerShell’s invoke-webrequest. Note that curl is an alias for this inside a PowerShell session, which is confusing because the command arguments aren’t the same as standard curl, so the example below is assumed to be run from a command prompt, not a PowerShell session.

Copy and paste the bearer token you get by sending it to your PowerShell window.

Copy 
$t.access_token

In your command prompt session, call the Azure API to enumerate certificates in the vault.

Note that we add a header with the standard authorization: bearer value followed by the bearer token. We call the URL that you noted earlier when the key vault was created.

Copy 
curl https://ajmkeyvaulttest.vault.azure.net/certificates?api-version=7.4 -H "authorization: bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6IjlHbW55RlBraGMzaE91UjIybXZTdmduTG83WSIsImtpZCI6IjlHbW55RlBraGMzaE91UjIybXZTdmduTG83WSJ9.eyJhdWQiOiJodHRwczovL3ZhdWx0LmF6dXJlLm5ldCIsImlzcyI6Imh0dHBzOi8vc3RzLndpbmRvd3MubmV0LzgxZjJiMmQ1LTU2ZGItNGQ3NS04NmQxLWI2NDk3NGJkMzViZC8iLCJpYXQiOjE2OTkzNjUxMzksIm5iZiI6MTY5OTM2NTEzOSwiZXhwIjoxNjk5MzY5MDM5LCJhaW8iOiJFMlZnWUhqd04vZ3AvMG0vcXpwdHR5ZjlTK1JaQ1FBPSIsImFwcGlkIjoiM2Y3NTI4MzctMzMwMy00ZDBlLTllMzktNTc1N2RkYTYwYWEyIiwiYXBwaWRhY3IiOiIyIiwiaWRwIjoiaHR0cHM6Ly9zdHMud2luZG93cy5uZXQvODFmMmIyZDUtNTZkYi00ZDc1LTg2ZDEtYjY0OTc0YmQzNWJkLyIsIm9pZCI6IjNlYWIyYmI0LTRmZWQtNGM5NC1iMjBiLTk0NGY4Y2Y0MGJhMiIsInJoIjoiMC5BWGtBMWJMeWdkdFdkVTJHMGJaSmRMMDF2VG16cU0taWdocEhvOGtQd0w1NlFKT1VBQUEuIiwic3ViIjoiM2VhYjJiYjQtNGZlZC00Yzk0LWIyMGItOTQ0ZjhjZjQwYmEyIiwidGlkIjoiODFmMmIyZDUtNTZkYi00ZDc1LTg2ZDEtYjY0OTc0YmQzNWJkIiwidXRpIjoiXzZBSXQ2N2tURXVzM2FkLW1lNGRBQSIsInZlciI6IjEuMCJ9.pukCyM3w4zl2-jkOjJJ1HUw5Itc8C3PpcOVnC8qgpPGPG75dunE4YJh16hX9lzfc6xPaZtSRxVLol-b82YWC2gdQSubz7Yqw68eLs8TF1gc8B6sHRlLjHlLAfWClx_8qaRQYC4c3sKnPNs0OFrP9IulOzuEVDzFVqDX1XFAZQqZ8NiG6CCuUVrzwapEg-FcUxI70hpRYYXakKPUfb80q5v-ZTSZIVlXggJAes30wcnkjhpVpi_IVuwxxOF3EvFgPjpg2qZRVutHSc2mr0yHVzSPmABB2pTLzPtq0UZIGUYpXZ-DKraqiRc0KTxLy73eKbfX9usq86e9mHn8WIZMoow"

We see the certificate that we uploaded earlier enumerated successfully.

Copy 
{"value":[{"id":"https://ajmkeyvaulttest.vault.azure.net/certificates/Corellian","x5t":"bMF28pA5tCA7UmjS6f2XS7Af8xo","attributes":{"enabled":true,"nbf":1681199253,"exp":1712822853,"created":1699364566,"updated":1699364566},"subject":""}],"nextLink":null}

Using a shared secret

To avoid the circular reference of using a certificate to obtain certificates, we’ll add a shared secret to the registered app and then perform a grant flow using that secret instead.

Adding a shared secret to the app

From the Certificates & secrets page of the application, click to add a new client secret.

Provide a secret name. Note that by default, client secrets have a six-month lifetime. You must refresh them within that period.

The secret has been created. Copy the value to the clipboard.

Now request a bearer token using the PowerShell Toolkit, but this time, supply the secret instead of a certificate. Note that in the example below, part of the secret has been redacted.

Copy 
$t = get-1ebearertoken -appid 3f752837-3303-4d0e-9e39-5757dda60aa2 -secret bxj8Q~...(redacted) -metadataendpoint https://login.microsoftonline.com/81f2b2d5-56db-4d75-86d1-b64974bd35bd/v2.0/.well-known/openid-configuration -scope https://vault.azure.net/.default

You should now be able to query the key vault with this bearer token just as you did previously.

Make sure you note the secret and put it somewhere safe as once you close the above screen, there’s no way to get it back again from Azure.

Using a managed identity to obtain an oAuth bearer token

Managed identities can be assigned to a variety of SaaS-hosted resources, such as VMs. In this scenario, we will use a VM in a Visual Studio subscription that has been assigned a user-defined managed identity.

Managed identities are effectively service principals that get allocated to entities in the Azure environment such as guests (VMs). In that scenario, any process running on that VM inherits the managed identity and can leverage this to obtain an oAuth bearer token automatically without providing any kind of credentials.

This is done by communicating with an Azure subsystem known as the Azure Instance Metadata Service. This is discussed further below.

Some Azure subsystems and their associated entities, such as Azure Functions, can also be associated with managed identities. However, they may not be able to obtain a bearer token from the Azure Instance Metadata Service directly. This isn’t well documented but always remember that you can use shared secrets to get a bearer token whether or not you have a managed identity to potentially leverage.

Granting the managed identity access to the key vault

For information on how to create a key vault, refer to Creating an Azure Key Vault. Assuming in this example that the key vault is AJMKeyVault (and not AJMKeyVaultTest as in the Corellian environment), and that the associated managed identity is AJMManagedIdentity, firstly take a note of the client Id of the managed identity as shown below.

Obtaining and using a bearer token from within a resource associated with the managed identity

Assume we have a VM which has been assigned that managed identity. From within the VM, issue the following PowerShell command. As with the 1E PowerShell Toolkit cmdlet we ran earlier, this will return a token in $t.access_token.

Copy 
$t = Invoke-RestMethod -Uri 'http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&client_id=4022133b-af0a-4c19-bf27-64f68db5aae5&resource=https%3A%2F%2Fvault.azure.net' -Method GET -Headers @{Metadata="true"}

As you can see, an unsecured HTTP request is made to a local endpoint which is automatically mapped into the calling process’s network space. This will return a bearer token directly, given the client id of the managed identity.

This endpoint is part of a set of services known as the Azure Instance Metadata Service.

This endpoint may not be available within the context of an Azure Function, or at least one running under standard consumption mode. This is not well documented however. If you want to retrieve something from the key vault in an Azure Function, you are probably going to have to use a shared secret rather than a managed identity.

We can also use a command prompt with curl.

Copy 
curl "http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&client_id=4022133b-af0a-4c19-bf27-64f68db5aae5&resource=https%3A%2F%2Fvault.azure.net" -H "Metadata: True"

This should work in linux as well as Windows; however, note that you may need to install curl on linux. It is included in all modern versions of Windows automatically.

This should return an oAuth bearer token just as with the previous grant flows, but in this case, we didn’t need to provide any secrets or certificates because the VM’s managed identity vouched for us.

We can now, as we did previously, enumerate the certificates in the key vault using the same API. This time, we’re going to use ajmkeyvault.vault.azure.net instead.

Copy 
curl https://ajmkeyvault.vault.azure.net/certificates?api-version=7.4 -H "authorization: bearer <bearer token>