Databricks integration

This article discusses using the 1E PowerShell Toolkit to obtain an access token that allows external integrators to retrieve data from Databricks.

To learn about the 1E PowerShell Toolkit cmdlets used for Databricks token management, refer to Platform server management cmdlets.

About Databricks

Databricks provides data storage and retrieval for 1E platform data that is collected over an extended time period. This data includes, for example, performance information relating to endpoints.

It can often be useful to access this data outside of the 1E platform itself. The 1E PowerShell Toolkit allows you to obtain an access token through which you can then retrieve data.

Databricks tokens and RBAC access rights

• You require write access to the securable type DataAccess to be able to create a Databricks token. Write permission is also requested to enable and disable the Data Access feature.

• You require write access to the securable type DataAccess to request the token activation URL. Requesting the activation URL appears to be a read operation, but the activation URL is used to modify the token state from pending to active. For this reason, requesting the activation URL is a write operation.

• You require read access to the securable type DataAccess to be able to retrieve information about a Databricks token.

As with all RBAC access rights, having write access does not imply read access. By default, a full administrator has both read and write operations for Data Access.

Enabling data access

You must enable data access for a tenant before you can create and retrieve tokens. Use the following cmdlet to enable data access:

Enable-1EDataAccess

Note that this operation may take some time and the process occurs asynchronously. When the feature is enabled, the 1E Platform will enable the sharing of data in two main steps:

1. It provisions the required objects to enable the sharing of data.

2. It copies data to be shared.

When run, this cmdlet will wait up to 60 seconds for step 1 to complete. Once step 1 is complete, new tokens can be created. However, note that tokens will only have access to data once step 2 completes. Tokens created before step 2 completes will have access to data once both steps complete.

Currently, the data copy operation occurs at midnight (UTC) and, depending on the amount of data, may take multiple hours to complete.

The Get-DataAccess cmdlet can be used to query the state of either the feature or the data sync operations.

Disabling data access

You can disable data access for a tenant using the following cmdlet:

Disable-1EDataAccess

This cmdlet will wait up to 60 seconds before returning to ensure that feature disablement is complete.

When disabling the feature, all existing tokens are first deleted. Once all tokens are deleted, the feature is disabled. Note that once a token is deleted, it cannot be used to access data. If the feature is later re-enabled, new tokens must be created.

Checking the status of Databricks access

You can verify whether the Databricks access feature is enabled or disabled using the following cmdlet:

Get-DataAccess -Status Feature

The output of -Status Feature is one of the following:

You can verify whether data is being synchronized correctly from the 1E Platform to Databricks using the following cmdlet:

Get-DataAccess -Status DataSync

You can only query the data sync status if the feature is enabled. This will return the successful sync operations for the last seven days. For example:

1
2
3
4
5
6
7
8
9
StartTimeUTC             FinishTimeUTC
------------             -------------
2025-07-21T00:00:02.739Z 2025-07-21T09:56:26.781Z
2025-07-20T00:00:02.687Z 2025-07-20T07:28:21.715Z
2025-07-19T00:00:01.772Z 2025-07-19T06:43:46.449Z
2025-07-18T00:00:02.884Z 2025-07-18T07:07:07.001Z
2025-07-17T00:00:03.153Z 2025-07-17T07:20:20.671Z
2025-07-16T00:00:01.722Z 2025-07-16T07:12:22.943Z
2025-07-15T00:00:03.651Z 2025-07-15T06:37:15.704Z

Token lifecycle

The 1E Platform is used to manage tokens and allows a user to create, list, and delete tokens. Each token has one of the following states:

All new tokens are created in the pending state. These tokens are not currently usable, and must first be activated to access data. To activate a token, the activation URL must be retrieved. The activation URL allows a user to download a credential file which provides access to data. The credential file can be downloaded only once. The token is considered active as soon as the credential file has been downloaded. To learn how to create and use a token, refer to Using tokens to retrieve Databricks data.

Creating a new access token

To create a new access token, use the Add-1EDataAccessToken cmdlet. For example:

Add-1EDataAccessToken -Name MyDataAccessToken -Description "An example description for the new token" -ExpirationTimeUTC 2025-10-05T12:00Z

Token names must be unique. You will receive an error if you attempt to create a token with the same name more than once. The name check is case-insensitive.

A token can have a description and an expiry datetime. A token that does not have an expiry datetime will use the default setting configured within Databricks, which is currently one year. Once the expiry datetime is reached, the token will cease to be valid.

Removing an access token

To delete an access token, use the Remove-1EDataAccessToken cmdlet. For example:

Remove-1EDataAccessToken -Name MyTokenName

When you do this, the token immediately becomes invalid.

Retrieving information on active access tokens

You can retrieve information on all active access tokens using the Get-1EDataAccessToken cmdlet.

Get-1EDataAccessToken

Running this cmdlet without a specific name will retrieve all tokens. The Name parameter can be used to return details for a specific token.

Retrieving a specific access token

You can retrieve details about a specific token using the Get-1EDataAccessToken cmdlet. This can be used to return the current state of a token and its activation URL. The activation URL allows the activation and download of the token.

To get the state of a specific token, run the following:

Get-1EDataAccessToken -Name MyTokenName

To get the activation URL of a token in the pending state, run the following:

Get-1EDataAccessToken -Name MyTokenName -ActivationUrl

You can only retrieve the activation URLs for tokens in the pending state. An error is returned for tokens in a different state.

About the activation URL

The activation URL is an internal Databricks URL, which is used to actually retrieve the token. To retrieve the actual token (credential file) you need to copy/paste the activation URL into a browser. This will give the option to download the credential file. Once the file has been downloaded, the token is activated. The download of the credential file is a one-time operation. Once downloaded, the token is considered active. The same activation URL cannot be used to download the credential file again. Once the token has been activated, the cmdlet to retrieve the activation URL will return an error.

Using tokens to retrieve Databricks data

The steps below show how to create, activate, and use a token. The example below shows how to access the data in PowerBI Desktop. The token can be used in other applications. Note that the activation URL provides guides for a number of different applications including PowerBI Desktop.

Creating a token

The following command creates a token called Demo which expires on July 31, 2025.

Add-1EDataAccessToken -Name Demo -ExpirationTimeUTC 2025-07-31T00:00Z

The result of the above is as follows. Note that the status is 1 (pending).

1
2
3
4
5
6
Name              : Demo
Description       :
Status            : 1
CreatedBy         : DavidTestUser1@1esaasplatform.onmicrosoft.com
CreatedAtUtc      : 2025-07-21T14:21:19.963Z
ExpirationTimeUtc : 2025-07-31T00:00:00.0Z

Activating the token

The pending token needs to be activated. The process of activating the token will result in a credential file which can be used in PowerBI. Get the activation URL by running the following:

Get-1EDataAccessToken -Name Demo -ActivationUrl

The result of the above is the following:

https://ukwest.azuredatabricks.net/delta_sharing/retrieve_config.html?mD90nNVbjnO0N5IGsyfDEvhqrBoTpbhjJhp0usqch_vZ_iRG8lJTjy4yyk_sEqxb

Opening the activation URL in a browser displays the following:

Click Download Credential File to download the file containing the required bearer token and the URL to access the data in PowerBI. The Access this share in section provides links for guides on how to use the credential file.

An example of the content of a downloaded credential file is shown below.

1
2
3
4
5
6
{
    "shareCredentialsVersion":1,
    "bearerToken":"8XvJ7Mka8lkrkiZuEMWcoeFaRmmI5uBeJb5Esl0ybTd7TmLTXv-stahFLLqEzufN",
    "endpoint":"https://ukwest.azuredatabricks.net/api/2.0/delta-sharing/metastores/591653a3-eed6-42f3-b3d6-874ba3f9be24",
    "expirationTime":"2025-07-31T00:59:59.964Z"
}

Note that the download button is now disabled.

Additionally, getting the token via the Get-1EDataAccessToken cmdlet shows the token has now been activated. The status is now 2 (activated).

Get-1EDataAccessToken -Name Demo

1
2
3
4
5
6
Name              : Demo
Description       :
Status            : 2
CreatedBy         : DavidTestUser1@1esaasplatform.onmicrosoft.com
CreatedAtUtc      : 2025-07-21T14:21:19.963Z
ExpirationTimeUtc : 2025-07-31T00:00:00.0Z

The activation URL can no longer be retrieved.

1
Get-1EDataAccessToken -Name Demo -ActivationUrl

1
2
3
4
5
6
Invoke-RestMethod : [{"ErrorCode":"DataAccess.NoTokenActivationUrlAvailable","Message":"No token activation URL is available.","Data":[]}]
At C:\Development\Platform\working\Server\Code\tools\PowershellToolkit\PS1EToolkit.psm1:2272 char:22
+ ...      $res = Invoke-RestMethod -Method $verb -Uri $fullurl -Header $hd ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebException
    + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand

Using the credential file in PowerBI

When downloading the credential file, the Access this share in section provides this Microsoft link for PowerBI. The steps are provided below, but as PowerBI is under development these may change. Please refer to the link provided on the Activation URL screen for up-to-date steps.

The steps below will require the credential file downloaded above. The example is shown again below.

1
2
3
4
5
6
{
    "shareCredentialsVersion":1,
    "bearerToken":"8XvJ7Mka8lkrkiZuEMWcoeFaRmmI5uBeJb5Esl0ybTd7TmLTXv-stahFLLqEzufN",
    "endpoint":"https://ukwest.azuredatabricks.net/api/2.0/delta-sharing/metastores/591653a3-eed6-42f3-b3d6-874ba3f9be24",
    "expirationTime":"2025-07-31T00:00:00.0Z"
}

You will need the endpoint on line 4 and the bearerToken on line 3.

1. Run PowerBI Desktop and select Get data from other sources. In the Get Data dialogue, search for Delta Sharing. You can type in the search box to easily find Delta Sharing.

   

   

2. Click Delta Sharing and then click Connect.

3. Enter the Delta Sharing Server URL. Enter the endpoint field from the credential file as the Delta Sharing Server URL.

   

4. Click OK.

5. On the next screen, select Authentication and paste the bearerToken field from the credential file as the Bearer Token.

   

   The screenshot above shows a We couldn’t authenticate…. warning. This occurred as the same Delta Sharing Server URL was previously used on this machine and PowerBI had cached the old bearer token.

6. Click Connect. PowerBI should now connect to Databricks via Delta Sharing. The first screen shown should be Navigator, and you can see below the tables being shared with PowerBI.

   

7. Clicking one of the tables will load a small sample set of data, indicating that it can load data successfully.

   

Troubleshooting

Cannot enable the feature

The Data Access feature is a separate licensable feature. If this feature is not currently licensed, you will not be able to enable the feature.

Enable-1EDataAccess

If the feature is unlicensed, you will receive the following error.

1
2
3
4
5
6
Invoke-RestMethod : {"ErrorCode":"Licensing.FeatureNotLicensed","Message":"Feature SelfService is not licensed.","Data":["SelfService"]}
At C:\Development\Platform\working\Server\Code\tools\PowershellToolkit\PS1EToolkit.psm1:2272 char:22
+ ...      $res = Invoke-RestMethod -Method $verb -Uri $fullurl -Header $hd ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebException
    + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand

If the feature is licensed, it may take some time to enable it. The Enable-1EDataAccess cmdlet should enable the feature and then wait until the resources required to create tokens are provisioned. Once complete, the cmdlet should return 1 (enabled). This cmdlet will wait up to one minute for this to occur, and in some cases this may not be long enough. The cmdlet may return before the feature is enabled, in which case it returns 3 (enabling). If this occurs, re-running Enable-1EDataAccess will result in an error as the feature is already in the process of being enabled. To get the current state you can run the following:

Get-1EDataAccess -Status Feature

This will return whether the feature is still being enabled (3) or if the feature is enabled (1). Once this returns enabled (1), you will be able to create tokens.

Unable to create a token

To create a token, the Data Access feature must be licensed and enabled. Note that when licensed, the feature is disabled by default, so you need to ensure the feature is enabled first. If the feature is unlicensed or disabled, an appropriate error will be returned when creating a token.

For example, trying to create a token when the feature is disabled:

Add-1EDataAccessToken -Name "MyToken"

The response is the following:

1
2
3
4
5
6
Invoke-RestMethod : [{"ErrorCode":"DataAccess.DataAccessIsNotEnabled","Message":"Data Access feature is not enabled. Current state is disabled.","Data":["disabled"]}]
At C:\Development\Platform\working\Server\Code\tools\PowershellToolkit\PS1EToolkit.psm1:2276 char:25
+ ...      $res = Invoke-RestMethod -Method $verb -Uri $fullurl -Header $hd ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebException
    + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand

To check if the feature is enabled, you can run the following:

Get-1EDataAccess -Status Feature

This will return 1 if the feature is enabled.

Also note that token names must be unique. The uniqueness check is case-insensitive. If you try to create a token that clashes with an existing name, an error is returned.

For example, creating a token with the name MyToken will succeed if the name is unique:

Add-1EDataAccessToken -Name "MyToken"

1
2
3
4
5
6
Name              : MyToken
Description       :
Status            : 1
CreatedBy         : DavidTestUser1@1esaasplatform.onmicrosoft.com
CreatedAtUtc      : 2025-07-22T09:52:38.413Z
ExpirationTimeUtc : 2026-07-22T09:52:38.415Z

If we now try and create another token called mytoken:

Add-1EDataAccessToken -Name "mytoken"

1
2
3
4
5
6
Invoke-RestMethod : [{"ErrorCode":"DataAccess.TokenAlreadyExists","Message":"Token with the name 'mytoken' already exists.","Data":["mytoken"]}]
At C:\Development\Platform\working\Server\Code\tools\PowershellToolkit\PS1EToolkit.psm1:2276 char:25
+ ...      $res = Invoke-RestMethod -Method $verb -Uri $fullurl -Header $hd ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebException
    + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand

Activated tokens cannot access data

If the feature is enabled, you will be able to create tokens. However, when the feature is first enabled, it may take some time before the data is available. When the feature is enabled, two main steps are performed:

1. It provisions the required objects to enable the sharing of data.

2. It copies data to be shared.

As soon as step 1 is complete, you can create tokens. Data will only be available when step 2 completes. If tokens are created before step 2 completes, they are still valid and usable, but you must wait for step 2 to complete before they have access to data.

To check if step 1 is complete, you can run the following. This checks if the feature is enabled, and if it is, you can create tokens.

Get-1EDataAccess -Status Feature

If this returns 1 (enabled), you can create tokens and step 1 is complete.

To check if step 2 is complete, you can run the following. This returns the successful data sync operations. If rows are returned, step 2 has completed, and data will be available.

Get-1EDataAccess -Status DataSync

This will return zero rows when no data is available. If at least one row is returned, you should have access to the data. For example:

1
2
3
StartTimeUTC             FinishTimeUTC
------------             -------------
2025-07-22T00:00:02.739Z 2025-07-22T09:56:26.781Z