Installing 1E DataSync Agent

The latest release of the 1E Platform does not require all downloadable components to be updated. Refer to the Release notes for what is new, and what is compatible. You can download tools for the 1E Platform and the 1E Client from the 1E Support Portal.

The DataSync Agent enables your 1E SaaS instance to retrieve data from on-premises sources. Installation and configuration of the 1E DataSync Agent are necessary if you use any of the following components in your 1E Platform instance:


Requirements

You will need the following items before you can start the installation.

1E will provide you with instructions for each agent installed on different servers.

Requirement

Details

SaaS instance

1E must have already provisioned your 1E SaaS instance, and readied the relevant Syncs.

1E will provide you with the following necessary to install the Agent(s) in one or more TXT files:

  • Sync name(s).

  • Client cert token(s).

Host computer(s)

You will need one host computer for each of your Sync use-cases:

  • Content Distribution CMSync to a ConfigMgr SQL Server, this only applies if you have installed the ​DataSyncAgentNomad.txt​​ supplied by 1E.

  • VDX connection to a Citrix SQL Server.

This version of DataSync supports only one Agent per computer. If you need to use a single host computer to support multiple Sync use-cases, then please contact your 1E Account Team, who can work with you to determine the correct number of DataSync computers that you need, their hardware specification, and custom installation steps.

  • A physical computer or a VM with 4 CPU and 8GB RAM should be sufficient for most cases.

  • The Agent can be installed on any Windows version, server or client, as long as it has .NET 8.0 Runtime installed, although in most production scenarios one would prefer a Window Server OS.

  • The computer on which the Agent is installed must be domain joined.

  • .NET 8.0 Runtime must be present on the machine on which the Agent is to be installed. Refer to Download .NET 8.0 Runtime (v8.0.8) - Windows x64 Installer.

  • .NET Framework 4.7.2 or later, the installer has a dependency on this version of the .NET Framework (although the Agent itself does not depend on it).

Coexistence with other on-premises components:

  • 1E Client can coexist with the DataSync Agent on the same host computer.

  • ConfigMgr site server or its SQL Server can coexist with the DataSync Agent, but best practice dictates different purposes should have different hosts.

Installation account permissions

The installation account running the installer must be:

  • Windows domain account.

  • Local admin on the host computer on which the Agent is to be installed.

For every source database from which data will be synced, the installation account must have the following permissions:

  • Have a securityadmin role at the server level and db_accessadmin role at the database level.

  • Alternatively, if they can’t have the securityadmin role or the db_accessadmin role, have the requisite granular permissions to perform the following actions:

    • Create logins in the server.

    • Create users in the database.

    • Grant SELECT and EXECUTE permissions on all objects in the database.

The Agent service runs using the Network Service account. If the installation account has sufficient permissions, the Agent installer creates a SQL Login on the instance, and grants the required permissions on the database:

SQL Server source(s)

The SQL Server instance which hosts the source database must be configured to use SSL certificates. If this is not possible, then please contact your 1E Account Team for help. Refer to Configure SQL Server Database Engine for encrypting connections.

  • You do not need to enable ForceEncryption (force encrypted communications for all clients) therefore this change should not affect your existing operations.

  • You only need to configure the instance with an appropriate SSL certificate (has Subject, and ServerAuthentication as usage). DataSync Agent does not require a client certificate.

  • The Agent communicates with source database(s) on whatever ports they are running on.

  • Ensure the host computer firewall allows access to each SQL Server source(s).

  • The Agent can sync from any version of SQL Server.

Network access to 1E Azure

The Agent communicates with the following hosts in 1E Azure on port 443. Ensure the host computer firewall allows access to each of these Azure hosts.

  • prod-datasyncsvc1e.azurewebsites.net

  • prod-datasyncehns1e.servicebus.windows.net

  • prod-authservice1e.azurewebsites.net

  • dc.applicationinsights.azure.com

  • dc.applicationinsights.microsoft.com

Proxy support

The 1E DataSync Agent supports proxy configurations. During installation, the installer automatically detects the system winhttp proxy settings and updates the configuration accordingly.

Configuration File: %ProgramFiles%\1E\DataSyncAgent\appsettings.json

Proxy Configuration Example:

Copy
"ProxyEnabled": true,
"ProxyUrl": "http://10.20.30.43:80"

All agent communication will be routed through the configured proxy.

You can manually provide the flag to enable the proxy using the installer command-line flag PROXY_ENABLED:

Copy
msiexec /i DataSyncAgent.msi PROXY_ENABLED=1 /qn <rest of command line>

If PROXY_ENABLED=1 is set, the agent will use the system proxy settings.

Preparation

To install the DataSync Agent, you'll need the MSI installer and the following information from 1E. You can download the installer from the 1E Support Portal. The parameters are included in one or more TXT files provided during the SaaS provisioning process. Typically, 1E supplies one TXT file for each Sync use case. All parameters are required.

Installation parameters

Installer property name

Description

DataSync Service URL

DATA_SYNC_API_URL

The URL of DataSync Service URL running in Azure which would normally be:

https://prod-datasyncsvc1e.azurewebsites.net/

Auth Service URL

AUTH_SERVICE_URL

The URL of Auth Service running in Azure used by the Agent to obtain authentication tokens which would normally be:

https://prod-authservice1e.azurewebsites.net/

Client Cert Token

CLIENT_CERT_TOKEN

A token used by the Agent installer to perform the initial setup of authentication for the Agent. This is a unique time-based token that includes encoded details of the Azure authentication certificate, and the specific Sync use-case(s).

Data Source Connection details

SQL_SERVER_DATA_SOURCES

Use the table below to construct the connection details for a specific Sync use-case.

<SyncName>|<DbServerName>|<DbName>

Although the Data Source Connection details field supports multiple sync use-cases, for this version, you should only enter details for one Sync. Use the pipe character | as a delimiter.

Data Source Connection details

Description

SyncName

The SyncName is provided to you by 1E, specific to a single Sync use-case. You should not change this.

DbServerName

The SQL Server instance name, using a format as specified by Microsoft, refer to Logging In to SQL Server.

For example:

  • DbServerFQDN

  • DbServerFQDN\InstanceName

  • DbServerFQDN\InstanceName.port

DbServerFQDN must match the subject name in the SQL Server instance’s SSL certificate.

The installation account must have permission to this SQL Server instance and database, as described in Requirements.

DbName

The SQL Server database name.

The installation account must have permission to this SQL Server instance and database, as described in Requirements.

Interactive installation

To run the installation interactively, log on to the host computer using the installation account, then start the installer.

  1. Run the DatasyncAgent.msi.

  2. On the Welcome screen, click Next.

  3. On the End-User License Agreement screen, check I accept the terms in the License Agreement, and click Next.

  4. On the DataSync API Parameters screen, enter the parameter values you have been provided, and click Next.

  5. On the Data Source Connection Details screen, enter the parameter value that you constructed earlier, and click Next.

  6. On the Ready to install 1E DataSync Agent screen, click Install to begin the installation.

  7. On completion, click Finish to close the installer.

  8. The DataSync service will be installed as 1E DataSync Agent and run under the Network Service account.

Silent installation

  1. To run the installation silently, log on to the host computer using the installation account.

  2. Download the latest DataSyncAgent.msi.

  3. Start the installer using a command line similar to below. You should have been given a TXT file with the command line you need for installing a single DataSync Agent supporting a single Sync. However, you must first reconstruct the SQL_SERVER_DATA_SOURCES parameter by replacing the placeholder text using the guidance in Preparation.

    Copy
    msiexec.exe /i DataSyncAgent.msi /l*vx DataSyncAgent.install.log /qn DATA_SYNC_API_URL=https://prod-datasyncsvc1e.azurewebsites.net/ AUTH_SERVICE_URL=https://prod-authservice1e.azurewebsites.net/ CLIENT_CERT_TOKEN=<token> SQL_SERVER_DATA_SOURCES="CMSync|ACME-CM01.acme.local|CM_CAS" PROXY_ENABLED=1

    The installer will automatically detect and configure your system proxy.

    If your environment uses a proxy server, it's recommended you use a command like the example to enable proxy server mode for the Data Sync Agent during installation. Otherwise, the traffic might not recognize the proxy settings.

  4. The service will be installed as 1E DataSync Agent and run under the Network Service account.

Upgrade

To upgrade an existing installation:

  1. Ensure the current DataSync Agent is running.

  2. Run the upgrade command using the relevant command line (No token required). For example:

    Copy
    msiexec.exe /i DataSyncAgent.msi /l*vx DataSyncAgent.install.log DATA_SYNC_API_URL=https://prod-datasyncsvc1e.azurewebsites.net/ AUTH_SERVICE_URL=https://prod-authservice1e.azurewebsites.net/ SQL_SERVER_DATA_SOURCES="CMSync|ACME-CM01.acme.local|CM_CAS" PROXY_ENABLED=1
  3. The upgrade will keep your existing settings, and no token is required.

Verify your installation

  1. Check if the 1E DataSync Agent service is running:

    Copy
    Get-Service -Name "1E DataSync Agent"
  2. Verify your proxy settings in: %ProgramFiles%\1E\DataSyncAgent\appsettings.json

  3. Confirm certificate validity:

    Copy
    $certPath = "C:\ProgramData\1E\DataSyncAgent\Certs\ClientCert.p12"
    $cert = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2
    $cert.Import($certPath)
    $cert.NotAfter

Post installation

  1. Once the installer finishes it will have created a Windows service for the Agent called 1E DataSync Agent.

  2. The service will start immediately and start sending data from the specified sources.

  3. The first sync is full, afterward only differences are synced. The Agent maintains a snapshot for each dataset at:

    %ALLUSERSPROFILE%\1E\DataSyncAgent\Snapshots\{syncname}\{dataset}

Review Agent logs

You should review the installation and service logs for errors. Unless otherwise specified on the command-line, the Agent installer creates an installation log in the %temp% folder of the logged on user. The DataSync Agent service log file is created by default at:

%ProgramData%\1E\DataSyncAgent\Logs\DataSyncAgent.log

The log shows details of each Sync cycle:

  • The frequency of each cycle depends on the Sync use-case. The cycle starts when the Agent is started.

  • Within each cycle, there are pairs of entries for each data set, saying Syncing data set and Done syncing data set in HH:MM:SS.SSSSSSSS.

  • Syncing data set does a SQL query then uploads events. If there is an issue with the SQL query, an entry is displayed showing the query and the error, and the process moves onto the next data set.

  • Done syncing indicates events for the data set have been uploaded, and either says No events sent if no new data, or lists the types of new events (Truncates, Inserts, Updates, Deletes, SnapshotCompletes).

Device grooming

The 1E Platform can execute a device grooming task that removes devices which haven't connected to the platform within a specified number of days (99 days by default). When devices are groomed, a deletion event is triggered. To utilize this feature, you need to request its activation for your platform instance, with values tailored to your organization's needs.