Installing 1E DataSync Agent

The DataSync Agent is the means by which your 1E SaaS instance gets data from your on-premises data sources. You will need to install and configure the 1E DataSync Agent if you use any of the following in your 1E Platform instance.


Requirements

You will need the following items before you can start the installation.

1E will provide you with instructions for each agent installed on different servers.

Requirement

Details

SaaS instance

1E must have already provisioned your 1E SaaS instance, and readied the relevant Syncs.

1E will provide you with the following necessary to install the Agent(s) in one or more TXT files:

  • Sync name(s).

  • Client cert token(s).

Host computer(s)

You will need one host computer for each of your Sync use-cases:

  • Content Distribution CMSync to a ConfigMgr SQL Server, this only applies if you have installed the ​DataSyncAgentNomad.txt​​ supplied by 1E.

  • VDX connection to a Citrix SQL Server.

This version of DataSync supports only one Agent per computer. If you need to use a single host computer to support multiple Sync use-cases, then please contact your 1E Account Team, who can work with you to determine the correct number of DataSync computers that you need, their hardware specification, and custom installation steps.

  • Hardware: A physical computer or a VM with 4 CPU and 8GB RAM should be sufficient for most cases.

  • Operating System: The Agent can be installed on any Windows version, server or client, as long as it has .NET 8.0 Runtime installed, although in most production scenarios one would prefer a Window Server OS.

  • Domain membership: The computer on which the Agent is installed must be domain joined.

  • .NET 8.0 Runtime : Must be present on the machine on which the Agent is to be installed. Download and install it from Download .NET 8.0 Runtime (v8.0.8) - Windows x64 Installer.

  • .NET Framework 4.7.2 or later: The installer has a dependency on this version of the .NET Framework (although the Agent itself does not depend on it).

Coexistence with other on-premises components

  • 1E Client - can coexist with the DataSync Agent on the same host computer.

  • ConfigMgr site server or its SQL Server - can coexist with the DataSync Agent, but best practice dictates different purposes should have different hosts.

Installation account permissions

The installation account running the installer must be:

  • Windows domain account.

  • Local admin on the host computer on which the Agent is to be installed.

For every source database from which data will be synced, the installation account must have the following permissions:

  • Have a securityadmin role at the server level and db_accessadmin role at the database level.

  • Alternatively, if they can’t have the securityadmin role or the db_accessadmin role, have the requisite granular permissions to perform the following actions:

    • Create logins in the server.

    • Create users in the database.

    • Grant SELECT and EXECUTE permissions on all objects in the database.

The Agent service runs using the Network Service account. If the installation account has sufficient permissions, the Agent installer creates a SQL Login on the instance, and grants the required permissions on the database:

  • Remote SQL Servers - the SQL Login is the computer$ account of the Agent host.

  • Local SQL Server - the SQL Login uses a Service SID (Microsoft uses Server SID accounts for its own various SQL Server services.

SQL Server source(s)

The SQL Server instance which hosts the source database must be configured to use SSL certificates. If this is not possible, then please contact your 1E Account Team for help.

  • You do not need to enable ForceEncryption (force encrypted communications for all clients) therefore this change should not affect your existing operations.

  • You only need to configure the instance with an appropriate SSL certificate (has Subject, and ServerAuthentication as usage). DataSync Agent does not require a client certificate.

The Agent communicates with source database(s) on whatever ports they are running on.

Ensure the host computer firewall allows access to each SQL Server source(s).

The Agent can sync from any version of SQL Server.

Network access to 1E Azure

The Agent communicates with the following hosts in 1E Azure on port 443. Ensure the host computer firewall allows access to each of these Azure hosts.

  • prod-datasyncsvc1e.azurewebsites.net

  • prod-datasyncehns1e.servicebus.windows.net

  • prod-authservice1e.azurewebsites.net

  • dc.applicationinsights.azure.com

  • dc.applicationinsights.microsoft.com

Preparation

To run the installation you will need the DataSync Agent MSI installer along with the following pieces of information from 1E. You can download the installer from support.1e.com/.

The parameters are provided to you as part of the SaaS provisioning process, in one or more TXT files. Normally, 1E provides one TXT file for each Sync use case. All parameters are mandatory.

Installation parameters

Installer property name

Description

DataSync Service URL

DATA_SYNC_API_URL

The URL of DataSync Service URL running in Azure.

This should normally be: https://prod-datasyncsvc1e.azurewebsites.net/

Auth Service URL

AUTH_SERVICE_URL

The URL of Auth Service running in Azure used by the Agent to obtain authentication tokens.

This should normally be: https://prod-authservice1e.azurewebsites.net/

Client Cert Token

CLIENT_CERT_TOKEN

A token used by the Agent installer to perform the initial setup of authentication for the Agent.

This is a unique time-based token that includes encoded details of the Azure authentication certificate, and the specific Sync use-case(s).

Data Source Connection details

SQL_SERVER_DATA_SOURCES

<SyncName>|<DbServerName>|<DbName>

Use the table below to construct the connection details for a specific Sync use-case.

Although the Data Source Connection details field supports multiple sync use-cases, for this version, you should only enter details for one Sync. Use the pipe character | as a delimiter.

Data Source Connection details

Description

SyncName

The SyncName is provided to you by 1E, specific to a single Sync use-case. You should not change this.

DbServerName

The SQL Server instance name, using a format as specified by Microsoft. For example:

  • DbServerFQDN

  • DbServerFQDN\InstanceName

  • DbServerFQDN\InstanceName,port

DbServerFQDN must match the subject name in the SQL Server instance’s SSL certificate.

The installation account must have permission to this SQL Server instance and database, as described in Requirements.

DbName

The SQL Server database name.

The installation account must have permission to this SQL Server instance and database, as described in Requirements.

Interactive installation

  1. To run the installation interactively, logon to the host computer using the installation account, then start the installer.

  2. Run the DatasyncAgent.msi.

  3. On the Welcome screen, click Next.

  4. On the End-User License Agreement screen, check I accept the terms in the License Agreement, and click Next.

  5. On the DataSync Service URL Parameters screen, enter the parameter values you have been provided, and click Next.

  6. On the Data Source Connection Details screen, enter the parameter value that you constructed earlier, and click Next.

  7. On the Ready to install 1E DataSync Agent screen, click Install to begin the installation.

  8. On completion, click Finish to close the installer.

Silent installation

To run the installation silently, logon to the host computer using the installation account, then start the installer using a command line similar to below. You should have been given a TXT file with the command line you need for installing a single DataSync Agent supporting a single Sync. However, you must first reconstruct the SQL_SERVER_DATA_SOURCES parameter by replacing the placeholder text using the guidance in Preparation.

For example:

Copy 
msiexec.exe /i DataSyncAgent.msi /l*v %TEMP%\DataSyncAgent.install.log /qn DATA_SYNC_API_URL=https://prod-datasyncsvc1e.azurewebsites.net/ 
AUTH_SERVICE_URL=https://prod-authservice1e.azurewebsites.net/ CLIENT_CERT_TOKEN=eyJhbGciOiJSUzI1NiIsImtpZCI6IkQ3MjE4MkQzQzIwN0I3ODg1ODZCMkVGN0
MxMDhDRkI3IiwidHlwIjoiSldUIn0.eyJ2ZXIiOiIxIiwianRpIjoiMDMxNzY4NDYtNTk4Yi00YjkxLTljMDItMDNlZTkxNjhiOTgzIiwiaWF0IjoxNjYzNzI5MjU5LCJleHAiOjE2OTUyNjUyNTksImFwcF
9pZCI6ImNkYzY3MTNhLWUxYjQtNGFkYy05NTUwLTgyYzc1MWYwZDBlNCIsImNlcnRfaWQiOiJDZXJ0MSIsInRlbmFudF9pZCI6IjU1YWNjOTY4LTRmZjItNDQ1NC05OGY0LTliZmE5NzgxMGQwYyJ9
.bU20wok0mR1_vi9W4wQ4RAjrXykTpbXETWWBHdNhizR9-zhk1uZY8necFzpS6JJoKma_Aurd9lM3br9k21TUwMigzFSxQMJlcac2Dpn0Wp1Z4adhYnoH-pdTiwBYwHh_9jUssDb8RHohkMtUWHjwlmi8
qFJ1tnWUEUo9l2Agw1rX2VDuYXFdHrtl8Vv7nYasiF4ohxEgppjk1HSt7OjAfd-0evejjcvCGie_a9CsFf8RlENlsMkLpK-3uOZ7wmnWpMZXBtu5g65AkK4K8RlNGHD9EtUiR4v6DmeH-d50Y6Uqw3y8af
_egtrAguMGMiTlLfOMOnW16S7zjXUIHyF0fA SQL_SERVER_DATA_SOURCES="CMSync|ACME-CM01.acme.local|CM_CAS"

Post installation

  • Once the installer finishes it will have created a Windows service for the Agent called 1E DataSync Agent.

  • The service will start immediately and start sending data from all the sources you specified.

  • The first sync is full, and thereafter only the differences. The Agent maintains a snapshot for each dataset in %ALLUSERSPROFILE%\1E\DataSyncAgent\Snapshots\{syncname}\{dataset}

Review Agent logs

You should review installation and service logs for any errors. Unless otherwise specified on the command-line, the Agent installer creates an installation log in the %temp% folder of the logged on user. The DataSync Agent service log file is created by default at:

%ProgramData%\1E\DataSyncAgent\Logs\DataSyncAgent.log

The log shows details of each Sync cycle:

  • The frequency of each cycle depends on the Sync use-case. The cycle starts when the Agent is started.

  • Within each cycle, there are pairs of entries for each data set, saying "Syncing data set" and "Done syncing data set in HH:MM:SS.SSSSSSSS".

  • "Syncing data set" does a SQL query then uploads events. If there is an issue with the SQL query, an entry is displayed showing the query and the error, and the process moves onto the next data set.

  • "Done syncing" indicates events for the data set have been uploaded, and either says "No events sent" if no new data, or lists the types of new events (Truncates, Inserts, Updates, Deletes, SnapshotCompletes).

Device grooming

1E Platform can run a device grooming task that deletes devices that have not connected to the platform in a set number of days (99 by default). When devices are groomed a deleted event is sent. If you want to take advantage of this feature, you will need to request that it is enabled for your platform instance with values suited to your organization.