Remote differential compression (RDC) integration

Not only is Nomad aware of the file level differences between different versions of a package so that only changed files are downloaded, it is also aware of the differences within individual files. Refer to Download resumption and consistency checking.

This is sometimes known as binary differential replication or binary deltas but is more commonly known as remote differential compression (RDC) integration, for details refer to https://docs.microsoft.com/en-us/previous-versions/windows/desktop/rdc/about-remote-differential-compression.

Refer to Using FanOut for large subnets for an example scenario about deploying software in a large network and how you can monitor those deployments using the Content Distribution app.

In our example, a package that has been previously downloaded has one of its files updated. Nomad compares the Configuration Manager RDC signature file (where each block in the file is given a number that identifies it according to its contents) for the new version of the file and the one before it. By identifying the changes, Nomad is able to download just the file blocks containing the differences.

In combination with other Nomad features such as Bandwidth Protection and dynamic elections, remote differential compression (RDC), when used with Configuration Manager (CM), results in an even more efficient and reliable mechanism for transferring content across the WAN. The Nomad client automatically generates a manifest file for each version of a package when the first client attempts to download the newest version of a package. Refer to Manifest files. If RDC is enabled, file delta downloads becomes a feature of the manifest file and the client will download just the differences from either the DP or the local peer.

RDC integration occurs on:

  • Configuration Manager distribution point (DP).

  • Nomad client master.

  • Nomad client peers.

The only configuration required for RDC is:

On Configuration Manager, RDC is not used for applications. This is because an application's content version always remains at 1, only the content revision gets updated with a new content ID, so RDC for applications will not occur. This is not a limitation of Nomad, but simply the way the application model is implemented.

RDC on distribution points

RDC signature files must be generated and copied to the default signature folder (C:\SMSSIG$) by Configuration Manager. If you use a different location for the signature files, you must update its location in the SigsFolder registry value. Signature files are generated for every package, including OSD images.

RDC on the elected client

The RDC file copy feature only works between the DP and the elected master. The package download process remains almost the same as in earlier versions of Nomad, except for files that have an RDC value specified for them in the manifest file.

The Nomad master checks for the package version (n-1) on the subnet. If the version (n-1) is cached locally or available on another client on the same subnet, it uses RDC to copy delta blocks from the DP. Once the elected master starts to download the newest version of a package, it first fetches the manifest file and RDC data generated by the NomadBranch service for that package on the DP. The RDC data contains the needs list (.needs.bin) for all the files in the package that have changed between versions n-1 to n. These .needs.bin files are used to create the target files by copying blocks from the files cached locally and from the DP.

If the master becomes unavailable during this process, a re-election takes place and the new master re-downloads the needs list from the DP.

The NomadBranch service skips RDC calculations while generating the LSZ of a package where the LSZ of version n-1 has not been generated earlier or if the LSZgen request is for the first version of the package.

RDC on the peer

RDC file copy is done between the DP and the elected master only in a subnet. The remaining devices or peers in the subnet copies all the blocks of files modified in the package using normal P2P.

Alternate source copy

Alternate source copy, often referred to as alt-copy, takes place when the elected master does not have the previous version of the package, but another device on the subnet has it. The elected master can still do RDC file copy by copying the required blocks using RDC_alt copy from the other device on the subnet.

If a version n of a package is not available from Nomad peers but version n-1 is, Nomad fetches n-1 on the assumption that at least some files are unchanged between versions, then it will download just the remaining, different, files in version n from the DP.

This mechanism relies on the name of the content remaining the same between versions with just a change in the version number, and so it only works for packages. A new version of an application or software update has a completely new name, and version numbers are not meaningful. There is no configuration option for this, Nomad does it automatically.

On Windows Server DPs without a primary or secondary site server, RDC must be installed manually using Windows Server Manager.

Alternate source copy will only work when clients download from the same DP.