Offloading data to an external target

This article describes how to set up a 1E Platform consumer to offload data to an external target and then invoke an instruction which sends the data to that target.

The 1E Platform supports the execution of instructions with the option to offload the instruction results to a different API. When this is specified, the results from an instruction are not stored in the 1E Platform responses database. Instead, they are sent to an API device which has been defined for the appropriate consumer specified when the instruction was invoked.

To explore the offloading feature, we start by configuring one of the existing 1E Platform consumers to support an optional offloading device.

In a production environment you should create a new consumer to support this functionality. However, in a test lab you can override the functionality of an existing consumer if desired.

From the settings application, modify the RunInstructionUI consumer to add an offload target URL as shown below. Change the server name to be the name of the machine on which you are going to run the test PowerShell cmdlet. Also disable Use Windows Authentication.

We will now stand up a web server on this device. Take the PowerShell code included below and run it in PowerShell under an administrator prompt.

Change the server name appropriately to match the above consumer configuration. If you have something running already bound to port 80, you can always set a different port, and make sure to also configure the consumer appropriately. If the 1E server is not running on the same machine as the PowerShell script, you may need to open an inbound port on your firewall.

Copy

# This is a super **SIMPLE** example of how to create a very basic powershell webserver
  
# Http Server
$http = New-Object System.Net.HttpListener
  
# Hostname and port to listen on
$http.Prefixes.Add("http://urth-dev.urth.local:80/") # change this to match the device you're running on.
  
# Start the Http Server
$http.Start()
  
# Log ready message to terminal
if ($http.IsListening)
{
    write-host " HTTP Server Ready!  " -f 'black' -b 'gre'
}
  
# INFINITE LOOP
# Used to listen for requests
while ($http.IsListening)
{
    # Get Request Url
    # When a request is made in a web browser the GetContext() method will return a request object
    # Our route examples below will use the request object properties to decide how to respond
    $context = $http.GetContext()
  
    if ($context.Request.RawUrl -eq '/Kill')
    {
        write-host "Kill received, exiting"
        break
    }
  
    if ($context.Request.HttpMethod -eq 'POST' -and $context.Request.RawUrl -eq '/Foo')
    {
        write-host "post"
        $streamReader = New-Object System.IO.StreamReader -ArgumentList $context.Request.InputStream
        $FormContent = $streamReader.ReadToEnd()
  
        # We can log the request to the terminal
        write-host "$($context.Request.UserHostAddress)  =>  $($context.Request.Url)" -f 'mag'
        Write-Host $FormContent -f 'Green'
        
        $buffer = [System.Text.Encoding]::UTF8.GetBytes("OK")
        $context.Response.ContentLength64 = $buffer.Length
        $context.Response.OutputStream.Write($buffer, 0, $buffer.Length) #stream to browser
        $context.Response.OutputStream.Close() # close the response
    }
  
    # powershell will continue looping and listen for new requests...
}
  
# here if we break from the loop (by receiving a Kill request)
$http.stop()
  
# Note:
# To end the loop you have to kill the powershell terminal. ctrl-c wont work :/

You should now see that the web server is ready to accept requests.

The server will remain running until you terminate it. You cannot terminate it with ctrl+C, so use the following:

Copy

curl http://urth-dev.urth.local/kill

This will terminate the web server if you wish to shut it down.

With the web server now running, first change the default consumer to RunInstructionUI, then use the invoke-1Einstruction cmdlet to invoke an instruction against that consumer, specifying that the results should be offloaded.

Copy

Set-1EDefaultConsumer RunInstructionUI
Invoke-1EInstruction 1E-Explorer-TachyonAgent-Echo -TargetScope urth -msg hello -offload

You will see the offloaded data sent to the listening web server as shown below.

If you invoke an instruction and specify the -Offload parameter, then if the listening device is not accepting data, the 1E Platform will not update the received count for the instruction statistics. This will cause the invoke-1Einstruction cmdlet to wait until a timeout occurs. You can, of course, interrupt the cmdlet with ctrl+C at any time.