Phone Lines icon GFI Support Home

Articles in this section

See more

LanGuard Remediation Process Workflow and Troubleshooting Guidelines

Author: Andriy Rybalchenko Updated

Overview

This article provides information on the LanGuard Remediation Process Workflow and troubleshooting guidelines. 

Introduction

LanGuard allows deploying Microsoft and non-Microsoft patches, service packs, custom applications. All of these activities are performed by Remediation (Deployment) jobs. There are two main categories of remediation:

  • Automatic Remediation - triggered automatically after a scan to install approved updates (if any)
  • Manual Remediation - performed by the administrator, based on reported scan results

Regardless of the trigger type, and whether this is patch deployment, custom software installation, or any other remediation job, the actual remediation process always follows the same steps.

 

Description

The diagram below shows an overview of the entire remediation job process:

mceclip0.png

Each step in detail is described in the Patch Deployment Process in GFI LanGuard customer-facing article.

 

Yet there are some extra steps, involving the database backend, which is not something you will see or troubleshoot daily. These steps, along with the troubleshooting suggestions, will be shown here to let you understand the full process. On the live troubleshooting sessions, this information may help to confirm that the remediation is set correctly from a database perspective and that you need to investigate communication issues instead.

Note: See the Scan Results Database Structure document for the referenced below database tables structure, values, and the meaning of each value.

  1. LanGuard writes the remediation entry to the SQL database in the dbo.Deployments table, that contains information about deployment sessions performed on one or more targets.

    mceclip0.png

  2. Since remediation can be set for multiple machines, each set of deployments references the dbo.Deployment table, which contains data on deployment target(s).

    mceclip1.png

  3. dbo.DeploymentDetail contains the list of patches set for remediation for each job, tied to the DeploymentsID from Step 1. It contains information about each remediation operation to be performed.

  4. The dbo.DeploymentsOptions contains configured deployment options, such as reboot time, patch verification scans and the likes, also tied to the DeploymentsID from Step 1.

  5. This step is generally where troubleshooting starts under normal circumstances. The job is initiated by the LanGuard Attendant service. It is constantly monitoring the database and tables above via the remediationplugin.dll (the default location is C:\Program Files (x86)\GFI\LanGuard 12 Agent\) and executes the jobs as scheduled (this .dll is NOT the plugin that communicates with the machines directly).

    You can follow the Remediation job corresponding entries through lanss_v***_attendantservice.csv log, by filtering the “Source” field to RemediationPlugin.

    Reminder: The logs are located in %Data%\GFI\LanGuard 12\DebugLogs\. Refer to Data Structure and Log Review for more information about the LanGuard Data Structure and Logs format understanding.

    mceclip3.png

    Here you can see the DeploymentsID initiation, which then goes through and determines the following:

    • The machines which have been selected for remediation, and their network information;

    • What to install; and

    • Any variations on settings that may have been made.

  6. The remediationplugin.dll plugin continues to work within the database, updating the dbo.DeploymentsStatus and dbo.DeploymentStatus tables. The entries found in these tables are reflected as remediation job progress in the LanGuard console.

  7. The plugin checks if the patch selected for Remediation is already downloaded and can be found in the local repository. If the patch requires downloading, the system will communicate with the PatchAutodownload plugin to download the needed patch before continuing.

    When using Relay Agents, files are copied from another location. See the Relay Agents section below for more details.

  8. After the patches have been confirmed to be ready and available, the remediationplugin.dll resolves the machine name to ensure proper communication channels.
    mceclip4.png

    Refer to the Testing Network Connectivity and Security Permissions for GFI LanGuard Operations if machine resolution is not working correctly. 

  9. At this point, the system has confirmed that it is time for the remediation job to take place, on what machines it will occur, and which patches to deploy on those machines.

    The plugin initiates a connection with the LNSSComm plugin to monitor remediation job status. 

    mceclip7.png

  10. The LNSSComm Remediation Engine plugin, responsible for communication, writes to lanss_v***_patchdeployment.csv log. The Initialization timestamp should be the same as on the screenshot above.

    mceclip6.png

  11. The lanss_v***_patchdeployment.csv lists the target computers, commands to run, files to copy, this should match the list from the Attendant Service RemediationPlugin.

    mceclip8.png

    All commands without a rem prefix will send a status message back to the console.

  12. When remediation targets multiple machines, separate threads are created for each machine.

    mceclip9.png

  13. The LanGuard server begins connecting to the machines over the network using the credentials set either by the service or in the UI.

    See the Best Practices for Setting up Account Permissions with Alternative Credentials. 

  14. If the Remote Registry service is not already running, LanGuard will attempt to start the service. Once started, it connects to the registry and gathers the system language and remote system path. 

    mceclip10.png

  15. Next, the Remediation engine connects to the Administrative (or configured) share path on the remote computer, preparing for the patch files copying.
    mceclip11.png

  16. The system checks if the PatchAgent service is already installed and running on the machine, and waits until it is terminated to avoid running multiple remediation jobs at the same time. 

  17. All the files required for deployment are copied to the target machine (Patch Deployment Process step 2).

  18. The batch file deploy_patches.bat containing the commands with parameters to install all selected patches silently is created and copied to the target computer.
    mceclip12.png

  19. PatchAgent.exe is copied to the target machine, then a service called GFI LanGuard Patch Agent Service (PatchAgent.exe) is installed and started on the machine. This service creates/writes to the PatchAgent.log, calls and executes each command in the deploy_patches.bat file, and monitors the status of each patch deployment.
    mceclip13.png

    The credentials for this service, specified in the deployment settings, should have local administrator permissions on the target machine as well as the Log on as a service rights. 

  20. At this point, the lanss_v***_patchdeployment.csv log shows status message responses from the target machine PatchAgent Service sent to the Attendant Service RemediationPlugin.

    These are the responses seen in the LanGuard console for each deployment:

    mceclip2.png 

  21. On the target machine commands execution is logged to the PatchAgent.log at %system%\Windows\Patches (unless otherwise changed in the settings). It should list the command to install the service at roughly the same time as the beginning of the remediation job. 

    mceclip14.png

  22. The PatchAgent then proceeds to open the previously created and copied deploy_patches.bat, which contains the instructions to execute patches selected for this remediation job.

  23. The PatchAgent verifies the SHA hash to make sure the correct patch is going to be executed, and not some malicious code, as well as ensuring that the full copy was completed.

    mceclip15.png

  24. The PatchAgent creates a separate batch file as needed for each command in the primary deploy_patches.bat file and executes just that one command/patch.

    mceclip16.png

  25. The PatchAgent Service returns the final result (success or fail) of each individual patch deployment to the GFI LanGuard server via the communications port 1072 (by default).

    This port needs to be open on the LanGuard server to receive the status messages that will be presented on the LanGuard console.

    mceclip17.png

    The exit code in the screenshot above is 3010, which is the MSIExec exit code for Success_Reboot_Required.

  26. Steps 24 and 25 are repeated until all commands are completed.

  27. Once the patches have been installed, the system will delete the installation files by default.

    This depends on the setting in After Deployment options (Delete copied files from remote computers after deployment).

  28. If the option for Reboot was configured among the After Deployment options, the system would run an extra command to initiate the Reboot option selection.

     

  29. The PatchAgent service then performs a cleanup, unregistering and removing the service before shutting down.

  30. Once the Apache server listening on the LanGuard server receives the status messages, they are sent to the Remediation Engine (lanss_v***_patchdeployment.csv log), which forwards the message on to the RemediationPlugin (lanss_v***_attendantservice.csv log). 

  31. After the deployment job completion, RemediationPlugin updates the deployment job status in the database and then removes the job id from the list of active deployments.

    mceclip18.png

  32. The status messages logged to the database are stored to present the results in the LanGuard Dashboard under the Activity monitor. 

It is possible to receive Timed out: PatchAgent did not respond in the permitted time interval error message when deploying patches, but that doesn't mean that the process actually failed. Refer to the Remediation Error: "Timed out: PatchAgent did not respond in the permitted time interval" When Attempting to Deploy Patches for more information and the solution.

If you are completely stuck, there is a chance other log files may provide clues as well. Just search them for errors and warnings around the time when the issue occurred.

 

Relay Agents

Spread out geographically or large environments may be using Relay Agents, with a repository cached at a secondary location(s) to reduce bandwidth usage across the WAN and load on the LanGuard server.

NOTE: When the target computer has a GFI LanGuard Agent that is configured to use a Relay Agent, only the PatchAgent.exe and deploy_patches.bat are copied initially. The Patch Agent Service once started would request patch installer executables from the assigned Relay Agent.

Relay agents cache updates in a particular cache folder and configure Apache HTTPD for process requests. LanGuard uses Apache Module mod_cache for caching.

The Relay cache folder is specified in C:\ProgramData\GFI\LanGuard 12\toolcfg_relay.xml.
By default, the relay cache folder is C:\ProgramData\GFI\LanGuard 12\RelayCache\. 

<supportagent>

Ticket Examples

Below are some examples of tickets were agents are using the logs analysis results to identify issues and probable root causes:

</supportagent>

Related articles