FIX: The agent could not connect to the Microsoft Operations Management Suite service.

Please check that the system either has internet access, or that a valid HTTP proxy has been configured for the agent.

This blog post is about resolving a problem with the Microsoft Monitoring Agent that is not connected to the Operation management Suite in Azure.

Today I had another great error message. Colleagues were unable to connect a Windows 2008 R2 server with Log Analytics (OMS). The Microsoft Monitoring agent had the following error message:

“The agent could not connect to the Microsoft Operations Management Suite service. Please check that the system either has internet access, or that a valid HTTP proxy has been configured for the agent.”

FIX: "The agent could not connect to the Microsoft Operations Management Suite service. Please check that the system either has internet access, or that a valid HTTP proxy has been configured for the agent."
FIX: “The agent could not connect to the Microsoft Operations Management Suite service. Please check that the system either has internet access, or that a valid HTTP proxy has been configured for the agent.”

Let’s fix ‘The agent could not connect to the Microsoft Operations Management Suite service.’

I have divided the tutorial into two parts.

  • A short but clear solution; Just give me a fix.
  • A longer solution about how I came up with the solution; Tell me how.

The problem can be due to various things, but with one of the tutorials above you will find the reason why you get the error message.


Quick fix: Just give me a fix!

There is a troubleshooter that is installed with the Microsoft Monitoring Agent ‘TestCloudConnection.exe‘ which you can find in the installation folder.

Start Command Prompt,
Set the location to the Microsoft Monitoring Agent installation folder,
Type in TestCloudConnection.exe and press Enter.

See the results and troubleshoot from there.

Always start with the Windows Firewall. You can temporarily disable it and then restart the Microsoft Monitoring Agent Service to see if a connection is made. Don’t forget to turn the firewall back on though.

The issue is ultimately the same, but it will generally be easiest to solve in Azure.


Server in Azure?

If it’s not the Windows Firewall, go to the Azure portal and check either any Azure firewall tooling (Or default) or the Network Security Group.

it eventually turned out that all traffic was blocked outbound on the Network Security Group.

On the NSG you can create a new Outbound Security rule that is open for outbound on port 443 to Azure Monitoring via Service Tags.

for more about Service Tags or Network Security Groups:

https://docs.microsoft.com/en-us/azure/virtual-network/service-tags-overview

https://docs.microsoft.com/en-us/azure/virtual-network/security-overview

See the screenshot below.

FIX: The agent could not connect to the Microsoft Operations Management Suite service.

Server on-premise?

Okay I can’t help you too much with this, but somewhere in your network port 443 is outbound blocked. Check the traffic from the respective server and monitor the flow per device. See if port 443 is blocked here.


Full explanation: Tell me how..

Okay, the error message says a lot, but actually it doesn’t.

The system either has internet access, or that a valid HTTP proxy has been configured for the agent.

The error indicates that the server is not able to connect to OMS. So it is clear that there is a blockage on the network traffic somewhere. This could be DNS or a firewall.


Could it be the Windows Firewall?

The first and the easiest to test is the Windows Firewall. Small chance that port 443 is blocked outbound, but you never know ..

  • Go to Start and search for Windows Firewall with Advanced Security.
  • Right click Windows Firewall with Advanced Security click Properties.
  • Save the Firewall state to Off.
  • Test again, and turn the Firewall state back on.
FIX: The agent could not connect to the Microsoft Operations Management Suite service.
FIX: The agent could not connect to the Microsoft Operations Management Suite service.

Where do we start?

I always start with a Google search. How many users are already running into this? Is the solution the same on the internet?

First thing I came across was a troubleshoot tutorial from Microsoft.

https://docs.microsoft.com/bs-cyrl-ba/azure/azure-monitor/platform/agent-windows-troubleshoot

There is a troubleshooter that is installed with the Microsoft Monitoring Agent ‘TestCloudConnection.exe‘ which you can find in the installation folder. For me this is:

C:\Program Files\Microsoft Monitoring Agent\Agent

To run it open Command Prompt and set the location to installation folder.

CD  C:\Program Files\Microsoft Monitoring Agent\Agent

Type in TestCloudConnection.exe and press Enter.
Below you’ll see my result.

C:\Program Files\Microsoft Monitoring Agent\Agent>TestCloudConnection.exe
Starting Azure Log Analytics connectivity test.

Proxy Configuration: None. The agent will directly connect to the Internet.

Starting connectivity test for workspace id "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx".
This test may take several minutes to complete if a firewall is blocking access.


Making initial connection to xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx.oms.opinsights
.azure.com.
This is to download the list of Azure resources neccesary for communication with
 this specific workspace.

Connection failed to xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx.oms.opinsights.azure.c
om.
Firewall troubleshooting details: "Unable to connect to the remote server"

Oké great, so I have no connection to one of these Agent Resources according to the troubleshooter (Docs.Microsoft.com)

Agent ResourcePortsDirectionBypass HTTPS inspection
ods.opinsights.azure.comPort 443OutboundYes
*.oms.opinsights.azure.comPort 443OutboundYes
*.blob.core.windows.netPort 443OutboundYes

So if it is not the default firewall then elsewhere on the network outbound traffic to one of these 4 must be blocked. And in my case to all.

The server I troubleshooted is in Azure.
So for me, it’s a matter of tracking the network infrastructure until I get to the device that’s blocking port 443 outbound.

In Azure I see that the relevant server is in a DMZ that has its own Network Security Group.

And what do you know? Outbound traffic is completely blocked. The NSG is configured to communicate only with its own network.

FIX: "Please check that the system either has internet access, or that a valid HTTP proxy has been configured for the agent."
FIX: “Please check that the system either has internet access, or that a valid HTTP proxy has been configured for the agent.

Okay, great. Outbound traffic must therefore be completely blocked.

Fortunately, when creating a new outbound security rule you have the option to provide Service Tags. It also contains tag AzureMonitor. If you put the tag on AzureMonitor, the outbound traffic over port 443 will only be allowed to Azure Monitoring.

For more about Azure Service Tags or Network Security Groups please take a look here:

https://docs.microsoft.com/en-us/azure/virtual-network/service-tags-overview

https://docs.microsoft.com/en-us/azure/virtual-network/security-overview

FIX: "Please check that the system either has internet access, or that a valid HTTP proxy has been configured for the agent."
FIX: “Please check that the system either has internet access, or that a valid HTTP proxy has been configured for the agent.

Now, when I re-run TestCloudConnection.exe on the server it’s able to connect to Azure Monitoring. Unfortunately I’m getting a new error:

The agent could not authenticate with the Microsoft Operations Management Suite service. Please check that the Workspace Key is correct.

but that’s something for the next chapter.

C:\Program Files\Microsoft Monitoring Agent\Agent>TestCloudConnection.exe
Starting Azure Log Analytics connectivity test.

Proxy Configuration: None. The agent will directly connect to the Internet.

Starting connectivity test for workspace id "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx".
This test may take several minutes to complete if a firewall is blocking access.


Making initial connection to xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx.oms.opinsights
.azure.com.
This is to download the list of Azure resources neccesary for communication with
 this specific workspace.

Completed connectivity test for workspace id "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx".  Connectivity check indicated a firewall may be blocking requests.

Blocked Hosts
        Firewall Rule: *.agentsvc.azure-automation.net
                xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx.agentsvc.azure-automation.n
et      The request was aborted: Could not create SSL/TLS secure channel.

Successful Connections
        Firewall Rule: *.blob.core.windows.net
                aoiweu201510151254.blob.core.windows.net
                opinsightsweuomssa.blob.core.windows.net
                scadvisorcontent.blob.core.windows.net
        Firewall Rule: *.ods.opinsights.azure.com
                xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx.ods.opinsights.azure.com
        Firewall Rule: *.oms.opinsights.azure.com
                xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx.oms.opinsights.azure.com

New error: “The agent could not authenticate with the Microsoft Operations Management Suite service. Please check that the Workspace Key is correct.

When you finally think that everything is working again, you come across something new.

Luckily I caused this message myself. By opening the workspace you have to enter the workspace key again. I probably accidentally pressed OK without entering a key.

Go to Log Analytics Workspace in Azure.
Click on Windows, Linux, and other sources.

FIX: "The agent could not authenticate with the Microsoft Operations Management Suite service. Please check that the Workspace Key is correct."
FIX: “The agent could not authenticate with the Microsoft Operations Management Suite service. Please check that the Workspace Key is correct.

Go to Connected Sources,
Open Windows Servers,

Copy the Workspace primary key.

FIX: "The agent could not authenticate with the Microsoft Operations Management Suite service. Please check that the Workspace Key is correct."
FIX: “The agent could not authenticate with the Microsoft Operations Management Suite service. Please check that the Workspace Key is correct.

Now go back to the server and open the Microsoft Monitoring Agent again.

Edit your Azure Log Analytics(OMS) workspace.

Copy and paste the key.
Click OK.

FIX: "The agent could not authenticate with the Microsoft Operations Management Suite service. Please check that the Workspace Key is correct."
FIX: “The agent could not authenticate with the Microsoft Operations Management Suite service. Please check that the Workspace Key is correct.

Success!

The Microsoft Monitoring Agent has successfully connected to the Microsoft Operations Management Suite Service.

Published by

Bas Wijdenes

My name is Bas Wijdenes and I work as a PowerShell DevOps Engineer. In my spare time I write about interesting stuff that I encounter during my work.

One thought on “FIX: The agent could not connect to the Microsoft Operations Management Suite service.”

  1. Thanks for your article.

    Even though my TestCloudConnection.exe troubleshooting tool was passing all firewall tests, and confirming Workspace ID and Key were correct, my MMA Agent was still failing to connect (as per the Control Panel agent).

    Resources were showing as “Connected” to the Log Workspace in the Azure Portal, yet no data was being received.

    *** SOLUTION ***
    Ended up logging support ticket with Microsoft, who after some trial and error, were eventually able to advise a solution to ENABLE the two items under “Network Isolation” under Settings in the Azure Portal for the Log Workspace which are used when you have an AMPLS (Azure Monitor Private Link Setup):
    – “Allow public network access for ingestion”
    – “Allow public network access for queries”

    If you are using Terraform, set the following settings for the “azurerm_log_analytics_workspace” resource:
    – internet_ingestion_enabled = true
    – internet_query_enabled = true

    Once these were enabled, restart the MMA Agent service on the VMs, and then the Control Panel agent was showing as connecting successfully.

    I requested that these settings be added to their troubleshooting OMS Agent documentation, as there was no mention of these settings to be found.

    Hope this helps someone out there, as this cost me over a week of time spent troubleshooting…

    Regards
    Darren

Leave a Reply

Your email address will not be published. Required fields are marked *