Troubleshooting ConfigMgr Enhanced HTTP and Azure Directory Group Sync

Today I got to help my buddy Adam Juelich with getting ConfigMgr Azure Directory Group Sync working. It’s an awesome new feature that allows you to sync ConfigMgr collection memberships to your Azure tenant. Adam had followed all of the steps and ensured that prerequisites were all configured properly but sync would never work.

Over a Teams meeting we double checked everything and walked through Ronny Dejong’s (He covers pretty much all of troubleshooting steps needed for this!), Nickolaj Andersen’s, and Jake Stoker’s blogs on configuring and troubleshooting the sync, which helped put us on the right track to the issue.

Finding the Issue

HTTPS or Enhanced HTTP is required for Azure Directory Group Sync - Adam confirmed that he had EHTTP enabled. According to Ronny’s blog, EHTTP is responsible for updating the AADTenantID value in the ClientDataKeys table in the ConfigMgr SQL database. We were confused because in the ConfigMgr console, you can turn on the Azure Active Directory Tenant ID column in any Device view and it was populated for all of the devices in the collection to be synced.

A Square Dozen Image

However when we queried the ClientDataKeys table (Query from Nickolaj’s blog) we found that the AADTenantID column was NULL while the AADTenantID column in System_DISC (Used by the console views) was populated.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
SELECT 
	System_DISC.Name0,
	ClientKeyData.SMSID, 
	System_DISC.SMS_Unique_Identifier0, 
	[ClientKeyData AADTenantID] = ClientKeyData.AADTenantID, 
	[System_DISC AADTenantID] = System_DISC.AADTenantID
FROM 
	ClientKeyData JOIN 
	System_DISC ON ClientKeyData.SMSID = System_DISC.SMS_Unique_Identifier0
ORDER BY
	ClientKeyData.AADTenantID DESC

A Square Dozen Image

Thanks to Ronny for a great description of the difference between the AADTenantID values between System_Disc and ClientDataKeys:

A Square Dozen Image
https://ronnydejong.com/2020/02/27/troubleshooting-device-collection-membership-azure-ad-group-sync/

Ronny’s blog section titled Validate SSL Communication would have helped with this next part if I had scrolled down a bit more, but instead we went about it the hard way and got to the same result - Enhanced HTTP wasn’t configured even though everything looked correct from the console.

Rolling back Enhanced HTTP

When we took a look IIS, we found that HTTPS was enabled but the binding was what Adam described as ‘An Ancient PKI Certificate’. I thought this was the fix, just swap the cert to the SMS Role SSL Certificate and we’d be good. However, that certificate wasn’t in the list. We checked the local certificate store and it wasn’t there either. This was a good clue. I verified in my lab that the cert should have been issued. We checked the docs and didn’t see anything listed about the new cert until we checked the docs for the Cloud Management Gateway where there’s this note, confirming our suspicion:

When you enable Enhanced HTTP, the site server generates a self-signed certificate named SMS Role SSL Certificate, issued by the root SMS Issuing certificate. The management point adds this certificate to the IIS Default Web site bound to port 443. https://docs.microsoft.com/mem/configmgr/core/clients/manage/cmg/certificates-for-cloud-management-gateway#enhanced-http-certificate-for-management-points

Our fix was to disable EHTTP (just uncheck the box shown in the next section and wait for the change to process (watch sitecomp.log and mpcontrol.log). We also removed the HTTPS binding from IIS. Then we re-enabled EHTTP following the steps below.

Enable and Verify Enhanced HTTP Configuration in IIS

Follow the steps from the Docs to enable Enhanced HTTP.

Enable Enhanced HTTP
Enable Enhanced HTTP

Check sitecomp.log to see the change get processed.

Detected change in SSLState for client settings.
Detected change in SSLState for client settings.

This will trigger a change that you can watch in mpcontrol.log (partial log shown here. There will be quite a few log entries here.)

Detected token auth flag is changed. Currently enabled.
Detected token auth flag is changed. Currently enabled.

A new cert will be generated on the MP. The Friendly Name will be SMS Role SSL Certificate issued by SMS Issuing.

Newly issued Cert Issued By SMS Signing
Newly issued Cert Issued By SMS Signing

In IIS on the Default Web Site, HTTPS will be added to the site bindings and the SMS Role SSL Certificate will be attached.

HTTPS enabled and new Cert attached
HTTPS enabled and new Cert attached

Test Collection Sync

Once you’ve verified that EHTTPS has been enabled successfully, you can clients will begin updating the ClientDataKeys table. Re-run the query above. Once you see the AADTenantID begin populating, any devices with a value in that column will be eligible to sync if they are in a collection you’ve configured for sync. Once you’ve verified the data, follow the steps in any of the blogs linked to set up collection sync. Jake has the simplest writeup on the steps. CollectionAADGroupSyncWorker.log and SMS_AZUREAD_DISCOVERY_AGENT.log are good logs to watch when working with Collection Sync. - See notes here on log file changes after 1910.

Root Cause

IIS had an HTTPS/443 binding configured with Certificate attached to it before enabling EHTTP. In my lab testing, if I had already enabled the HTTPS/443 binding (which requires you to select a certificate), when I enabled Enhanced HTTP, the process didn’t remove/replace the IIS configuration that was already in place. It also didn’t generate the SMS Role SSL Certificate as expected.

If you disable EHTTP, the configuration and certificates aren’t removed either. I suspect this is by design to prevent the enablement process from overwriting pre-existing configuration.

Summary

There are lots of moving parts here and the key takeaway should be Trust but Verify. As robust as the systems/processes may be within ConfigMgr, sometimes you just need to dig into the logs to verify that things are working as intended - especially if you’ve enabled a Preview feature, things change. In this case, we walked through each component, verifying configurations, until we found the misconfiguration.

Special thanks to Ronny de Jong, Nickolaj Andersen and Jake Stoker for their great posts that helps save a bunch of time and thanks Adam Juelich for letting me lend a hand, though next time it’ll have to include some smoked meats.