Do you know how hard it is to find things when you don’t know what to search for??
I’m paranoid. We recently upgraded our site to ConfigMgr 2002, first fast ring, then the hotfix. We also installed a Cloud Management Gateway, re-worked our Boundary Groups to handle VPN better, added a second Management Point and generally made a butt-load of changes to our environment to help our remote clients have access to content more easily. I’ve been frantically spot checking clients to ensure they are working now and hoping that one of the many changes hasn’t caused them to drop off the face of the earth. There was also a bug that got fixed in slow ring/hotfix, related to client failing to authenticate the CMG that wanted to ensure was working now. Unfortunately, when you go looking into client logs, you can end up chasing lots of rabbits. On of the issues that I’m having is that some of our clients that are likely at the office are just going to be offline until everyone returns to work post-COVID-19. At the same time, we have some other number of online clients that are showing no client or no recent heartbeats but when I ping them, a decent number respond. On a regular basis, we have about 300 machines in some sort of unknown state and we have just learned to live with it since it’s largely environmental. We have the PFE client health tool implemented, but honestly, I’m not sure if it’s helping or hurting at this point.
I feel like I should stat the purpose here - I want to disable metering and get ConfigMgr clients to check back in without having to reboot the client.
DISCLAIMER: The ‘solution’ provided below uses several ‘unsupported’ steps to achieve the intended results and is provided for informational purposes only.
Without further ado, here’s the brain dump firehose…
See, what had happened was…
I used Recast Right Click tools (free community edition) not a sponsored post :-) to trigger a client repair on around 500 devices. About 60 of the machines were online even though they were showing no client in the console. I waited to see if they would check in and they never did. Started looking at client logs and noticed that the repairs seemed to be working and the clients appeared to be fine, but when I checked CcmMessaging.log, it was full of these errors:
I opened a remote PowerShell console to some of the machines and I started checking to see if metering had been enabled on the wireless adapters by the end user. I ran some NETSH commands to find the profiles where metering was enabled and the clients began working. I was excited until I got to my next machine and discovered that it was on the LAN at the office and we don’t enable metering on any of our adapters by default.
I tried the same NETSH commands and they didn’t work on the LAN adapters. It took me a while but I stumbled on several posts that had bits and pieces that helped find what I needed. There’s a fair amount of detail upcoming, but the script is at the end. I want to include all of the components that I discovered so I can consolidate everything into one place.
Also, I’m just making most of this up based on what I have been able to piece together. I have spent WAY too much time digging and just need to move on. I could be REALLY wrong, but it’s the best I’ve got. If you know more, send me a note and I’ll delete it, I mean, I’ll update the post.
Network Adapter Metering
When setting up your network adapters in Windows 10, I’m sure you’ve seen this section under the properties for your network adapters. By enabling metering, you tell Windows to limit the amount of data that it and other applications can put through the adapter. This is a way to reduce consumption of pay-per-use internet plans and or wifi hotspots, etc.
One problem with this setting is that it will actually disable Windows updates (there’s an option to override this as well) AND more importantly, it will prevent the installation of the ConfigMgr client. Also, if you haven’t changed your Client Settings policy from the default ‘Block’, devices with the client installed won’t communicate with your MP - as shown in the error log above.
Another problem is that there appear to be Group Policies for disabling the ability for users to change this setting for wireless adapters or cellular cards, but I wasn’t able to find one that prevented users from setting metering on their wired LAN adapters.
It also appears that there have been changes to where/how you manage metering in the registry/Group Policy in later versions of Windows (1709 and later).
Computer Configuration \ Administrative Templates \ Network \ WLAN Service \ WLAN Media Cost
Basically, there are some options that are somewhat difficult to lock down that users can change and break things.
ConfigMgr Client Event Handling
The ConfigMgr client has lots of event handlers that it uses to monitor system processes and trigger processes in response to the events. One of the items it monitors is network connectivity and adapter events. Whenever you enable metering on your adapter, you can watch the CcmNotificationAgent.log to see that the network cost was changed to 2 (Enabled/Fixed/Metered) or if you disable, it will show 1 (Disabled/Unlimited/Not Metered). If you enable Debug logging on your ConfigMgr client, you will see even more info (not shown). Martin has a great script to enable debug logging here.
You can even run a PowerShell command to check the ConfigMgr client for it’s current Network Cost - it will return 1 or 2. From what I can tell, ConfigMgr monitors for a State Change event on the network adapter, then updates it’s network cost value and caches it. As far as I could determine, there doesn’t appear to be a way to force the client to check for the network cost again nor is there a way to trigger a network state change that it would see.
You can watch the NetworkProfile event log and see the State Change event when you enable/disable metering on the adapter.
You can see the
ConnectionCostChanged entry. This is what ConfigMgr is looking for.
Here’s what ccmsetup.log looks like if you have metering enabled and try to install the ConfigMgr client. It’s a hardcoded check too, so no overriding it.
The Windows Network Adapter Black Hole
I’ll be the first to admit, I don’t like networking. I dislike it even more after this adventure. So we’ve seen where in the UI you can enable/disable metering. We’ve seen in GPO where you can configure WLAN cost options. We’ve seen the network event that gets triggered when you make a change. What we haven’t seen is how you can programatically change costing on a LAN adapter in a way that causes the ConfigMgr to see the event and trigger a re-cache of costing for the network adapter. Spoiler - I can’t find a way, but here’s how far I got.
There’s a Windows system DLL
dusmapi.dll(Data Usage API) and a service
dusmsvc (Data Usage) that are used to manage your network adapter profile settings related to - you guessed it - DATA USAGE!!
By watching ProcMon, I could see that
dusmapi.dll was being called but I can’t figure out how to import the dll into PowerShell or VS to try to manually call it. However, I did find find a dump of all of the functions in the dll.
On line 25 we can see that there’s a function called
DusmSetUserCost. That’s the one we want. I’ve searched GitHub, in the Windows networking PowerShell module and everywhere else and I can’t find any info on how to call this but I’m certain it’s the key to triggering the costing state change event on the adapter.
The other related bit here is that the setting that gets created by toggling metering on an adapter is a registry key
Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\DusmSvc\Profiles\<AdapterGUID>\*\UserCost. The value will be set to 2 if it’s enabled and 0 if you disable it. If you never touch it, you will only have the base key of
A light in the darkness
There are other references to network costs, but only for querying them, nothing for setting them.
Run this in PowerShell and you can dump out the cost info for all of your network adapter profiles.
A nice guy named Wil (with 1 L) Taylor even has a module to help dump cost info out. Unfortunately, there’s not a module for modifying the settings.
I have to give credit to Franck Richard for his nice write ups on this topic as well. He provided some of the only pieces to this puzzle that I found initially. These 2 posts talked about the issues I was facing and he even has a script for it editing it that I used bits of in my final solution.
The script has 3 main parts. LAN, WLAN and ConfigMgr. It checks the registry for any LAN profiles set using the Settings app in Windows. These will be under the dusmsvc registry key here:
Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\DusmSvc\Profiles. Then it uses NETSH to find WLAN profiles with metering not set to
Unlimited. And finally it checks to see of the ConfigMgr client costing is set to metered. Examples and info are included in the script.
As soon as I hit publish, I considered that perhaps I could have handled this slightly differently. Honestly, I wrote all of the code before I found the ConfigMgr CCM_NetworkSettings class. Now that I’m thinking about it and understand things more, here’s how I would ACTUALLY do it.
This is part of the final code in the repo. It’s the part that resets the CCM_NetworkSettings class. The cool part here is that it allows the client to check in and get new policies, even if metering is enabled. My issue was that we had our client settings policy set to Block. If I remove the policy (with the code below), the clients can get an updates.
Next, I would add a Configuration Item or a scheduled script to check for and remove metering. I haven’t test Group Policy preferences but since the registry key has a * in the path (who decided that was a good idea??) and * is a wildcard pretty much EVERYWHERE, I’m not sure how well that will work. Either way, having a mechanism to block or remove this from being set again would be the next step.
And instead of a script, I think I could have just updated the client settings policy to Allow ConfigMgr over metered connections then used Right Click Tools to reset client policies on any devices that I suspected had metering enabled.
This has been a fun dive into the bowels of network connectivity. I’m REALLY hoping someone sends me an email with the code to update costs using the Windows API. Feels like there’s still some opportunity for polishing this turd.