When we were preparing to upgrade our domain controllers from 2008 R2 to 2016, we of course began inventorying all functions that were going to be migrated. One of these was Network Policy Server (NPS). NPS is one of the easiest services to migrate to a new system, since it’s basically a simple backup and restore, and we weren’t expecting any difficulties. Of course had things gone smoothly, this post wouldn’t exist.

A little background:

We use NPS for multiple functions, including Cisco AAA and wireless device authentication. Our Meraki wireless infrastructure has multiple SSIDs, some of which are for domain-joined devices, and others for non-domain devices. The domain-joined devices use 802.1x certificate-based authentication and the non-domain joined devices use various other methods.

NPS was migrated from 2008 R2 to 2016 and everything other than the 802.1x certificate authentication worked. The 2016 servers were correctly setup as RAS servers and had certificates that were valid for client & server authentication, as needed. I spent quite a long time troubleshooting this with no success, and eventually had to back-burner it. The 2008 R2 servers were due to be retired, so as a temporary measure I migrated NPS to a couple 2012 R2 servers, all of which worked flawlessly after about two minutes worth of effort.

This was pretty early in 2016’s lifecycle, so since it worked in 2008 R2 and 2012 R2, we chalked it up to a bug in the OS and intended to pursue it with Microsoft at a later date.

Before I had a chance to call Microsoft, another project had us looking at our PKI configuration. We were having issues using LDAPS with this particular system (despite using it elsewhere), and after extensive troubleshooting the vendor told us they thought the hash algorithm used by our PKI was unsupported.

Our PKI was originally setup on a 2003 server back when that was the current server OS. It had been migrated to a newer server since then but the hash algorithm had never been changed; it was still using md5RSA. Now upgrading the hash algorithm isn’t something to be taken lightly, as it can break a lot of things, but in our case since our PKI is pretty simple I decided (after backing everything up) that the upgrade risk was minimal. I changed the hash algorithm to SHA256, reissued RAS certificates to the NPS servers and just like that things started working.

I can’t find any changelog indicating why NPS would not work with an older certificate hash on 2016 but empirical evidence shows the hash was the problem.

There are plenty of resources about upgrading your root CA’s hash algorithm, but this is a good start from Jason Bender at Microsoft.