Background

Microsoft announced that it will release an update to help strengthen the security of configurations for LDAP channel binding and LDAP signing on Active Directory domain controllers. Below is the Microsoft KB for further details on this update.

ADV190023 | Microsoft Guidance for Enabling LDAP Channel Binding and LDAP Signing

To help support this Microsoft update, there are steps you must take for both Workspace ONE UEM and IDM (Access). The required steps are outlined in two separate VMware KBs listed below.

• Support LDAP Signing and LDAP Channel Binding with VMware Workspace ONE Unified Endpoint Management (UEM) (77539)
• [Resolved] HW-111374: Support LDAP Signing and LDAP Channel Binding with VMware Workspace ONE Access, Identity Manager (77158)

Almost everyone has at least a Workspace ONE UEM environment. Some such as myself also have the Workspace ONE Access/Identity Manager environment. So let’s ‘free two birds with one key’ with this post.

Changes to be made in Workspace ONE UEM

In Workspace ONE UEM, go to GROUPS & SETTINGS > All Settings > System > Enterprise Integration > Directory Services. Next to Encryption Type, select SSL which will turn the port number from 389 to 636.

Changes to be made in Workspace ONE Access/Identity Manager

In Workspace ONE Access/Identity Manager Administration Console, go to Identity & Access Management. Then, click on your directory name. In the Certificates section, check off the box This Directory requires all connections to use SSL. Upon doing so, you will see that the Server Port in the Server Location section changes from 389 to 636. Then, append the Intermediate (if used) and RootCA certificates from your domain controller next to SSL Certificate in the Certificates section. Finally, enter the Bind User Password in the Bind User Details section and click Save.

In my case, I  also had to change the Server Host in the Server Location to one that supports SSL LDAP.

Troubleshooting

While there was no issue with enabling SSL in Workspace ONE UEM, I did encounter three different issues in Workspace ONE Access. One of them turns out to be a known issue/bug with version 19.03 and it’s resolved with version 20.01.

As part of the troubleshooting, I review both the connector.log and the connector-dir-sync.log on the Workspace ONE Access connector under C:\VMware\VMwareIdentityManager\Connector\opt\vmware\horizon\workspace\logs.

When troubleshooting with VMware support, you may be asked to upload these logs from your IDM connector. Instead of zipping the entire log folder, you can also export the log bundle easily by visiting https://name_of_your_IDM_server:8443/cfg/login. Keep in mind that you will only need to upload the log bundle from the server that is configured for directory sync.

The file size for the log bundle is quite large so you will want to upload it via the VMware FTP server via the steps outlined in this KB: Uploading diagnostic information for VMware through the Secure FTP portal.

Issue #1

Upon enabling SSL, I noticed my scheduled directory sync was failing. I also noticed I was no longer able to search for group/user for scheduled sync within Sync Settings.

The below was observed in the connector logs:

• The Workspace ONE Access connector continued to connect to the domain controller that was initially configured instead of the new one that I changed to.
• The Workspace ONE Access connector continued to connect via port 389 instead of 636.

Per VMware support, this issue is a confirmed bug with the config-state.json file in version 19.03 and was fixed in version 20.01. Here’s a snippet from my connector log:

2020-06-02T10:35:00,396 ERROR (Thread-6) [Tenant;Email Adress@Tenant;127.0.0.1;] com.vmware.horizon.connector.rest.SyncConfigurationRestController – Directory service exception.

com.vmware.horizon.directory.DirectoryServiceException: Problem connecting to directory. Domain Controller(s) failed to respond.
at com.vmware.horizon.connector.admin.LdapService.isGroupDNValid(LdapService.java:254) ~[classes/:19.03.0.0 Build 13322315]

at org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter.java:389) [urlrewritefilter-4.0.4.jar:4.0.4]

at org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter.java:389) [urlrewritefilter-4.0.4.jar:4.0.4]

Caused by: javax.naming.CommunicationException: simple bind failed: Server_Host:389

Solution for issue #1

For this issue, the solution is to update the config-state.json file manually with the correct domain controller name and/or server port number. Before making this change, you must first stop the VMwareIDMConnector service from running.

Then, browse to the config-state.json file at C:\VMware\VMwareIdentityManager\Connector\usr\local\horizon\conf\states\tenant\#### (4 digit numbers) and make a copy of this file as backup before making any changes. Afterward, restart the VMwareIDMConnector service and try directory sync again in the VMware IDM console.

Here’s a snippet from the config-state.json file where the changes were made:

“directory” : {
“isConfigured” : true,
“type” : “ActiveDirectory”,
“host” : “updated domain controller that supports LDAP SSL“,
“port” : 636,
“isSsl” : true,
“useSRV” : false,
“useStartTls” : false,
“ignoreUsersOutsideUserSyncScope” : false,
“baseDN” : “DC=XXX,DC=XXX,DC=XXX”,

If you have more than one VIDM connectors in your environment, be sure to repeat the same steps above on all of them for consistency.

Issue #2

For my second issue, no one can log into the VMware IDM console with his/her AD credential even though the account was in fact present. Initially, VMware support insisted this was because SSL wasn’t enabled. However, I swiftly disproved that theory after pointing out that we didn’t experience this issue in our other environment without SSL enabled.

After providing the log bundle via the method described above, VMware support suspected that there might be a communication issue between the VMware IDM connector and domain controller. Here’s a snippet from my connector log:

2020-05-29T10:35:10,126 INFO  (Thread-3) [TENANT;-;127.0.0.1;] com.vmware.horizon.directory.ldap.LdapConnector – Starting LDAP Query: Host: ldap://FQDN_of_DC:389 PageSize – 1000 SearchDN – distinguishedName=dc=xxx,dc=xxx SearchFilter – (&(objectCategory=person)(sAMAccountName=username)) Scope – 22020-05-29T10:35:10,126 INFO  (Thread-3) [TENANT;-;127.0.0.1;] com.vmware.horizon.directory.ldap.LdapConnector – Query Completed for SearchDN – dc=XXX,dc=XXX SearchFilter – (&(objectCategory=person)(sAMAccountName=username))2020-05-29T10:35:10,126 INFO  (Thread-3) [TENANT;-;127.0.0.1;] com.vmware.horizon.directory.ldap.LdapDirectoryService – User username@MYDOMAIN.COM not found under base DN dc=xxx,dc=xxx – FAILURE8:562020-05-29T10:39:02,533 INFO

Solution for issue #2

VMware support has two suggestions:

• Confrim both the config-state.json file and the IDM admin console have the same domain controller info. This is similar to the troubleshooting steps for issue #1.
• On the VMware IDM connector, run the LDAP Admin tool to confirm if the bind user is able to fetch the user from the domain controller.

Since we already reviewed the first bullet point under issue #1, let’s talk about the second bullet point further. First, head to the website to download the installer. Then, save it to the VMware IDM connector server.

Once it’s executable is extracted from the zip file and launched, click on the icon shown below.

Then, click New connection.

Fill in the remaining information and click Test connection.

If the test connection is successful, you will get this prompt.

If the test connection is not successful (i.e. when an invalid credential was entered), you will get this prompt.

When the connection is successful, clicking on Fetch DNs will give you all the DNs within your domain. Go ahead and click OK.

This proves the bind user is able to fetch the user from the domain controller.

In my case, I was not able to establish a connection successfully without checking off the box for SSL. Below was the error when trying to do so.

So this might suggest SSL was required after all, but then again it wasn’t the case in my other environment and I had no issue with logging in with any AD account to the VMware IDM console.

Unfortunately, enabling SSL still does not allow me to log into the IDM console with any AD account. This solution to this issue, as it turns out, was very simple.

For my Base DN, it was initially configured with just the top OU info such as DC=COMPANY,DC=COM while my Bind User DN was configured with more detailed info. By updating my Base DN to match the Bind User DN, I was able to log into the VIDM console with any AD account.

Possible Solution for BOTH issue #1 and #2

In some cases, the Server Host in the Server Location of your IDM console might point to a load balancer name instead of a single domain controller. To troubleshoot further, you can:

1. Modify both the IDM console and JSON file with a single domain controller hostname to confirm it’s working with or without SSL enabled.
2. Use Wireshark to monitor the traffic between the IDM connector and the FQDN of the load balancer name. Perhaps there are additional firewall rules to be created/modified.
3. Update the domain_krb.properties and krb5.conf files on the connector server with all the available domain controller hostnames.

VMware support recommends the links below for bullet point #3.

• VMware Identity Manager is slow during Active Directory sync (2144953)
• VMware Identity Manager – Could not Pull the Required Object From Identity Manager
• Editing the domain_krb.properties file

Issue #3

Directory Sync Frequency is set to run every hour. However, it’s running twice per hour.

Solution for issue #3

• The config-state.json file might still have incorrect information after changes have been made updated on the VIDM console (see issue #1 above).
• If you have multiple VIDM connector servers, the pertinent info within the config-state.json file such as server hostname and port number should be the same on all of the connector servers.
• Try restarting either the VMwareIDMConnector service or the connector server.
• If there is a duplicate folder created for the config-state.json file, then directory sync will occur twice.

I hope you find this post helpful!