Dude! Where’s my vCSA SSL Cert chain?

Well, it certainly has been a while since my last post. The justification for my absence in recent months is due to the birth of my son! He is our first and so work/career life has taken a bit of back burner so I can enjoy family time being a new Dad. It’s a great experience and I’m loving it!

Right, to the issue at hand. Recently, a few of my colleagues were working on applying SSL certificates to a vCSA which drives our test environment. We were applying a trusted third party SSL certificate (from Quo Vadis) to our appliance and used the following KB:

Replacing vSphere 6.0 SSL Certificate with a custom CA Signed Cert

However, we needed to modify the .CSR but were having difficulty so this KB cleared things up for us:

Certificate Manager Utility not utilizing certool.cfg for CSR generation

Finally, we had what we needed but kept seeing roll back. This was because we had to download the certificate chain and present it to vCenter using this KB:

Replacing certificates using VMware vSphere 6.0 Certificate Manager fails at 0% with the error: Operation failed, performing automatic rollback

This appeared to work. Browsing to the web console of the vCSA showed a valid certificate from a variety of browsers on Windows machines but something wasn’t quite right our bespoke provisioning system stopped working..

Upon a little investigation when connecting via openssl to the vCSA address, we received the errors:

“Unable to get local issuer certificate”
“certificate not trusted”
“unable to verify the first certificate”

This was a problem for us as our bespoke provisioning system was not able to establish a connection to the vCSA.

The full error output is here (I appreciate it’s not code but its much easier to read on my blog):

My colleague resolved the issue by noting that the proxy configurations for service endpoints were not updated with the intermediate certificate. This can be fixed by doing the following:

1) Navigate to /etc/vmware-rhttpproxy/ssl on the vCSA.

2) Note the trustedCerts.pem file which upon initial investigation has no content! Copy and paste in the content of your Intermediate certificate (from your issuing CA) into trustedCerts.pem.

vcsaSSL3

3) Open config.xml file an an editor and find the line:

vcsaSSL1

4) Uncomment the line to ensure it is read in the config:

/etc/vmware-rhttpproxy/ssl/trustedCerts.pem

vcsaSSL2

5) Save the file and run a service restart:

Once complete, with another test to openssl the following should be observed – error free:

That is it really, nothing too special. We couldn’t find this fix in any of the VMware KB articles detailing SSL certificates. For most people, I doubt that would even notice due to browsers understanding the chain already with their built-in trusts. When you are programatically accessing the vCSA to make API calls, that is when the fun started.

100% of the credit and hardwork goes to my colleagues @claytonpeters and @dfgrain.

vROPS 6.2 for Horizon: Broker Agent

In the last post of this mini-series, I’m going to be covering the broker agent install and configuration which is required for a View environment to talk to vROPS. The agent resides on a connection server of your choosing and reports back to vROPS to get all the fancy stats that you need.

Broker Agent Config

1. The first thing you need to do is to login to the vROPS appliance to change the firewall. At first login, use root and no password. You will be prompted to change the no password to something of your choosing!

brokeragent1

2. Run the following commands once logged in:

brokeragent2

3. You will find yourself in the firewall config, at which point you need to amend the open TCP ports list to include the range as documented here.

brokeragent3

4. Save the config and restart the firewall with a

brokeragent4

5. Once complete, login to one of your connection servers. Run the broker agent installer that you’ve downloaded. Simple install, run the configuration utility when you’re done.

brokeragent5

brokeragent6

brokeragent7

6. At the config screen, enter in the IP/FQDN of your vROPS server. Enter in the pairing key as configured in my previous post. Select Pair and after a successful test, select Next.

brokeragent8

brokeragent9

7. On the next screen, enter in the details of a Horizon Administrator configured on your View Admin page. I use a service account for this, Test it and then click next.

brokeragent10

NB: During the original install (which was actually an upgrade), I had problems being unable to ever connect/test for the credential or the DB. It turns out this was due to the “locked.properties” file in the View installaton fodlers which was there from a legacy version of Horizon View and setting default protocol to HTTP. I deleted the file and everything started to work.

8. The next page, configure the username and password that is configured for the Event DB. I used the same account that is already configured in the View Admin portal. Test it and click next.

brokeragent11

9. If you wish, you can change the interval and timeouts, I left mine at default.

brokeragent12

10. Similarly, it is possible to change the logging level if you rewquire more information on the broker agent. Useful for troubleshooting agent issues.

brokeragent13

11. Make sure the service is running and then click Finish.

brokeragent14

brokeragent15

12. Login to the vROPS admin portal, navigate to “Inventory Explorer” and find “View Adapter Instance” in the list. You can see the credential you conifgured and paired with View. This should start showing objects collecting which proves that the agent on the connection server is sending stats through to vROPS. If this doesn’t change, something is wrong!

brokeragent16

The best thing is to leave vROPS alone now and give it a good amount of time before the decent statistics start to come in.
It is also worth configuring the vCenter that controls the VDI infrastructure hosts into vROPS too, so that vROPS has the complete picture of the entire platform.

This ends the vROPS for Horizon 6.2 series, I hope it’s been useful!

vROPS 6.2 for Horizon: View Adapter

In this post I’m going to be running through the additional configuration required to get the newly installed vROPS working with Horizon View, specifically the Horizon Adapter. There are a few additional specific configuration options that are required above and beyond standard vROPS which follow on from my previous two posts on this subject.

View Adapter Configuration

To get this part of the installation working, you must have downloaded the vROPS View Adapter.pak file from the VMware site.

1. From within the vROPS admin portal, select “Solutions” in the navigation pane. Click the green plus symbol to add a solution and browse to the ViewAdapter.pak file.

ViewAdapter1

2. Select upload and when complete, select Next.

ViewAdapter2

3. Accept the EULA and move on, this next step whilst the solution installs can take 15+ minutes so grab a coffee at this point!

ViewAdapter3

ViewAdapter4

4. After this was complete, I repeated the same process but installing the Management Pack for storage devices for extra VSAN visibility.

ViewAdapter5

5. Once finished. Navigate to the licensing tab on the navigation pane. Select the VMware Horizon Licensing and select edit.

ViewAdapter6

6. Under the vROPS for Horizon option, ensure the license key entered earlier is selected and hit next.

ViewAdapter7

7. Things get a bit mad here. The only understanding I have of it is to associated objects that are VDI specific to the Horizon license.

In the first Select the Object Type that matches all of the following criteria drop-down menu,select Host System, define the criteria Relationship, Descendant of, is, and type. All Hosts in the Object name text box.
In the second Select the Object Type that matches all of the following criteria drop-down menu,select Virtual Machine, define the criteria Relationship, Descendant of, is, and type. All Desktop VMs in the Object name text box.
In the third Select the Object Type that matches all of the following criteria drop-down menu,select Datastore, define the criteria Relationship, Descendant of, is, and type. All Storage in the Object name text box.

ViewAdapter8

8. Hit next and finish when done.

ViewAdapter9

9. Next up, head to the licensing tab on the navigation pane. Select the Product Licensing and select edit.

ViewAdapter10

10. Under the vRealize Operations Manager option, ensure the license keys entered earlier is selected and hit next.

ViewAdapter11

11. There is some more magic that now needs to be done, similar to step 7.

In the first Select the Object Type that matches all of the following criteria drop-down menu, select Host System, define the criteria Relationship, Descendant of, is not, and type All Hosts in the Object name text box.
In the second Select the Object Type that matches all of the following criteria drop-down menu, select Virtual Machine, define the criteria Relationship, Descendant of, is not, and type All Desktop VMs in the Object name text box.
In the third Select the Object Type that matches all of the following criteria drop-down menu, select Datastore, define the criteria Relationship, Descendant of, is not, and type All Storage in the Object name text box.

ViewAdapter12

12. When complete, hit next and finish.

ViewAdapter1

13. Head back to the solutions section and select VMware Horizon. Click the Cogs symbol to edit…

ViewAdapter1

14. Select the Horizon Adapter and then the green cross to add an instance.

ViewAdapter1

15. Enter in the name and also a key that you can use later to pair your servers with. This can be anything secure. Click OK and Save the settings of the Adapter.

ViewAdapter1

vROPS 6.2 for Horizon: Configuration

This weeks post is a continuation on the vROPS 6.2 for Horizon install I’ve done, focusing mainly on the post configuration tasks after the initial appliance deploy.

Configuration

1. Once your appliance has been deployed, you will be able to navigate to https://serverip/admin

2. You are presented with a Getting Started page. If you are deploying multiple appliances in a HA configuration then you can expand on your installation. For me, I am doing a new installation of a single instance of vROPS as there is no requirement for HA.

vROPCFG1
 

3. Plow through the getting started menu.

vROPCFG2
 

4. Enter in some admin credentials for when you need to login later on.

vROPCFG3
 

5. If you have a certificate for the service, now is the time. Failing that, use the defaults.

vROPCFG4
 

6. Give your cluster node a name (I used FQDN) and enter in your NTP server address(s).

vROPCFG5
 

7. Finish at the initial setup finalization screen.

vROPCFG7
 

8. You are taken to the admin page where you can see a system status. You need to select to start vROPs.

vROPCFG8
 

9. You will receive a prompt which is warning about a cluster configuration with multiple nodes. Click Yes to accept and proceed.

vROPCFG9
 

10. This part can take a while, you will notice the node information change and the status as “Going Online” and then to “Online”.

vROPCFG10
 
vROPCFG11
 

11. At this stage, you can logout of the web interface and go back. Be careful not to go to http:///admin otherwise you’ll end up in the previous setups menu.

12. Login using the admin credentials setup in step 4 and you will be presented with the remaining core vROPS configuration menus.

vROPCFG12
 
vROPCFG13
 

13. Enter in your product key infromation here unless on evaluation.

vROPCFG14
 

14. If you want to enable customer experience, do so here. Then finish.

vROPCFG15
 
vROPCFG16
 

15. The next stage for me was to add in the rest of our licenses, so on the right hand side, navigate to Licensing and add in your allocations.

vROPCFG17
 

I’m going to end the post here as there is a fair bit to do still and it is easier for me to chunk it up as to not have one long mammoth post! The last remaining bits to cover will be installing the View adapter and also the Broker agent on the connection servers to actually collect statistics.

vROPS 6.2 for Horizon: Installation

In recent days, VMware have released vRealize Operations Manager 6.2. I’ve been meaning to overhaul our VDI environment with vROPS 6.2 for Horizon monitoring, so why not blog about it? I recently installed VMTurbo to monitor our Production estate which I’ll post about in the future, but this vROPS instance is specifically going to be working on VDI.

I’ve broken the effort down into several posts as there is surprisingly a lot to cover if it is to be done thoroughly. This is a greenfield deployment on latest versions, following the documentation from VMware.

Installation

This section is mainly running through the brief steps to get the appliance up and working in your environment.

1) On the vCenter where you want to deploy the vROPS appliance, deploy a new OVF template…
vROP1

2) Click next, accept the EULA.
vROP2

vROP3

3) Enter in your server name and select where it’s folder location.
vROP4

4) For your deployment type, select the best for your environment, given the advice below the drop down menu.
vROP5

5) Select the cluster where you want to deploy the appliance, resource pool and datastore settings:
vROP6

vROP7

vROP8

6) Select the disk provisioning type and the network port group where you want to place the VM.
vROP9

vROP10

7) Enter in the configuration details of your appliance, making sure: Correct Time Zone, Default Gateway for network, DNS and Appliance IP information are all filled in.
vROP11

8) Once the VM has deployed, boot it up!
vROP12

The best thing about VMware appliances is the ease of deployment. This post doesn’t show a great deal to most VMware admins, but some of the steps get a bit more in depth and complicated later on. My next post will be on the configuration of the appliance through the admin interface.