VMware & Azure Site Recovery – Part 1: Pre-Requisites

I’ve had the opportunity of investigating Disaster Recovery in my role recently. I have been looking at costs and methods of bringing our critical systems online in the event of a primary data center outage.

Without going into too much detail on my existing employer, there are many things to review and architecting DR into the existing infrastructure isn’t the easiest thing to do. Given our relationship with Microsoft, I was asked to investigate Azure Site Recovery to see if it was a viable option to provide us with a DR site in the cloud.

I’m going to be blogging in a small series on the technical implementation required to achieve VMware VM’s failing over from an on-site VMware cluster to an Azure Site Recovery instance. Hopefully if all goes well I’ll add to the series as I go, but for now I’m going to keep it simple with basic deployment.

Pre-Requisites

The entire process that I am following has been documented by Microsoft and gives some good detail on how to achieve VM replication into the cloud.

It is important to read through the checklist of required items before starting the setup. This can be done beforehand or during the actual implementation. I surmised it down to the following:

Cloud:

1) An Azure account, free trial possible (I have MSDN sub)
2) Azure Storage, somewhere to put your data.
3) Azure Network, VM’s location after fail over.

On-Site:

1) Build a new 2012 R2 Process/Management Server with necessary specification (Ready for installing ASR components)
2) External network connectivity to cloud services.
3) VMware vCenter + ESXi 5.5 or greater.
4) Guest machines that do not exceed certain limitations of the service (e.g. – No disks larger than 1TB)

Once the pre-steps are complete, it was on to configuring the magic….

1) Signed in to my MSDN subscription and setup the Azure Free trial ($150 a month)
ASR1

2) Login to https://portal.azure.com

3) Navigate to the market place, Networking, Virtual Network.
ASR2

4) If this is all new, it’s best to stick with the Resource Manager deployment model as that is the latest and greatest. Click Create.
ASR3

5) Create your virtual network by filling in your requirements. I went for the large default address space, naming it and then a small subnet within that for testing. In this instance I also created a new Resource Group for
ASR4

ASR5

NB:- A handy tip is to pin certain objects to the dashboard so you can see them on your main screen. I found this useful for the on-site Process/Management server.

6) Navigate to the market place, Storage, Storage account.
ASR6

7) Create a storage account as instructed. I opted for standard performance and keeping the data geographically local as I’m only performing a PoC. Maintaining the resource in the previously created group.
ASR7

8) Navigate to the market place, Monitoring + management, Backup and Site Recovery. Create a recovery vault to group resources together.
ASR8

ASR9

The next thing worth doing is creating an account in your Directory Services that VMware and ASR can use.

1) Create AD Service account
2) Add to vCenter/Datacenter object where VM’s will be replicated from.
3) Create a role for “Azure Site Recovery” and give it the following permissions:
ASR10

ASR11

Here is a good place to stop with this part of the guide. The hard part of this post was making sure all the pre-configuration bits are done and that you are ready to proceed.

In the next post I’ll run through configuring the actual Site Recovery and making the components communicate. The most notable comment I’d have for all of this is that Microsoft have gone quite a distance in making this process as easy as possible. That doesn’t mean, however, that it goes completely without technical caveats which I’ll cover later on in the series.
Until next time!

vCSA 6.5 Upgrade Fail – Migration Assistant & VUM

I, like many have recently tried to update my vCSA 6.0 to 6.5. Part of this work involves running the migration assistant on the existing Windows Server running VMware Update Manager.

I had a failure recently and there was nothing noting how to fix it. The issue was trying to run the Migration assistant, entering in the administrator@vsphere.local password and getting a big fail message.

Error message:

vumerror2


Error: A problem occurred during authentication to the legacy vCenter Server using the provided credentials. Resolution make sure the vCenter Server is up and running. Verify you have entered the correct credentials.

Now, before I started my upgrade (blogs to follow) I checked I had all my passwords and that they were correct. I went as far as logging in to everything to test this, so I know the password was correct.

VUM was working through console and services were there, so what was the issue?

Then I remembered that a while ago I applied SSL Certificates to the vCSA for some automation testing!

The fix:

1) Login to the VUM server

2) Navigate to C:\Program Files (x86)\VMware\Infrastructure\Update Manager (or install path of your environment)

3) Load the VMwareUpdateManagerUtility.exe and login with SSO Admin (it worked!)

4) Navigate to “re-register vCenter Server” and enter in the details again.
vumerror5

5) Restart the VMware Update Server Service when prompted.

6) Retry the Migration Assistant
vumerror4

7) Success!

I couldn’t find any articles detailing the fix when I encountered this issue, so hopefully this will help someone in need of a quick fix! 🙂

Enjoy!

Recent training and certification

It’s been a while since my last post but thought I would write about my most recent technology related experiences. I’ve been quite lucky in that I’ve been sent on multiple training courses with work recently which have really enabled me to learn some new skills and build on ones that I already had…

vmwarelogo


VMware vSphere 6: Design and Deploy

I had the pleasure of attending the most recent version of the advanced vSphere course in London a few weeks ago. The actual course information can be found here:

VMware vSphere: Design and Deploy Fast Track [V6]

I was very lucky to be staying in the QA Training building overlooking Tower Bridge which was an excellent place to study, if anyone is ever training in the UK I’d recommend it!

The course was lead by Gareth Baguley (no relation to Joe) who has been a VMware trainer for quite some time. It was evident by his knowledge on all things vSphere, clearly a passion of his. When a guy remotes in to their home labs to demonstrate technology and concepts then you know you are in the right room!

The technical level was just right for VCAP study. The actual content is split around 50:50 between the VCAP-DCA/DCD material. Having passed the VCAP DCA in 5.5 I was very much more interested in VMware’s design methodologies and concepts. On a personal note I wish the design was a larger part of the course but it was still invaluable to progress to the DCD (soon!).

Overall I’d highly recommend this for anyone who is either looking to progress from VCP to VCAP or for those who might be very into VMware but wants course to attend. Once you have attended this course you are able to sit all certifications. This might be good for veterans who have expired VCP’s but don’t want to sit through VCP level study. I’m definitely aiming to do the V6 DCD in the next year .

Red Hat Training

redhatlogo

Also in the past month or so I’ve been lucky enought to attend 2 weeks of Red Hat Linux 7 training:

RH124
RH134

The first course was more of an introduction into basic RHEL administration. This is a great course as an absolute back to basics style where if you have no previous knowledge you will be in good hands. I found parts of it a little slow but also at the same time it filled in quite a few gaps in my knowledge; not being from a Linux background, having only dabbled in the past.

The second course was a bit more in depth and started to go a bit more advanced into things like storage, security, networking and other concepts which are vital to any system administrator. At the end of the weeks course I had a 2.5 hr exam which reminded me very much of the VCAP-DCA I did for VMware 5.5. Fully practical exam which followed the course material very well and was true to the blueprint that Red Hat publish.

I sat the exam and there were some tricky parts that I had not quite fully prepared for. I managed to take my time and work through the problems, ultimately coming out thinking I’d done well enough for a pass. The following day I received a notification stating that I obtained a score of 283/300 which I was very happy with!

I’m going to start study for the VCAP-DCD and possibly RHCE in the future so will post my experience in order to hopefully help anyone who stumbles across my ramblings!

Dude! Where’s my vCSA SSL Cert chain?

Well, it certainly has been a while since my last post. The justification for my absence in recent months is due to the birth of my son! He is our first and so work/career life has taken a bit of back burner so I can enjoy family time being a new Dad. It’s a great experience and I’m loving it!

Right, to the issue at hand. Recently, a few of my colleagues were working on applying SSL certificates to a vCSA which drives our test environment. We were applying a trusted third party SSL certificate (from Quo Vadis) to our appliance and used the following KB:

Replacing vSphere 6.0 SSL Certificate with a custom CA Signed Cert

However, we needed to modify the .CSR but were having difficulty so this KB cleared things up for us:

Certificate Manager Utility not utilizing certool.cfg for CSR generation

Finally, we had what we needed but kept seeing roll back. This was because we had to download the certificate chain and present it to vCenter using this KB:

Replacing certificates using VMware vSphere 6.0 Certificate Manager fails at 0% with the error: Operation failed, performing automatic rollback

This appeared to work. Browsing to the web console of the vCSA showed a valid certificate from a variety of browsers on Windows machines but something wasn’t quite right our bespoke provisioning system stopped working..

Upon a little investigation when connecting via openssl to the vCSA address, we received the errors:

“Unable to get local issuer certificate”
“certificate not trusted”
“unable to verify the first certificate”

This was a problem for us as our bespoke provisioning system was not able to establish a connection to the vCSA.

The full error output is here (I appreciate it’s not code but its much easier to read on my blog):

My colleague resolved the issue by noting that the proxy configurations for service endpoints were not updated with the intermediate certificate. This can be fixed by doing the following:

1) Navigate to /etc/vmware-rhttpproxy/ssl on the vCSA.

2) Note the trustedCerts.pem file which upon initial investigation has no content! Copy and paste in the content of your Intermediate certificate (from your issuing CA) into trustedCerts.pem.

vcsaSSL3

3) Open config.xml file an an editor and find the line:

vcsaSSL1

4) Uncomment the line to ensure it is read in the config:

/etc/vmware-rhttpproxy/ssl/trustedCerts.pem

vcsaSSL2

5) Save the file and run a service restart:

Once complete, with another test to openssl the following should be observed – error free:

That is it really, nothing too special. We couldn’t find this fix in any of the VMware KB articles detailing SSL certificates. For most people, I doubt that would even notice due to browsers understanding the chain already with their built-in trusts. When you are programatically accessing the vCSA to make API calls, that is when the fun started.

100% of the credit and hardwork goes to my colleagues @claytonpeters and @dfgrain.

vROPS 6.2 for Horizon: Broker Agent

In the last post of this mini-series, I’m going to be covering the broker agent install and configuration which is required for a View environment to talk to vROPS. The agent resides on a connection server of your choosing and reports back to vROPS to get all the fancy stats that you need.

Broker Agent Config

1. The first thing you need to do is to login to the vROPS appliance to change the firewall. At first login, use root and no password. You will be prompted to change the no password to something of your choosing!

brokeragent1

2. Run the following commands once logged in:

brokeragent2

3. You will find yourself in the firewall config, at which point you need to amend the open TCP ports list to include the range as documented here.

brokeragent3

4. Save the config and restart the firewall with a

brokeragent4

5. Once complete, login to one of your connection servers. Run the broker agent installer that you’ve downloaded. Simple install, run the configuration utility when you’re done.

brokeragent5

brokeragent6

brokeragent7

6. At the config screen, enter in the IP/FQDN of your vROPS server. Enter in the pairing key as configured in my previous post. Select Pair and after a successful test, select Next.

brokeragent8

brokeragent9

7. On the next screen, enter in the details of a Horizon Administrator configured on your View Admin page. I use a service account for this, Test it and then click next.

brokeragent10

NB: During the original install (which was actually an upgrade), I had problems being unable to ever connect/test for the credential or the DB. It turns out this was due to the “locked.properties” file in the View installaton fodlers which was there from a legacy version of Horizon View and setting default protocol to HTTP. I deleted the file and everything started to work.

8. The next page, configure the username and password that is configured for the Event DB. I used the same account that is already configured in the View Admin portal. Test it and click next.

brokeragent11

9. If you wish, you can change the interval and timeouts, I left mine at default.

brokeragent12

10. Similarly, it is possible to change the logging level if you rewquire more information on the broker agent. Useful for troubleshooting agent issues.

brokeragent13

11. Make sure the service is running and then click Finish.

brokeragent14

brokeragent15

12. Login to the vROPS admin portal, navigate to “Inventory Explorer” and find “View Adapter Instance” in the list. You can see the credential you conifgured and paired with View. This should start showing objects collecting which proves that the agent on the connection server is sending stats through to vROPS. If this doesn’t change, something is wrong!

brokeragent16

The best thing is to leave vROPS alone now and give it a good amount of time before the decent statistics start to come in.
It is also worth configuring the vCenter that controls the VDI infrastructure hosts into vROPS too, so that vROPS has the complete picture of the entire platform.

This ends the vROPS for Horizon 6.2 series, I hope it’s been useful!