Category Archives: VMware

Detailed VMware Host Network Information

Following my previous script ‘Extracting Cisco Discovery Protocol Info‘ I decided to tidy up the script, give a more controlled output and add a few items to the output.

The following script will add some nice host network information into an object which is exported to a csv file for passing to the network guys or can be used to find your server in that mess of cables that are always meaning to be tidied in the data center.

You will get:

  • Host,
  • Physical Nic Name
  • Speed
  • MAC
  • Switch Device ID
  • Port ID
  • Observed Network ranges
  • VLAN’s

Who deleted my VM ?

Today I had a colleague come to me and say someone had deleted his VM, he didn’t know when and thinks it may have been a couple of months ago, he didn’t know which host or which datastore it was in. could I tell him when and who. Hmmmm, time to start trawling through the logs I thought.

A quick Twit from @stahler told me that this would be in the Virtual Center Database so, here is a quick powershell which I used to find the culprit…

# Fill in the following information:
$SqlServer = "MYDBSERVER";
$SqlDB = "VMwareDataBase";
$MYVM = "TESTSERVER1"
$TypeofEvent = "vim.event.VmRemovedEvent"
# The vim.event.VmRemovedEvent is a Removed action from VC you can also use :
# vim.event.VmGuestShutdownEvent
# vim.event.VmPoweredOffEvent
# vim.event.VmConnectedEvent

Function Read-VIDB ($SqlQuery)
{
# Setup SQL Connection
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Server = $SqlServer; Database = $SqlDB; Integrated Security = True"

# Setup SQL Command
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand
$SqlCmd.CommandText = $SqlQuery
$SqlCmd.Connection = $SqlConnection

# Setup .NET SQLAdapter to execute and fill .NET Dataset
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $SqlCmd
$DataSet = New-Object System.Data.DataSet

#Execute and Get Row Count
$nRecs = $SqlAdapter.Fill($DataSet)

if ($nRecs -gt 0)
{
# Do Stuff
$dataSet.Tables | Select-Object -Expand Rows
}
}

$SqlQuery = "SELECT CREATE_TIME, USERNAME, VM_NAME, HOST_NAME, EVENT_TYPE FROM VMWareDS.VPX_EVENT WHERE (VM_NAME = N'$MYVM') AND (EVENT_TYPE = '$TypeofEvent')"
$MyResults = Read-VIDB $SqlQuery
$MyResults

Extracting Cisco Discovery Protocol Info

Inspired by Gabe over at GabesVirtual World (Check out his site he has some great posts) I have added the Cisco Discovery Protocol info, its not the nicest looking output but the info is there !

Extracts all info for each ESX host into a text file, don’t forget to change the VI server you are connecting to and the path on lines 1 and 2 of the following script:

Connect-VIServer MYVISERVER
$filename = "c:\VirtualNetworkConfig.txt"
$vmhosts = Get-VMHost | Where-Object {$_.State -eq "Connected"} | Get-View
Write-output (Get-Date) | out-file -encoding ASCII -filepath $Filename
foreach ($vmhost in $vmhosts)
{
$networkSystem = Get-view $vmhost.ConfigManager.NetworkSystem
Write-output $vmhost.Name | out-file -Append -encoding ASCII -filepath $Filename
foreach($netSys in $networkSystem)
{
foreach($pnic in $netSys.NetworkConfig.Pnic)
{
$subnets = $netSys.QueryNetworkHint($pnic.Device)
foreach($pnicHint in $subnets)
{
Write-Output $pnicHint.Device | out-file -Append -encoding ASCII -filepath $Filename
foreach($pnicIpHint in $pnicHint.Subnet)
{
$info = $hint.connectedSwitchPort | Select *
$moreinfo = $hint.connectedSwitchPort.deviceCapability
Write-Output $info | Select * | out-file -Append -encoding ASCII -filepath $Filename
Write-Output $moreinfo | Select * | out-file -Append -encoding ASCII -filepath $Filename
Write-Output "Observed IPRanges: " $pnicIpHint.IpSubnet | out-file -Append -encoding ASCII -filepath $Filename
}
}
}
}
}

Managing VMWare Infrastructure with PowerShell

So you have used VMWare Virtual Infrastructure 3, you know how powerful it is and how great the features are within the VI client but what happens when you want to get more details or you want to automate some features… You wait for the next version and hope they have included them in there – WRONG !

You wait till early next year and you buy “Managing VMWare Infrastructure with PowerShell“ written by Hal Rottenberg and published by SAPIEN Press.

I have just finished technically reviewing this great book, if you haven’t bothered learning PowerShell yet (why not) and you haven’t looked at the VI Toolkit then please do yourself a favour and buy this book when it is released.

Not only does it give you an amazing insight into the way things work below the GUI it also teaches you the basics of PowerShell and how to use this technology to manage the VMware feature set we have come to know and love.

Hal’s unique writing style will take you through learning how to perform many extensive tasks which would normally take hours, sometimes days through the client and show you how to perform these actions within minutes or sometimes seconds. He even adds a little of his own sense of humour to lighten some of the more mundane tasks.

Learn how to deploy multiple virtual machines in minutes, report on the extensive areas of virtual infrastructure and much much more.

Believe me, this is one book I would recommend for any VI Admin to read and store net to your management workstation.

If you are a VI admin reading this then stop doing things the long way and get a bit of Posh in your life.

Table of Contents
Introduction – Why You Need This Book
  1. Windows PowerShell Crash Course
  2. Getting Started
  3. Scripting with Virtual Infrastructure
  4. Understanding Your Verbs and Nouns
  5. Basic Cmdlets
  6. Deploying Your Virtual Infrastructure
  7. Configuring Your Environment
  8. Reporting and Discovery
  9. Keeping Things Running (Maintenance)
  10. When Things Break (Troubleshooting)
  11. Management and Automation Case Studies
Appendices
Other Resources

Observed IP Ranges

Recently I needed to check a whole bunch of servers to see if we had a particular VLAN tagged on any of the ESX Hosts, after a quick post on the VI Toolkit forum and a little messing, the following script tells me quickly which observed IP ranges are on each of the nics in each of the hosts…

Connect-VIServer MYVISERVER
$vmhosts = Get-VMHost | Get-View
foreach ($vmhost in $vmhosts)
{
$networkSystem = Get-view $vmhost.ConfigManager.NetworkSystem
Write-Host $vmhost.Name -foregroundcolor yellow
foreach($netSys in $networkSystem)
{
foreach($pnic in $netSys.NetworkConfig.Pnic)
{
$subnets = $netSys.QueryNetworkHint($pnic.Device)
foreach($pnicHint in $subnets)
{
Write-Host $pnicHint.Device
foreach($pnicIpHint in $pnicHint.Subnet)
{
Write-Host "`t" $pnicIpHint.IpSubnet
}
}
}
}
}

Citrix on VMware, 1vCPU or 2vCPU ?

Following my previous Citrix testing on VMware I thought it best to try a dual vCPU VM to see if I can get double the users of a single vCPU, sounds logically sensible to me !

So just a reminder that a single vCPU with 25 users ran at around 73% average, a nice figure to run at giving plenty of CPU for peaks and troughs.

This test of a dual vCPU was undertaken following the sucesfull test of 25 users on one VM with 1 x vCPU this was run as a baseline to see how it relates to the single vCPU VM.

Virtual Machine Specification

2 x vCPU (Using VMware process affinity to tie the VM to a specific CPU core)
3072 GB RAM (reduced from 4GB as memory was not being used)

1 x 20GB Hard Disk Stored on SAN

1 x vNic (1GB)

Users Logged on: 50

  • Logged 10 users on and working over 5 minutes
  • Logged another 10 users on and working over 5 minutes
  • Logged another 5 users on and working over 3 minutes
  • Kept 25 users logged on and working for 15 minutes
  • Logged all 25 users off over a 5 minute period

clip_image002

As you can see from the above, a 3000Mhz server runs at around 60% which is not a good start bearing in mind the single vCPU VM.

Next Test 50 Users and Dual vCPU

This test was undertaken following the sucesfull test of 25 users on one server with 1xVCPU allowing for a reasonoble amount of capacity for application usage.

Virtual Machine Specification

2 x vCPU (Using VMware process affinity to tie the VM to a specific CPU core)
3072 GB RAM

1 x 20GB Hard Disk Stored on SAN

1 x vNic (1GB)

Users Logged on: 50

  • Logged 20 users on and working over 5 minutes
  • Logged another 20 users on and working over 5 minutes
  • Logged another 10 users on and working over 3 minutes
  • Kept 50 users logged on and working for 15 minutes
  • Logged all 50 users off over a 5 minute period

clip_image002[7]

The peak CPU usage of this graph was reached before 50 users were connected and therefore proves that the best method to get more users per physical server is to scale out rather than increase CPU resources.

This will obviously make sense in some cases where the cost of Operating System, Application software and licenses and management costs may make sense to build a 2 x vCPU and get slightly more users.

 

Getting VMware Host Physical Nic Speeds

I had an issue with one of my hosts this week who’s nic had auto-negotiated to the wrong speed, following this I wrote a quick script to check the rest of my hosts.

Good job as there were a fair few nics running at 100MB rather than 1GB causing a few issues.

The following script will list each host and there nics and speeds….

Connect-VIServer MYVISERVER

$VMhosts = Get-VMHost | Get-View
Foreach ($vmhost in $vmhosts){
	Write-Output $vmhost.Name
	$pnic = 0
	Do {    
		$Speed = $VMhost.Config.Network.Pnic[$pnic].LinkSpeed.SpeedMb    
		Write "Pnic$pnic $Speed"    
		$pnic ++
	} Until ($pnic -eq ($VMhost.Config.Network.Pnic.Length))}

Unable to format a LUN

The SAN team issued a new LUN to my hosts and I was unable to format it as I recieved the following error: “Unable to read partition information” on LUN 180 on vmhba1.

Running fdisk from the service console cleared this up, heres my output :

[root@myhost]# fdisk /vmfs/devices/disks/vmhba1\:0\:180\:0

Command (m for help): p

Disk /vmfs/devices/disks/vmhba1:0:180:0 (Sun disk label): 15 heads, 64 sectors, 24686 cylinders
Units = cylinders of 960 * 512 bytes

Device Flag Start End Blocks Id System
/vmfs/devices/disks/vmhba1:0:180:0p3 u 0 24686 11849280 5 Whole disk
/vmfs/devices/disks/vmhba1:0:180:0p4 u 0 2 960 f Unknown
/vmfs/devices/disks/vmhba1:0:180:0p5 u 2 24686 11848320 e Unknown

Command (m for help): o
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won’t be recoverable.

The number of cylinders for this disk is set to 24686.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

After this I was able to use the LUN as normal, really put here for my reference in the future 🙂

Checking CPU Affinity

I needed to double check the CPU Affinity settings I had set on some VM’s which I wanted tied to the cores in my host, just to double check I had these correct the following one liner works a treat….

Get-Cluster “My Cluster” | Get-VM | Get-View | foreach {Write-output $_.Name $_.Config.CpuAffinity.AffinitySet}

Is MSCS Supported on a HA/DRS Cluster ?

(Straight answer in red at the bottom if you want to miss out the novel)

I needed to install Microsoft Cluster Service (MSCS) on a couple of virtual machines acting as an Active-Passive cluster, with a rule to separate these VM’s onto different hosts within VMware 3.5 U1.

Some may ask why (as I did) when you have the wonders that are HA and Vmotion why would you do this ? the short answer was that it took less time (slightly) for the MSCS cluster to fail over to another virtual machine on a different host in the event of hardware failure than it did for HA to realise there was a failure and move the machine and power it back on and the machine to start up.

And as this was a urgent application the MSCS cluster on top of VMware seemed to be the way to go.

So after reading the fantastic white paper by VMware:

Setup for Microsoft Cluster Service
Update 2 Release for ESX Server 3.5, ESX Server 3i version 3.5, VirtualCenter 2.5

Which can be downloaded here

Unfortunately I hit upon a problem in my planned design, on Page 16 of the above document which summarises the caveats for setting up a MSCS cluster on VMware, one of the bullet points was as follows:

  • Clustered virtual machines cannot be part of VMware clusters (DRS or HA)

I will not use the exact language here but lets just say I was not best pleased !

On thinking about this and the way that it worked and after adding a post on the VMware communities to confirm what I was thinking (great bunch of guys on there) I came up with the following…

As the MSCS Cluster has a private network which it uses for a heartbeat LAN there may be issues VMotioning the server over to another host, this as we know can cause a slight blip in the network communication and if this was to happen it may cause a problem with the MSCS cluster causing it to issue a failover and suddenly the service would be unavailable.

OK it makes sense, but that is quite a broad statement to make, “Clustered virtual machines cannot be part of VMware clusters (DRS or HA)”. Why couldn’t I just edit the DRS Cluster and set the Virtual Machines Automation Level to Disabled ? Surely this would stop the machines from being VMotioned and everyone would be happy ?

Whilst in theory this would work forever and a day and I would not get any issues I decided to confirm why the document did not reflect this setup as even if I did get it working it was point blankly “Not Supported” by VMware, and as we know working in a corporate environment this is a big NO NO.

Next Step: I raised a call with VMware to ask why this was the case, after a few games of Ping Pong I received my answer…..

Hi Alan,

You are correct in your assumption. It is not recommended to have MSCS VM’s within a Vmware Cluster. This is for the reasons you mentioned above regarding interrupted heartbeats. You could set individual HA/DRS settings on these individual MSCS VM’s in order to keep them running on the specific hosts. However, there would be issues in the future if these values were ever changed unknowingly by another administrator with access to the system.

Yes, I can confirm that this would be supported by VMware. However, if DRS ever tried to move these VM’s even with the DRS setting disabled then this would be a bug and would need to be investigated.

I would recommend that you test this setup using a network monitoring tool such as IO Meter to ensure that no heartbeats are dropped from the VM.