vscsistats is a great tool used for troubleshooting virtual machines storage performance. The tool works at the virtual SCSI device level in the kernel. This means that you can use it for virtual machines on local disk, fiber channel, iSCSI or NFS. It reports performance pr. VMDK.
Vscsistats will report the following about a virtual machine:
- ioLength
- seekDistance
- outstandingIOs
- latency
- interarrival
To use the tool you need to login to your ESX service console or ESXi unsupported tech-mode console. The tool is available from right there. The tool is not bundled with ESXi 4.0 or prior. Update
to ESXi 4.1 and the tool will be there.
To use the tool you need to list your virtual machines on your host by typing
vscsistats -l
This will output your virtual machines with their worldGroupID and their individual vmdk’s handleID
If you want to start collecting info for all disks on a virtual machine use the following command
vscsistats -s -w worldGroupID
If you want to collect just for one specific disk on a virtual machine do the following
vscsistats -s -w worldGroupID -i handleID
The collection will run for 30 minutes. If you want to stop it type vscsistats -x
During or after collection you can use the -p switch to show the collected information. If you want to see the latency
vscsistats -p latency
The output will be something similar to this (It actually also show the READ IO latency and WRITE IO latency but I left those out for the example)
Histogram: latency of IOs in Microseconds (us) for virtual machine worldGroupID : 785088, virtual disk handleID : 8340 {min : 415max : 6156mean : 801count : 582{0 (<= 1)0 (<= 10)0 (<= 100)22 (<= 500)469 (<= 1000)88 (<= 5000)3 (<= 15000)0 (<= 30000)0 (<= 50000)0 (<= 100000)0 (> 100000)}}
This tool is simply brilliant. You will not find this information anywhere in either ESXTOP or the vCenter performance graphs.
Thanks for blogging a about vscsiStats. I originally wrote this tool in 2006. Take a look at my IISWC 2007 paper on this topic as well.
On my esx 4.1 I get this response when I pullout -x latency
VscsiStats: No valid stats for worldgroup xxxz, handle 10201. Please enable vscsiStats collection
It works on my 4.0 hosts
Not sure what build you were using, but this isn’t there by default in any of my versions of ESX/ESXi 4.1 or higher.
I have to cd /usr/lib/vmware/bin
./vscsiStats –
Weird