Virtual machine does not power on because of missing or locked files

Virtual machine does not power on because of missing or locked files KB Article 10051 Updated Mar. 02, 2009

Products

VMware ESX VMware VirtualCenter

Details

A virtual machine does not power on because of missing or locked files. File problems include: Missing swap file Unable to open swap or file lock on virtual machine files vmdk file locked by another VM A virtual machine that has not successfully removed the .vswp file or locks on files in the virtual machine directory when it was shut down cannot be powered on.   Error messages during power up include:   Unable to open Swap File   You might find entries in the /var/log/vmkernel file that look similar to:   WARNING: World: VM xxxx:  xxx: Failed to open swap file <path>: Lock was not free WARNING: World: VM xxxx:  xxx: Failed to initialize swap file <path>

Solution

To resolve this issue through the VirtualCenter Client: Locate the virtual machine and check its state. Attempt to power on from VI Client. Open a remote console window for the virtual machine. Make sure the VMware splash screen appears. If there is no response in the virtual machine's remote console and the state continues to be powered off: As root, log onto the ESX Server system where the virtual machine resides. Confirm that the virtual machine is registered on the server using the command: [root@esxhostname root]# vmware-cmd -l Ensure the virtual machine is powered off using the command: [root@esxhostname root]# vmware-cmd <path.vmx> getstate Where <path.vmx> is the full path to the virtual machine configuration file. If the output from this command is getstate() = on, return to the VIC to see if the state has changed. To correct the situation when the getstate () = on  but the virtual machine is not responding, issue the commands: [root@esxhostname root]# cd /vmfs/volumes/<storage>/VMname [root@esxhostname root]# ls Look for the virtual machine's VMname.vswp file. To determine whether the VMDK file of the virtual machine is locked by another running virtual machine: [root@esxhostname root]# vmkfstools -D <entire path/vmname.vmdk> This command returns the MAC address of any host that is locking the vmdk file. If the state is still on try to stop the virtual machine from the command line. [root@esxhostname root]# vmware-cmd <entire path/VMname.vmx> stop hard [root@esxhostname root]# vmware-cmd <entire path/VMname.vmx> getstate If the state persists to report it is on (see step 3), determine the virtual machines master world ID and try to kill the process from the command line. [root@esxhostname root]# cat /proc/vmware/vm/*/names Look for the virtual machine in the output. Record the the vmid. [root@esxhostname root]# less -S /proc/vmware/vm/<vmid from above>/cpu/status Locate the Group field for the virtual machine. This represents the master world number for each virtual machine. [root@esxhostname root]# vmkload_app -kill 9 <master world number from above> Use the ls command again to check for the swap file named VMname.vswp. If this file exists, the virtual machine does not power up. The swap file may be locked by another server with access to the LUN. Check for this using the command:  [root@esxhostname root]# vmkfstools –D <entire path/VMname.vswp> To locate the other server look at the last few lines of the vmkernel log file. [root@esxhostname root]# tail /var/log/vmkernel Look for lines similar to the following: Apr  5 09:45:26 Hostname vmkernel: 17:00:38:46.977 cpu1:1033)Lock [type 10c00001 offset 13058048 v 20, hb offset 3499520 Apr  5 09:45:26 Hostname vmkernel: gen 532, mode 1, owner 45feb537-9c52009b-e812-00137266e200 mtime 1174669462] Apr  5 09:45:26 Hostname vmkernel: 17:00:38:46.977 cpu1:1033)Addr <4, 136, 2>, gen 19, links 1, type reg, flags 0x0, uid 0, gid 0, mode 600 Apr  5 09:45:26 Hostname vmkernel: 17:00:38:46.977 cpu1:1033)len 297795584, nb 142 tbz 0, zla 1, bs 2097152 Apr  5 09:45:26 Hostname vmkernel: 17:00:38:46.977 cpu1:1033)FS3: 132: <END supp167-w2k3-VC-a3112729.vswp> (END) Note the string on the second line in this sample output.  The text after the word "owner" corresponds to the MAC address of the system with the lock on the file. In this example: "owner 45feb537-9c52009b-e812-00137266e200" The other system in this example is identified as  00137266e200. Check the current host's UUID. [root@esxhostname root]# esxcfg-info |grep -i 'system uuid' This command returns a sting like the following: |----System UUID....................450e849d-cbfe-b70e-8dac-00137266e200 Compare the last field to determine if you are on the ESX Server machine that holds the lock. If you are not on the correct ESX host machine, you must find the correct server by running this command on other servers that share access to the LUN with the virtual machine directory For more information related troubleshooting information, see: Troubleshooting import or export failures in Lab Manager (1006333) Troubleshooting a failure to deploy or undeploy a virtual machine (1006336)