If you’ve ever tried to execute a task in vCenter only to be told that another task is already in progress but you can’t see it then this post may be of use to you.
In this example I tried to initiate a removal of all snapshots against a target virtual machine – vCenter then indicated that another task was in progress but when we look in the web interface to see running tasks nothing shows. This can be a common occurrence especially with long running tasks or where somebody reboots a vCenter.
To see what is happening we will connect via SSH to the host running the virtual machine and then leverage the vim-cmd
commands to determine what is happening and how far the task has progressed.
The first command we will run is vim-cmd vmsvc/getallvms
which will return a list of the running virtual machines on this host. This will allow us to identify the virtual machine ID which is required by our subsequent commands.
[root@BSA-ESXI04:~] vim-cmd vmsvc/getallvms
Vmid Name File Guest OS Version Annotation
122 BSA-NP [BSA-NEW-3par01-Datastore04] BSA-NP/BSA-NP.vmx windows8Server64Guest vmx-15 [REMOVED]
[root@BSA-ESXI04:~]
In the above example our virtual machine ID is 122 – we can now feed this into our next command, vim-cmd vmsvc/get.tasklist
This command takes the VMID we determined previously and displays a list of all the tasks, as we see below there is already a snapshot removal task running.
[root@BSA-ESXI04:~] vim-cmd vmsvc/get.tasklist 122
(ManagedObjectReference) [
'vim.Task:haTask-122-vim.VirtualMachine.removeAllSnapshots-869472401'
]
[root@BSA-ESXI04:~]
Now we know what the running task is it would be useful to understand how far it has progressed and what our completion percentage is. We leverage vim-cmd vimsvc/task_info
command which takes the task information from our previous command. We need to take the value for vim.Task:
which in our case is haTask-122-vim.VirtualMachine.removeAllSnapshots-869472401
[root@BSA-ESXI04:~] vim-cmd vimsvc/task_info haTask-122-vim.VirtualMachine.removeAllSnapshots-869472401
(vim.TaskInfo) {
key = "haTask-122-vim.VirtualMachine.removeAllSnapshots-869472401",
task = 'vim.Task:haTask-122-vim.VirtualMachine.removeAllSnapshots-869472401',
description = (vmodl.LocalizableMessage) null,
name = "vim.VirtualMachine.removeAllSnapshots",
descriptionId = "VirtualMachine.removeAllSnapshots",
entity = 'vim.VirtualMachine:122',
entityName = "BSA-NP",
locked = <unset>,
state = "running",
cancelled = false,
cancelable = true,
error = (vmodl.MethodFault) null,
result = <unset>,
progress = 93,
reason = (vim.TaskReasonUser) {
userName = "vpxuser"
},
queueTime = "2019-07-05T08:12:03.51941Z",
startTime = "2019-07-05T08:12:03.519649Z",
completeTime = <unset>,
eventChainId = 869472401,
changeTag = <unset>,
parentTaskKey = <unset>,
rootTaskKey = <unset>,
activationId = <unset>
}
[root@BSA-ESXI04:~]
We get quite a bit of information returned but for me the most useful part is the progress value which in this case shows the task is 93% complete. We can run the command multiple times through the day to see how the task is doing and when it has completed.
Hopefully this information proves useful to you and helps identify those ‘hidden’ tasks which the web management interface may no longer display but which are still active.
Excelente aporte…
I like the info but it would be useful to go into how to abort the task if it’s a runaway task or something.
Estava com o mesmo problema executei os passos abaixo e resolveu,
Do Console Local ou SSH:
Faça login no SSH ou no console local como root.2.
Execute esses comandos:/etc/init.d/hostd restart
/etc/init.d/vpxa restart
I would like this info too.
vim-cmd vimsvc/task_cancel
example: vim-cmd vimsvc/task_cancel haTask-122-vim.VirtualMachine.removeAllSnapshots-869472401
Thank you for this quick detail and thank you to Tracy for the cancel command. I had a stalled vStorage vMotion that was stopping a VM from being powered on after a migration. This allowed me to find and cancel the offending registration command that was hung up.
Glad you found the post and comments useful, really cool knowing it’s benefiting people