What is Ghost VRA, and How to Recover from Ghost VRA

KB Number:
000001006

Symptoms:
Zerto Virtual Replication keeps track of relevant vSphere components. In the event that an object (see list below in the "Solution"), is removed from vCenter, Zerto is unable to continue to track the host, even if the object is later re-added. This means that replication will stop for any VPGs that reside on that host, and the VRA on that host will become a "Ghost VRA." Once a Ghost VRA exists, the situation that caused it to become a Ghost VRA either needs to be resolved, or the VRA needs to be replaced in order to restart replication.

Cause:
There are several ways for Ghost VRA status to appear:

1. A VRA VM, or the VM of a Shadow VRA (AKA VRA-H or diskbox) associated with the VRA, has been removed from the VC (either deleted or just removed from inventory).

2. A recovery volume that was attached to the VRA VM, or to an associated Shadow VRA, has been removed from the VC (either detached from the VM or completely deleted).

3. A journal volume that was attached to the VRA VM, or to an associated Shadow VRA, has been removed from the VC (either detached from the VM or completely deleted).

4. The host on which the VRA lays was disconnected and re-added to the vCenter inventory (it will get a new internal ID from the VC MOB, thus Zerto will not be able to track the VRA installed on it).

5. Deleted Failover Test VM (testing recovery VM).

Solution:

There are several solutions, depending on where the Ghost VRA is reported, or how it was generated.

A Ghost VRA at a source site:

1) vMotion protected VMs from the Ghost VRA’s host to another host, with a VRA, in the source cluster.

2) Note the Ghost VRA’s network/storage configurations for use during re-install.

3) Uninstall the Ghost VRA.

4) Install a new VRA on the host.

5) vMotion protected VMs back to the host as desired.

A Ghost VRA due to the removal of a Zerto-managed VM from vCenter inventory:

1) Delete the VPGs in which the removed VMs are configured.

2) The VPG deletion will cause the VRA to exit Ghost VRA state. If the deletion fails, and the VPGs enter “Pending Remove” state, choose to delete them again, and choose the “Force Delete” option.

3) Using the Zerto Diagnostics utility on a site ZVM, choose to import a preserved VPG configuration XML to recreate the removed VPGs.  By default, the five most recent of these files (one per day), are kept at the following path: C:\Program Files (x86)\Zerto\Zerto Virtual Replication\ExportedSettings

A Ghost VRA at a target site due to the removal of a VRA’s host from vCenter inventory:

1) Delete the affected VPGs, while choosing “Keep target disks at peer site.”  If the deletion fails, and the VPG enter “Pending Remove” state, choose to delete it again, and choose the “Force Delete” option.

2) Note the Ghost VRA’s network/storage configurations for use during re-install.

3) Remove the Ghost VRA VM from vCenter inventory. This will dissociate disks that are still attached to the VRA, and leave them in their datastore locations.

4) Install a new VRA on the host.

5) Recreate the removed VPGs, while manually pre-seeding the preserved target disks.  The preserved target disk locations are referenced in the CSV file exported in Step 1.

6) Once the VPGs have been re-created, the target disks will have been moved to new datastore locations.  As such, it is safe to delete the old VRA’s folder, under which they were previously located.

A Ghost VRA due to the removal of a recovery/journal volume from vCenter inventory:

1) Delete the VPGs in which the removed volumes are configured.

2) The VPG deletion will cause the VRA to exit Ghost VRA state. If the deletion fails, and the VPG enters “Pending Remove” state, choose to delete it again, and choose the “Force Delete” option.

3) Using the Zerto Diagnostics utility on one of the site ZVMs, choose to import a preserved VPG configuration XML to recreate the removed VPGs.  By default, the five most recent of these files (one per day), are kept at the following path: C:\Program Files (x86)\Zerto\Zerto Virtual Replication\ExportedSettings

A Ghost VRA with no environment changes:

1) Delete the affected VPGs, while choosing “Keep target disks at peer site.”  If the deletion fails, and the VPG enters “Pending Remove” state, choose to delete it again, and choose the “Force Delete” option.

If Step 2 doesn’t provide relief, continue with the following steps.

1) Note the Ghost VRA’s network/storage configurations for use during re-install.

2) Remove the Ghost VRA VM from vCenter inventory. This will dissociate disks that are still attached to the VRA, and leave them in their datastore locations.

3) Install a new VRA on the host.

4) Using the Zerto Diagnostics utility on one of the site ZVMs, choose to import a preserved VPG configuration XML to recreate the removed VPGs.  By default, the five most recent of these files (one per day), are kept at the following path: C:\Program Files (x86)\Zerto\Zerto Virtual Replication\ExportedSettings

5) Once the VPGs have been re-created, the target disks will have been moved to new datastore locations.  As such, it is safe to delete the old VRA’s folder, under which they were previously located.

Since there might be a large number of VPGs in an error state, you may contact Zerto Support in order to locate the problematic VPG -- assuming the issue is only related to a single VPG.

Affected Versions:
3.0x-4.0x

Hypervisor:
VMWare, Hyper-V

1 Star2 Stars3 Stars4 Stars5 Stars (1 votes, average: 3.00 out of 5)
Loading...