How to Recover from a Ghost VRA (ZVR 4.0 and greater)

KB Number:
000001032

Cause:

Zerto Virtual Replication keeps track of relevant vSphere components. In the event that an object, listed below in the solution, is removed from vCenter, Zerto is unable to continue to track the host, even if the object is later re-added.  This means that replication will stop for any VPGs that reside on that host, and the VRA on that host will become a "Ghost VRA".  Once a Ghost VRA exists, the situation that caused it to become a “Ghost VRA” either needs to be resolved, or the VRA needs to be replaced in order to restart replication.

Ghost VRAs can arise from the following circumstances:

  • A VRA VM, or the VM of a Shadow VRA (AKA VRA-H or diskbox) associated with the VRA, has been removed from the VC (either deleted or just removed from inventory)

  • A recovery volume that was attached to the VRA VM, or to an associated Shadow VRA, has been removed from the VC (either detached from the VM or completely deleted)

  • A journal volume that was attached to the VRA VM, or to an associated Shadow VRA, has been removed from the VC (either detached from the VM or completely deleted).

  • A VRA’s host is removed from vCenter inventory and re-added to vCenter inventory.

  • A VM undergoing a ZVR operation (Failover Test, Failover, Move) is deleted from vCenter inventory.

Solution:

To recover from a Ghost VRA, follow the relevant instructions below:

A Ghost VRA at a source site:

  1. vMotion protected VMs from the Ghost VRA’s host to another host, with a VRA, in the source cluster.

  2. Note the Ghost VRA’s network/storage configurations for use during re-install.

  3. Uninstall the Ghost VRA.

  4. Install a new VRA on the host.

  5. vMotion protected VMs back to the host as desired.

 

A Ghost VRA due to the removal of a Zerto-managed VM from vCenter inventory:

  1. Delete the VPGs in which the removed VMs are configured.

  2. The VPG deletion will cause the VRA to exit Ghost VRA state. If the deletion fails, and the VPGs enter “Pending Remove” state, choose to delete them again, and choose the “Force Delete” option.

  3. Using the Zerto Diagnostics utility on a site ZVM, choose to import a preserved VPG configuration XML to recreate the removed VPGs.  By default, the five most recent of these files (one per day), are kept at the following path: C:\Program Files (x86)\Zerto\Zerto Virtual Replication\ExportedSettings
    In version 4.5 and above, the path by default is C:\Program Files (x86)\Zerto\Zerto Virtual Replication\ExportedSettings

A Ghost VRA at a target site due to the removal of a VRA’s host from vCenter inventory:

  1. For each affected VPG on the recovery side (the action on the source site will export only partial information), via the VPGs tab, choose to “Actions” > “Export CSV” to export the VPG configurations, including recovery disk locations, to a CSV file on the local machine.
  2. Delete the affected VPGs, while choosing “Keep target disks at peer site”.  If the deletion fails, and the VPG enter “Pending Remove” state, choose to delete it again, and choose the “Force Delete” option.
  3. Note the Ghost VRA’s network/storage configurations for use during re-install.
  4. Remove the Ghost VRA VM from vCenter inventory - this will dissociate disks that are still attached to the VRA, and leave them in their datastore locations.
  5. Install a new VRA on the host.
  6. Recreate the removed VPGs, while manually preseeding the preserved target disks.  The preserved target disk locations are referenced in the CSV file exported in step 1.
  7. Once the VPGs have been re-created, the target disks will have been moved to new datastore locations.  As such, it is safe to delete the old VRA’s folder, under which they were previously located.

 

A Ghost VRA due to the removal of a recovery/journal volume from vCenter inventory:

  1. Delete the VPGs in which the removed volumes are configured.

  2. The VPG deletion will cause the VRA to exit Ghost VRA state. If the deletion fails, and the VPG enter “Pending Remove” state, choose to delete it again, and choose the “Force Delete” option.

  3. Using the Zerto Diagnostics utility on one of the site ZVMs, choose to import a preserved VPG configuration XML to recreate the removed VPGs.  By default, the five most recent of these files (one per day), are kept at the following path: C:\Program Files (x86)\Zerto\Zerto Virtual Replication\ExportedSettings
    In version 4.5 and above, the path by default is C:\Program Files (x86)\Zerto\Zerto Virtual Replication\ExportedSettings

 

A Ghost VRA with no environment changes:

  1. For each affected VPG on the recovery side (the action on the source site will export only partial information), via the VPGs tab, choose to “Actions” > “Export CSV” to export the VPG configurations, including recovery disk locations, to a CSV file on the local machine.

  2. Delete the affected VPGs, while choosing “Keep target disks at peer site”.  If the deletion fails, and the VPG enter “Pending Remove” state, choose to delete it again, and choose the “Force Delete” option.

If step 2 doesn’t provide relief, continue with the following steps.

  1. Note the Ghost VRA’s network/storage configurations for use during re-install.

  2. Remove the Ghost VRA VM from vCenter inventory - this will dissociate disks that are still attached to the VRA, and leave them in their datastore locations.

  3. Install a new VRA on the host.

  4. Using the Zerto Diagnostics utility on one of the site ZVMs, choose to import a preserved VPG configuration XML to recreate the removed VPGs.  By default, the five most recent of these files (one per day), are kept at the following path: C:\Program Files (x86)\Zerto\Zerto Virtual Replication\ExportedSettings
    In version 4.5 and above, the path by default is C:\Program Files (x86)\Zerto\Zerto Virtual Replication\ExportedSettings

  5. Once the VPGs have been re-created, the target disks will have been moved to new datastore locations.  As such, it is safe to delete the old VRA’s folder, under which they were previously located.

Affected Versions:
ZVR 4.0 and greater

Hypervisor:
VMWare, Hyper-V

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading...