Article number
000004568
Affected Versions
7.5 Update 3
7.5 Update 2
7.5 Update 1
7.5 Patch 1
7.5
7.0 Update 3
7.0 Update 2 Patch 2
7.0 Update 2 Patch 1
7.0 Update 2
7.0 Update 1 Patch 1
7.0 Update 1
7.0 Patch 2
7.0 Patch 1
7
8.0
8.0 Update 1
8.0 Update 1 Patch 1
8.0 Update 2
Source Hypervisor
All
Target Hypervisor
AWS

Failover To AWS Fails With Error “Instance i-xxxxx did not appear in AWS yet.: did not happen after 00:07:00”

Viewed 63 times

Summary

This article discusses a known issue of zimporters failing to start properly due to missing permissions in specific setups in AWS, leading to Failover failures.

Root Cause

Starting in 7.0, EBS encryption is enabled in Zerto by default. Due to this, all EBS volumes created, including the ones created for zimporters' OS disk and data disk that is used for the import, are encrypted. Zerto utilizes AWS KMS encryption and expects to see the default AWS-provided KMS keys.

However, if an administrator replaces these AWS-provided KMS keys with their own generated KMS keys, then Zerto will fail to decrypt the EBS volumes properly. Since the OS volume is an EBS volume and is not accessible, per AWS design, the instance is terminated automatically.

Source: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/troubleshooting-launch.html#troubleshooting-launch-internal

Symptoms

Failover to AWS fails with the below error:
 
Instance i-xxxxx did not appear in AWS yet.: did not happen after 00:07:00


Additionally, while viewing the EC2 Dashboard in AWS during a Failover, it will be noticed that a zimporter instance will spin up but almost immediately terminate.

Solution

Workaround:

1) The following KMS permissions must be added to the IAM Role attached to the ZCA that is facilitating the Failover to AWS. Once they are added in, retry the Failover.
"kms:Encrypt",
"kms:Decrypt",
"kms:ReEncrypt*",
"kms:GenerateDataKey*",
"kms:DescribeKey",
"kms:CreateGrant",
"kms:ListAliases"

2) If the administrator would rather not provide this permission to the ZCA's IAM Role, the customer KMS keys will need to be replaced with the original default AWS-provided KMS keys. Once replaced, retry the Failover.

3) The final option if the above are not acceptable would be to completely disable the EBS encryption feature for the ZCA by following this Zerto KB Article. Once the necessary steps have been taken per the referenced article, retry the Failover.