Article number
000003973
Affected Versions
All
Source Hypervisor
Azure
Target Hypervisor
Azure

Error: “Scaleout orchestrator failed” Immediately Following Restart of ZCA

Viewed 195 times

Summary

An administrator may notice an issue after restarting the ZVM or VRA service in an Azure ZCA or rebooting the Azure ZCA itself.

Root Cause

Following a restart of the ZCA, or a restart of the ZVM service or VRA service on the ZCA, the Scaleout Orchestrator will be in a reset state for up to 15 minutes. During this time, the ZCA cannot process tasks that require a scale set in Azure such as a test failover, live failover, or move.

Symptoms

A Failover or Move for a VPG replicating to Azure fails, and the following event error is generated under the Monitoring section of the ZVM GUI:

Protection group failover test. Failed: Scaleout orchestrator failed

The ZVM log file from the ZCA covering the time of the failed task contains the following error:

GlobalExceptionFilter,OnException,"[REQUEST #2] FAIL. Request=https://localhost:7072/ManifestJobs/v1/start?RequestId=2. StatusCode=OK, Response={
""ResponseId"": 2,
""ResponseTimestamp"": ""2020-04-02T15:07:13.0248918Z"",
""ReturnCode"": 205,
""ReturnDescription"": ""[JobId=0]: Error: ScaleOut Orchestrator in invalid state: Orchestrator in invalid state 'Reset' (ORCHESTRATOR_INVALID_STATE)"",
""RequestId"": 2
}","Zerto.Platform.PublicCloud.ScaleoutJobsOrchestrator.Application.Exceptions.OrchestratorInvalidStateException T Invoke[T](Zerto.Platform.PublicCloud.ScaleoutJobsOrchestrator.Common.Models.Enums.ComponentState, System.Func`1[T]) Zerto.Platform.PublicCloud.ScaleoutJobsOrchestrator.Application.Exceptions.OrchestratorInvalidStateException: [JobId=0]: Error: ScaleOut Orchestrator in invalid state: Orchestrator in invalid state 'Reset'

Solution

This is a current product limitation/expected behavior.

Kindly wait 15 minutes before attempting a test failover, live failover, or move operation after the ZCA is restarted, or after the ZVM or VRA service on the ZCA is restarted.