- This topic has 47 replies, 33 voices, and was last updated May 10, 2022 by Carlos C.
Driver I/O cache ran out of memory on host
Just upgraded from 8.5.3 to 9.0 Update one and now one of our host is getting a alert message – : Driver I/O cache ran out of memory on host
. This may impair replication performance. Current I/O cache size: 96 MB. Last full I/O cache memory occurrence: 9 seconds ago. Full I/O cache memory has occurred 1 times in the last 9 seconds.
Not sure what to doMatthew HSeptember 17, 2021 03:12:10 PM
Having the same issue. Updated from 9.0 –> 9.0 U1 and now getting this message on multiple VRAs in vSphere.
You can add one more here. It’s so far occurred on multiple hosts across different vcenters.Jeroen SSeptember 20, 2021 07:32:30 AM
Same here after updating to 9.0U1
Same issue for us as well after upgrading to 9.0U1. Happening on multiple hosts, but not all yetPetar SSeptember 20, 2021 09:22:18 PM
Another.Matthew KSeptember 21, 2021 06:00:44 PM
Another. Is Zerto paying attention? I don’t see actual responses to the issue.Matthew HSeptember 21, 2021 06:11:09 PM
I created a support request and have been told to do one of three things:
1. Ignore the warning since everything is working
2. I can manually disable the warning
3. Escalate to Level 2 support and they can increase the I/O cache (but cannot guarantee it will fix the issue…)Matthew KSeptember 21, 2021 06:12:36 PM
3 relatively bad options. Nice!
This is what I received when I engaged support, and this is not palatable especially since all but one of our hosts now display the warning:
1. vMotion all protected VMs from one of the hosts that have problem with IO cache;
2. Uninstall the VRA on the host where VMs were vMotioned from;
3. On the ZVM VM, Edit zloadmod.sh file:
– Open zloadmod.sh on ZVM VM (it should be at C:\Program Files\Zerto\Zerto Virtual Replication).
– Edit zloadmod.sh as follows:
add iocache=XXX At the end of the following line:
vmkload_mod -m $DRIVER_RANDOM_NAME /tmp/$DRIVER_MODULE_NAME.o $MODULE_PARAMS
vmkload_mod -m $DRIVER_RANDOM_NAME /tmp/$DRIVER_MODULE_NAME.o $MODULE_PARAMS iocache=192
1. If iocache parameter is present increase value as stated above
2. Once the file is edited and saved, Re-install VRA
3. vMotion VM “usmlrs1949.arrow.com_ShopFloor_PROD” back to Host and ideally set an affinity rule so it stays there and is not migrated to another Host that has the default iocacheMatthew KSeptember 21, 2021 07:53:06 PM
This doesn’t sound good at all. Are you to set affinity rules for all of the guests on the host having an issue?Brian MSeptember 22, 2021 01:46:03 PM
This is not really a solution in my opinion. I’m not setting affinity rules on all my VM’s/host in that cluster.Kevin SSeptember 23, 2021 05:23:59 PM
The 3 Hyper-V hosts at our primary site send me those alerts. Hopefully its not causing any issues and fixed in U2 because those steps seem like to much work.
Same here, so far on a handful of hosts.Patrick RSeptember 28, 2021 11:18:43 PM
Having the same issue here as well. Has anyone received any helpful information from Zerto?