AWS VolumeModificationSizeLimitExceeded

If you are dealing with big amounts of EBS volume and need to extend it you might face one day an AWS VolumeModificationSizeLimitExceeded error.

We are occasionally extending our EBS volumes, It is done by updating the CloudFormation for these instances and then there is a script (that was written by people much more AWS knowledgable than me) that checks CloudFormation parameters and if it sees that volume size has increased it will do all the AWS and Linux “black magic” to actually extend the volumes and make the OS aware of it.

One happy day we were extending volumes for our 60 members strong fleet of EC2 instances from 2000GB to 3000GB.  So I’ve updated the CloudFormation and the extension kicked off.. On 51 out of 60 instances it has completed successfully, but on the rest, I could see that cfn-script was complaining “An error occurred (VolumeModificationSizeLimitExceeded) when calling the ModifyVolume operation: You have exceeded your maximum gp2 volume modification storage limit of 100 TiB in this region. Please contact AWS Support to request an Elastic Block Store volume modification storage limit increase or retry once existing volume modifications have completed”

Apparently, there is a limit on the “original” total volumes size that can be extended in one go and the default limit is 100TB and which we have hit:  52 x 2000GB = 101.56TB.

It is mentioned in https://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html as “Maximum modifying storage”

VolumeModificationSizeLimitExceeded
Maximum Modifying Storage

(As of December 2019) I couldn’t find where one can see the current limit and the only way to increase it is by logging a support case with AWS. By the way, you will need to log a “generic” support case and not a “limit increase” case as this limit is not available for selection.

Also if you plan to extend from more than 300TB in one go you better contact AWS as soon as possible as here is what I’ve got from AWS  Support when trying to increase the value to 400TB

“With regards to your question, the 300TB is the value we can provide at this point because request with higher value requires further review/approval from our EBS team. The outcome depends on their review and we don’t guarantee whether the request will get approved or not.”

P.S. you’ve reached the maximum modification rate per volume limit. wait at least 6 hours between modifications per ebs volume.

The “you’ve reached the maximum modification rate per volume limit. wait at least 6 hours between modifications per ebs volume.” error is different and happens when you are trying to increase an EBS volume again within less than 6 hours from the previous time you’ve modified the same volume.

Update October 2022

These limits are now better documented at: https://docs.aws.amazon.com/general/latest/gr/ebs-service.html and it seems that one can request the limit increase using a dedicated “limit increase” case.

VOLUMEMODIFICATIONSIZELIMITEXCEEDED

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.