EFS - Intelligent Tiering

Opportunity Name:

EFS - Intelligent Tiering

AWS Resource Type:

Amazon Elastic File Service (EFS)

Opportunity Description:

Amazon EFS offers multiple storage classes that differ based on the cost of storage and cost of accessing the data:

• Standard - stores data across multiple AZ
• One Zone - stores data in a single AZ
• Standard Infrequent Access - has lesser price than Standard, but has access cost
• One Zone Infrequent Access - has lesser price than One Zone, but has access cost

A research done by AWS shows that only 20% of files are actively used, and the rest 80% are rarely accessed. Thus the cost of EFS can be reduced by leveraging Infrequent Access storage classes.

EFS also comes with an Intelligent Tiering option. This creates 2 lifecycle policies:

• Transition into IA - defaults to None, and can be changed to 7, 14, 30, 60 or 90 days since last access
• Transition out of IA - defaults to None, and can be changed to “On first access”

There are 2 throughput modes supported by EFS:

• Bursting: provides 50 MBps baseline throughput per terabyte of storage and a minimum of 1 MBps. Unused throughput is accumulated in burst credits, providing an increased throughput up to twice that of the baseline throughput for short duration.
• Provisioned: provides a dedicated throughput defined by you. Only the extra throughput than what would have been provided by Bursting mode is charged.

While Infrequent Access is a great option to reduce the storage cost, one important thing to consider is that the Bursting throughput is calculated based on the amount of data in Standard/One zone tier. The data that is moved to Infrequent Access does not contribute to the throughput calculation.

So, for file systems with Bursting throughput mode, you should also consider the IO throughput requirements of the file system. To continue to receive high throughput even in the Infrequent Access tier, Provisioned throughput mode can be used. Based on the workload pattern and amount of storage, the increased cost of Provisioned throughput, may still be less than the cost saving provided by the Infrequent Access storage class.

Criteria for identifying the opportunity:

CloudFix will create recommendations to enable Intelligent Tiering for all EFS file systems matching the following criteria:

• The file system does not have lifecycle policy to transition to IA
• For file system with Bursting throughput, the average IO for past 30 days should be less than
• 90% of baseline throughput for Standard storage class
• 40% of baseline throughput for One Zone storage class

Potential savings (range in % on annual basis):

Enabling Intelligent Tiering results in EFS costs reduction by up to 92%.

What happens when the Fixer is executed?

The files accessed before 30 days are moved to Infrequent Access storage class. The duration for this operation depends on the amount of storage and workload on the EFS file system. The IO throughput is first allocated to active workloads, and then to lifecycle policy operations. 

In Cost Explorer, the following usage type will show the effect of enabling Intelligent Tiering:

• <region>-TimedStorage-ByteHrs
• <region>-IATimedStorage-ByteHrs
• <region>-IADataAccess-Bytes
• <region>-ProvisionedTP-MiBpsHrs
• <region>-IncludedTP-MiBpsHrs

Is it possible to rollback once CloudFix implements the fixer?

Manual rollback on your AWS account is possible. One-click rollback via CloudFix for this opportunity is currently not available.

Few things to consider for manual rollback:

• Disabling Intelligent Tier or removing lifecycle policies will leave the files in their current storage class. To move the files from Infrequent Access to the original tier, you can connect the EFS file system to an EC2 instance, and run a rsync command. As all files will be accessed by rsync, they will be moved back to the original tier. After this, the throughput mode can also be changed back to the original value.
• Another simpler option would be to modify the provisioned throughput value to accommodate the increased IO due to change in workload pattern.

Can CloudFix implement the fix automatically once I accept the recommendation?

Yes. Once you accept the recommendation and schedule the fix, execution is automatic.

Does this fix require downtime?

No. Modifying EFS Intelligent tiering lifecycle rules or provisioned mode does not require any downtime. It is executed as a zero downtime fix.

Do it yourself (DIY) Instructions:

If you have CloudFix subscription, CloudFix will execute these automatic cost savings for you. However, if you'd like to do it yourself - please follow these instructions:

AWS Console:

1. Use CloudFix recommendations to identify the EFS file system you want to enable Intelligent Tiering on.
2. For each file system, click on Edit button and then Modify the rules under Lifecycle management section
   1. Transition to IA - 30 days since last access
   2. Transition out of IA - On first access
   3. Enable Provisioned, and set the Provisioned Throughput value
3. Click on “Save changes” button

AWS CLI

1. Use CloudFix recommendations to identify the EFS file system you want to enable Intelligent Tiering on
2. For each file system, to enable Intelligent Tier, execute the following command

   aws efs put-lifecycle-configuration \
   --file-system-id <File-System-ID> \
   --lifecycle-policies "[{\"TransitionToIA\":\"AFTER_30_DAYS\"},{\"TransitionToPrimaryStorageClass\":\"AFTER_1_ACCESS\"}]"

3. For each file system, to set provisioned throughput, execute the following command

   aws efs update-file-system \
   --file-system-id <File-System-ID> \
   --throughput-mode provisioned \
   --provisioned-throughput-in-mibps <Throughput-Value>