AWS Resource Type:
Simple Storage Service (S3)
AWS S3 defaults to the most expensive "standard" storage class for all new objects, assuming frequent access. S3 Intelligent-Tiering (SIT) allows you to automatically optimize the storage costs by transitioning infrequently accessed objects to less expensive access tiers.
Most data stored in the standard tier is almost never accessed, and can easily move to the infrequent access tier. One could move objects to the infrequent access tier by API calls, or more typically, by creating a lifecycle policy.
However, retrieving data from the infrequent access tier costs $10 / TB every time it is accessed, so ops teams are generally conservative with their lifecycle policy settings. With S3 intelligent tiering, the transition between these two tiers is automatic and free. The fee for S3 intelligent tiering is well worth that cost given the operational and cost benefits of turning it on. Based on thousands of buckets analyzed, we generally get at least 50% reduction in costs when data is moved from standard to S3 intelligent tiering.
Criteria for identifying the opportunity:
CloudFix will create recommendations to enable S3 Intelligent-Tiering for all buckets matching the following criteria:
- The bucket does not have a global deletion policy removing objects within 30 days
- The bucket does not have a global policy archiving objects within 30 days
- The bucket has an annual cost of more than $100
Potential savings (range in % on annual basis):
Enabling SIT results in S3 costs reduction by 5% to 15%.
What happens when the Fixer is executed?
When intelligent tiering is turned on, your S3 objects will move from standard and infrequent access tiers to the intelligent tiering frequent access tier (shows up as “INT-FA” on Cost Explorer). Thirty days later, all unaccessed data moves to the Intelligent tiering infrequent access tier (shows up as “INT-IA”), which is roughly half the cost of INT-FA. Ninety days later, unaccessed data moves to the archive instant access tier (“INT-IA”), which is 20% of the cost of the standard tier.
Transitioning data between tiers does not affect availability, so this fix does not involve any downtime or performance drop compared to standard tier.
Almost all client side applications can continue to access the data with the same APIs / methods / URLs. The only (very rare) exception is applications which use the jclouds library with versions older than 2.3.0. In this case, you will have to upgrade your jclouds library, which is fairly straightforward.
Is it possible to rollback once CloudFix implements the fixer?
Manual rollback on your AWS account is possible. One-click rollback via CloudFix for this opportunity is currently not available.
You can turn off intelligent tiering by deleting the lifecycle rule created by the fixer. However, individual objects will then have to be transitioned to S3 standard tier.
Can CloudFix implement the fix automatically once I accept the recommendation?
Yes. Enabling / disabling SIT policy on S3 buckets is automatic – once you accept the recommendation and schedule the fix, execution is automatic.
Does this fix require downtime?
No. Executing S3 Intelligent-Tiering (SIT) lifecycle rule does not require any downtime. It is executed as a zero downtime fix.
Do it yourself (DIY) Instructions:
If you have CloudFix subscription, CloudFix will execute these automatic cost savings for you. However, if you'd like to do it yourself - please follow these instructions:
|Finding cost saving recommendations with CloudFix is FREE! You'll need a paid subscription only if you'd like CloudFix to execute those recommendations.|
- Use CloudFix recommendations to identify which S3 buckets you want to enable SIT on. Alternatively, you can go through all your S3 buckets in the AWS console.
- For each S3 bucket, navigate to <bucket-name> -> Management -> Lifecycle rules
- Click on ‘create lifecycle rule’
- Give it a name
- Choose the rule scope to apply to the entire bucket (unless you have a special case, you may not want to filter objects for this rule)
- Configure transition to S3 Intelligent Tiering
- Click on ‘create rule’
Check this AWS documentation for more details.
- Use CloudFix recommendations to identify which S3 buckets you want to enable SIT on. Alternatively, you can go through all your S3 buckets in the AWS console, and identify all buckets matching the eligibility criteria.
- For each S3 bucket, execute the following command, replacing [bucket_name] with the name of the bucket.
aws s3api put-bucket-lifecycle-configuration \
--bucket [bucket_name] \
You can use this sample lifecycle, save it as lifecycle.json.
Check this AWS Documentation -> Using AWS CLI section for more details.