AWS Resource Type:
Simple Storage Service (S3)
AWS S3 defaults to the most expensive "standard" storage class for all new objects, assuming frequent access. S3 Intelligent-Tiering (SIT) allows you to automatically optimize the storage costs by transitioning infrequently accessed objects to less expensive access tiers.
Most data stored in the standard tier is almost never accessed, and can easily move to the infrequent access tier. One could move objects to the infrequent access tier by API calls, or more typically, by creating a lifecycle policy.
However, retrieving data from the infrequent access tier costs $10 / TB every time it is accessed, so ops teams are generally conservative with their lifecycle policy settings. With S3 intelligent tiering, the transition between these two tiers is automatic and free. The fee for S3 intelligent tiering is well worth that cost given the operational and cost benefits of turning it on. Based on thousands of buckets analyzed, we generally get at least 50% reduction in costs when data is moved from standard to S3 intelligent tiering.
Criteria for identifying the opportunity:
- Enable Recommendations: CloudFix estimates potential cost savings from enabling SIT on your S3 buckets and creates recommendations.
- When you accept those recommendations, CloudFix enables SIT on each of those buckets for which you accepted SIT recommendations.
- CloudFix analyzes and recommends opportunities at the bucket level, not at the object level because complexity of object level management does not justify incremental savings.
- CloudFix SIT recommendations are limited to S3 and exclude Glacier.
Potential savings (range in % on annual basis):
Enabling SIT results in S3 costs reduction by 5% to 15%.
What happens when the Fixer is executed?
When intelligent tiering is turned on, your S3 objects will move from standard and infrequent access tiers to the intelligent tiering frequent access tier (shows up as “INT-FA” on Cost Explorer). Thirty days later, all unaccessed data moves to the Intelligent tiering infrequent access tier (shows up as “INT-IA”), which is roughly half the cost of INT-FA. Ninety days later, unaccessed data moves to the archive instant access tier (“INT-IA”), which is 20% of the cost of the standard tier.
Transitioning data between tiers does not affect availability, so this fix does not involve any downtime or performance drop compared to standard tier.
Almost all client side applications can continue to access the data with the same APIs / methods / URLs. The only (very rare) exception is applications which use the jclouds library with versions older than 2.3.0. In this case, you will have to upgrade your jclouds library, which is fairly straightforward.
Is it possible to rollback once CloudFix implements the fixer?
Manual rollback on your AWS account is possible. One-click rollback via CloudFix for this opportunity is currently not available.
You can turn off intelligent tiering by deleting the lifecycle rule created by the fixer. However, individual objects will then have to be transitioned to S3 standard tier.
Can CloudFix implement the fix automatically once I accept the recommendation?
Yes. Enabling / disabling SIT policy on S3 buckets is automatic – once you accept the recommendation and schedule the fix, execution is automatic.
Does this fix require downtime?
No. Executing S3 Intelligent-Tiering (SIT) lifecycle rule does not require any downtime. It is executed as a zero downtime fix.
Do it yourself (DIY) Instructions:
If you have CloudFix subscription, CloudFix will execute these automatic cost savings for you. However, if you'd like to do it yourself - please follow these instructions:
|Finding cost saving recommendations with CloudFix is FREE! You'll need a paid subscription only if you'd like CloudFix to execute those recommendations.|
- Use CloudFix recommendations to identify which S3 buckets you want to enable SIT on. Alternatively, you can go through all your S3 buckets in the AWS console.
- For each S3 bucket, navigate to <bucket-name> -> Management -> Lifecycle rules
- Click on ‘create lifecycle rule’
- Give it a name
- Choose the rule scope to apply to the entire bucket (unless you have a special case, you may not want to filter objects for this rule)
- Configure transition to S3 Intelligent Tiering
- Click on ‘create rule’
Check this AWS documentation for more details.
aws s3api put-bucket-lifecycle-configuration \
--bucket bucketname \
Sample lifecycle json file here:
Check this AWS Documentation -> Using AWS CLI section for more details.