(Preview) ElasticSearch Optimize Storage

Opportunity Name:

ElasticSearch Optimize Storage

AWS Resource Type:

Amazon OpenSearch Service (formerly Amazon ElasticSearch)

Opportunity Description:

Amazon OpenSearch is a managed service for ElasticSearch. It provides search, monitoring, and analysis for real-time data streams like application metrics and logs.

Amazon OpenSearch service supports gp2, io1, and standard magnetic EBS volumes for storage. gp2 volumes are -by far- the most commonly used.

This opportunity right sizes the OpenSearch gp2 storage volumes. CloudFix checks the available free storage as well as growth in storage for past 30 days from CloudWatch metrics. It then calculates the ideal storage that should accommodate the growth factor as well as keep additional buffer of free storage.

Criteria for identifying the opportunity:

  • ClusterStatus is green and Automated snapshot has not failed in past 24 hours
  • OpenSearch domains is using General Purpose gp2 volume
  • The gp2 storage cost for the domain is more than $100 annually
  • The domain has more than 30 days of CloudWatch metrics data

Potential savings (range in % on annual basis):

The cost savings depend on the current storage allocation for your OpenSearch domains. With each GB reduced, the monthly costs will drop by $0.135 multiplied by number of nodes. Based on our experience, you can expect up to 25% cost-savings on the OpenSearch storage costs.

What happens when the Fixer is executed?

The fixer will resize the cluster volume, during this process, the fixer will monitor following metrics:

  • ClusterStatus.red
  • FreeStorageSpace
  • ClusterIndexWritesBlocked

Risks

Although this change has no risk, the change should be performed when there’s less traffic on the OpenSearch domain.

Is it possible to rollback once CloudFix implements the fixer?

Manual rollback on your AWS account is possible. One-click rollback via CloudFix for this opportunity is currently not available.

  • AWS can provide support to investigate if the OpenSearch domain has been stuck in the processing state for more than 24 hours
  • If the OpenSearch domain cluster has entered into the "Red" state, the red indices need to be deleted and restored from an automated snapshot.
  • During the change, if the OpenSearch volume's free storage becomes less than 20%, CloudFix will automatically resize it to have 30% free storage space.

Can CloudFix implement the fix automatically once I accept the recommendation?

Yes. OpenSearch domain storage resizing is automatic – once you accept the recommendation and schedule the fix, execution is automatic.

Does this fix require downtime?

No. OpenSearch storage resizing via CloudFix does not require any downtime. It is executed as a zero downtime fix.

Do it yourself (DIY) Instructions:

If you have CloudFix subscription, CloudFix will execute these automatic cost savings for you. However, if you'd like to do it yourself - please follow these instructions:

Finding cost saving recommendations with CloudFix is FREE! You'll need a paid subscription only if you'd like CloudFix to execute those recommendations.

 

AWS Console:

  • Use CloudFix recommendations to identify the OpenSearch domains you want to resize the storage.
  • For each domain, select the click on the domain and click on Edit button under Cluster configuration section

mceclip0.png

  • Change the "EBS storage size per node" value

mceclip1.png

  • Click on "Save changes" button.

AWS CLI:

  • Use CloudFix recommendations to identify the OpenSearch domains you want to rightsize the storage
  • For each OpenSearch domain execute following command
aws opensearch update-domain-config --domain-name <Name of domain> \
--ebs-options '{
"VolumeSize": <Volume size>
}'

Comments

0 comments

Please sign in to leave a comment.

Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request