How do I automate AWS snapshot scheduling?
Amazon Web Services (AWS) offers its customers several methods to help automate manual tasks such as snapshot scheduling. However, it can be a challenge to automate these tasks on various AWS resources across multiple regions and accounts.
To help customers more easily manage cross-account and cross-region automation, AWS offers the AWS Ops Automator solution. This solution enables customers to easily configure schedules to automatically create, copy, and delete Amazon Elastic Block Store (Amazon EBS) snapshots, and copy and delete Amazon Redshift snapshots. The solution also enables you to automatically set the throughput capacity for Amazon DynamoDB on a schedule.
The AWS Ops Automator is designed to provide a core framework for automated tasks, allowing you to focus on extending the solution’s functionality rather than managing the underlying infrastructure operations. The solution can be customized to include task audit trails, logging, resource selection, scaling, API request retries, completion handing for long tasks, and concurrency handling.
General Best Practices
When running infrastructure on the cloud, automate the monitoring and management of resources whenever possible. Automated actions are more reliable than manual processes, helping to reduce the operational complexity of managing several resources across multiple regions and accounts. For optimal efficiency and cost savings, consider these best practices when automating resource management:
- Carefully consider your snapshot frequency and retention settings before deploying the AWS Ops Automator to avoid incurring unnecessary charges.
- Implement a resource-identification system, such as tags for AWS resources. This helps to ensure that automated actions are targeted to the correct resource, and also allows for easier filtering, modification, and troubleshooting according to categories that you define.
- Grant least-privilege access to individuals or systems that perform automated actions on resources.
- Deploy automation tools in a tightly controlled, isolated account with limited access. This will prevent unauthorized users from creating and deleting resources.
AWS Solution
AWS offers a solution that allows you to use time-based or event-based triggers to automatically manage AWS resources across regions and accounts. The diagram below presents the components and functionality you can build using the AWS Ops Automator implementation guide and accompanying AWS CloudFormation template.
- The AWS CloudFormation template launches the core framework, which includes a suite of microservices (AWS Lambda functions) that manage triggering events, resource selection, task execution, concurrency control, and completion.
- Task configuration data, which defines the triggering event, how the task should be performed, which resources will be selected by the actions, and where these resources are located, is stored in an Amazon DynamoDB table.
- Solution-generated AWS CloudFormation templates configure tasks based on parameters you define, and the roles necessary to perform actions across accounts.
- The solution tracks all steps in the process, the selected resources, and the results of the actions, including possible errors, in a DynamoDB table.
- The solution also leverages Amazon CloudWatch Logs for logging. Warning and error messages are published to a solution-created Amazon Simple Notification Service (Amazon SNS) topic which sends messages to a subscribed email address.
What you’ll accomplish:
Deploy the AWS Ops Automator using AWS CloudFormation. The CloudFormation template will automatically launch and configure the components necessary to automatically manage AWS resources across regions and accounts.
Automate operational tasks on specific AWS resources using time-based or event-based triggers. When a trigger occurs, the solution automatically runs the task on the applicable resources.
Implement a core framework that allows you to focus on developing new functionality rather than underlying infrastructure operations. The framework includes task audit trails, logging, resource selection, scaling, API request retries, completion handing for long tasks, and concurrency handling.
What you’ll need before starting:
An AWS account: You will need an AWS account to begin provisioning resources. Sign up for AWS.
Skill level: This solution is intended for IT infrastructure and DevOps professionals who have practical experience with automation, Python, the Boto3 API, and architecting on the AWS cloud.
Solution FAQs
Q: What initial actions does the AWS Ops Automator include?
The solution includes a set of actions that automatically create, copy, and delete Amazon Elastic Block Store (Amazon EBS) snapshots; copy and delete Amazon Redshift snapshots; and set the throughput capacity for Amazon DynamoDB.
Q: Can I develop custom actions?
Yes. You can use the Ops Automator’s default actions as a reference to quickly and consistently develop custom actions that extend the solution’s functionality to other AWS services.
Q: Can I manage AWS resources in multiple regions and accounts?
Yes. The solution includes AWS CloudFormation templates that create the AWS Identity and Access Management (IAM) roles necessary to automate tasks on resources in multiple accounts. The templates also include parameters where you can specify regions where your task will run.
Q: How do I identify resources that will receive automated actions?
During initial configuration of the primary AWS CloudFormation template, you define a tag key you will use to identify resources that will receive automated actions. When you deploy a task template, the stack name you specify is used as the tag value that identifies the task you want the solution to perform on the tagged resource. For detailed information, see the implementation guide.
Q: Can I deploy the AWS Ops Automator in any AWS Region?
Customers can deploy the Ops Automator AWS CloudFormation template only in AWS Regions where AWS Lambda, Amazon DynamoDB, and Amazon CloudWatch are available (see AWS service offerings by region). However, once deployed, the solution’s can execute actions on resources in any AWS Region.