As most who have adopted AWS services can confirm, the larger and more complex your cloud, the more overwhelming and time consuming it is to manage. To avoid the loss of productivity that comes with this management, many cloud administrators are doing their best to embrace automation wherever possible.
Thankfully, automating AWS can be fairly straightforward, with a number of built-in services and supported integrations available. Here, we’ll introduce a few of these services and cover some ways in which they might benefit you.
Recommended Read: Guide to install Ansible on Ubuntu
Also Read: Ansible Tutorial: Introduction to simple Ansible commands
Considerations for Automating AWS
In AWS, you can automate pretty much any service or function that is accessible through API or the CLI. A number of services, such as Backup and CloudWatch, even include built-in automation functions, such as alerting or scheduling. The downside of this is that if automation isn’t used responsibly, you can quickly rack up additional costs. These costs can arise from misconfiguration of resources, over-provisioning, or simply the costs of performing computation actions. If applied carefully, however, automation can help you standardize your configurations and centralize the management of many, if not all, of your services.
For automating AWS, you often need to first create the appropriate user roles and permissions, granting access to services and allowing modification of their configurations or data. This is done through either Identity Access Management (IAM) or CloudFormation. The exact roles and permissions needed will depend on the services you employ and their requirements can be found in the appropriate service documentation.
Automation Methods
AWS offers several methods for automating services, tasks, and configuration management.
CloudFormation
CloudFormation is an AWS service that allows you to create and manage AWS resources in “stacks” of applications or services via JSON templates. It allows you to quickly configure infrastructure components in a standardized way, easily troubleshoot issues, and provision and deploy resources. All of this means less manual effort from you and a smaller chance of misconfiguring resources.
CloudFormation is often used as part of Continuous Delivery (CD) pipeline, in combination with AWS CodePipeline, by DevOps teams. This requires identifying a CloudFormation template as a deployment action in your CodePipeline workflow. Once done, you can easily deploy and terminate test or pre-production environments, manage application stacks, and run changesets. These changesets let you test how changes will affect your resources and applications before pushing those changes into production
AWS Systems Manager
Systems Manager is a service that provides operational data from services, such as the size or health of EC2 instances. This data is then used to automate tasks related to services. For example, Systems Manager can be used to take bulk actions, like deploying instances, perform configuration changes, or deploying patches. With it, you can group resources, such as instances or storage buckets, for easier management and monitoring. Systems Manager performs these automation functions through automation documents that are written in either JSON or YAML.
To make applying automation easier, there are predefined documents available for common tasks that can be customized according to your needs. These documents allow you to specify specific actions as well as define those actions’ schedules and success criteria. For example, you can define different outcomes depending on whether an action fails or is completed as anticipated. When creating documents you can specify whether actions are role dependent, configure them to use branching logic, and define how they are triggered by different resource states.
Some of the most common uses of Systems Manager include the management of CloudFormation stacks, resource tagging, initiation of instances, creation and updating of machine images, and performing API actions.
AWS Lambda
Lambda is Amazon’s serverless computing service. It allows you to use Amazon servers to run compute functions and eliminates the need to provision or manage servers independently. Lambda is typically used with third-party integrations, such as cron. In Lambda, you can upload custom scripts written in JavaScript, Ruby, Go, Python, and .NET. To use this service, you simply need to upload the code for the functions you wish to run and configure their runtime conditions. These functions can be run via manual activation, time scheduling, being triggered by AWS services, or by directly calling functions through web or mobile applications.
The functions you create can be used to manage the bulk of your cloud monitoring and maintenance responsibilities. For example, creating alerts for data changes in high priority storage buckets or initiating the deployment of applications through CodePipeline.
Your functions can be initiated by 15 different services, including CloudWatch, S3, and CodeCommit and event information can be read from Kinesis, SQS, and DynamoDB.
A possible downside of Lambda is that it requires at least basic programming knowledge to create custom functions. However, if you don't have this ability you can still use pre-built functions supplied by the AWS user community. It is relatively easy to find tutorials with code samples on AWS site directly or through a variety of websites, such as this tutorial showing how to automate EBS snapshots
Lambda is frequently used for serverless backends, allowing for highly available, scalable web apps, and data processing, allowing for real-time file processing, Extract, Transfer, Load (ETL) processing, and real-time stream processing. .
Third-Party Services
Many third-party services work with automating AWS configurations. Most can be directly integrated with AWS services and can be deployed either in containers or on top of EC2 instances, depending on your workflow requirements.
Two commonly used examples are:
- Ansible—includes built-in modules for managing AWS, including functions for provisioning, authentication, inventory, security grouping, autoscaling, and tagging. This functionality is applied through playbooks written in YAML format for easier human understanding.
- Jenkins—provides automation server used for CI/CD workflows that includes broad variety of plug-ins that allow interaction with AWS through API. This is accomplished by installing the AWS CLI on your Jenkins server and adding build steps that execute shell commands.
Conclusion
Applying automation to the configuration and management of your AWS services allows you to significantly reduce the time and effort you must spend. It lets you optimize your cloud performance while leaving you free to work on more productive and less routine tasks. This is especially valuable as your cloud system grows larger and more complex.
Although we only covered a few of your automation options in this article, you should now have a better idea of what is available to you. From here, you can begin applying automation where the appropriate and better plan for more complex workflow modifications to maximize your productivity.
--------------------
Author Bio
Gilad David Maayan is a technology writer who has worked with over 150 technology companies including SAP, Imperva, Samsung NEXT, NetApp and Ixia, producing technical and thought leadership content that elucidates technical solutions for developers and IT leadership. Today he heads Agile SEO, the leading marketing agency in the technology industry.
LinkedIn: https://www.linkedin.com/in/giladdavidmaayan/
