An Introduction to the AWS Command Line Tool Part 2

In Part 1 of this series, I introduced awscli, a powerful command line interface which can be used to manage AWS services.

I’d like to continue digging deeper into awscli by provisioning a Volume with Amazon Elastic Block Store (EBS) and creating an Amazon Relational Database Service (RDS) MySQL instance.

By using awscli to provision EBS volumes, we have a simple and cost effective way via the command line, to expand our Ec2 disk capacity without any downtime. EBS volumes also support snapshots, providing us with low level block device backups which can be restored or copied to new or existing Ec2 instances.

RDS instances are a way System Administrators can run our own Databases within AWS without having to spend the extra effort on complex performance tuning, routine maintenance and redundancy configuration.

Both Amazon EBS and RDS are available to new AWS customers under the AWS Free Usage Tier. Please ensure you understand AWS pricing before proceeding.

Create and Attach a Volume with Elastic Block Store

Let’s get started with creating and then attaching a 1GB EBS volume to our Ec2 Instance created in Part 1 of our tutorial.

$ aws ec2 create-volume 
    --size 1 
    --availability-zone ap-southeast-2a

The output of create-volume will provide us with a VolumeId which we will use to attach the volume to our Ec2 instance. Note that I’ve chosen ap-southeast-2a as the availability-zone which shares the same Availability Zone of my Ec2 instance created in Part 1.

$ aws ec2 attach-volume 
    --volume-id vol-5dda4d0a 
    --instance-id i-0d9c2b31 
    --device /dev/xvdb

From within our Ec2 instance, we should now see the new EBS volume as /dev/xvdb.

$ fdisk -l /dev/xvdb
Disk /dev/xvdb: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
...

We are now free to use this block device as a normal disk within Linux. Let’s go ahead and partition the disk, apply a filesystem and then mount it on /srv.

$ parted -s -a optimal /dev/xvdb mklabel msdos
$ parted -s -a optimal /dev/xvdb mkpart primary ext2 0% 100%
$ mkfs.ext4 /dev/xvdb1
$ mount /dev/xvdb1 /srv

EBS volumes support snapshots which are a point in time copy of the volume.

Snapshots are ideal for simple and cost effective backups of block devices which contain file systems. Snapshots can be copied across AWS regions and turned into volumes which can be mounted on other Ec2 Linux instances.

$ aws ec2 create-snapshot 
    --volume-id vol-5dda4d0a 
    --description "My First Snapshot"

If we wanted to attach this snapshot to any other Ec2 instance, we would use —snapshot-id as an argument to aws ec2 create-volume. We could then attach the volume created from a snapshot to another Ec2 instance, which is a very efficient way to copy large blocks of data across servers.

Provision a MySQL Database with Relational Database Service

Before we create our first RDS MySQL DB instance, lets create another Security Group which only allows inbound traffic on port 3306/tcp.

$ aws ec2 create-security-group 
    --group-name MySecurityGroupDBOnly 
    --description "Inbound DB only"

authorize-security-group-ingress allows us to permit source traffic for a Security Group by providing a source network address range (eg; 172.31.0.0/20), an AWS Account ID (eg; 561434394141) or a Security Group Name.

$ aws ec2 authorize-security-group-ingress 
    --group-name MySecurityGroupDBOnly 
    --source-group MySecurityGroupDBOnly 
    --protocol tcp --port 3306

The above rule, perhaps slightly confusing at first, allows any Ec2 instance associated with the Security Group MySecurityGroupDBOnly to access any other Ec2 or RDS instance associated with the MySecurityGroupDBOnly Security Group on port 3306/tcp. We’ll come back to this Security Group later.

Let’s move on to creating our first RDS instance by making use of the rds service within awscli.

New RDS DB instances are provisioned using the create-db-instance option. create-db-instance requires a password for our instance which we will rely on apg, an automated password generator, to create for us. We will also disable our shell’s history file so we don’t save any sensitive passwords to disk in clear text.

$ apg -a0 -n10 -m16
$ unset HISTFILE
$ aws rds create-db-instance 
    --db-name MyDatabase 
    --db-instance-identifier my-db-instance 
    --allocated-storage 5 
    --db-instance-class db.t1.micro 
    --engine MySQL 
    --master-username dbadmin 
    --master-user-password @SECURE_PASSWORD_GENERATED_WITH_APG@ 
    --vpc-security-group-ids sg-c53b20a7

Do replace @SECURE_PASSWORD_GENERATED_WITH_APG@ with a password generated from apg.

Also note the Security Group Id passed to —vpc-security-group-ids which is the Id of MySecurityGroupDBOnly returned by awscli when we created the group with create-security-group. You will need to substitute this with your Security Group Id for MySecurityGroupDBOnly.

After a few minutes, we can discover the RDS instance Address which is within the Endpoint section of describe-db-instances.

$ aws rds describe-db-instances

By default, all inbound traffic to our new RDS DB instance will be denied though by modifying the Security Groups our current Ec2 instance belongs to, we can neatly grant access to our DB instance.

$ aws ec2 modify-instance-attribute 
    --instance-id i-95bd36aa 
    --groups sg-8a051ee8 sg-c53b20a7

Ec2 instances can belong up to 5 Security Groups which provides us with a lot of flexibility when designing our Internal and External traffic policies. By using the modify-instance-attribute command above, we’ve confined our Ec2 Instance to the Security Groups with Ids of sg-8a051ee8 and sg-c53b20a7. The first Security Group Id, sg-8a051ee8, is the original Security Group MySecurityGroupSSHOnly created in Part 1. The second, sg-c53b20a7, is the Security Group Id of MySecurityGroupDBOnly.

Once we install the MySQL client on our Ec2 instance, we should now be able to connect into our RDS DB instance which will have our MyDatabase database waiting for us.

$ yum install mysql
$ mysql --user=dbadmin 
    --password 
    --host=my-db-instance.ccyld3xbhrju.ap-southeast-2.rds.amazonaws.com
mysql> SHOW DATABASES;

An expert in the Linux Infrastructure space, Rene Cunningham is a passionate Systems Engineer with a strong background in Open Source Software Stacks and large scale infrastructure. Based in Melbourne, Australia, Rene works on a highly experienced, distributed team of professional System Administrators at The Linux Foundation.