Installing AWS Command Line Tools from Amazon Downloads

By on September 5, 2012
aws

 

When you need an AWS command line toolset not provided by Ubuntu packages, you can download the tools directly from Amazon and install them locally.

In a previous article I provided instructions on how to install AWS command line tools using Ubuntu packages. That method is slightly easier to set up and easier to upgrade when Ubuntu releases updates. However, the Ubuntu packages aren’t always up to date with the latest from Amazon and there are not yet Ubuntu packages published for every AWS command line tools you might want to use.

Unfortunately, Amazon does not have one single place where you can download all the command line tools for the various services, nor are all of the tools installed in the same way, nor do they all use the same format for accessing the AWS credentials.

The following steps show how I install and configure the AWS command line tools provided by Amazon when I don’t use the packages provided by Ubuntu.

Prerequisites

Install required software packages:

sudo apt-get update sudo apt-get install -y openjdk-6-jre ruby1.8-full libxml2-utils unzip cpanminus build-essential 

Create a directory where all AWS tools will be installed:

sudo mkdir -p /usr/local/aws 

Now we’re ready to start downloading and installing all of the individual software bundles that Amazon has released and made available in scattered places on their web site and various S3 buckets.

Download and Install AWS Command Line Tools

These steps should be done from an empty temporary directory so you can afterwards clean up all of the downloaded and unpacked files.

EC2 API command line tools:

wget --quiet http://s3.amazonaws.com/ec2-downloads/ec2-api-tools.zip unzip -qq ec2-api-tools.zip sudo rsync -a --no-o --no-g ec2-api-tools-*/ /usr/local/aws/ec2/ 

EC2 AMI command line tools:

wget --quiet http://s3.amazonaws.com/ec2-downloads/ec2-ami-tools.zip unzip -qq ec2-ami-tools.zip sudo rsync -a --no-o --no-g ec2-ami-tools-*/ /usr/local/aws/ec2/ 

IAM (Identity and Access Management) commmand line tools:

wget --quiet http://awsiammedia.s3.amazonaws.com/public/tools/cli/latest/IAMCli.zip unzip -qq IAMCli.zip sudo rsync -a --no-o --no-g IAMCli-*/ /usr/local/aws/iam/ 

RDS (Relational Database Service) command line tools:

wget --quiet http://s3.amazonaws.com/rds-downloads/RDSCli.zip unzip -qq RDSCli.zip sudo rsync -a --no-o --no-g RDSCli-*/ /usr/local/aws/rds/ 

ELB (Elastic Load Balancer) command line tools:

wget --quiet http://ec2-downloads.s3.amazonaws.com/ElasticLoadBalancing.zip unzip -qq ElasticLoadBalancing.zip sudo rsync -a --no-o --no-g ElasticLoadBalancing-*/ /usr/local/aws/elb/ 

AWS CloudFormation command line tools:

wget --quiet https://s3.amazonaws.com/cloudformation-cli/AWSCloudFormation-cli.zip unzip -qq AWSCloudFormation-cli.zip sudo rsync -a --no-o --no-g AWSCloudFormation-*/ /usr/local/aws/cfn/ 

Auto Scaling command line tools:

wget --quiet http://ec2-downloads.s3.amazonaws.com/AutoScaling-2011-01-01.zip unzip -qq AutoScaling-*.zip sudo rsync -a --no-o --no-g AutoScaling-*/ /usr/local/aws/as/ 

AWS Import/Export command line tools:

wget --quiet http://awsimportexport.s3.amazonaws.com/importexport-webservice-tool.zip sudo mkdir /usr/local/aws/importexport sudo unzip -qq importexport-webservice-tool.zip -d /usr/local/aws/importexport 

CloudSearch command line tools:

wget --quiet http://s3.amazonaws.com/amazon-cloudsearch-data/cloud-search-tools-1.0.0.1-2012.03.05.tar.gz tar xzf cloud-search-tools*.tar.gz sudo rsync -a --no-o --no-g cloud-search-tools-*/ /usr/local/aws/cloudsearch/ 

CloudWatch command line tools:

wget --quiet http://ec2-downloads.s3.amazonaws.com/CloudWatch-2010-08-01.zip unzip -qq CloudWatch-*.zip sudo rsync -a --no-o --no-g CloudWatch-*/ /usr/local/aws/cloudwatch/ 

ElastiCache command line tools:

wget --quiet https://s3.amazonaws.com/elasticache-downloads/AmazonElastiCacheCli-2012-03-09-1.6.000.zip unzip -qq AmazonElastiCacheCli-*.zip sudo rsync -a --no-o --no-g AmazonElastiCacheCli-*/ /usr/local/aws/elasticache/ 

Elastic Beanstalk command line tools:

wget --quiet https://s3.amazonaws.com/elasticbeanstalk/cli/AWS-ElasticBeanstalk-CLI-2.1.zip unzip -qq AWS-ElasticBeanstalk-CLI-*.zip sudo rsync -a --no-o --no-g AWS-ElasticBeanstalk-CLI-*/ /usr/local/aws/elasticbeanstalk/ 

Elastic MapReduce command line tools:

wget --quiet http://elasticmapreduce.s3.amazonaws.com/elastic-mapreduce-ruby.zip unzip -qq -d elastic-mapreduce-ruby elastic-mapreduce-ruby.zip sudo rsync -a --no-o --no-g elastic-mapreduce-ruby/ /usr/local/aws/elasticmapreduce/ 

Simple Notification Serivice (SNS) command line tools:

wget --quiet http://sns-public-resources.s3.amazonaws.com/SimpleNotificationServiceCli-2010-03-31.zip unzip -qq SimpleNotificationServiceCli-*.zip sudo rsync -a --no-o --no-g SimpleNotificationServiceCli-*/ /usr/local/aws/sns/ sudo chmod 755 /usr/local/aws/sns/bin/* 

Route 53 (DNS) command line tools:

sudo mkdir -p /usr/local/aws/route53/bin for i in dnscurl.pl route53tobind.pl bindtoroute53.pl route53zone.pl; do sudo wget --quiet --directory-prefix=/usr/local/aws/route53/bin http://awsmedia.s3.amazonaws.com/catalog/attachments/$i sudo chmod +x /usr/local/aws/route53/bin/$i done cpanm --sudo --notest --quiet Net::DNS::ZoneFile NetAddr::IP Net::DNS Net::IP Digest::HMAC Digest::SHA1 Digest::MD5 

CloudFront command line tool:

sudo wget --quiet --directory-prefix=/usr/local/aws/cloudfront/bin http://d1nqj4pxyrfw2.cloudfront.net/cfcurl.pl sudo chmod +x /usr/local/aws/cloudfront/bin/cfcurl.pl 

S3 command line tools:

wget --quiet http://s3.amazonaws.com/doc/s3-example-code/s3-curl.zip unzip -qq s3-curl.zip sudo mkdir -p /usr/local/aws/s3/bin/ sudo rsync -a --no-o --no-g s3-curl/ /usr/local/aws/s3/bin/ sudo chmod 755 /usr/local/aws/s3/bin/s3curl.pl 

Now that we have all of the software installed under /usr/local/aws we need to set up the AWS credentials and point the tools to where they can find everything.

Set up AWS Credentials and Envronment

Create a place to store the secret AWS credentials:

mkdir -m 0700 $HOME/.aws-default/ 

Copy your AWS X.509 certificate and private key to this subdirectory. These files will have names that look something like this:

$HOME/.aws-default/cert-7KX4CVWWQ52YM2SUCIGGHTPDNDZQMVEF.pem $HOME/.aws-default/pk-7KX4CVWWQ52YM2SUCIGGHTPDNDZQMVEF.pem 

Create the file $HOME/.aws-default/aws-credential-file.txt with your AWS access key id and secret access key in the following format:

AWSAccessKeyId=<insert your AWS access id here> AWSSecretKey=<insert your AWS secret access key here> 

Create the file $HOME/.aws-default/aws-credentials.json in the following format:

{ "access-id": "<insert your AWS access id here>", "private-key": "<insert your AWS secret access key here>", "key-pair": "<insert the name of your Amazon ec2 key-pair here>", "key-pair-file": "<insert the path to the .pem file for your Amazon ec2 key pair here>", "region": "<The region where you wish to launch your job flows. Should be one of us-east-1, us-west-1, us-west-2, eu-west-1, ap-southeast-1, or ap-northeast-1, sa-east-1>" } 

Create the file $HOME/.aws-secrets in the following format:

%awsSecretAccessKeys = ( 'default' => { id => '<insert your AWS access id here>', key => '<insert your AWS secret access key here>', }, ); 

Create a symbolic link for s3curl to find its hardcoded config file and secure the file permissions

ln -s $HOME/.aws-secrets $HOME/.s3curl chmod 600 $HOME/.aws-default/* $HOME/.aws-secrets 

Add the following lines to your $HOME/.bashrc file so that the AWS command line tools know where to find themselves and the credentials. We put the new directories in the front of the$PATH so that we run these instead of any similar tools installed by Ubuntu packages.

export JAVA_HOME=/usr export EC2_HOME=/usr/local/aws/ec2 export AWS_IAM_HOME=/usr/local/aws/iam export AWS_RDS_HOME=/usr/local/aws/rds export AWS_ELB_HOME=/usr/local/aws/elb export AWS_CLOUDFORMATION_HOME=/usr/local/aws/cfn export AWS_AUTO_SCALING_HOME=/usr/local/aws/as export CS_HOME=/usr/local/aws/cloudsearch export AWS_CLOUDWATCH_HOME=/usr/local/aws/cloudwatch export AWS_ELASTICACHE_HOME=/usr/local/aws/elasticache export AWS_SNS_HOME=/usr/local/aws/sns export AWS_ROUTE53_HOME=/usr/local/aws/route53 export AWS_CLOUDFRONT_HOME=/usr/local/aws/cloudfront for i in $EC2_HOME $AWS_IAM_HOME $AWS_RDS_HOME $AWS_ELB_HOME $AWS_CLOUDFORMATION_HOME $AWS_AUTO_SCALING_HOME $CS_HOME $AWS_CLOUDWATCH_HOME $AWS_ELASTICACHE_HOME $AWS_SNS_HOME $AWS_ROUTE53_HOME $AWS_CLOUDFRONT_HOME /usr/local/aws/s3 do PATH=$i/bin:$PATH done PATH=/usr/local/aws/elasticbeanstalk/eb/linux/python2.7:$PATH PATH=/usr/local/aws/elasticmapreduce:$PATH export EC2_PRIVATE_KEY=$(echo $HOME/.aws-default/pk-*.pem) export EC2_CERT=$(echo $HOME/.aws-default/cert-*.pem) export AWS_CREDENTIAL_FILE=$HOME/.aws-default/aws-credential-file.txt export ELASTIC_MAPREDUCE_CREDENTIALS=$HOME/.aws-default/aws-credentials.json 

Set everything up in your current shell:

source $HOME/.bashrc 

Test

Make sure that the command line tools are installed and have credentials set up correctly. These commands should not return errors:

ec2-describe-regions ec2-ami-tools-version iam-accountgetsummary rds-describe-db-engine-versions elb-describe-lb-policies cfn-list-stacks cs-describe-domain mon-version elasticache-describe-cache-clusters eb --version elastic-mapreduce --list --all sns-list-topics dnscurl.pl --keyname default https://route53.amazonaws.com/2010-10-01/hostedzone | xmllint --format - cfcurl.pl --keyname default https://cloudfront.amazonaws.com/2008-06-30/distribution | xmllint --format - s3curl.pl --id default http://s3.amazonaws.com/ | xmllint --format - 

Are you aware of any other command line tools provided by Amazon? Let other readers know in the comments on this article.

 

Source : http://alestic.com/2012/09/aws-command-line-tools?utm_source=feedburner&utm_medium=email&utm_campaign=Feed%3A+alestic+%28Alestic.com+-+Ubuntu+on+EC2%29

About jeevan

Geek under construction, Tech Speaker, Tech Blogger @Geekospace.com, foodie, Cloud Evangelist, Moviez freak passionate learner, startup lover, business and tech savya, social media addict, biker, musician, extrovert, realistic, seminal, approachable.


Leave a Reply

Your email address will not be published. Required fields are marked *

*


*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>