AWS has its own continuous integration tool known as CodeDeploy, using a simple command you would be able to deploy on multiple servers when you want to change something on code base.
Installing code deploy to instance
If code deploy client is not installed at your instance, you would need to do that:
sudo yum install -y ruby wget cd /opt wget https://aws-codedeploy-ap-south-1.s3.amazonaws.com/latest/install chmod +x ./install sudo ./install auto
Create CodeDeploy Application
You have to create Code Deploy application with Deployment type to Inplace deployment, and deployment Configuration set to CodeDeployDefault.OneAtATime.
Give it a name under Ec2 configuration
and Amazon ec2 instance
, say the name is Code deploy instance
. Now you have to add the same tag to all your code deploy instances.
Set IAM Permissions
Now that we are done with installation, we would need to setup IAM rules:
First create an IAM group called CodeDeployGroup
. This group needs AmazonS3FullAccess
and AWSCodeDeployFullAccess permissions. Create a user and add it to this group. This user only needs programmatic access.Save key and key id to somewhere safe.
Create role that has Trusted entities
and Policies
are ec2.amazonaws.com
and AWSCodeDeployRole AmazonS3FullAccess
, respectively.
Edit trust relationship to following:
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": [ "ec2.amazonaws.com", "codedeploy.ap-south-1.amazonaws.com" ] }, "Action": "sts:AssumeRole" } ] }
Create new s3 bucket with previously created IAM rules.
CodeDeploy configuration
My codebase structure is something like following:
- src - <django project> - scripts install_dependencies start_server stop_server appspec.yml codedeploy_deploy.py deploy.sh
appspec.yml
is the file that contains our hooks and configuration for code deploy.
version: 0.0 os: linux files: - source: src destination: /home/centos/proj_name hooks: BeforeInstall: - location: scripts/install_dependencies timeout: 300 runas: root ApplicationStop: - location: scripts/stop_server timeout: 300 runas: root ApplicationStart: - location: scripts/start_server timeout: 300 runas: root
for django scripts/install_dependencies
may look like following:
sudo yum install -y gcc openssl-devel bzip2-devel wget sudo yum install -y make git cd /opt command -v python3.6 || { wget https://www.python.org/ftp/python/3.6.3/Python-3.6.3.tgz tar xzf Python-3.6.3.tgz cd Python-3.6.3 sudo ./configure --enable-optimizations sudo make altinstall } sudo yum install -y mysql-devel
for scripts/start_server
I have following:
cd /home/centos/evaly pip3.6 install -r requirements.txt nohup uwsgi --http :80 --module evaly.wsgi > /dev/null 2>&1 &
for scripts/stop_server
I have following:
pkill uwsgi
I have borrowed a python script from bitbucket team which looks like following:
# Copyright 2016 Amazon.com, Inc. or its affiliates. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file # except in compliance with the License. A copy of the License is located at # # http://aws.amazon.com/apache2.0/ # # or in the "license" file accompanying this file. This file is distributed on an "AS IS" # BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations under the License. """ A BitBucket Builds template for deploying an application revision to AWS CodeDeploy narshiva@amazon.com v1.0.0 """ from __future__ import print_function import os import sys from time import strftime, sleep import boto3 from botocore.exceptions import ClientError VERSION_LABEL = strftime("%Y%m%d%H%M%S") BUCKET_KEY = os.getenv('APPLICATION_NAME') + '/' + VERSION_LABEL + \ '-bitbucket_builds.zip' def upload_to_s3(artifact): """ Uploads an artifact to Amazon S3 """ try: client = boto3.client('s3') except ClientError as err: print("Failed to create boto3 client.\n" + str(err)) return False try: client.put_object( Body=open(artifact, 'rb'), Bucket=os.getenv('S3_BUCKET'), Key=BUCKET_KEY ) except ClientError as err: print("Failed to upload artifact to S3.\n" + str(err)) return False except IOError as err: print("Failed to access artifact.zip in this directory.\n" + str(err)) return False return True def deploy_new_revision(): """ Deploy a new application revision to AWS CodeDeploy Deployment Group """ try: client = boto3.client('codedeploy') except ClientError as err: print("Failed to create boto3 client.\n" + str(err)) return False try: response = client.create_deployment( applicationName=str(os.getenv('APPLICATION_NAME')), deploymentGroupName=str(os.getenv('DEPLOYMENT_GROUP_NAME')), revision={ 'revisionType': 'S3', 's3Location': { 'bucket': os.getenv('S3_BUCKET'), 'key': BUCKET_KEY, 'bundleType': 'zip' } }, deploymentConfigName=str(os.getenv('DEPLOYMENT_CONFIG')), description='New deployment from BitBucket', ignoreApplicationStopFailures=True ) except ClientError as err: print("Failed to deploy application revision.\n" + str(err)) return False """ Wait for deployment to complete """ while 1: try: deploymentResponse = client.get_deployment( deploymentId=str(response['deploymentId']) ) deploymentStatus=deploymentResponse['deploymentInfo']['status'] if deploymentStatus == 'Succeeded': print ("Deployment Succeeded") return True elif (deploymentStatus == 'Failed') or (deploymentStatus == 'Stopped') : print ("Deployment Failed") return False elif (deploymentStatus == 'InProgress') or (deploymentStatus == 'Queued') or (deploymentStatus == 'Created'): continue except ClientError as err: print("Failed to deploy application revision.\n" + str(err)) return False return True def main(): if not upload_to_s3('/Users/sadafnoor/Projects/evaly/artifact.zip'): sys.exit(1) if not deploy_new_revision(): sys.exit(1) if __name__ == "__main__": main()
I have written a script to zip up my source code so that the script can upload it to s3 and eventually all my ec2 instances will be downloading that zip from s3.
export APPLICATION_NAME="CodeDeployApplicationName" export AWS_ACCESS_KEY_ID="IAMUserKeyId" export AWS_DEFAULT_REGION="ap-south-1" export AWS_SECRET_ACCESS_KEY="IAMUserSecretKey" export DEPLOYMENT_CONFIG="CodeDeployDefault.OneAtATime" export DEPLOYMENT_GROUP_NAME="CodeDeployDeploymentGroup" export S3_BUCKET="S3BucketName" zip -r ../artifact.zip src/* appspec.yml scripts/* python codedeploy_deploy.py