What is DevOps?
DevOps is a set of software development practices that combines software development (Dev) and information technology operations (Ops) to build, test and release software at high velocity, more reliably and in close alignment with business objectives. This speed enables organizations to better serve their customers and compete more effectively in the market.
PSL Group Overview
PSL Group is a global organization dedicated to putting information at the service of medicine. The companies and people of the PSL Group aim to improve medical care by serving those who need it, those who provide it and those who seek to improve it. PSL Group collects data about physicians via surveys and web data collection methods. The goal is to locate a personal email for each physician. Email campaigns are sent out to a physician’s personal email address and aggregations are performed regarding physician’s skills. PSL Group sells recommendation (aggregate) data to physicians. The Integrated Data Hub (IDH) is a composite of multiple storage types. The Raw and Ingestion Zones utilize AWS S3. A relational database design is used for the Lima DB Zone. Snowflake is used for the Data Warehouse (DW) Zone.
Goals
The Northbay team brought into board to achieve the following DevOps goals.
- To create CloudFormation templates (IAAC) to provision infrastructure.
- To implement CI/CD processes for Microservices based software deployments into multiple environments.
- To set up logs/events to monitor deployment.
Challenge
- To write highly parameterized and reusable CloudFormation templates to set up infrastructure across all environments.
- To provision resources, across account boundaries, required in the target accounts from the CloudFormation stack present in Shared Services account using custom stackset resources.
- To allow services in the target accounts to be able to access encrypted S3 bucket artifacts in the Shared Services account.
- To access ECS images in the Shared Services ECS repository during ECS task executions in the target accounts.
Delivered Solution
- CloudFormation Based Infrastructure Stack
Infrastructure was provisioned for different environments using nested CloudFormation templates
- Serverless Templates for SAM application
Serverless templates were used to deploy Serverless application (Lambdas, APIs)
- CI/CD Stack
Multiple CodePipelines were used for continuous integration and continuous deployment. The source code, resided within a CodeCommit repository, was built using CodeBuild and deployed using CloudFormation.
- Logging and Monitoring using CloudWatch
CloudWatch was used to maintain application level logging as well as CloudWatch based rules were set up to notify about the deployment status.
CloudFormation based Infrastructure Stack
Nested CloudFormation stacks were used to provision infrastructure across all multiple environments spread across multiple accounts. In the Shared Services account, StackSets were used to create resources, in the deployment accounts, required for cross-account software deployments. The following services were created during infrastructure provisioning along with their IAM roles and policies.
- Amazon VPC
- Amazon EC2
- AWS Aurora PostgreSQL
- Amazon ElastiCache
- AWS ElasticSearch
- AWS IAM Roles and Policies
- AWS SSM Parameters
- AWS S3 Buckets
- AWS KMS Keys
- AWS Lambda
- AWS CloudWatch Rules
- AWS SNS Topics
- AWS ECR Repositories
- AWS ECS Clusters
- AWS CodeCommit Repositories
- AWS CodeBuild Projects
- AWS CodePipelines
CloudFormation stack provisioned CodePipelines for continuous integration and deployment for all the microservices in an environment.
Following figure shows the flow and implementation of CICD in PSL:
Continuous Integration and Continuous Deployment
CodePipeline was used to automate steps in the software delivery process by integrating code, building artefacts and deploying resources on every code update.
Continuous Integration
Developers work on feature branches and create pull requests in order to merge their changes into develop branch. The pull requests are approved only by senior developers group. Upon approval of the pull request, the CodePipeline is triggered against the develop branch and code is deployed to Development environment.
Each permanent branch i.e. develop, staging, pre-prod and master has an associated CodePipeline that will be executed based on a commit event in that branch and deploy software to Development, Staging, Pre-production and Production environment respectively.
Continuous Delivery
The CodePipelines are triggered by changes being committed to the CodeCommit repository. Upon execution the Pipelines package the SAM template, upload artifacts to S3 bucket and create docker images for ECS tasks and push them to AWS ECR during the build stage. The Pipelines then deploy that template in the target account using CloudFormation stack after manual approval. Lastly, the Pipelines copy files to the cross-account bucket and execute Fargate tasks in the ECS cluster in the target account.
CodeBuild
Codebuild projects have been used for the following purposes:
- Shared Services Account
- Package SAM template and upload it to S3 artifact bucket
- Create docker image and push it to ECR repository
- Copy file to the cross-account S3 bucket
- Target Account
- Apply API authentication using Python script
- Create CloudWatch rule to schedule ECS tasks
- Execute Fargate jobs on the ECS cluster
Shared Service Account Model
Shared Services Account model was implemented in PSL Group. The CodeCommit repositories, CodePipelines, artifact store and logging S3 buckets are present in the Shared Services account. Access to these resources was made enabled through IAM based cross-account access for the deployment accounts (Non-production and Production).
Shared Service Account Model allowed the source code and deployment pipelines to reside in a single account while at the same time enabling deployments to other accounts. This approach led to a defined, secure and more manageable CI/CD infrastructure. The services in the target accounts are given granular access to only the required resources using IAM roles, S3 bucket policies, KMS key and ECR repository permissions.
About NorthBay
We are a fast-growing, 100% AWS focused onshore/offshore AWS Premier Consulting Partner, supporting our customers to accelerate the reinvention of their applications and data for a Cloud-native world. Our >350 AWS Certified Employees excel in developing and deploying Database & Application Migrations, Data Lakes and Analytics, Machine Learning/AI, DevOps and Application and Data Modernization/Development that drive measurable business impact.
