Learn about the difficulties of manually performing the deployment pipeline in large projects and how the CI/CD pipeline can help.

CI/CD

The deployment pipeline is automated, allowing developers to focus on building new features!

Overall, the CI/CD pipeline is the implementation of DevOps culture through automation. The relationships between them can be summarized in three points:
1. The CI/CD pipeline optimizes the production system as a whole, removing bottlenecks at each stage of the deployment process.
2. The CI/CD pipeline enables learning and experimentation due to the ability to rapidly release changes and update the application. 
3. Feedback loops are built into the CI/CD pipeline through continuous testing. As changes move through the pipeline, developers can track the testing progress of their code at each stage of the pipeline.
 
CI/CD not only improves each individual part of the deployment pipeline, it optimizes the entire system. The pipeline allows for developers to react quickly to changing customer demands, and allows customers to view the most recent version of the application. Additionally, by automating many of the processes, developers can focus on developing new features. No wonder many companies are incorporating the CI/CD pipeline into their projects!


DevOps Culture & The CI/CD Pipeline

Continuous testing is featured in continuous integration, delivery, and deployment.

_It happened again. A bug was accidentally released to the public in one of Anita’s apps. This rarely happened when she wrote the code on her own. Now with so many developers merging in code and fixing conflicts all while reviewing different PRs, it seems to be inevitable._
 
Using automation to control the frequency and location of tests can minimize this issue. This automation of testing is called **Continuous Testing**. It involves automatically triggering tests to be executed once an application is built in a new environment. Using continuous testing casts a wide net, catching bugs early and ensuring that the requirements of the project are met. 
 
Tests can take a variety of forms with the most common being:
 
* Tests during development
  * Unit Tests
  * Integration Tests
* Tests before deployment to production
  * Acceptance Tests
  * End-to-end Tests
 
Since tests are triggered automatically each time the application is built in a new environment, developers don't need to constantly monitor the state of the tests. Developers are simply notified of any failed tests by the automation tool, allowing them to quickly address the problem. When tests pass in a given environment, the approved code can automatically be moved into the next environment where even more tests may be executed. 


Continuous Testing

Code moves from a developers laptop through the deployment process. Along the way there is version control, testing, and various environments.

_Anita has been an independent programmer for years, working solo on most of her projects. Recently, she has begun working on larger projects with a team and some of her development and deployments practices aren't working as well as before. When merging her code, she found that there were a lot of conflicts with the other developers’ code. Some bugs were slipping through the merges and causing more issues. Even some of the deployments failed! They did not have a prototype ready to show and faced many delays._ 

_Anita began to realize that her processes needed to change to scale up for larger projects._

Throughout this course, we have learned about various DevOps practices such as version control, environment configuration, and testing. These practices enable developers to deploy their code with great speed and reliability. However, when attempting to use these practices and tools at scale, issues like Anita's can arise. 

Teams practicing DevOps will turn to **automation** to scale their development and deployment practices. 

Automation is the process of using tools and scripts to perform tasks for us. Many aspects of the deployment process can be automated, from version control to testing and finally to deployment. Altogether, the automation of the deployment process creates the **Continuous Integration / Continuous Delivery (CI/CD) Pipeline**. This pipeline also encompasses **Continuous Testing** and **Continuous Deployment**.

In this lesson, we will take a look at each of the components of the CI/CD Pipeline and how it relates to the overall culture of DevOps. We will cover:

* Bottlenecks of Manual Deployment
* Automation in DevOps
* Continuous Testing
* Continuous Integration
* Continuous Delivery
* Continuous Deployment
* The Complete CI/CD Pipeline

Let's get started!


Introduction

Continuous testing, integration, delivery, and deployment make up the CI/CD pipeline.

In this lesson, we learned about the problems which can occur when using manual deployment processes with large team projects. Although manual processes may be fine for small projects, customers in the modern age require teams to move quickly and reliably.
 
The CI/CD pipeline provides this consistent and agile solution. It automates every part of the deployment process using:
* **Continuous Integration** --> Merging small changes and triggering builds and tests
* **Continuous Delivery** --> Preparing the application for deployment using intermediate environments and more tests
* **Continuous Deployment** --> Moving the application to production with final tests
 
Thanks to the practice of **Continuous Testing**, tests are automatically executed in every part of the CI/CD pipeline. Due to the high frequency of tests, we can "shift left" and catch bugs early on.
 
The entire CI/CD pipeline is the direct implementation of DevOps culture concepts. Each part of DevOps culture (systems-level thinking, continuous learning and experimentation, feedback loops) is put into practice by CI/CD. When we put it all together, implementing a CI/CD pipeline reduces human error, speeds up project development, and improves quality.
 
The next time manual processes are getting tiresome, or problems in your deployment process start occurring, consider using a CI/CD pipeline!


Review

Many aspects of releasing software can be automated including testing, infrastructure configuration, deployment, and monitoring.

Automating processes leads to many benefits when compared to traditional manual processes. Automated systems are:
 
* Faster — automated processes can perform operations much faster than people.
* Less error-prone — automation is able to perform a task more consistently than a person.
* Cheaper — workers don’t have to be paid to do these repetitive workflows.

Automation can be incorporated into every part of the deployment pipeline. For example, automated tools can be set up to monitor repositories, detect new merges, and trigger the new version of the application to be built and tested. 

Some other ways to use automation in the deployment pipeline include:
 
* Configuring servers
* Moving the application through testing and staging servers
* Executing tests
* Deploying to production
* Monitoring the application
 
Over the next few exercises, we will see which parts of the CI/CD pipeline are responsible for automating each of these processes. First, we'll look at **Continuous Testing**.

Automation in DevOps

Continuous deployment automates the release of new code to users.

The application is finally ready for production, but who is going to approve it and move it onto the production server? Traditionally, an approval process is controlled by the deployment team. The team would ensure that the production server is ready, all tests from continuous delivery have passed, and the feature meets the business requirements. Afterwards, the application would be manually deployed onto the production environment. 

This process typically requires entire features to be completed before deploying to production, resulting in slower release cycles. For many businesses, this is desirable. For others businesses, releasing new updates on a faster and more regular schedule may be preferred. To achieve, this, **Continuous Deployment** is the solution.

**Continuous deployment** is the automatic process of deploying a project to the production server after it has been tested in testing and staging environments.

Automated deployments might look like this:
* If previous stages were successful, then the deployment is approved automatically. 
* An automated system deploys the application onto the production environment. 
* Final tests, feedback from users, and monitoring tools identify any bugs in production.
* Developers can quickly react and push bug fixes.
* The automated pipeline is triggered. This results in another deployment which updates the live application.

Once automated deployment is in place, users are always able to access the most up-to-date version of the application. Keep in mind that this may or may not be the desired effect depending on the business. Some businesses may automate continuous testing, integration, and delivery while choosing to maintain manual deployments.


Continuous Deployment

Small changes trigger automated testing before merging into the main branch.

Continuous integration is a practice that consists of two main components:
 
* Merging source code changes on a frequent basis
* Building and testing the changes in an automated process
 
The main focus of continuous integration is frequency. By merging, testing, and building rapidly, we can prevent a lot of problems that could occur with many developers working on a single piece of software. Let's take a closer look at how.
 
#### No more long-lived branches
 
One of Anita’s problems was that she used large branches which she worked on for a long time. As other team members start contributing to the main branch, her branch will start to fall behind.

By switching to continuous integration practices, Anita now merges smaller changes more frequently. This follows the branching strategy of **trunk-based development** rather than **feature branch development**. Here’s the difference:
 
* Feature branch development — Merging in long-lived branches containing entire new features
 
* Trunk-based development — Merging in small changes frequently into the main branch (called the trunk)
 
Take a look at this diagram showing how each branching strategy performs differently.

> Hint: You can expand this "Learn" panel to get a better look

![Branching Strategies](https://static-assets.codecademy.com/Courses/DevOps/fundamentals/ci-trunk-vs-feature-branch.svg)

#### Less merge conflicts

Rapidly merging smaller changes means that there is less of a risk for merge conflicts. If any are found, they can be addressed quickly and require fewer changes due to the small size.
 
#### Frequent tests
 
Each new change automatically triggers building the application in an integration environment (sometimes referred to as a "CI server"). Through continuous testing, tests are executed immediately as changes are introduced. As a result, bugs are caught early on in the process.


Continuous Integration

Long-lived feature branches, infrequent testing, and manual infrastructure management can lead to many merge conflicts, missed bugs, and failed deployments.

Although manual deployment isn’t always a bad thing, it can cause a lot of problems when trying to work on a large project with many developers. Let's take a look at some common bottlenecks found within the deployment process.

For each of these bottlenecks, think about what issues can occur when scaled up to a large project. Select the dropdowns to see the answers.
 
#### Version Control Management
 
When Anita was working by herself, she used to create a branch and code a new feature. Merging wasn't a concern since she was the only one working on the project. As a result, she typically waited until an entire feature was complete before merging into the main branch.

<details>
<summary>See the issue!</summary>

> This can cause some problems on a large scale if developers are working on long-lived branches containing new features. Once the developers try to merge their code, it can cause a lot of merge conflicts and introduce new bugs. 
</details>

#### Testing
 
Anita used to run tests when she felt it was necessary. She usually ran tests before merging a large feature branch or if she found a problem herself.
 
<details>
<summary>See the issue!</summary>

> It is usually not enough to only run a test before merging a branch. Testing infrequently can lead to bugs being overlooked. Additionally, testing issues can compound once merge conflicts begin to occur between developers. 
</details>

#### Infrastructure and Environments
 
Anita previously set up the servers and the environments herself. Before deploying her code on her staging or production server, she would configure the environment variables manually. A deployment would sometimes fail due to human error, but it wasn’t such a problem when she was working by herself. Now, the team relies on the servers working properly and can’t afford any simple mistakes.
 
<details>
<summary>See the issue!</summary>

> Differences between various deployment environments can be tricky to manage. Setting up environments manually makes room for human errors that can cause the server to go down and delay production. 
</details>

Bottlenecks of Manual Deployment

Continuous delivery prepares code for production by running tests in intermediate environments.

Building and testing an application in testing and staging environments ensures that the code is compatible with production infrastructure.  When done manually, subtle differences between environments can make this a complicated process that is prone to human error. **Continuous delivery** addresses this issue through automation. 

Continuous delivery is the automated process of preparing new versions of an application to be deployed into the production environment. Picking up where continuous integration left off, continuous delivery automates:
* Environment configuration through containerization and infrastructure as code
* Deployment to intermediary environments (testing, staging, etc…)
* Further testing (acceptance and end-to-end tests)

Sometimes, an application that runs well in the development or testing environment will crash if deployed to production. DevOps practices, such as containerization and Infrastructure as Code (IaC) can be incorporated into continuous delivery to resolve these issues. 

Through continuous delivery, developers can be confident that the application has been thoroughly tested and is ready to be deployed at any time.
