Accelerate the Software Development Lifecycle with Jenkins

1.Setup the Github Repository 

To begin with, you will need to create an account on Github. (You can skip the registration part if you already have an account). Please ensure that you push the application's source code to a branch called "master". Once the code is in master, you can create another branch (from master) for your tests called "develop".

1. To create the Github account, head to the sign up page and fill out the form to sign up for a new account. (Only applicable if you don't have an account already. If you have one, please use it).

2. Once logged in, create a new repo at https://github.com/new. Ensure that you provide a meaningful name and then create it.

3. To access this repo you will need to generate an access token. Go to https://github.com/settings/tokens. Select "repo" permissions only and create it. Ensure that you save your Github token since it will be used in the future.

4. Clone your Github repository and configure how your name will be displayed in every commit.

git clone https://<username>:<token>@github.com/<your-github-username>/<your-repo-name>.git ~/code/my-repo
cd ~/code/my-repo
git config --global user.email "<my-email-id>@example.com"
git config --global user.name "<your-name> <last-name>"

Please ensure that you replace the placeholders surrounded by <> with real values.

5. Locate the main.go file (attached as an asset) and ensure that you create a new file with the same name and the same content. Once copied, it's time to create the "master" and "develop" branches.

cd ~/code/my-repo
nano main.go # Copy the content of the main.go asset. Then, save it.
 
git add main.go && git commit -m "First commit" && git push -u origin master # Creates master branch
git checkout -b develop && git push -u origin develop # Creates develop branch

6. Ensure that you can see both branches in your Github repo.

2.Create a MultiBranch Pipeline and Integrate it with Github

The next step is to create Multibranch Pipeline in Jenkins. The Pipeline should read the steps from a Jenkinsfile hosted in your Github repo.

Something to keep in mind: Due to security policies in the company, we are not allowed to configure Github webhooks. Therefore, Jenkins polling should be used instead.

1. Log in to your Jenkins instance and create a new pipeline by clicking on "New Item". Ensure that you provide a meaningful name and that the type is "Multibranch Pipeline". Once done, click ok.

2. To integrate this Pipeline with Jenkins, we need to configure the GitHub token that you created for authentication. Locate "Branch Sources", then select "Github" as your source. You will need to create a new "Credential" to be able to authenticate against your Github account.

3. Add your Github username and token as well as your repo URL to complete the Github integration. Jenkins offers a "Validation" button to double-check that everything is working as expected.

4. Navigate to "Behaviors" and ensure that you only have one behavior. This "behavior" should be "Discover Branches/Strategy/All Branches".

5. To conclude with the Pipeline configuration, head to the "Build Configuration" section and ensure that the "Script Path" is "Jenkinsfile". Also, ensure that you set the scanning interval to 3 minutes. Lastly, save the changes.

3.Install and Configure Golang as a jenkins Plugin

Before building the application, we need to have an installation of "Golang" totally functional. Please go on and download/configure Golang in Jenkins to be able to build and test the application.

1. Go to the Jenkins plugin manager at https://<your-jenkins-url>/pluginManager. Locate and install the Golang Plugin. You should see green ticks at the end of the installation to ensure that everything worked as expected.

2. Head to the Jenkins Global Tool Configuration at  https://<your-jenkins-url>/configureTools  to tweak the Go plugin installation. First off, locate the "Go" section, add a new "Go Installation" and delete the default installer.

3. Add a new installer of type "Run Shell Command". The script that will be run by the installer is attached as an asset with the name "go-plugin.txt".  Ensure that the "Tool Home" is set to "./go".

4.Build And Test the Go Applications with Jenkins

Please go ahead and create a Jenkinsfile that will build and test the application. The build.sh script expects an environment variable called "ENV" that describes the target environment depending on the branch.

Remember: The develop branch points to dev and master points to prod. The scripts that you will use for each pipeline stage are attached as assets.

1. To start writing the Jenkinsfile, we need to use the Golang installation, matching the exact name that we provided when the plugin was first configured.

NOTE: ... Simply means that there is more text, but it has been cut for readability.

pipeline {
    agent any
    tools { go 'go-1.19' } // Comes from the jenkins global config
    ...
}

2. To specify the target environment, we need to set the ENV variable pointing to the correct destination based on the branch name.

pipeline {
    ...
    environment {
        ENV = "${env.BRANCH_NAME == 'master' ? 'PROD' : 'DEV'}"
    }
    ...
}

3. To create the build and test stages, we have to add a parent section called "stages". Within this section, we start creating child sections named "stage".

pipeline {
    ...
    stages {
        stage('Build') {
            steps {
                sh 'bash scripts/build.sh' // Run the build.sh asset
            }
        }
        stage('Test') {
            steps {
                sh 'bash scripts/test.sh' // Run the test.sh asset
            }
        }
    }
}

4. To ensure that this Jenkinsfile works, we have to create new files with the content of the "build.sh" and "test.sh" assets to a new "scripts" folder, as mentioned in the Jenkinsfile.

mkdir -p ~/code/my-repo/scripts # Create the scripts directory
cd ~/code/my-repo/scripts # cd into the script directory
 
# Create the scripts that jenkins will run
 
nano build.sh # Copy the content of the build.sh asset, then save the file.
nano test.sh # Copy the content of the test.sh asset, then save the file.

5. Ensure that the folder structure in your workspace looks like this:

6. Push the Jenkinsfile and the scripts directory to Github.

cd ~/code/my-repo
git checkout develop
git status
git add Jenkinsfile scripts
git commit -m "Adds build and test pipeline steps"
git push origin develop

7. The pipeline execution should start automatically in under 3 minutes. Ensure that your Pipeline succeeds.

5.Deploy the Application to the Dev Environment

In this task, you will have to write the deployment steps to deploy the application automatically after a commit occurs to the develop or master branch. The deployment script MUST be executed exactly as follows:

export JENKINS_NODE_COOKIE=do_not_kill ; bash scripts/deploy.sh

Remember,  develop deploys to the dev environment. Once the deployment is completed, you can test if the app is deployed by running the below command:

curl localhost:4040 # Returns the application status in the DEV environment

NOTE: DO NOT modify the content of the deploy.sh file under any circumstances.

1. You can start with a quick test to ensure that there are no services running in your workspace. To do that, you can run the commands mentioned in the task description:

labsuser@udemy-labs:~$ curl localhost:4040 # Checks dev
curl: (7) Failed to connect to localhost port 4040: Connection refused
 
labsuser@udemy-labs:~$ curl localhost:5050 # Checks prod
curl: (7) Failed to connect to localhost port 5050: Connection refused

2. Add the deployment step into your existing Jenkinsfile. It should be a new stage that will run the script as mentioned in the task.

pipeline {
    environment {
        ...
        BRANCH = "${env.BRANCH_NAME}" // Needed by the deployment script
    }
    stages {
        ...
        stage('Deploy') {
            when {
                anyOf {
                    branch 'master';
                    branch 'develop'
                }
            }
            steps {
                sh 'export JENKINS_NODE_COOKIE=do_not_kill ; bash scripts/deploy.sh'
            }
        }
    }
}
 

3. Ensure that you copy the deploy.sh script to the "scripts" directory. Once done, you can push the changes to Github.

cd ~/code/my-repo
git status
git add Jenkinsfile scripts
git commit -m "Adds deployment step"
git push origin develop

4. Ensure that you can see the file on Github. You should also ensure that the Pipeline execution gets triggered in under 3 mins and it should be successful.

5. Double check that the application was deployed into your workspace after the pipeline execution.

labsuser@udemy-labs:~/code/my-repo$ curl localhost:4040
# Expected output
{"description":"You're viewing the DEV api, deployed from branch develop","environment":"DEV","time":"2022-08-14T10:45:35.739457583Z"}

6.Deploy the Application to the Production 

In this task, you will deploy to production, but first, you should test the changes in dev. Please modify the main.go and change the description on line #47 from  "You're viewing the %v api, deployed from branch %v"  to "You're viewing the AMAZING %v api, deployed from branch %v" .

Ensure that you push these changes to develop, then verify that the new changes are deployed. If everything goes well, repeat the process by pushing the changes to master to trigger the production deployment.

Note: To test the app in the production environment, use curl localhost:5050.

1. Modify the main.go file with the new changes.

// main.go
46    ...
47        Description: fmt.Sprintf("You're viewing the AMAZING %v api, deployed from branch %v", env, branch),
48        ...

2. To re-trigger the dev pipeline, all you need to do is push the changes to develop.

cd ~/code/my-repo
git status
git add main.go
git commit -m "Changes API description in Dev"
git push origin develop

3. Wait for the pipeline to get triggered. Once the execution is successfully completed, ensure that you can see the new changes deployed in Dev.

labsuser@udemy-labs:~/code/my-repo$ curl localhost:4040
# Expected output
{"description":"You're viewing the AMAZING DEV api, deployed from branch develop","environment":"DEV","time":"2022-08-14T10:55:20.063465255Z"}

4. After ensuring that the changes are deployed to the dev environment, it's time to deploy to production. To do that, merge the develop branch into the master branch.

cd ~/code/my-repo
git checkout master # Switches to the master branch
git merge develop # Merges develop into master
git push origin master # Pushes the changes to master 

5. Ensure that the pipeline successfully completes for master, then check that the application is deployed in production.

1. labsuser@udemy-labs:~/code/my-repo$ curl localhost:5050
2. # Expected output
3. {"description":"You're viewing the AMAZING PROD api, deployed from branch master","environment":"PROD","time":"2022-08-14T11:05:52.328821689Z"}
4. 7.Implement Stack Notifications

5. Now that we can build, test, and deploy, we have to put notifications in place so that the development team can get insights about the release process. Please go ahead and write steps to trigger Slack notifications at the beginning and at the end of the Pipeline.

   1. Create a Slack account at https://slack.com/get-started#/createnew. (If you already have one, feel free to skip this step).

   2. Once you've signed up, create a Slack workspace at https://slack.com/get-started#/landing.

   3. Add a Jenkins integration to your new Slack workspace. Search for "apps" in the main search bar, find "Jenkins" then click on "Add to Slack".

   

   

   

   

   4. Once you click on "Add Jenkins CI Integration", a new page will open with some details and tokens (as shown below) that you will use in the next steps. Please keep this at hand.

   

   5. Head to the Jenkins plugin manager at https://<your-jenkins-url>/pluginManager and install the Slack plugin. Ensure that you search for it in the "Available" tab, and that you click on "Install without restart".

   

   

   6. Go to the Jenkins Configuration page at https://<your-jenkins-url>/configure and find the "Slack" section. You need to provide your Slack workspace, as well as the token that was generated for you in step 3. Finally, click on the "Test Connection" button to ensure that it works.

   

   

   

   6. Now that Jenkins is configured with Slack, write the steps to send notifications at the beginning and at the end of the build.

   pipeline {
       ...
       stages {
           stage('Send initial notification') {
               when {
                   anyOf {
                       branch 'master';
                       branch 'develop'
                   }
               }
               steps {
                   slackSend channel: '#general',
                             message: "Build for job ${env.JOB_NAME} has started - (<${env.BUILD_URL}|Open>)"
               }
           }
           ...
       }
       post {
           always {
               slackSend channel: '#general',
                         color: "${currentBuild.currentResult == 'SUCCESS' ? 'good' : 'danger'}",
                         message: "Build for job ${env.JOB_NAME} finished with status ${currentBuild.currentResult} - (<${env.BUILD_URL}|Open>)"
           }
       }
   }

   7. As always, the changes have to be pushed to develop first for the pipeline to run the new notification steps.

   cd ~/code/my-repo
   git checkout develop # Switch to develop
   git status
   git add Jenkinsfile
   git commit -m "Adds slack notifications"
   git push origin develop

   8. Ensure that the pipeline gets triggered automatically in less than 3 minutes as well as that the execution is successful. Also, double-check that notifications are working as expected.

   

   

7.8

8

7

8.Apply DevsecOps to Your Pipeline

In this task, you need to implement DevSecOps continuous checks in the Jenkinsfile to ensure that secrets are not committed to Github.

The secret scanning stage should happen after the first notification gets triggered and before the application build starts.

1. Add a new stage to your pipeline to scan for secrets. This stage will use "gitleaks" for the scan. Since gitleaks is not part of Jenkins, you need to ensure that you download the binary before attempting to run it.

pipeline {
    ...
    stages {
        stage('Send initial notification') {
            ...
        }
        stage('Scan for secrets') {
            steps {
                sh '''
                    curl -LO https://github.com/zricethezav/gitleaks/releases/download/v8.9.0/gitleaks_8.9.0_linux_x64.tar.gz
                    tar -xzf gitleaks_8.9.0_linux_x64.tar.gz
                    ./gitleaks protect -v // Scan for commonly leaked secrets
                    rm -rf gitleaks*
                '''
            }
        }
        ...
    }
    ...
}

2. Push the Jenkinfile changes to the develop branch to trigger the pipeline and start scanning for secrets.

cd ~/code/my-repo
git checkout develop # Ensure that you are on develop
git status
git add Jenkinsfile
git commit -m "Adds gitleaks step"
git push origin develop

3. Wait for the pipeline to trigger in under 3 minutes. Once the trigger occurs, you need to ensure that you can see the new stage in the right place (after the build notifications). You should also ensure that the pipeline execution is successful, which means that no secrets were found.