ECS-C1: Continuous Integration with Docker and Jenkins

Continuous Integration is about building your apps on every change and on a regular schedule. It used to mean maintaining and managing build servers and complex build scripts - where the build for production was nothing like the developer workflow. Not any more…

In this episode we’ll combine multi-stage Dockerfiles with a build infrastructure running in containers, to make for a simple and portable approach to CI. We’ll use Jenkins and run all the components locally using Docker.

Here it is on YouTube - ECS-C1: Continuous Integration with Docker and Jenkins

Pre-reqs

You can use Docker Desktop on Windows 10 or Mac, or Docker Community Edition on Linux to follow the demos.

Demo 1 - Spinning up the build infrastructure

We’ll run a full build stack - Git server, automation server and container registry.

The spec is in infrastructure/docker-compose.yml, using project team images from Docker Hub, except Jenkins which is a custom build in images/jenkins/Dockerfile.

Spin up the infrastructure:

docker-compose -f ./infrastructure/docker-compose.yml up -d

Check the registry server:

curl http://registry.local:5000/v2/_catalog

And setup Gogs:

http://localhost:3000

Finish the configuration and create a user and repo:

Select Sqlite DB & install with defaults
Create user ecs
Create repo ecs - Gogs has issues & wiki like GitHub

Add the local Git server as a remote:

git remote add local http://localhost:3000/ecs/ecs.git

git push -u local

See Diamol ep. 10 for automation options

Demo 2 - Building Docker images with Jenkins

We have a distributed app to build in the src directory:

access-log/Dockerfile - a Node.js REST API
image-of-the-day/Dockerfile - a JAVA REST API
image-gallery/Dockerfile - a Go web server
docker-compose.yml - the Compose file to build and run the app

Build arguments get used as labels to provide an audit trail from the final image.

Build the app locally:

docker-compose -f ./src/docker-compose.yml build

docker image inspect sixeyed/image-of-the-day:ecs-c1

Default label values for the local build

Check Jenkins is up and running:

docker logs infrastructure_jenkins_1

It’s provisioned with two scripts - admin.groovy sets the admin user credentials and install-plugins.groovy installs the Pipeline plugin.

Browse to Jenkins at http://localhost:8080

Log in with creds ecs/ecs
Create a new item - select Pipeline Job
Set the schedule @daily
And poll scm * * * * *

Configure the source repo to use Gogs:

git - http://gogs:3000/ecs/ecs.git
path - episodes/ecs-c1/src/Jenkinsfile

Build now and check the logs

Jenkins is using this Jenkinsfile. It runs Docker commands using the local Docker Engine - from the bind mounted socket.

Check the images it builds are in the local image cache:

docker image ls registry.local:5000/*/*

docker image inspect registry.local:5000/ecs/image-gallery:ecs-c1

Demo 3 - Running tests in containers

Your CI process can run end-to-end tests in containers and push the app images to a local registry.

Then it would go on to do security scanning and signing before pushing to a production repo for release.

There’s a simple test container in e2e-test/Dockerfile - it connects to the image API container using the container name.

Change the Jenkins job at http://localhost:8080/job/ecs-c1/configure

Set path - episodes/ecs-c1/src/Jenkinsfile.v2

Build now and check logs

The new pipeline uses the job in Jenkinsfile.v2.

Verify the new build:

docker image inspect registry.local:5000/ecs/image-gallery:ecs-c1

And check the images in the local registry:

curl http://registry.local:5000/v2/_catalog

curl http://registry.local:5000/v2/ecs/access-log/tags/list

Coming next

ECS-C2: CI/CD with Docker and GitHub Actions

Elton's Container Show