How to create Docker image and upload to AWS ECR using Git Lab CI/CD pipeline

Technical Discussion
#1

Creating Docker image and pushing to ECR using GitLab CI/CD

Pre requisite

  1. Your GitLab Project should have a Docker file at root level
  2. Create a .gitlab-ci.yml file at root level in the repository, this file contains the pipeline.

Variables:

These are placeholders for values used throughout the pipeline.

  • DOCKER_HOST: The location where Docker is running.
  • MANUAL_TRIGGER: Should be set to false in variables by default, set value to true while running manually
  • APP_NAME: Preferred name for your docker image.
  • DOCKER_IMAGE_NAME: The name of your Docker image.
  • GIT_USERNAME and GIT_PASSWORD: Your credentials for accessing a Git repository.
  • Moqui_*: Configuration details for Moqui, including database information.
  • ELASTICSEARCH_HOST: The address where Elasticsearch is running.
  • Ofbiz_TRANSACTION_DB_*: Details related to the OFBiz database.
  • TIME_ZONE: Your local time zone.
  • REPOSITORY_URL: The web address of your Docker image repository.

Stages:

1. Build Stage:

  • Job Name: build_job

  • What it does: Builds a Docker image, creating a package with necessary files and settings.

    • How it works:

      • Connects to Amazon ECR Public to store the Docker image securely.
      • Builds the Docker image with specific configurations.
      • Pushes the image to the specified repository.
      • Cleans up by removing the locally built Docker image.

    • When it runs: Triggered when a tag starting with ‘v’ is created on a protected branch.

2. Release Stage:

  • Job Name: release_job

  • What it does: Creates a release, a snapshot of your software at a specific point.

    • How it works:

      • Outputs a message saying it’s running.
      • Creates a GitLab release with a tag and description based on the commit tag.

    • When it runs: Triggered when a tag starting with ‘v’ is created on a protected branch.

Additional Notes:

  • retry: 2: If something goes wrong, the pipeline will try to run the failed job up to two times.

How to Use:

  1. Set Up Variables:

    • Fill in values for variables with your specific details.

  2. Create a Tag:

    • Create a tag starting with ‘v’ on a protected branch.

  3. Watch the Pipeline:

    • Go to your GitLab project, and you’ll see a “CI/CD” section.
    • Watch as the pipeline runs through the stages.

  4. Check the Results:

    • If set up correctly, you should have a new Docker image in your specified repository and a GitLab release created.

Alternate:

To run manually

  1. Set $MANUAL_TRIGGER to true
  2. Set value of $CI_COMMIT_TAG

Remember to keep sensitive information (like passwords) secure by using GitLab’s environment variables within project settings.

stages:
  - build
  - release

variables:
  DOCKER_HOST: $DOCKER_HOST
  #These variables can specified when running the job manually, the values will be overridden 
  DOCKER_IMAGE_NAME: $DOCKER_IMAGE_NAME
  GIT_USERNAME: $GIT_USERNAME
  GIT_PASSWORD: $GIT_PASSWORD
  Moqui_HOST: $Moqui_HOST
  Moqui_DB_HOST: $Moqui_DB_HOST
  Moqui_DB_USER: $Moqui_DB_USER
  Moqui_DB_PASSWORD: $Moqui_DB_PASSWORD
  Moqui_DB_NAME: $Moqui_DB_NAME
  Moqui_analytical_DB_Name: $Moqui_analytical_DB_Name
  Moqui_configuration_DB_Name: $Moqui_configuration_DB_Name
  ELASTICSEARCH_HOST: $ELASTICSEARCH_HOST
  Ofbiz_TRANSACTION_DB_HOST: $Ofbiz_TRANSACTION_DB_HOST
  Ofbiz_TRANSACTION_DB_USER: $Ofbiz_TRANSACTION_DB_USER
  Ofbiz_TRANSACTION_DB_PASSWORD: $Ofbiz_TRANSACTION_DB_PASSWORD
  Ofbiz_TRANSACTION_DB_NAME: $Ofbiz_TRANSACTION_DB_NAME
  TIME_ZONE: $TIME_ZONE
  REPOSITORY_URL: $REPOSITORY_URL
  APP_NAME: $APP_NAME


build_job:
  image:
    name: amazon/aws-cli
    entrypoint: [""]

  services:
    - docker:18.09-dind

  before_script:
    - amazon-linux-extras install docker
    - aws --version
    - docker --version

  stage: build
  
  allow_failure: false

  rules:
    - if: $CI_COMMIT_TAG =~/^v.*/   
    - if: $MANUAL_TRIGGER == "true"
      when: manual

  script:
    - echo $MANUAL_TRIGGER
    - aws ecr-public get-login-password --region us-east-1 | docker login --username AWS --password-stdin public.ecr.aws
    - echo "Building Docker image... $DOCKER_IMAGE_NAME"
    - docker build --force-rm=true --no-cache=true --build-arg TIME_ZONE=$TIME_ZONE --build-arg Ofbiz_TRANSACTION_DB_HOST=$Ofbiz_TRANSACTION_DB_HOST --build-arg Ofbiz_TRANSACTION_DB_USER=$Ofbiz_TRANSACTION_DB_USER --build-arg Ofbiz_TRANSACTION_DB_PASSWORD=$Ofbiz_TRANSACTION_DB_PASSWORD --build-arg Ofbiz_TRANSACTION_DB_NAME=$Ofbiz_TRANSACTION_DB_NAME --build-arg GIT_USERNAME=$GIT_USERNAME --build-arg GIT_PASSWORD=$GIT_PASSWORD --build-arg Moqui_HOST=$Moqui_HOST --build-arg Moqui_DB_HOST=$Moqui_DB_HOST --build-arg Moqui_DB_USER=$Moqui_DB_USER --build-arg Moqui_DB_PASSWORD=$Moqui_DB_PASSWORD --build-arg Moqui_DB_NAME=$Moqui_DB_NAME --build-arg Moqui_analytical_DB_Name=$Moqui_analytical_DB_Name --build-arg Moqui_configuration_DB_Name=$Moqui_configuration_DB_Name --build-arg ELASTICSEARCH_HOST=$ELASTICSEARCH_HOST -t $REPOSITORY_URL:$APP_NAME-$CI_COMMIT_TAG .
    
    - echo "Pushing Docker Image . . . "
    - aws ecr-public get-login-password --region us-east-1 | docker login --username AWS --password-stdin public.ecr.aws  
    - docker push $REPOSITORY_URL:$APP_NAME-$CI_COMMIT_TAG
    - docker rmi $REPOSITORY_URL:$APP_NAME-$CI_COMMIT_TAG
  retry: 2

release_job:
  stage: release
  image: registry.gitlab.com/gitlab-org/release-cli:latest
  rules:
    - if: $CI_COMMIT_TAG =~/^v.*/                    
  script:
    - echo "running release_job"
  release:                               
    tag_name: '$CI_COMMIT_TAG'
    description: '$CI_COMMIT_TAG'
  when: always
  retry: 2

Feel free to ask for clarification or help with specific steps!