GitLab CI/CD pipeline: Write your first .gitlab-ci.yml file from scratch

GitLab CI/CD is a tool built into GitLab for software development through the continuous methodologies: Continuous Integration (CI), Continuous Delivery (CD), and Continuous Deployment (CD). & creating your first .gitlab-ci.yml file is a significant step towards automating your development processes using GitLab's powerful CI/CD pipeline.

This guide will walk you through how to write your first .gitlab-ci.yml file, explaining essential fields and providing detailed examples to help you get started. Before we walk you through the GitLab CI/CD pipeline & steps to write your 1st .gitlab-ci.yml file from scratch, lets discuss check,

Advantages of Using GitLab CI/CD with .gitlab-ci.yml

1. Automation of the Software Development Lifecycle (SDLC):

   • GitLab CI/CD automates the stages of your software development from building and testing code to deploying it to production, reducing the manual effort required and speeding up the entire process.

2. Improved Code Quality:

   • Continuous Integration ensures that your code is tested automatically and frequently. This means errors are detected earlier, improving the overall quality of the code.

3. Efficient Parallel Execution:

   • GitLab CI/CD can run multiple jobs concurrently, which means you can run tests for different modules or services at the same time, significantly reducing the time required for the CI/CD process.

4. Flexible and Scalable:

   • GitLab CI/CD pipelines are defined in the .gitlab-ci.yml file, which can be version controlled and shared across projects. This makes it easy to scale across multiple projects and teams.

5. Customizable and Extensible:

   • You can customize the pipeline to meet the specific needs of your project. Whether it’s defining specific stages, setting up different environments, or using various tools, GitLab’s CI/CD is highly adaptable.

6. Visibility and Insight:

   • GitLab provides detailed insights into the pipeline’s processes, which helps in monitoring the health and status of your applications. This visibility is crucial for maintaining the reliability and stability of your systems.

7. Environment and Deployment Management:

   • GitLab allows you to define and manage dynamic environments, supports canary deployments, and has capabilities for feature flags, making it easier to manage deployments and rollbacks in a controlled manner.

8. Integrated with GitLab:

   • Since GitLab CI/CD is fully integrated into the GitLab platform, it provides a unified experience without the need for additional integration or switching between different tools.

9. Security and Compliance:

   • GitLab CI/CD pipelines can include security scanning and compliance checks as part of the workflow, ensuring that every release meets your security standards and regulatory requirements.

10. Community and Support:

    • GitLab has a strong community and professional support that can provide assistance, share best practices, and offer templates for various use cases.

Now that we have some understanding of Gitlab CI/CD, let's discuss .gitlab-ci.yml file.

Understanding .gitlab-ci.yml

The .gitlab-ci.yml file is a YAML file where you configure specific instructions for GitLab CI/CD. This file is placed in the root of your repository and is used by GitLab Runner to manage your project's jobs.

Key Components of .gitlab-ci.yml

1. stages: Defines stages in the pipeline (e.g., build, test, deploy).
2. image: Specifies the Docker image to use for the jobs.
3. variables: Declares variables that are passed to jobs.
4. before_script: Commands that run before each job's script.
5. after_script: Commands that run after each job's script.
6. jobs: Defines what to execute using scripts.
7. script: Shell script commands executed by Runner.
8. only/except: Defines the policy for job execution.
9. cache: Configures caching between jobs to speed up execution.
10. artifacts: Defines list of files and directories to attach to a job on success.

Writing Your First .gitlab-ci.yml File from scratch

Here’s a step-by-step guide to creating a basic .gitlab-ci.yml file:

Step 1: Define the Image

Specify the Docker image that the Runner should use. For instance, if you’re using Python:

image: python:3.9

Step 2: Set Up Stages

Define the stages of your CI/CD pipeline. Common stages include build, test, and deploy.

stages:
  - build
  - test
  - deploy

Step 3: Create Job Definitions

Each job must have a stage and a script. Here’s an example of a job that installs dependencies:

install_dependencies:
  stage: build
  script:
    - pip install -r requirements.txt

This job falls under the build stage and runs the command to install Python packages listed in requirements.txt.

Step 4: Add Testing and Deployment

Add jobs for testing and deployment. For testing, you might run a script to execute tests:

run_tests:
  stage: test
  script:
    - python -m unittest discover

For deployment, you might have a script to deploy your application:

deploy_to_production:
  stage: deploy
  script:
    - echo "Deploying to production server"
  only:
    - main

This job deploys your code only when changes are pushed to the main branch.

Step 5: Use Variables and Caching

You can define variables to make your configuration more dynamic and use cache to speed up the pipeline execution:

variables:
  PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"

cache:
  paths:
    - .cache/pip
    - node_modules/

Step 6: Configure Artifacts

Specify artifacts to pass data between jobs or store results:

artifacts:
  paths:
    - output/
  expire_in: 1 week

Example .gitlab-ci.yml

Combining all the elements, here’s a simple .gitlab-ci.yml file for a Python project:

image: python:3.9

stages:
  - build
  - test
  - deploy

variables:
  PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"

cache:
  paths:
    - .cache/pip
    - node_modules/

install_dependencies:
  stage: build
  script:
    - pip install -r requirements.txt

run_tests:
  stage: test
  script:
    - python -m unittest discover

deploy_to_production:
  stage: deploy
  script:
    - echo "Deploying to production server"
  only:
    - main

artifacts:
  paths:
    - output/
  expire_in: 1 week

Here is another example for how a .gitlab.ci.yml for Node CI/CD pipeline.

image: node:latest

stages:
  - build
  - test
  - deploy

cache:
  paths:
    - node_modules/

install_dependencies:
  stage: build
  script:
    - npm install

run_tests:
  stage: test
  script:
    - npm test

deploy_to_production:
  stage: deploy
  script:
    - echo "Deploying to production server"
  only:
    - main

Conclusion

Writing .gitlab-ci.yml file is a foundational skill for automating your development workflow with GitLab CI/CD. By understanding and utilizing the key components of the .gitlab-ci.yml file, you can effectively configure your CI/CD pipeline to build, test, and deploy your applications seamlessly.

As you become more familiar with GitLab CI/CD, you can explore more advanced configurations and integrations to enhance your automation processes. This concludes tutorial on GitLab CI/CD pipeline: Write your 1st .gitlab-ci.yml file from scratch.