Multi-Environment Variable Management in Gitlab CI
Using Gitlab CI, Docker and Docker Compose (No Kubernetes)
The Objective
Let’s say we have simple Gitlab CI setup. Pushing to a development branch, a staging branch or a production branch triggers Gitlab CI deploy stages that Gitlab-Runners on remote hosts are registered to. No docker swarm, no Kubernetes, no clusters. Just single remote servers for each environment. Here’s an approach how to keep things simple, clear and secure.
The Setup
In our gitlab-ci.yml we would have a deploy service that looks similar to this:
For the sake of clarity we have a docker-compose.<env>.yml file for each environment (development/staging/production).
To share environment variables across our containers we want to define a env_file that keeps all our environment variables in one place.
The corresponding .env file in this case would include all environment variables being used in and across our two containers (nodejs and postgres).
As we all know, we should never publicly share any .env files containing sensitive information (like the postgres password in this case). To prevent them from being committed the repository we usually add them to our .gitignore file.
What we usually do provide instead is a .env.sample file that functions as a blueprint for our actual .env file since we leave sensitive information blank.
Gitlab CI Variables
So, where do wo store our password instead? This is where Gitlab CI Variables (Settings → CI/CD → Variables) comes in the game.
Setting a variable come with a protect and a mask flag that ensures the value never to be displayed anywhere. (Even if the branch is public accessible to other collaborators the value will only be visible for administrators!)
Now we have stored our sensitive information in a Gitlab variable and our insensitive in a .env.sample but our docker-compose.yml is still expecting a single .env.staging file containing all environments in one place! So, what do we do?!
The Script
Exactly! We write a little shell script that reads all the pre-defined variables from our provided .env.sample file, replace it’s value if provided as a environment variable via Gitlab CI Variables (otherwise we just copy it) and write it to a .env.staging file.
Finally, we run this script in the deploy job in our gitlab-ci.yml file before docker-compose.
Doing so, docker compose will find the env_file we defined in our docker-compose.staging.yml containing all the environment variables our containers need in one place while our sensitive can easily be managed and kept safe using Gitlab CI Variables!
Hope this was helpful — feel free to leave a comment.
