Run GitLab on a K8s-based private cloud
After Microsoft announced the acquisition of GitHub, many developers raised concerns on social media about Microsoft’s history of unsuccessfully running the acquired businesses like Skype, Nokia’s handset business, Navision, and other 150 companies (you probably haven’t noticed) they have swallowed up over the years.
Other than keeping the developer’s life-support plugged, one of the biggest concern is that MS will use its power on GitHub repositories to analyze trends among software development in order to launch competing products. Fears that GitHub privacy may be in jeopardy have already led many developers to jump off the ship or consider alternatives. GitLab’s publicly available status graphs show spikes of 70x increase in imported repositories (average 100 vs 7.5K), a validation of increased user apprehension.
Whether you are considering moving your code out of GitHub or not, here is one of the fastest way to get your private repository with GitLab up and running on your Kubernetes environment – Let’s “Make DevOps lifecycle private again” (c)
Currently, the easiest and recommended way to install GitLab on Kubernetes is using the Gitlab-Omnibus Helm charts.
Gitlab-Omnibus deploys every feature a small deployment requires including the Container Registry, Ingress Controller (NGINX), Mattermost, and Runner.
Prerequisites
Minimum requirements for a multi-node cluster:
Hardware
- Boot node: 1x 1+ core(s) >= 2.4 GHz CPU, 4GB RAM, >=100 GB disk space
- Master node: 1 or 3x 2+ cores >= 2.4 GHz CPU, 4+GB RAM, >=151 GB disk space
- Proxy node: 1 or 3x 2+ cores >= 2.4 GHz CPU, 4GB RAM, >=40 GB disk space
- Worker node: 3x 2+ cores >= 2.4 GHz CPU, 4+GB RAM, >=100 GB disk space
Since I’m not planning to run anything heavy, I’ll be using 3 nodes, and install Master, Proxy, and Workers on all 3 nodes.
Software
- Ubuntu 16.04 LTS (RHEL 7.x is also supported)
- Docker 1.12 to 17.03
- Kubernetes 1.7+ Cluster (You can use IBM Cloud Private 2.1.0.2 or Red Hat OpenShift Origin)
- kubectl
- Helm client
- A GitLab Omnibus Pod, including Mattermost, Container Registry, and Prometheus
- An auto-scaling GitLab Runner using the Kubernetes executor
- Redis
- PostgreSQL
- NGINX Ingress
- OpenEBS persistent volumes for Data, Registry, Postgres, and Redis
Kubernetes instructions described below using Helm are generic and should work on all other platforms.
Installing GitLab and OpenEBS using the Helm Chart
GitLab depends on stateful applications like Redis and PostgeSQL and requires persistent volumes for its data and the registry. I will simplify the storage provisioning using OpenEBS.
Install OpenEBS using the chart.
Optional: If you would like to customize your OpenEBS installation, you can also use a copy of value.yaml file from the OpenEBS chart and modify parameters listed here.
Add the predefined storage classes.
There are many ways to enable OpenEBS to be used by GitLab. The fastest one is to make one of the OpenEBS storage classes a default StorageClass:
List available OpenEBS storage classes in your cluster.
NAME PROVISIONER AGE
openebs-cassandra openebs.io/provisioner-iscsi 18d
openebs-es-data-sc openebs.io/provisioner-iscsi 18d
openebs-jupyter openebs.io/provisioner-iscsi 18d
openebs-kafka openebs.io/provisioner-iscsi 18d
openebs-mongodb openebs.io/provisioner-iscsi 18d
openebs-percona openebs.io/provisioner-iscsi 18d
openebs-redis openebs.io/provisioner-iscsi 18d
openebs-standalone openebs.io/provisioner-iscsi 18d
openebs-standard openebs.io/provisioner-iscsi 18d
openebs-zk openebs.io/provisioner-iscsi 18d
Either create your StorageClass or pick one of the predefined classes. openebs-standard
creates 3 replicas and ideal candidate here to be used for most of the stateful workloads. Let’s mark this StorageClass as default.
Verify that your chosen StorageClass is now the default.
NAME PROVISIONER AGE
openebs-cassandra openebs.io/provisioner-iscsi 18d
openebs-es-data-sc openebs.io/provisioner-iscsi 18d
openebs-jupyter openebs.io/provisioner-iscsi 18d
openebs-kafka openebs.io/provisioner-iscsi 18d
openebs-mongodb openebs.io/provisioner-iscsi 18d
openebs-percona openebs.io/provisioner-iscsi 18d
openebs-redis openebs.io/provisioner-iscsi 18d
openebs-standalone openebs.io/provisioner-iscsi 18d
openebs-standard (default) openebs.io/provisioner-iscsi 18d
openebs-zk openebs.io/provisioner-iscsi 18d
Now, we can install the GitLab Omnibus chart. It is recommended to save your configuration options in a values.yaml file for future use.
Edit the values.yaml
file and at least add the externalUrl field, otherwise, you might end up with a non-functioning release.
Here is how my values.yaml file looks like after my changes.
externalUrl: http://containerized.me/
serviceType: LoadBalancer
ingress:
annotations:
enabled: false
tls:
url: gitlab.cluster.local
sshPort: 22
httpPort: 80
httpsPort: 443
livenessPort: http
readinessPort: http
resources:
requests:
memory: 1Gi
cpu: 500m
limits:
memory: 2Gi
cpu: 1
persistence:
gitlabEtc:
enabled: true
size: 1Gi
storageClass: openebs-standard
accessMode: ReadWriteOnce
gitlabData:
enabled: true
size: 10Gi
storageClass: openebs-standard
accessMode: ReadWriteOnce
postgresql:
imageTag: "9.6"
cpu: 1000m
memory: 1Gi
postgresUser: gitlab
postgresPassword: gitlab
postgresDatabase: gitlab
persistence:
size: 10Gi
storageClass: openebs-standard
accessMode: ReadWriteOnce
redis:
redisPassword: "gitlab"
resources:
requests:
memory: 1Gi
persistence:
size: 10Gi
storageClass: openebs-standard
accessMode: ReadWriteOnce
Now, install the chart.
List the pods and confirm that all pods are ready and running.
NAME READY STATUS RESTARTS AGE
kubectl get pods
NAME READY STATUS RESTARTS AGE
gitlab-test-gitlab-ce-dd69cdf4b-69vmb 1/1 Running 0 11m
gitlab-test-postgresql-75bf9b667d-lwj2b 1/1 Running 0 11m
gitlab-test-redis-998998b59-hzztj 1/1 Running 0 11m
openebs-gitlab-test-apiserver-68fc4488fd-jf8gz 1/1 Running 0 1h
openebs-gitlab-test-provisioner-7dfdf646d8-9wpmg 1/1 Running 0 1h
pvc-cb0fc1b2-6904-11e8-9f57-06a0a9acf800-ctrl-74d4b59c9f-bjtg2 2/2 Running 0 11m
pvc-cb0fc1b2-6904-11e8-9f57-06a0a9acf800-rep-64f56667d-6ds26 1/1 Running 0 11m
pvc-cb0fc1b2-6904-11e8-9f57-06a0a9acf800-rep-64f56667d-99mbh 1/1 Running 0 11m
pvc-cb0fc1b2-6904-11e8-9f57-06a0a9acf800-rep-64f56667d-d8d4z 1/1 Running 0 11m
pvc-cb1064ee-6904-11e8-9f57-06a0a9acf800-ctrl-bd7cff65f-ph8dr 2/2 Running 0 11m
pvc-cb1064ee-6904-11e8-9f57-06a0a9acf800-rep-595dd9c997-2lm4x 1/1 Running 0 11m
pvc-cb1064ee-6904-11e8-9f57-06a0a9acf800-rep-595dd9c997-jldjs 1/1 Running 0 11m
pvc-cb1064ee-6904-11e8-9f57-06a0a9acf800-rep-595dd9c997-kzlrc 1/1 Running 0 11m
pvc-cb111261-6904-11e8-9f57-06a0a9acf800-ctrl-668f5988c5-hv8vb 2/2 Running 0 11m
pvc-cb111261-6904-11e8-9f57-06a0a9acf800-rep-74974f6644-hsn49 1/1 Running 0 11m
pvc-cb111261-6904-11e8-9f57-06a0a9acf800-rep-74974f6644-lj64g 1/1 Running 0 11m
pvc-cb111261-6904-11e8-9f57-06a0a9acf800-rep-74974f6644-z6kfd 1/1 Running 0 11m
pvc-cb11a791-6904-11e8-9f57-06a0a9acf800-ctrl-585cf7c97d-58pnq 2/2 Running 0 11m
pvc-cb11a791-6904-11e8-9f57-06a0a9acf800-rep-79d658d94c-5bzn6 1/1 Running 0 11m
pvc-cb11a791-6904-11e8-9f57-06a0a9acf800-rep-79d658d94c-9dz5f 1/1 Running 0 11m
pvc-cb11a791-6904-11e8-9f57-06a0a9acf800-rep-79d658d94c-snkfb 1/1 Running 0 11m
Get the list of persistent volumes.
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pvc-cb0fc1b2-6904-11e8-9f57-06a0a9acf800 10Gi RWO Delete Bound default/gitlab-test-postgresql openebs-standard 17m
pvc-cb1064ee-6904-11e8-9f57-06a0a9acf800 10Gi RWO Delete Bound default/gitlab-test-redis openebs-standard 17m
pvc-cb111261-6904-11e8-9f57-06a0a9acf800 10Gi RWO Delete Bound default/gitlab-test-gitlab-ce-data openebs-standard 17m
pvc-cb11a791-6904-11e8-9f57-06a0a9acf800 1Gi RWO Delete Bound default/gitlab-test-gitlab-ce-etc openebs-standard 17m
You can see that four persistent volumes were created (postgresql, redis, gitlab-ce-etc, gitlab-ce-data) and each volume is protected by 3 replicas.
Now you can visit the external endpoint address you have defined and start using GitLab after you set your new password.
Now, click Create a project, import your existing project from GitHub, and start using GitLab.
Also published on Medium.
Zhiyong Zhao
•3 years ago
Can you provide instructions with new omnibus charts? This chart different. https://docs.gitlab.com/ee/install/kubernetes/gitlab_omnibus.html
We use ICP (2.1.0.1) and want to use openebs. thank you!
me
•3 years ago
Hi Zhiyong, thanks for your feedback. I’ll publish a seperate blog on using ICP Dashboard to install GitLab shortly. Thanks!
KubeWeekly #137 – KubeWeekly
•3 years ago
[…] Git freedom on Kubernetes Murat Karslioglu, CloudByte […]