源碼編譯 GitLab Runner

  • 2019 年 10 月 8 日
  • 筆記

本文使用「署名 4.0 國際 (CC BY 4.0)」許可協議，歡迎轉載、或重新修改使用，但需要註明來源。 署名 4.0 國際 (CC BY 4.0)

本文作者: 蘇洋

創建時間: 2019年08月04日 統計字數: 10252字 閱讀時間: 21分鐘閱讀 本文鏈接: https://soulteary.com/2019/08/04/source-code-compilation-gitlab-runner.html

源碼編譯 GitLab Runner

從源碼安裝 GitLab 你或許聽說過，但是從源碼安裝 GitLab Runner ，或許這將是你聽到的第一篇相關部落格。

最近遇到一個問題，需要手動編譯構建 GitLab Runner，而官方文檔陳舊、命令過時，如果按照官方錯誤的指引搞下去，難免會浪費很多時間，而且得不到你想要的結果。

如果你也有類似需求，跟隨本篇文章，大概十分鐘左右就能折騰出一個屬於你自己的 GitLab Runner。

前置準備

這次的前置準備真的不多，大概就需要兩樣：

  • 一台虛擬機，我選擇的作業系統是 Ubuntu18.0464位
  • 一顆耐心

在編譯 GitLab Runner 之前，我們需要製作編譯工具，而編譯工具依賴訂製的系統環境，所以第一步就是簡單折騰一個系統環境。

準備系統環境

安裝基礎環境的第一步，無非是升級系統環境、安裝必要依賴。

這裡推薦安裝 git 依賴，而非官方文檔中註明的 git-core ，詳細原因見這篇文章。

apt update && apt upgrade -yapt install software-properties-common git mercurial unzip vim binfmt-support qemu-user-static -y

與其他軟體不同的是，這裡必須安裝 Docker ，因為後面的構建過程中會使用到 Docker （哪怕你編譯的不是 Runner 的 Docker 鏡像）。

截止文章發布，使用 Dockerversion19.03.1 並無任何異常。如果你使用的是阿里雲，可以使用下面的命令進行安裝，更詳細的內容，可以參考之前的文章。

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -add-apt-repository "deb [arch=amd64] https://mirrors.aliyun.com/docker-ce/linux/ubuntu $(lsb_release -cs) stable"apt install -y docker-ce

由於軟體是使用 Go 語言編寫，所以安裝 Go 語言運行時必不可少。但可惜的是，我們無法使用最新的 Go1.12，必須老老實實使用 2017 年推出的 1.8.7。

wget https://storage.googleapis.com/golang/go1.8.7.linux-amd64.tar.gzsudo tar -C /usr/local -xzf go*-*.tar.gzecho "export GOPATH=$HOME/Go" >> ~/.profileecho "export PATH=$PATH:$GOPATH/bin:/usr/local/go/bin" >> ~/.profilesource ~/.profile

接下來需要做的事情是獲取軟體包源碼、以及開發需要使用的依賴包。

go get gitlab.com/gitlab-org/gitlab-runnercd $GOPATH/src/gitlab.com/gitlab-org/gitlab-runner/go get -u github.com/jteeuwen/go-bindata/...

準備編譯工具

在上述命令都執行完畢，且沒有報錯的情況下，繼續執行下面的語句，就能獲得可用的編譯工具啦。

make helper-build helper-docker

如果你比較順利，將會看到類似下面的日誌：

# make helper-build helper-dockergo get github.com/mitchellh/goxgox -osarch=windows/amd64 -ldflags "-X gitlab.com/gitlab-org/gitlab-runner/common.NAME=gitlab-runner -X gitlab.com/gitlab-org/gitlab-runner/common.VERSION=12.2.0~beta.1803.g41d5c6ad -X gitlab.com/gitlab-org/gitlab-runner/common.REVISION=41d5c6ad -X gitlab.com/gitlab-org/gitlab-runner/common.BUILT=2019-08-01T16:59:49+0000 -X gitlab.com/gitlab-org/gitlab-runner/common.BRANCH=master -s -w" -output=dockerfiles/build/binaries/gitlab-runner-helper.x86_64-windows gitlab.com/gitlab-org/gitlab-runner/apps/gitlab-runner-helperNumber of parallel builds: 7-->   windows/amd64: gitlab.com/gitlab-org/gitlab-runner/apps/gitlab-runner-helpergox -osarch=linux/amd64 -ldflags "-X gitlab.com/gitlab-org/gitlab-runner/common.NAME=gitlab-runner -X gitlab.com/gitlab-org/gitlab-runner/common.VERSION=12.2.0~beta.1803.g41d5c6ad -X gitlab.com/gitlab-org/gitlab-runner/common.REVISION=41d5c6ad -X gitlab.com/gitlab-org/gitlab-runner/common.BUILT=2019-08-01T16:59:49+0000 -X gitlab.com/gitlab-org/gitlab-runner/common.BRANCH=master -s -w" -output=dockerfiles/build/binaries/gitlab-runner-helper.x86_64 gitlab.com/gitlab-org/gitlab-runner/apps/gitlab-runner-helperNumber of parallel builds: 7-->     linux/amd64: gitlab.com/gitlab-org/gitlab-runner/apps/gitlab-runner-helpergox -osarch=linux/arm -ldflags "-X gitlab.com/gitlab-org/gitlab-runner/common.NAME=gitlab-runner -X gitlab.com/gitlab-org/gitlab-runner/common.VERSION=12.2.0~beta.1803.g41d5c6ad -X gitlab.com/gitlab-org/gitlab-runner/common.REVISION=41d5c6ad -X gitlab.com/gitlab-org/gitlab-runner/common.BUILT=2019-08-01T16:59:49+0000 -X gitlab.com/gitlab-org/gitlab-runner/common.BRANCH=master -s -w" -output=dockerfiles/build/binaries/gitlab-runner-helper.arm gitlab.com/gitlab-org/gitlab-runner/apps/gitlab-runner-helperNumber of parallel builds: 7-->       linux/arm: gitlab.com/gitlab-org/gitlab-runner/apps/gitlab-runner-helperdocker build -t gitlab/gitlab-runner-helper:x86_64-41d5c6ad -f dockerfiles/build/Dockerfile.x86_64 dockerfiles/buildSending build context to Docker daemon  40.32MBStep 1/6 : FROM alpine:3.93.9: Pulling from library/alpinee7c96db7181b: Pull completeDigest: sha256:7746df395af22f04212cd25a92c1d6dbc5a06a0ca9579a229ef43008d4d1302aStatus: Downloaded newer image for alpine:3.9 ---> 055936d39205Step 2/6 : RUN apk add --no-cache bash ca-certificates git git-lfs miniperl     && ln -s miniperl /usr/bin/perl ---> Running in 93e1c5a12b93fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/main/x86_64/APKINDEX.tar.gzfetch http://dl-cdn.alpinelinux.org/alpine/v3.9/community/x86_64/APKINDEX.tar.gz(1/14) Installing ncurses-terminfo-base (6.1_p20190105-r0)(2/14) Installing ncurses-terminfo (6.1_p20190105-r0)(3/14) Installing ncurses-libs (6.1_p20190105-r0)(4/14) Installing readline (7.0.003-r1)(5/14) Installing bash (4.4.19-r1)Executing bash-4.4.19-r1.post-install(6/14) Installing ca-certificates (20190108-r0)(7/14) Installing nghttp2-libs (1.35.1-r0)(8/14) Installing libssh2 (1.8.2-r0)(9/14) Installing libcurl (7.64.0-r2)(10/14) Installing expat (2.2.7-r0)(11/14) Installing pcre2 (10.32-r1)(12/14) Installing git (2.20.1-r0)(13/14) Installing git-lfs (2.5.1-r2)Executing git-lfs-2.5.1-r2.post-installGit LFS initialized.(14/14) Installing miniperl (5.26.3-r0)Executing busybox-1.29.3-r10.triggerExecuting ca-certificates-20190108-r0.triggerOK: 42 MiB in 28 packagesRemoving intermediate container 93e1c5a12b93 ---> cf8c23304de7Step 3/6 : RUN git lfs install --skip-repo ---> Running in cdd487902490Git LFS initialized.Removing intermediate container cdd487902490 ---> 995fbdc82d24Step 4/6 : COPY ./scripts/ /usr/bin ---> d1a8aabbcd3cStep 5/6 : COPY ./binaries/gitlab-runner-helper.x86_64 /usr/bin/gitlab-runner-helper ---> fcb3bbe621aaStep 6/6 : RUN echo 'hosts: files dns' >> /etc/nsswitch.conf ---> Running in d5a9a71bbe2fRemoving intermediate container d5a9a71bbe2f ---> 73b181e7c05aSuccessfully built 73b181e7c05aSuccessfully tagged gitlab/gitlab-runner-helper:x86_64-41d5c6addocker rm -f gitlab-runner-prebuilt-x86_64-41d5c6adError: No such container: gitlab-runner-prebuilt-x86_64-41d5c6adMakefile.runner_helper.mk:55: recipe for target 'out/helper-images/prebuilt-x86_64.tar' failedmake: [out/helper-images/prebuilt-x86_64.tar] Error 1 (ignored)docker create --name=gitlab-runner-prebuilt-x86_64-41d5c6ad gitlab/gitlab-runner-helper:x86_64-41d5c6ad /bin/shbe14d18aa9153578b35e7c58fce754093b89d6b50228c02baeaa440c753d723adocker export -o out/helper-images/prebuilt-x86_64.tar gitlab-runner-prebuilt-x86_64-41d5c6addocker rm -f gitlab-runner-prebuilt-x86_64-41d5c6adgitlab-runner-prebuilt-x86_64-41d5c6adxz -f -9 out/helper-images/prebuilt-x86_64.tardocker build -t gitlab/gitlab-runner-helper:arm-41d5c6ad -f dockerfiles/build/Dockerfile.arm dockerfiles/buildSending build context to Docker daemon  40.32MBStep 1/6 : FROM multiarch/alpine:armhf-v3.9armhf-v3.9: Pulling from multiarch/alpined2a76f42393b: Pull complete0e48aa23384e: Pull completeDigest: sha256:138eab1a3a8b31f4d0df75a752b5fdda95612505522de9e9f8d2f96d3d97abd9Status: Downloaded newer image for multiarch/alpine:armhf-v3.9 ---> daef464a9e14Step 2/6 : RUN apk add --no-cache bash ca-certificates git git-lfs miniperl     && ln -s miniperl /usr/bin/perl ---> Running in a87dfb239af0fetch https://uk.alpinelinux.org/alpine/v3.9/main/armhf/APKINDEX.tar.gzfetch https://uk.alpinelinux.org/alpine/v3.9/community/armhf/APKINDEX.tar.gz(1/15) Installing ncurses-terminfo-base (6.1_p20190105-r0)(2/15) Installing ncurses-terminfo (6.1_p20190105-r0)(3/15) Installing ncurses-libs (6.1_p20190105-r0)(4/15) Installing readline (7.0.003-r1)(5/15) Installing bash (4.4.19-r1)Executing bash-4.4.19-r1.post-install(6/15) Installing ca-certificates (20190108-r0)(7/15) Installing nghttp2-libs (1.35.1-r0)(8/15) Installing libssh2 (1.8.2-r0)(9/15) Installing libcurl (7.64.0-r2)(10/15) Installing libgcc (8.3.0-r0)(11/15) Installing expat (2.2.7-r0)(12/15) Installing pcre2 (10.32-r1)(13/15) Installing git (2.20.1-r0)(14/15) Installing git-lfs (2.5.1-r2)Executing git-lfs-2.5.1-r2.post-installGit LFS initialized.(15/15) Installing miniperl (5.26.3-r0)Executing busybox-1.29.3-r10.triggerExecuting ca-certificates-20190108-r0.triggerOK: 40 MiB in 34 packagesRemoving intermediate container a87dfb239af0 ---> 919719cb8063Step 3/6 : RUN git lfs install --skip-repo ---> Running in 292da8a82fffGit LFS initialized.Removing intermediate container 292da8a82fff ---> da40dc15d497Step 4/6 : COPY ./scripts/ /usr/bin ---> 65df18912961Step 5/6 : COPY ./binaries/gitlab-runner-helper.arm /usr/bin/gitlab-runner-helper ---> eb4e1e5c8f94Step 6/6 : RUN echo 'hosts: files dns' >> /etc/nsswitch.conf ---> Running in 6478fa57668aRemoving intermediate container 6478fa57668a ---> 9dc701e1cac4Successfully built 9dc701e1cac4Successfully tagged gitlab/gitlab-runner-helper:arm-41d5c6addocker rm -f gitlab-runner-prebuilt-arm-41d5c6adError: No such container: gitlab-runner-prebuilt-arm-41d5c6adMakefile.runner_helper.mk:55: recipe for target 'out/helper-images/prebuilt-arm.tar' failedmake: [out/helper-images/prebuilt-arm.tar] Error 1 (ignored)docker create --name=gitlab-runner-prebuilt-arm-41d5c6ad gitlab/gitlab-runner-helper:arm-41d5c6ad /bin/shd6f6155e534039619d616944528205ec3e388c9eda51787cbae329fcec10ec03docker export -o out/helper-images/prebuilt-arm.tar gitlab-runner-prebuilt-arm-41d5c6addocker rm -f gitlab-runner-prebuilt-arm-41d5c6adgitlab-runner-prebuilt-arm-41d5c6adxz -f -9 out/helper-images/prebuilt-arm.tar

開始構建軟體

當上面的步驟都就緒後，就可以正式開始編譯構建 GitLab Runner 啦。

source ci/touch_make_dependenciesmake build BUILD_PLATFORMS="-osarch='linux/amd64'"

-osarch 參數支援同時填寫多個平台，來達到輸出多個平台的可執行文件，比如填寫 linux/amd64 darwin/amd64，將輸出64位的Linux、OSx 系統的應用軟體。

目前支援的平台有：

  • darwin/386 / darwin/amd64
  • freebsd/386 / freebsd/amd64 / freebsd/arm
  • linux/386 / linux/amd64 / linux/arm
  • windows/386 / windows/amd64

如果我們只構建 Linux 64 位系統下的應用，將看到類似下面的日誌輸出：

# make build BUILD_PLATFORMS="-osarch='linux/amd64'"go get github.com/mitchellh/gox# Building gitlab-runner in version 12.2.0~beta.1803.g41d5c6ad for -osarch='linux/amd64'gox -osarch='linux/amd64'     -ldflags "-X gitlab.com/gitlab-org/gitlab-runner/common.NAME=gitlab-runner -X gitlab.com/gitlab-org/gitlab-runner/common.VERSION=12.2.0~beta.1803.g41d5c6ad -X gitlab.com/gitlab-org/gitlab-runner/common.REVISION=41d5c6ad -X gitlab.com/gitlab-org/gitlab-runner/common.BUILT=2019-08-01T17:04:48+0000 -X gitlab.com/gitlab-org/gitlab-runner/common.BRANCH=master -s -w"     -output="out/binaries/gitlab-runner-{{.OS}}-{{.Arch}}"     gitlab.com/gitlab-org/gitlab-runnerNumber of parallel builds: 7-->     linux/amd64: gitlab.com/gitlab-org/gitlab-runner

命令執行完畢後，我們可以在 ./out/binaries/ 目錄內看到我們想要的二進位程式。試著運行一下吧：

# ./out/binaries/gitlab-runner-linux-amd64 -vVersion:      12.2.0~beta.1803.g41d5c6adGit revision: 41d5c6adGit branch:   masterGO version:   go1.8.7Built:        2019-08-01T17:04:48+0000OS/Arch:      linux/amd64

如果你也能看到類似的輸出，那麼源碼編譯 GitLab Runner 的任務，就這麼愉快的結束啦。

上述問題解決方案來自項目 .gitlab-ci.yml 持續集成配置文件，感興趣的同學可以了解下。

最後

《編程匠藝》曾提過不應把過時錯誤的資訊提供給你的夥伴，要維護良好的文檔。然而現實中充滿了過時錯誤的資訊，就像本例中一樣，作為一款開源軟體，這些錯誤的資訊難免會澆滅外部貢獻者的熱情。

下一篇文章，我將講講我為什麼要編譯 GitLab。

—EOF

VirMach 便宜 VPS

QNews

QNews