搭建树莓派持续集成环境

正在做的树莓派应用需要做持续集成,需要挂一台实机跑CI runner吗,不,官方提供了一个的docker镜象raspbian/stretch,利用QEMU执行ARM,可以让x86计算机能直接用上raspbian环境。

这篇文章记录了一次树莓派持续集成环境的搭建过程,用的CI是gitlab-ci。

主机环境布置

docker镜象中的程序是ARM指令集的,主机需要安装QEMU并使用Linux的binfmt_misc机制调用虚拟机,实现兼容运行。

Debian系:

1
$ sudo apt install qemu qemu-user-static

Arch系:

1
$ sudo pacman -S qemu qemu-user-static binfmt-qemu-static

Docker环境搭建

先简单测试一下这个官方的镜象:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
$ docker run -itd raspbian/stretch
$ docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
ef20fb2bc508 raspbian/stretch "/bin/sh -c 'tail -f…" 41 seconds ago Up 40 seconds rasp
$ docker exec -it rasp /bin/bash
root@ef20fb2bc508:/#
root@ef20fb2bc508:/# cat /etc/os-release
PRETTY_NAME="Raspbian GNU/Linux 9 (stretch)"
NAME="Raspbian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"
root@ef20fb2bc508:/#

货真价实的rasbian环境,比较担心的是CPU性能,跑一下基准测试:

1
2
3
4
5
6
7
8
9
10
Host: AMD 3400G, Manjaro, sysbench 1.0.20
QEMU Rasbian: */Rasbian skretch/sysbench 0.4.12
Raspi 4b : Broadcom BCM2711, Quad core Cortex-A72/Rasbian buster/sysbench 0.4.12
---
测试项cpu(单线程)
sysbench --test=cpu --max-time=30 --num-threads=1 run
测试项cpu(4线程)
sysbench --test=cpu --max-time=30 --num-threads=4 run
测试项cpu(8线程)
sysbench --test=cpu --max-time=30 --num-threads=8 run
Host QEMU Rasbian Raspi 4b Rasbian
测试项cpu(单线程) 56758 1778 2724
测试项cpu(4线程) 187554 5822 7107
测试项cpu(8线程) 235309 9634 6663

再花了点时间跑了个UnixBench:

很明显,QEMU和实机相比,单核性能差了不少,但IO性能强了不少(废话)。
影响就是,比如在跑autoconf的配置阶段这种吃单核速度的东西,就会很慢。

肉眼可见的慢,用了两年AMD 3400G第一次感受到CPU不够用…

建立自己的Docker镜象

接下来正式开搞,做一个我们自己的docker镜象,换上交大的源,把常用的工具装进来,新建Dockerfile

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
FROM raspbian/stretch:041518
ENV TZ Asia/Shanghai
RUN echo "deb http://mirrors.sjtug.sjtu.edu.cn/raspbian/raspbian/ stretch main contrib non-free rpi firmware" > /etc/apt/sources.list \
&& echo "deb-src http://mirrors.sjtug.sjtu.edu.cn/raspbian/raspbian/ stretch main contrib non-free rpi firmware" >> /etc/apt/sources.list \
&& apt-get update \
&& apt-get install -y vim build-essential wget python2.7 python3 unzip rsync bc cpio libz1 libncurses5 libstdc++6 \
texinfo zlib1g-dev liblzo2-dev uuid-dev pkg-config libncurses-dev \
asciidoc binutils bzip2 gawk gettext git libncurses5-dev libz-dev patch zlib1g-dev \
libgcc1 libc6-dev subversion flex uglifyjs git-core p7zip p7zip-full \
msmtp libssl-dev libglib2.0-dev xmlto upx libelf-dev autoconf automake \
libtool autopoint device-tree-compiler gperf swig \
git scons \
&& apt-get clean \
&& ln -sf bash /bin/sh

构建docker build . -t sk-rasbian,等了少说30分钟才构建好。

发现安装java的VM也就是openjdk(jre)会失败…

Gitlab CI测试

接下来将docker runner注册到gitlab的ci-runner中,gitlab的ci-runner已经以docker容器(名为“gitlab-runner”)的方式事先架设好了,直接用命令注册:

1
2
3
4
5
6
7
8
9
10
11
12
13
docker exec -it gitlab-runner gitlab-runner register \
--non-interactive \
--executor "docker" \
--docker-image sk-rasbian \
--description "skteam rasbian" \
--url "http://192.168.99.xx" \
--registration-token "xxxxxxxxxxxxxxxxx" \
--tag-list "rasbian" \
--run-untagged \
--locked="false" \
--docker-extra-hosts "skgit:192.168.99.xx" \
--docker-pull-policy "if-not-present"

写个C的helloword,再建个仓库做CI测试,.gitlab-ci

1
2
3
4
5
6
7
8
9
10
build:
stage: build
tags:
- rasbian
script:
- gcc helloworld.c -o helloworld
- readelf helloworld -h
artifacts:
paths:
- helloworld

push后CI运行成功,生成了ARM机器的ELF:

遇到的问题:

1
2
/dev/mapper/control: open failed: Operation not permitted
Failure to communicate with kernel device-mapper driver.
待完成

Todo: 补全

直接在树莓派实机上搭建 Gitlab CI Runner

1
2
$ curl -LJO "https://gitlab-runner-downloads.s3.amazonaws.com/latest/deb/gitlab-runner_armhf.deb"
$ dpkg -i gitlab-runner_armhf.deb