Files
documents/Auto-Builder-Server.md
Xu, Shunxuan 938f4bd653 添加补充,开始使用发现的问题修改(nbd模块加载,对时服务)
修改:     Auto-Builder-Server.md

Signed-off-by: Xu, Shunxuan <sxxu@linx-info.com>
2016-01-29 15:36:52 +08:00

592 lines
19 KiB
Markdown
Raw Permalink Blame History

This file contains invisible Unicode characters
This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# 搭建Auto-builder服务器过程记录
## 1 概述
由于原编译机服务器硬盘损坏,导致数据丢失。查阅文档记录并不全面,只能尝试重新搭建。现搭建时进行详细记录搭建过程。便于后来实施人员重搭环境及测试。
## 2 基础环境搭建及配置
### 2.1 系统安装及基本配置
root口令builder配置默认root登录配置ssh可root登录配置双网卡bonding
分区:(50G / 32G swap 两个50G空闲分区 ,剩下的挂/home)
```
root@rocky:/home# parted -l
Model: ATA ST2000DM001-1ER1 (scsi)
Disk /dev/sda: 2000GB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt
Number Start End Size File system Name Flags
1 17.4kB 50.0GB 50.0GB ext3
2 50.0GB 100GB 50.0GB ext3
3 100GB 132GB 32.0GB linux-swap(v1)
4 132GB 182GB 50.0GB ext3
5 182GB 2000GB 1818GB ext3
```
### 2.2 安装kvm虚拟机
kvmkernel-based virtual machine安装 配置源、安装包。
#### 2.2.1 配置源
配置/etc/apt/sources.list配置完后执行apt-get update
```
deb http://172.16.0.234/debian squeeze main contrib non-free
deb-src http://172.16.0.234/debian squeeze main contrib non-free
deb http://172.16.0.234/sid squeeze main
deb-src http://172.16.0.234/sid squeeze main
```
#### 2.2.2 安装kvm包
```
root@rocky:/home# aptitude install libvirt-bin
libvirt-dev python-libvirt virt-top virtinst virt-manager qemu-kvm
```
#### 2.2.3 安装虚拟机
virt-manager启动kvm安装虚拟机。
分配100G“/”,4096M内存配置默认有root登录注意需update-grub2。
#### 2.2.4 虚拟机网络配置
配置桥接gw172.16.250.1 ip172.16.250.220
在virt-manager启动的图形界面上点击虚拟机管理器--编辑--主机详情--虚拟网络(左下方“+”)--创建虚拟网络--输入“虚拟网络名”--选择ipv4地址空间“172.16.250.0/24”--前进到连接到物理网络,选择“转发到物理网络NAT
在单独的虚拟机图形界面上点击:选择虚拟机--显示虚拟机硬件详情--选择虚拟网络接口e1000
### 2.3 配置nfs共享
#### 2.3.1 开机启动服务
在172.16.0.250上,添加开机启动服务
```
chkconfig portmap on
chkconfig nfs-common on
chkconfig nfs-kernel-server on
```
#### 2.3.2 配置nfs服务器
创建目录/home/builder配置/etc/exports
```
root@rocky:/home/builder# cat /etc/exports
# /etc/exports: the access control list for filesystems which may be exported
# to NFS clients. See exports(5).
#
# Example for NFSv2 and NFSv3:
# /srv/homes hostname1(rw,sync,no_subtree_check) hostname2(ro,sync,no_subtree_check)
#
# Example for NFSv4:
# /srv/nfs4 gss/krb5i(rw,sync,fsid=0,crossmnt,no_subtree_check)
# /srv/nfs4/homes gss/krb5i(rw,sync,no_subtree_check)
#
/home/builder 172.16.250.220(rw,anonuid=1001,anongid=1001,sync,no_subtree_check,no_root_squash)
```
手动启动nfs服务器
```
/etc/init.d/portmap start
/etc/init.d/nfs-common start
/etc/init.d/nfs-kernel-server start
```
#### 2.3.3 配置nfs客户端
在172.16.250.220虚拟机上,创建目录/home/builder修改配置/etc/fstab配置开机挂载在最后添加一行
```
localhost:~ # cat /etc/fstab
#
# /etc/fstab: static file system information
#
# <file system> <dir> <type> <options> <dump> <pass>
devpts /dev/pts devpts defaults 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
#tmp /tmp tmpfs defaults 0 0
tmpfs /dev/shm tmpfs defaults 0 0
#usb /proc/bus/usb usbdevfs defaults 0 0
UUID=b253098b-da2c-4b5b-adde-3a4c2a227a48 / ext3 acl 0 1
172.16.0.250:/home/builder /home/builder nfs nolock 0 0
```
注意:重启测试,未挂载,需在/etc/rc.d/rc.local中添加mount -a
### 2.4 配置ssh服务
配置250编译服务器可root登录生成密钥对配置和编译虚拟机的等效性。
#### 2.4.1 生成密钥对
在172.16.0.250上生成ssh密钥对。
```
root@rocky:~/.ssh# ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
/root/.ssh/id_rsa already exists.
Overwrite (y/n)? y
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
49:1c:2f:41:a7:b7:6f:e0:72:36:60:07:a1:9c:81:19 root@rocky
The key's randomart image is:
+--[ RSA 2048]----+
| E+..= . |
| o. = B |
| + * o |
| . = . |
| S + |
| . + o |
| . = o |
| + o |
| |
+-----------------+
root@rocky:~/.ssh#
```
#### 2.4.2 设置等效性
```
root@Auto-builder:~/.ssh# cp id_rsa.pub authorized_keys
root@Auto-builder:~/.ssh# ls
authorized_keys id_rsa id_rsa.pub known_hosts
root@Auto-builder:~/.ssh#
```
配置完172.16.0.250服务器后配置172.16.250.220虚拟机。将服务器上的.ssh目录拷贝到虚拟机的root家目录下即可。
### 2.5 配置gitlab用户
此处使用原有用户linx6.0.42builder@linx-info.com口令rockyrocky
只需添加新的ssh公钥即可。
使用原有的linx6.0.42builder@linx-info.com登录后在个人配置里将密钥配置里原有的公钥删除添加新的公钥前一节做出的ssh密钥对的id_rsa.pub
在172.16.0.250上测试配置好gitlab库的配置。git clone一个用户有权限的库测试
```
root@Auto-builder:~# git clone git@gitlab.rd.in.linx:linx6.0.42/autobuild-tools.git
Cloning into autobuild-tools...
The authenticity of host 'gitlab.rd.in.linx (172.17.150.10)' can't be established.
RSA key fingerprint is 3a:0d:22:14:a2:21:8b:9d:52:12:57:c9:7a:0e:6a:3c.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'gitlab.rd.in.linx,172.17.150.10' (RSA) to the list of known hosts.
--------------------------------------------------------
Linx Code server Powered By Debian GNU/Linx 8.0 && Gitlab
--------------------------------------------------------
remote: Counting objects: 1597, done.
remote: Compressing objects: 100% (1514/1514), done.
remote: Total 1597 (delta 896), reused 87 (delta 37)
Receiving objects: 100% (1597/1597), 627.08 KiB, done.
Resolving deltas: 100% (896/896), done.
root@Auto-builder:~#
```
登录编译虚拟机测试git clone。克隆失败如下。
```
localhost:/home/x86_64-workdir # git clone git@gitlab.rd.in.linx:linx6.0.42/autobuild-tools.git
Initialized empty Git repository in /home/x86_64-workdir/autobuild-tools/.git/
ssh: Could not resolve hostname gitlab.rd.in.linx: Temporary failure in name resolution
fatal: The remote end hung up unexpectedly
localhost:/home/x86_64-workdir # route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
172.16.250.0 * 255.255.255.0 U 0 0 0 eth0
169.254.0.0 * 255.255.0.0 U 1002 0 0 eth0
default 172.16.250.1 0.0.0.0 UG 0 0 0 eth0
localhost:/home/x86_64-workdir # ping 172.17.150.10
PING 172.17.150.10 (172.17.150.10) 56(84) bytes of data.
64 bytes from 172.17.150.10: icmp_seq=1 ttl=61 time=1.25 ms
64 bytes from 172.17.150.10: icmp_seq=2 ttl=61 time=1.35 ms
^C
--- 172.17.150.10 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 1.151/1.252/1.354/0.087 ms
localhost:/home/x86_64-workdir # ping gitlab.rd.in.linx
ping: unknown host gitlab.rd.in.linx
localhost:/home/x86_64-workdir #
```
此时现象为可以ping通但解析不了域名。需配置resolv.conf。
```
localhost:/home/builder/x86_64 # cat /etc/resolv.conf
#
# /etc/resolv.conf: resolver configuration file
#
search in.linx
nameserver 172.31.255.3
domain in.linx
# End of file
localhost:/home/builder/x86_64 #
```
再进行clone测试成功。
## 3 布置自动编译环境及测试调试
关于自动编译虚拟机的搭建及配置可以参考文档《linx6.0.42.41自动编译环境搭建说明》
文档所在git库连接地址 http://gitlab.rd.in.linx/linx6.0.42/documents/blob/master/autobuild.md
### 3.1 布置自动编译环境
在编译虚拟机上执行以下操作
#### 3.1.1 创建工作目录
```
# mkdir -p /home/x86_64-workdir
```
克隆autobuild-tools.git
```
# git clone git@gitlab.rd.in.linx:linx6.0.42/autobuild-tools.git
```
使用工具获取相关git库等待执行结束即可
```
# ./autobuild-tools/build-pkg/pre.sh
```
在编译机上获取chroot_x86_64_git最小环境的git库并准备好最小环境。
```
# cd /home/x86_64-workdir
# git clone git@gitlab.rd.in.linx:chroot_git/chroot_x86_64_git.git
# cd chroot_x86_64_git
# ./preps.sh
```
创建日常编译、日常编译发布版、发布正式版、虚拟机默认安装等目录。
目录如下:
```
localhost:/home/builder # find .
.
./x86_64
./x86_64/2015-12-16-005001
./x86_64/2015-12-16-005001/pkgRecord
./x86_64/2015-12-16-005001/pkgRecord/base
./x86_64/2015-12-16-005001/pkgRecord/security
./x86_64/2015-12-16-005001/commitRecord
./x86_64/2015-12-16-005001/cfg
./x86_64/2015-12-16-005001/cfg/iso_env
./x86_64/2015-12-16-005001/cfg/sys_env
./x86_64/2015-12-16-005001/cfg/xmlDir
./kvm_autocreate
./kvm_autocreate/mnt
./kvm_autocreate/kvmimage
./stable-iso
./stable-iso/4.2-i686-builds
./stable-iso/4.2-x86_64-builds
./daily-builds
./daily-builds/4.2-i686-builds
./daily-builds/4.2-x86_64-builds
./daily-builds/4.2-x86_64-builds/20151216
./daily-builds/4.2-x86_64-builds/isoUpdates
localhost:/home/builder #
```
#### 3.1.2 添加git检测记录
为执行编译创建了/home/builder/x86_64/2015-12-16-005001/commitRecord/applications,applications文件中存放了上次发布版20150911的applications的git commit提交。此commitRecord目录下应该有所有要集成的git库的某一日期的commit记录此时为测试记录暂未添加别的。因为检测是否更新包出盘的脚本中是以检测commit是否有新提交来决定是否编包出盘的。
```
localhost:/home/builder # cat ./x86_64/2015-12-16-005001/commitRecord/applications
x86_64:dedff700fda645fe2117f67d1c26347dc3b04ecc
```
添加各个库的最新的git commit记录到信息记录的文件中与上述application一样。到/home/x86_64-workdir/autobuild-tools/git_dir/git下的各个git库中git log获取最新的git commit提交写入配置文件中。如下
```
localhost:/home/x86_64-workdir/autobuild-tools/build-pkg # ls /home/builder/x86_64/2015-12-17-015346/commitRecord/*
/home/builder/x86_64/2015-12-17-015346/commitRecord/applications
/home/builder/x86_64/2015-12-17-015346/commitRecord/liblinxsn
/home/builder/x86_64/2015-12-17-015346/commitRecord/linux-firmware
/home/builder/x86_64/2015-12-17-015346/commitRecord/linx-app
/home/builder/x86_64/2015-12-17-015346/commitRecord/linx-serialnumber
/home/builder/x86_64/2015-12-17-015346/commitRecord/manipulating_build_pkgs
/home/builder/x86_64/2015-12-17-015346/commitRecord/new_install
/home/builder/x86_64/2015-12-17-015346/commitRecord/StateGrid
localhost:/home/x86_64-workdir/autobuild-tools/build-pkg # cat /home/builder/x86_64/2015-12-17-015346/commitRecord/*
x86_64:10f23d6721fb088b84cde5e2ba530922b79d8580
x86_64:60e3b78c2a1d2a022c129918ac21f1e2d1d05faa
x86_64:bbe4917c054eb0a73e250c6363341e3bf6725839
x86_64:f85c7f5dd767d0ec2af8bbb5df187851077ef006
x86_64:12e446f3e8e211897ee2fb7bd61897877adbd336
x86_64:6792c5af9cbbec4f83dd076e198ea006bb455d99
x86_64:911c08a72aa3bb6bd72aa6f91f0cc39dc2ff577a
x86_64:591be5e8a8dc13c7012dca6b7fd37b7deee0d6cf
localhost:/home/x86_64-workdir/autobuild-tools/build-pkg #
```
#### 3.1.3 测试自动检测编译
将上次正式版发布的iso挂载拷贝里面的安装包到/home/builder/x86_64/2015-12-16-005001/pkgRecord/security中。
在编译机执行auto_gitcheck.sh测试编译。(此时应该已可以检测编译,但是无法出盘)
```
localhost:/home/x86_64-workdir # ./autobuild-tools/build-pkg/auto_gitcheck.sh -m
localhost:/home/x86_64-workdir #
```
添加自动检测编译到开机启动rc.local中。如下rc.local中最后一句。
```
localhost:/home/x86_64-workdir # cat /etc/rc.d/rc.local
#!/bin/sh
#
# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don't
# want to do the full Sys V style init stuff.
mount -a
echo 1 > /proc/sys/kernel/sysrq
nohup /home/x86_64-workdir/autobuild-tools/build-pkg/auto_gitcheck.sh >> /home/x86_64-workdir/auto_gitcheck.log &
```
### 3.2 配置环境至出盘
#### 3.2.1 查看如何得到base包
查看文档《Rocky6.0.42版本构建工具说明文档V0.1.pdf》中get_mkiso_info工具说明在之前测试编译出的目录/home/builder/x86_64/2015-12-17-015346中的cfg中有光盘环境包列表和系统环境包列表。
根据光盘环境包列表开始着手编译base包集合。
查看原来的记录文档及说明文档。没有此类包如何得到及特殊编译的记录。
首先手动编译base包kernel
git clone git@gitlab.rd.in.linx:linx6.0.42/kernel.git
在6.0系统中编译。此处是在172.16.0.250服务器上进行编译的。注意此时才发现此250服务器需要使用gdzhang的key因为编译kernel需要从172.16.0.4获取包只有gdzhang的key可以使用
执行编译,在/tmp/work中可找到编译完成的包等
```
root@Auto-builder:/home/sxxu/kernel/kernel# ./build.sh -a x86_64
-v 2.6.32.41 -b
```
拷贝编译好的内核到base目录下拷贝光盘环境的所有包除了内核到base目录下。
注意此时使用的base包的其他所有包均来自查找原来的备份所得。备份包为原swei的本机硬盘备份
#### 3.2.2 添加mkiso所需的tag
查看auto_gitcheck.sh脚本及之前的log可以运行到mkiso。查看mkiso脚本MINOR_VERSION需要配置/home/builder/$ARCH的git tag。
在/home/builder/x86_64下创建git库提交git tag
```
localhost:/home/builder/x86_64 # git init
Initialized empty Git repository in /home/builder/x86_64/.git/
localhost:/home/builder/x86_64 # git add .
localhost:/home/builder/x86_64 #
localhost:/home/builder/x86_64 # ls -a
. .. 2015-12-16-005001 2015-12-17-015346 .git
localhost:/home/builder/x86_64 # git commit -s
[master (root-commit) 6db3265] Initial commit
943 files changed, 25739 insertions(+), 0 deletions(-)
...
localhost:/home/builder/x86_64 #
localhost:/home/builder/x86_64 # git tag 41
localhost:/home/builder/x86_64 # git log
commit 6db3265e00dcf4ddaeed02f0f203c697464f5c58
Author: Xu, Shunxuan <sxxu@linx-info.com>
Date: Mon Dec 21 06:04:20 2015 +0800
Initial commit
Signed-off-by: Xu, Shunxuan <sxxu@linx-info.com>
localhost:/home/builder/x86_64 # git tag
41
localhost:/home/builder/x86_64 #
```
## 4 发布环境布置
### 4.1 配置apache服务器
重装apache的包首先将原来的apache的包全部卸载然后配置源/etc/apt/sources.list只配置debian源。安装debian源里的apache包。
卸载包更新源为debian源后再重新安装
```
root@Auto-builder:~# aptitude purge apache2
root@Auto-builder:~# aptitude purge apache2.2-common apache2.2-bin apache2-mpm-prefork
root@Auto-builder:~# apt-get update
root@Auto-builder:~# aptitude install apache2
```
修改apache配置文件/etc/apache2/httpd.conf(添加servername)
修改apache配置文件/etc/apache2/sites-enabled/000-default(指定headername)
拷贝apache浏览器界面配置文件到/var/www下(github-markdown.css footer.html)修改footer.html。
注意改完配置启动或重启apache服务。
### 4.2 配置邮件发送服务
创建邮件服务器用户Auto-builder设置sudo权限NOPASSWDALL
postfix服务配置安装包heirloom-mailx注意查看/tmp的权限为777。
postfix配置文件/etc/postfix/main.cf
```
root@Auto-builder:/etc/postfix# cat main.cf
myhostname = Auto-builder
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
mydestination = Auto-builder, localhost.localdomain, , localhost
relayhost =
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_command = procmail -a "$EXTENSION"
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
inet_protocols = ipv4
root@Auto-builder:/etc/postfix#
```
设置postfix开机自启动chkconfig postfix on
测试发送邮件:
```
root@Auto-builder:/etc/postfix# su - Auto-builder
没有目录,将以 HOME=/ 登录
Auto-builder@Auto-builder:/$ touch /tmp/111
Auto-builder@Auto-builder:/$ sudo -u Auto-builder mail -s "test1" sxxu@linx-info.com < /tmp/111
```
## 5 测试调试
基本配置如上所述。之后可进行git库更新测试调试。
## 6 后期使用添加补充
### 6.1 添加nbd模块加载
经测试调试发现安装虚拟机会出错。原因是未加载nbd设备。导致配置已经初始化的kvm_install.img失败。
修改250服务器的rc.local添加开机启动加载nbd设备如下
```
root@Auto-builder:~# cat /etc/modules
# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.
# Parameters can be specified after the module name.
loop
linx_sec
linx_sysinfo
nbd max_part=8
```
### 6.2 出盘发布时间变化
多次出盘时间定在09:00左右。查看发现编译虚拟机的时间不对。配置编译虚拟机的对时任务配置250服务器的ntp对时服务。
虚拟机对时任务配置:(添加对时脚本,配置定时任务)
```
localhost:/var/log # ls /usr/sbin/update_time.sh -l
-rwxr-xr-x 1 sysadmin sysadmin 181 1月 29 11:53 /usr/sbin/update_time.sh
localhost:/var/log # cat /usr/sbin/update_time.sh
#!/bin/bash
NTPSERVER_A='172.16.0.250'
NTPSERVER_B=''
NTPDATE='/usr/sbin/ntpdate'
${NTPDATE} ${NTPSERVER_A}
#${NTPDATE} ${NTPSERVER_A} || ${NTPDATE} ${NTPSERVER_B}
/sbin/hwclock -w
localhost:/var/log # cat /etc/crontab
#
# Vixie cron /etc/crontab
#
# globals
SHELL=/bin/bash
PATH=/bin:/sbin:/usr/bin:/usr/sbin
MAILTO=""
0 20 * * * sysadmin /usr/sbin/runjobs /etc/cron/daily
5 20 * * mon sysadmin /usr/sbin/runjobs /etc/cron/weekly
10 20 1 * * sysadmin /usr/sbin/runjobs /etc/cron/monthly
*/1 * * * * sysadmin /usr/sbin/update_time.sh
```
服务器ntp对时服务配置ntp服务默认是开机启动
```
root@Auto-builder:~# cat /etc/ntp.conf
# /etc/ntp.conf, configuration for ntpd; see ntp.conf(5) for help
driftfile /var/lib/ntp/ntp.drift
# Enable this if you want statistics to be logged.
#statsdir /var/log/ntpstats/
statistics loopstats peerstats clockstats
filegen loopstats file loopstats type day enable
filegen peerstats file peerstats type day enable
filegen clockstats file clockstats type day enable
server 172.31.255.8 iburst
server 172.31.255.9 iburst
server 172.17.0.240 iburst noselect
server 172.17.0.239 iburst noselect
server 172.17.0.235 iburst noselect
```