diff --git a/Auto-Builder-Server.md b/Auto-Builder-Server.md new file mode 100644 index 0000000..b6462a4 --- /dev/null +++ b/Auto-Builder-Server.md @@ -0,0 +1,503 @@ +# 搭建Auto-builder服务器过程记录 + +## 1 概述 + +由于原编译机服务器硬盘损坏，导致数据丢失。查阅文档记录并不全面，只能尝试重新搭建。现搭建时进行详细记录搭建过程。便于后来实施人员重搭环境及测试。 + +## 2 基础环境搭建及配置 + +### 2.1 系统安装及基本配置 + +root口令builder，配置默认root登录，配置ssh可root登录，配置双网卡bonding + +分区:(50G / ，32G swap ，两个50G空闲分区 ，剩下的挂/home) + +``` +root@rocky:/home# parted -l +Model: ATA ST2000DM001-1ER1 (scsi) +Disk /dev/sda: 2000GB +Sector size (logical/physical): 512B/4096B +Partition Table: gpt + +Number Start End Size File system Name Flags + 1 17.4kB 50.0GB 50.0GB ext3 + 2 50.0GB 100GB 50.0GB ext3 + 3 100GB 132GB 32.0GB linux-swap(v1) + 4 132GB 182GB 50.0GB ext3 + 5 182GB 2000GB 1818GB ext3 + +``` + +### 2.2 安装kvm虚拟机 + +kvm（kernel-based virtual machine）安装： 配置源、安装包。 + +#### 2.2.1 配置源 + +配置/etc/apt/sources.list，配置完后执行apt-get update + +``` +deb http://172.16.0.234/debian squeeze main contrib non-free +deb-src http://172.16.0.234/debian squeeze main contrib non-free +deb http://172.16.0.234/sid squeeze main +deb-src http://172.16.0.234/sid squeeze main +``` +#### 2.2.2 安装kvm包 + +``` +root@rocky:/home# aptitude install libvirt-bin + libvirt-dev python-libvirt virt-top virtinst virt-manager qemu-kvm + +``` + +#### 2.2.3 安装虚拟机 + +virt-manager启动kvm安装虚拟机。 + +分配100G“/”,4096M内存，配置默认有root登录，注意需update-grub2。 + +#### 2.2.4 虚拟机网络配置 + +配置桥接（gw：172.16.250.1 ip：172.16.250.220）： + +在virt-manager启动的图形界面上点击：虚拟机管理器--编辑--主机详情--虚拟网络（左下方“+”）--创建虚拟网络--输入“虚拟网络名”--选择ipv4地址空间“172.16.250.0/24”--前进到连接到物理网络,选择“转发到物理网络（NAT）” + +在单独的虚拟机图形界面上点击：选择虚拟机--显示虚拟机硬件详情--选择虚拟网络接口（e1000） + +### 2.3 配置nfs共享 + +#### 2.3.1 开机启动服务 + +在172.16.0.250上，添加开机启动服务 + +``` +chkconfig portmap on +chkconfig nfs-common on +chkconfig nfs-kernel-server on +``` + +#### 2.3.2 配置nfs服务器 + +创建目录/home/builder，配置/etc/exports + +``` +root@rocky:/home/builder# cat /etc/exports +# /etc/exports: the access control list for filesystems which may be exported +# to NFS clients. See exports(5). +# +# Example for NFSv2 and NFSv3: +# /srv/homes hostname1(rw,sync,no_subtree_check) hostname2(ro,sync,no_subtree_check) +# +# Example for NFSv4: +# /srv/nfs4 gss/krb5i(rw,sync,fsid=0,crossmnt,no_subtree_check) +# /srv/nfs4/homes gss/krb5i(rw,sync,no_subtree_check) +# +/home/builder 172.16.250.220(rw,anonuid=1001,anongid=1001,sync,no_subtree_check,no_root_squash) +``` +手动启动nfs服务器 + +``` +/etc/init.d/portmap start +/etc/init.d/nfs-common start +/etc/init.d/nfs-kernel-server start + +``` + +#### 2.3.3 配置nfs客户端 + +在172.16.250.220虚拟机上，创建目录/home/builder，修改配置/etc/fstab配置开机挂载（在最后添加一行）： + +``` +localhost:~ # cat /etc/fstab +# +# /etc/fstab: static file system information +# +# + +devpts /dev/pts devpts defaults 0 0 +sysfs /sys sysfs defaults 0 0 +proc /proc proc defaults 0 0 +#tmp /tmp tmpfs defaults 0 0 +tmpfs /dev/shm tmpfs defaults 0 0 +#usb /proc/bus/usb usbdevfs defaults 0 0 + +UUID=b253098b-da2c-4b5b-adde-3a4c2a227a48 / ext3 acl 0 1 +172.16.0.250:/home/builder /home/builder nfs nolock 0 0 +``` + +注意：重启测试，未挂载，需在/etc/rc.d/rc.local中添加mount -a + + +### 2.4 配置ssh服务 + +配置250编译服务器可root登录，生成密钥对，配置和编译虚拟机的等效性。 + +#### 2.4.1 生成密钥对 + +在172.16.0.250上，生成ssh密钥对。 + +``` +root@rocky:~/.ssh# ssh-keygen +Generating public/private rsa key pair. +Enter file in which to save the key (/root/.ssh/id_rsa): +/root/.ssh/id_rsa already exists. +Overwrite (y/n)? y +Enter passphrase (empty for no passphrase): +Enter same passphrase again: +Your identification has been saved in /root/.ssh/id_rsa. +Your public key has been saved in /root/.ssh/id_rsa.pub. +The key fingerprint is: +49:1c:2f:41:a7:b7:6f:e0:72:36:60:07:a1:9c:81:19 root@rocky +The key's randomart image is: ++--[ RSA 2048]----+ +| E+..= . | +| o. = B | +| + * o | +| . = . | +| S + | +| . + o | +| . = o | +| + o | +| | ++-----------------+ +root@rocky:~/.ssh# +``` +#### 2.4.2 设置等效性 + +``` +root@Auto-builder:~/.ssh# cp id_rsa.pub authorized_keys +root@Auto-builder:~/.ssh# ls +authorized_keys id_rsa id_rsa.pub known_hosts +root@Auto-builder:~/.ssh# + +``` + +配置完172.16.0.250服务器后，配置172.16.250.220虚拟机。将服务器上的.ssh目录拷贝到虚拟机的root家目录下即可。 + +### 2.5 配置gitlab用户 + +此处使用原有用户linx6.0.42builder@linx-info.com口令rockyrocky + +只需添加新的ssh公钥即可。 + +使用原有的linx6.0.42builder@linx-info.com登录后，在个人配置里将密钥配置里原有的公钥删除，添加新的公钥（前一节做出的ssh密钥对的id_rsa.pub）。 + +在172.16.0.250上测试配置好gitlab库的配置。git clone一个用户有权限的库测试： + +``` +root@Auto-builder:~# git clone git@gitlab.rd.in.linx:linx6.0.42/autobuild-tools.git +Cloning into autobuild-tools... +The authenticity of host 'gitlab.rd.in.linx (172.17.150.10)' can't be established. +RSA key fingerprint is 3a:0d:22:14:a2:21:8b:9d:52:12:57:c9:7a:0e:6a:3c. +Are you sure you want to continue connecting (yes/no)? yes +Warning: Permanently added 'gitlab.rd.in.linx,172.17.150.10' (RSA) to the list of known hosts. +-------------------------------------------------------- + +Linx Code server Powered By Debian GNU/Linx 8.0 && Gitlab + +-------------------------------------------------------- +remote: Counting objects: 1597, done. +remote: Compressing objects: 100% (1514/1514), done. +remote: Total 1597 (delta 896), reused 87 (delta 37) +Receiving objects: 100% (1597/1597), 627.08 KiB, done. +Resolving deltas: 100% (896/896), done. +root@Auto-builder:~# +``` + +登录编译虚拟机，测试git clone。克隆失败如下。 + +``` +localhost:/home/x86_64-workdir # git clone git@gitlab.rd.in.linx:linx6.0.42/autobuild-tools.git +Initialized empty Git repository in /home/x86_64-workdir/autobuild-tools/.git/ +ssh: Could not resolve hostname gitlab.rd.in.linx: Temporary failure in name resolution +fatal: The remote end hung up unexpectedly +localhost:/home/x86_64-workdir # route +Kernel IP routing table +Destination Gateway Genmask Flags Metric Ref Use Iface +172.16.250.0 * 255.255.255.0 U 0 0 0 eth0 +169.254.0.0 * 255.255.0.0 U 1002 0 0 eth0 +default 172.16.250.1 0.0.0.0 UG 0 0 0 eth0 +localhost:/home/x86_64-workdir # ping 172.17.150.10 +PING 172.17.150.10 (172.17.150.10) 56(84) bytes of data. +64 bytes from 172.17.150.10: icmp_seq=1 ttl=61 time=1.25 ms +64 bytes from 172.17.150.10: icmp_seq=2 ttl=61 time=1.35 ms +^C +--- 172.17.150.10 ping statistics --- +3 packets transmitted, 3 received, 0% packet loss, time 2003ms +rtt min/avg/max/mdev = 1.151/1.252/1.354/0.087 ms +localhost:/home/x86_64-workdir # ping gitlab.rd.in.linx +ping: unknown host gitlab.rd.in.linx +localhost:/home/x86_64-workdir # +``` + +此时现象为可以ping通，但解析不了域名。需配置hosts，添加gitlab.rd.in.linx。 + +``` +localhost:/home/x86_64-workdir # cat /etc/hosts +# +# /etc/hosts: static lookup table for host names +# + +127.0.0.1 localhost +192.168.1.109 localhost.in.linx localhost + +172.17.150.10 gitlab.rd.in.linx +# End of file +localhost:/home/x86_64-workdir # +``` +再进行clone测试，成功。 + +## 3 布置自动编译环境及测试调试 + +关于自动编译虚拟机的搭建及配置，可以参考文档《linx6.0.42.41自动编译环境搭建说明》 + +文档所在git库连接地址：　http://gitlab.rd.in.linx/linx6.0.42/documents/blob/master/autobuild.md + +### 3.1 布置自动编译环境 + +在编译虚拟机上执行以下操作 + +#### 3.1.1 创建工作目录 + +``` +# mkdir -p /home/x86_64-workdir +``` + +克隆autobuild-tools.git + +``` +# git clone git@gitlab.rd.in.linx:linx6.0.42/autobuild-tools.git +``` + +使用工具获取相关git库（等待执行结束即可） + +``` +# ./autobuild-tools/build-pkg/pre.sh +``` + +在编译机上，获取chroot_x86_64_git最小环境的git库，并准备好最小环境。 + +``` +# cd /home/x86_64-workdir +# git clone git@gitlab.rd.in.linx:chroot_git/chroot_x86_64_git.git +# cd chroot_x86_64_git +# ./preps.sh +``` + +创建日常编译、日常编译发布版、发布正式版、虚拟机默认安装等目录。 + +目录如下： + +``` +localhost:/home/builder # find . +. +./x86_64 +./x86_64/2015-12-16-005001 +./x86_64/2015-12-16-005001/pkgRecord +./x86_64/2015-12-16-005001/pkgRecord/base +./x86_64/2015-12-16-005001/pkgRecord/security +./x86_64/2015-12-16-005001/commitRecord +./x86_64/2015-12-16-005001/cfg +./x86_64/2015-12-16-005001/cfg/iso_env +./x86_64/2015-12-16-005001/cfg/sys_env +./x86_64/2015-12-16-005001/cfg/xmlDir +./kvm_autocreate +./kvm_autocreate/mnt +./kvm_autocreate/kvmimage +./stable-iso +./stable-iso/4.2-i686-builds +./stable-iso/4.2-x86_64-builds +./daily-builds +./daily-builds/4.2-i686-builds +./daily-builds/4.2-x86_64-builds +./daily-builds/4.2-x86_64-builds/20151216 +./daily-builds/4.2-x86_64-builds/isoUpdates +localhost:/home/builder # +``` + +#### 3.1.2 添加git检测记录 + +为执行编译创建了/home/builder/x86_64/2015-12-16-005001/commitRecord/applications,applications文件中存放了上次发布版20150911的applications的git commit提交。此commitRecord目录下应该有所有要集成的git库的某一日期的commit记录（此时为测试记录，暂未添加别的）。因为检测是否更新包出盘的脚本中，是以检测commit是否有新提交来决定是否编包出盘的。 + +``` +localhost:/home/builder # cat ./x86_64/2015-12-16-005001/commitRecord/applications +x86_64:dedff700fda645fe2117f67d1c26347dc3b04ecc + +``` + +添加各个库的最新的git commit记录到信息记录的文件中，与上述application一样。到/home/x86_64-workdir/autobuild-tools/git_dir/git下的各个git库中，git log获取最新的git commit提交，写入配置文件中。如下： + +``` +localhost:/home/x86_64-workdir/autobuild-tools/build-pkg # ls /home/builder/x86_64/2015-12-17-015346/commitRecord/* +/home/builder/x86_64/2015-12-17-015346/commitRecord/applications +/home/builder/x86_64/2015-12-17-015346/commitRecord/liblinxsn +/home/builder/x86_64/2015-12-17-015346/commitRecord/linux-firmware +/home/builder/x86_64/2015-12-17-015346/commitRecord/linx-app +/home/builder/x86_64/2015-12-17-015346/commitRecord/linx-serialnumber +/home/builder/x86_64/2015-12-17-015346/commitRecord/manipulating_build_pkgs +/home/builder/x86_64/2015-12-17-015346/commitRecord/new_install +/home/builder/x86_64/2015-12-17-015346/commitRecord/StateGrid +localhost:/home/x86_64-workdir/autobuild-tools/build-pkg # cat /home/builder/x86_64/2015-12-17-015346/commitRecord/* +x86_64:10f23d6721fb088b84cde5e2ba530922b79d8580 +x86_64:60e3b78c2a1d2a022c129918ac21f1e2d1d05faa +x86_64:bbe4917c054eb0a73e250c6363341e3bf6725839 +x86_64:f85c7f5dd767d0ec2af8bbb5df187851077ef006 +x86_64:12e446f3e8e211897ee2fb7bd61897877adbd336 +x86_64:6792c5af9cbbec4f83dd076e198ea006bb455d99 +x86_64:911c08a72aa3bb6bd72aa6f91f0cc39dc2ff577a +x86_64:591be5e8a8dc13c7012dca6b7fd37b7deee0d6cf +localhost:/home/x86_64-workdir/autobuild-tools/build-pkg # + +``` + +#### 3.1.3 测试自动检测编译 + +将上次正式版发布的iso，挂载拷贝里面的安装包到/home/builder/x86_64/2015-12-16-005001/pkgRecord/security中。 + +在编译机执行auto_gitcheck.sh测试编译。(此时应该已可以检测编译，但是无法出盘) + +``` + +localhost:/home/x86_64-workdir # ./autobuild-tools/build-pkg/auto_gitcheck.sh -m +localhost:/home/x86_64-workdir # + +``` + +添加自动检测编译到开机启动rc.local中。如下，rc.local中最后一句。 + +``` +localhost:/home/x86_64-workdir # cat /etc/rc.d/rc.local +#!/bin/sh +# +# This script will be executed *after* all the other init scripts. +# You can put your own initialization stuff in here if you don't +# want to do the full Sys V style init stuff. +mount -a +echo 1 > /proc/sys/kernel/sysrq + +nohup /home/x86_64-workdir/autobuild-tools/build-pkg/auto_gitcheck.sh >> /home/x86_64-workdir/auto_gitcheck.log & +``` + +### 3.2 配置环境至出盘 + +#### 3.2.1 查看如何得到base包 + +查看文档《Rocky6.0.42版本构建工具说明文档V0.1.pdf》中get_mkiso_info工具说明，在之前测试编译出的目录/home/builder/x86_64/2015-12-17-015346中的cfg中有光盘环境包列表和系统环境包列表。 + +根据光盘环境包列表，开始着手编译base包集合。 + +查看原来的记录文档及说明文档。没有此类包如何得到及特殊编译的记录。 + +首先，手动编译base包kernel + +git clone git@gitlab.rd.in.linx:linx6.0.42/kernel.git + +在6.0系统中编译。此处，是在172.16.0.250服务器上进行编译的。（注意：此时才发现，此250服务器需要使用gdzhang的key，因为编译kernel需要从172.16.0.4获取包，只有gdzhang的key可以使用） + +执行编译，在/tmp/work中可找到编译完成的包等： + +``` +root@Auto-builder:/home/sxxu/kernel/kernel# ./build.sh -a x86_64 + -v 2.6.32.41 -b +``` + +拷贝编译好的内核到base目录下，拷贝光盘环境的所有包（除了内核）到base目录下。 + +注意：此时使用的base包的其他所有包，均来自查找原来的备份所得。（备份包为原swei的本机硬盘备份） + +#### 3.2.2 添加mkiso所需的tag + +查看auto_gitcheck.sh脚本及之前的log，可以运行到mkiso。查看mkiso脚本，MINOR_VERSION需要配置/home/builder/$ARCH的git tag。 + +在/home/builder/x86_64下创建git库，提交git tag + +``` +localhost:/home/builder/x86_64 # git init +Initialized empty Git repository in /home/builder/x86_64/.git/ +localhost:/home/builder/x86_64 # git add . +localhost:/home/builder/x86_64 # +localhost:/home/builder/x86_64 # ls -a +. .. 2015-12-16-005001 2015-12-17-015346 .git +localhost:/home/builder/x86_64 # git commit -s +[master (root-commit) 6db3265] Initial commit + + 943 files changed, 25739 insertions(+), 0 deletions(-) +... +localhost:/home/builder/x86_64 # +localhost:/home/builder/x86_64 # git tag 41 +localhost:/home/builder/x86_64 # git log +commit 6db3265e00dcf4ddaeed02f0f203c697464f5c58 +Author: Xu, Shunxuan +Date: Mon Dec 21 06:04:20 2015 +0800 + + Initial commit + + Signed-off-by: Xu, Shunxuan +localhost:/home/builder/x86_64 # git tag +41 +localhost:/home/builder/x86_64 # + +``` + +## 4 发布环境布置 + +### 4.1 配置apache服务器 + +重装apache的包，首先将原来的apache的包全部卸载，然后配置源/etc/apt/sources.list，只配置debian源。安装debian源里的apache包。 + +卸载包，更新源为debian源后，再重新安装： + +``` +root@Auto-builder:~# aptitude purge apache2 +root@Auto-builder:~# aptitude purge apache2.2-common apache2.2-bin apache2-mpm-prefork +root@Auto-builder:~# apt-get update +root@Auto-builder:~# aptitude install apache2 +``` + +修改apache配置文件/etc/apache2/httpd.conf(添加servername) + +修改apache配置文件/etc/apache2/sites-enabled/000-default(指定headername) + +拷贝apache浏览器界面配置文件到/var/www下(github-markdown.css footer.html)，修改footer.html。 + +注意，改完配置，启动或重启apache服务。 + +### 4.2 配置邮件发送服务 + +创建邮件服务器用户Auto-builder，设置sudo权限NOPASSWD：ALL + +postfix服务配置，安装包heirloom-mailx，注意查看/tmp的权限为777。 + +postfix配置文件/etc/postfix/main.cf + +``` +root@Auto-builder:/etc/postfix# cat main.cf +myhostname = Auto-builder +alias_maps = hash:/etc/aliases +alias_database = hash:/etc/aliases +mydestination = Auto-builder, localhost.localdomain, , localhost +relayhost = +mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 +mailbox_command = procmail -a "$EXTENSION" +mailbox_size_limit = 0 +recipient_delimiter = + +inet_interfaces = all +inet_protocols = ipv4 + +root@Auto-builder:/etc/postfix# +``` + +设置postfix开机自启动chkconfig postfix on + +测试发送邮件： + +``` +root@Auto-builder:/etc/postfix# su - Auto-builder +没有目录，将以 HOME=/ 登录 +Auto-builder@Auto-builder:/$ touch /tmp/111 +Auto-builder@Auto-builder:/$ sudo -u Auto-builder mail -s "test1" sxxu@linx-info.com < /tmp/111 +``` + +## 5 测试调试 + +基本配置如上所述。之后可进行git库更新测试调试。 diff --git a/README.md b/README.md index 870717e..e430858 100644 --- a/README.md +++ b/README.md @@ -21,3 +21,6 @@ ## [git reset远程git库主分支到merge前状态的操作纪录](Modified_git_default_branch.md) ## ## [网线制作说明](twist-pair/twist-pair.md) + +## [linx6.0.42.41自动编译服务器搭建](Auto-Builder-Server.md) +