User Tools

Site Tools


documentation:technical_docs:poudriere

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

documentation:technical_docs:poudriere [2019/03/02 14:33] (current)
Line 1: Line 1:
 +====== Poudriere image ======
 +{{description>​Unofficial Poudriere technical resources}}
 +====== Why using poudriere in place of nanobsd ? ======
  
 +BSDRP use a heavy customized nanobsd script that include package generation. This code need to be adapted each time the port build infrastructure change.
 +
 +Why not using the new shinny "​poudriere image" feature that will avoid to re-invent the wheel ? 
 +
 +====== Externals links about Poudriere ======
 +
 +Here are a list of docs and tutorials about Poudriere:
 +  *  [[https://​github.com/​freebsd/​poudriere/​wiki|Poudriere wiki]]: Official Wiki
 +  *  [[https://​www.freebsd.org/​doc/​handbook/​ports-poudriere.html|Building Packages with Poudriere]] : Presentation page on FreeBSD Handbook
 +  *  [[https://​github.com/​freebsd/​poudriere/​wiki/​poudriere.8|poudriere man page]] : The man page
 +
 +====== Understanding Poudriere image ======
 +
 +===== What is Poudriere ? =====
 +
 +It's a shell script used for build package in a clean (jail) environment.
 +But once you get a clean jail in one side, and a list of fresh generated package in other side, why not mix them together in a "​nanobsd like" firmware ?
 +
 +This is the "​poudriere image" feature includes in poudriere-devel-3.2.99.20180601
 +that is presented here.
 +
 +===== Images generated by Poudriere =====
 +
 +Poudriere can generate multiple "​image"​ types (default is iso+zmfs):
 +  * iso: An ISO 9660 format image
 +  * iso+mfs: An ISO 9660 format image where the root filesystem is MFS mounted
 +  * iso+zmfs (default):​ An ISO 9660 format image where the root filesystem is LZ77 compressed and is MFS mounted.
 +  * usb: A GPT-layout prepared UFS2 image containing a UEFI boot loader.
 +  * usb+mfs : A GPT-layout prepared UFS2 image containing a UEFI boot loader where the root filesystem is MFS mounted
 +  * usb+zmfs: A GPT-layout prepared UFS2 image containing a UEFI boot loader where the root filesystem is LZ77 compressed and is MFS mounted.
 +  * firmware: A NanoBSD style image with a GPT partitions and a UEFI boot loader
 +  * rawfirmware:​ A raw disk image (Update image that includes only one system partition of the firmware?)
 +  * rawdisk: A raw UFS2, softupdates-enabled,​ disk image
 +  * zrawdisk: A raw ZFS disk image
 +  * tar: An XZ-compressed tarball
 +  * embedded: Create a u-boot ready embedded image
 +
 +===== The 6 minimum steps to build a poudriere firmware image =====
 +
 +Using poudriere from a ZFS is not mandatory but strongly advised.
 +  - Install poudriere and configure it: <​code>​pkg install poudriere-devel
 +echo "​ZPOOL="​`zpool list -H | cut -f1` >> /​usr/​local/​etc/​poudriere.conf</​code>​
 +  - Create a poudriere jail WITH a GENERIC kernel (by default kernel is not build & installed): ​ <​code>​poudriere jail -c -j router -v 12.0-RELEASE -K GENERIC</​code>​
 +  - Create a port-tree using "​poudriere ports":<​code>​poudriere ports -c -p router-ports</​code>​
 +  - Generate list of ports to be build & added into the firmware image:<​code>​
 +cat > ~/​router-pkglist <<EOF
 +sysutils/​tmux
 +net/frr6
 +net/bird
 +net/mpd5
 +EOF</​code>​
 +  - Build them using "​poudriere bulk":<​code>​poudriere bulk -j router -p router-ports -f ~/​router-pkglist</​code>​
 +  - Generate your disk image (4Gb total, because 2 systems partitions of 2Gb) using "​poudriere image":​ <​code>​
 +poudriere image -t firmware -j router -s 4g -p router-ports -h router -n router -f ~/​router-pkglist
 +(...)
 +[00:00:40] Image available at: /​usr/​local/​poudriere/​data/​images//​router.img</​code>​
 +
 +===== Comparing poudriere firmware image and nanobsd image =====
 +
 +==== Final firmware file size ====
 +
 +Start by checking poudriere firmware file size:
 +<​code>​
 +root@lame4:​~ # # ls -alh /​usr/​local/​poudriere/​data/​images/​router.img
 +-rw-r--r-- ​ 1 root  wheel   3.6G Feb 19 23:48 /​usr/​local/​poudriere/​data/​images/​router.img
 +
 +</​code>​
 +
 +We've obtained a 3.6GiB disk image file... which fit into a [[https://​github.com/​freebsd/​poudriere/​pull/​638|marketed-size 4GB flash disk]].
 +
 +
 +==== Partition scheme ====
 +
 +The resulting images partition layout will have the same behavior than a nanobsd:
 +  * GPT partition with EFI bootloader (nanobsd uses a MBR scheme with BIOS bootloader)
 +  * first 991M (calculated from user input) system partition called gpt/​${IMAGENAME}1
 +  * second system partition called gpt/​${IMAGENAME}2
 +  * configuration configuration partition (hard-coded to 32M) called gpt/cfg
 +  * data partition (hard-coded to 32M) called gpt/data
 +
 +<​code>​
 +root@router:​~ # gpart show -l
 +=>      4  4191926 ​ vtbd0  GPT  (2.0G)
 +        4     ​1600 ​     1  (null) ​ (800K)
 +     ​1604 ​     118      2  (null) ​ (59K)
 +     ​1722 ​ 2029568 ​     3  router1 ​ (991M)
 +  2031290 ​ 2029568 ​     4  router2 ​ (991M)
 +  4060858 ​   65536      5  cfg  (32M)
 +  4126394 ​   65536      6  data  (32M)
 +
 +=>      4  4191926 ​ diskid/​DISK-BHYVE-59FF-34DA-1F19 ​ GPT  (2.0G)
 +        4     ​1600 ​                                ​1 ​ (null) ​ (800K)
 +     ​1604 ​     118                                 ​2 ​ (null) ​ (59K)
 +     ​1722 ​ 2029568 ​                                ​3 ​ router1 ​ (991M)
 +  2031290 ​ 2029568 ​                                ​4 ​ router2 ​ (991M)
 +  4060858 ​   65536                                 ​5 ​ cfg  (32M)
 +  4126394 ​   65536                                 ​6 ​ data  (32M)
 +
 +root@router:​~ # df -h
 +Filesystem ​         Size    Used   Avail Capacity ​ Mounted on
 +/​dev/​gpt/​router1 ​   1.8G    1.2G    424M    75%    /
 +devfs               ​1.0K ​   1.0K      0B   ​100% ​   /dev
 +tmpfs                32M    3.2M     ​29M ​   10%    /etc
 +tmpfs                32M    2.7M     ​29M ​    ​8% ​   /var
 +</​code>​
 +==== /etc ram disk ====
 +
 +Poudriere image is correctly compliant to a nanobsd /etc & /var ramdisk:
 +<​code>​
 +root@router:​~ # mount
 +/​dev/​gpt/​router1 on / (ufs, local, read-only)
 +devfs on /dev (devfs, local, multilabel)
 +/dev/md0 on /etc (ufs, local)
 +/dev/md1 on /var (ufs, local)
 +</​code>​
 +
 +Comparing to a standard nanobsd:
 +<​code>​
 +[root@nanobsd]~#​ mount
 +/​dev/​ufs/​NANOs1a on / (ufs, local, read-only)
 +devfs on /dev (devfs, local)
 +/dev/md0 on /etc (ufs, local)
 +/dev/md1 on /var (ufs, local)
 +</​code>​
 +==== fstab ====
 +
 +fstab is compliant to nanobsd too:
 +
 +<​code>​
 +root@router:​~ # cat /etc/fstab
 +/​dev/​gpt/​router1 / ufs ro 1 1
 +/​dev/​gpt/​cfg ​ /cfg  ufs rw,​noatime,​noauto ​       2 2
 +/​dev/​gpt/​data /data ufs rw,​noatime,​noauto,​failok 2 2
 +</​code>​
 +
 +<​code>​
 +[root@nanobsd]~#​ cat /etc/fstab
 +/​dev/​ufs/​NANOs2a / ufs ro 1 1
 +/​dev/​ufs/​NANOs3 /cfg ufs rw,​noatime,​noauto 2 2
 +/​dev/​ufs/​NANOs4 /data ufs rw,​noauto,​failok 2 2
 +</​code>​
 +
 +====== Migrating from NanoBSD to poudriere image ======
 +
 +===== Features matrix comparison =====
 +
 +nanobsd configuration file is a shell script,and BSDRP use a highly nanobsd customized configuration file generated by a wrapper (make.sh).
 +
 +The challenge of migrating the customization done on nanobsd to poudriere image can be resumed here:
 +
 +^ BSDRP customized nanobsd usage  ^ poudriere image     ^ Purpose ^
 +| Support differents src.conf: One for buildworld and another for installword | src.conf used for buildworld and installworld,​ image-JAILNAME[-SET]-src.conf added for installworld/​delete-old. Allow to use the -X exclude_list too | Using WITHOUT_INCLUDES or WITHOUT_DEBUG just for installworld but not for buildworld |
 +| Downloading a specific source revision of a stable/​releng/​head ​  | use svn url@rev (or custom source tree with '-b -m src=/​usr/​src'​) | Allow reproducing the same build on different systems |
 +| Build a specific kernel file  | Just had to install custom kernel int the custom source tree | Customized kernel |
 +| Build only list of kernel modules | FIXME How to do that? | Customized module list |
 +| Advanced scripting for building non-ports softwares | FIXME Not available, no idea of how to cleanly add this feature | Need to build some of /​usr/​src/​tools |
 +| Advanced tunning of final image | FIXME Need to add a hook | Generating a mtree (for host-IDS), creating specific users |
 +| Building ports using own (outdated) embedded script | NATIVE role of poudriere | Adding ports to image |
 +| System upgrade by changing MBR active mode on system partition | System upgrade by setting "​bootonce"​ attribute on system partition (and need to be removed from older?) | Upgrading system partition (new/old)|
 +====== Customized poudriere image for BSDRP ======
 +
 +How to manually (without a "​wrapper"​ script) configure poudriere for generating a BSDRP firmware image.
 +
 +===== Set of configuration files =====
 +
 +We need to start creating a set of configuration files, named BSDRP:
 +  * poudriere.d/​BSDRP-src.conf : Include all src.conf parameters used for reference jail buildworld/​installworld
 +  * poudriere.d/​BSDRP-make.conf : Include all common ports parameters
 +  * poudriere.d/​BSDRP-options/​$CATEGORY_$PORTNAME/​options : Include specific options for $CATEGORY/​$PORTNAME
 +  * poudriere.d/​image-$JAILNAME[-$SETNAME]-src.conf :​ parameters added for installworld (followed by a delete-old)
 +
 +Then need other configuration files:
 +  * BSDRP-pkglist : List of packages to be build and included in the final image
 +
 +==== poudriere.d/​BSDRP-src.conf ====
 +
 +The previous section of the NanoBSD configuration files found in variables CONF_BUILD and CONF_WORLD in file [[https://​github.com/​ocochard/​BSDRP/​blob/​4febbb26dad24a38275eba344b85f05bc123e59f/​BSDRP/​BSDRP.nano|BSDRP/​BSDRP.nano]] should be copied in this file.
 +
 +As for the BSDRP example, this give this file:
 +<​code>​
 +MALLOC_PRODUCTION=
 +BOOT_BOOT0_COMCONSOLE_SPEED=0
 +WITHOUT_ACCT=
 +WITHOUT_AMD=
 +WITHOUT_APM=
 +WITHOUT_ASSERT_DEBUG=
 +WITHOUT_ATF=
 +WITHOUT_ATM=
 +WITHOUT_AUDIT=
 +WITHOUT_AUTHPF=
 +WITHOUT_AUTOFS=
 +WITHOUT_BHYVE=
 +WITHOUT_BLUETOOTH=
 +WITHOUT_BOOTPARAMD=
 +WITHOUT_BOOTPD=
 +WITHOUT_BSDINSTALL=
 +WITHOUT_CALENDAR=
 +WITHOUT_CCD=
 +WITHOUT_CTM=
 +WITHOUT_CVS=
 +WITHOUT_DICT=
 +WITHOUT_EE=
 +WITHOUT_EXAMPLES=
 +WITHOUT_FINGER=
 +WITHOUT_FLOPPY=
 +WITHOUT_FREEBSD_UPDATE=
 +WITHOUT_GAMES=
 +WITHOUT_GCOV=
 +WITHOUT_GNUCXX=
 +WITHOUT_GPIB=
 +WITHOUT_GPIO=
 +WITHOUT_GROFF=
 +WITHOUT_HAST=
 +WITHOUT_HTML=
 +WITHOUT_ICONV=
 +WITHOUT_INFO=
 +WITHOUT_IPX=
 +WITHOUT_IPX_SUPPORT=
 +WITHOUT_ISCSI=
 +WITHOUT_LIB32=
 +WITHOUT_LINT=
 +WITHOUT_LOADER_FIREWIRE=
 +WITHOUT_LOADER_GELI=
 +WITHOUT_LOCATE=
 +WITHOUT_LPR=
 +WITHOUT_NCP=
 +WITHOUT_NDIS=
 +WITHOUT_NETCAT=
 +WITHOUT_NIS=
 +WITHOUT_NLS=
 +WITHOUT_NLS_CATALOGS=
 +WITHOUT_NS_CACHING=
 +WITHOUT_PC_SYSINSTALL=
 +WITHOUT_PORTSNAP=
 +WITHOUT_PROFILE=
 +WITHOUT_QUOTAS=
 +WITHOUT_RBOOTD=
 +WITHOUT_RCMDS=
 +WITHOUT_RCS=
 +WITHOUT_RESCUE=
 +WITHOUT_ROUTED=
 +WITHOUT_SENDMAIL=
 +WITHOUT_SERVICESDB=
 +WITHOUT_SHAREDOCS=
 +WITHOUT_SVNLITE=
 +WITHOUT_SYSCONS=
 +WITHOUT_SYSINSTALL=
 +WITHOUT_TALK=
 +WITHOUT_TESTS=
 +WITHOUT_TESTS_SUPPORT=
 +WITHOUT_TFTP=
 +WITHOUT_TIMED=
 +WITHOUT_UNBOUND=
 +WITHOUT_USB_GADGET_EXAMPLES=
 +WITHOUT_WIRELESS=
 +WITHOUT_WPA_SUPPLICANT_EAPOL=
 +WITHOUT_ZFS=
 +WITH_IDEA=
 +WITH_OFED=
 +WITH_DEBUG_FILES=
 +WITH_REPRODUCIBLE_BUILD=
 +WITH_DIRDEPS_BUILD=
 +WITH_RETPOLINE=
 +</​code>​
 +
 +==== poudriere.d/​image-JAILNAME-src.conf ====
 +
 +Allow to ADD WITHOUT that will be removed during installworld
 +
 +<​code>​
 +cat <<EOF > /​usr/​local/​etc/​poudriere.d/​image-BSDRP-src.conf
 +WITHOUT_DEBUG_FILES=
 +WITHOUT_TOOLCHAIN=
 +WITHOUT_INCLUDES=
 +EOF
 +</​code>​
 +==== poudriere.d/​BSDRP-make.conf ====
 +
 +This file contains configuration parameters common for all ports.
 + 
 +For BSDRP, it is simple:
 +<​code>​
 +cat <<EOF > /​usr/​local/​etc/​poudriere.d/​BSDRP-make.conf
 +OPTIONS_UNSET+= DOCS NLS X11 EXAMPLES
 +EOF
 +</​code>​
 +
 +==== poudriere.d/​BSDRP-options/​$CATEGORY_$PORTNAME/​options ====
 +
 +If the ports builded need to use specific options, they need to be declared.
 +
 +Using the command line "​poudriere option -z BSDRP CATEGORY/​PORTNAME"​ is the way of doing this easly.
 +
 +Or you can do this:
 +<​code>​
 +rm -rf /​var/​db/​ports/​*
 +cd /​usr/​ports/​net/​bird
 +make config
 +cd ../frr
 +make config
 +cd ../​../​security/​ipsec-tools/​
 +make config
 +cd ../​strongswan
 +make config
 +cd ../​../​sysutils/​flashrom/​
 +make config
 +cp -r /​var/​db/​ports/​* poudriere.d/​BSDRP-options/​
 +</​code>​
 +==== BSDRP-pkglist ====
 +
 +Now the list of package to be builded and added to the final image:
 +
 +<​code>​
 +security/​ca_root_nss ​                                                                                                              ​[15/​9237]
 +net-mgmt/​bsnmp-regex
 +net-mgmt/​bsnmp-ucd
 +lang/​python36
 +lang/​python3
 +lang/python
 +devel/​py-setuptools
 +net-mgmt/​rtrlib
 +net/frr6
 +net/bird2
 +net/​freevrrpd
 +net/​openldap24-client
 +security/​ipsec-tools
 +security/​strongswan
 +net-mgmt/​pmacct
 +net/ucarp
 +net/arping
 +net/​smcroute
 +net/​pim6-tools
 +net/pim6dd
 +net/pim6sd
 +net/mrouted
 +net/pimdd
 +net/pimd
 +net/tayga
 +net/​isc-dhcp44-server
 +net/​dhcprelya
 +net/dhcp6
 +sysutils/​fswatch-mon
 +sysutils/​monit
 +sysutils/​tmux
 +sysutils/​ipmitool
 +security/​sudo
 +net/mpd5
 +net/mlvpn
 +security/​easy-rsa
 +security/​openvpn
 +benchmarks/​iperf3
 +net/exabgp
 +sysutils/​flashrom
 +sysutils/​x86info
 +sysutils/​devcpu-data
 +sysutils/​intel-pcm
 +net/​ntraceroute
 +net-mgmt/​bgpq3
 +net/​ixl_unlock
 +net/​graphpath
 +net/​quagga-bgp-netgen
 +</​code>​
 +
 +
 +
 +===== Building the jail =====
 +
 +
 +The simplest solution is to re-use already existing BSDRP patched source tree: specific kernel configuration files can be installed into these source tree and use after.
 +
 +Start by only patching BSDRP sources (sources and ports) using the BSDRP make.sh script:
 +<​code>​
 +./make.sh -U
 +Update ONLY done
 +</​code>​
 +
 +There is now 2 sources ready patched:
 +  * /​usr/​local/​BSDRP/​BSDRP/​FreeBSD/​src (including BSDRP specific kernel configuration file)
 +  * /​usr/​local/​BSDRP/​BSDRP/​FreeBSD/​ports
 +
 +
 +<​code>​
 +poudriere jail -c -j BSDRP -a amd64 -z BSDRP -b -m src=/​usr/​local/​BSDRP/​BSDRP/​FreeBSD/​src -K amd64
 +</​code>​
 +Command line details:
 +  * -b: Build from source
 +  * -c: create a jail
 +  * -j: SHORT name for the jail (I can't use BSDRP-amd64-10.3R here because later it will generate a long directory name and long name aren't well supported)
 +  * -m src=: Path to the patched source branch we want to use
 +  * -z: Configuration set name, used for loading the BSDRP-src.conf
 +  * -K: The kernel configuration file, was copied here during patching BSDRP code trees 
 +
 +===== Creating port tree =====
 +
 +Now we need to create a port-tree using the patched existing port tree:
 +
 +<​code>​
 +poudriere ports -c -p BSDRP-ports -m null -M /​usr/​local/​BSDRP/​BSDRP/​FreeBSD/​ports
 +</​code>​
 +
 +===== Build packages =====
 +
 +poudriere native role, we just give the jail name, sets name and port-tree name to use then the list of packages.
 +<​code>​
 +poudriere bulk -j BSDRP -z BSDRP -p BSDRP-ports -f /​usr/​local/​etc/​poudriere.d/​BSDRP-pkglist
 +</​code>​
 +
 +===== Generating firmware image =====
 +
 +Here I'm instructing to build a 4GB image using the previous sets, jail, port-tree.
 +<​code>​
 +poudriere image -t firmware -s 4g -j BSDRP -p BSDRP-ports -z BSDRP -n BSDRP -h router.bsdrp.net -c /​usr/​local/​BSDRP/​BSDRP/​Files/​ -f /​usr/​local/​etc/​poudriere.d/​BSDRP-pkglist
 +</​code>​
 +
 +Command line explanation:​
 +  * -s: Size of full image size (same as the flash media)
 +  * -n: Image name, will be use as the partition name too
 +  * -h: Hostname configured on the image
 +  * -f: List of package to be installed on the image
 +  * -c: Directory tree to be copied on the image
 +
 +FIXME: It needs a hook at the end of image generation for advanced task like generating an mtree (used for host-IDS) or specific user creation.
documentation/technical_docs/poudriere.txt · Last modified: 2019/03/02 14:33 (external edit)