Differences

This shows you the differences between two versions of the page.

@@ Line 1: / Line 1: @@
+====== Poudriere image ======
+{{description>Unofficial Poudriere technical resources}}
+====== Why using poudriere in place of nanobsd ? ======
+BSDRP use a heavy customized nanobsd script that include package generation. This code need to be adapted each time the port build infrastructure change.
+Why not using the new shinny "poudriere image" feature that will avoid to re-invent the wheel ?
+====== Externals links about Poudriere ======
+Here are a list of docs and tutorials about Poudriere:
+  *  [[https://github.com/freebsd/poudriere/wiki|Poudriere wiki]]: Official Wiki
+  *  [[https://www.freebsd.org/doc/handbook/ports-poudriere.html|Building Packages with Poudriere]] : Presentation page on FreeBSD Handbook
+  *  [[https://github.com/freebsd/poudriere/wiki/poudriere.8|poudriere man page]] : The man page
+====== Understanding Poudriere image ======
+===== What is Poudriere ? =====
+It's a shell script used for build package in a clean (jail) environment.
+But once you get a clean jail in one side, and a list of fresh generated package in other side, why not mix them together in a "nanobsd like" firmware ?
+This is the "poudriere image" feature includes in poudriere-devel-3.2.99.20180601
+that is presented here.
+===== Images generated by Poudriere =====
+Poudriere can generate multiple "image" types (default is iso+zmfs):
+  * iso: An ISO 9660 format image
+  * iso+mfs: An ISO 9660 format image where the root filesystem is MFS mounted
+  * iso+zmfs (default): An ISO 9660 format image where the root filesystem is LZ77 compressed and is MFS mounted.
+  * usb: A GPT-layout prepared UFS2 image containing a UEFI boot loader.
+  * usb+mfs : A GPT-layout prepared UFS2 image containing a UEFI boot loader where the root filesystem is MFS mounted
+  * usb+zmfs: A GPT-layout prepared UFS2 image containing a UEFI boot loader where the root filesystem is LZ77 compressed and is MFS mounted.
+  * firmware: A NanoBSD style image with a GPT partitions and a UEFI boot loader
+  * rawfirmware: A raw disk image (Update image that includes only one system partition of the firmware?)
+  * rawdisk: A raw UFS2, softupdates-enabled, disk image
+  * zrawdisk: A raw ZFS disk image
+  * tar: An XZ-compressed tarball
+  * embedded: Create a u-boot ready embedded image
+===== The 6 minimum steps to build a poudriere firmware image =====
+Using poudriere from a ZFS is not mandatory but strongly advised.
+  - Install poudriere and configure it: <code>pkg install poudriere-devel
+echo "ZPOOL="`zpool list -H | cut -f1` >> /usr/local/etc/poudriere.conf</code>
+  - Create a poudriere jail WITH a GENERIC kernel (by default kernel is not build & installed):  <code>poudriere jail -c -j router -v 12.0-RELEASE -K GENERIC</code>
+  - Create a port-tree using "poudriere ports":<code>poudriere ports -c -p router-ports</code>
+  - Generate list of ports to be build & added into the firmware image:<code>
+cat > ~/router-pkglist <<EOF
+sysutils/tmux
+net/frr6
+net/bird
+net/mpd5
+EOF</code>
+  - Build them using "poudriere bulk":<code>poudriere bulk -j router -p router-ports -f ~/router-pkglist</code>
+  - Generate your disk image (4Gb total, because 2 systems partitions of 2Gb) using "poudriere image": <code>
+poudriere image -t firmware -j router -s 4g -p router-ports -h router -n router -f ~/router-pkglist
+(...)
+[00:00:40] Image available at: /usr/local/poudriere/data/images//router.img</code>
+===== Comparing poudriere firmware image and nanobsd image =====
+==== Final firmware file size ====
+Start by checking poudriere firmware file size:
+<code>
+root@lame4:~ # # ls -alh /usr/local/poudriere/data/images/router.img
+-rw-r--r--  1 root  wheel   3.6G Feb 19 23:48 /usr/local/poudriere/data/images/router.img
+</code>
+We've obtained a 3.6GiB disk image file... which fit into a [[https://github.com/freebsd/poudriere/pull/638|marketed-size 4GB flash disk]].
+==== Partition scheme ====
+The resulting images partition layout will have the same behavior than a nanobsd:
+  * GPT partition with EFI bootloader (nanobsd uses a MBR scheme with BIOS bootloader)
+  * first 991M (calculated from user input) system partition called gpt/${IMAGENAME}1
+  * second system partition called gpt/${IMAGENAME}2
+  * configuration configuration partition (hard-coded to 32M) called gpt/cfg
+  * data partition (hard-coded to 32M) called gpt/data
+<code>
+root@router:~ # gpart show -l
+=>      4  4191926  vtbd0  GPT  (2.0G)
+     1600      1  (null)  (800K)
+      118      2  (null)  (59K)
+  2029568      3  router1  (991M)
+  2031290  2029568      4  router2  (991M)
+  4060858    65536      5  cfg  (32M)
+  4126394    65536      6  data  (32M)
+=>      4  4191926  diskid/DISK-BHYVE-59FF-34DA-1F19  GPT  (2.0G)
+     1600                                 1  (null)  (800K)
+      118                                 2  (null)  (59K)
+  2029568                                 3  router1  (991M)
+  2031290  2029568                                 4  router2  (991M)
+  4060858    65536                                 5  cfg  (32M)
+  4126394    65536                                 6  data  (32M)
+root@router:~ # df -h
+Filesystem          Size    Used   Avail Capacity  Mounted on
+/dev/gpt/router1    1.8G    1.2G    424M    75%    /
+devfs               1.0K    1.0K      0B   100%    /dev
+tmpfs                32M    3.2M     29M    10%    /etc
+tmpfs                32M    2.7M     29M     8%    /var
+</code>
+==== /etc ram disk ====
+Poudriere image is correctly compliant to a nanobsd /etc & /var ramdisk:
+<code>
+root@router:~ # mount
+/dev/gpt/router1 on / (ufs, local, read-only)
+devfs on /dev (devfs, local, multilabel)
+/dev/md0 on /etc (ufs, local)
+/dev/md1 on /var (ufs, local)
+</code>
+Comparing to a standard nanobsd:
+<code>
+[root@nanobsd]~# mount
+/dev/ufs/NANOs1a on / (ufs, local, read-only)
+devfs on /dev (devfs, local)
+/dev/md0 on /etc (ufs, local)
+/dev/md1 on /var (ufs, local)
+</code>
+==== fstab ====
+fstab is compliant to nanobsd too:
+<code>
+root@router:~ # cat /etc/fstab
+/dev/gpt/router1 / ufs ro 1 1
+/dev/gpt/cfg  /cfg  ufs rw,noatime,noauto        2 2
+/dev/gpt/data /data ufs rw,noatime,noauto,failok 2 2
+</code>
+<code>
+[root@nanobsd]~# cat /etc/fstab
+/dev/ufs/NANOs2a / ufs ro 1 1
+/dev/ufs/NANOs3 /cfg ufs rw,noatime,noauto 2 2
+/dev/ufs/NANOs4 /data ufs rw,noauto,failok 2 2
+</code>
+====== Migrating from NanoBSD to poudriere image ======
+===== Features matrix comparison =====
+nanobsd configuration file is a shell script,and BSDRP use a highly nanobsd customized configuration file generated by a wrapper (make.sh).
+The challenge of migrating the customization done on nanobsd to poudriere image can be resumed here:
+^ BSDRP customized nanobsd usage  ^ poudriere image     ^ Purpose ^
+| Support differents src.conf: One for buildworld and another for installword | src.conf used for buildworld and installworld, image-JAILNAME[-SET]-src.conf added for installworld/delete-old. Allow to use the -X exclude_list too | Using WITHOUT_INCLUDES or WITHOUT_DEBUG just for installworld but not for buildworld |
+| Downloading a specific source revision of a stable/releng/head   | use svn url@rev (or custom source tree with '-b -m src=/usr/src') | Allow reproducing the same build on different systems |
+| Build a specific kernel file  | Just had to install custom kernel int the custom source tree | Customized kernel |
+| Build only list of kernel modules | FIXME How to do that? | Customized module list |
+| Advanced scripting for building non-ports softwares | FIXME Not available, no idea of how to cleanly add this feature | Need to build some of /usr/src/tools |
+| Advanced tunning of final image | FIXME Need to add a hook | Generating a mtree (for host-IDS), creating specific users |
+| Building ports using own (outdated) embedded script | NATIVE role of poudriere | Adding ports to image |
+| System upgrade by changing MBR active mode on system partition | System upgrade by setting "bootonce" attribute on system partition (and need to be removed from older?) | Upgrading system partition (new/old)|
+====== Customized poudriere image for BSDRP ======
+How to manually (without a "wrapper" script) configure poudriere for generating a BSDRP firmware image.
+===== Set of configuration files =====
+We need to start creating a set of configuration files, named BSDRP:
+  * poudriere.d/BSDRP-src.conf : Include all src.conf parameters used for reference jail buildworld/installworld
+  * poudriere.d/BSDRP-make.conf : Include all common ports parameters
+  * poudriere.d/BSDRP-options/$CATEGORY_$PORTNAME/options : Include specific options for $CATEGORY/$PORTNAME
+  * poudriere.d/image-$JAILNAME[-$SETNAME]-src.conf : parameters added for installworld (followed by a delete-old)
+Then need other configuration files:
+  * BSDRP-pkglist : List of packages to be build and included in the final image
+==== poudriere.d/BSDRP-src.conf ====
+The previous section of the NanoBSD configuration files found in variables CONF_BUILD and CONF_WORLD in file [[https://github.com/ocochard/BSDRP/blob/4febbb26dad24a38275eba344b85f05bc123e59f/BSDRP/BSDRP.nano|BSDRP/BSDRP.nano]] should be copied in this file.
+As for the BSDRP example, this give this file:
+<code>
+MALLOC_PRODUCTION=
+BOOT_BOOT0_COMCONSOLE_SPEED=0
+WITHOUT_ACCT=
+WITHOUT_AMD=
+WITHOUT_APM=
+WITHOUT_ASSERT_DEBUG=
+WITHOUT_ATF=
+WITHOUT_ATM=
+WITHOUT_AUDIT=
+WITHOUT_AUTHPF=
+WITHOUT_AUTOFS=
+WITHOUT_BHYVE=
+WITHOUT_BLUETOOTH=
+WITHOUT_BOOTPARAMD=
+WITHOUT_BOOTPD=
+WITHOUT_BSDINSTALL=
+WITHOUT_CALENDAR=
+WITHOUT_CCD=
+WITHOUT_CTM=
+WITHOUT_CVS=
+WITHOUT_DICT=
+WITHOUT_EE=
+WITHOUT_EXAMPLES=
+WITHOUT_FINGER=
+WITHOUT_FLOPPY=
+WITHOUT_FREEBSD_UPDATE=
+WITHOUT_GAMES=
+WITHOUT_GCOV=
+WITHOUT_GNUCXX=
+WITHOUT_GPIB=
+WITHOUT_GPIO=
+WITHOUT_GROFF=
+WITHOUT_HAST=
+WITHOUT_HTML=
+WITHOUT_ICONV=
+WITHOUT_INFO=
+WITHOUT_IPX=
+WITHOUT_IPX_SUPPORT=
+WITHOUT_ISCSI=
+WITHOUT_LIB32=
+WITHOUT_LINT=
+WITHOUT_LOADER_FIREWIRE=
+WITHOUT_LOADER_GELI=
+WITHOUT_LOCATE=
+WITHOUT_LPR=
+WITHOUT_NCP=
+WITHOUT_NDIS=
+WITHOUT_NETCAT=
+WITHOUT_NIS=
+WITHOUT_NLS=
+WITHOUT_NLS_CATALOGS=
+WITHOUT_NS_CACHING=
+WITHOUT_PC_SYSINSTALL=
+WITHOUT_PORTSNAP=
+WITHOUT_PROFILE=
+WITHOUT_QUOTAS=
+WITHOUT_RBOOTD=
+WITHOUT_RCMDS=
+WITHOUT_RCS=
+WITHOUT_RESCUE=
+WITHOUT_ROUTED=
+WITHOUT_SENDMAIL=
+WITHOUT_SERVICESDB=
+WITHOUT_SHAREDOCS=
+WITHOUT_SVNLITE=
+WITHOUT_SYSCONS=
+WITHOUT_SYSINSTALL=
+WITHOUT_TALK=
+WITHOUT_TESTS=
+WITHOUT_TESTS_SUPPORT=
+WITHOUT_TFTP=
+WITHOUT_TIMED=
+WITHOUT_UNBOUND=
+WITHOUT_USB_GADGET_EXAMPLES=
+WITHOUT_WIRELESS=
+WITHOUT_WPA_SUPPLICANT_EAPOL=
+WITHOUT_ZFS=
+WITH_IDEA=
+WITH_OFED=
+WITH_DEBUG_FILES=
+WITH_REPRODUCIBLE_BUILD=
+WITH_DIRDEPS_BUILD=
+WITH_RETPOLINE=
+</code>
+==== poudriere.d/image-JAILNAME-src.conf ====
+Allow to ADD WITHOUT that will be removed during installworld
+<code>
+cat <<EOF > /usr/local/etc/poudriere.d/image-BSDRP-src.conf
+WITHOUT_DEBUG_FILES=
+WITHOUT_TOOLCHAIN=
+WITHOUT_INCLUDES=
+EOF
+</code>
+==== poudriere.d/BSDRP-make.conf ====
+This file contains configuration parameters common for all ports.
+For BSDRP, it is simple:
+<code>
+cat <<EOF > /usr/local/etc/poudriere.d/BSDRP-make.conf
+OPTIONS_UNSET+= DOCS NLS X11 EXAMPLES
+EOF
+</code>
+==== poudriere.d/BSDRP-options/$CATEGORY_$PORTNAME/options ====
+If the ports builded need to use specific options, they need to be declared.
+Using the command line "poudriere option -z BSDRP CATEGORY/PORTNAME" is the way of doing this easly.
+Or you can do this:
+<code>
+rm -rf /var/db/ports/*
+cd /usr/ports/net/bird
+make config
+cd ../frr
+make config
+cd ../../security/ipsec-tools/
+make config
+cd ../strongswan
+make config
+cd ../../sysutils/flashrom/
+make config
+cp -r /var/db/ports/* poudriere.d/BSDRP-options/
+</code>
+==== BSDRP-pkglist ====
+Now the list of package to be builded and added to the final image:
+<code>
+security/ca_root_nss                                                                                                               [15/9237]
+net-mgmt/bsnmp-regex
+net-mgmt/bsnmp-ucd
+lang/python36
+lang/python3
+lang/python
+devel/py-setuptools
+net-mgmt/rtrlib
+net/frr6
+net/bird2
+net/freevrrpd
+net/openldap24-client
+security/ipsec-tools
+security/strongswan
+net-mgmt/pmacct
+net/ucarp
+net/arping
+net/smcroute
+net/pim6-tools
+net/pim6dd
+net/pim6sd
+net/mrouted
+net/pimdd
+net/pimd
+net/tayga
+net/isc-dhcp44-server
+net/dhcprelya
+net/dhcp6
+sysutils/fswatch-mon
+sysutils/monit
+sysutils/tmux
+sysutils/ipmitool
+security/sudo
+net/mpd5
+net/mlvpn
+security/easy-rsa
+security/openvpn
+benchmarks/iperf3
+net/exabgp
+sysutils/flashrom
+sysutils/x86info
+sysutils/devcpu-data
+sysutils/intel-pcm
+net/ntraceroute
+net-mgmt/bgpq3
+net/ixl_unlock
+net/graphpath
+net/quagga-bgp-netgen
+</code>
+===== Building the jail =====
+The simplest solution is to re-use already existing BSDRP patched source tree: specific kernel configuration files can be installed into these source tree and use after.
+Start by only patching BSDRP sources (sources and ports) using the BSDRP make.sh script:
+<code>
+./make.sh -U
+Update ONLY done
+</code>
+There is now 2 sources ready patched:
+  * /usr/local/BSDRP/BSDRP/FreeBSD/src (including BSDRP specific kernel configuration file)
+  * /usr/local/BSDRP/BSDRP/FreeBSD/ports
+<code>
+poudriere jail -c -j BSDRP -a amd64 -z BSDRP -b -m src=/usr/local/BSDRP/BSDRP/FreeBSD/src -K amd64
+</code>
+Command line details:
+  * -b: Build from source
+  * -c: create a jail
+  * -j: SHORT name for the jail (I can't use BSDRP-amd64-10.3R here because later it will generate a long directory name and long name aren't well supported)
+  * -m src=: Path to the patched source branch we want to use
+  * -z: Configuration set name, used for loading the BSDRP-src.conf
+  * -K: The kernel configuration file, was copied here during patching BSDRP code trees
+===== Creating port tree =====
+Now we need to create a port-tree using the patched existing port tree:
+<code>
+poudriere ports -c -p BSDRP-ports -m null -M /usr/local/BSDRP/BSDRP/FreeBSD/ports
+</code>
+===== Build packages =====
+poudriere native role, we just give the jail name, sets name and port-tree name to use then the list of packages.
+<code>
+poudriere bulk -j BSDRP -z BSDRP -p BSDRP-ports -f /usr/local/etc/poudriere.d/BSDRP-pkglist
+</code>
+===== Generating firmware image =====
+Here I'm instructing to build a 4GB image using the previous sets, jail, port-tree.
+<code>
+poudriere image -t firmware -s 4g -j BSDRP -p BSDRP-ports -z BSDRP -n BSDRP -h router.bsdrp.net -c /usr/local/BSDRP/BSDRP/Files/ -f /usr/local/etc/poudriere.d/BSDRP-pkglist
+</code>
+Command line explanation:
+  * -s: Size of full image size (same as the flash media)
+  * -n: Image name, will be use as the partition name too
+  * -h: Hostname configured on the image
+  * -f: List of package to be installed on the image
+  * -c: Directory tree to be copied on the image
+FIXME: It needs a hook at the end of image generation for advanced task like generating an mtree (used for host-IDS) or specific user creation.