- en
- fr
This is an old revision of the document!
Table of Contents
Poudriere image
Unofficial Poudriere technical resources
Why using poudriere in place of nanobsd ?
BSDRP use a heavy customized nanobsd script that include package generation. This code need to be adapted each time the port build infrastructure change.
Why not using the new shinny “poudriere image” feature that will avoid to re-invent the wheel ?
Externals links about Poudriere
Here are a list of docs and tutorials about Poudriere:
- Poudriere wiki: Official Wiki
- Building Packages with Poudriere : Presentation page on FreeBSD Handbook
- poudriere man page : The man page
Understanding Poudriere image
What is Poudriere ?
It's a shell script used for build package in a clean (jail) environment. But once you get a clean jail in one side, and a list of fresh generated package in other side, why not mix them together in a “nanobsd like” firmware ?
This is the “poudriere image” feature includes in poudriere-git-3.3.99.20220713 that is presented here.
Images generated by Poudriere
Poudriere can generate multiple “image” types (default is iso+zmfs):
- iso: ISO 9660 format image
- iso+mfs: ISO 9660 variant where the root filesystem is MFS mounted
- iso+zmfs (default): ISO 9660 variant where the root filesystem is LZ77 compressed and is MFS mounted.
- usb: GPT-layout prepared UFS2 image containing a UEFI boot loader.
- usb+mfs : variant where the root filesystem is MFS mounted
- usb+zmfs: variant where the root filesystem is LZ77 compressed and is MFS mounted.
- rawdisk: raw UFS2, softupdates-enabled, disk image
- zrawdisk: raw ZFS disk image
- tar: XZ-compressed tarball
- firmware: NanoBSD style image with a GPT partitions and a UEFI boot loader
- rawfirmware: raw disk image (Update image that includes only one system partition of the firmware?)
- embedded: u-boot ready embedded image
- zsnapshot: zfs snapshot full and incremental to be used in a jail
The 6 minimum steps to build a poudriere firmware image
Using poudriere from a ZFS is not mandatory but strongly advised.
- Install poudriere and configure it:
pkg install poudriere-devel echo "ZPOOL="`zpool list -H | cut -f1` >> /usr/local/etc/poudriere.conf
- Create a poudriere jail WITH a GENERIC kernel (by default kernel is not build & installed), here named “router”:
poudriere jail -c -j router -v 13.2-RELEASE -K GENERIC
- Create a port-tree using “poudriere ports”:
poudriere ports -c -p router-ports
- Generate list of ports to be build & added into the firmware image:
cat > ~/router-pkglist <<EOF sysutils/tmux net/frr8 net/bird2 net/mpd5 EOF
- Build the ports (ie: generate binary packages) from the corresponding jail using “poudriere bulk”:
poudriere bulk -j router -p router-ports -f ~/router-pkglist
- Generate your disk image (4Gb total, because 2 systems partitions of 2Gb) using “poudriere image”:
poudriere image -t firmware -j router -s 4g -p router-ports -h router -n router -f ~/router-pkglist (...) [00:00:15] Creating ESP image [00:00:15] ESP Image created [00:00:21] Image available at: /usr/local/poudriere/data/images/router.img
Comparing poudriere firmware image and nanobsd image
Final firmware file size
Start by checking poudriere firmware file size:
# ls -alh /usr/local/poudriere/data/images/router.img -rw-r--r-- 1 root wheel 3.8G Jul 22 19:24 /usr/local/poudriere/data/images/router.img
We've obtained a 3.8GiB disk image file… which fit into a marketed-size 4GB flash disk.
Partition scheme
The resulting images partition layout will have the same behavior than a nanobsd:
- 10M GPT partition with EFI bootloader (nanobsd uses a MBR scheme with BIOS bootloader)
- first 991M (calculated from user input) system partition called gpt/${IMAGENAME}1
- second system partition called gpt/${IMAGENAME}2
- configuration configuration partition (hard-coded to 32M) called gpt/cfg
- data partition (hard-coded to 32M) called gpt/data
# mdconfig -a -t vnode -f /usr/local/poudriere/data/images/router.img md0 # gpart show -l md0 => 4 7995515 md0 GPT (3.8G) 4 20480 1 (null) (10M) 20484 123 2 (null) (62K) 20607 3921920 3 router1 (1.9G) 3942527 3921920 4 router2 (1.9G) 7864447 65536 5 cfg (32M) 7929983 65536 6 data (32M) # mount /dev/gpt/router1 /mnt/ # df -h /mnt Filesystem Size Used Avail Capacity Mounted on /dev/gpt/router1 1.8G 1.5G 103M 94% /mnt
/etc ram disk
Poudriere image is correctly compliant to a nanobsd /etc & /var ramdisk:
root@router:~ # mount /dev/gpt/router1 on / (ufs, local, read-only) devfs on /dev (devfs, local, multilabel) /dev/md0 on /etc (ufs, local) /dev/md1 on /var (ufs, local)
Comparing to a standard nanobsd:
[root@nanobsd]~# mount /dev/ufs/NANOs1a on / (ufs, local, read-only) devfs on /dev (devfs, local) /dev/md0 on /etc (ufs, local) /dev/md1 on /var (ufs, local)
fstab
poudriere's fstab is using gpt labels, while nanobsd one using ufs labels:
root@router:~ # cat /etc/fstab /dev/gpt/router1 / ufs ro 1 1 /dev/gpt/cfg /cfg ufs rw,noatime,noauto 2 2 /dev/gpt/data /data ufs rw,noatime,noauto,failok 2 2
[root@nanobsd]~# cat /etc/fstab /dev/ufs/NANOs2a / ufs ro 1 1 /dev/ufs/NANOs3 /cfg ufs rw,noatime,noauto 2 2 /dev/ufs/NANOs4 /data ufs rw,noauto,failok 2 2
Migrating from NanoBSD to poudriere image
Features matrix comparison
nanobsd configuration file is a shell script,and BSDRP use a highly nanobsd customized configuration file generated by a wrapper (make.sh).
The challenge of migrating the customization done on nanobsd to poudriere image can be resumed here:
BSDRP customized nanobsd usage | poudriere image | Purpose |
---|---|---|
Support differents src.conf: One for buildworld and another for installword | src.conf used for buildworld and installworld, image-JAILNAME[-SET]-src.conf added for installworld/delete-old. Allow to use the -X exclude_list too | Using WITHOUT_INCLUDES or WITHOUT_DEBUG just for installworld but not for buildworld |
Downloading a specific source revision of a stable/releng/head | use svn url@rev (or custom source tree with '-b -m src=/usr/src') or using branches | Allow reproducing the same build on different systems |
Build a specific kernel file | Just had to install custom kernel int the custom source tree | Customized kernel |
Build only list of kernel modules | Declaring MODULES_OVERRIDE in -src.conf | Customized module list |
Advanced scripting for building non-ports softwares | Not available, no idea of how to cleanly add this feature | Need to build some of /usr/src/tools |
Advanced tuning of final image | -A post-script and -B pre-script | Generating a mtree (for host-IDS), creating specific users |
Building ports using own (outdated) embedded script | NATIVE role of poudriere | Adding ports to image |
System upgrade by changing MBR active mode on system partition | System upgrade by setting “bootonce” attribute on system partition (and need to be removed from older?) | Upgrading system partition (new/old) |
Customized poudriere image for BSDRP
How to configure poudriere to generate a BSDRP firmware image.
Set of configuration files
To be able to reproduce the highly customized BSDRP firmware image, we need multiples configuration files.
Poudriere will requiere jail and port so prepending j and p to avoid confusion:
- BSDRPj for the jail name
- BSDRPp for the port tree name
We need to start creating a set of configuration files, named prefixed with the name BSDRP-:
- poudriere.d/BSDRPj-src.conf : Include all src.conf parameters used for reference jail buildworld/installworld
- poudriere.d/BSDRPj-make.conf : Include all common ports parameters (and ports build options)
- poudriere.d/image-BSDRPj-src.conf : parameters added for installworld (followed by a delete-old)
Then need other configuration files:
- BSDRP-pkglist : List of packages to be build and included in the final image
- A kernel configuration file: I'm using the BSDRP amd64 configuration
- excluded.files: A list of file we want to be exculed during the installworld
- overlaydir/usr/local/etc/pkg.conf: with a FILES_IGNORE_GLOB list that will exclude files from being extracted during packages installation
poudriere.d/BSDRP-src.conf
The previous section of the NanoBSD configuration files found in variables CONF_BUILD and CONF_WORLD in file BSDRP/BSDRP.nano should be copied in this file.
As for the BSDRP example, this give this file, that will include the MODULES_OVERRIDE list too
cat <<EOF > /usr/local/etc/poudriere.d/BSDRPj-src.conf BOOT_BOOT0_COMCONSOLE_SPEED=0 WITHOUT_ACCT= WITHOUT_AMD= WITHOUT_APM= WITHOUT_ASSERT_DEBUG= WITHOUT_ATF= WITHOUT_ATM= WITHOUT_AUDIT= WITHOUT_AUTHPF= WITHOUT_AUTOFS= WITHOUT_BHYVE= WITHOUT_BLUETOOTH= WITHOUT_BOOTPARAMD= WITHOUT_BOOTPD= WITHOUT_BSDINSTALL= WITHOUT_CALENDAR= WITHOUT_CCD= WITHOUT_CROSS_COMPILER= WITHOUT_CTM= WITHOUT_CVS= WITHOUT_DICT= WITHOUT_EE= WITHOUT_EXAMPLES= WITHOUT_FINGER= WITHOUT_FLOPPY= WITHOUT_FREEBSD_UPDATE= WITHOUT_GAMES= WITHOUT_GCOV= WITHOUT_GNUCXX= WITHOUT_GPIB= WITHOUT_GPIO= WITHOUT_GROFF= WITHOUT_HAST= WITHOUT_HTML= WITHOUT_INFO= WITHOUT_IPX= WITHOUT_IPX_SUPPORT= WITHOUT_ISCSI= WITHOUT_KERBEROS= WITHOUT_LIB32= WITHOUT_LINT= WITHOUT_LLVM_ASSERTIONS= WITHOUT_LLVM_TARGET_ALL= WITHOUT_LOADER_FIREWIRE= WITHOUT_LOADER_GELI= WITHOUT_LOCATE= WITHOUT_LPR= WITHOUT_NCP= WITHOUT_NDIS= WITHOUT_NIS= WITHOUT_NLS= WITHOUT_NLS_CATALOGS= WITHOUT_NS_CACHING= WITHOUT_PC_SYSINSTALL= WITHOUT_PORTSNAP= WITHOUT_PROFILE= WITHOUT_QUOTAS= WITHOUT_RBOOTD= WITHOUT_RCMDS= WITHOUT_RCS= WITHOUT_RESCUE= WITHOUT_ROUTED= WITHOUT_SENDMAIL= WITHOUT_SERVICESDB= WITHOUT_SHAREDOCS= WITHOUT_SVNLITE= WITHOUT_SYSCONS= WITHOUT_SYSINSTALL= WITHOUT_TALK= WITHOUT_TESTS= WITHOUT_TESTS_SUPPORT= WITHOUT_TFTP= WITHOUT_TIMED= WITHOUT_UNBOUND= WITHOUT_USB_GADGET_EXAMPLES= WITHOUT_WIRELESS= WITHOUT_WPA_SUPPLICANT_EAPOL= WITHOUT_ZFS= WITH_IDEA= WITH_OFED= WITH_DEBUG_FILES= WITH_RETPOLINE= WITH_REPRODUCIBLE_BUILD= WITH_MALLOC_PRODUCTION= WITH_OPENSSL_KTLS= MODULES_OVERRIDE= \ backlight \ blake2 \ bridgestp \ carp \ dtrace \ dummynet \ fdescfs \ if_bridge \ if_disc \ if_epair \ if_gre \ if_infiniband \ if_lagg \ if_stf \ if_tuntap \ if_vxlan \ ipdivert \ ipfilter \ ipfw \ ipfw_nat \ ipfw_nat64 \ ipfw_pmod \ ipfw_nptv6 \ ipsec \ i2c \ fib_dxr \ ksyms \ libalias \ mlx4 \ mlx4en \ netgraph \ nullfs \ opensolaris \ pf \ pfsync \ pflog \ rc4 \ unionfs \ usb/uether \ usb/urndis \ amdsbwd \ amdsmn \ amdtemp \ cpuctl \ coretemp \ dpdk_lpm4 \ dpdk_lpm6 \ ena \ ichwd \ ioat \ ipmi \ ispfw \ hifn \ hyperv \ qat \ qatfw \ qat_c2xxx \ hwpmc \ padlock \ qlxgb \ qlxgbe \ safe \ vmware EOF
poudriere.d/image-BSDRPj-src.conf
Allow to ADD WITHOUT that will be removed during installworld
cat <<EOF > /usr/local/etc/poudriere.d/image-BSDRPj-src.conf WITHOUT_DEBUG_FILES= WITHOUT_TOOLCHAIN= WITHOUT_INCLUDES= EOF
poudriere.d/BSDRPj-make.conf
This file contains configuration parameters common for all ports.
Example:
OPTIONS_UNSET+=DOCS EXAMPLES INFO NLS X11 DOXYGEN lang_perl5.32_UNSET+=DTRACE textproc_expat2_SET+=STATIC net_frr9_SET+=MULTIPATH net_openldap25-client_UNSET+=GSSAPI security_ipsec-tools_SET+=RC5 IDEA RADIUS LDAP security_strongswan_SET+=MEDIATION GCM security_strongswan_UNSET+=EAPAKA3GPP2 EAPSIMFILE SQLITE UNBOUND LOADTESTER TESTVECTOR security_openvpn_SET+=DCO benchmarks_netperf_UNSET+=OMNI sysutils_flashrom_UNSET+=DMIDECODE FTDI sysutils_mstflint_SET+=FWMANAGER ADAB net_mtr_UNSET+=JSON ftp_curl_SET+=GSSAPI_NONE ftp_curl_UNSET+=GSSAPI_BASE IMAP GOPHER HTTP2 LIBSSH2 POP3 PSL RTSP SMTP TELNET
poudriere.d/BSDRPj-BSDRPp-options/$CATEGORY_$PORTNAME/options
If the ports builded need to use specific options, they need to be declared.
Using the command line “poudriere options -j BSDRPj -p BSDRPp -n CATEGORY/PORTNAME” is the way of doing this easily.
BSDRP-pkglist
Now the list of package to be builded and added to the final image:
benchmarks/iperf benchmarks/iperf3 benchmarks/netperf editors/vim@tiny emulators/open-vm-tools@nox11 lang/perl5.32 lang/python net-mgmt/bgpq4 net-mgmt/bsnmp-regex net-mgmt/bsnmp-ucd net-mgmt/nrpe3 net-mgmt/pmacct net-mgmt/rtrlib net/arping net/bird2@netlink net/dhcp6 net/dhcprelya net/exabgp4 net/freevrrpd net/frr8 net/frr8-pythontools net/graphpath net/isc-dhcp44-server net/ixl_unlock net/mlvpn net/mpd5 net/mrouted net/mrtparse net/mtr net/nc net/netmap-fwd net/ntraceroute net/pimd net/pkt-gen net/quagga-bgp-netgen net/realtek-re-kmod net/tayga net/trafshow net/ucarp net/wireguard-tools ports-mgmt/pkg security/ca_root_nss security/ipsec-tools security/openvpn-devel security/strongswan security/sudo security/tinc sysutils/devcpu-data sysutils/dtrace-toolkit sysutils/flashrom sysutils/fswatch-mon sysutils/intel-pcm sysutils/ipmitool sysutils/monit sysutils/mstflint-lite sysutils/tmux sysutils/x86info
Building the jail
The simplest solution is to re-use already existing BSDRP patched source tree: specific kernel configuration files can be installed into these source tree and use after.
Start by only patching BSDRP sources (sources and ports) using the BSDRP make.sh script:
./make.sh -U Update ONLY done
There is now 2 sources ready patched:
- /usr/local/BSDRP/BSDRP/FreeBSD/src (including BSDRP specific kernel configuration file)
- /usr/local/BSDRP/BSDRP/FreeBSD/ports
poudriere jail -c -j BSDRPj -a amd64 -b -m src=/usr/local/BSDRP/BSDRP/FreeBSD/src -K amd64
Command line details:
- -b: Build from source
- -c: create a jail
- -j: SHORT name for the jail (I can't use BSDRP-amd64-10.3R here because later it will generate a long directory name and long name aren't well supported)
- -m src=: Path to the patched source branch we want to use
- -K: The kernel configuration file, was copied here during patching BSDRP code trees
Creating port tree
Now we need to create a port-tree using the patched existing port tree:
poudriere ports -c -p BSDRPp -m null -M /usr/local/BSDRP/BSDRP/FreeBSD/ports
Build packages
poudriere native role, we just give the jail name and port-tree name to use then the list of packages.
poudriere bulk -j BSDRPj -p BSDRPp -f /usr/local/etc/poudriere.d/BSDRP-pkglist
Generating firmware image
Here I'm instructing to build a 4GB image using the previous sets, jail, port-tree.
poudriere image -t firmware -s 4g -j BSDRPj -p BSDRPp -n BSDRP -h router.bsdrp.net -c /usr/local/BSDRP/BSDRP/Files/ -f /usr/local/etc/poudriere.d/BSDRP-pkglist
Command line explanation:
- -s: Size of full image size (same as the flash media)
- -n: Image name, will be use as the partition name too
- -h: Hostname configured on the image
- -f: List of package to be installed on the image
- -c: Directory tree to be copied on the image
: Needs to add a post-script to be added with option “-A post-script” for advanced task like generating an mtree (used for host-IDS) or specific user creation.