User Tools

Site Tools


Technical documentation for developers


The first step was to understand what is NanoBSD ?

How to build BSDRP images

All theses steps are done from a FreeBSD system (release 9.1 advised).


You need 3G of free space (1,5G for full FreeBSD+ports source, and 1,5G for the working dir).

Getting BSDRP source code

You can download the source code of BSDRP with svnlite (included in FreeBSD 10.0):

svnlite co BSDRP

or using git if you had installed it:

git clone BSDRP

Once you download the code, you can kept your BSDRP sources up-to-date with these commands:

svnlite update | git pull flowshart

Here is the big picture regarding BSDRP image build process:

 BSDRP build framework flowshart

Running the build script

Display the options proposed by the script :

./ -h

The FreeBSD code include all architecture and limited cross-compiling tools: You can generate an i386 BSDRP image from a FreeBSD amd64, but you can't generate a sparc64 image from an i386/amd64 host.

Current problems with make

Disk image creation can fail

On FreeBSD dektop, sometimes NanoBSD can't umount the md during image creation.

“fstat | grep BSDRP” display the name of the process locking the temporary mount point. It can be:

  • gam_server
  • gvfsd-trash

here is the log of a failed build:

### log : /usr/obj/BSDRP.amd64/_.di
Error encountered. Check errors in last log file.
Running exit trap code
root@dev # tail -n 4 /usr/obj/BSDRP.amd64/_.di
unmount: unmount of /usr/obj/BSDRP.amd64/_.mnt failed: Device busy


For preventing gamin to lock the working mount point, create this configuration file and force a restart of gamin:

mkdir /usr/local/etc/gamin
echo "fsset ufs poll 10" > /usr/local/etc/gamin/gaminrc
killall -9 gam_server


I don't know how to fix this gvfs bug.

How to generate customized BSDRP images

If you want to generate your customized BSDRP image, first try to build one generic BSDRP image from source.

You can check the files of the project “BSDRP” and it's child project “BSDRPcur”. A child project overwrite the MASTER_PROJECT files/kernels settings.

If this step works for you, then you can begin to customize it.

Customizing BSDRP in few slides

main files


This file include the main global information for building the image:

  • NAME: Name of the Project
  • MASTER_PROJECT: For a child project, name of the father project
  • SVN_SRC_PATH: SVN URL for the source tree
  • SVN_PORTS_PATH: SVN URL for the port source tree
  • FREEBSD_SRC: directory for localy stored FreeBSD source tree
  • SRC_PATCH_DIR: Directory for FreeBSD patches
  • PORTS_SRC: Directory for localy stored ports source tree
  • PORT_PATCH_DIR: Directory for port patches
  • NANOBSD_DIR: Where the nanobsd tree lives
  • NANO_MODULES_ARCH: List of kernel modules to build for ARCH


This file include all the customization steps:

You can change the:

  • Size of target BSDRP media (UsbDevice Generic)
  • Size of configuration partition (NANO_CONFSIZE)
  • Size of data partition (NANO_DATASIZE)
  • Size of the /etc RAM disk (NANO_RAM_ETCSIZE)
  • Size of the /tmp, /var disk (NANO_RAM_TMPVARSIZE)
  • and so one…

You can declare new package (and their dependency, but only if they use special option) to be installed: For example, if you want to add vim-lite, you add theses lines:

add_port "editors/vim-lite" "-DWITHOUT_X11 -DNO_INSTALL_MANPAGES -DNOPORTDOCS"

If you need to set special permission on some file after their installation, you need to add these steps on the bsdrp_custom () function.


This directory include the kernel configuration files. You need to edit these files and the NANO_MODULES_$ARCH variable in the make.conf for customizing your own kernel and modules.

$PROJECT/Files directory

All files put in the Files/ directory will be copied as it on the BSDRP image.

Little child project example

Here is a little example (minimum modification) for building a new project based on BSDRP, but for a web server appliance: This project will be a child project of BSDRP.

Start by downloading BSDRP source code (refers to getting BSDRP source code chapter) and go in the BSDRP directory.

Then create a new directory using your project name:

mkdir WEBSRV


Now we need to configure a minimum project configuration file:

echo 'NAME="WEBSRV"' > WEBSRV/make.conf
echo 'MASTER_PROJECT="BSDRP"' >> WEBSRV/make.conf

Listing all run-dependency of your ports

We want to add the port www/mohawk.

The first step is to list all run-deps of this port.

You need to have downloaded the FreeBSD port-tree first (automatically done if you've already generated one BSDRP image).

If you've installed BSDRP on /usr/local/BSDRP, here is an example for displaying the run-deps:

setenv PORTSDIR /usr/local/BSDRP/BSDRP/FreeBSD/ports
cd $PORTSDIR/www/mohawk
make run-depends-list

Here we need devel/libevent2 is a running dependency for www/mohawk.


Now we will copy the nanobsd configuration file from BSDRP:


Edit the file WEBSRV/WEBSRV.nano and delete all lines related to routing ports like that:

add_port "category/port_name" "build options"

And add (in the #### Ports list section #####) all the running dependency and your port:

add_port "devel/libevent2"
add_port "www/mohawk"

Remove theses lines that compile and install extra small tools too:

customize_cmd add_netrate
customize_cmd add_quagga-bgp-netgen

Then modify the bsdrp_custom () function in this file too for removing the quagga chown hack.


Now set-up the version number:

mkdir -p WEBSRV/Files/etc
echo '1' > WEBSRV/Files/etc/version 

Generating image

Now you should generate a full new image:

root@laptop:/usr/local/BSDRP # ./ -p WEBSRV
BSD Router Project image build script

Will generate an WEBSRV image with theses values:
- Target architecture: amd64
- Console : -vga
- Source Updating/installing: NO
- Build the full world (take about 1 hour): YES
- FAST mode (skip compression and checksumming): NO
- Debug image type: NO
Copying amd64 Kernel configuration file
Launching NanoBSD build process...
00:00:00 # NanoBSD image WEBSRV build starting
00:00:00 ## Clean and create object directory (/usr/obj/WEBSRV.amd64)
00:00:00 ## Construct build make.conf (/usr/obj/WEBSRV.amd64/
00:00:00 ## run buildworld
00:00:00 ### log: /usr/obj/WEBSRV.amd64/
00:15:03 ## build kernel (amd64)
00:15:03 ### log: /usr/obj/WEBSRV.amd64/_.bk
00:17:50 ## Clean and create world directory (/usr/obj/WEBSRV.amd64/_.w)
00:17:50 ## Construct install make.conf (/usr/obj/WEBSRV.amd64/make.conf.install)
00:17:50 ## installworld
00:17:50 ### log: /usr/obj/WEBSRV.amd64/_.iw
00:18:29 ## install /etc
00:18:29 ### log: /usr/obj/WEBSRV.amd64/_.etc
00:18:30 ## configure nanobsd /etc
00:18:30 ## install kernel (amd64)
00:18:30 ### log: /usr/obj/WEBSRV.amd64/_.ik
00:18:33 ## run customize scripts
00:18:33 ## customize "add_port_devel_libevent2"
00:18:33 ### log: /usr/obj/WEBSRV.amd64/_.cust.add_port_devel_libevent2
00:18:33 ## customize "add_port_www_mohawk"
00:18:33 ### log: /usr/obj/WEBSRV.amd64/_.cust.add_port_www_mohawk
00:18:34 ## customize "cleanup_ports"
00:18:34 ### log: /usr/obj/WEBSRV.amd64/_.cust.cleanup_ports
00:18:34 ## customize "shrink_md_fbsize"
00:18:34 ### log: /usr/obj/WEBSRV.amd64/_.cust.shrink_md_fbsize
00:18:34 ## customize "cust_install_files"
00:18:34 ### log: /usr/obj/WEBSRV.amd64/_.cust.cust_install_files
00:18:34 ## customize "bsdrp_custom"
00:18:34 ### log: /usr/obj/WEBSRV.amd64/_.cust.bsdrp_custom
00:18:38 ## customize "cust_allow_ssh_root"
00:18:38 ### log: /usr/obj/WEBSRV.amd64/_.cust.cust_allow_ssh_root
00:18:38 ## customize "bsdrp_console_vga"
00:18:38 ### log: /usr/obj/WEBSRV.amd64/_.cust.bsdrp_console_vga
00:18:38 ## configure nanobsd setup
00:18:38 ### log: /usr/obj/WEBSRV.amd64/_.dl
00:18:39 ## run late customize scripts
00:18:39 ## build diskimage
00:18:39 ### log: /usr/obj/WEBSRV.amd64/_.di
00:19:02 # NanoBSD image WEBSRV completed
unmounting  /usr/local/BSDRP/WEBSRV/kernels
NanoBSD build seems finish successfully.
Compressing WEBSRV upgrade image...
/usr/obj/WEBSRV.amd64/WEBSRV-1-upgrade-amd64-vga.img (1/1)
  100 %        26.2 MiB / 101.9 MiB = 0.257   3.0 MiB/s       0:34             
Generating checksum for WEBSRV upgrade image...
WEBSRV upgrade image file here:
Compressing WEBSRV full image...
/usr/obj/WEBSRV.amd64/WEBSRV-1-full-amd64-vga.img (1/1)
  100 %        26.2 MiB / 244.1 MiB = 0.107   5.3 MiB/s       0:45             
Generating checksum for WEBSRV full image...
Zipped WEBSRV full image file here:
Zipping and renaming mtree...
/usr/obj/WEBSRV.amd64/WEBSRV-1-amd64-vga.mtree (1/1)
  100 %      262.3 KiB / 1753.4 KiB = 0.150                                    
HIDS reference file here:
Done !

How to modify an existing image

All theses step are done from a FreeBSD system using a unziped BSDRP full-image.

Partition layout of a BSDRP image

Here are all partition found in a BSDRP-full image:

  • s1a: first system partition (UFS labeled as BSDRPs1a)
  • s2a: Second system partition, didn't exist if the system was never upgraded (UFS labeled as BSDRPs2a)
  • s3: cfg partition (UFS labeled as BSDRPs3)
  • s4: data partition (UFS labeled as BSDRPs4)

FreeBSD call “slide” (s) an MBR partition.

Mounting a BSDRP image as a memory disk

Automated way

Use the script include with BSDRP sources: ./ mount <filename> ./ umount

Manual way

Using the BSDRP image file, create a memory disk (md):

mdconfig -a -t vnode -f BSDRP_0.35_full_i386_serial.img -x 63 -y 16

The system will display the md name created:


Now list all partitions on this md:

# ls /dev/md0*
# /dev/md0    /dev/md0s1  /dev/md0s1a /dev/md0s2  /dev/md0s3

You should found the s1a (system) and s3 (cfg) partition. Mount the system partition for example:

mount /dev/md0s1a /mnt

Now you can do all your changes on the image.

And the end, umount and close the memory disk:

umount /mnt
mdconfig -d -u 0

Increasing /etc and /var RAM disk size

Mount the filesystem in read-write mode:

[root@BSDRP]/# mount -uw /

And change the value in these files:

  • /conf/base/etc/md_size
  • /conf/base/var/md_size

The remount the filesystem in read-only and reboot (answer “no” if it detect configuration changes)

[root@BSDRP]/conf/base/var# mount -ur /
[root@BSDRP]/conf/base/var# reboot

How to debug

Here are some advice for debugging.

Performance optimization

You can found more information about FreeBSD on the FreeBSD forwarding Performance page.

Shell script

Start your script with “sh -x”:

[root@router]~#sh -x /usr/local/sbin/system

RC script

If you want to create some RC scripts, you need to start by reading Practical rc.d scripting in BSD.

Saving a crash dump

For extracting a crash dump you need:

  • A data partition with a size for storing the extracted dump file
  • A swap partition with a size for the RAM dump

If you've installed BSDRP on a large disk that have enough space you can:

  1. Expand partition 4 (/data) to full disk space
  2. Create 2 BSD partitions in the large partition 4:
    • partition s4a (4.2BSD) with a size= half size of your RAM (because extracted dump file is zipped)
    • partition s4b (swap) with a size= RAM size
  3. Format s4a in UFS and enable dumpon on the swap

If you can plug an USB key with a size=RAM size, you can avoid to expand your partition 4 and just using this device.

Here is an example if BSDRP is installed on a big “da0” disk with a 16G RAM system:

gpart delete -i 4 da0
gpart add -t freebsd da0
bsdlabel -w /dev/da0s4 
bsdlabel -e /dev/da0s4

# /dev/da0s4:
8 partitions:
#          size     offset    fstype   [fsize bsize bps/cpg]
  a:   8G          16        4.2BSD        0     0     0
  b:   16G          *         swap
  c:  284191428          0    unused        0     0     # "raw" part, don't edit

newfs -UjL BSDRPs4 /dev/da0s4a
dumpon /dev/da0s4b && ln -fs /dev/da0s4b /dev/dumpdev

And after a crash/reboot, here is how to extract the dump:

savecore -C /dev/dumpdev
mount /data
savecore -z /data /dev/dumpdev
documentation/technical_docs.txt · Last modified: 2015/05/26 11:12 by olivier