User Tools

Site Tools


documentation:examples:multi-tenant_router_and_firewall

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

documentation:examples:multi-tenant_router_and_firewall [2017/08/31 16:25] (current)
Line 1: Line 1:
 +====== Multi-tenant router or firewall ======
 +{{description>​Multi-tenant router and firewall example}}
 +This lab shows how to create multi-tenant router or firewall using jail/vnet (available since BSDRP 1.80).
 +
 +===== Presentation =====
 +
 +==== Network diagram ====
 +
 +Lab build following [[documentation:​examples:​How to build a BSDRP router lab]]: 5 routers with full-meshed link and one shared LAN.
 +
 +Here is the logical and physical view:
 +
 +{{:​documentation:​examples:​multi-tenant-router-firewall.png}}
 +
 +==== Setting-up a virtual lab ====
 +
 +=== Downloading BSD Router Project images ===
 +
 +Download BSDRP serial image (prevent to have to use an X display) on Sourceforge.
 +
 +=== Download Lab scripts ====
 +
 +More information on these BSDRP lab scripts available on [[documentation:​examples:​How to build a BSDRP router lab]].
 +
 +Start the lab with full-meshed 5 routers and one shared LAN, on this example using bhyve lab script on FreeBSD:
 +
 +<​code>​
 +[root@FreeBSD]~#​ tools/​BSDRP-lab-bhyve.sh -i BSDRP-1.71-full-amd64-serial.img.xz -n 5 -l 1
 +BSD Router Project (http://​bsdrp.net) - bhyve full-meshed lab script
 +Setting-up a virtual lab with 5 VM(s):
 +- Working directory: /tmp/BSDRP
 +- Each VM have 1 core(s) and 256M RAM
 +- Switch mode: bridge + tap
 +- 1 LAN(s) between all VM
 +- Full mesh Ethernet links between each VM
 +VM 1 have the following NIC:
 +- vtnet0 connected to VM 2
 +- vtnet1 connected to VM 3
 +- vtnet2 connected to VM 4
 +- vtnet3 connected to VM 5
 +- vtnet4 connected to LAN number 1
 +VM 2 have the following NIC:
 +- vtnet0 connected to VM 1
 +- vtnet1 connected to VM 3
 +- vtnet2 connected to VM 4
 +- vtnet3 connected to VM 5
 +- vtnet4 connected to LAN number 1
 +VM 3 have the following NIC:
 +- vtnet0 connected to VM 1
 +- vtnet1 connected to VM 2
 +- vtnet2 connected to VM 4
 +- vtnet3 connected to VM 5
 +- vtnet4 connected to LAN number 1
 +VM 4 have the following NIC:
 +- vtnet0 connected to VM 1
 +- vtnet1 connected to VM 2
 +- vtnet2 connected to VM 3
 +- vtnet3 connected to VM 5
 +- vtnet4 connected to LAN number 1
 +VM 5 have the following NIC:
 +- vtnet0 connected to VM 1
 +- vtnet1 connected to VM 2
 +- vtnet2 connected to VM 3
 +- vtnet3 connected to VM 4
 +- vtnet4 connected to LAN number 1
 +For connecting to VM'​serial console, you can use:
 +- VM 1 : cu -l /dev/nmdm1B
 +- VM 2 : cu -l /dev/nmdm2B
 +- VM 3 : cu -l /dev/nmdm3B
 +- VM 4 : cu -l /dev/nmdm4B
 +- VM 5 : cu -l /dev/nmdm5B
 +</​code>​
 +===== Configuration =====
 +
 +  * Router 4 (R4) hosts the 3 routers/​firewalls for each 3 customers.
 +  * Router 1 (R1) belongs to customer 1, router 2 (R2) to customer 2 and router 3 (R3) to customer 3.
 +  * Router 5 (R5) simulates a simple Internet host.
 +
 +==== Router 5: Simple Internet host ====
 +
 +R5 simulate a simple Internet host:
 +
 +<​code>​
 +sysrc hostname=R5
 +hostname R5
 +sysrc ifconfig_vtnet3="​inet 10.254.254.5/​24"​
 +sysrc -x gateway_enable
 +sysrc -x ipv6_gateway_enable
 +service netif restart
 +service routing restart
 +config save
 +</​code>​
 +
 +==== Router 1: Customer 1 workstation====
 +
 +R1 simulate customer 1's workstation,​ generate customer 1' SSH keys:
 +
 +<​code>​
 +sysrc hostname=R1
 +hostname R1
 +sysrc ifconfig_vtnet4="​up"​
 +sysrc vlans_vtnet4="​1"​
 +sysrc ifconfig_vtnet4_1="​inet 10.0.0.1/​24"​
 +sysrc defaultrouter="​10.0.0.254"​
 +sysrc -x gateway_enable
 +sysrc -x ipv6_gateway_enable
 +service netif restart
 +service routing restart
 +ssh-keygen -f /​root/​.ssh/​id_rsa -N ''​
 +config save
 +</​code>​
 +
 +Then display the public SSH key (need to declare it into the customer 1's firewall):
 +<​code>​
 +cat .ssh/​id_rsa.pub
 +ssh-rsa (...) root@R1
 +</​code>​
 +
 +==== Router 2: Customer 2 workstation====
 +
 +R2 simulate customer 2's workstation,​ and host customer 2 SSH keys too.
 +
 +<​code>​
 +sysrc hostname=R2
 +hostname R2
 +sysrc ifconfig_vtnet4="​up"​
 +sysrc vlans_vtnet4="​2"​
 +sysrc ifconfig_vtnet4_2="​inet 10.0.0.1/​24"​
 +sysrc defaultrouter="​10.0.0.254"​
 +sysrc -x gateway_enable
 +sysrc -x ipv6_gateway_enable
 +service netif restart
 +service routing restart
 +ssh-keygen -f /​root/​.ssh/​id_rsa -N ''​
 +config save
 +</​code>​
 +
 +Then display the public SSH key (need to declare it into the customer 2's firewall):
 +<​code>​
 +cat .ssh/​id_rsa.pub
 +ssh-rsa (...) root@R2
 +</​code>​
 +==== Router 3: Customer 3 workstation====
 +
 +R3 simulate customer 3's workstation,​ and host customer 3 SSH keys too.
 +
 +<​code>​
 +sysrc hostname=R3
 +hostname R3
 +sysrc ifconfig_vtnet4="​up"​
 +sysrc vlans_vtnet4="​3"​
 +sysrc ifconfig_vtnet4_3="​inet 10.0.0.1/​24"​
 +sysrc defaultrouter="​10.0.0.254"​
 +sysrc -x gateway_enable
 +sysrc -x ipv6_gateway_enable
 +service netif restart
 +service routing restart
 +ssh-keygen -f /​root/​.ssh/​id_rsa -N ''​
 +config save
 +</​code>​
 +
 +Then display the public SSH key (need to declare it into the customer 3's firewall):
 +<​code>​
 +cat .ssh/​id_rsa.pub
 +ssh-rsa (...) root@R3
 +</​code>​
 +==== Router 4: multi-tenant ipfw firewall ====
 +
 +Router 4 is a multi-tenant ipfw firewall: It hosts 3 firewalls for each customer.
 +
 +Then we will configure:
 +  * Bridge and VLAN interfaces
 +  * Enable ipfw (firewall modules needs to be loaded on the host for being available into jails)
 +
 +<​code>​
 +sysrc hostname=R4
 +sysrc cloned_interfaces="​bridge0"​
 +sysrc ifconfig_bridge0="​inet 10.254.254.4/​24 addm vtnet3"​
 +sysrc ifconfig_vtnet3="​up"​
 +sysrc ifconfig_vtnet4="​up"​
 +sysrc vlans_vtnet4="​1 2 3"
 +sysrc ifconfig_vtnet4_1="​up"​
 +sysrc ifconfig_vtnet4_2="​up"​
 +sysrc ifconfig_vtnet4_3="​up"​
 +sysrc firewall_enable="​YES"​
 +sysrc firewall_nat_enable="​YES"​
 +sysrc firewall_type="​open"​
 +service netif restart
 +hostname R4
 +service ipfw start
 +config save
 +</​code>​
 +
 +Then install the customers SSH public keys:
 +
 +<​code>​
 +echo "​ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABA... root@R1"​ > /​tmp/​cust1.ssh.pub
 +echo "​ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABA... root@R2"​ > /​tmp/​cust2.ssh.pub
 +echo "​ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABA... root@R3"​ > /​tmp/​cust3.ssh.pub
 +</​code>​
 +
 +Create 3 jailed firewalls, one for each customers:
 +<​code>​
 +tenant -c -j customer1 -f /​tmp/​cust1.ssh.pub -i bridge0/​10.254.254.1/​24,​vtnet4.1/​10.0.0.254/​24 -g 10.254.254.5
 +tenant -c -j customer2 -f /​tmp/​cust2.ssh.pub -i bridge0/​10.254.254.2/​24,​vtnet4.2/​10.0.0.254/​24 -g 10.254.254.5
 +tenant -c -j customer3 -f /​tmp/​cust3.ssh.pub -i bridge0/​10.254.254.3/​24,​vtnet4.3/​10.0.0.254/​24 -g 10.254.254.5
 +</​code>​
 +
 +Last step, because they are virtual firewalls and not simple routers, we will enable firewall in open mode into their internal rc.conf for allowing customers to SSH into them:
 +
 +<​code>​
 +sysrc -f /​etc/​jails/​customer1/​rc.conf firewall_enable="​YES" ​
 +sysrc -f /​etc/​jails/​customer1/​rc.conf firewall_nat_enable="​YES"​
 +sysrc -f /​etc/​jails/​customer1/​rc.conf firewall_type="​open"​
 +sysrc -f /​etc/​jails/​customer2/​rc.conf firewall_enable="​YES" ​
 +sysrc -f /​etc/​jails/​customer2/​rc.conf firewall_nat_enable="​YES"​
 +sysrc -f /​etc/​jails/​customer2/​rc.conf firewall_type="​open"​
 +sysrc -f /​etc/​jails/​customer3/​rc.conf firewall_enable="​YES" ​
 +sysrc -f /​etc/​jails/​customer3/​rc.conf firewall_nat_enable="​YES"​
 +sysrc -f /​etc/​jails/​customer3/​rc.conf firewall_type="​open"​
 +</​code>​
 +
 +Configuration will be now automatically saved when changed detected into /etc, then you do not need to use "​config save" on the host once a jail is created.
 +
 +Then to start the jails:
 +
 +<​code>​
 +service jail start
 +</​code>​
 +
 +
 +===== Customers firewalls configuration =====
 +
 +Each customer should be able to ssh into their new firewalls using their SSH keys.
 +
 +==== Customer 1 ====
 +
 +From customer 1's workstation R1:
 +<​code>​
 +[root@R1]~# ssh 10.0.0.254
 +The authenticity of host '​10.0.0.254 (10.0.0.254)'​ can't be established.
 +ECDSA key fingerprint is SHA256:​extHiTI3L94Ks1TPnMI66zq+4t+frkAnvRVSkYk3qak.
 +No matching host key fingerprint found in DNS.
 +Are you sure you want to continue connecting (yes/no)? yes
 +Warning: Permanently added '​10.0.0.254'​ (ECDSA) to the list of known hosts.
 +BSD Router project (BSDRP) (c) 2009-2017, The BSDRP Development Team
 +All rights reserved.
 +BSDRP is under the Simplified BSD license.
 +
 +Documentation:​ http://​bsdrp.net
 +
 +Discover BSDRP tools with "​help"​ command
 +
 +Keyboard layout can be changed with this command:
 +kbdcontrol -l keymap_file (<​TAB>​ for list available maps)
 +root has logged on pts/0 from 10.0.0.1.
 +[root@customer1]~#​
 +</​code>​
 +
 +Now connected to his firewall, this customer can configure its own firewall rules:
 +
 +<​code>​
 +sysrc -x firewall_type
 +sysrc firewall_script="/​etc/​ipfw.rules"​
 +
 +cat > /​etc/​ipfw.rules <<'​EOF'​
 +#!/bin/sh
 +fwcmd="/​sbin/​ipfw -q"
 +ext_if="​epair1b"​
 +int_if="​vtnet4.1"​
 +${fwcmd} -f flush
 +${fwcmd} nat 1 config if ${ext_if} same_ports deny_in unreg_only reset
 +${fwcmd} add pass ip from any to any via lo0
 +${fwcmd} add pass ip from any to any via ${int_if}
 +${fwcmd} add nat 1 ip from any to any via ${ext_if}
 +'​EOF'​
 +service ipfw restart
 +config save
 +</​code>​
 +
 +Check firewall rules:
 +<​code>​
 +[root@customer1]~#​ ipfw show
 +00100  0    0 allow ip from any to any via lo0
 +00200 91 7756 allow ip from any to any via vtnet4.1
 +00300  0    0 nat 1 ip from any to any via epair1b
 +65535  0    0 deny ip from any to any
 +</​code>​
 +
 +Now, from R1, try to reach public Internet server R5:
 +
 +<​code>​
 +[root@R1]~# ping -c 3 10.254.254.5
 +PING 10.254.254.5 (10.254.254.5):​ 56 data bytes
 +64 bytes from 10.254.254.5:​ icmp_seq=0 ttl=63 time=0.211 ms
 +64 bytes from 10.254.254.5:​ icmp_seq=1 ttl=63 time=0.186 ms
 +64 bytes from 10.254.254.5:​ icmp_seq=2 ttl=63 time=0.181 ms
 +
 +--- 10.254.254.5 ping statistics ---
 +3 packets transmitted,​ 3 packets received, 0.0% packet loss
 +round-trip min/​avg/​max/​stddev = 0.181/​0.193/​0.211/​0.013 ms
 +</​code>​
 +
 +==== Customer 2 ====
 +
 +From customer 2's workstation R1:
 +<​code>​
 +[root@R2]~# ssh 10.0.0.254
 +The authenticity of host '​10.0.0.254 (10.0.0.254)'​ can't be established.
 +ECDSA key fingerprint is SHA256:​zC+ryVAd9v1lTvSb+THFj5i8aYfFi2I6VvayF1TIhVo.
 +No matching host key fingerprint found in DNS.
 +Are you sure you want to continue connecting (yes/no)? yes
 +Warning: Permanently added '​10.0.0.254'​ (ECDSA) to the list of known hosts.
 +BSD Router project (BSDRP) (c) 2009-2017, The BSDRP Development Team
 +All rights reserved.
 +BSDRP is under the Simplified BSD license.
 +
 +Documentation:​ http://​bsdrp.net
 +
 +Discover BSDRP tools with "​help"​ command
 +
 +Keyboard layout can be changed with this command:
 +kbdcontrol -l keymap_file (<​TAB>​ for list available maps)
 +root has logged on pts/0 from 10.0.0.1.
 +[root@customer2]~#​
 +</​code>​
 +
 +Now connected to his firewall, this customer can configure its own firewall rules:
 +
 +<​code>​
 +sysrc -x firewall_type
 +sysrc firewall_script="/​etc/​ipfw.rules"​
 +
 +cat > /​etc/​ipfw.rules <<'​EOF'​
 +#!/bin/sh
 +fwcmd="/​sbin/​ipfw -q"
 +ext_if="​epair2b"​
 +int_if="​vtnet4.2"​
 +${fwcmd} -f flush
 +${fwcmd} nat 1 config if ${ext_if} same_ports deny_in unreg_only reset
 +${fwcmd} add pass ip from any to any via lo0
 +${fwcmd} add pass ip from any to any via ${int_if}
 +${fwcmd} add nat 1 ip from any to any via ${ext_if}
 +'​EOF'​
 +service ipfw restart
 +config save
 +</​code>​
 +
 +Check firewall rules:
 +<​code>​
 +[root@customer2]~#​ ipfw show
 +00100  0    0 allow ip from any to any via lo0
 +00200 91 7756 allow ip from any to any via vtnet4.2
 +00300  0    0 nat 1 ip from any to any via epair2b
 +65535  0    0 deny ip from any to any
 +</​code>​
 +
 +Now, from R2, try to public Internet server R5:
 +
 +<​code>​
 +[root@R2]~# ping -c 3 10.254.254.5
 +PING 10.254.254.5 (10.254.254.5):​ 56 data bytes
 +64 bytes from 10.254.254.5:​ icmp_seq=0 ttl=63 time=0.211 ms
 +64 bytes from 10.254.254.5:​ icmp_seq=1 ttl=63 time=0.186 ms
 +64 bytes from 10.254.254.5:​ icmp_seq=2 ttl=63 time=0.181 ms
 +
 +--- 10.254.254.5 ping statistics ---
 +3 packets transmitted,​ 3 packets received, 0.0% packet loss
 +round-trip min/​avg/​max/​stddev = 0.181/​0.193/​0.211/​0.013 ms
 +</​code>​
 +
 +==== Customer 3 ====
 +
 +From customer 3's workstation R1:
 +<​code>​
 +[root@R3]~# ssh 10.0.0.254
 +The authenticity of host '​10.0.0.254 (10.0.0.254)'​ can't be established.
 +ECDSA key fingerprint is SHA256:​iCkc1w5zzeQL+X3qyEwovuEAGNvD+rfftsitMAlK+Xk.
 +No matching host key fingerprint found in DNS.
 +Are you sure you want to continue connecting (yes/no)? yes
 +Warning: Permanently added '​10.0.0.254'​ (ECDSA) to the list of known hosts.
 +BSD Router project (BSDRP) (c) 2009-2017, The BSDRP Development Team
 +All rights reserved.
 +BSDRP is under the Simplified BSD license.
 +
 +Documentation:​ http://​bsdrp.net
 +
 +Discover BSDRP tools with "​help"​ command
 +
 +Keyboard layout can be changed with this command:
 +kbdcontrol -l keymap_file (<​TAB>​ for list available maps)
 +root has logged on pts/0 from 10.0.0.1.
 +[root@customer3]~#​
 +</​code>​
 +
 +Now connected to his firewall, this customer can configure its own firewall rules:
 +
 +<​code>​
 +sysrc -x firewall_type
 +sysrc firewall_script="/​etc/​ipfw.rules"​
 +
 +cat > /​etc/​ipfw.rules <<'​EOF'​
 +#!/bin/sh
 +fwcmd="/​sbin/​ipfw -q"
 +ext_if="​epair3b"​
 +int_if="​vtnet4.3"​
 +${fwcmd} -f flush
 +${fwcmd} nat 1 config if ${ext_if} same_ports deny_in unreg_only reset
 +${fwcmd} add pass ip from any to any via lo0
 +${fwcmd} add pass ip from any to any via ${int_if}
 +${fwcmd} add nat 1 ip from any to any via ${ext_if}
 +'​EOF'​
 +service ipfw restart
 +config save
 +</​code>​
 +
 +Check firewall rules:
 +<​code>​
 +[root@customer3]~#​ ipfw show
 +00100  0    0 allow ip from any to any via lo0
 +00200 91 7756 allow ip from any to any via vtnet4.3
 +00300  0    0 nat 1 ip from any to any via epair3b
 +65535  0    0 deny ip from any to any
 +</​code>​
 +
 +Now, from R3, try to public Internet server R5:
 +
 +<​code>​
 +[root@R3]~# ping -c 3 10.254.254.5
 +PING 10.254.254.5 (10.254.254.5):​ 56 data bytes
 +64 bytes from 10.254.254.5:​ icmp_seq=0 ttl=63 time=0.211 ms
 +64 bytes from 10.254.254.5:​ icmp_seq=1 ttl=63 time=0.186 ms
 +64 bytes from 10.254.254.5:​ icmp_seq=2 ttl=63 time=0.181 ms
 +
 +--- 10.254.254.5 ping statistics ---
 +3 packets transmitted,​ 3 packets received, 0.0% packet loss
 +round-trip min/​avg/​max/​stddev = 0.181/​0.193/​0.211/​0.013 ms
 +</​code>​
 +===== Using pf firewall in place of ipfw =====
 +
 +pf need a little more configuration because by default /dev/pf is hidden from jail.
 +
 +Then, on the host we need to: 
 +  - In place of loading the ipfw/​ipfw-nat modules we need to load the pf module (but still disabling pf on our host for this example)
 +  - Modify default devd rules for allowing jails to see /dev/pf (if you want to use tcpdump inside your jail, you should use bpf device too)
 +  - Replacing nojail tag by nojailvnet tag into /​etc/​rc.d/​pf ([[https://​github.com/​ocochard/​BSDRP/​blob/​master/​BSDRP/​patches/​freebsd.rc.jailvnet.patch|already done into BSDRP]] and into [[https://​svnweb.freebsd.org/​base?​view=revision&​revision=320802|FreeBSD -head]])
 +
 +Preparing configuration:​
 +
 +<​code>​
 +cat > /​etc/​devfs.rules <<'​EOF'​
 +[devfsrules_jailpf=4]
 +add include $devfsrules_hide_all
 +add include $devfsrules_unhide_basic
 +add include $devfsrules_unhide_login
 +add path '​pf'​ unhide
 +'​EOF'​
 +pf_enable="​YES"​
 +pf_flags="​-d" ​
 +echo "set skip on {lo1 vtnet4}"​ > /​etc/​pf.conf
 +</​code>​
 +
 +Now reloading devd and loading pf module:
 +<​code>​
 +service pf start
 +service devfs restart
 +</​code>​
 +
 +You can now declare pf in place of ipfw into your jails rc.conf:
 +
 +<​code>​
 +sysrc -f /​etc/​jails/​customer1/​rc.conf pf_enable="​YES"​
 +echo "pass all" > /​etc/​jails/​customer1/​pf.conf
 +sysrc -f /​etc/​jails/​customer2/​rc.conf pf_enable="​YES"​
 +echo "pass all" > /​etc/​jails/​customer2/​pf.conf
 +sysrc -f /​etc/​jails/​customer3/​rc.conf pf_enable="​YES"​
 +echo "pass all" > /​etc/​jails/​customer3/​pf.conf
 +</​code>​
 +
 +Now you can start customers jails, and let customers SSH into their firewalls and configure their own rules:
 +<​code>​
 +[root@customer2]~#​ pfctl -s rules
 +pass all flags S/SA keep state
 +</​code>​
 +
 +
 +===== Under the hood: jails-on-nanobsd =====
 +
 +[[https://​github.com/​ocochard/​BSDRP/​blob/​master/​BSDRP/​Files/​usr/​local/​sbin/​tenant|BSDRP'​s tenant shell script]] creates jail configuration compliant with a host running nanobsd.
 +
 +Then these jails need to be configured for a nanobsd:
 +  - Being nullfs based for being hosted on a read-only root filesystem
 +  - Have  their /etc and /var into tmpfs disks (then we need to populate these directory before each start)
 +  - Configuration changes need to be saved with nanobsd configuration tools, like "​config save" on BSDRP
 +And on the host:
 +  - [[https://​github.com/​ocochard/​BSDRP/​blob/​master/​BSDRP/​Files/​usr/​local/​sbin/​autosave|autosave daemon]] need to be enabled: Each time a customer will issue a "​config save" inside a jail, his configuration diffs will be save into host's /​etc/​jails/​. And this directory is a RAM disk too, then we need to automatically save hosts configuration on changes.
 +
 +Here are examples of configuration files generated:
 +
 +host jail.conf: ​
 +
 +<​code>​
 +customer1 {
 +    jid = 1;
 +    path          = "/​var/​jails/​customer1";​
 +    # Because we are using jail on nanobsd, the jail directories are volatil (mounted into /var/jails)
 +    # They didn't exist after a reboot, then we need to create jail directories with exec.prestart
 +    # But mount.* instructions are called before exec.prestart,​ then we need to call mount manually
 +    # into the exec.prestart
 +    #​mount.devfs;​
 +    #​mount.fstab ​  = "/​etc/​fstab.customer1";​
 +    #​devfs_ruleset = 4;
 +    host.hostname = "​customer1";​
 +    vnet new;
 +    allow.chflags = 1;
 +    exec.start ​   = "/​bin/​sh /​etc/​rc";​
 +    exec.stop ​    = "/​bin/​sh /​etc/​rc.shutdown";​
 +    exec.clean;
 +    exec.consolelog = "/​var/​log/​jail.customer1";​
 +    exec.poststop ​ = "​logger poststop jail customer1";​
 +    # Commands to run on host before jail is created
 +    exec.prestart ​ = "​logger pre-starting jail customer1";​
 +    exec.prestart ​ += "mkdir -p /​var/​jails/​customer1/​dev";​
 +    exec.prestart ​ += "mkdir -p /​var/​jails/​customer1/​etc";​
 +    exec.prestart ​ += "mkdir -p /​var/​jails/​customer1/​var";​
 +    exec.prestart ​ += "mkdir -p /​var/​jails/​customer1/​cfg";​
 +    exec.prestart ​ += "mkdir -p /​var/​jails/​customer1/​root";​
 +    exec.prestart ​ += "mkdir -p /​var/​jails/​customer1/​bin";​
 +    exec.prestart ​ += "mkdir -p /​var/​jails/​customer1/​sbin";​
 +    exec.prestart ​ += "mkdir -p /​var/​jails/​customer1/​lib";​
 +    exec.prestart ​ += "mkdir -p /​var/​jails/​customer1/​libexec";​
 +    exec.prestart ​ += "mkdir -p /​var/​jails/​customer1/​usr";​
 +    exec.prestart ​ += "mkdir -p /​var/​jails/​customer1/​conf/​base";​
 +    exec.prestart ​ += "test -L /​var/​jails/​customer1/​tmp || ln -s /var/tmp /​var/​jails/​customer1/​tmp";​
 +    exec.prestart ​ += "mount -F /​etc/​fstab.customer1 -a";
 +    exec.prestart ​ += "mount -t devfs -o rw,​ruleset=4 devfs /​var/​jails/​customer1/​dev";​
 +
 +    # Copy reference and backuped files to /etc
 +    exec.prestart ​ += "cp -a /conf/base/ /​var/​jails/​customer1";​
 +    exec.prestart ​ += "cp -a /​etc/​jails/​customer1/​ /​var/​jails/​customer1/​etc/";​
 +    # Prevent diskless
 +    exec.prestart ​ += "test -f /​var/​jails/​customer1/​etc/​diskless && rm /​var/​jails/​customer1/​etc/​diskless
 +|| true";
 +    vnet.interface ​ += "​epair1b";​
 +    exec.prestart ​ += "​ifconfig epair1 create up";
 +    exec.prestart ​ += "​ifconfig epair1a up";
 +    exec.prestart ​ += "​ifconfig bridge0 addm epair1a up";
 +    # fix bug that assing same MAC to all epairXb interface
 +    # TO DO: convert this id into hexa
 +    exec.prestart ​ += "​ifconfig epair1b ether 02:​ff:​00:​00:​ff:​1";​
 +    exec.poststop ​ += "​ifconfig bridge0 deletem epair1a";​
 +    exec.poststop ​ += "​ifconfig epair1a destroy";​
 +    vnet.interface ​ += "​vtnet4.1";​
 +    exec.poststop ​ += "​ifconfig vtnet4.1 -vnet 1";
 +    exec.prestart ​ += "​logger jail customer1 pre-started";​
 +    exec.poststop ​ += "​umount /​var/​jails/​customer1/​dev";​
 +    exec.poststop ​ += "​umount -a -F /​etc/​fstab.customer1";​
 +    exec.poststop ​ += "​logger jail customer1 post-stopped";​
 +}
 +</​code>​
 +
 +/​etc/​fstab.customer1:​
 +<​code>​
 +tmpfs /​var/​jails/​customer1/​etc tmpfs rw,​size=16000000 0 0
 +tmpfs /​var/​jails/​customer1/​var tmpfs rw,​size=16000000 0 0
 +/root /​var/​jails/​customer1/​root nullfs ro 0 0
 +/bin /​var/​jails/​customer1/​bin nullfs ro 0 0
 +/sbin /​var/​jails/​customer1/​sbin nullfs ro 0 0
 +/lib /​var/​jails/​customer1/​lib nullfs ro 0 0
 +/libexec /​var/​jails/​customer1/​libexec nullfs ro 0 0
 +/usr /​var/​jails/​customer1/​usr nullfs ro 0 0
 +/conf/base /​var/​jails/​customer1/​conf/​base nullfs ro 0 0
 +/​etc/​jails/​customer1 /​var/​jails/​customer1/​cfg nullfs rw,noatime 0 0
 +</​code>​
  
documentation/examples/multi-tenant_router_and_firewall.txt ยท Last modified: 2017/08/31 16:25 (external edit)