From TBP Wiki
Jump to: navigation, search


FreeBSD Logo.png

FreeBSD is a free and open-source Unix-like operating system descended from Research Unix via the Berkeley Software Distribution (BSD). FreeBSD is a direct descendant of BSD of which was historically called "BSD Unix" or "Berkeley Unix" (in violation of the UNIX trademark). The first version of FreeBSD was released in 1993 and, as of 2005, FreeBSD was the most widely used open-source BSD operating system, accounting for more than three-quarters of all installed BSD systems.

FreeBSD shares similarities with Linux but has two major differences in scope and licensing; FreeBSD maintains a complete system, i.e. the project delivers a kernel, device drivers, userland utilities, and documentation, as opposed to Linux only delivering a kernel, drivers, and relying upon third-parties for system software. FreeBSD source code is generally released under a permissive BSD license, as opposed to the copyleft GPL used by Linux.

The FreeBSD project includes a security team overseeing all software shipped in the base distribution. A wide range of additional third-party applications may be installed using the pkg package management system, FreeBSD Ports, or by compiling source code.

Due to its licensing, much of FreeBSD's codebase has become an integral part of other operating systems, such as Apple's Darwin (the basis for macOS, iOS, watchOS, and tvOS), the open-source NAS/SAN operating system FreeNAS, the Nintendo Switch system software, and the system software for Sony's PlayStation 3 and PlayStation 4.


To list pools:

   zpool import

To import a pool:

   zpool import POOLNAME

This pool has to be mounted manually if moved from another system.

   zfs set mountpoint=/mnt/dirname poolname
   zfs mount -a

Attach a mirror to existing hard drive in FreeBSD/FreeNAS

Let's assume ada0 is your existing disk, ada1 is the new one, tank is the pool name.

   gpart create -s gpt /dev/ada1
   gpart add -i 1 -b 128 -t freebsd-swap -s 2g /dev/ada1
   gpart add -i 2 -t freebsd-zfs /dev/ada1
  • Run zpool status and note the gptid of the existing disk
  • Run glabel status and find the gptid of the newly created partition. It is the gptid associated with ada1p2.
   zpool attach tank /dev/gptid/[gptid_of_the_existing_disk] /dev/gptid/[gptid_of_the_new_partition]

It may take a while to resilver your drive after this - you will not have access to it whilst this is running.


Unlock Geli-encrypted ZFS Volume:

   geli attach -k [geli_key_file] [dev_to_unlock]


   geli attach -k /data/geli/geli.key /dev/ada0p2

To import the pool, see Pools

Iocage/Warden Jails

To migrate jails from one pool to another:

   zfs snapshot -r poolname/jails@relocate
   zfs send -R poolname/jails@relocate | zfs receive -vF newpool/jails

To migrate a jail from one computer to another:

   iocage stop jailname
   iocage export jailname

Exporting jails will create a zip file "" inside "/mnt/iocage/images/". To import these backups, copy the exported backup files into "/mnt/iocage/images/" and then restore:

   iocage import

If iocage gives trouble, use the jail name instead:

       iocage import jailname

Change iocage pool location:

    iocage activate NEWPOOLNAME

To clone jail1 to jail2, run:

   iocage clone jail1 --name jail2

Manual import of a jail:

   zfs create zpool1/iocage/jails/jail1
   zfs recv -F zpool1/iocage/jails/jail1 < jail1_2020-10-24
   zfs recv -F zpool1/iocage/jails/jail1/data < jail1_2020-10-24_data
   zfs recv -F zpool1/iocage/jails/jail1/root < jail1_2020-10-24_root

Automatically stop, make an export backup, and start all available iocage jails in a for loop into zpool1/iocage/images:

   for i in $(iocage list |awk '{print $4}' |grep -vi name|awk NF); do iocage stop $i && iocage export $i && iocage start $i; done

Iohyve PCI passthrough

The following is how to get Iohyve PCI passthrough working in FreeNAS with pfsense.

Get the PCI addresses for the ethernet card.

   pciconf -lv

Find the PCI addresses for the ethernet card. A multi-port card will have several. You will need them for the pptdev2 tunable in a x/y/z format. This example is for two ethernet ports with PCI addresses x1/y1/z1 and x2/y2/z2.

Go to System > Tunables and configure the following options to enable iohyve and PCI passthrough. pptdevs2 is used because regular pptdevs did not work so it depends on the setup.

Variable | Value | Type

iohyve_enable | YES | rc

iohyve_flags | kmod=1 net=<eth0,eth1> | rc

pptdevs2 | x1/y1/z1 x2/y2/z2 | loader

vmm_load | YES | loader

Configure the virtual machine using iohyve within terminal:

   iohyve setup pool=(pool name)
   iohyve create pfsense 8G
   iohyve set pfsense ram=2048mb
   iohyve set pfsense cpu=2
   iohyve set pfsense pcidev:7=passthru,x1/y1/z1
   iohyve set pfsense pcidev:8=passthru,x2/y2/z2
   iohyve set pfsense os=pfsense
   iohyve set pfsense bargs="-S -A -H -P"

Some have to dd the image to the zvol. It can be installed any other way so long as it boots properly. Make sure the paths and files are correct. You can disregard the following if you are able to boot using other methods.

   iohyve fetch
   zfs rename zeus/iohyve/ISO/pfSense-CE-memstick-serial-2.4.4-RELEASE-p1-amd64.img.gz zeus/iohyve/ISO/pfSense-CE-memstick-serial-2.4.4-RELEASE-p1-amd64.img
   cd /iohyve/ISO/pfSense-CE-memstick-serial-2.4.4-RELEASE-p1-amd64.img/
   gunzip pfSense-CE-memstick-serial-2.4.4-RELEASE-p1-amd64.img.gz
   dd if=/iohyve/ISO/pfSense-CE-memstick-serial-2.4.4-RELEASE-p1-amd64.img/pfSense-CE-memstick-serial-2.4.4-RELEASE-p1-amd64.img/pfSense-CE-memstick-serial-2.4.4-RELEASE-p1-amd64.img of=/dev/zvol/zeus/iohyve/pfsense/disk0 bs=1m

Start the VM

   iohyve start pfsense

In another shell session, connect to the console to perform the installation.

   iohyve console pfsense

Set it to automatically boot.

   iohyve set pfsense boot=1

Here are some good resources to use for this in case this doesn't work:

Iohyve manual man page

Iohyve wiki

USB passthrough example

CentOS useful for tunables for FreeNAS

Limiting Jail Resources with RCTL

Here is how you limit the amount of RAM or CPU each jail can have. A dd the following line to /boot/loader.conf:


Reboot to activate.

The following is how to constrain CPU usage, in percentage:

   rctl -a jail:JAILNAME:pcpu:deny=75

The following is how to constrain virtual and physical RAM usage, in percentage:

   rctl -a jail:JAILNAME:vmemoryuse:deny=512M
   rctl -a jail:JAILNAME:memoryuse:deny=1024M

To view the currently applied limits:


To view the resources used by a jail:

   rctl -u jail:JAILNAME

Install Ubuntu Linux 20.04 LTS in vm-bhyve


This guide is how to install Ubuntu in vm-bhyve.


   pkg install vm-bhyve qemu-tools cdrkit-genisoimage
   pkg install grub2-bhyve bhyve-firmware

Configure Install

   zfs create -o mountpoint=/vm tank1/vm
   cp /usr/local/share/examples/vm-bhyve/* /vm/.templates/

Add this to rc.conf:


Then run:

   vm init

Configure networking

   vm switch create public
   vm switch add public eth0

If this does not work, use the following:

   vm switch create -t manual -b bridge0 public

Fetch image

Download the | Cloud Init image:

vm img

Resize the disk

Resize to desired

   qemu-img resize  /tank/bhyve/.img/focal-server-cloudimg-amd64.img +20G

Create the VM

vm create -c 8 -m 16G -t ubuntu -i focal-server-cloudimg-amd64.img -C -k ~/.ssh/ ubuntu

To change the number of CPUs, change "-c 8" to desired. Value "-m 16G" is for RAM. A maximum of 16 vCPUs is currently supported in bhyve.

Start the VM

   vm start ubuntu


Determine the IP address and ssh to the vm:

   ssh ubuntu@

vm-bhyve doesn't have any way of showing the actual IP so you need to search the DHCP logs or use nmap.

Set hostname

   hostnamectl set-hostname ubuntu.vmhostname

Package management

Do not install recommended and suggested packages

   cat <<EOT >/etc/apt/apt.conf.d/61norecommends
   APT::Install-Recommends "false";
   APT::Install-Suggests "false"; 

Update the software

   apt update && apt -y upgrade

Enable autostart

Make sure the VM is listed in vm_list in /etc/rc.conf.

vm_list="ubuntu vm1 vm2 ..."

Resize a root disk

Reboot into single user. This is assuming that da0 is the root drive and da0p2 is the root partition.

   gpart recover da0
   gpart resize -i 2 da0
   zpool online -e zroot da0p2