Hello Vivek Gite;
Been trying to get vnet jails to work on a gateway host. Having problems with vnet network access to the public internet. I need some help. From this article I can tell you have the networking knowledge that can help me figure out what is wrong with my vnet configuration. I wrote my problem as a draft for an article that you may use to post on this site. I can be reached at joeb_722 at yahoo dot com
This is an example of a vnet jail configuration that is run on a real hardware computer having a network interface connected to a public internet service provider. FreeBSD 12.0 is the operating system running on this hardware.
VIMAGE which is required for vnet jails is included in the FreeBSD base kernel in this release, and the PF firewall and its NAT (network address translation) function have also been adapted to be vnet jail aware. A vnet jail requires NAT to access the public internet.
The bridge/epair method is used to connect the vnet jails network to the hosts network. The setting up and the tearing down of the vnet jails bridge/epair is done in the vnet jails jail.conf file.
The hosts "service jail" command is used to start/stop the vnet jails. The setup configurations are further separated into all the vnet jails being defined in a single jail.conf file or each jail definition being in a separate jail.conf file.
Information on the sysrc command and the "service jail" command.
The sysrc command adds or changes statements in the hosts /etc/rc.conf file.
Additions are placed at the bottom of the /etc/rc.conf file.
$ sysrc jail_enable=YES
Used to enable the "service jail" command. Will auto start jails at boot time and auto stop jails at halt & shutdown time.
$ sysrc jail_list=""
Used by the "service jail" command. Empty list means all jails are in a single /etc/jail.conf file.
$ sysrc jail_list="jail-name jail-name …"
Used by the "service jail" command. Means search /etc for jail.xxx.conf files where xxx is jail-name from the list.
This "service jail" command setup works for both vnet jails and non-vnets.
Format of service command
$ service jail start
$ service jail restart
$ service jail stop
$ service jail start jail-name
$ service jail restart jail-name
$ service jail stop jail-name
Creating vnet jails directory tree the easy way.
mkdir -p /usr/local/jails
fetch -avrA "http://ftp.freebsd.org/pub/FreeBSD/releases/amd64/12.0-RELEASE/base.txz"
Repeat the following 2 commands changing the jail-name to create as many vnet jails as you want.
xzdec base.txz | tar --unlink -xpJf - -C "jail-name"
Copy host files needed by the jail.
cp /etc/resolv.conf jail-name/etc
cp /etc/localtime jail-name/etc
Preparing the host’s /etc/rc.conf for jails.
Sample 1: Single jail.conf(5) method.
All jail definitions in single jail.conf file.
On the host system console issue following commands one time.
Copy the jail.conf statements shown below and paste into /etc/jail.conf
Edit the /etc/jail.conf following the edit instructions below.
Sample 2: Per-jail jail.conf(5)
Each jail definition in individual jail.jail-name.conf file.
On the host system console issue following commands one time.
sysrc jail_list+="jail-name jail-name jail-name ..."
Copy the jail.conf statements shown below and paste
Edit the /etc/jail.jail-name.conf following the edit instructions below.
This is the example vnet jail jail.conf content.
host.hostname = "jail-name";
path = "/usr/local/jails/jail-name";
exec.consolelog = "/var/log/jail.jail-name.console.log";
devfs_ruleset = "70";
vnet = "new";
vnet.interface = "epair15b";
exec.prestart = "ifconfig bridge15 create up";
exec.prestart += "ifconfig epair15 create up";
exec.prestart += "ifconfig bridge15 addm epair15a";
exec.prestart += "ifconfig bridge15 addm re0";
exec.start = "ifconfig epair15b inet 10.0.110.25/24";
exec.start += "route add default 10.0.110.2";
exec.start += "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
exec.poststop = "ifconfig epair15a destroy";
exec.poststop += "sleep 2";
exec.poststop += "ifconfig bridge15 destroy";
Standard edit instructions for the jail.conf file.
Substitute all 4 places where the word jail-name is shown with the real jail name created above.
Vnet jails must have an ip address assigned to it. Change the ip address 10.0.110.25/24 to what ever ip address you’re using on your host system leaving the /24 suffix.
All vnet jails must have a route pointing to the hosts default gateway ip address assigned by the ISP. Issue "route -n get default" command. Take the gateway ip address shown in the output display and substitute it for the 220.127.116.11 ip address. ???
The re0 in the exec.prestart line represents the interface name of the interface facing the public internet. Replace re0 with the interface name facing your public internet. You can determine the interface name to use from the "route -n get default" command issued on the host.
Setup PF firewall to run inside of vnet jail.
Copy the following and paste it into the /etc/rc.conf file of the vnet jail you’re setting up.
Pre-allocate the pflog file
Vnet jails need devfsrules to enable pf firewall to run.
On the host create /etc/devfs.rules file and populate it with the following content. Then issue the "service devfs restart" command to enable this new rule.
This only needs to be done one time.
devfsrules for pf to function in a vnet jail.
add include $devfsrules_hide_all
add include $devfsrules_unhide_basic
add include $devfsrules_unhide_login
add include $devfsrules_jail
add path ‘bpf*’ unhide
add path pf unhide
add path pflog unhide
Copy the following and paste it into the /etc/pf.conf file of the vnet jail you’re setting up. This is very simple PF rules, pass every thing in and out and log, so traffic can be viewed.
set block-policy drop
set fail-policy drop
set state-policy if-bound
scrub in on epair15b all
set skip on lo0
nat on epair15b from 10.0.110.25 to any -> host-public-ip
pass out log (all) quick on epair15b from any to any
pass in log (all) quick on epair15b from any to any
PF is now a loadable module. If PF is used as the hosts firewall than nothing further needs to be done. If the host is running IPFW or IPFILTER firewall then PF has to be loaded into the kernel one time per boot using this command "kldload pf.ko pflog.ko" or added to boot/loader.conf. To see what modules have been loaded issue kldstat -v. To unload the PF modules
use "kldunload -f pf.ko pflog.ko" command.
Testing your vnet jail.
From the hosts console.
service jail start
service jail stop
It may take up to 30 seconds for the vnet jail to complete starting.
You can issue the "jls" command to verify the jail is running.
To log into the running jail issue "jexec jail-name login -f root"