OPNSense – Installing On Intel Apollo Lake

So, I’ve had some issues getting OPNSense (or pFsense, anything based on FreeBSD) to install and boot properly on my shiny new router. The router is based on the Intel J3455 CPU, which uses the Apollo Lake architecture.

The Gigabyte (Gigabyte GA-J3455N-D3H) board’s BIOS is rather basic – it does not allow you to configure ACPI or HPET settings; these are what cause issues in FreeBSD 11. To get around this, some stuff is needed to be done:

Booting the USB:

  • Ensure the CSM is enabled and set “Other Devices” to “Legacy”
  • Boot the USB installer in UEFI mode
  • Press “3” 
  • Type: set hint.hpet.0.clock=0
  • Type: set hint.acpi.0.msi=2 – repeat four times for “0” to “3”
  • Type: set machdep.disable_msix_migration=1
  • Type: boot

Install OPNSense/pFsense as normal.

Once installed:

  • When the machine reboots, repeat the steps above
  • Login to the console
  • Select the “Shell” option
  • Type: cd /boot
  • Use vi to add the following lines to loader.conf.local:
    • hint.hpet.0.clock=”0″
    • hint.ahci.0.msi=”2″ (add 4 lines from “0” to “3”)
    • machdep.disable_msix_migration=”1″
  • Reboot the machine

Now, the device should boot up without hanging or intervention required. This guide is what worked for me on this particular system installing to an SSD connected via SATA – hopefully FreeBSD 12 fixes this problem.

Hope this helps – good luck!


Configuring DNS-over-TLS with OPNSense

Privacy is becoming more and more important in the ever-developing world of the Internet. Of course, you have the ongoing issues with security on the Internet – a lot of the main workings are ancient. And the other thing we now have more and more is targeted advertisement – where companies exist solely to gather as much information about an individual as possible and present to them adverts that they think would matter to them.

One mechanism that is used for this is the ancient DNS system – used to translate something like “google.com” to it’s IP address “”. Typically, DNS operates unencrypted using UDP – where a request is sent in plain-text and is open to being interrupted and replaced with something else (called a “man-in-the-middle” attack).


The main problem here is that there is no way with traditional DNS of knowing if the returning answer is from the DNS server you originally requested. The effect of leaving everything unencrypted is allowing companies (like your ISP) to be able to log your DNS traffic (and maybe sell it to advertisers).

So we need to do two things:

  • Verify connections back from the DNS server are legit
  • Encrypt the connection

This is where DNS-over-TLS comes in! TLS uses a TDP connection to form a two-way conversation with the DNS server and then also encrypts the traffic to prevent snooping. There is also DNS-over-HTTPS which employs a similar method to “hide” your DNS traffic as standard HTTPS requests to websites.

TLS is an encrypted protocol that makes it extremely difficult (although not impossible) to read the DNS requests sent from your browser to the DNS server. A handshake is made using keys to ensure all the traffic is between the router and DNS server is legitimate. Mozilla is currently trialling built in DNS-over-TLS functionality in Firefox, however this still limits you to one browser on your desktop PC – you may also want to use your phone for instance. The best way to go about this is to perform the encryption of DNS requests in the router.

Fortunately, pfSense and OPNsense support the new service provided by CloudFlare. There are alternatives (such as Quad9 – that can also be used – infact you can use these as backups in case CloudFlare goes down (or use Quad9 as your primary). Either way it is recommended to use a secondary DNS service as a backup, and then a tertiary service such as can be used as a final backup to ensure you never go offline.

I am running OPNsense – though the setup for pfSense is very similar to this. The bulk of the configuration is achieved through the Unbound DNS > General page:

Above is my configuration – you should add multiple “forward-addr” lines to allow for multiple DNS servers to be used if one or more are offline. You will want “DNS Query Forwarding” to be disabled.

And that is it! Once that is set, make sure that Unbound is restarted using the restart icon at the top of the screen. You can test if it works using the “Packet Capture” tool under Interfaces > Diagnostics and looking at data transmitted over port 853:

You will see your public IP connecting to the DNS service using TCP over port 853! 

And that’s it! Your DNS requests will now be encrypted. Safe browsing!


IPv6 bogons – OPNsense errors?

I must admit I am still getting used to a more complex firewall setup, as detailed in earlier posts. I’m constantly finding new things in the settings panel which have absolutely no applicable use to me, but absolutely do to those in the larger enterprise space.

That’s all well and good, and I’ve been tootling along just fine leaving most of the more advanced settings alone. Until…


[ There were error(s) loading the rules: /tmp/rules.debug:15: cannot define table bogonsv6: Cannot allocate memory – The line in question reads [15]: table persist file /usr/local/etc/bogonsv6]

This was weird. As far as I’m aware, my ISP doesn’t have IPv6 support yet – but I can’t ignore it since it will come in the future. I was having problems getting ports forwarded – I would apply the settings but I still couldn’t get through, unless I rebooted the router. This didn’t make sense to me – this was all working before. One nice thing about OPNsense is the ability to live view packets being allowed or blocked by the firewall – this way I could still tell the default rule was blocking packets that should be allowed through.

That message appeared at the top of the screen each time I applied the firewall settings. Hmm… What actually does it mean? I did some digging and discovered that the IPv6 bogon list was recently expanded – and this has now tipped over the default maximum number of firewall entries.

Since IPv6 is still relatively new, the bogon list (which specifies things like loopback addresses to block on the incoming side of the WAN – those sorts of things should never be on the incoming side so they’re blocked for security purposes) is still being updated.

The solution to this is to bump up the maximum size from the default.


I’ve set mine to 1 million since I have plenty of memory and CPU and didn’t want to have another bogon update trip the limit again. 200,000 is the default limit for versions around the time of writing this (though I’ve read that 400,000 is a decent setting to use). Maybe a million is too many but I’ll see!

Another option is to disable IPv6 bogon filtering in the Interfaces settings page – though IPv6 support may drop at any time I’m going to leave it enabled and just increase the firewall entry table size instead.


I wouldn’t recommend turning off bogon filtering since they are actually quite important, especially with IPv6 covering a much larger set.

Anyway – now that’s been dealt with I can move onto DNS-over-TLS (since it’s all over the web at the moment). Stay tuned!


OPNsense/pfSense: What it brings to the table

I’ve just finished my 2018 router and am enjoying the experience so far. I’ve been wondering… what does it give me over a standard ISP-issued router, and what am I sacrificing? Well – I thought I’d list a few things I have come to appreciate over the last little while…


I am running OPNsense – and so far it has been great. As for a routing platform it seems quite similar to pfSense – though I do prefer the look of the OPNsense UI (subjectively, mind). But this post isn’t about the software running – it’s about why I did this project at all. I mean, I could have just used the standard router the ISP issued me – and that was working very well before this project.

The first reason: fun! There’s something about setting up your own stuff to work the way you want it to – having ultimate control over everything to the point where you mess everything up and spend hours and hours troubleshooting. That, for me at least, is fun! It’s why I like Linux, it’s why I build my own PCs, its why I built my network from scratch. I believe it’s fine to break stuff by fiddling, as long as you can work out how to put it right again and learning along the way!

The second reason: security. OPNsense is updated on a regular basis with bug fixes and security patches:


I bet there isn’t a single router anywhere on the planet that gets updated 4-5 times a month! You even get a full description of what goes into each update, rather than a generic “security fixes” reason that you would get from the likes of, for instance, Netgear.

The third reason: configuration. There are lots and lots of settings in this thing… many of which I don’t know what they do. That adds to the fun aspect of reason #1, but also gives you the ability to fine tune how you want things to behave. There is a very steep learning curve, and it took me a good while to figure out how to forward ports even! It gives you a level of control you won’t get from any off-the-shelf router.

The fourth reason: privacy. This seems to be getting more and more important as time goes on, and VPN’s are becoming more and more ubiquitous. You can use this to tunnel everything on your network through a VPN provider transparently, or only certain parts of your network. One big privacy-centred feature for me is being able to use DNS-over-TLS – where DNS requests that are typically unencrypted become unreadable by, for instance, your ISP or government. Future privacy-related content to come soon!

The fifth reason: reliability. I’ve had this setup running for over two months and not once have I had a problem with connectivity (except for ones I caused!). I’ve had many different routers in the past from ISPs and third-party companies in the past – not one hasn’t required me to reboot it every-so-often. Virgin Media routers especially are notorious for being unstable (in my own experience). Instead of having one box with a combined router, switch and WiFi access point, I have separated everything.


This separation means that if one bit stops working (often the WiFi bit, for instance) then that part can be rebooted without affecting other parts of the network.

Those are the top five reasons I did this – and the reward is so worth the effort and headache. Let’s go through what disadvantages I can think of:

  • Cost. The ISP-router was free. All in, this cost me just shy of £200 (though I did have a fair few bits lying around that I could use).
  • Power consumption (more detail here). It’s low – but not as low as a normal router.
  • Not being something that “just works”. This took a lot of setting up, and didn’t “Just work” for me.
  • The steep learning curve. I’m not a newbie when it comes to networking, and even for me this was a challenge. If you’re new to networks and all you’ve ever done is forward a port through a Belkin router, then this is not for you (unless you like a challenge and want to learn something – then it is definitely for you).

Would I do this again? Absolutely. I don’t think I’ll need to for a long while – this thing is so overkill its hilarious. I’ve yet to properly benchmark it but just based on the CPU and RAM usage – it’s barely even lifting a finger.

To conclude – I am very happy with this router. It’s far exceeded my expectations with what it is capable of, and I am slowly learning about what it can do. It feels like I discover something new each time I go into the admin panel and poke around – there’s a reason there are courses people go on to learn all this stuff. I mentioned before there will be future content on stuff it can do – so stay tuned!


An Overview of my Home Network

I must admit that it is nice to have ultimate control over your network. I was not happy with the limited control that comes with ISP routers – it was time to step things up with a reliable setup.

So behold! Below is my slightly over-complicated network setup:

Network Map

So let’s start at the Internet jack in the living room. I have a phone table on which there is a Netgear DM200 modem:


This is a cheap and simple modem that takes a VDSL/2 signal and chucks it down a 15m ethernet cable that snakes around to the study. Once I’ve disabled the routing mode of the modem and setup the internet connection, it can be plugged into one of the ports on the OPNSense router.

From there it gets, well, routed and sent over to my TP-Link SG108E Gigabit Smart Switch:


This is an 8-port switch that has some additional capabilities – such as being able to set up VLANs or do things like link aggregation (the NAS now has a 2Gbit link). This switch connects the NAS, printer, PC and WiFI access point, which is a Netgear AC750 Access Point that plugs into the wall. It provides plenty of WiFi around the flat at pretty decent 802.11ac speeds.

For the smart TV and Steam Link I am passing the local network over to the living room via TP-Link 1200Mbps PowerLine. The wiring in this flat is quite new so the PowerLine link is very good – meaning I can use the Steam Link to stream games to the TV from the PC as if I was playing them directly on the PC.


This is a shot of the wall sockets in use – the equipment “rack” is connected to the power meter on the left, and the 1200MB Mbit PowerLine with the AC750 WiFi access point on the right.

I like this setup since its compact and still provides easy access to power buttons. The other end of the PowerLine link comes out behind the TV, which then plugs into a very basic 10/100 Mbit switch that supplies the TV and Steam Link (which are both 100 Mbit devices anyway).


Above you can see the equipment “rack” (an old TV stand) with the gear all set up. This was taken when I was using a different WiFi access point that proved to be too flaky, so I moved to the Netgear AC750. The printer is a Brother MFC-J5910DW I got from a friend – totally overkill for me but it works and is networked in!

The bottom shelf has the router and switch, the middle holds the 2016 NAS and on top is the massive printer.

This has been a very fun project to put together – and it works very nicely. Aside from my own blunders, this has been rock-solid. If I had to improve something – I should have got a 16-port switch instead of the 8. I have used up every port so if I want to host a LAN party then I have to unplug things… not ideal!

Fortunately I can just use the ethernet cable running to the PC (which is fairly long) and use yet another switch!

Though… thinking about it… I could also jump on the 10 Gbit bandwagon at some point! NICs still need to come down in price a little bit before I go down that road! Maybe in the future – but for now regular 1 gigabit is fine.

That’s all for now! Let me know what you would add/improve in the comments!


The 2018 Router – Power Consumption

So then – the build is done. There are a few questions that need to be answered though, and today we will go into the power consumption of this router.

Of course, this is a custom build based on a PC architecture – it’s not going to be at the levels of a typical router that you would get from your ISP. But… just how low is it?

Firstly, let’s address what tames the 230VAC to a much more manageable 12VDC:

Now, I’ve got to admit I went a bit over the top with this bit. This is a 150W adapter that will output 12.5A of 12V DC. I can safely be assured that my new router won’t even approach a tenth of that… but it is an FSP unit with solid build quality and efficiency. I had some ideas for the future to build a 12V UPS using a battery and some circuit modules – things like the router and modem could run off the 12V battery for a decent length of time, and this chunky 12V power brick would be good for charging up the battery.

So this get’s 12V into the system. Obviously PC’s don’t just use 12V – there are all these pesky other voltages like 5V and 3.3V. Some motherboards have onboard power conversion but mine doesn’t – so I need something to convert it to all the voltages I need.

Step in the picoPSU-150-XT. This is a very small adapter that plugs into the 24-pin ATX connector and supplies all the other voltages the system needs. A main 12V DC feed comes in and 12V, 5V, 3.3V etc come out, with extra connectors for things like the CPU header, Molex and SATA drives.

I figured I had a 150W power brick, so why not get a 150W picoPSU as well? The nice thing is, that if it manages to last a decent amount of time – I would consider getting these parts again for the 2016 NAS if the PSU in that decided to pack it in. It’s got plenty of capacity to run my entire NAS 24/7 with decent efficiency. For now though, it’s going in the router.

Something to be careful of – eBay is a great source for these items (I got this kit) but there are a lot of knockoff items floating about that may ruin your day. I spent a bit extra getting the “good stuff” since this router is the most critical part of my network – nothing works without this working at 100%.

The picoPSU is installed in the case as shown above – the kit I got came with an adapter for the FSP PSU that also fits the case perfectly. The extra connector with Molex and SATA attached is not needed so I have detached it and stored it away.

That’s the power delivery – now let’s see how well it works!

43.5W is our baseline reading on the power meter. This is just the NAS, switch and printer connected – lets plug in and boot the router to see what it uses:

43.5W to 53.6W – that’s 10.1W of power usage. For the average electricity price in the UK, that’s around £10-£11 per year. Not quite as low as a router you would get from your ISP, but then again it’s not bad! I wouldn’t be surprised actually if routers do exist with similar or higher power consumption.

For the control and performance you get, 10W is very respectable when you consider that it’s a quad core router! My internet connection is a 80/20 VDSL link – it handles it with ease and could probably manage much, much larger connections.

Speaking of performance – stay tuned!


The 2018 Router – The Motherboard

Now we’re getting into the meat of this project – the single, absolutely most important part of any router: The Motherboard!

This is the Gigabyte GA-J3455N-D3H – a Mini-ITX board with a soldered Intel Celeron J3455 which runs up to 2.3GHz at maximum turbo. This is a quad core part using the Apollo Lake architecture. It is considerably more powerful than the Bay-Trail based N2830 chip that was in my Intel NUC.

This board is by no means a server-oriented board, but it is the only one I could find at a reasonable price that had:

  • A passively cooled CPU that supported AES-NI
  • Two gigabit LAN ports
  • Support for DDR3L SODIMMs – I had old RAM lying around I wanted to use
  • Reasonably good build quality (eg solid capacitors).
  • Low power consumption
  • A low-end soldered CPU. Routers really don’t need much horsepower.

That first bullet point is critical for the latest versions of pfSense and OPNSense. A lot of other boards I considered all ran J1900 parts which don’t support AES-NI. I wanted this system to run the latest versions of software for security purposes – and being stuck at an old version was not an option for me.

On the rear you get a decent selection of IO – PS2 ports, 2x Serial ports, VGA and HDMI, two gigabit LAN ports, two USB 2.0 and two USB 3.1 Gen 1 ports alongside basic audio outputs. The video output ports are perfect for the initial setup of the box.

You get two 1.35V DDR3L slots (good for power consumption), four SATA 6 Gbit/s ports, a USB 3.1 Gen 1 header and two USB 2.0 headers. There are also things like TPM headers and naturally the front panel headers. Present there is also a 24-pin ATX header along the right hand edge and a 4-pin CPU power header (though I’m not sure why this is needed – some other low-power boards don’t seem to bother with this).

The onboard buzzer is quite loud – which is annoying when the board starts up. It is very useful, however, for pfSense and OPNSense to notify you when the system is booted, and when the system is powering down.

There was one, glaring issue with this board that really made me worried about using it… the two gigabit ethernet ports run off Realtek controllers. For FreeBSD-based things like pfSense and OPNSense you really want Intel based NICS.

Fortunately, for this board, the NICs work perfectly with OPNSense. They show up and work just fine – and the performance is very good (more on this in future articles). I must admit I was relived to see them working without manually shoehorning in drivers – otherwise the board would have been returned to sender!

What’s it like whilst its running – I hear you ask? Does that puny heatsink and case ventilation mentioned yesterday do enough to cool that 10W monster of a CPU? Well, yes actually. During normal routing duties I don’t see anything above 47-48 degrees C, and so far after two months not once has it let me down. It’s been absolutely rock solid running OPNSense – though I had a lot of challenges getting the install completed… that’s a saga meaty enough for it’s own post! I could not for the life of me get pfSense installed – so I moved to OPNSense (which is basically the same thing).

So finally, time to conclude my experience using this board as my routing platform over the last couple of months:

The good:

  • It meets my requirements as discussed earlier
  • Considering it includes a CPU, it is well priced (~£80)
  • Excellent build quality
  • Silent operation
  • Sips power

And the not-so-good:

  • Realtek NICs – not the best for pfSense and OPNSense.
  • No ECC support… not the end of the world though
  • BIOS buzzer really goes wild at boot with no keyboard, mouse or screen attached
  • Not server-grade – something you want with critical 24/7 equipment.
  • pfSense wouldn’t play ball at all
  • OPNSense was a bit tricky to get installed and needed some crowbar-ing

So yeah – it’s worked out to be a good board for OPNSense once the install is done. That’s the most headachey part of this project – getting the OS installed. After that – it’s been smooth sailing. More to come on performance, power and the software!