pfSense on the PC Engines APU2

Hey! Listen! This post is part of a series on pfSense. Check them all out!

DateURLPart
2019-08-25pfSense on the PC Engines APU2
  • Migrated to a PC Engines APU2D4
  • 2019-07-17My SG-1100 died
  • Migrated back the the EdgeRouter Lite
  • 2019-06-28Migrating away from the Ubiquiti EdgeRouter Lite
  • Migrated to a Netgate SG-1100
  • Introduction

    In the past few weeks, I replaced my EdgeRouter Lite with a Netgate SG-1100. Two weeks later, my SG-1100 died, and I had to put the EdgeRouter Lite back. However, I still wanted to replace the ERL with a pfSense device (albeit, not Netgate hardware).

    PC Engines APU2

    Again, my requirements for hardware were as follows:

    • Hardware that is small, low power, and fanless (this device is in my living room, not a server rack)
    • Have Intel NICs (they generally have better compatibility with Linux/BSD than Realtek)
    • Be around $250 or less, including RAM (but not storage)

    Going back to my hardware chart, I further narrowed it down to the PC Engines APU2 and the Shuttle DS77U (in fact, the DS10U was just released, which is a DS77U with an 8th Gen Intel processor). While the Shuttle DS77U/DS10U would have better hardware (newer CPU, DDR4 instead of DDR3, etc…), I chose to go with the APU2. The APU2 comes highly recommended on reddit and the pfSense forums, and it receives frequent BIOS updates (including Coreboot support, and most recently AMD Core Performance Boost).

    There are four main differentiators of the APU2 lineup (thanks to Cee Jay for clearing this up):

    • the APU platform generation (e.g., 2, 3, 4, etc…) – this is where you get APU2, APU3, APU4, etc…
    • the board revision (e.g., a, b, c, d, etc…) – this is where you get APU2A, APU2B, APU2C, etc…
    • the amount of RAM (2GB or 4GB) – this is where you get APU2D2, APU2D4, etc…
    • the type of NIC (Intel i210AT vs i211AT)

    When purchasing an APU model, do not assume that a bigger model number is better. For example, the APU4 is not “better” than the APU2 because the model number is larger. You should always do your research to determine what model you need for your application.

    For networking applications, it’s generally known that the i210AT is considered “better” than the i211AT because it has four transmit and four receive queues per port, while the i211AT only has two transmit and two receive queues per port. So when purchasing an APU, I made sure to look for one with an i210AT.

    I ended up purchasing the following items directly from PC Engines directly. Shipping took a total of 10 days from Switzerland to Pennsylvania. It arrived via USPS and required a signature, since it originated outside the United States.

    Pre-install

    Assembly

    I chose to assemble my APU2 myself, but there is an option to pay to have it assembled. If you choose to assemble it yourself, there is a really good video here. Pro-tip, make sure you remove the screws on the serial connection before you try to assemble anything.

    Connect to serial

    Connect the serial cable using the settings from the manual. These are the same settings that pfSense uses as well.

    • Speed: 115200
    • Data Bits: 8
    • Parity Bits: None
    • Stop Bits: 1

    These are the PuTTY settings I used (your COM port may be different).

    Press F10 at boot when connected via the console cable and you will see the boot menu.

    Memtest86

    I always test my memory before I use it, and I always recommend Memtest86 (not to be confused with Memtest86+, which is no longer maintained). There is a build of Memtest86 built-in to the APU2 BIOS, just press F10 at boot when connected via the console cable and you can run a memory test with option 3.

    Setup

    Install

    Installing pfSense was easy enough, especially if you’re using the amazing install guide (seriously, use it). For my install, I chose the options below from the download page. The APU2 does not have a VGA port, so you don’t have any option other than to do an install over serial.

    Configuration

    The initial configuration was easy. The middle port is the LAN port, which will give you a 192.168.1.1/24 address. From a browser, follow the prompts to do your initial setup. I won’t detail exactly what I did to my setup, since everyone’s install will be different.

    I recommend getting a cheap label maker and labeling the interfaces, since they are not marked on the case anywhere. I also make a label for the boot menu shortcut key and the serial settings.

    BIOS update

    This is personal preference, but I always try to update the BIOS on my devices. With the recent Meltdown and Spectre vulnerabilities, it is crucial to keep your BIOS updated (since these vulnerabilities can only be mitigated with firmware updates). For the APU2, the BIOS updates are located here.

    PC Engines recommends you flash the firmware from a separate Linux-based USB drive, but you can do it from inside pfSense, after pfSense is installed, as shown here.

    Speedtest

    Here are the iPerf results when running a test with my ERL as the router. Keep in mind, my internet at home is only 400/400, so that’s my current maximum speed.

    Connecting to host loganmarchione.com, port 5201
    [ 4] local 10.10.2.34 port 55818 connected to 68.183.148.132 port 5201
    [ ID] Interval Transfer Bandwidth Retr Cwnd
    [ 4] 0.00-1.00 sec 48.0 MBytes 402 Mbits/sec 28 1.62 MBytes 
    [ 4] 1.00-2.00 sec 64.8 MBytes 543 Mbits/sec 23 1.35 MBytes 
    [ 4] 2.00-3.00 sec 67.2 MBytes 564 Mbits/sec 0 1.42 MBytes 
    [ 4] 3.00-4.00 sec 67.2 MBytes 564 Mbits/sec 0 1.47 MBytes 
    [ 4] 4.00-5.00 sec 67.2 MBytes 564 Mbits/sec 0 1.50 MBytes 
    [ 4] 5.00-6.00 sec 67.2 MBytes 564 Mbits/sec 0 1.52 MBytes 
    [ 4] 6.00-7.00 sec 66.4 MBytes 557 Mbits/sec 0 1.53 MBytes 
    [ 4] 7.00-8.00 sec 66.6 MBytes 559 Mbits/sec 0 1.54 MBytes 
    [ 4] 8.00-9.00 sec 67.2 MBytes 564 Mbits/sec 0 1.54 MBytes 
    [ 4] 9.00-10.00 sec 67.2 MBytes 564 Mbits/sec 0 1.56 MBytes 
    - - - - - - - - - - - - - - - - - - - - - - - - -
    [ ID] Interval Transfer Bandwidth Retr
    [ 4] 0.00-10.00 sec 649 MBytes 544 Mbits/sec 51 sender
    [ 4] 0.00-10.00 sec 647 MBytes 542 Mbits/sec receiver

    And here are the results of same test, but with the APU2 acting as the router. As you can see, it’s able to max out my connection without breaking a sweat.

    Connecting to host loganmarchione.com, port 5201
    [ 4] local 10.10.2.34 port 51548 connected to 68.183.148.132 port 5201
    [ ID] Interval Transfer Bandwidth Retr Cwnd
    [ 4] 0.00-1.00 sec 53.7 MBytes 450 Mbits/sec 9 2.21 MBytes
    [ 4] 1.00-2.00 sec 67.2 MBytes 563 Mbits/sec 1 2.23 MBytes
    [ 4] 2.00-3.00 sec 67.2 MBytes 564 Mbits/sec 0 2.25 MBytes
    [ 4] 3.00-4.00 sec 67.2 MBytes 564 Mbits/sec 0 2.27 MBytes
    [ 4] 4.00-5.00 sec 67.2 MBytes 563 Mbits/sec 0 2.29 MBytes
    [ 4] 5.00-6.00 sec 67.2 MBytes 564 Mbits/sec 0 2.32 MBytes
    [ 4] 6.00-7.00 sec 67.2 MBytes 563 Mbits/sec 0 2.34 MBytes
    [ 4] 7.00-8.00 sec 67.2 MBytes 564 Mbits/sec 0 2.38 MBytes
    [ 4] 8.00-9.00 sec 67.1 MBytes 563 Mbits/sec 0 2.47 MBytes
    [ 4] 9.00-10.00 sec 67.2 MBytes 564 Mbits/sec 1 2.51 MBytes
    - - - - - - - - - - - - - - - - - - - - - - - - -
    [ ID] Interval Transfer Bandwidth Retr
    [ 4] 0.00-10.00 sec 658 MBytes 552 Mbits/sec 11 sender
    [ 4] 0.00-10.00 sec 657 MBytes 551 Mbits/sec receiver

    I have heard nothing but good things about the APU2, and am so far impressed!

     

    -Logan

    17 thoughts on “pfSense on the PC Engines APU2”

    1. Hi you mentioned “the number of NICs (2, 3, or 4 – this is where you get APU2, APU3, or APU4)”

      Slight correction here. The number of NICs is not determined by the APU#. Case and point, “APU2d4” is “APU” platform generation “2” board revision “d” with “4” gigs of ECC RAM. The current models of the APU2 platform consist of the following.

      apu2d0 (2 GB DRAM, 2 i211AT NICs)
      apu2e2 (2 GB DRAM, 3 i211AT NICs)
      apu2e4 (4 GB DRAM, 3 i210AT NICs)
      apu3c2 (2 GB DRAM, 3 i211AT NICs, optimized for 3G/LTE modems)
      apu3c4 (4 GB DRAM, 3 i211AT NICs, optimized for 3G/LTE modems)
      apu4d2 (2 GB DRAM, 4 i211AT NICs)
      apu4d4 (4 GB DRAM, 4 i211AT NICs)

      Reply
    2. No prob. BTW I have the APU2e4 and get full speed [940 dn/940 up] on multi-que speed test using iperf3, but single-que maxes out at around [740 dn /940 up]. This is with all the mentioned tweaks from Teklager. My guest is that there are some other settings to look at. If you haven’t found this page yet take a look. It has some nice configuration suggestions. https://nguvu.org/

      Reply
      • Yep, I’ve seen both the Teklager and nguvu guides, but I haven’t set anything in my config file since my internet is only 400/400 right now. I didn’t however use the nguvu guide for general setup and guidance. You’re not doing anything else on your APU2, right? Like IDS or IPS?

        Reply
    3. VLAN’s for network segmentation for IOT, Gaming, Printer, Guest Wifi, NAS. All separated with limited access in case one device is compromised the net result is minimal. Also using DNSSEC via TLS for secure DNS queries. pfBlockerNG for domain restrictions.

      Reply
    4. If you can, verify your laptop and USB to Serial Adapter with another device beyond just hairpinning SD and RD with a paperclip.

      I am having some real trouble here. Can’t get a reliable connection into the serial port. Definitely have a good null modem cable and my settings are according to instructions ( 115200-8N1 ). It spits out “garbage” then the pfSense initial dialog shows up but when it starts booting it seems to hang with more garbage characters being emitted.

      I eventually determined this was a problem between my laptop and serial adapter. I broke out my old 2012 HP Laptop, albeit running Mint 19.3, and I had no issues from there. That was almost two days wasted. The HP laptop was my work laptop when I was connecting to a lot of Cisco switches via console so I knew it was a sure thing. Problem is, I am at home so don’t have other equipment to verify with, like a switch with a console port. If I hadn’t been able to get a switch to talk to me I would have known it was my issue with adapter or laptop.

      Reply
      • Ugh that sucks, who would have guessed it would be the serial adapter/laptop combo? This is the only device I use that has an actual RS-232 port, so I may pickup an extra adapter just to be safe. Thanks for sharing!

        Reply
    5. Hows this setup working now that you’ve been using it for a while? I’m on edge lite and interested in what you consider improvements from that to your new setup? Anything cons?

      Thanks

      Reply
      • That’s a really good question!

        I love pfSense and the APU2 combo. I’ve performed a pfSense version upgrade and multiple APU2 BIOS upgrades without issue. To be honest, this is the one device on my network I can basically forget about because it runs so smoothly. I basically just expect it to work 100% of the time.

        My only con of the APU2 is the old/slow CPU. It’s fine for my use case, which is only routing and firewall (e.g., no IDS/IPS). I think if I started doing any IDS/IPS, the CPU would start to choke. Also, I’ve heard the APU2 can only handle connections speeds of around 600Mbps (without tuning), but my internet is only 400Mbps, so I have yet to see a bottleneck.

        Also, not a con of the APU2, but pfSense lacks an API or way to configure the router programmatically. You basically need to write down all of your settings in the web UI if you’re doing a new install, whereas on EdgeOS you can copy/paste your configuration commands.

        For me, I wasn’t a fan of the direction Ubiquiti was heading. They seemed to be creating products no one was asking for, while pushing beta software to users as “stable”. I want my router to be secure and stable, not full of flashy new features. However, I do prefer the zone-based firewall of EdgeOS, whereas the interface-based firewall of pfSense can be sometimes hard to wrap my head around (e.g., traffic from out of VLAN10, into the LAN, etc…).

        If I were to switch from EdgeOS, I think I would give VyOS a try. It’s built on the same base as EdgeOS, but isn’t customized by Ubiquiti. They offer an open-source rolling release, or a stable release that is paid. However, I believe you can get the stable if you either build it yourself from source, or contribute back to the project.

        Reply
    6. Hi Logan, great, timely blog post! I had a similar journey, went from Edgerouter X to SG-1100. My Netgate router went bad after a couple of months (kept rebooting periodically, sometimes in the middle of my “work from home” day). Netgate processed RMA, but it was a repair, rather than a swap of the hardware. Good that I still had my old router handy. I’m glad that I saw your post before I ordered the APU2, got the model with the i210AT NICs (APU2E4) directly from PC Engines in Switzerland. Very good shopping experience, fast shipping via UPS (S&H was 25 USD to California). Your links were helpful, especially the tips related to BIOS update. I’m still learning to navigate pfSense, picked up some limited knowledge online. I’d be interested to see a pfSense blog from you in the future, best practices, especially related to firewall rules, etc. Thanks, stay safe and healthy!

      Reply
      • Glad I was able to help you get started! I was going to post a how-to guide for pfSense, but it can quickly get out of date and there is already so much good material out there.

        Lawrence Systems has a really good YouTube channel that I recommend. The owner (Tom) does cabling and network installs for a living, so he knows his networking. They publish a ton of videos on pfSense and Ubiquiti solutions.
        https://www.youtube.com/watch?v=fsdm5uc_LsU

        Also, this site has an always up-to-date pfSense setup guide.
        https://nguvu.org/pfsense/pfsense-baseline-setup/

        Reply
    7. Good write up. I’m going to add this to my list of considerations. I also would like to go low power, but it’s my least important requirement. I am also considering a server between $250-400. I can get a decent HP DL360 G8 or Dell PowerEdge for around the same price. Not in the long run, of course, if you consider the electricity!

      Reply
    8. Logan,
      This is an amazing post and has really helped me make a decision on the pfsense box I need. I was using the USG3 and good Lord, it sucks, I agree 100% that Ubiquiti needs to clear their thoughts on “stable” release vs beta. I am a100% remote employee and need my internet to be up and running at al times.
      I had one question about the throughput- call me crazy but will I be able to run pfsense and an IDS/IPS like Suricata and get a decent throughput? I currently have 500/500 FiOS, have about 50 connected devices.

      Appreciate the good work!
      Sam

      Reply
      • Glad to help!

        I don’t run an IDS/IPS myself (yet). I believe the APU2 can run some basic services like Suricata, but you may not be able to get your full 500/500 while doing it. You may need to step up to a Protectli box for that kind of horsepower.

        Reply

    Leave a Comment

    This site uses Akismet to reduce spam. Learn how your comment data is processed.