Understanding certain websites connectivity issues and possible relevance to Unbound DNS
Hello everyone, I have moved from using commercially available routers to building my OPNsense router using an SFF PC with dual 10G card (WAN/LAN) and I have been quite happy with OPNsense and the flexibility it provides.
I have, however, noticed quite a bit of throttling on certain websites, like YouTube and Facebook where connection may get dropped, or just refuse to load then load fine after refresh.
My current set up:
VLANs: I have 4 VLANs (Admin, General, Guest, Webserver). I have two managed switches that are also configured to support those VLANs and no device on the whole network is VLAN aware as it is all handled by switches/router.
Unbound DNS: listens on port 53 across my 4 VLAN interfaces (WAN interface excluded). It is set up to use DNS over TLS with DNSSEQ enabled. I had Quad9 servers initially, but found them very unreliable and currently using Cloudfair and Google as secondary.
DNSMASQ: for local queries only (listens on port 53053) to resolve local network hostnames, queries targeting my router hostname will be forwarded to this by Unbound.
----------------------------------
I previously had many issues with Unbound timing out and thrashing, and if a sudden rush of DNS queries came from a device, Unbound would just stop responding altogether. I am talking about ~500 queries/s causing the service to shutdown. I have since expanded the cache size and the threads available, and stress tested the service by running +30k queries per second and it handled the test very gracefully and kept running just fine.
I also had issues with blocklists blocking many things I use, so they have now been completely disabled.
Last but not least, I do have several domains hosted and exposed using my public IP. That includes a mailserver, and several other domains/subdomains.
---------------------------------
With that in mind, I run into many issues with YouTube specifically, where it would disconnect completely, refuse loading comment/chat sections, constantly saying "Action not allowed", and this behavior is the same across all devices on the network, and on different YouTube accounts. Whenever I connect to VPN, those issues disappear for YouTube.
I have tried forcing DHCP release from my ISP by spoofing a MAC address for my WAN interface, it worked and I was able to get new IP (my ISP does not allow static IPs). That resolved the issue with YouTube for a few days, then it went back exactly to how it was before.
--------------------------------
I have thought of different reasons that could cause the issue:
- Misconfiguration in OPNsense on my end causing the behavior after some time passes.
- YouTube flagging my IP and throttling it.
- ISP is throttling my traffic to YouTube, given that is my most visited website in my network.
- My IP is being blacklisted on CDN level? Maybe the fact I have several sites hosted on the same IP is causing some issues flagging my IP in CDNs? I do know I am on two blacklists: RATS Dyna and Spamhaus ZEN, both of which list all DHCP ranges from commercial ISPs as blocked to prevent mail delivery. I could not find my IP blacklisted in other blacklists.
My IP is NOT behind GCNAT, my router WAN address is the same address exposed to the public.
My network devices are not compromised nor does any device act as part of any botnet, I do monitor my traffic.
I have been trying for months to determine what is happening and I am at a loss. I would like to rule out OPNsense from the equation, any suggestions to help me do that will be greatly appreciated.
Note: I will be changing ISP soon, and getting 2 static IPs, one will be dedicated specifically for my mailserver.
-----------------------------------
Edit: as a previous suspecion of mine was around the state of the connections, I decided to go back to that line of enquiry. I manually set WAN interface MTU to 1500, and MSS to 1440. Then I went to my General VLAN Firewall rules and changed State Type to "Sloppy State" instead of "Keep State" and immediately noticed the bad connectivity issues were gone. Of course this could be anecdotal, so I will be testing it out and I will update the post accordingly.
For reference, my Firewall rules are as follows:
Admin VLAN -> Can initiate to anywhere, nothing can initiate connections to it, they may only respond.
General VLAN -> Can initiate to Guest/Webserver VLANs, they can't initiate back.
Guest VLAN -> Can initiate to Webserver VLAN, Webserver can't initiate back.
Webserver -> Strictly internet traffic, cannot communicate to any internal VLANs, except if it's responding.