u/Rich-Engineer2670

OK -- before I lose what little sanity I have.... this SHOULD work, but doesn't. Here is the netplan file I have for Ubuntu. Notice one IPv6 address is oommented out. It doens't matter which one I comment out, but if both are active, at random, only one works. If I comment out one, the other routes. I THOUGHT I had source routing rgiht.... I guess not.

network:
  version: 2
  renderer: networkd
  ethernets:
    enp4s0:
      dhcp4: false
      dhcp6: false
      addresses:
        - 10.0.1.2/16
        - 50.154.105.131/28
        - 199.181.204.131/25
        - 2001:55d:9080::0a00:0102/64
        #- 2602:805:A000::0a00:0102/64
      gateway4: 10.0.0.2
      gateway6: 2001:55d:9080::1
      nameservers:
        addresses:
        - 10.0.1.1
      routes:
        - to: default
          via: 50.154.105.129
          table: 100
        - to: default
          via: 199.181.204.129
          table: 101
        - to: default
          via: 2001:55d:9080::1
          table: 102
        - to: default
          via: 2602:805:A000::1
          table: 103
      routing-policy:
        - from: 50.154.105.128/28
          table: 100
        - from: 199.181.204.128/25
          table: 101
        - from: 2001:55d:9080::/64
          table: 102
        - from: 2602:805:A000::/64
          table: 103

What I THINK this says is (ignoring V4 which works), if the source address is 2602:805:A000::/48, route to 2602:805:A000::1. If it's 2001:55d:9080::/48 route to 2001:55d:9080::1. BUT....l.

This is Ubuntu 26.04 desktop. So if you want to win the Dodge Desoto with Groucho Marx, say the secret words here -- I've been saying a lot secret words, but it's frustrating so I can't say any of them in public.

The 2Gb fiber is installed, but it has its quicks.

• It's SFP+ to me, it expects to hand me IPs via a /30 and /126.
• I run it through a 2Gb switch and I'll send it through a 5009. This means SFP+ comes in, and the 5009, captures the internal IPs of the /30 and /126. It then puts them on its SFP+ port
• Now, on that SFP+ I have the /28 and /48 as expected
• One of those /28s feeds yet another 5009 which sets up two GRE tunnels. (Primary and backup). It captures the returned IPs, and sends them to the same switch the SFP+ of the first 5009. Now at THAT switch, I have all captured IPs, the /28 and /48 from the fiber link, and the /24 and /40 from the GRE tunnel
• That switch feeds internal nodes INCLUDE (gag!) a THIRD 5009, which does things like NATing when needed

Please tell me there's a better way. A /28 and /48 from fiber and a /24 and /40 from GRE, have to go internal and possibly be NATed. In the old days, I would have a server running VMWare, and one or more instances of the CHRs -- what is the thing to do today?

After the great flow control debate, when the fiber was installed, we were able to prove (shock!), Comcast has problems. The fiber connection, does not. The fiber comes in to a 2.5Gb switch on the SFP+ port, and then 2.5 connections go to two Mikrotik RB5009s. (One is for special tunnels).

It would be nice if I had a switch/router from MKT that could take multiple 2.5Gb connections, then the tunnel RB5009 could just feed the switch which would do basic layer-3 firewall work to systems downstream. The 5009 could just concentrate on tunnels and leave the filtering to the node downstream.

Is there such a device right now, and, for example, can it layer-3 rules (no encryption, no tunnels, just access rules at 2.5Gb).

I have a setup right now, where the 5009 feeds a 4011.

OK -- some progress, but more questions than answers....

What we have:

• New RB5009 with a 2Gb copper connection on ether1 (7.22.2)
• New 6A cables
• New 2Gb with switch 10Gb uplink
• Either
  • 10Gb connection on SFP+ port to SFP+ port on a 2Gb switch (the port is 10Gb of course). 5009 ports are NOT using a bridge (makes no difference if they are)
  • 1Gb copper on ether2 to 2Gb switch via 2Gb port
• What we see
  • If we're using the 2Gb/1Gb copper arrangement, I'd expect to see around 1Gb/385 from the ISP. I get about 650/385. (CPU flow control on)
  • With CPU flow control OFF, it's around 250'/385

HELP! I have no idea what's going on! The Mikrotik forum folks suggest I turn flow control off and then fix the CPU frequency at 1400Mhz. For what it's worth, if I take a laptop and correct it directly to the cable modem, I get the expected 2Gb/365 or so as expected. When I run it through the Mikrotik, I get the throughput loss.

To verify things, I've even done a truly default config -- no change. To absolutely confirm if it's the Mikrotik, I'll try a 2.5Gb router as an alternate.

I'm not sure if I'm in the right place for this but I was wondering if there's a solution for IPv6 portability.

In V4, ugly as it is, we had NAT so if you switched providers, your internal addresses never knew about it. In v6, there is no NAT (thank God), but it makes moving ISPs a pain. What I imagined, such as it is --

• When you go to your RIR, they have a new question -- basically, are you an ISP who can do all of the items we do today, BGP, RPKI etc. If so, no changes. If not:
• We get your your portable V6 allocation and an ASN -- no changes. You don't have to do the other items, because the RIR fills it all out autoamtically.
• You choose your ISP and they give you a "provider ID"
• You give your new ISP your ASN and a one-time code (much like phone number portability) and it "transfers" the BGP for that ASN to their control.
• When you decide to change, you get the new provider ID and give them your ASN
• They can look up that ASN at the RIR and find your still with ISP-A.
• They (ISP-B) send you a key, you give that ISP-A to say "transfer contorl to ISP-B"

Is this crazy?

I'm not sure what I want to do works the way I imagine, but then agian, people question my imagination all the time 😄

• I happen to be using a Mikrotik RB5009 software release 7.22.2
• I have two GRE tunnels to transit providers available to me
• I was hoping to have tunnel1 connect to ISP1 and its BGP announcements and tunnel2 to ISP2 and its announcements
• That seems OK -- it's multi-homed BGP and one gets pre-pended BUT
  • If GRE tunnel1 goes down, OK, I'd lose the announcements from BGP1. BGP2 would eventually take over
  • If GRE tunnel2 goes down, same thing
• But what if I have this

BGP1. BGP2
|tunnel1 | tunnel2
Router1 --- link -Router2

If I'm on default, everything gtoes to BGP1 though tunnel1, and router2 gets its traffic from router1 over link.

If BGP1 or tunnel1 fails, BGP will send everything to router2 and it has to know to reach router1 via the link. In effect, how does Router1 know Router2 is handling the routes via itself or vice versa.

I'm trying to have two BGP announcement points from two ISPs to two routers across the country. Each router also has a private link to he other. If I had large enough blocks, say a /16, I could give each router a sub block and let its sister router handle the other block by default, but my V4 is only a /24. (I have more than enough V6)

OK, I clearly don't understand the use of the RB5009 switch chip.... What I'm trying to do:

1. WAN port: 2Gb/s over copper (ether1)
2. LAN (SFP+ to 2Gb switch) or 1Gb on ether2 over copper
3. I've tried it both with no conifugration (no switch setup) and the default configuration

What I get:

• If I try to use the 2Gb->SFP+ with and without default configurations, with a 2Gb connection, I see perhaps 450Mb down and 355Mb up
• If I use the copper connection ether1<->ether2 (no SFP+, pure copper), things improve, but it's still quite variable
• If I just take a test laptop and connect it directly over 2Gb copper to the cable modem, I get the expected speeds

I've replaced the cables (cat6), the 5009 itself, and the switch.

So, assuming I am clueless, how would YOU do the following:

• Latest RB5009 w. 7.22.3
• 2.5Gb 24 port switch with two 10Gb ports and 24 2.5 ports
• ISP providing 2Gb over copper
• Test laptop has 2Gb copper
• The switch can be connected via SFP+ cable between switch and RB5009 or copper

How would yuou set this up using SFP or copper-to-copper? I had imagined, but I'm probably wrong, that I could remove all ports on the bridge and the bridge itself and just two ether1 and ether2 as separate interfaces (or ether1 and SFP+) -- it seemed to work in the past. And, I hate to even ask this, but I'm waiting for the 2Gb fiber install... If Mikrotik can't do it until issues are fixed, what do people recommend that can? For example, can a Protectli box running OpnSense do this reliably, or can CHR running under a VM?

The problem -- for a couple of years now, Comcast (not my choice), provided 1Gb/355Mb business service to my site. It was directly connected to an RB5009. And, for the most part, it worked.

About four months ago, it dropped to 600Mb/355 -- I called Comcast out, and they did their ususal replace bad lines, rusted taps, etc, but this time, no improvements.

The RB5009 has a minimal configuration - a WAN static IPv4 block, LAN block via NAT, literally, for testing all that exists in the firewall section are a passthrough rule, and in the NAT section a masquerade rule. That's it!

Credit to Greg Bob @ Comcast SF Bay maintenance who spent nearly two hours trying to figure this out, step by step, link by link, but he's as confused as I am.

If he connects his 1Gb laptop directly to the replaced Comcast modem, he gets around 1Gb down (850-950 sometimes but that's OK) and 350+ Mb up. We'd expect thatt. Plug the same laptop into a spare port on the 5009 which directly connects to the modem and the throughput collapse. Why???? The router is fine so far as we can tell, the cables are new cat 6, the modem is brand new.

What are we both missing here?

# software id = RXQW-C5J2

#

# model = RB5009UG+S+

# serial number = HDK08H3R35B

/interface bridge

add admin-mac=48:A9:8A:25:67:B6 auto-mac=no comment=defconf name=bridge

/interface list

add comment=defconf name=WAN

add comment=defconf name=LAN

/interface wireless security-profiles

set [ find default=yes ] supplicant-identity=MikroTik

/ip pool

add name=default-dhcp ranges=192.168.88.10-192.168.88.254

/disk settings

set auto-media-interface=bridge auto-media-sharing=yes auto-smb-sharing=yes

/interface bridge port

add bridge=bridge comment=defconf interface=ether2

add bridge=bridge comment=defconf interface=ether3

add bridge=bridge comment=defconf interface=ether4

add bridge=bridge comment=defconf interface=ether5

add bridge=bridge comment=defconf interface=ether6

add bridge=bridge comment=defconf interface=ether7

add bridge=bridge comment=defconf interface=ether8

add bridge=bridge comment=defconf interface=sfp-sfpplus1

/ip neighbor discovery-settings

set discover-interface-list=LAN

/interface list member

add comment=defconf interface=bridge list=LAN

add comment=defconf interface=ether1 list=WAN

/ip address

add address=50.247.114.17/28 interface=ether1 network=50.247.114.16

add address=10.0.0.2/16 interface=sfp-sfpplus1 network=10.0.0.0

/ip dhcp-server

add address-pool=default-dhcp interface=bridge name=defconf

/ip dhcp-server network

add address=192.168.88.0/24 comment=defconf dns-server=192.168.88.1 gateway=\

192.168.88.1

/ip dns

set allow-remote-requests=yes servers=1.1.1.1,8.8.8.8

/ip dns static

add address=192.168.88.1 comment=defconf name=router.lan type=A

/ip firewall filter

add action=accept chain=input comment=\

"defconf: accept established,related,untracked" connection-state=\

established,related,untracked

add action=drop chain=input comment="defconf: drop invalid" connection-state=\

invalid

add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp

add action=accept chain=input comment=\

"defconf: accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1 \

in-interface=lo src-address=127.0.0.1

add action=drop chain=input comment="defconf: drop all not coming from LAN" \

in-interface-list=!LAN

add action=accept chain=forward comment="defconf: accept in ipsec policy" \

ipsec-policy=in,ipsec

add action=accept chain=forward comment="defconf: accept out ipsec policy" \

ipsec-policy=out,ipsec

add action=fasttrack-connection chain=forward comment="defconf: fasttrack" \

connection-state=established,related

add action=accept chain=forward comment=\

"defconf: accept established,related, untracked" connection-state=\

established,related,untracked

add action=drop chain=forward comment="defconf: drop invalid" \

connection-state=invalid

add action=drop chain=forward comment=\

"defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat \

in-interface-list=WAN

/ip firewall nat

add action=masquerade chain=srcnat comment="defconf: masquerade" \

ipsec-policy=out,none out-interface-list=WAN

/ip route

add disabled=no dst-address=0.0.0.0/0 gateway=50.247.114.30 routing-table=\

main

/ip service

set ftp disabled=yes

set telnet disabled=yes

set www address=10.0.0.0/16

set ssh address=10.0.0.0/16 port=2212

set winbox address=10.0.0.0/16

set api address=10.0.0.0/16

set api-ssl address=10.0.0.0/16

/ipv6 firewall address-list

add address=::/128 comment="defconf: unspecified address" list=bad_ipv6

add address=::1/128 comment="defconf: lo" list=bad_ipv6

add address=fec0::/10 comment="defconf: site-local" list=bad_ipv6

add address=::ffff:0.0.0.0/96 comment="defconf: ipv4-mapped" list=bad_ipv6

add address=::/96 comment="defconf: ipv4 compat" list=bad_ipv6

add address=100::/64 comment="defconf: discard only " list=bad_ipv6

add address=2001:db8::/32 comment="defconf: documentation" list=bad_ipv6

add address=2001:10::/28 comment="defconf: ORCHID" list=bad_ipv6

add address=3ffe::/16 comment="defconf: 6bone" list=bad_ipv6

/ipv6 firewall filter

add action=accept chain=input comment=\

"defconf: accept established,related,untracked" connection-state=\

established,related,untracked

add action=drop chain=input comment="defconf: drop invalid" connection-state=\

invalid

add action=accept chain=input comment="defconf: accept ICMPv6" protocol=\

icmpv6

add action=accept chain=input comment="defconf: accept UDP traceroute" \

dst-port=33434-33534 protocol=udp

add action=accept chain=input comment=\

"defconf: accept DHCPv6-Client prefix delegation." dst-port=546 protocol=\

udp src-address=fe80::/10

add action=accept chain=input comment="defconf: accept IKE" dst-port=500,4500 \

protocol=udp

add action=accept chain=input comment="defconf: accept ipsec AH" protocol=\

ipsec-ah

add action=accept chain=input comment="defconf: accept ipsec ESP" protocol=\

ipsec-esp

add action=accept chain=input comment=\

"defconf: accept all that matches ipsec policy" ipsec-policy=in,ipsec

add action=drop chain=input comment=\

"defconf: drop everything else not coming from LAN" in-interface-list=\

!LAN

add action=fasttrack-connection chain=forward comment="defconf: fasttrack6" \

connection-state=established,related

add action=accept chain=forward comment=\

"defconf: accept established,related,untracked" connection-state=\

established,related,untracked

add action=drop chain=forward comment="defconf: drop invalid" \

connection-state=invalid

add action=drop chain=forward comment=\

"defconf: drop packets with bad src ipv6" src-address-list=bad_ipv6

add action=drop chain=forward comment=\

"defconf: drop packets with bad dst ipv6" dst-address-list=bad_ipv6

add action=drop chain=forward comment="defconf: rfc4890 drop hop-limit=1" \

hop-limit=equal:1 protocol=icmpv6

add action=accept chain=forward comment="defconf: accept ICMPv6" protocol=\

icmpv6

add action=accept chain=forward comment="defconf: accept HIP" protocol=139

add action=accept chain=forward comment="defconf: accept IKE" dst-port=\

500,4500 protocol=udp

add action=accept chain=forward comment="defconf: accept ipsec AH" protocol=\

ipsec-ah

add action=accept chain=forward comment="defconf: accept ipsec ESP" protocol=\

ipsec-esp

add action=accept chain=forward comment=\

"defconf: accept all that matches ipsec policy" ipsec-policy=in,ipsec

add action=drop chain=forward comment=\

"defconf: drop everything else not coming from LAN" in-interface-list=\

!LAN

/system clock

set time-zone-name=America/Los_Angeles

/tool mac-server

set allowed-interface-list=LAN

/tool mac-server mac-winbox

set allowed-interface-list=LAN