u/therealmcz

Hi everyone,

please help, I'm unable to configure cilium + gateway API... the gateway just is stuck in the "programmed=false" state. Tried so many things but no success, also checked this post but if I'm correct, my current version should have the fix included:

https://github.com/cilium/cilium/pull/46350

Nodes:

NAME                         STATUS   ROLES           AGE     VERSION   INTERNAL-IP    EXTERNAL-IP   OS-IMAGE                         KERNEL-VERSION                          CONTAINER-RUNTIME
n1.k8s.net   Ready    control-plane   3d13h   v1.37.1   192.168.78.3   <none>        AlmaLinux 10.2 (Lavender Lion)   6.12.0-211.61.1.el10_2.x86_64 (amd64)   cri-o://1.37.2
n2.k8s.net   Ready    control-plane   13h     v1.37.1   192.168.78.4   <none>        AlmaLinux 10.2 (Lavender Lion)   6.12.0-211.61.1.el10_2.x86_64 (amd64)   cri-o://1.37.2
n3.k8s.net   Ready    <none>          13h     v1.37.1   192.168.78.2   <none>        AlmaLinux 10.2 (Lavender Lion)   6.12.0-211.61.1.el10_2.x86_64 (amd64)   cri-o://1.37.2

Cilium:

cilium status
    /¯¯\
 /¯¯\__/¯¯\    Cilium:             OK
 \__/¯¯\__/    Operator:           OK
 /¯¯\__/¯¯\    Envoy DaemonSet:    disabled (using embedded mode)
 \__/¯¯\__/    Hubble Relay:       disabled
    \__/       ClusterMesh:        disabled

DaemonSet              cilium                   Desired: 3, Ready: 3/3, Available: 3/3
Deployment             cilium-operator          Desired: 1, Ready: 1/1, Available: 1/1
Containers:            cilium                   Running: 3
                       cilium-operator          Running: 1
                       clustermesh-apiserver
                       hubble-relay
Cluster Pods:          6/6 managed by Cilium
Helm chart version:    1.20.2
Image versions         cilium             quay.io/cilium/cilium:v1.20.2@sha256:2939231d0d3e3ebddcd80fffa168b7ddcc78fdf0dc864d1c8c126ff523c54f01: 3
                       cilium-operator    quay.io/cilium/operator-generic:v1.20.2@sha256:64d8798350e8569b8e7622563fed6e44dce2625f311e4651b774816516c744fc: 1

GatewayClass:

kubectl describe gc
Name:         cilium
Namespace:
Labels:       app.kubernetes.io/managed-by=Helm
Annotations:  meta.helm.sh/release-name: cilium
              meta.helm.sh/release-namespace: kube-system
API Version:  gateway.networking.k8s.io/v1
Kind:         GatewayClass
Metadata:
  Creation Timestamp:  2026-10-10T07:23:46Z
  Generation:          1
  Resource Version:    592430
  UID:                 2d99e2fe-ab22-451f-8f71-f09968df4e52
Spec:
  Controller Name:  io.cilium/gateway-controller
  Description:      The default Cilium GatewayClass
Status:
  Conditions:
    Last Transition Time:  2026-10-10T07:24:02Z
    Message:               Valid GatewayClass
    Observed Generation:   1
    Reason:                Accepted
    Status:                True
    Type:                  Accepted
  Supported Features:
    Name:  BackendTLSPolicy
    Name:  GRPCRoute
    Name:  GRPCRouteNamedRouteRule
    Name:  Gateway
    Name:  GatewayAddressEmpty
    Name:  GatewayFrontendClientCertificateValidationInsecureFallback
    Name:  GatewayHTTPListenerIsolation
    Name:  GatewayInfrastructurePropagation
    Name:  GatewayPort8080
    Name:  GatewayStaticAddresses
    Name:  HTTPRoute
    Name:  HTTPRoute303RedirectStatusCode
    Name:  HTTPRoute307RedirectStatusCode
    Name:  HTTPRoute308RedirectStatusCode
    Name:  HTTPRouteBackendProtocolH2C
    Name:  HTTPRouteBackendProtocolWebSocket
    Name:  HTTPRouteBackendRequestHeaderModification
    Name:  HTTPRouteBackendTimeout
    Name:  HTTPRouteCORS
    Name:  HTTPRouteDestinationPortMatching
    Name:  HTTPRouteHostRewrite
    Name:  HTTPRouteMethodMatching
    Name:  HTTPRouteNamedRouteRule
    Name:  HTTPRoutePathRedirect
    Name:  HTTPRoutePathRewrite
    Name:  HTTPRoutePortRedirect
    Name:  HTTPRouteQueryParamMatching
    Name:  HTTPRouteRequestMirror
    Name:  HTTPRouteRequestMultipleMirrors
    Name:  HTTPRouteRequestPercentageMirror
    Name:  HTTPRouteRequestTimeout
    Name:  HTTPRouteResponseHeaderModification
    Name:  HTTPRouteRetry
    Name:  HTTPRouteRetryBackendTimeout
    Name:  HTTPRouteRetryConnectionError
    Name:  HTTPRouteSchemeRedirect
    Name:  ListenerSet
    Name:  Mesh
    Name:  MeshClusterIPMatching
    Name:  MeshHTTPRouteBackendRequestHeaderModification
    Name:  MeshHTTPRouteNamedRouteRule
    Name:  MeshHTTPRouteQueryParamMatching
    Name:  MeshHTTPRouteRedirectPath
    Name:  MeshHTTPRouteRedirectPort
    Name:  MeshHTTPRouteRewritePath
    Name:  MeshHTTPRouteSchemeRedirect
    Name:  ReferenceGrant
    Name:  TCPRoute
    Name:  TLSRoute
    Name:  TLSRouteModeMixed
    Name:  UDPRoute
Events:    <none>

Gateway:

kubectl describe gtw
Name:         nodeport-gateway
Namespace:    default
Labels:       <none>
Annotations:  <none>
API Version:  gateway.networking.k8s.io/v1
Kind:         Gateway
Metadata:
  Creation Timestamp:  2026-10-10T07:35:16Z
  Generation:          4
  Resource Version:    607512
  UID:                 f6b3b043-d072-4e6c-8f37-bf28cb37a1b2
Spec:
  Gateway Class Name:  cilium
  Listeners:
    Allowed Routes:
      Namespaces:
        From:  Same
    Name:      web-gw-80
    Port:      80
    Protocol:  HTTP
Status:
  Conditions:
    Last Transition Time:  2026-10-10T08:53:31Z
    Message:               Gateway successfully scheduled
    Observed Generation:   4
    Reason:                Accepted
    Status:                True
    Type:                  Accepted
    Last Transition Time:  2026-10-10T08:53:31Z
    Message:               Gateway waiting for address
    Observed Generation:   4
    Reason:                AddressNotAssigned
    Status:                False
    Type:                  Programmed
  Listeners:
    Attached Routes:  1
    Conditions:
      Last Transition Time:  2026-10-10T08:53:31Z
      Message:               Resolved Refs
      Observed Generation:   4
      Reason:                ResolvedRefs
      Status:                True
      Type:                  ResolvedRefs
      Last Transition Time:  2026-10-10T08:53:31Z
      Message:               Listener Accepted
      Observed Generation:   4
      Reason:                Accepted
      Status:                True
      Type:                  Accepted
      Last Transition Time:  2026-10-10T08:53:31Z
      Message:               Address not ready yet
      Observed Generation:   4
      Reason:                Pending
      Status:                False
      Type:                  Programmed
    Name:                    web-gw-80
    Supported Kinds:
      Group:  gateway.networking.k8s.io
      Kind:   HTTPRoute
      Group:  gateway.networking.k8s.io
      Kind:   GRPCRoute
Events:       <none>

Happy for any input, thanks a lot!

Hi everyone,

I've been playing around with bgpalerter for a AS and so far it's working fine. I do get notifications once prefixes are withdrawn according to the threshold, however I do NOT get notifications once upstream bgp-sessions are lost, which seems weird. Of course, your prefix would still be announce through other upstream-providers, but in any case I'd like to get notified when I loose connection to an upstream...

My current config looks like this:

config.yaml

configVersion: 3
environment: production
connectors:
  - file: connectorRIS
    name: ris
    params:
      carefulSubscription: true
      url: ws://ris-live.ripe.net/v1/ws/
      perMessageDeflate: true
      subscription:
        moreSpecific: true
        type: UPDATE
        host: null
        socketOptions:
          includeRaw: false
monitors:
  - file: monitorHijack
    channel: hijack
    name: basic-hijack-detection
    params:
      thresholdMinPeers: 3
  - file: monitorPath
    channel: path
    name: path-matching
    params:
      thresholdMinPeers: 1
  - file: monitorPathNeighbors
    channel: neighbors
    name: path-neighbors
    params:
      thresholdMinPeers: 1
  - file: monitorNewPrefix
    channel: newprefix
    name: prefix-detection
    params:
      thresholdMinPeers: 3
  - file: monitorVisibility
    channel: visibility
    name: withdrawal-detection
    params:
      thresholdMinPeers: 1
      notificationIntervalSeconds: 3600
  - file: monitorAS
    channel: misconfiguration
    name: asn-monitor
    params:
      skipPrefixMatch: false
      thresholdMinPeers: 3
  - file: monitorRPKI
    channel: rpki
    name: rpki-monitor
    params:
      thresholdMinPeers: 3
      checkUncovered: false
      checkDisappearing: false
  - file: monitorROAS
    channel: rpki
    name: rpki-diff
    params:
      enableDiffAlerts: true
      enableExpirationAlerts: true
      enableExpirationCheckTA: true
      enableDeletedCheckTA: true
      enableAdvancedRpkiStats: false
      roaExpirationAlertHours: 2
      checkOnlyASns: true
      toleranceDeletedRoasTA:
        ripe: 20
        apnic: 20
        arin: 20
        lacnic: 20
        afrinic: 50
      toleranceExpiredRoasTA:
        ripe: 20
        apnic: 20
        arin: 20
        lacnic: 20
        afrinic: 50
  - file: monitorPathNeighbors
    channel: path
    name: path-neighbors
    params:
      thresholdMinPeers: 1
reports:
  - file: reportFile
    channels:
      - hijack
      - newprefix
      - visibility
      - path
      - misconfiguration
      - rpki
      - neighbors
  - file: reportEmail
    channels:
      - hijack
      - newprefix
      - visibility
      - path
      - misconfiguration
      - rpki
      - roa
      - neighbors
    params:
      <removed>
notificationIntervalSeconds: 86400
alarmOnlyOnce: false
monitoredPrefixesFiles:
  - prefixes.yml
persistStatus: true
generatePrefixListEveryDays: 0
logging:
  directory: logs
  logRotatePattern: YYYY-MM-DD
  maxRetainedFiles: 10
  maxFileSizeMB: 15
  compressOnRotation: false
rpki:
  vrpProvider: rpkiclient
  refreshVrpListMinutes: 15
  markDataAsStaleAfterMinutes: 120
rest:
  host: localhost
  port: 8011
checkForUpdatesAtBoot: true
pidFile: bgpalerter.pid
fadeOffSeconds: 360
checkFadeOffGroupsSeconds: 30

prefixes.yaml

<prefix(removed)>:
  description: No description provided
  asn:
    - my-asn-no#
  ignoreMorespecifics: false
  ignore: false
  group: noc
options:
  monitorASns:
    'my-asn-no#':
      group: noc
      upstreams:
        - asn1
        - asn2
        - asn3
      downstreams: []
      neighbors:
        - asn1
        - asn2
        - asn3
  generate:
    exclude: []
    excludeDelegated: true
    monitoredASes:
      - 'my-asn-no#'
      - 'asn1'
      - 'asn2'
      - 'asn3'
    historical: false
    group: noc
    asnList:
      - 'my-asn-no#'

Thanks a lot!

Hi everyone,

tried to create a cronjob that would be run every minute. So on the FTD, I did the following (yes, wannt to execute that on the ftd):

"crontab -e" and added the following line:

* * * * * /etc/cron.hourly/cleanup

The "cleanup" in the referenced directory has the following permissions:

-rwxr-xr-x 1 root root

Now I realized that the command that is within the "cleanup" has to be run in the default FTD shell and not within expert mode...

Is that possible via FTD cronjob? If not, would it work via FMC?

thanks a lot!

Hi everyone,

got a FTD and a core firewall. The philosophy is to route everything to the core firewall and this won't be changed, so please be aware of that and don't answer "just create a policy on the FTD" - thanks.

There is a S2S-VPN on the FTD to provider A, hosting 10.10.10.0/24. Then there is the remote access VPN for Cisco anyconnect. So far, so good. The default for the remote access points to the core firewall, but when the user connects to the FTD, traffic to 10.10.10.0 will be routed to the S2S directly as the route is more specific...

According to a collegue, you can't override that route just for the remote access, you could only create a separate default route (which doesn't help here).

So the option was to use VRFs, but here it gets even more complicated as you can't use the same VLAN ID on two seperate physical interfaces (forti could do that, so cisco, why can't you?) which means that you had to create seperate VLANS, seperate routing, seperate subnets.... So much effort for such a "little problem".

Has anyone an idea how to fix that on the FTD without having to kind of install the whole system from scratch? Of course, I could just let the tunnel terminate elsewhere, but that prefered way was to leave everything on the FTD and just change the config or just add some cables for a seperate VRF. Reminder: It's not a valid option to let the traffic directly break out on the FTD, it has to be routed to the core firewall.

Thanks a lot!

Hi everyone,

I'm using a virtual server in front of a webserver, so far it works perfectly with SSL offload and what have you but now I have to redirect/forward calls from "mydomain.com" to "mydomain.com/entrypoint" and now I'm struggling.

I guess this is not provided by forti and so you'd have to use nginx or other webervers? Or is there a baked in solution?

Thanks a lot!

Hi everyone,

can please someone give me and indication what cilium enterprise might cost? Just an indication, so that I know if it costs 5k, 20k or maybe 50k per year.

Thanks a lot!

Hi everyone,

as almost everyone here knows, FortiClient sucks. So, is there a good alternative if you're using only IPsec VPN in combination with SSO (EntraID)? FortiOS version 7.4.x

Curious about your experiences and recommendations?

Thanks!

Hi everyone,

got an annoying topic here... A customer is using openshift k8s and the ovn stack in combination with egressFirewall... As soon as you're having cloudservices like smtp.office365.com or others, you're in trouble.

What we have found out so far is that ovn is hitting a race condition: The production pod tries to connect to smtp.office365.com and does a dns-lookup, same query is being sent by the OVN pod to allow the outgoing connection. As long as both pods get the same results, everything is fine. But due to the short TTL and some strange DNS-cache-facts, it (and this is confirmed) can happen that those pods get different results - leading to the fact that the connection might fail here and there.

Now here comes the real question: As soon as we're doing the firewalling on the forti, everything is fine. But even there it must be the same situation: The production pod does a DNS lookup and forti does the same - sooner or later it should lead into the race condition that the pod gets an "old" result, while the forti gets the current version of the DNS reply and boom. When you're having multiple DNS servers you can even have different results from DNS to DNS depending on the TTL/caching mechanism.

So how does forti handle FQDN policies? Does it ask every DNS-Server and then merge everything together into a map? And does it maybe extend the TTL to make sure that you don't have those race conditions?

Because it could even happen with a longer TTL - when someone changes the DNS-records and sessions are connected to destination A while the current DNS query returns destination B, it shouldn't drop the session, right?

Thanks a lot for the insights!

PS: Please don't just give advice like "shift to another k8s network stack" because right now I'm just interested into the logic of the forti and it's not possible to change anything on that stack right now. thank you

Hi everyone,

I'm struggling as hell with just getting a valid uplink on the r5000-series. Just plugged in a F5-SFP+ (OPT-0016-00) and plugged the "other" side of the patchcable (OM4) into a cisco transceiver. RX signals are OK there but both the switch and the F5 tells me "notconnected". No matter what, used different transceivers, switches, cables, breakout cables, etc. Played around setting the portspeed, but no chance.

Also created a loop by using 2 of those transceivers, plugged them into the F5 and used the patchcable - still down in the GUI.

The port in the F5-GUI is enabled and that's more or less all that I can change there.

Any advice? Does F5 ignore other transceivers than the ones from F5? Can I disable that?

Thanks a lot!

Hi everyone,

can anyone please give me an indication what a WAF-license for a single VM with say 1 core and 100 mbit/s bandwidth might cost? Or in other words: What's the lowest price for a WAF-license on a vm and what specs would that cover?

Thanks!

Hi everyone,

I'm having a Cisco IR1101 (managed via SD-WAN) which has a M2M-SIM configured. On one device there is quite a bad signal, but the device wouldn't switch to a "better" network. If I execute "cellular 0/1/0 lte plmn select Auto" it suddenly selects a better provider (which is provided by the SIM) but I also had the case where it suddenly would fall back to the first provider with much worser signal rates.

Does anyone know

- why it would never switch providers on its own
- why it does it when triggered manually when PLMN is configured in auto-mode
- why it doesn't stick to the better network
- how to manually pin it to a certain provider
- is there a way to trigger an automatic provider switch on a certain threshold

Thanks a lot!

Hi everyone,

I#m having an issue with an IR1101, which doesn't establish a valid SDWAN-connection. Support is focusing on the message "%CELLWAN-2-CALL_SETUP_FAIL: Cellular0/1/0 data call setup failed due to mismatching IP address", but I'm very sure that this isn't the issue as I can pin it down to the modem. Replace the modem and the same config works.

Used the same SIM, different SIM, upgraded firmware, etc. - no changes. The SIM is a M2M-SIM which has a fixed IP on this ICCID and so the modem should always get this IP - which it does because you can ping this IP from "the outside".

Now the issue is on thing but what I'd like to understand is the message:

"%CELLWAN-2-CALL_SETUP_FAIL: Cellular0/1/0 data call setup failed due to mismatching IP address"

What does it try to tell me? What is "mismatching"? With what is it matched? btw, the IP it receives is a RFC1918 IP. Please note that I don't see the IP on the cellular interface when doing "show ip int brief", it just lists "unassigned", which is weird, because you can ping this IP in the meantime.

Thanks a lot!