Skip to content

[DNM] Ovs cpu part2 saga revert set4 #2561

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: master
Choose a base branch
from

Conversation

tssurya
Copy link
Contributor

@tssurya tssurya commented May 10, 2025

MTU changes from RH and NVIDIA

@openshift-ci openshift-ci bot requested review from abhat and jcaamano May 10, 2025 18:31
Copy link
Contributor

openshift-ci bot commented May 10, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: tssurya

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 10, 2025
@tssurya tssurya changed the title Ovs cpu part2 saga revert set4 [DNM] Ovs cpu part2 saga revert set4 May 10, 2025
@tssurya
Copy link
Contributor Author

tssurya commented May 10, 2025

/hold

@openshift-ci openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 10, 2025
Copy link
Contributor

openshift-ci bot commented May 10, 2025

@tssurya: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-aws-ovn-hypershift-kubevirt 9985676 link false /test e2e-aws-ovn-hypershift-kubevirt
ci/prow/security 9985676 link false /test security
ci/prow/e2e-aws-ovn-hypershift-conformance-techpreview 9985676 link false /test e2e-aws-ovn-hypershift-conformance-techpreview
ci/prow/e2e-gcp-ovn-techpreview 9985676 link true /test e2e-gcp-ovn-techpreview
ci/prow/4.20-upgrade-from-stable-4.19-e2e-gcp-ovn-rt-upgrade 9985676 link true /test 4.20-upgrade-from-stable-4.19-e2e-gcp-ovn-rt-upgrade
ci/prow/e2e-gcp-ovn 9985676 link true /test e2e-gcp-ovn
ci/prow/e2e-aws-ovn-upgrade-local-gateway 9985676 link true /test e2e-aws-ovn-upgrade-local-gateway

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-merge-robot openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 11, 2025
trozet and others added 7 commits May 12, 2025 09:38
Tests:
 - host networked pod -> nodeport -> ovnk backend (UDP)
 - VM (1500 mtu) -> host networked pod on another node (TCP)
 - ovnk pod -> host networked pod on another node (UDP)

Each of these test would trigger a node to install a lower MTU route
in its cache towards another node, due to PMTUD. The test fails if it
detects such a route.

Signed-off-by: Tim Rozet <[email protected]>
(cherry picked from commit f8bf30b)
Create nftables rules to block sending ICMP needs frag/packet too big
for known Kubernetes node IPs. PMTUD between nodes can be deterimental
to the cluster. Note, this does not affect PMTUD messages received from
an intermediary router. For shared gateway mode, also install openflow
rules to drop needs frag packets from OVN GR that are destined to known
kubernetes nodes.

Signed-off-by: Tim Rozet <[email protected]>
(cherry picked from commit b23bcb0)
This test works under the premise that sending UDP packets via netcat
would trigger ICMP needs frag.

For shared gateway mode, packets to a host networked pod endpoint are
not fragmented and are rerouted out of GR back to the next host endpoint.
However, for OVN networked endpoints and shared gateway mode, the packet
will hit the GR and then even if DF bit is not set, OVN GR will send
back an ICMP needs frag (because OVS is incapable of fragmentation
across different MTU boundary interfaces).

For local gateway mode, large packets that hit an intermediary node are
DNAT'ed in iptables to the cluster IP service. Then there is a route for
cluster IP service set to 1400 byte MTU. This will trigger the kernel to
drop the packet when DF bit is set.

Since our new logic prevents ICMP needs frag from installing a cached
MTU route, the client will continue to send packets with a 1500 byte MTU
and they will be dropped. We choose not to fix this for now as it was
identified as not a practical use case:
https://issues.redhat.com/browse/OCPBUGS-7609

It could be fixed in the future by using an ip rule to match on nodeport
port range, and then redirecting traffic to another routing table with a
lower MTU on the route.

Signed-off-by: Tim Rozet <[email protected]>
(cherry picked from commit bc261a1)
This allows ovn_cluster_router to send ICMP needs frag when too large of
a packet is sent from a pod (such as from a kubevirt VM with too large
an MTU).

See https://issues.redhat.com/browse/FDP-1295 for more details.

Signed-off-by: Tim Rozet <[email protected]>
(cherry picked from commit 9577e9f)
Today, we require that VF accelerated gateway interface must be
explicitly configured. This change allows VF based accelerated gateway
interface without explict configuration.

Signed-off-by: Yun Zhou <[email protected]>
(cherry picked from commit 0851d4c)
Signed-off-by: nithyar <[email protected]>
(cherry picked from commit 4dcedbe)
@tssurya tssurya force-pushed the OVS-CPU-part2-saga-revert-set4 branch from 9985676 to 7796d12 Compare May 12, 2025 07:44
@openshift-merge-robot openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 12, 2025
tssurya added 2 commits May 12, 2025 09:54
This reverts commit b1525c3.

Signed-off-by: Surya Seetharaman <[email protected]>
(cherry picked from commit 936e621)
@tssurya
Copy link
Contributor Author

tssurya commented May 12, 2025

/payload 4.20 nightly blocking

Copy link
Contributor

openshift-ci bot commented May 12, 2025

@tssurya: trigger 11 job(s) of type blocking for the nightly release of OCP 4.20

  • periodic-ci-openshift-release-master-ci-4.20-e2e-aws-upgrade-ovn-single-node
  • periodic-ci-openshift-release-master-nightly-4.20-e2e-aws-ovn-upgrade-fips
  • periodic-ci-openshift-release-master-ci-4.20-e2e-azure-ovn-upgrade
  • periodic-ci-openshift-release-master-ci-4.20-upgrade-from-stable-4.19-e2e-gcp-ovn-rt-upgrade
  • periodic-ci-openshift-hypershift-release-4.20-periodics-e2e-aws-ovn-conformance
  • periodic-ci-openshift-release-master-nightly-4.20-e2e-aws-ovn-serial
  • periodic-ci-openshift-release-master-ci-4.20-e2e-aws-ovn-techpreview
  • periodic-ci-openshift-release-master-ci-4.20-e2e-aws-ovn-techpreview-serial
  • periodic-ci-openshift-release-master-nightly-4.20-fips-payload-scan
  • periodic-ci-openshift-release-master-nightly-4.20-e2e-metal-ipi-ovn-bm
  • periodic-ci-openshift-release-master-nightly-4.20-e2e-metal-ipi-ovn-ipv6

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/01a9bb70-2f08-11f0-9b12-087fc33ce2b4-0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants