Skip to content

feat(conformance): Add HTTPRouteMultipleGatewaysDifferentPools test #838

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 105 commits into from
Jun 23, 2025
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
105 commits
Select commit Hold shift + click to select a range
87d1756
copy of accepted inference pool test to start from.
SinaChavoshi May 14, 2025
a9dcb62
add yaml file for the test
SinaChavoshi May 14, 2025
c36e5f8
update time out
SinaChavoshi May 14, 2025
aaf23e2
update the yaml file to add port 9002
SinaChavoshi May 14, 2025
2a67e7d
read timeout config from local repo
SinaChavoshi May 14, 2025
8ac7d72
remove excess comments
SinaChavoshi May 14, 2025
c859896
copy from existing test to start with
SinaChavoshi May 14, 2025
fe540a7
update test and add gateway to main manifest
SinaChavoshi May 15, 2025
63fedab
fix spelling error
SinaChavoshi May 19, 2025
66f325d
correct spelling for scenarios
SinaChavoshi May 20, 2025
fd0fa10
check route condition on RouteConditionResolvedRefs
SinaChavoshi May 20, 2025
68812d7
check RouteConditionResolvedRefs instead of Reconciled
SinaChavoshi May 20, 2025
59b1f9c
Merge branch 'main' into inf_pool_tests_11
SinaChavoshi May 27, 2025
8a078dc
remove empty lines in yaml
SinaChavoshi May 27, 2025
c55a803
set optional/defaulted fields as unspecified
SinaChavoshi May 27, 2025
5efaeb2
add new line at the end of yaml files.
SinaChavoshi May 27, 2025
2cd8dab
delete blank lines in yaml files.
SinaChavoshi May 27, 2025
26704a1
change variable names to use primary / secondary
SinaChavoshi May 27, 2025
bb62da7
add to do for name change across the project.
SinaChavoshi May 27, 2025
796b584
fix comment
SinaChavoshi May 27, 2025
403b588
Merge branch 'main' into inf_pool_tests_12
SinaChavoshi May 27, 2025
6506d44
change all variable names to use primary/secondary consistantly
SinaChavoshi May 27, 2025
dafd21e
add to do for http route testing
SinaChavoshi May 27, 2025
ecf11e4
HTTPRouteMustBeAcceptedAndResolved
SinaChavoshi May 27, 2025
f1ce162
add helper method.
SinaChavoshi May 27, 2025
f3d8e8c
using helper method for pool route accepted.
SinaChavoshi May 28, 2025
1cd8790
combline pool and route accpetnace
SinaChavoshi May 28, 2025
44b1427
add http route validation
SinaChavoshi May 28, 2025
c6bacb4
Merge branch 'main' into inf_pool_tests_11
SinaChavoshi May 28, 2025
d15c25e
Merge branch 'main' into inf_pool_tests_12
SinaChavoshi May 28, 2025
1451a90
remove extra comments and yaml fields.
SinaChavoshi May 28, 2025
271b7a8
Merge branch 'main' into inf_pool_tests_11
SinaChavoshi May 28, 2025
adcc104
Merge branch 'main' into inf_pool_tests_12
SinaChavoshi May 28, 2025
1535ccc
remove duplicate method
SinaChavoshi May 28, 2025
c92a402
fix timeout
SinaChavoshi May 28, 2025
ff25051
fix timeout
SinaChavoshi May 28, 2025
5a61a70
change varialbe names to use primary secondary consistently.
SinaChavoshi May 28, 2025
6fa352d
remove extra comments
SinaChavoshi May 28, 2025
5f1bcad
factor out common code
SinaChavoshi May 28, 2025
e5fc860
Add actual http traffic validation using echo-basic
SinaChavoshi May 28, 2025
62791b5
remove extra comments from manifest
SinaChavoshi May 28, 2025
2f08131
remove modifiedTimeoutConfig.HTTPRouteMustHaveCondition per review co…
SinaChavoshi May 28, 2025
cf587a6
remove modifiedTimeoutConfig.HTTPRouteMustHaveCondition per review co…
SinaChavoshi May 28, 2025
afc8bc3
remove excessive logging from conformance.go
SinaChavoshi May 28, 2025
8588183
Update conformance/tests/basic/httproute_multiple_gateways_different_…
SinaChavoshi May 28, 2025
8b6a873
Update conformance/tests/basic/httproute_multiple_gateways_different_…
SinaChavoshi May 28, 2025
efe3990
Update conformance/tests/basic/httproute_multiple_gateways_different_…
SinaChavoshi May 28, 2025
b63459d
Update conformance/tests/basic/httproute_multiple_gateways_different_…
SinaChavoshi May 28, 2025
b903d8f
Update conformance/tests/basic/httproute_multiple_gateways_different_…
SinaChavoshi May 28, 2025
2d52812
Remove unecessary logs for accpeted reason.
SinaChavoshi May 28, 2025
fc2a410
Merge remote-tracking branch 'upstream/main'
SinaChavoshi May 29, 2025
939ae57
Remove all comments from yaml and go files and fix yaml
SinaChavoshi May 29, 2025
d34457a
Merge remote-tracking branch 'upstream/main'
SinaChavoshi Jun 5, 2025
753917a
Merge remote-tracking branch 'upstream/main'
SinaChavoshi Jun 6, 2025
f4bcc9a
Merge branch 'main' into inf_pool_tests_11
SinaChavoshi Jun 6, 2025
d5530dc
Merge remote-tracking branch 'upstream/main'
SinaChavoshi Jun 6, 2025
0ce459c
Merge branch 'main' into inf_pool_tests_11
SinaChavoshi Jun 6, 2025
3b884d6
intermediate update
SinaChavoshi Jun 6, 2025
d90eae0
fix the test run
SinaChavoshi Jun 6, 2025
73731e3
factor out common code
SinaChavoshi Jun 6, 2025
b57fb8a
Merge remote-tracking branch 'upstream/main'
SinaChavoshi Jun 9, 2025
6ee73d5
Merge branch 'main' into inf_pool_tests_11
SinaChavoshi Jun 9, 2025
b239930
move epp def to shared manifest
SinaChavoshi Jun 9, 2025
66b8426
remove extra comments
SinaChavoshi Jun 9, 2025
9432280
revert back to two epps
SinaChavoshi Jun 9, 2025
4b1f44a
Merge branch 'main' into inf_pool_tests_12
SinaChavoshi Jun 9, 2025
450a5c5
Merge branch 'inf_pool_tests_11' into inf_pool_tests_12
SinaChavoshi Jun 9, 2025
5990c51
update to use epp filter
SinaChavoshi Jun 9, 2025
392171d
remove extra comments
SinaChavoshi Jun 9, 2025
117652e
Merge remote-tracking branch 'upstream/main'
SinaChavoshi Jun 13, 2025
ad82e88
Merge remote-tracking branch 'upstream/main' into inf_pool_tests_11
SinaChavoshi Jun 13, 2025
ab4e4bc
Merge remote-tracking branch 'upstream/main' into inf_pool_tests_12
SinaChavoshi Jun 13, 2025
04f52f3
add to do for epp image
SinaChavoshi Jun 16, 2025
d6b588e
switch to GeneralMustHaveConditionTimeout
SinaChavoshi Jun 16, 2025
112d878
undo gateway version changes
SinaChavoshi Jun 16, 2025
120feb3
remove unused HTTPRouteMustHaveConditions
SinaChavoshi Jun 16, 2025
8214cdf
update doc string for GetPod
SinaChavoshi Jun 16, 2025
76cb644
update docstring
SinaChavoshi Jun 16, 2025
f50054f
Merge branch 'inf_pool_tests_11' into inf_pool_tests_12
SinaChavoshi Jun 16, 2025
b235dbd
fix defaults in timing.go
SinaChavoshi Jun 16, 2025
5dca67b
Remove resource type from names in manifests.
SinaChavoshi Jun 16, 2025
f2c0602
remove type from name
SinaChavoshi Jun 17, 2025
daeb8e6
remove health check
SinaChavoshi Jun 17, 2025
79b3015
Merge remote-tracking branch 'upstream/main'
SinaChavoshi Jun 17, 2025
f1ab79f
Merge remote-tracking branch 'upstream/main' into inf_pool_tests_11
SinaChavoshi Jun 17, 2025
81562a2
add todo for combining getpod methods
SinaChavoshi Jun 17, 2025
871d4d6
Merge branch 'inf_pool_tests_11' into inf_pool_tests_12
SinaChavoshi Jun 17, 2025
ae33369
Merge remote-tracking branch 'upstream'
SinaChavoshi Jun 17, 2025
43c1fa1
Merge branch 'main' into inf_pool_tests_12
SinaChavoshi Jun 17, 2025
182ee70
remove duplicate methods
SinaChavoshi Jun 18, 2025
2afea17
Merge remote-tracking branch 'upstream/main' into inf_pool_tests_12
SinaChavoshi Jun 20, 2025
948956c
revert changes to conformance.go
SinaChavoshi Jun 20, 2025
eae8202
revert chagnes to conformance.go
SinaChavoshi Jun 20, 2025
e88bf77
revert changes to helpers.go
SinaChavoshi Jun 20, 2025
89869d7
use epp and pool from shared manifest
SinaChavoshi Jun 20, 2025
be03841
add todo to factor out common constants
SinaChavoshi Jun 20, 2025
7a9133c
Merge remote-tracking branch 'upstream' into inf_pool_tests_12
SinaChavoshi Jun 23, 2025
810e055
remove port number for EPP call
SinaChavoshi Jun 23, 2025
17ab77b
remove port number for inf pool ref
SinaChavoshi Jun 23, 2025
fbce2d5
use GetPodsWithLabel
SinaChavoshi Jun 23, 2025
af6b681
removed unused method
SinaChavoshi Jun 23, 2025
22bf7f9
fix routing issue
SinaChavoshi Jun 23, 2025
d491509
switch to using MakeRequestAndExpectEventuallyConsistentResponse
SinaChavoshi Jun 23, 2025
413ec25
undo changes to GatewayFollowingEPPRouting
SinaChavoshi Jun 23, 2025
0b94f9f
revert changes to traffic.go
SinaChavoshi Jun 23, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 10 additions & 3 deletions conformance/conformance.go
Original file line number Diff line number Diff line change
Expand Up @@ -68,8 +68,9 @@ import (

// Constants for the shared Gateway
const (
SharedGatewayName = "conformance-gateway" // Name of the Gateway in manifests.yaml
SharedGatewayNamespace = "gateway-conformance-infra" // Namespace of the Gateway
SharedGatewayName = "conformance-gateway" // Name of the primary Gateway in manifests.yaml
SharedGatewayNamespace = "gateway-conformance-infra" // Namespace of the shared Gateways
SecondaryGatewayName = "conformance-secondary-gateway" // Name of the secondary Gateway in manifests.yaml
)

// GatewayLayerProfileName defines the name for the conformance profile that tests
Expand Down Expand Up @@ -214,9 +215,15 @@ func RunConformanceWithOptions(t *testing.T, opts confsuite.ConformanceOptions)
cSuite.Setup(t, tests.ConformanceTests)

sharedGwNN := types.NamespacedName{Name: SharedGatewayName, Namespace: SharedGatewayNamespace}
secondaryGwNN := types.NamespacedName{Name: SecondaryGatewayName, Namespace: SharedGatewayNamespace}

// Validate Gateway setup.
// Validate Gateway setup for both Gateways.
t.Logf("Validating primary Gateway setup: %s/%s", sharedGwNN.Namespace, sharedGwNN.Name)
ensureGatewayAvailableAndReady(t, cSuite.Client, opts, sharedGwNN)

t.Logf("Validating secondary Gateway setup: %s/%s", secondaryGwNN.Namespace, secondaryGwNN.Name)
ensureGatewayAvailableAndReady(t, cSuite.Client, opts, secondaryGwNN)

t.Log("Running Inference Extension conformance tests against all registered tests")
err = cSuite.Run(t, tests.ConformanceTests)
require.NoError(t, err, "error running conformance tests")
Expand Down
26 changes: 24 additions & 2 deletions conformance/resources/manifests/manifests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ metadata:
gateway-conformance: backend

---
# Namespace for simple web server backends. This is expected by
# Namespace for simple web server backends. This is expected by
# the upstream conformance suite's Setup method.
apiVersion: v1
kind: Namespace
Expand All @@ -50,8 +50,30 @@ spec:
protocol: HTTP
allowedRoutes:
namespaces:
from: All
from: All
kinds:
# Allows HTTPRoutes to attach, which can then reference InferencePools.
- group: gateway.networking.k8s.io
kind: HTTPRoute

---
# --- Conformance Secondary Gateway Definition ---
# A second generic Gateway resource for tests requiring multiple Gateways.
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: conformance-secondary-gateway
namespace: gateway-conformance-infra
spec:
gatewayClassName: "{GATEWAY_CLASS_NAME}"
listeners:
- name: http
port: 80
protocol: HTTP
hostname: "secondary.example.com" # Distinct hostname to differentiate from conformance-gateway
allowedRoutes:
namespaces:
from: All
kinds:
- group: gateway.networking.k8s.io
kind: HTTPRoute
119 changes: 119 additions & 0 deletions conformance/tests/basic/httproute_multiple_gateways_different_pools.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
/*
Copyright 2025 The Kubernetes Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package basic

import (
"testing"

metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/types"
gatewayv1 "sigs.k8s.io/gateway-api/apis/v1"
"sigs.k8s.io/gateway-api/conformance/utils/suite"

// Import the tests package to append to ConformanceTests
"sigs.k8s.io/gateway-api-inference-extension/conformance/tests"
"sigs.k8s.io/gateway-api-inference-extension/conformance/utils/config"
infrakubernetes "sigs.k8s.io/gateway-api-inference-extension/conformance/utils/kubernetes"
gatewaykubernetes "sigs.k8s.io/gateway-api/conformance/utils/kubernetes"
)

func init() {
tests.ConformanceTests = append(tests.ConformanceTests, HTTPRouteMultipleGatewaysDifferentPools)
}

var HTTPRouteMultipleGatewaysDifferentPools = suite.ConformanceTest{
ShortName: "HTTPRouteMultipleGatewaysDifferentPools",
Description: "Validates two HTTPRoutes on different Gateways successfully referencing different InferencePools.",
Manifests: []string{"tests/basic/httproute_multiple_gateways_different_pools.yaml"},
Test: func(t *testing.T, s *suite.ConformanceTestSuite) {
const (
appBackendNamespace = "gateway-conformance-app-backend"
infraNamespace = "gateway-conformance-infra"
gateway1Name = "conformance-gateway"
gateway2Name = "conformance-secondary-gateway"
routeForGW1Name = "route-for-gw1"
routeForGW2Name = "route-for-gw2"
poolAName = "pool-a"
poolBName = "pool-b"
)

routeForGW1NN := types.NamespacedName{Name: routeForGW1Name, Namespace: appBackendNamespace}
routeForGW2NN := types.NamespacedName{Name: routeForGW2Name, Namespace: appBackendNamespace}
poolANN := types.NamespacedName{Name: poolAName, Namespace: appBackendNamespace}
poolBNN := types.NamespacedName{Name: poolBName, Namespace: appBackendNamespace}
gateway1NN := types.NamespacedName{Name: gateway1Name, Namespace: infraNamespace}
gateway2NN := types.NamespacedName{Name: gateway2Name, Namespace: infraNamespace}

var timeoutConfig config.InferenceExtensionTimeoutConfig = config.DefaultInferenceExtensionTimeoutConfig()

t.Run("HTTPRoute for Gateway 1 should be Accepted and Reconciled", func(t *testing.T) {
acceptedCondition := metav1.Condition{
Type: string(gatewayv1.RouteConditionAccepted),
Status: metav1.ConditionTrue,
Reason: string(gatewayv1.RouteReasonAccepted),
}
gatewaykubernetes.HTTPRouteMustHaveCondition(t, s.Client, timeoutConfig.TimeoutConfig, routeForGW1NN, gateway1NN, acceptedCondition)
t.Logf("HTTPRoute %s is Accepted by Gateway %s", routeForGW1NN.String(), gateway1NN.String())

reconciledCondition := metav1.Condition{
Type: string(gatewayv1.RouteConditionType("Reconciled")),
Status: metav1.ConditionTrue,
Reason: "ReconciliationSucceeded",
}
gatewaykubernetes.HTTPRouteMustHaveCondition(t, s.Client, timeoutConfig.TimeoutConfig, routeForGW1NN, gateway1NN, reconciledCondition)
t.Logf("HTTPRoute %s is Reconciled by Gateway %s", routeForGW1NN.String(), gateway1NN.String())
})

t.Run("InferencePool A (pool-a) should be Accepted", func(t *testing.T) {
acceptedCondition := metav1.Condition{
Type: string(gatewayv1.RouteConditionAccepted),
Status: metav1.ConditionTrue,
Reason: string(gatewayv1.RouteReasonAccepted),
}
infrakubernetes.InferencePoolMustHaveCondition(t, s.Client, poolANN, acceptedCondition)
t.Logf("InferencePool %s parent status shows Accepted by Gateway %s (via HTTPRoute %s)", poolANN.String(), gateway1NN.String(), routeForGW1NN.String())
})

t.Run("HTTPRoute for Gateway 2 should be Accepted and Reconciled", func(t *testing.T) {
acceptedCondition := metav1.Condition{
Type: string(gatewayv1.RouteConditionAccepted),
Status: metav1.ConditionTrue,
Reason: string(gatewayv1.RouteReasonAccepted),
}
gatewaykubernetes.HTTPRouteMustHaveCondition(t, s.Client, timeoutConfig.TimeoutConfig, routeForGW2NN, gateway2NN, acceptedCondition)
t.Logf("HTTPRoute %s is Accepted by Gateway %s", routeForGW2NN.String(), gateway2NN.String())

reconciledCondition := metav1.Condition{
Type: string(gatewayv1.RouteConditionType("Reconciled")),
Status: metav1.ConditionTrue,
Reason: "ReconciliationSucceeded",
}
gatewaykubernetes.HTTPRouteMustHaveCondition(t, s.Client, timeoutConfig.TimeoutConfig, routeForGW2NN, gateway2NN, reconciledCondition)
t.Logf("HTTPRoute %s is Reconciled by Gateway %s", routeForGW2NN.String(), gateway2NN.String())
})

t.Run("InferencePool B (pool-b) should be Accepted", func(t *testing.T) {
acceptedCondition := metav1.Condition{
Type: string(gatewayv1.RouteConditionAccepted),
Status: metav1.ConditionTrue,
Reason: string(gatewayv1.RouteReasonAccepted),
}
infrakubernetes.InferencePoolMustHaveCondition(t, s.Client, poolBNN, acceptedCondition)
t.Logf("InferencePool %s parent status shows Accepted by Gateway %s (via HTTPRoute %s)", poolBNN.String(), gateway2NN.String(), routeForGW2NN.String())
})
},
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,199 @@
# httproute_multiple_gateways_different_pools.yaml
# This manifest defines resources for the httproute_multiple_gateways_different_pools conformance test.
# It includes two distinct backend deployments, their EPP services, two InferencePools,
# and two HTTPRoutes, each linking to one of the two shared Gateways from the base manifests.

# --- Backend Deployment A ---
apiVersion: apps/v1
kind: Deployment
metadata:
name: multi-gw-backend-a-deployment
namespace: gateway-conformance-app-backend
labels:
app: multi-gw-backend-a
spec:
replicas: 1
selector:
matchLabels:
app: multi-gw-backend-a
template:
metadata:
labels:
app: multi-gw-backend-a
spec:
containers:
- name: agnhost-echo-a
image: k8s.gcr.io/e2e-test-images/agnhost:2.39
args:
- serve-hostname
- --port=8080
ports:
- name: http
containerPort: 8080
readinessProbe:
httpGet:
path: /
port: 8080
initialDelaySeconds: 3
periodSeconds: 5
failureThreshold: 2

---
# --- Service for Backend Deployment A (EPP for pool-a) ---
apiVersion: v1
kind: Service
metadata:
name: pool-a-epp
namespace: gateway-conformance-app-backend
spec:
selector:
app: multi-gw-backend-a
ports:
- name: http
protocol: TCP
port: 8080
targetPort: 8080
- name: epp
protocol: TCP
port: 9002
targetPort: 9002

---
# --- Backend Deployment B ---
apiVersion: apps/v1
kind: Deployment
metadata:
name: multi-gw-backend-b-deployment
namespace: gateway-conformance-app-backend
labels:
app: multi-gw-backend-b
spec:
replicas: 1
selector:
matchLabels:
app: multi-gw-backend-b
template:
metadata:
labels:
app: multi-gw-backend-b
spec:
containers:
- name: agnhost-echo-b
image: k8s.gcr.io/e2e-test-images/agnhost:2.39
args:
- serve-hostname
- --port=8080
ports:
- name: http
containerPort: 8080
readinessProbe:
httpGet:
path: /
port: 8080
initialDelaySeconds: 3
periodSeconds: 5
failureThreshold: 2

---
# --- Service for Backend Deployment B (EPP for pool-b) ---
apiVersion: v1
kind: Service
metadata:
name: pool-b-epp
namespace: gateway-conformance-app-backend
spec:
selector:
app: multi-gw-backend-b
ports:
- name: http
protocol: TCP
port: 8080
targetPort: 8080
- name: epp
protocol: TCP
port: 9002
targetPort: 9002

---
# --- InferencePool A Definition ---
apiVersion: inference.networking.x-k8s.io/v1alpha2
kind: InferencePool
metadata:
name: pool-a
namespace: gateway-conformance-app-backend
spec:
selector:
app: "multi-gw-backend-a"
targetPortNumber: 8080
extensionRef:
name: pool-a-epp

---
# --- InferencePool B Definition ---
apiVersion: inference.networking.x-k8s.io/v1alpha2
kind: InferencePool
metadata:
name: pool-b
namespace: gateway-conformance-app-backend
spec:
selector:
app: "multi-gw-backend-b"
targetPortNumber: 8080
extensionRef:
name: pool-b-epp

---
# --- HTTPRoute for Gateway 1 (conformance-gateway) ---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: route-for-gw1
namespace: gateway-conformance-app-backend
spec:
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: conformance-gateway # Referencing shared conformance-gateway
namespace: gateway-conformance-infra # Gateway is in infra namespace
sectionName: http
hostnames:
- "gw1.example.com" # Specific hostname for this route
rules:
- backendRefs:
- group: inference.networking.x-k8s.io
kind: InferencePool
name: pool-a
port: 8080
weight: 1
matches:
- path:
type: PathPrefix
value: /test-gw1

---
# --- HTTPRoute for Gateway 2 (conformance-secondary-gateway) ---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: route-for-gw2
namespace: gateway-conformance-app-backend
spec:
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: conformance-secondary-gateway # Referencing shared conformance-secondary-gateway
namespace: gateway-conformance-infra # Gateway is in infra namespace
sectionName: http
hostnames:
- "secondary.example.com" # Matching hostname of conformance-secondary-gateway
rules:
- backendRefs:
- group: inference.networking.x-k8s.io
kind: InferencePool
name: pool-b
port: 8080
weight: 1
matches:
- path:
type: PathPrefix
value: /test-gw2
Loading