Skip to content

[DX-1992] Deconsolidate Traffic Transformation #6390

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 57 commits into from
May 28, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
1e7e6f2
Merge branch 'master' of https://github.com/TykTechnologies/tyk-docs
sharadregoti May 1, 2025
a782ebd
Merge branch 'master' of https://github.com/TykTechnologies/tyk-docs
sharadregoti May 2, 2025
c295712
Merge branch 'master' of https://github.com/TykTechnologies/tyk-docs
sharadregoti May 2, 2025
007acf1
Merge branch 'master' of https://github.com/TykTechnologies/tyk-docs
sharadregoti May 7, 2025
d047927
Merge branch 'master' of https://github.com/TykTechnologies/tyk-docs
sharadregoti May 9, 2025
3cf06d4
Allow List
sharadregoti May 9, 2025
6fd83e0
Block List
sharadregoti May 9, 2025
e5bb982
Do Not Track
sharadregoti May 9, 2025
9529af6
Ignore Authentication
sharadregoti May 9, 2025
22e69f0
Internal Endpoint
sharadregoti May 9, 2025
6febb0d
Request Method
sharadregoti May 9, 2025
b209518
Request Body, Headers, Size-Limit
sharadregoti May 9, 2025
270ea85
Response Body and headers
sharadregoti May 9, 2025
2161a24
Remaining Changes
sharadregoti May 9, 2025
defdc77
Merge master into DX-1992
buger May 12, 2025
5319af4
Merge master into DX-1992
buger May 12, 2025
1864606
Merge master into DX-1992
buger May 12, 2025
a473838
Merge master into DX-1992
buger May 12, 2025
50812f4
Merge master into DX-1992
buger May 12, 2025
e587acd
Merge master into DX-1992
buger May 12, 2025
eaeff14
Merge master into DX-1992
buger May 12, 2025
47b199b
Merge master into DX-1992
buger May 13, 2025
68c1d1f
Merge master into DX-1992
buger May 13, 2025
2b256cc
Merge master into DX-1992
buger May 13, 2025
39d92b0
Merge master into DX-1992
buger May 13, 2025
7e47d2f
Merge master into DX-1992
buger May 13, 2025
da48a99
Merge master into DX-1992
buger May 13, 2025
ed624db
Merge master into DX-1992
buger May 13, 2025
19b0700
Merge master into DX-1992
buger May 14, 2025
7631cd8
Merge master into DX-1992
buger May 14, 2025
e39c40c
Merge master into DX-1992
buger May 14, 2025
fbf6b37
Merge master into DX-1992
buger May 15, 2025
faaa351
Merge master into DX-1992
buger May 16, 2025
77fd175
Merge master into DX-1992
buger May 16, 2025
6e3b6ca
Merge master into DX-1992
buger May 16, 2025
e4b57e7
Merge master into DX-1992
buger May 16, 2025
e1532ed
Merge master into DX-1992
buger May 20, 2025
a82a31d
Merge master into DX-1992
buger May 20, 2025
3141fad
Merge master into DX-1992
buger May 21, 2025
589c3d5
Merge master into DX-1992
buger May 21, 2025
e30ff32
Merge master into DX-1992
buger May 22, 2025
e56b1fd
Merge master into DX-1992
buger May 22, 2025
f861e21
Merge master into DX-1992
buger May 22, 2025
ee95126
Merge master into DX-1992
buger May 22, 2025
5f6230b
Merge master into DX-1992
buger May 22, 2025
c6340da
Merge master into DX-1992
buger May 23, 2025
b5eefad
Merge master into DX-1992
buger May 23, 2025
8844d21
Merge branch 'master' of https://github.com/TykTechnologies/tyk-docs …
sharadregoti May 26, 2025
e7dc46d
Fixes
sharadregoti May 26, 2025
ce13a5e
CI Fixes 1
sharadregoti May 26, 2025
b41af76
Merge master into DX-1992
buger May 27, 2025
ae59d8c
Merge master into DX-1992
buger May 27, 2025
83185db
CI Fixes
sharadregoti May 28, 2025
cd3fab6
Merge branch 'DX-1992' of https://github.com/TykTechnologies/tyk-docs…
sharadregoti May 28, 2025
6e4fad5
Fixes
sharadregoti May 28, 2025
443a3e8
CI Fixes
sharadregoti May 28, 2025
e410443
CI Fixes
sharadregoti May 28, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions tyk-docs/content/api-management/api-versioning.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ API sunsetting is the process of phasing out or retiring an older version of an

When sunsetting API versions, you may have endpoints that become deprecated between versions. It can be more user friendly to retain those endpoints but return a helpful error, instead of just returning `HTTP 404 Not Found`.

This is easy to do with Tyk. You could, for example, include the deprecated endpoint in the new version of the API and configure the [mock response]({{< ref "api-management/traffic-transformation#mock-response-1" >}}) middleware to provide your clients with relevant information and instruction. Alternatively, you could return a `HTTP 302 Found` header and redirect the user to the new endpoint.
This is easy to do with Tyk. You could, for example, include the deprecated endpoint in the new version of the API and configure the [mock response]({{< ref "api-management/traffic-transformation/mock-response" >}}) middleware to provide your clients with relevant information and instruction. Alternatively, you could return a `HTTP 302 Found` header and redirect the user to the new endpoint.

---

Expand Down Expand Up @@ -158,7 +158,7 @@ When using Tyk Classic APIs you can explicitly grant access to specific versions

As explained, there are differences between the way that versioning works for Tyk OAS and Tyk Classic APIs.

These are largely due to the fact that a separate API definition is generated for each version of a Tyk OAS API, with one designated as the [base]({{< ref "api-management/api-versioning#base-and-child-apis" >}}) version (which should be exposed on Tyk Gateway with the other (child) versions set to [internal]({{< ref "api-management/traffic-transformation#internal-endpoint-1" >}}) visibility) whereas all versions of a Tyk Classic API are described by a single API definition.
These are largely due to the fact that a separate API definition is generated for each version of a Tyk OAS API, with one designated as the [base]({{< ref "api-management/api-versioning#base-and-child-apis" >}}) version (which should be exposed on Tyk Gateway with the other (child) versions set to [internal]({{< ref "api-management/traffic-transformation/internal-endpoint" >}}) visibility) whereas all versions of a Tyk Classic API are described by a single API definition.

The Tyk Classic approach limits the range of features that can differ between versions.

Expand Down
6 changes: 3 additions & 3 deletions tyk-docs/content/api-management/dashboard-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -3560,9 +3560,9 @@ At the top of the Endpoint Designer, you can see which version you are currently

{{< img src="/img/dashboard/endpoint-designer/classic-endpoint-designer-endpoint.png" alt="The Tyk Classic Endpoint Designer - Endpoint Designer tab" >}}

The **Endpoint Designer** is where you can define endpoints for your API so that you can enable and configure Tyk middleware to [perform checks and transformations]({{< ref "api-management/traffic-transformation#" >}}) on the API traffic.
The **Endpoint Designer** is where you can define endpoints for your API so that you can enable and configure Tyk middleware to [perform checks and transformations]({{< ref "api-management/traffic-transformation" >}}) on the API traffic.

In some cases, you will want to set global settings that affect all paths that are managed by Tyk. The **Global Version Settings** section will enable you to configure API-level [request]({{< ref "api-management/traffic-transformation#tyk-classic-api" >}}) and [response]({{< ref "api-management/traffic-transformation#tyk-classic-api" >}}) header transformation.
In some cases, you will want to set global settings that affect all paths that are managed by Tyk. The **Global Version Settings** section will enable you to configure API-level [request]({{< ref "api-management/traffic-transformation/request-headers#tyk-classic-api" >}}) and [response]({{< ref "api-management/traffic-transformation/request-headers#tyk-classic-api" >}}) header transformation.

### Advanced Options

Expand All @@ -3573,7 +3573,7 @@ The **Advanced Options** tab is where you can configure Tyk's other powerful fea
- [API-level caching]({{< ref "api-management/response-caching#configuring-the-cache-via-the-dashboard" >}}) including a button to invalidate (flush) the cache for the API
- [CORS]({{< ref "api-management/gateway-config-tyk-classic#cross-origin-resource-sharing-cors" >}})
- Add custom attributes to the API definition as *config data* that can be accessed by middleware
- Enable [context variables]({{< ref "api-management/traffic-transformation#request-context-variables" >}}) so that they are extracted from requests and made available to middleware
- Enable [context variables]({{< ref "api-management/traffic-transformation/request-context-variables" >}}) so that they are extracted from requests and made available to middleware
- Manage *segment tags* if you are working with [sharded gateways]({{< ref "api-management/multiple-environments#gateway-sharding" >}})
- Manage client IP address [allow]({{< ref "api-management/gateway-config-tyk-classic#ip-access-control" >}}) and [block]({{< ref "api-management/gateway-config-tyk-classic#ip-access-control" >}}) lists
- Attach [webhooks]({{< ref "api-management/gateway-events#event-handling-with-webhooks" >}}) that will be triggered for different events
Expand Down
4 changes: 2 additions & 2 deletions tyk-docs/content/api-management/data-graph.md
Original file line number Diff line number Diff line change
Expand Up @@ -396,7 +396,7 @@ Global headers can be configured via Tyk API Definition. The correct place to do
}
```

Global headers now have access to all [request context variables]({{< ref "api-management/traffic-transformation#request-context-variables" >}}).
Global headers now have access to all [request context variables]({{< ref "api-management/traffic-transformation/request-context-variables" >}}).

By default, any header that is configured as a global header, will be forwarded to all data sources of the UDG.

Expand All @@ -421,7 +421,7 @@ Data source headers can be configured via Tyk API Definition and via Tyk Dashboa
}
```

Data source headers now have access to all [request context variables]({{< ref "api-management/traffic-transformation#request-context-variables" >}}).
Data source headers now have access to all [request context variables]({{< ref "api-management/traffic-transformation/request-context-variables" >}}).

#### Headers priority order

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ API definition objects can be compact for a basic pass-through API, and can beco

## API Definitions

An *API definition* is the specification for an API proxy, providing Tyk with everything it needs to receive and process requests. Using Tyk's mock response, virtual endpoint and custom plugin functionality, you don't even need an upstream service - with a single API definition you can emulate a service entirely within Tyk, providing a [mock response]({{< ref "api-management/traffic-transformation#mock-response-1" >}}).
An *API definition* is the specification for an API proxy, providing Tyk with everything it needs to receive and process requests. Using Tyk's mock response, virtual endpoint and custom plugin functionality, you don't even need an upstream service - with a single API definition you can emulate a service entirely within Tyk, providing a [mock response]({{< ref "api-management/traffic-transformation/mock-response" >}}).

Tyk supports two types of API definition depending on the type of service that you are looking to proxy:

Expand Down
24 changes: 12 additions & 12 deletions tyk-docs/content/api-management/gateway-config-managing-oas.md
Original file line number Diff line number Diff line change
Expand Up @@ -381,10 +381,10 @@ On the Import API screen, there are three options for <b>Import Type</b>, it is

| Middleware | OpenAPI data used for configuration |
|------------|-------------------------------------|
| [Request validation]({{< ref "api-management/traffic-transformation#request-schema-in-openapi-specification" >}}) | Endpoints that have `requestBody` or `schema` |
| [Mock response]({{< ref "api-management/traffic-transformation#mock-responses-using-openapi-metadata" >}}) | Endpoints with `examples` or `schema` |
| [Request validation]({{< ref "api-management/traffic-transformation/request-validation#request-schema-in-openapi-specification" >}}) | Endpoints that have `requestBody` or `schema` |
| [Mock response]({{< ref "api-management/traffic-transformation/mock-response#mock-responses-using-openapi-metadata" >}}) | Endpoints with `examples` or `schema` |
| [Client authentication]({{< ref "api-management/client-authentication#how-does-tyk-implement-authentication-and-authorization" >}}) | Defined in `security` and `securitySchemes` |
| [Allow list]({{< ref "api-management/traffic-transformation#allow-list-1" >}}) | Restrict access only to declared endpoint paths |
| [Allow list]({{< ref "api-management/traffic-transformation/allow-list" >}}) | Restrict access only to declared endpoint paths |

8. Select **Import API** to complete the import and create the API based on your API definition.

Expand Down Expand Up @@ -422,9 +422,9 @@ The optional parameters are:
| `listenPath` | Set the listen path for the API | Defaults to `/` |
| `upstreamURL` | Set the upstream (target) URL | Defaults to the first URL in the `servers` section of the [OpenAPI description]({{< ref "api-management/gateway-config-managing-oas#api-base-path" >}}) |
| `authentication` | Configure [client authentication]({{< ref "api-management/client-authentication#how-does-tyk-implement-authentication-and-authorization" >}}) based on `security` and `securitySchemes` | Client authentication is not configured |
| `allowList` | Enable [allow list]({{< ref "api-management/traffic-transformation#allow-list-1" >}}) middleware for all endpoints declared in the OpenAPI description | Allow list not configured |
| `validateRequest` | Configure [request validation]({{< ref "api-management/traffic-transformation#request-schema-in-openapi-specification" >}}) for all endpoints with `requestBody` or `schema` defined | Request validation not configured |
| `mockResponse` | Configure [mock response]({{< ref "api-management/traffic-transformation#mock-responses-using-openapi-metadata" >}}) for all endpoints with `examples` or `schema` defined | Mock response not configured |
| `allowList` | Enable [allow list]({{< ref "api-management/traffic-transformation/allow-list" >}}) middleware for all endpoints declared in the OpenAPI description | Allow list not configured |
| `validateRequest` | Configure [request validation]({{< ref "api-management/traffic-transformation/request-validation#request-schema-in-openapi-specification" >}}) for all endpoints with `requestBody` or `schema` defined | Request validation not configured |
| `mockResponse` | Configure [mock response]({{< ref "api-management/traffic-transformation/mock-response#mock-responses-using-openapi-metadata" >}}) for all endpoints with `examples` or `schema` defined | Mock response not configured |
| `apiID` | Id to be assigned to the new API | Tyk will determine and assign a unique Id |
| `templateId` | Apply the selected [API template]({{< ref "api-management/dashboard-configuration#applying-a-template-when-creating-an-api-from-a-tyk-oas-api-definition" >}}) when creating the API | No template is applied |

Expand Down Expand Up @@ -467,9 +467,9 @@ The optional parameters are:
| `listenPath` | Set the listen path for the API | Defaults to `/` |
| `upstreamURL` | Set the upstream (target) URL | Defaults to the first URL in the `servers` section of the [OpenAPI description]({{< ref "api-management/gateway-config-managing-oas#api-base-path" >}}) |
| `authentication` | Configure [client authentication]({{< ref "api-management/client-authentication#how-does-tyk-implement-authentication-and-authorization" >}}) based on `security` and `securitySchemes` | Client authentication is not configured |
| `allowList` | Enable [allow list]({{< ref "api-management/traffic-transformation#allow-list-1" >}}) middleware for all endpoints declared in the OpenAPI description | Allow list not configured |
| `validateRequest` | Configure [request validation]({{< ref "api-management/traffic-transformation#request-schema-in-openapi-specification" >}}) for all endpoints with `requestBody` or `schema` defined | Request validation not configured |
| `mockResponse` | Configure [mock response]({{< ref "api-management/traffic-transformation#mock-responses-using-openapi-metadata" >}}) for all endpoints with `examples` or `schema` defined | Mock response not configured |
| `allowList` | Enable [allow list]({{< ref "api-management/traffic-transformation/allow-list" >}}) middleware for all endpoints declared in the OpenAPI description | Allow list not configured |
| `validateRequest` | Configure [request validation]({{< ref "api-management/traffic-transformation/request-validation#request-schema-in-openapi-specification" >}}) for all endpoints with `requestBody` or `schema` defined | Request validation not configured |
| `mockResponse` | Configure [mock response]({{< ref "api-management/traffic-transformation/mock-response#mock-responses-using-openapi-metadata" >}}) for all endpoints with `examples` or `schema` defined | Mock response not configured |
| `apiID` | Id to be assigned to the new API | Tyk will determine and assign a unique Id |

**Check request response**
Expand Down Expand Up @@ -637,10 +637,10 @@ If you have an updated OpenAPI description or Tyk OAS API definition, in YAML or

| Middleware | OpenAPI data used for configuration |
|------------|-------------------------------------|
| [Request validation]({{< ref "api-management/traffic-transformation#request-schema-in-openapi-specification" >}}) | Endpoints that have `requestBody` or `schema` |
| [Mock response]({{< ref "api-management/traffic-transformation#mock-responses-using-openapi-metadata" >}}) | Endpoints with `examples` or `schema` |
| [Request validation]({{< ref "api-management/traffic-transformation/request-validation#request-schema-in-openapi-specification" >}}) | Endpoints that have `requestBody` or `schema` |
| [Mock response]({{< ref "api-management/traffic-transformation/mock-response#mock-responses-using-openapi-metadata" >}}) | Endpoints with `examples` or `schema` |
| [Client authentication]({{< ref "api-management/client-authentication#how-does-tyk-implement-authentication-and-authorization" >}}) | Defined in `security` and `securitySchemes` |
| [Allow list]({{< ref "api-management/traffic-transformation#allow-list-1" >}}) | Restrict access only to declared endpoint paths |
| [Allow list]({{< ref "api-management/traffic-transformation/allow-list" >}}) | Restrict access only to declared endpoint paths |

8. Select **Import API** to complete the update.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -353,7 +353,7 @@ If set to `true` when matching the URL path for requests to this API, the case o
* `/my-api/getUser`
* `/my-api/GetUser`

If set to true, this will override the endpoint level settings in [Ignore]({{< ref "api-management/traffic-transformation#case-sensitivity-2" >}}), [Allowlist]({{< ref "api-management/traffic-transformation#case-sensitivity" >}}) and [Blocklist]({{< ref "api-management/traffic-transformation#case-sensitivity-1" >}}) middleware. This setting can be overriden at the Tyk Gateway level, and so applied to all APIs, by setting `ignore_endpoint_case` to `true` in your `tyk.conf` file. See [ignore_endpoint_case]({{< ref "tyk-oss-gateway/configuration#ignore_endpoint_case" >}}) for details.
If set to true, this will override the endpoint level settings in [Ignore]({{< ref "api-management/traffic-transformation/ignore-authentication#case-sensitivity" >}}), [Allowlist]({{< ref "api-management/traffic-transformation/allow-list#case-sensitivity" >}}) and [Blocklist]({{< ref "api-management/traffic-transformation/block-list#case-sensitivity" >}}) middleware. This setting can be overriden at the Tyk Gateway level, and so applied to all APIs, by setting `ignore_endpoint_case` to `true` in your `tyk.conf` file. See [ignore_endpoint_case]({{< ref "tyk-oss-gateway/configuration#ignore_endpoint_case" >}}) for details.

**Field: `enable_batch_request_support`**
Set to true to enable batch support
Expand Down
2 changes: 1 addition & 1 deletion tyk-docs/content/api-management/gateway-events.md
Original file line number Diff line number Diff line change
Expand Up @@ -228,7 +228,7 @@ It is very likely that an `AuthFailure` event will fire on the same endpoint mor

##### Webhook payload

When your webhook event handler is triggered, it will send an HTTP request to the configured target. For HTTP methods that support a request body, for example `POST`, the event handler will process a [Go template]({{< ref "api-management/traffic-transformation#go-templates" >}}) to produce the payload.
When your webhook event handler is triggered, it will send an HTTP request to the configured target. For HTTP methods that support a request body, for example `POST`, the event handler will process a [Go template]({{< ref "api-management/traffic-transformation/go-templates" >}}) to produce the payload.

If no template is provided in the webhook event handler configuration in the API definition, Tyk Gateway will look for the default file `templates/default_webhook.json`. Any text file accessible to the Gateway can be used to store the Go template to be used by the event handler when constructing the payload.

Expand Down
4 changes: 2 additions & 2 deletions tyk-docs/content/api-management/graphql.md
Original file line number Diff line number Diff line change
Expand Up @@ -958,7 +958,7 @@ You can enrich any GraphQL request proxied through Tyk Gateway with additional i

{{< img src="/img/dashboard/graphql/headers-gql-request.png" alt="Request headers" >}}

**Request headers** values can be defined as context variables. To know how to refer to request context variables check [this page]({{< ref "api-management/traffic-transformation#request-context-variables" >}}).
**Request headers** values can be defined as context variables. To know how to refer to request context variables check [this page]({{< ref "api-management/traffic-transformation/request-context-variables" >}}).

Any header key/value pair defined in **Request headers** will only be used to inject headers into requests proxied through the Gateway. It will not be used to introspect the upstream schema from Tyk Dashboard.

Expand Down Expand Up @@ -1110,7 +1110,7 @@ If you run a request to your proxy, you should get a response similar to this:

#### Dynamic variables

We have seen support for passing static variable values via the API definition, but there will be cases where we want to extract variables from the request header or URL. More information about available request context variables in Tyk can be found [here]({{< ref "api-management/traffic-transformation#request-context-variables" >}})
We have seen support for passing static variable values via the API definition, but there will be cases where we want to extract variables from the request header or URL. More information about available request context variables in Tyk can be found [here]({{< ref "api-management/traffic-transformation/request-context-variables" >}})

Below is an examples of using an incoming `code` header value as a variable in `persist_graphql` middleware configuration:

Expand Down
2 changes: 1 addition & 1 deletion tyk-docs/content/api-management/logs-metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -233,7 +233,7 @@ The transaction records generated by the Gateway are stored in Redis, from which

The Gateway will not, by default, include the request and response payloads in the transaction records. This minimizes the size of the records and also avoids logging any sensitive content. The [detailed recording]({{< ref "api-management/logs-metrics#capturing-detailed-logs" >}}) option is provided if you need to capture the payloads in the records.

You can suppress the generation of transaction records for any endpoint by enabling the [do-not-track middleware]({{< ref "api-management/traffic-transformation#do-not-track-overview" >}}) for that endpoint. This provides granular control over request tracking.
You can suppress the generation of transaction records for any endpoint by enabling the [do-not-track middleware]({{< ref "api-management/traffic-transformation/do-not-track" >}}) for that endpoint. This provides granular control over request tracking.

You can find details of all the options available to you when configuring analytics in the Gateway in the [reference documentation]({{< ref "tyk-oss-gateway/configuration#analytics_config" >}}).

Expand Down
Loading
Loading