feat(ai-proxy): add modelMapping regexp support #2358

daixijun · 2025-05-31T17:01:27Z

Ⅰ. Describe what this PR did

openrouter.ai 中模型调用是需要指定 Provider 前缀的，如 openai/gpt-4o-mini
我们有场景需要在 openai 不可用时，fallback 到 openrouter.ai，这时需要保持调用模型名不变，但需要加上 openai/前缀
所以希望是通过正则来做替换，而不是单纯的前缀匹配

比如 gpt(.*) 需要替换为 openai/gptXXX

Ⅱ. Does this pull request fix one issue?

Ⅲ. Why don't you add test cases (unit test/integration test)?

Ⅳ. Describe how to verify it

docker-compose.yaml

services:
  envoy:
    image: higress-registry.cn-hangzhou.cr.aliyuncs.com/higress/gateway:v2.1.3
    entrypoint: /usr/local/bin/envoy
    command: -c /etc/envoy/envoy.yaml --component-log-level wasm:debug
    networks:
      - wasmtest
    ports:
      - '10000:10000'
    volumes:
      - ./envoy.yaml:/etc/envoy/envoy.yaml
      - ./plugin.wasm:/etc/envoy/plugin.wasm

networks:
  wasmtest: {}

envoy.yaml

# File generated by hgctl. Modify as required.

admin:
  address:
    socket_address:
      protocol: TCP
      address: 0.0.0.0
      port_value: 9901
static_resources:
  listeners:
    - name: listener_0
      address:
        socket_address:
          protocol: TCP
          address: 0.0.0.0
          port_value: 10000
      filter_chains:
        - filters:
            - name: envoy.filters.network.http_connection_manager
              typed_config:
                "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                scheme_header_transformation:
                  scheme_to_overwrite: https
                stat_prefix: ingress_http
                # Output envoy logs to stdout
                access_log:
                  - name: envoy.access_loggers.stdout
                    typed_config:
                      "@type": type.googleapis.com/envoy.extensions.access_loggers.stream.v3.StdoutAccessLog
                # Modify as required
                route_config:
                  name: local_route
                  virtual_hosts:
                    - name: local_service
                      domains: ["*"]
                      routes:
                        - match:
                            prefix: "/"
                          route:
                            cluster: openrouter
                            timeout: 300s
                http_filters:
                  - name: wasmtest
                    typed_config:
                      "@type": type.googleapis.com/udpa.type.v1.TypedStruct
                      type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
                      value:
                        config:
                          name: wasmtest
                          vm_config:
                            runtime: envoy.wasm.runtime.v8
                            code:
                              local:
                                filename: /etc/envoy/plugin.wasm
                          configuration:
                            "@type": "type.googleapis.com/google.protobuf.StringValue"
                            value: |
                              {
                                "provider": {
                                  "type": "openai",
                                  "openaiCustomUrl": "openrouter.ai/api/v1/",
                                  "modelMapping": {
                                    "qwen3-235b-a22b": "qwen/qwen3-235b-a22b",
                                    "deepseek-v3*":  "deepseek/deepseek-chat-v3-0324",
                                    "~gpt(.*)": "openai/gpt$1",
                                    "*": "openai/gpt-4.1-mini"
                                  },
                                  "apiTokens": [
                                      "sk-xxx"
                                  ]
                                }
                              }
                  - name: envoy.filters.http.router
                    typed_config:
                      "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
  clusters:
    - name: openrouter
      connect_timeout: 30s
      type: LOGICAL_DNS
      dns_lookup_family: V4_ONLY
      lb_policy: ROUND_ROBIN
      load_assignment:
        cluster_name: google
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: openrouter.ai
                      port_value: 443
      transport_socket:
        name: envoy.transport_sockets.tls
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
          "sni": "openrouter.ai"

$ curl -s http://127.0.0.1:10000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
      "messages": [
        {
          "role": "user",
          "content": "今天是2025年05月31日。用json输出最近三天的日期，包含字段： today, next_day, day_after_tomorrow."
        }
      ],
      "model": "gpt-4o-mini"
    }' 
{
  "id": "gen-1748710730-bV1xwfObpNhNgR5SG037",
  "provider": "OpenAI",
  "model": "openai/gpt-4o-mini",
  "object": "chat.completion",
  "created": 1748710730,
  "choices": [
    {
      "logprobs": null,
      "finish_reason": "stop",
      "native_finish_reason": "stop",
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "```json\n{\n    \"today\": \"2025-05-31\",\n    \"next_day\": \"2025-06-01\",\n    \"day_after_tomorrow\": \"2025-06-02\"\n}\n```",
        "refusal": null,
        "reasoning": null
      }
    }
  ],
  "system_fingerprint": "fp_62a23a81ef",
  "usage": {
    "prompt_tokens": 40,
    "completion_tokens": 47,
    "total_tokens": 87,
    "prompt_tokens_details": {
      "cached_tokens": 0
    },
    "completion_tokens_details": {
      "reasoning_tokens": 0
    }
  }
}


日志输出

![image](https://github.com/user-attachments/assets/4edd9780-907d-4081-bc07-dff26774bbc3)


### Ⅴ. Special notes for reviews

Signed-off-by: Xijun Dai <[email protected]>

lingma-agents · 2025-05-31T17:02:08Z

feat(ai-proxy): 新增基于正则表达式的模型映射功能

变更文件

文件路径	变更说明
plugins/wasm-go/extensions/ai-proxy/provider/provider.go	新增正则表达式匹配模型映射逻辑，支持通过正则表达式动态替换模型名称
envoy.yaml	配置示例中添加modelMapping正则表达式映射规则，包含通配符和具体模式匹配

时序图

sequenceDiagram
    participant Client as HTTP客户端
    participant Envoy as Envoy代理
    participant Provider as 模型服务
    Client->>Envoy: 发送/v1/chat/completions请求
    Envoy->>Envoy: wasm插件处理模型名称
    Envoy->>Envoy: 正则匹配模型映射规则(gpt.* → openai/gpt$1)
    Envoy->>Provider: 转发请求到openrouter.ai服务
    Provider-->>Envoy: 返回处理结果
    Envoy-->>Client: 响应结果到客户端

💡 小贴士

与 lingma-agents 交流的方式

📜 直接回复评论
直接回复本条评论，lingma-agents 将自动处理您的请求。例如：

在当前代码中添加详细的注释说明。
请详细介绍一下你说的 LRU 改造方案，并使用伪代码加以说明。

📜 在代码行处标记
在文件的特定位置创建评论并 @lingma-agents。例如：

@Lingma-Agent 分析这个方法的性能瓶颈并提供优化建议。
@Lingma-Agent 对这个方法生成优化代码。

📜 在讨论中提问
在任何讨论中 @lingma-agents 来获取帮助。例如：

@Lingma-Agent 请总结上述讨论并提出解决方案。
@Lingma-Agent 请根据讨论内容生成优化代码。

lingma-agents

🔍 代码评审报告

📋 评审意见详情

💡 单文件建议

✅ 未发现需要特别关注的代码问题。

🚀 跨文件建议

以下是对代码架构和设计的综合分析，聚焦于跨文件交互、系统一致性和潜在优化空间。

🔍 1. 正则表达式在循环中重复编译导致性能问题

在循环中每次调用regexp.MustCompile(k)会重复编译正则表达式，导致不必要的CPU和内存开销。建议将所有模式预编译为regexp.Regexp对象并缓存，避免重复编译。

📌 关键代码：

plugins/wasm-go/extensions/ai-proxy/provider/provider.go (617-619)

if strings.Contains(k, "(") && strings.Contains(k, ")") {
    re := regexp.MustCompile(k)
    ...

⚠️ 潜在风险： 高频模型映射请求时会导致资源浪费，可能引发性能瓶颈或延迟，尤其在大规模模型映射场景下。

🔍 2. 匹配策略未模块化影响可扩展性

当前将前缀匹配和正则表达式匹配逻辑混合在同一个循环中，违反开闭原则。建议将匹配策略抽象为独立接口（如Strategy模式），便于未来添加新匹配方式（如通配符组合）而无需修改核心代码。

📌 关键代码：

plugins/wasm-go/extensions/ai-proxy/provider/provider.go (605-624)

for k, v := range modelMapping {
    // 混合前缀匹配和正则表达式逻辑
    ...

⚠️ 潜在风险： 新增匹配逻辑时需修改现有代码，增加维护成本；未来扩展复杂策略（如优先级排序）时代码结构将难以维护。

🔍 3. 正则表达式可能引发ReDoS安全风险

允许任意正则表达式可能导致指数级时间复杂度匹配（如包含大量嵌套分组的模式）。建议对用户提供的正则表达式进行复杂度验证（如限制捕获组数量）或采用安全正则引擎。

⚠️ 潜在风险： 恶意构造的正则表达式可能引发CPU占用过高，导致服务崩溃或拒绝服务攻击。

🔍 4. 缺乏正则表达式模式合法性校验

当前未对正则表达式语法有效性进行校验，可能导致运行时panic（如语法错误的正则表达式）。建议在配置加载时预校验所有模式的合法性。

📌 关键代码：

plugins/wasm-go/extensions/ai-proxy/provider/provider.go (617-619)

re := regexp.MustCompile(k) // 语法错误的k会直接panic

⚠️ 潜在风险： 配置错误可能导致服务启动失败或运行时崩溃，影响系统稳定性。

💡 小贴士

与 lingma-agents 交流的方式

📜 直接回复评论
直接回复本条评论，lingma-agents 将自动处理您的请求。例如：

在当前代码中添加详细的注释说明。
请详细介绍一下你说的 LRU 改造方案，并使用伪代码加以说明。

📜 在代码行处标记
在文件的特定位置创建评论并 @lingma-agents。例如：

@Lingma-Agent 分析这个方法的性能瓶颈并提供优化建议。
@Lingma-Agent 对这个方法生成优化代码。

📜 在讨论中提问
在任何讨论中 @lingma-agents 来获取帮助。例如：

@Lingma-Agent 请总结上述讨论并提出解决方案。
@Lingma-Agent 请根据讨论内容生成优化代码。

codecov-commenter · 2025-05-31T17:06:31Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 46.07%. Comparing base (ef31e09) to head (26cea50).
Report is 533 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #2358       +/-   ##
===========================================
+ Coverage   35.91%   46.07%   +10.16%     
===========================================
  Files          69       81       +12     
  Lines       11576    13010     +1434     
===========================================
+ Hits         4157     5995     +1838     
+ Misses       7104     6670      -434     
- Partials      315      345       +30

see 78 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

CH3CHO

对应的文档也需要更新

CH3CHO · 2025-06-01T08:47:09Z

plugins/wasm-go/extensions/ai-proxy/provider/provider.go

+			}
+		}
+
+		if strings.Contains(k, "(") && strings.Contains(k, ")") {


这个的意思是说包含 capture group 的就是正则表达式吗？感觉是不是应该做的更通用一点，比如 ~ 开头的就代表是正则表达式（正常模型名字不会以它开头吧）。这样替换的时候也不要求必须是走 capture group 了。

这里之所以要走 capture group，是需要获取到匹配的内容，并用于之后替换到目标模型名称上

我是说“是不是必须要走 capture group”，而不是“是不是可以走 capture group”，也就是“走 capture group” 是“使用正则映射”的必要还是充分条件。目前这种基于括号的正则表达式判断方式要求的是前者，而你所描述的需求目前我只能看出对后者的要求，看不出前者。因此，建议用更合适的方式来进行这个判断。

我能想到的是必须要用 capture group, 或者有更好的方式实现也可以的
比如我这边的场景是希望能做到如下,以 openai 模型为例:
第一种: gpt-xxx 系列能映射到 openai/gpt-xxx
第二种: 反之 openai/gpt-xxx 也能映射到 gpt-xxx

如果是之前提到的以 ~ 开头代表正则式，第二种好像不太好配置

~ 开头代表正则表达式只是影响 if strings.Contains(k, "(") && strings.Contains(k, ")") { 这句判定，以及实际做匹配的时候要移除开头的 ~，其他的和你现在的是一样的。

而且，有括号不代表一定是正则，建议这里还是用更明确的语义来进行配置。

那么就是以 ~ 开头代表使用正则匹配，而不是判断是否带了 "()"，但 capture group 的能力我觉得还是需要保留

…matching in modelMapping Signed-off-by: Xijun Dai <[email protected]>

Signed-off-by: Xijun Dai <[email protected]>

CH3CHO

LGTM

feat(ai-proxy): add modelMapping regexp support

562d538

Signed-off-by: Xijun Dai <[email protected]>

daixijun requested review from cr7258, CH3CHO and rinfx as code owners May 31, 2025 17:01

lingma-agents bot reviewed May 31, 2025

View reviewed changes

CH3CHO reviewed Jun 1, 2025

View reviewed changes

daixijun added 2 commits June 1, 2025 18:30

docs(ai-proxy): Update the usage instructions for regular expression …

2a00f12

…matching in modelMapping Signed-off-by: Xijun Dai <[email protected]>

feat(ai-proxy): Prefixed with ~ means regexp matches are required

26cea50

Signed-off-by: Xijun Dai <[email protected]>

CH3CHO approved these changes Jun 3, 2025

View reviewed changes

hanxiantao and others added 2 commits June 3, 2025 19:32

Merge branch 'main' into feat/add-model-mapping-regexp

3d3e8aa

Merge branch 'main' into feat/add-model-mapping-regexp

f522067

CH3CHO merged commit 896780b into alibaba:main Jun 3, 2025
7 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(ai-proxy): add modelMapping regexp support #2358

feat(ai-proxy): add modelMapping regexp support #2358

Uh oh!

daixijun commented May 31, 2025 •

edited

Loading

Uh oh!

lingma-agents bot commented May 31, 2025

与 lingma-agents 交流的方式

Uh oh!

lingma-agents bot left a comment •

edited

Loading

Uh oh!

codecov-commenter commented May 31, 2025 •

edited

Loading

Uh oh!

CH3CHO left a comment

Uh oh!

CH3CHO Jun 1, 2025

Uh oh!

daixijun Jun 1, 2025

Uh oh!

CH3CHO Jun 3, 2025 •

edited

Loading

Uh oh!

daixijun Jun 3, 2025

Uh oh!

CH3CHO Jun 3, 2025

Uh oh!

CH3CHO Jun 3, 2025

Uh oh!

daixijun Jun 3, 2025

Uh oh!

CH3CHO Jun 3, 2025

Uh oh!

CH3CHO left a comment

Uh oh!

Uh oh!

Uh oh!

feat(ai-proxy): add modelMapping regexp support #2358

feat(ai-proxy): add modelMapping regexp support #2358

Uh oh!

Conversation

daixijun commented May 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Ⅰ. Describe what this PR did

Ⅱ. Does this pull request fix one issue?

Ⅲ. Why don't you add test cases (unit test/integration test)?

Ⅳ. Describe how to verify it

Uh oh!

lingma-agents bot commented May 31, 2025

feat(ai-proxy): 新增基于正则表达式的模型映射功能

变更文件

时序图

与 lingma-agents 交流的方式

Uh oh!

lingma-agents bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

🔍 代码评审报告

📋 评审意见详情

💡 单文件建议

🚀 跨文件建议

与 lingma-agents 交流的方式

Uh oh!

codecov-commenter commented May 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

CH3CHO left a comment

Choose a reason for hiding this comment

Uh oh!

CH3CHO Jun 1, 2025

Choose a reason for hiding this comment

Uh oh!

daixijun Jun 1, 2025

Choose a reason for hiding this comment

Uh oh!

CH3CHO Jun 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

daixijun Jun 3, 2025

Choose a reason for hiding this comment

Uh oh!

CH3CHO Jun 3, 2025

Choose a reason for hiding this comment

Uh oh!

CH3CHO Jun 3, 2025

Choose a reason for hiding this comment

Uh oh!

daixijun Jun 3, 2025

Choose a reason for hiding this comment

Uh oh!

CH3CHO Jun 3, 2025

Choose a reason for hiding this comment

Uh oh!

CH3CHO left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

daixijun commented May 31, 2025 •

edited

Loading

lingma-agents bot left a comment •

edited

Loading

codecov-commenter commented May 31, 2025 •

edited

Loading

CH3CHO Jun 3, 2025 •

edited

Loading