Skip to content

test_llm_chain_filter is flaky in libs/langchain #26273

Closed
@efriis

Description

@efriis

Checked other resources

  • I added a very descriptive title to this issue.
  • I searched the LangChain documentation with the integrated search.
  • I used the GitHub search to find a similar question and didn't find it.
  • I am sure that this is a bug in LangChain rather than my code.
  • The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

https://github.com/langchain-ai/langchain/actions/runs/10798432256/job/29951972859?pr=26270

Error Message and Stack Trace (if applicable)

  At index 1 diff: Document(metadata={'c': 3}, page_content='The moon is round.') != Document(metadata={'b': 2}, page_content='Candlepin bowling balls are smaller.')
  Full diff:
    [
     Document(metadata={'a': 1}, page_content='Candlepin bowling is popular in New England.'),
  -  Document(metadata={'b': 2}, page_content='Candlepin bowling balls are smaller.'),
  +  Document(metadata={'c': 3}, page_content='The moon is round.'),
    ]```

### Description

This unit test mostly succeeds, sometimes documents seem to come back in wrong order

Likely a bug in the document compressor

### System Info

CI

Metadata

Metadata

Assignees

No one assigned

    Labels

    🤖:bugRelated to a bug, vulnerability, unexpected error with an existing feature

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions