Skip to content

Add DuckDB integration with example usage and tests #1193

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

AshAnand34
Copy link

This pull request introduces a new integration with DuckDB, an in-process SQL OLAP database, to enhance support for analytical workloads and similarity search in Retrieval-Augmented Generation (RAG) workflows. The changes include the addition of a new module for DuckDB integration, an example script, tests, and updates to documentation and dependencies.

Related issue: #924

DuckDB Integration:

  • New DuckDB Integration Module:

    • Added the DuckDBIntegration class in llmware/duckdb_integration.py, providing methods to initialize the database, create tables, insert data, query data, and close the connection.
  • Example Script:

    • Added examples/Models/duckdb_rag_example.py, demonstrating how to use the DuckDBIntegration class for creating tables, inserting data, and querying a DuckDB database.
  • Unit Tests:

    • Added tests/models/test_duckdb_integration.py to validate the functionality of the DuckDBIntegration class, including creating tables, inserting data, querying data, and closing the database connection.

Documentation and Dependencies:

  • Documentation Update:

    • Updated docs/components/model_catalog.md to include a new section on DuckDB integration, detailing its features, use cases, and references to the example script and tests.
  • Dependency Update:

    • Added duckdb>=0.7.1 to llmware/requirements.txt to ensure the necessary library is available for the integration.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant