Skip to content

Feature: USearch VectorStore with custom DataStore #602

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
3 tasks done
winxalex33 opened this issue May 6, 2025 · 0 comments
Open
3 tasks done

Feature: USearch VectorStore with custom DataStore #602

winxalex33 opened this issue May 6, 2025 · 0 comments
Labels
enhancement New feature or request

Comments

@winxalex33
Copy link

winxalex33 commented May 6, 2025

Integration with DI C#.

If some want to use USearch with any datastore in C# can use this template until official support. `using System;
using System.Collections.Generic;
using System.Configuration;
using System.Linq;
using System.Threading.Tasks;
using Cloud.Unum.USearch; // Import USearch
using Microsoft.Data.SqlClient;
using Microsoft.Extensions.AI;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Options;
using Microsoft.Extensions.VectorData;

namespace USearchVectorDataExample
{
class Program
{
// --- Configuration ---

    const string SqliteDbPath = "glossary_datastore.db"; // Path for SQLite data store
    const string CollectionName = "glossary"; // Name for the collection
    const string CollectionNameMovie = "movies"; // Name for the collection

    static async Task Main(string[] args)
    {
         

        

        IEmbeddingGenerator<string, Embedding<float>> generator =
            new OllamaEmbeddingGenerator(new Uri("http://localhost:11434/"), "nomic-embed-text:latest");

        var host = Host.CreateDefaultBuilder(args).ConfigureAppConfiguration((context, config) =>
            {
                config.AddJsonFile("appsettings.json", optional: false, reloadOnChange: true);
            })
           .ConfigureServices((context, services) =>
           {
               // Access the IConfiguration instance from the context
               var configuration = context.Configuration;

               // Register services and configure options using the IConfiguration instance
               services.Configure<USearchVectorStoreOptions>(configuration.GetSection("USearchSettings"));

               // 1. Register the specific IDataStore implementation(s)
               services.AddSingleton<IDataStore<ulong, Glossary>>(new SqliteDataStore<Glossary>(SqliteDbPath));
               services.AddSingleton<IDataStore<ulong, Movie>>(new SqliteDataStore<Movie>(SqliteDbPath));

               // 2. Register the USearchVectorStore
               services.AddSingleton<IVectorStore>(sp =>
               {
                   var options = sp.GetRequiredService<IOptions<USearchVectorStoreOptions>>();
                   return new USearchVectorStore(options, sp);
               });
           })
           .Build();

        // --- Get Services and Run Logic ---
        // Get the IVectorStore from the DI container
        var vectorStore = host.Services.GetRequiredService<IVectorStore>();
        // ==================================================
        // --- Column defintion ---
        // ==================================================
        string columnSearchTerm = "Дај ми ги сите случаеви што се отворени 20се затворени";
        var columnSearchVector = await GenerateEmbeddingVector(generator, columnSearchTerm);
        //let say vector store returns colums that are semanticly close to the search term


        // ==================================================
        // --- Movie Example ---
        // ==================================================
        Console.WriteLine("\n\n--- Processing Movie Collection ---");

        // Get the Movie Collection
        IVectorStoreRecordCollection<ulong, Movie> movieCollection = vectorStore.GetCollection<ulong, Movie>(
            CollectionNameMovie
        );

        await movieCollection.CreateCollectionIfNotExistsAsync();

        // Populate Movie Data
        Console.WriteLine($"Checking/Populating Movie data in '{CollectionNameMovie}'...");
        await PopulateSampleMovieDataAsync(movieCollection, generator); // Use the new movie population method

        // Movie Searches
        //string movieSearchTerm = "sci-fi movie about space travel and survival";
        //string movieSearchTerm = "sci-fi movie about space travel and survival";
        string movieSearchTerm = "A movie that’s fun for the whole family";
        var movieSearchVector = await GenerateEmbeddingVector(generator, movieSearchTerm);

        if (movieSearchVector.Length > 0) // Only search if vector generation succeeded
        {
            Console.WriteLine($"\n--- Movie Searches for: '{movieSearchTerm}' ---");

            Console.WriteLine("\n--- Hybrid Search: Vector + Keywords (Tags) ---");
            var hybridSearchOptionsM1 = new HybridSearchOptions<Movie>
            {
                AdditionalProperty = r => r.Tags, // Search keywords against Tags
                Top = 5
            };
            // Use keywords relevant to the search term and potential tags
            var searchResultM1 = await ((IKeywordHybridSearch<Movie>)movieCollection).HybridSearchAsync(movieSearchVector, new[] { "nolan", "space", "classic" }, hybridSearchOptionsM1);
            await PrintMovieResultsAsync("Vector + Keywords ('nolan', 'space', 'classic')", searchResultM1);

            Console.WriteLine("\n--- Hybrid Search: Vector + Filter (Genre and Tag) ---");
            var hybridSearchOptionsM2 = new HybridSearchOptions<Movie>
            {
                Filter = r => r.Genre == "Sci-Fi" && r.Tags.Contains("dystopian"),
                Top = 3,
                // VectorProperty defaults to the single ReadOnlyMemory<float> property if unambiguous,
                // otherwise specify: VectorProperty = r => r.MovieDescriptionEmbedding
            };
            // Provide empty keyword list when focusing on vector + filter
            var searchResultM2 = await ((IKeywordHybridSearch<Movie>)movieCollection).HybridSearchAsync(movieSearchVector, new List<string>(), hybridSearchOptionsM2);
            await PrintMovieResultsAsync("Vector + Filter (Genre='Sci-Fi', Tags='dystopian')", searchResultM2);


            Console.WriteLine("\n--- Hybrid Search: Vector + Filter (Genre) + IncludeVectors ---");
            var hybridSearchOptionsM3 = new HybridSearchOptions<Movie>
            {
                Filter = r => r.Genre == "Animation",
                Top = 4,
                IncludeVectors = true // Request vectors in the results
            };
            var searchResultM3 = await ((IKeywordHybridSearch<Movie>)movieCollection).HybridSearchAsync(movieSearchVector, new List<string>(), hybridSearchOptionsM3);
            await PrintMovieResultsAsync("Vector + Filter (Genre='Animation') + IncludeVectors", searchResultM3, includeVector: true);


            Console.WriteLine($"\n--- Vector Search Only '{movieSearchTerm}' (using VectorizedSearchAsync) ---");
            var searchOptionsM4 = new VectorSearchOptions<Movie>
            {
                Top = 5,
                IncludeVectors = false
            };
            var searchResultM4 = await movieCollection.VectorizedSearchAsync(movieSearchVector, searchOptionsM4);
            await PrintMovieResultsAsync("Vector Search Only (Top 5)", searchResultM4);


            Console.WriteLine($"\n--- Vector Search of '{movieSearchTerm}' with some score filter (using VectorizedSearchAsync) ---");
            var searchOptionsM5 = new HybridSearchOptionsWithScoreFiltering<Movie>
            {
                ScoreFilter = distance => distance < 0.38f, // Example score filter
                Top = 20,
                IncludeVectors = false
            };
            var searchResultM5 = await ((IKeywordHybridSearch<Movie>)movieCollection).HybridSearchAsync(movieSearchVector,Array.Empty<string>(), searchOptionsM5);
            await PrintMovieResultsAsync("Vector Search Only (Top 5)", searchResultM5);


            OllamaChatClient ollamaChatClient = new OllamaChatClient("http://localhost:11434/", "llama3.1:latest");


            ChatResponse movieChatResponse = await ollamaChatClient.GetResponseAsync(await CreateMoviePrompt(movieSearchTerm, searchResultM5));

            Console.WriteLine(movieChatResponse.Messages[movieChatResponse.Messages.Count - 1].Text); // Set the last message to assistant role
        }
        else { Console.WriteLine($"Skipping Movie searches as embedding generation failed for '{movieSearchTerm}'."); }

        // --- Cleanup ---
        Console.WriteLine("\nDisposing USearchVectorStore...");
        ((USearchVectorStore)vectorStore).Dispose(); // Dispose the store (which should handle its indexes)

        Console.WriteLine("\n--- Example Complete ---");

       


        return;

    // Get the Collection (passing dependencies using the specific overload)
    //    Cast to the specific search interface needed later.

    IVectorStoreRecordCollection<ulong, Glossary> vectorStoreRecordCollection = vectorStore.GetCollection<ulong, Glossary>(
            CollectionName

        );

        await vectorStoreRecordCollection.CreateCollectionIfNotExistsAsync();



        // --- Populate Data (if index is new or corresponding data seems missing) ---
        if (vectorStoreRecordCollection != null)
        {
            Console.WriteLine("Populating collection with sample data...");
            // Pass the collection instance, which implements the required interface
            await PopulateSampleDataAsync(vectorStoreRecordCollection,generator); // Pass the specific collection instance

        }


        // --- Example Searches ---
        string searchTerm = "semantic kernel";

        // Generate a sample search vector (replace with actual embedding)
        var searchVector = await GenerateEmbeddingVector(generator,searchTerm);

        Console.WriteLine("\n--- Hybrid Search: Vector + Keywords ---");
        var hybridSearchOptions1 = new HybridSearchOptions<Glossary>
        {
            AdditionalProperty = r => r.Tags,
            Top = 5
        };
        var searchResult1 = await ((IKeywordHybridSearch<Glossary>)vectorStoreRecordCollection).HybridSearchAsync(searchVector, new[] { "semantic", "kernel" }, hybridSearchOptions1);
        await PrintResultsAsync("Vector + Keywords ('semantic', 'kernel')", searchResult1);


        Console.WriteLine("\n--- Hybrid Search: Vector + Keywords + Top/Skip ---");
        var hybridSearchOptions2 = new HybridSearchOptions<Glossary>
        {
            AdditionalProperty = r => r.Definition,
            Top = 2,
            Skip = 1
        };
        var searchResult2 = await ((IKeywordHybridSearch<Glossary>)vectorStoreRecordCollection).HybridSearchAsync(searchVector, new[] { "data" }, hybridSearchOptions2);
        await PrintResultsAsync("Vector + Keyword ('data') + Skip 1, Top 2", searchResult2);


        Console.WriteLine("\n--- Hybrid Search: Vector + Filter ---");
        var hybridSearchOptions3 = new HybridSearchOptions<Glossary>
        {
            Filter = r => r.Category == "AI" && r.Tags.Contains("core"),
            Top = 5,
            VectorProperty = r => r.DefinitionEmbedding
        };
        var searchResult3 = await ((IKeywordHybridSearch<Glossary>)vectorStoreRecordCollection).HybridSearchAsync(searchVector, new List<string>(), hybridSearchOptions3);
        await PrintResultsAsync("Vector + Filter (Category='AI', Tags='core')", searchResult3);


        Console.WriteLine("\n--- Hybrid Search: Vector + Filter + IncludeVectors ---");
        var hybridSearchOptions4 = new HybridSearchOptions<Glossary>
        {
            Filter = r => r.Category == "Framework",
            Top = 3,
            IncludeVectors = true
        };
        var searchResult4 = await ((IKeywordHybridSearch<Glossary>)vectorStoreRecordCollection).HybridSearchAsync(searchVector, new List<string>(), hybridSearchOptions4);
        await PrintResultsAsync("Vector + Filter (Category='Framework') + IncludeVectors", searchResult4, includeVector: true);


        Console.WriteLine("\n--- Vector Search Only (using SearchAsync) ---");
        var searchOptions5 = new VectorSearchOptions<Glossary>
        {
            Top = 3,
            IncludeVectors = false
        };
        var searchResult5 = await vectorStoreRecordCollection.VectorizedSearchAsync(searchVector, searchOptions5);
        await PrintResultsAsync("Vector Search Only (Top 3)", searchResult5);


        // --- Cleanup ---
        // Dispose USearchIndex explicitly. SqliteDataStore is disposed by 'await using'.


        ((USearchVectorStore)vectorStore).Dispose();
        Console.WriteLine("\n--- Example Complete ---");
    }

    // Helper to populate sample data


    static async Task<string> CreateMoviePrompt(string searchQuery, VectorSearchResults<Movie> searchResults)
    {
        // Build context description
        string contextDescription = $"The following movies were retrieved based on the search query: \"{searchQuery}\"";

        // Format the search results
        string resultsText = "Retrieved Movies:\n";
        await foreach (var result in searchResults.Results)
        {
            resultsText += $"Movie: {result.Record?.MovieName}\n";
            resultsText += $"Description: {result.Record?.MovieDescriptionText}\n";
            resultsText += $"Genre: {result.Record?.Genre}\n";
            resultsText += $"Tags: {string.Join(", ", result.Record?.Tags ?? Enumerable.Empty<string>())}\n";
            resultsText += $"Similarity Score: {result.Score:F4}\n\n";
        }

        // Create the prompt for Ollama
        string prompt = $@"{contextDescription}

{resultsText}

Based on the search query and only these movie results, please provide:

  1. A brief analysis of why these movies match the search query
  2. Any common themes or patterns among the results

Please keep the response concise and focused on these aspects.";

        return prompt;
    }


    static async Task PopulateSampleDataAsync(IVectorStoreRecordCollection<ulong, Glossary> collection, IEmbeddingGenerator<string,Embedding<float>> generator)
    {
        var random = new Random();
        // Fix for IDE0028: Simplify collection initialization
        var sampleData = new List<Glossary>
        {
            new Glossary
            {
                Key = 1,
                Term = "Semantic Kernel",
                Definition = "An SDK that integrates LLM AI models with conventional programming languages.",
                Category = "Framework",
                Tags = new List<string> { "AI", "SDK", "core","kernel" },
                DefinitionEmbedding = await GenerateEmbeddingVector(generator, "An SDK that integrates LLM AI models with conventional programming languages.")
            },
            new Glossary
            {
                Key = 2,
                Term = "Vector Database",
                Definition = "A database designed to store and query high-dimensional vector data.",
                Category = "Data",
                Tags = new List<string> { "database", "vector", "search" },
                DefinitionEmbedding = await GenerateEmbeddingVector(generator, "A database designed to store and query high-dimensional vector data.")
            },
            new Glossary
            {
                Key = 3,
                Term = "Embedding",
                Definition = "A numerical representation of text, images, or other data in a vector space.",
                Category = "AI",
                Tags = new List<string> { "vector", "representation", "core" },
                DefinitionEmbedding = await GenerateEmbeddingVector(generator, "A numerical representation of text, images, or other data in a vector space.")
            },
            new Glossary
            {
                Key = 4,
                Term = "USearch",
                Definition = "An efficient library for approximate nearest neighbor search in vector data.",
                Category = "Library",
                Tags = new List<string> { "vector", "search", "ANN" },
                DefinitionEmbedding = await GenerateEmbeddingVector(generator, "An efficient library for approximate nearest neighbor search in vector data.")
            },
            new Glossary
            {
                Key = 5,
                Term = "Hybrid Search",
                Definition = "A search strategy combining vector similarity search with traditional keyword or metadata filtering.",
                Category = "Search",
                Tags = new List<string> { "vector", "keyword", "filter" },
                DefinitionEmbedding = await GenerateEmbeddingVector(generator, "A search strategy combining vector similarity search with traditional keyword or metadata filtering.")
            },
            new Glossary
            {
                Key = 6,
                Term = "LLM",
                Definition = "Large Language Model - A type of AI model trained on vast amounts of text data.",
                Category = "AI",
                Tags = new List<string> { "model", "language", "core" },
                DefinitionEmbedding = await GenerateEmbeddingVector(generator, "Large Language Model - A type of AI model trained on vast amounts of text data.")
            }
        };




        // With this corrected code:
        await foreach (var key in collection.UpsertBatchAsync(sampleData))
        {
            // Optionally handle the returned keys here if needed
        }
    }

    private static async Task<ReadOnlyMemory<float>> GenerateEmbeddingVector(IEmbeddingGenerator<string, Embedding<float>> generator, string input)
    {
        // Fix for CS0029: Extract the vector from the Embedding<float> and convert it to ReadOnlyMemory<float>
        var generatedEmbeddings = await generator.GenerateAsync(new List<string> { input });
        var embedding = generatedEmbeddings.FirstOrDefault(); // Assuming you want the first embedding
        return embedding?.Vector ?? ReadOnlyMemory<float>.Empty; // Return the vector or an empty memory if null
    }


    //GenerateEmbeddingVector

    // Helper to generate placeholder vectors (REPLACE with actual embedding generation)
    //static ReadOnlyMemory<float> GenerateEmbeddingVector(uint dimensions, int seed)
    //{
    //    var random = new Random(seed);
    //    var vector = new float[dimensions];
    //    float norm = 0;
    //    for (int i = 0; i < dimensions; i++)
    //    {
    //        vector[i] = (float)(random.NextDouble() * 2 - 1);
    //        norm += vector[i] * vector[i];
    //    }
    //    norm = (float)Math.Sqrt(norm);
    //    if (norm > 0)
    //    {
    //        for (int i = 0; i < dimensions; i++) vector[i] /= norm;
    //    }
    //    return new ReadOnlyMemory<float>(vector);
    //}

    // Helper to print search results
    static async Task PopulateSampleMovieDataAsync(IVectorStoreRecordCollection<ulong, Movie> collection, IEmbeddingGenerator<string, Embedding<float>> generator)
    {
        // Basic check: If the collection seems empty, populate it.
        bool needsPopulation = true;
        try
        {
            var checkResult = await collection.VectorizedSearchAsync(new float[1], new VectorSearchOptions<Movie> { Top = 1 });
            await foreach (var item in checkResult.Results)
            {
                needsPopulation = false; // Found at least one item
                break;
            }
        }
        catch { /* Ignore errors during check, assume population needed */ }

        if (!needsPopulation)
        {
            Console.WriteLine("Movie collection already contains data. Skipping population.");
            return;
        }

        Console.WriteLine("Populating movie collection with sample data...");
        var sampleMovies = new List<Movie>
        {
            // --- Initial 5 Movies ---
            new Movie { MovieId = 1, MovieName = "The Lion King", MovieDescriptionText = "...", Genre = "Animation", Tags = new List<string> { "disney", "classic", "family", "musical" } },
            new Movie { MovieId = 2, MovieName = "Inception", MovieDescriptionText = "...", Genre = "Sci-Fi", Tags = new List<string> { "action", "thriller", "nolan", "mind-bending" } },
            new Movie { MovieId = 3, MovieName = "Toy Story", MovieDescriptionText = "...", Genre = "Animation", Tags = new List<string> { "pixar", "family", "comedy", "adventure" } },
            new Movie { MovieId = 4, MovieName = "Pulp Fiction", MovieDescriptionText = "...", Genre = "Crime", Tags = new List<string> { "tarantino", "non-linear", "dark comedy", "classic" } },
            new Movie { MovieId = 5, MovieName = "Shrek", MovieDescriptionText = "...", Genre = "Animation", Tags = new List<string> { "comedy", "fantasy", "family", "parody" } },
            // --- Additional 10 Movies ---
            new Movie { MovieId = 6, MovieName = "The Matrix", MovieDescriptionText = "...", Genre = "Sci-Fi", Tags = new List<string> { "action", "cyberpunk", "dystopian", "classic" } },
            new Movie { MovieId = 7, MovieName = "Forrest Gump", MovieDescriptionText = "...", Genre = "Drama", Tags = new List<string> { "comedy", "romance", "historical", "classic" } },
            new Movie { MovieId = 8, MovieName = "The Dark Knight", MovieDescriptionText = "...", Genre = "Action", Tags = new List<string> { "superhero", "crime", "thriller", "dc" } },
            new Movie { MovieId = 9, MovieName = "Spirited Away", MovieDescriptionText = "...", Genre = "Animation", Tags = new List<string> { "fantasy", "adventure", "anime", "ghibli" } },
            new Movie { MovieId = 10, MovieName = "Interstellar", MovieDescriptionText = "...", Genre = "Sci-Fi", Tags = new List<string> { "adventure", "drama", "space", "nolan" } },
            new Movie { MovieId = 11, MovieName = "Parasite", MovieDescriptionText = "...", Genre = "Thriller", Tags = new List<string> { "dark comedy", "drama", "social commentary", "korean" } },
            new Movie { MovieId = 12, MovieName = "Gladiator", MovieDescriptionText = "...", Genre = "Action", Tags = new List<string> { "historical", "drama", "epic" } },
            new Movie { MovieId = 13, MovieName = "Finding Nemo", MovieDescriptionText = "...", Genre = "Animation", Tags = new List<string> { "adventure", "comedy", "family", "pixar" } },
            new Movie { MovieId = 14, MovieName = "Blade Runner 2049", MovieDescriptionText = "...", Genre = "Sci-Fi", Tags = new List<string> { "neo-noir", "dystopian", "mystery" } },
            new Movie { MovieId = 15, MovieName = "Coco", MovieDescriptionText = "...", Genre = "Animation", Tags = new List<string> { "fantasy", "adventure", "family", "pixar", "music" } }
        };

        // Assign the full descriptions back for embedding generation
        // (Descriptions were shortened above for brevity in the code listing)
        sampleMovies[0].MovieDescriptionText = "The Lion King is a classic Disney animated film that tells the story of a young lion named Simba who embarks on a journey to reclaim his throne as the king of the Pride Lands after the tragic death of his father.";
        sampleMovies[1].MovieDescriptionText = "Inception is a mind-bending science fiction film directed by Christopher Nolan. It follows the story of Dom Cobb, a skilled thief who specializes in entering people’s dreams to steal their secrets. However, he is offered a final job that involves planting an idea into someone’s mind.";
        sampleMovies[2].MovieDescriptionText = "Toy Story is a groundbreaking animated film from Pixar. It follows the secret lives of toys when their owner, Andy, is not around. Woody and Buzz Lightyear are the main characters in this heartwarming tale.";
        sampleMovies[3].MovieDescriptionText = "Pulp Fiction is a crime film directed by Quentin Tarantino. It weaves together interconnected stories of mobsters, hitmen, and other colorful characters in a non-linear narrative filled with dark humor and violence.";
        sampleMovies[4].MovieDescriptionText = "Shrek is an animated comedy film that follows the adventures of Shrek, an ogre who embarks on a quest to rescue Princess Fiona from a dragon-guarded tower in order to get his swamp back.";
        sampleMovies[5].MovieDescriptionText = "A computer hacker learns about the true nature of his reality and his role in the war against its controllers.";
        sampleMovies[6].MovieDescriptionText = "The presidencies of Kennedy and Johnson, the Vietnam War, the Watergate scandal and other historical events unfold from the perspective of an Alabama man with an IQ of 75, whose only desire is to be reunited with his childhood sweetheart.";
        sampleMovies[7].MovieDescriptionText = "When the menace known as the Joker wreaks havoc and chaos on the people of Gotham, Batman must accept one of the greatest psychological and physical tests of his ability to fight injustice.";
        sampleMovies[8].MovieDescriptionText = "During her family's move to the suburbs, a sullen 10-year-old girl wanders into a world ruled by gods, witches, and spirits, and where humans are changed into beasts.";
        sampleMovies[9].MovieDescriptionText = "A team of explorers travel through a wormhole in space in an attempt to ensure humanity's survival.";
        sampleMovies[10].MovieDescriptionText = "Greed and class discrimination threaten the newly formed symbiotic relationship between the wealthy Park family and the destitute Kim clan.";
        sampleMovies[11].MovieDescriptionText = "A former Roman General sets out to exact vengeance against the corrupt emperor who murdered his family and sent him into slavery.";
        sampleMovies[12].MovieDescriptionText = "After his son is captured in the Great Barrier Reef and taken to Sydney, a timid clownfish sets out on a journey to bring him home.";
        sampleMovies[13].MovieDescriptionText = "Young Blade Runner K's discovery of a long-buried secret leads him to track down former Blade Runner Rick Deckard, who's been missing for thirty years.";
        sampleMovies[14].MovieDescriptionText = "Aspiring musician Miguel, confronted with his family's ancestral ban on music, enters the Land of the Dead to find his great-great-grandfather, a legendary singer.";


        // Generate embeddings in batches
        var descriptions = sampleMovies.Select(m => m.MovieDescriptionText).ToList();
        var embeddings = await generator.GenerateAsync(descriptions);

        if (embeddings.Count == sampleMovies.Count)
        {
            for (int i = 0; i < sampleMovies.Count; i++)
            {
                sampleMovies[i].MovieDescriptionEmbedding = embeddings[i]?.Vector ?? ReadOnlyMemory<float>.Empty;
            }

            Console.WriteLine($"Generated {embeddings.Count} embeddings for movies. Upserting batch...");
            await foreach (var key in collection.UpsertBatchAsync(sampleMovies)) { /* Optional handling */ }
            Console.WriteLine("Movie data upsert complete.");
        }
        else
        {
            Console.WriteLine("Error: Embedding generation count mismatch for movies. Skipping population.");
        }
    }

    static async Task PrintMovieResultsAsync(string title, VectorSearchResults<Movie> queryResult, bool includeVector = false)
    {
        Console.WriteLine($"--- Results for: {title} ---");
        int count = 0;
        await foreach (var result in queryResult.Results)
        {
            count++;
            var movie = result.Record;
            Console.WriteLine($"  {count}. Key: {movie.MovieId}, Score: {result.Score:F4}, Name: {movie?.MovieName}");
            Console.WriteLine($"     Description (Snippet): {movie?.MovieDescriptionText?.Substring(0, Math.Min(movie.MovieDescriptionText.Length, 100))}..."); // Show snippet
            Console.WriteLine($"     Genre: {movie?.Genre}, Tags: [{string.Join(", ", movie?.Tags ?? Enumerable.Empty<string>())}]");
            if (includeVector && movie != null && movie.MovieDescriptionEmbedding.Length > 0)
            {
                const int maxVectorElementsToShow = 5;
                var vectorSnippet = movie.MovieDescriptionEmbedding.Length <= maxVectorElementsToShow
                   ? movie.MovieDescriptionEmbedding.ToArray()
                   : movie.MovieDescriptionEmbedding.Slice(0, maxVectorElementsToShow).ToArray();

                string vectorString = string.Join(", ", vectorSnippet.Select(v => v.ToString("F2")));
                if (movie.MovieDescriptionEmbedding.Length > maxVectorElementsToShow) { vectorString += ", ..."; }
                Console.WriteLine($"     Vector (first {maxVectorElementsToShow}): [{vectorString}] (Total Dims: {movie.MovieDescriptionEmbedding.Length})");
            }
            else if (includeVector) { Console.WriteLine($"     Vector: Not included, not found, or empty."); }
        }
        if (count == 0) { Console.WriteLine("  No results found."); }
        Console.WriteLine("-------------------------------------");
    }

    static async Task PrintResultsAsync(string title, VectorSearchResults<Glossary> queryResult, bool includeVector = false)
    {
        Console.WriteLine($"--- Results for: {title} ---");
        int count = 0;
        // Use IAsyncEnumerable iteration
        await foreach (var result in queryResult.Results)
        {
            count++;
            Console.WriteLine($"  {count}. Key: {result.Record.Key}, Score: {result.Score:F4}, Term: {result.Record?.Term}");
            Console.WriteLine($"     Definition: {result.Record?.Definition}");
            Console.WriteLine($"     Category: {result.Record?.Category}, Tags: [{string.Join(", ", result.Record?.Tags ?? Enumerable.Empty<string>())}]");
            if (includeVector)
            {
                // Safely access the vector value
                Console.WriteLine($"     Vector: [{string.Join(", ", result.Record.DefinitionEmbedding.ToArray().Select(v => v.ToString("F2")))}]");
            }
            else if (includeVector)
            {
                Console.WriteLine($"     Vector: Not included or not found.");
            }
        }
        if (count == 0)
        {
            Console.WriteLine("  No results found.");
        }
        Console.WriteLine("-------------------------------------");
    }

`

P.S Why I need to do all memalloc, L2 cache management and all those low level mambo jumbo. Why we didn't automatize that so we can focus on creativity. With LLMs are we gone from bits lower to quants or we gone from programming languages to natural languages.

HybridSearchOptionsWithScoreFiltering - Copy.txt
IDataStore - Copy.txt
SqliteDataStore - Copy.txt
USearchVectorStore - Copy.txt
USearchVectorStoreOptions - Copy.txt

Can you contribute to the implementation?

  • I can contribute

Is your feature request specific to a certain interface?

It applies to everything

Contact Details

No response

Is there an existing issue for this?

  • I have searched the existing issues

Code of Conduct

  • I agree to follow this project's Code of Conduct
@winxalex33 winxalex33 added the enhancement New feature or request label May 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant