New Rust 🦀 & C 🐂 APIs in USearch v2.11

#include <stdio.h> // For printing.
#include <assert.h> // For assertions!
#include <usearch/usearch.h> // For the win 🚀

Filtering with Predicates

The new USearch release exposes low-level C++ predicated-search functionality to Rust and C 99, also providing several convenience methods already present in the Python and JavaScript APIs. In Rust you may use it like this:

let is_odd = |key: Key| key % 2 == 1;
let query = vec![0.2, 0.1, 0.2, 0.1, 0.3];
let results = index.filtered_search(&query, 10, is_odd).unwrap();
assert!(
    results.keys.iter().all(|&key| key % 2 == 1),
    "All keys must be odd"
);

The same using the C:

int is_odd(usearch_key_t key, void* state) {
    return key % 2;
}

int main() {
    ...
    usearch_key_t found_keys[10];
    usearch_distance_t found_distances[10];
    usearch_filtered_search(
        index, &query[0], usearch_scalar_f32_k, 10, 
        &is_odd, NULL, // no state needed for this callback
        &found_keys[0], &found_distances[0], &error);

User Defined Metrics

While most vector search packages concentrate on just two metrics, "Inner Product distance" and "Euclidean distance", USearch allows arbitrary user-defined metrics. This flexibility allows you to customize your search for various applications, from computing geospatial coordinates with the rare [Haversine][haversine] distance to creating custom metrics for composite embeddings from multiple AI models, like joint image-text embeddings. You could already use Numba, Cppyy, or PeachPy to define your custom metric even in Python:

from numba import cfunc, types, carray
from usearch.index import Index, MetricKind, MetricSignature, CompiledMetric

@cfunc(types.float32(types.CPointer(types.float32), types.CPointer(types.float32)))
def python_inner_product(a, b):
    a_array = carray(a, ndim)
    b_array = carray(b, ndim)
    c = 0.0
    for i in range(ndim):
        c += a_array[i] * b_array[i]
    return 1 - c

metric = CompiledMetric(pointer=python_inner_product.address, kind=MetricKind.IP, signature=MetricSignature.ArrayArray)
index = Index(ndim=ndim, metric=metric, dtype=np.float32)

Similar effect was much easier to achieve in the C++ layer, and is now also exposed to Rust and C.

simsimd_distance_t callback(void const* a, void const* b, void* state) {
    // Your custom metric implementation here
}

int main() {
    ...
    void callback_state = NULL;
    usearch_change_metric(index, callback, callback_state, usearch_metric_unknown_k, &error);

Let's say you are implementing a weighted distance function to search through joint embeddings of images and textual descriptions of some products in a catalog, taking some UForm or CLIP-like multimodal embedding models. Implementing that in Rust using SimSIMD for unimodal slices may look like:

use simsimd::SpatialSimilarity;

let image_dimensions: usize = 768;
let text_dimensions: usize = 512;
let image_weights: f32 = 0.7;
let text_weights: f32 = 0.9;

let weighted_distance = Box::new(move |a: *const f32, b: *const f32| unsafe {
    let a_slice = std::slice::from_raw_parts(a, image_dimensions + text_dimensions);
    let b_slice = std::slice::from_raw_parts(b, image_dimensions + text_dimensions);

    let image_similarity = f32::cosine(a_slice[0..image_dimensions], b_slice[0..image_dimensions]);
    let text_similarity = f32::cosine(a_slice[image_dimensions..], b_slice[image_dimensions..]);
    let similarity = image_weights * image_similarity + text_weights * text_similarity / (image_weights + text_weights);
    
    1.0 - similarity
});
index.change_metric(weighted_distance);

Broader Type System for Rust

USearch supports the Rust-native f32 and f64 scalar types, as well as the i8 for quantized 8-bit scalars. Going beyond that, USearch supports non-native f16 and b1x8 for half-precision floating point and binary vectors, respectively.

Half Precision Floating Point

Rust has no native support for half-precision floating-point numbers, but USearch provides a f16 type. It has no advanced functionality - it is a transparent wrapper around i16 and can be used with half or any other half-precision library. Assuming USearch uses the IEEE 754 no conversion is needed, you can unsafe-cast the outputs of other IEEE-compliant libraries to usearch::f16.

use usearch::f16 as USearchF16;
use half::f16 as HalfF16;

let vector_a: Vec<HalfF16> = ...
let vector_b: Vec<HalfF16> = ...

let buffer_a: &[USearchF16] = unsafe { std::slice::from_raw_parts(a_half.as_ptr() as *const SimF16, a_half.len()) };
let buffer_b: &[USearchF16] = unsafe { std::slice::from_raw_parts(b_half.as_ptr() as *const SimF16, b_half.len()) };

index.add(42, buffer_a);
index.add(43, buffer_b);

Binary Vectors

USearch also implement binary distance functions and natively supports bit-vectors. If you initialize the index with quantization: ScalarKind::B1, you can add floating-point vectors and they will be quantized mapping positive values to 1 and negative and zero values to 0. Alternatively, you can use the b1x8 type to represent packed binary vectors directly.

let index = Index::new(&IndexOptions {
    dimensions: 8,
    metric: MetricKind::Hamming,
    quantization: ScalarKind::B1,
    ..Default::default()
})
.unwrap();

// Binary vectors represented as `b1x8` slices
let vector42: Vec<b1x8> = vec![b1x8(0b00001111)];
let vector43: Vec<b1x8> = vec![b1x8(0b11110000)];
let query: Vec<b1x8> = vec![b1x8(0b01111000)];

// Adding binary vectors to the index
index.reserve(10).unwrap();
index.add(42, &vector42).unwrap();
index.add(43, &vector43).unwrap();

let results = index.search(&query, 5).unwrap();

// Validate the search results based on Hamming distance
assert_eq!(results.keys.len(), 2);
assert_eq!(results.keys[0], 43);
assert_eq!(results.distances[0], 2.0); // 2 bits differ between query and vector43
assert_eq!(results.keys[1], 42);
assert_eq!(results.distances[1], 6.0); // 6 bits differ between query and vector42

Metadata

Both C and Rust now provide:

• memory_usage API, that reports the number of bytes consumed.
• hardware_acceleration API that reports the codename for used SIMD extensions.

Changelog 2.11.0 (2024-04-08)

• b1x8 for Rust (540fc75)
• filtered_search API in C & C++ (5bb38aa)
• filtered_search in Rust (e1b24e1)
• Hyper-param configs in C (c7ed1d4)
• Metadata extraction in C (2c698cd)
• Guides for Rust and C (fe05ad8)
• -Wdeprecated-this-capture (#387) (a233e20), closes #387
• Explicit capture (208e383)
• Filter the entry point in the 0 layer (4ff568b)
• JS imports & Doxygen (#384) (7356158), closes #384
• More adequate default Rust params (c4f9e65)
• metric_punned_t static methods (220ef57)
• Configure Rust compilation options (32c9f3b), closes #378
• Include sources in Crates (0d414e4)

v2.10.5

2.10.5 (2024-04-04)

Fix

• Ensure Rust build aborts on error (7c0095a)

Make

• JS CI cross compiling (#383) (6e1356c), closes #383

v2.10.4

2.10.4 (2024-04-02)

Fix

• TypeScript docs (28688ac)

Make

• NodeJS packaging (#379) (4b0cd31), closes #379

v2.10.3

2.10.3 (2024-04-02)

Make

• Create missing directories (39cf7c3)

v2.10.2

2.10.2 (2024-04-01)

Make

• Cleaner SQLite CI (e0bc522), closes #376 #373 #371

v2.10.1

2.10.1 (2024-04-01)

Fix

• Strip shared lib extension (9c10855), closes #373

Improve

• SQLite download process (47f94ef)

Make

• usearch.h path (d04f898)
• Override OSX arch target (aff6293)
• Versioning releases from main (27897e8)

Broader Compatibility in USearch v2.10.0

USearch v2.10.0 brings major compatibility improvements:

• Better CMake thanks to @Ngalstyan4 🤗
• Pre-build JavaScript binaries thanks to @sroussey 🤗
• Better docs thanks to @simonw 🤗

Notable new features include:

• Use AVX in Windows Python images (c694880)
• Standalone SQLite binaries (227083b)
• Introspecting SIMD capabilities in Rust builds (16fa90f)
• Pretty printing the index with __repr_pretty__ (7087138)
• Improved bit-casting for negative float-s (1d03a9d)

Moreover, new USearch ships 50 binaries to PyPi, covering more platforms than NumPy with 35, but still fewer than SimSIMD and StringZilla with 105. More improvements to come to the most portable fast approximate nearest neighbors search engine 🥳

v2.9.2

2.9.2 (2024-03-05)

Docs

• Comparison tolerance (01c5c89)
• New shields (8a964ec)

Fix

• index.vectors computed property (fe710cc), closes #349
• multi initialization in C tests (7f4dee4)
• Add BigInt validation in index creation,Node Support (5c61c95)
• Bugfix for importlib attrib error (1bc24dd)
• No Pearson correlation for 1d data (3b09259)
• ordering keys before comparison (6bf1df8)
• Reordering vectors before comparison (a4fb305)

Make

• Upgrade SimSIMD & StringZilla (e7aac02)

Hashes

• docs.tar.gz : 71c0d872c29a98d2fdbfbd27946682372138f54512e701d3f40810cb8566ab0d
• usearch-v2.9.2.tar.gz : 16736360f30c9e27d9544ac1fc7662939e02ea223c748e29e8e18ba282a2eadf
• usearch-v2.9.2.zip : 572988a414ddc8ebe6091648c6fe12edfe3fb01ddb2c4000a3538d56e547aa76
• usearch_linux_amd_2.9.2.deb : 4025bf124e3ee32029407a85b8eb7011e9a3ce1f19f5057cc674391264947ea2
• usearch_linux_arm_2.9.2.deb : a9ad80f35789b5f661ad8fd2d01663661d036c61ff04a9109a6e8ef9d20f3a99
• usearch_macOS_arm64_2.9.2.zip : ebfc16c1da6029110260dbb3bd5b3e1c20855cc5ffaa69ee00d30436f2bf377b
• usearch_macOS_x86_64_2.9.2.zip : 7860f96675d87ff3afc0587e5940a3732e54c933057ad1290802ae8110a945f0
• usearch_wasm_linux_arm64_2.9.2.tar.gz : a610edde242ee047156945b82dcf9ad02448fce5c815ae57f87296ffa81ddecd
• usearch_wasm_linux_x86_64_2.9.2.tar.gz : 4e389a217fad0c7f2802697330c8b19cbbee09201c9e85542404f694d17426e2
• usearch_wasm_macos_arm64_2.9.2.zip : d88faf6739742a6c141736a6565eaae1316e327d295023501c97b5d77abac895
• usearch_wasm_macos_x86_64_2.9.2.zip : dc9a5df62be5ffad2fa37f312cfce213f4cc669d9bd3b8e117c518e132db3ce7
• usearch_wasm_windows_x64_2.9.2.tar.gz : 80a8aac793c5bd537eef7b16c22186886ae1f6146a50fb4c06068eb3a8894d10
• usearch_wasm_windows_x86_2.9.2.tar.gz : 2746e5cc7a172017324f76922bc9803309a5d9b3caf21df14ded9ce29faedb57
• usearch_windows_x64_2.9.2.tar : 440924a5e7aba5fd95daf673043b7a6fee3447ecefc3c2f380acee8c5626bed6
• usearch_windows_x86_2.9.2.tar : bf851998175614f406ba31395c07f4d65a9f98ef1f09003e2fe00cb180c13ad0

v2.9.1

2.9.1 (2024-02-27)

Fix

• Explicit narrowing conversions (25436ae)
• Match enum values between C and C# (4a69086)
• Recovering quantization settings in load (fd53619), closes #353 #343
• wildcard ES module imports (#351) (d5091ac), closes #351

Improve

• Report missing arguments in debug builds (ce5bbaf)
• Test reopening index (24be07b)

Make

• Clean-up the C# dir (70d646c)
• invalid C++17 Clang arg (23684aa)

Hashes

• docs.tar.gz : 80b75daa7f627b9a96607fff2498d695f71dc4a12f3695bc9a3521ae8fc12f6a
• usearch-v2.9.1.tar.gz : 69f22afb6da763c11840ca28307523b1cd1154d6925b0f05235af9e49674eba0
• usearch-v2.9.1.zip : 771f434c5cd818c4e6b395ab5cc7ac755b167417a724a9bb70f95954f21cb603
• usearch_linux_amd_2.9.1.deb : e12ba570f5f8eba17221218e1d49f9fae5f5936cb7c21727df1f72f6712d3490
• usearch_linux_arm_2.9.1.deb : ce6fb426af55a96324f77d1ada9ab58cf741ea787358e681e17805601f359279
• usearch_macOS_arm64_2.9.1.zip : 6a1c951c429172867d2b0849ca775b1c87e0f051303337dbf09e1dccbfaa7d79
• usearch_macOS_x86_64_2.9.1.zip : 8e3a2af95febddde4cf78f6adb187cda7375655308a946f20b59a7fd5381eb88
• usearch_wasm_linux_arm64_2.9.1.tar.gz : 470440a59551fb9f3a980a5837f5bbe95b37f0527dbaa41cd5d0e662f9e51551
• usearch_wasm_linux_x86_64_2.9.1.tar.gz : bb577526cedfbf4ec3310e74a49b158145bb17ded857c0d80718fb1154470348
• usearch_wasm_macos_arm64_2.9.1.zip : e4c26aa027cab0f0322ecb8cc6d13ef124acf6191e250fee132b183568b9591e
• usearch_wasm_macos_x86_64_2.9.1.zip : 717e1a2a7d9b53bab74df2768c86863d597857bd7a2be7edbb4cc201e15a2600
• usearch_wasm_windows_x64_2.9.1.tar.gz : 34faf4748899305e9b8c688462a183295ed1e39bdf60bfc6da4348dd3de755a1
• usearch_wasm_windows_x86_2.9.1.tar.gz : 967f2c40e22a2cf51b0e92558c653bd2585440cc0fc827701160a3d8c718dc0e
• usearch_windows_x64_2.9.1.tar : f73729ce83e3501d994bc8dc336c188c91b14aad5a473846f1fe74ed53cccd9a
• usearch_windows_x86_2.9.1.tar : 0d68b669a7ae3607b74a7a27005653aaa7607be74e4dfa57ffcd6fb0148d2ba6

v2.9.0

2.9.0 (2024-02-22)

Add

• SQLite binding (222de55)
• String distances to SQLite (ae4d0f0)

Docs

• Header refreshed (7465c29)
• Py and SQLite extensions (550624b)
• README.md link to Joins (#327) (1279c54), closes #327

Fix

• bug reports were immediately marked invalid (c5fc825)
• Error handling, mem safety bugs #335 (#339) (4747ef4), closes #335 #339
• Passing SQLite tests (6334983)
• Reported number of levels (9b1a06a)
• Skip non-Linux SQLite tests (b02d262)
• SQLite cosine function + tests (55464fb)
• undefined var error in remove api (8d86a9e)

Improve

• Multi property lookup (e8bf02c)
• Support multi-column vectors (66f1716)

Make

• npi ci (#330) (5680920), closes #330
• Add 3.12 wheels (d66f697)
• Change include paths (21db294)
• invalid C++17 Clang arg (2a6d779)
• Link libpthread for older Linux GCC builds (#324) (6f1e5dd), closes #324
• Parallel CI for Python wheels (a9ad89e)
• Upgrade SimSIMD & StringZilla (5481bdf)

Revert

• Postpone Apache Arrow integration (5d040ca)

Hashes

• docs.tar.gz : 068f9712d50aa9734dd636ffce9a58453544754bc476f7df2aa07b3634816d46
• usearch-v2.9.0.tar.gz : f2ceef55ba874b1ab14b115f8a8487848ebdd1384425497b1a76a6e4c1ab6718
• usearch-v2.9.0.zip : 51ec2db7403bf31459f0a63f90fe0437031f9353134a51837a68afbc59605fba
• usearch_linux_amd_2.9.0.deb : 723cfa1d56dd909b3d6da2bf1c1d2d55625239574ccbad7d69ed4efb93c64e33
• usearch_linux_arm_2.9.0.deb : 4b277ccdfd93cfdddd983a215d3d588f4cb6cffda3dfe07cc4f9675aa47fd373
• usearch_macOS_arm64_2.9.0.zip : 0c3f9df3b62292ad8adabe03f36336ab08bc556e4a7dd082b39347fd95e9d062
• usearch_macOS_x86_64_2.9.0.zip : 5baa0a303b9d0cad829c995f6cfeb0eccc968cb01ef45313c3907ba1da2e0c8c
• usearch_wasm_linux_arm64_2.9.0.tar.gz : 2726dec40ad971f0140e444d2ed0857829be197b5fd42483f0d77c22f8c4b062
• usearch_wasm_linux_x86_64_2.9.0.tar.gz : 241fe2473a9baac74ef782432465176e77261c5c6bbb2de6cccdbb37ecae4d78
• usearch_wasm_macos_arm64_2.9.0.zip : 573bea5c54503b4ccb573169a367a9018b552d9c8de627443462da409c751376
• usearch_wasm_macos_x86_64_2.9.0.zip : feeb8f5a8e0a58d337f44fb59d86caf595bbd88ce1f8f99465f0de2535efc7a5
• usearch_wasm_windows_x64_2.9.0.tar.gz : b5551b5364c1f8cf72ff27f9930a7cacbbf68f3e643181cffb29e6389d02d191
• usearch_wasm_windows_x86_2.9.0.tar.gz : 1e232da2280c9bd10f39e06801c32859b72b2b22aff5d482f71aa04ec001cae6
• usearch_windows_x64_2.9.0.tar : 22dadba584463f93b22bc97c816b7d5451459211ed9b21bdc995bab53358fde5
• usearch_windows_x86_2.9.0.tar : 5421825873e60d6cc7083d95a02a4928f6187e3f4d4fe2f92a6706f30713db46