Introduction

Rust has been known for its performance and memory/thread safety. However, it takes an ecosystem to be successful. So far, Rust has found a lot of success in network programming and bit coin mining. Rust has an excellent support for async making it a great language any time we need distributed systems.

For the reason, Rust has found a lot of success in vector databases.

Vector Databases

There are several start-up companies that are building vector databases using Rust: Pinecone, Qdrant and Fennel.ai.

Const generics

Rust has an interesting feature called const generics that is uniquely suitable for vector databases. Const generics are generic arguments that range over constant values, rather than types or lifetimes. This allows, for instance, types to be parameterized by integers. The follow are some examples of const generics:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
// Examples where const generic parameters can be used.

// Used in the signature of the item itself.
fn foo<const N: usize>(arr: [i32; N]) {
    // Used as a type within a function body.
    let x: [i32; N];
    // Used as an expression.
    println!("{}", N * 2);
}

// Used as a field of a struct.
struct Foo<const N: usize>([i32; N]);

impl<const N: usize> Foo<N> {
    // Used as an associated constant.
    const CONST: usize = N * 4;
}

trait Trait {
    type Output;
}

impl<const N: usize> Trait for Foo<N> {
    // Used as an associated type.
    type Output = [i32; N];
}

Frameworks

Rust has been growing in machine learning frameworks as well, for example, Hugging Face candle and burn-rs. Each of these frameworks can only run some models. For example, for candle, candle-whisper and candle-llama2 and for burn, whisper-burn and llama2-burn.

These frameworks can work in spacial cases where frameworks like PyTorch are too large. For example, Candle’s core goal is to make serverless inference possible.

This frameworks gave proof that Rust is a viable language for machine learning. They also built-up an ecosystem of Rust libraries that could be very useful for other Rust projects, for example, tch-rs. Hugging Face also has several tools developed in Rust.

References

I have only touched a small part of machine learning in Rust. For more comprehensive reviews, please check out the following references:

Last modified on 2023-09-04