Langdash Documentation#

A simple library for interfacing with language models.

Repository: main / Gitlab mirror

Features:

  • Support for guided text generation, text classification (through prompting) and vector-based document searching.

  • Lightweight, build-it-yourself-style prompt wrappers in pure Python, with no domain-specific language involved.

  • Token healing and transformers/RNN state reuse for fast inference, like in Microsoft’s guidance.

  • First-class support for ggml backends.

Installation#

Use pip to install. By default, langdash does not come preinstalled with any additional modules. You will have to specify what you need like in the following command:

pip install --user langdash[embeddings,sentence_transformers]

List of modules:

  • core:

    • embeddings: required for running searching through embeddings

  • backends:

    • Generation backends: rwkv_cpp, llama_cpp, ctransformers, transformers

    • Embedding backends: sentence_transformers

Note: If running from source, initialize the git submodules in the langdash/extern folder to compile foreign backends.

Version:

1.20.3

Contents:

Indices and tables#