Specifying model config with LLMConfig

To specify unified configurations between models, Langdash supports using the LLMConfig class. This is a data class that allows you to specify common config arguments such as context length, number of inference threads, and so on. You can pass an instance of this class into the Langdash.model_from_type function as the config parameter, which will automatically initialize a template with the config specified in the class.

For example, if you want to create a new model using a specific configuration, you can use the following code:

from langdash import Langdash
from langdash.llm_config import LLMConfig

ld = Langdash()
ld.register_model(
  name="model",
  model=Langdash.model_from_type(
    model_type,
    config=LLMConfig(
      model=model_path,
      batch_size=32,
      context_length=1024,
    )
  ),
)

In this example, we pass an instance of the LLMConfig class with the path to the model, a context length of 1024 and a batch size of 32. We then pass this instance into the Langdash.model_from_type function to create a new model with these specifications.