Managing Models in Paradigm's Admin Panel

The Models section in Paradigm allows you to integrate any model into the platform, enabling users to interact with them. You have the flexibility to configure open-source models hosted by yourself or a provider, or to use paid models accessible via API.

Viewing the List of Models

The list of models displays the following information:

• Name: The public name displayed in Paradigm for users.
• Model Type: You can integrate Large Language Models (LLMs) or embedding models. Embedding models are used to convert text into numerical representations to understand semantic similarity.
• Deployment Type: Indicates where the model is deployed (e.g., Sagemaker, OpenAI, or Self-hosted).
• Enabled: Shows whether the model is available for users through the interface or API (green = accessible, red = not accessible).

Viewing Model Details

By clicking on the model's name, you can view or modify its details:

• Name: The public name of the model displayed in the Paradigm interface for end users (this field can be edited).
• Technical Name: A technical identifier used by various APIs (this field should not be modified).
• Model Type: You can integrate LLMs or embedding models.
• Deployment Type: The model's deployment type (e.g., Sagemaker, OpenAI, Self-hosted).
• Sagemaker Endpoint Name: Used only if the deployment type is Sagemaker (e.g., vllm-0-4-custom-2-dev-5--Meta-Llama-3-8B-Instruct-Sk9-ft).
• LoRA Model Name: Specify the name if you are using a LoRA (Low-Rank Adapter) model, a specific fine-tuning technique. We recommend using vLLM (https://docs.vllm.ai/en/latest/) to serve your models. If you've fine-tuned a model with LoRA (a specific finetuning technique) and want to use it in Paradigm, we suggest deploying it using vLLM and providing the Lora Model Name that matches the --lora-modules flag you used when starting the vLLM server.
• Model URL: used when your model is self-hosted. Think of it like a virtual address where your model is being served. For example, let's say you want to deploy a model called Llama-405 on a machine with the IP address 123.150.117.199. In this case, the Model URL would be http://123.150.117.199. You can also map this IP address to a domain name like https://llama-405b.lighton.ai for easier access.
• Start Message Template: This field defines a list of messages added at the beginning of a conversation with the model to customize interaction. It follows the Jinja templating style and allows for conditional operations and the use of variables. Available variables include:
  • {{ date }}: The current date.
  • {{ company_instructions }}: Instructions from the chat settings.
  • {{ user_instructions }}: Instructions set by the user.
  • {{ user_first_name }}: The user's first name.
  • {{ user_last_name }}: The user's last name.
  • Only these variables are available for now; using any other will result in an error when setting the start_messages_template field.
• Instruction: describes how the final model is going to be called.
  i.e. for rewriter: "rephrase this question {question}", for gen model "answer this {question} according to this {context}

Creating a New Model

To create a new model, administrators need to provide information on the model's public name, type, deployment type, and relevant configuration settings like the endpoint name or URL for self-hosted models. You can also specify how the model should be called using customized model instructions (e.g., "Rephrase this question {question}" for a rewriter or "Answer this {question} according to this {context}" for a generative model).

For self-hosted models, ensure the URL points to the service serving the model, and consider using vLLM for efficient model deployment.

By managing these configurations, you can ensure that the models integrated into Paradigm are optimized for various user interactions, enhancing the overall effectiveness and user experience across the platform.