Procedure for Configuring LLM Settings for Paradigm

This section details the required configuration for the Large Language Model (LLM) settings within the Paradigm application.

Configuration of the Paradigm Address

• In the LLM Settings section, locate the Paradigm Address field.
• Enter the complete URL address of the Paradigm application where the LLM will be deployed or accessible.
  • Address format: https://paradigm-[application-name].[domain-name]

Selection of the LLM Model

• Choose the LLM model you wish to use with your application:
  • Other model: If using a custom model, enter the exact name of the model.
    • Example model name: alfred-2-40b-0723-v1
  • You can also opt for a toy model or a model managed by SageMaker if these options are available.

Allocation of GPU Resources

• Specify the number of GPUs you wish to allocate to the LLM node.
  • Default value: 0

Configuration of Cache Slots

• Determine the number of cache slots needed for your LLM model.
  • Default value: 32

Saving and Applying Settings

• Ensure all entered information is correct and reflects your deployment needs.
• Save the settings to update the configuration of your LLM model in the Paradigm application.

Procedure for Configuring Additional LLM Settings for Paradigm

Continue configuring the LLM settings for your Paradigm application with the following steps:

Configuration of the Model Path

• In the Model Path field, indicate the access path to the LLM model you wish to use.
  • Default Value: /

GCP SA Private Key

• Provide the private key of the Google Cloud Platform (GCP SA) service account in Base64 format.
  • Base64 required: Ensure the private key is encoded in Base64 for security reasons.

LLM Storage Class Name

• Select the LLM Storage Class Name that will define the storage class for the LLM persistent volume.
  • Default Value: standard-rwo
  • Note that this choice is immutable and cannot be changed after the first deployment.

LLM Storage Capacity

• Determine the LLM Storage Capacity by specifying the desired size for LLM storage (unit in gigabytes).
  • Default Value: 100 Gi
• Note that this choice is immutable and cannot be changed after the first deployment.

LLM Persistent Volume Path

• Enter the LLM Persistent Volume Path to indicate the local path of the LLM's persistent volume.
  • Example path to enter: /my/volume/path
  • Required: The path is necessary to locate the persistent storage of the LLM model.

Saving and Applying Settings

• Verify that all entered information is correct, particularly that the GCP SA private key is correctly encoded in Base64 and that the access paths are valid.