Introduction

This tool allows you to compare the performance of two language models (LLMs) directly from the chat interface. Whether you are a simple user, an annotator, or part of a model management team, this feature provides valuable insights into the strengths and weaknesses of different models.

Using Comparison Models

Selecting Models:

Choose Model 1 and Model 2 from the list of available models.
Optionally, set the temperature to control the creativity of the responses.

Receiving Responses:

When comparison mode is active, you will receive two responses for each prompt entered in the chat interface.

After reviewing the responses, indicate your preference by clicking "Select to proceed." To ask another question, you must select one of the two responses.

Collecting and Using Feedback

Data Collection:

Each time you select your preferred response, the prompt and the chosen answers are securely saved in your database.
This data is invaluable for model improvement