How to add samples via API?

You can also add samples to your sample set via API for evaluation purposes.

When adding samples, you'll need your API key, which can be found on the EvalsOne settings page; and the ID of your sample set, which can be found in the API add sample instructions tab of your sample set.

Here are some examples of how to call the API to add samples in common programming languages:

Example of adding a sample via Curl:

curl -X POST https://api.evalsone.com/api/sample/add \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <YOUR_API_KEY>" \
  -d '{
      "sid": "<SAMPLE_SID>",
      "messages": [{"system": "example instruct"}, {"user": "example user message"}],
      "completion": "<Model generated completion>",
      "ideal": ["ideal answer 1", "ideal answer 2"],
      "context": ["Context 1", "Context 2"]
  }'

You'll need to replace <YOUR_API_KEY> and <SAMPLE_SID> with your actual API key and sample set ID.

Parameter	Required	Format	Description
sid	Yes	String	Unique identifier for the sample
messages	Yes	List of messages	A list of conversation messages, each message being a JSON object with 'system' or 'user' key
completion	No	String	The model generated completion, optional
ideal	No	List of strings	Ideal answers
context	No	List of strings	Context information

Example of adding a sample via Python:

Via EvalsOne Python SDK

With the EvalsOne Python SDK, you can add samples to your sample set with just a few lines of code.

First, install the EvalsOne Python SDK:

pip install evalsone

Then use the following code to add a sample:

eo_client = EvalsOne(api_key='<YOUR_API_KEY>', sid='<SAMPLE_SET_ID>')
sample_data = {'messages': {"role": "user", "content": "Introduce your develop team."}'}
response = eo_client.add_sample(sample_data)

Via requests library

Or, you can directly use the requests library to send a POST request:

import requests
import json

url = "https://api.evalsone.com/api/sample/add"
headers = {
    "Content-Type": "application/json",
    "Authorization": "Bearer <YOUR_API_KEY>"
}
data = {
    "sid": "<SAMPLE_SID>",
    "messages": [{"system": "example instruct"}, {"user": "example user message"}],
    "completion": "<Model generated completion>",  # Optional, completed answer, string
    "ideal": ["ideal answer 1", "ideal answer 2"],  # Optional, ideal answer list, string or list of string
    "context": ["Context 1", "Context 2"]  # Optional, context list, string or list of string
}

requests.post(url, headers=headers, data=json.dumps(data))

Example of adding a sample via PHP:

<?php
$url = "https://api.evalsone.com/api/sample/add";
$headers = array(
    "Content-Type: application/json",
    "Authorization: Bearer <YOUR_API_KEY>"
);
$data = array(
    "sid" => "<SAMPLE_SID>",
    "messages" => array(array("system" => "example instruct"), array("user" => "example user message")),
    "completion" => "<Model generated completion>",  // Optional, generated completion, string
    "ideal" => array("ideal answer 1", "ideal answer 2"),  // Optional, ideal answer list, string or list of string
    "context" => array("Context 1", "Context 2")  // Optional, context list, string or list of string
);

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($data));
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

$response = curl_exec($ch);
curl_close($ch);
echo $response;
?>

Example of adding a sample via NodeJS:

const axios = require('axios');

const url = "https://api.evalsone.com/api/sample/add";
const headers = {
    "Content-Type": "application/json",
    "Authorization": "Bearer <YOUR_API_KEY>"
};
const data = {
    sid: "<SAMPLE_SID>",
    messages: [{system: "example instruct"}, {user: "example user message"}],
    completion: "<Model generated completion>",  // Optional, generated completion, string
    ideal: ["ideal answer 1", "ideal answer 2"],  // Optional, ideal answer list, string or list of string
    context: ["Context 1", "Context 2"]  // Optional List
};

axios.post(url, data, { headers: headers })
    .then(response => {
        console.log(response.data);
    })
    .catch(error => {
        console.error('There was an error!', error);
    });