SingleStore RAG

A module that enhances the @singlestore/client package with Retrieval-Augmented Generation (RAG) functionality, enabling seamless integration of advanced RAG features into a chat application.

Installation

``bash
npm install @singlestore/rag @singlestore/ai @singlestore/client
`

Example Apps

- Terminal Chat

Usage

$3

To initialize RAG, you need to create instances of the AI class from @singlestore/ai and the SingleStoreClient from @singlestore/client. After that, establish a connection to your workspace and select the database where the required tables for the RAG application will be created to store chats, chat sessions and chat messages.

`ts
import { AI } from "@singlestore/ai";
import { SingleStoreClient } from "@singlestore/client";
import { RAG } from "@singlestore/rag";

const ai = new AI({ openAIApiKey: "" });
const client = new SingleStoreClient({ ai });

const connection = client.connect({
host: "",
user: "",
password: "",
});

const database = connection.database.use("");
const rag = new RAG({ database, ai });
`

#### Additional Notes

- It is possible to use custom LLMs instead of OpenAI. For more information, please refer to this guide.

---

$3

#### Get Models

Returns the list of available models that can be used in the ChatSession.submit method.

`ts
const models = await rag.getModels();
`

---

$3

#### Create Chat

Creates a chat where sessions and messages will be stored.

`ts
const chat = await rag.chat.create({
name: "", // Optional
systemRole: "", // Optional
tableName: "", // Optional; chats by default
sessionsTableName: "", // Optional; chat_sessions by default
messagesTableName: "", // Optional; chat_messages by default
store: true, // Optional; false by default,
tools: toolsList, // Optional
});
`

##### Additional Notes

- The store parameter determines whether the chat, sessions, and messages should be persisted in the database.
- The tools parameter accepts an array of tools that the LLM can use when executing the ChatSession.submit method. For more information, please refer to this guide.

---

#### Find Chat

Retrieves chat records, either all available chats or those that match specified conditions. You can control the result set using filters, ordering, pagination, and other optional parameters.

##### All Chats

Fetches all chat records without any filtering.

`ts
const chats = await rag.chat.find();
`

##### By Condition

Fetches a specific chat record based on a provided condition, such as id.

`ts
const chat = await rag.chat.find({ where: { id: "" } });
`

##### With Advanced Filtering

Fetches chat records with additional filtering options. You can apply conditions (where), sorting (orderBy), pagination (limit, offset), and provide custom parameters such as tableName and tools.

`ts
const chats = await rag.chat.find(
{
where: { columnName: "COLUMN_VALUE" }, // Optional
orderBy: { columnName: "asc" }, // Optional
limit: 10, // Optional
offset: 0, // Optional
}, // Optional
{
tableName: "chats", // Optional; chats by default
tools: customToolsList, // Optional
},
);
`

---

#### Update Chat

Allows to modify chat details, either by specifying conditions for matching multiple records or by targeting a specific chat instance.

##### By Condition

Updates chat records that meet the specified conditions. You can provide any combination of parameters in the where clause to filter which chats to update.

`ts
await rag.chat.update(
"chats",
// Updated values
{
name: "", // Optional
systemRole: "", // Optional
sessionsTableName: "", // Optional
messagesTableName: "", // Optional
createdAt: "NEW_CREATED_AT", // Optional
},
// Where condition
{
id: "", // Optional
name: "", // Optional
systemRole: "", // Optional
sessionsTableName: "", // Optional
messagesTableName: "", // Optional
},
);
`

##### Specific

Updates a specific chat instance by directly providing the new values for the chat's fields. Each field is optional and only provided fields will be updated.

`ts
await chat.update({
name: "", // Optional
systemRole: "", // Optional
sessionsTableName: "", // Optional
messagesTableName: "", // Optional
createdAt: "NEW_CREATED_AT", // Optional
});
`

---

#### Delete Chat

Deletes chat records from the database. You can delete chats based on specific conditions or directly target a specific chat instance.

##### By Condition

Deletes chats, sessions, and messages that match the specified conditions.

`ts
await rag.chat.delete(
"chats",
"chat_sessions",
"chat_messages",
// Where condition
{
id: "", // Optional
name: "", // Optional
systemRole: "", // Optional
sessionsTableName: "", // Optional
messagesTableName: "", // Optional
},
);
`

##### Specific

Deletes a specific chat instance, including its associated sessions and messages.

`ts
await chat.delete();
`

---

$3

#### Create Chat Session

Initializes a new chat session. You can optionally provide a name for the session, or create a session with default parameters.

`ts
const session = await chat.session.create(
{
name: "", // Optional
}, // Optional
);
`

#### Find Chat Session

Retrieves session records, either all available sessions or those that match specified conditions. You can control the result set using filters, ordering, pagination, and other optional parameters.

##### All Chat Sessions

Fetches all session records without any filtering.

`ts
const sessions = await chat.session.find();
`

##### By Condition

Fetches a specific session record based on a provided condition, such as id.

`ts
const session = await chat.session.find({ where: { id: "" } });
`

##### With Advanced Filtering

Fetches session records with additional filtering options. You can apply conditions (where), sorting (orderBy), pagination (limit, offset).

`ts
const sessions = await chat.session.find(
{
where: { columnName: "COLUMN_VALUE" }, // Optional
orderBy: { columnName: "asc" }, // Optional
limit: 10, // Optional
offset: 0, // Optional
}, // Optional
);
`

---

#### Update Chat Session

Allows to modify chat session details, either by specifying conditions for matching multiple records or by targeting a specific chat instance.

##### By Condition

Updates chat session records that meet the specified conditions. You can provide any combination of parameters in the where clause to filter which chats to update.

`ts
await chat.session.update(
// Updated values
{
chatID: "", // Optional
name: "", // Optional
createdAt: "", // Optional
},
// Where condition
{
id: "", // Optional
name: "", // Optional
createdAt: "", // Optional
},
);
`

##### Specific

Updates a specific chat session instance.

`ts
await session.update({
chatID: "", // Optional
name: "", // Optional
createdAt: "", // Optional
});
`

---

#### Delete Chat Session

Deletes chat session records from the database. You can delete chat sessions based on specific conditions or directly target a specific chat session instance.

##### By Condition

Deletes chat sessions, and messages that match the specified conditions.

`ts
await chat.session.delete({
// Where condition
id: "",
name: "",
createdAt: "",
});
`

##### Specific

Deletes a specific chat instance, including its associated messages.

`ts
await session.delete();
`

---

#### Submit

This method is used to initiate a chat session with a given prompt, model, and various configuration options. You can specify settings such as the model to be used, the history of the chat, database schema loading, and custom tools for function calling. Additionally, it supports streaming responses and handling tool calls and their results

`ts
const customToolsList = [describeDatabaseChatTool(database)];

const stream = await session.submit({
model: "gpt-4o-mini", // Optional; specifies the LLM model to use
prompt: "", // The prompt to be sent to the LLM
systemRole: "", // Optional; defines the role of the system (e.g., instructions or persona)
temperature: 0, // Optional; controls the randomness of the output (0 = deterministic, 1 = more creative)
stream: true, // Optional; enables streaming of responses; false by default
loadHistory: true, // Optional; loads the chat history into the session; false by default
loadDatabaseSchema: true, // Optional; loads the database schema into the session; false by default
messages: [], // Optional; array of messages to preload into the session; [] by default
maxMessagesLength: 2048, // Optional; limits the total length of messages; 2048 by default
tools: customToolsList, // Optional; list of tools to be used by the LLM during the session; [] by default
onMessagesLengthSlice: () => {}, // Optional; callback triggered when messages length is sliced
onMessageLengthExceededError: (error) => {}, // Optional; callback for when message length exceeds the limit
onMessagesLengthExceededError: (error) => {}, // Optional; callback for when messages length exceed the limit
toolCallHandlers: { toolName: async (tool, toolParams) => {} }, // Optional; handles calls to custom tools
toolCallResultHandlers: { toolName: async (tool, toolResult, toolParams) => {} }, // Optional; handles the result of tool calls
});

const chatCompletion = await ai.chatCompletions.handleStream(stream, async (chunk) => {
console.log(chunk); // Process each chunk of the streaming response
});
`

$3

#### Create Chat Message

`ts
const message = await session.message.create({
role: "user", // Supported values: user | assistant | system
content: "",
});
`

---

#### Find Chat Message

Retrieves chat message records, either all available messages or those that match specified conditions. You can control the result set using filters, ordering, pagination, and other optional parameters.

##### All Chat Messages

Fetches all chat message records without any filtering.

`ts
const messages = await chat.message.find();
`

##### By Condition

Fetches a specific message record based on a provided condition, such as id.

`ts
const message = await chat.message.find({ where: { id: "" } });
`

##### With Advanced Filtering

Fetches chat message records with additional filtering options. You can apply conditions (where), sorting (orderBy), pagination (limit, offset).

`ts
const messages = await chat.message.find(
{
where: { columnName: "COLUMN_VALUE" }, // Optional
orderBy: { columnName: "asc" }, // Optional
limit: 10, // Optional
offset: 0, // Optional
}, // Optional
);
`

---

#### Update Chat Message

Allows to modify chat message details, either by specifying conditions for matching multiple records or by targeting a specific chat instance.

##### By Condition

Updates chat message records that meet the specified conditions. You can provide any combination of parameters in the where clause to filter which chats to update.

`ts
await session.message.update(
// Updated values
{
sessionID: "", // Optional
role: "", // Optional; Supported values: user | assistant | system
content: "", // Optional
createdAt: "", // Optional
},
// Where condition
{
id: "", // Optional
role: "", // Optional
content: "", // Optional
createdAt: "", // Optional
},
);
`

##### Specific

Updates a specific chat message instance.

`ts
await message.update({
sessionID: "", // Optional
role: "", // Optional; Supported values: user | assistant | system
content: "", // Optional
createdAt: "", // Optional
});
`

---

#### Delete Chat Message

Deletes chat message records from the database. You can delete chat messages based on specific conditions or directly target a specific chat message instance.

##### By Condition

Deletes chat messages that match the specified conditions.

`ts
await session.message.delete(
// Where condition
{
id: "", // Optional
role: "", // Optional
content: "", // Optional
createdAt: "", // Optional
},
);
`

##### Specific

Deletes a specific chat message instance.

`ts
await message.delete();
`

---

$3

#### Upload File

Uploads a file buffer, creates embeddings based on the file content, and inserts the resulting embeddings into a specified table for use in vector searches. By default, the method stores embeddings in the vectors table, but you can customize the table name and column names.

`ts
const buffer = "";

await rag.file.upload({
buffer,
name: "",
ext: "", // Supported values: txt | csv | pdf
tableParams: {
name: "", // Optional; vectors by default
contentColumnName: "", // Optional; content by default
vColumnName: "", // Optional; v_content by default
}, // Optional
textSplitterOptions: {...}, // Optional; ai.textSplitter.split options
embeddingParams: {...}, // Optional; ai.embeddings.create params
});
`

#### Delete File

Deletes a file's corresponding embeddings from the specified table. By default, it removes entries from the vectors table, but you can customize the table name if needed.

`ts
await rag.file.delete(
"",
"", // Optional; vectors by default
);
`

---

$3

You can create custom tools to extend the functionality of the LLM when creating or finding chats, or when calling the ChatSession.submit method. These tools can also integrate with function calling, allowing the LLM to execute specific functions during interactions. For detailed information on how to create a tool, refer to this guide.

Additionally, this package includes ready-to-use tools, which you can find here.

Below are examples of how to use tools:

`ts
import { ChatCompletionTool } from "@singlestore/ai";
import { describeDatabaseChatTool, textToSQLChatTool, vectorSearchChatTool } from "@singlestore/rag";
import { z } from "zod";

const customTool = new ChatCompletionTool({
name: "",
description: "",
params: z.object({ paramName: z.string().describe("") }),
call: async (params) => {
const value = await anyFnCall(params);
return { name: "", params, value: JSON.stringify(value) };
},
});

const tools = [
customTool,
describeDatabaseChatTool(database),
textToSQLChatTool(database, ai, { model: "gpt-4o-mini" }),
vectorSearchChatTool(database),
];

const chat = await rag.chat.create({ tools });
// OR
const chats = await rag.chat.find({}, { tools });
// OR
const chatCompletion = await session.submit({ tools });
``