A module that enhances the [`@singlestore/client`](https://github.com/singlestore-labs/singlestore/tree/main/packages/client) package with Retrieval-Augmented Generation (RAG) functionality, enabling seamless integration of advanced RAG features into a ch
npm install @singlestore/ragA module that enhances the @singlestore/client package with Retrieval-Augmented Generation (RAG) functionality, enabling seamless integration of advanced RAG features into a chat application.
- Installation
- Example Apps
- Usage
- Initialization
- Additional Notes
- RAG
- Get Models
- Chat
- Create Chat
- Additional Notes
- Find Chat
- All Chats
- By Condition
- With Advanced Filtering
- Update Chat
- By Condition
- Specific
- Delete Chat
- By Condition
- Specific
- Chat Session
- Create Chat Session
- Find Chat Session
- All Chat Sessions
- By Condition
- With Advanced Filtering
- Update Chat Session
- By Condition
- Specific
- Delete Chat Session
- By Condition
- Specific
- Submit
- Chat Message
- Create Chat Message
- Find Chat Message
- All Chat Messages
- By Condition
- With Advanced Filtering
- Update Chat Message
- By Condition
- Specific
- Delete Chat Message
- By Condition
- Specific
- File
- Upload File
- Delete File
- Tools
``bash`
npm install @singlestore/rag @singlestore/ai @singlestore/client
To initialize RAG, you need to create instances of the AI class from @singlestore/ai and the SingleStoreClient from @singlestore/client. After that, establish a connection to your workspace and select the database where the required tables for the RAG application will be created to store chats, chat sessions and chat messages.
`ts
import { AI } from "@singlestore/ai";
import { SingleStoreClient } from "@singlestore/client";
import { RAG } from "@singlestore/rag";
const ai = new AI({ openAIApiKey: "
const client = new SingleStoreClient({ ai });
const connection = client.connect({
host: "
user: "
password: "
});
const database = connection.database.use("
const rag = new RAG({ database, ai });
`
#### Additional Notes
- It is possible to use custom LLMs instead of OpenAI. For more information, please refer to this guide.
---
#### Get Models
Returns the list of available models that can be used in the ChatSession.submit method.
`ts`
const models = await rag.getModels();
---
#### Create Chat
Creates a chat where sessions and messages will be stored.
`tschats
const chat = await rag.chat.create({
name: "
systemRole: "
tableName: " by defaultchat_sessions
sessionsTableName: " by defaultchat_messages
messagesTableName: " by defaultfalse
store: true, // Optional; by default,`
tools: toolsList, // Optional
});
##### Additional Notes
- The store parameter determines whether the chat, sessions, and messages should be persisted in the database.tools
- The parameter accepts an array of tools that the LLM can use when executing the ChatSession.submit method. For more information, please refer to this guide.
---
#### Find Chat
Retrieves chat records, either all available chats or those that match specified conditions. You can control the result set using filters, ordering, pagination, and other optional parameters.
##### All Chats
Fetches all chat records without any filtering.
`ts`
const chats = await rag.chat.find();
##### By Condition
Fetches a specific chat record based on a provided condition, such as id.
`ts`
const chat = await rag.chat.find({ where: { id: "
##### With Advanced Filtering
Fetches chat records with additional filtering options. You can apply conditions (where), sorting (orderBy), pagination (limit, offset), and provide custom parameters such as tableName and tools.
`tschats
const chats = await rag.chat.find(
{
where: { columnName: "COLUMN_VALUE" }, // Optional
orderBy: { columnName: "asc" }, // Optional
limit: 10, // Optional
offset: 0, // Optional
}, // Optional
{
tableName: "chats", // Optional; by default`
tools: customToolsList, // Optional
},
);
---
#### Update Chat
Allows to modify chat details, either by specifying conditions for matching multiple records or by targeting a specific chat instance.
##### By Condition
Updates chat records that meet the specified conditions. You can provide any combination of parameters in the where clause to filter which chats to update.
`ts`
await rag.chat.update(
"chats",
// Updated values
{
name: "
systemRole: "
sessionsTableName: "
messagesTableName: "
createdAt: "NEW_CREATED_AT", // Optional
},
// Where condition
{
id: "
name: "
systemRole: "
sessionsTableName: "
messagesTableName: "
},
);
##### Specific
Updates a specific chat instance by directly providing the new values for the chat's fields. Each field is optional and only provided fields will be updated.
`ts`
await chat.update({
name: "
systemRole: "
sessionsTableName: "
messagesTableName: "
createdAt: "NEW_CREATED_AT", // Optional
});
---
#### Delete Chat
Deletes chat records from the database. You can delete chats based on specific conditions or directly target a specific chat instance.
##### By Condition
Deletes chats, sessions, and messages that match the specified conditions.
`ts`
await rag.chat.delete(
"chats",
"chat_sessions",
"chat_messages",
// Where condition
{
id: "
name: "
systemRole: "
sessionsTableName: "
messagesTableName: "
},
);
##### Specific
Deletes a specific chat instance, including its associated sessions and messages.
`ts`
await chat.delete();
---
#### Create Chat Session
Initializes a new chat session. You can optionally provide a name for the session, or create a session with default parameters.
`ts`
const session = await chat.session.create(
{
name: "
}, // Optional
);
#### Find Chat Session
Retrieves session records, either all available sessions or those that match specified conditions. You can control the result set using filters, ordering, pagination, and other optional parameters.
##### All Chat Sessions
Fetches all session records without any filtering.
`ts`
const sessions = await chat.session.find();
##### By Condition
Fetches a specific session record based on a provided condition, such as id.
`ts`
const session = await chat.session.find({ where: { id: "
##### With Advanced Filtering
Fetches session records with additional filtering options. You can apply conditions (where), sorting (orderBy), pagination (limit, offset).
`ts`
const sessions = await chat.session.find(
{
where: { columnName: "COLUMN_VALUE" }, // Optional
orderBy: { columnName: "asc" }, // Optional
limit: 10, // Optional
offset: 0, // Optional
}, // Optional
);
---
#### Update Chat Session
Allows to modify chat session details, either by specifying conditions for matching multiple records or by targeting a specific chat instance.
##### By Condition
Updates chat session records that meet the specified conditions. You can provide any combination of parameters in the where clause to filter which chats to update.
`ts`
await chat.session.update(
// Updated values
{
chatID: "
name: "
createdAt: "
},
// Where condition
{
id: "
name: "
createdAt: "
},
);
##### Specific
Updates a specific chat session instance.
`ts`
await session.update({
chatID: "
name: "
createdAt: "
});
---
#### Delete Chat Session
Deletes chat session records from the database. You can delete chat sessions based on specific conditions or directly target a specific chat session instance.
##### By Condition
Deletes chat sessions, and messages that match the specified conditions.
`ts`
await chat.session.delete({
// Where condition
id: "
name: "
createdAt: "
});
##### Specific
Deletes a specific chat instance, including its associated messages.
`ts`
await session.delete();
---
#### Submit
This method is used to initiate a chat session with a given prompt, model, and various configuration options. You can specify settings such as the model to be used, the history of the chat, database schema loading, and custom tools for function calling. Additionally, it supports streaming responses and handling tool calls and their results
`ts
const customToolsList = [describeDatabaseChatTool(database)];
const stream = await session.submit({
model: "gpt-4o-mini", // Optional; specifies the LLM model to use
prompt: "
systemRole: "
temperature: 0, // Optional; controls the randomness of the output (0 = deterministic, 1 = more creative)
stream: true, // Optional; enables streaming of responses; false by defaultfalse
loadHistory: true, // Optional; loads the chat history into the session; by defaultfalse
loadDatabaseSchema: true, // Optional; loads the database schema into the session; by default[]
messages: [], // Optional; array of messages to preload into the session; by default2048
maxMessagesLength: 2048, // Optional; limits the total length of messages; by default[]
tools: customToolsList, // Optional; list of tools to be used by the LLM during the session; by default
onMessagesLengthSlice: () => {}, // Optional; callback triggered when messages length is sliced
onMessageLengthExceededError: (error) => {}, // Optional; callback for when message length exceeds the limit
onMessagesLengthExceededError: (error) => {}, // Optional; callback for when messages length exceed the limit
toolCallHandlers: { toolName: async (tool, toolParams) => {} }, // Optional; handles calls to custom tools
toolCallResultHandlers: { toolName: async (tool, toolResult, toolParams) => {} }, // Optional; handles the result of tool calls
});
const chatCompletion = await ai.chatCompletions.handleStream(stream, async (chunk) => {
console.log(chunk); // Process each chunk of the streaming response
});
`
#### Create Chat Message
`tsuser
const message = await session.message.create({
role: "user", // Supported values: | assistant | system`
content: "
});
---
#### Find Chat Message
Retrieves chat message records, either all available messages or those that match specified conditions. You can control the result set using filters, ordering, pagination, and other optional parameters.
##### All Chat Messages
Fetches all chat message records without any filtering.
`ts`
const messages = await chat.message.find();
##### By Condition
Fetches a specific message record based on a provided condition, such as id.
`ts`
const message = await chat.message.find({ where: { id: "
##### With Advanced Filtering
Fetches chat message records with additional filtering options. You can apply conditions (where), sorting (orderBy), pagination (limit, offset).
`ts`
const messages = await chat.message.find(
{
where: { columnName: "COLUMN_VALUE" }, // Optional
orderBy: { columnName: "asc" }, // Optional
limit: 10, // Optional
offset: 0, // Optional
}, // Optional
);
---
#### Update Chat Message
Allows to modify chat message details, either by specifying conditions for matching multiple records or by targeting a specific chat instance.
##### By Condition
Updates chat message records that meet the specified conditions. You can provide any combination of parameters in the where clause to filter which chats to update.
`tsuser
await session.message.update(
// Updated values
{
sessionID: "
role: " | assistant | system`
content: "
createdAt: "
},
// Where condition
{
id: "
role: "
content: "
createdAt: "
},
);
##### Specific
Updates a specific chat message instance.
`tsuser
await message.update({
sessionID: "
role: " | assistant | system`
content: "
createdAt: "
});
---
#### Delete Chat Message
Deletes chat message records from the database. You can delete chat messages based on specific conditions or directly target a specific chat message instance.
##### By Condition
Deletes chat messages that match the specified conditions.
`ts`
await session.message.delete(
// Where condition
{
id: "
role: "
content: "
createdAt: "
},
);
##### Specific
Deletes a specific chat message instance.
`ts`
await message.delete();
---
#### Upload File
Uploads a file buffer, creates embeddings based on the file content, and inserts the resulting embeddings into a specified table for use in vector searches. By default, the method stores embeddings in the vectors table, but you can customize the table name and column names.
`ts
const buffer = "
await rag.file.upload({
buffer,
name: "
ext: " | csv | pdfvectors
tableParams: {
name: " by defaultcontent
contentColumnName: " by defaultv_content
vColumnName: " by default`
}, // Optional
textSplitterOptions: {...}, // Optional; ai.textSplitter.split options
embeddingParams: {...}, // Optional; ai.embeddings.create params
});
#### Delete File
Deletes a file's corresponding embeddings from the specified table. By default, it removes entries from the vectors table, but you can customize the table name if needed.
`tsvectors
await rag.file.delete(
"
" by default`
);
---
You can create custom tools to extend the functionality of the LLM when creating or finding chats, or when calling the ChatSession.submit method. These tools can also integrate with function calling, allowing the LLM to execute specific functions during interactions. For detailed information on how to create a tool, refer to this guide.
Additionally, this package includes ready-to-use tools, which you can find here.
Below are examples of how to use tools:
`ts
import { ChatCompletionTool } from "@singlestore/ai";
import { describeDatabaseChatTool, textToSQLChatTool, vectorSearchChatTool } from "@singlestore/rag";
import { z } from "zod";
const customTool = new ChatCompletionTool({
name: "
description: "
params: z.object({ paramName: z.string().describe("
call: async (params) => {
const value = await anyFnCall(params);
return { name: "
},
});
const tools = [
customTool,
describeDatabaseChatTool(database),
textToSQLChatTool(database, ai, { model: "gpt-4o-mini" }),
vectorSearchChatTool(database),
];
const chat = await rag.chat.create({ tools });
// OR
const chats = await rag.chat.find({}, { tools });
// OR
const chatCompletion = await session.submit({ tools });
``