@sf-ai/rag-core
v0.2.1
Core RAG schema with pgvector support for document embeddings
0/weekUpdated 2 weeks agoMITUnpacked: 15.6 KB
Published by Dan Lynch
npm install @sf-ai/rag-core
@sf-ai/rag-core
Core RAG (Retrieval-Augmented Generation) schema with pgvector support for document embeddings.
Overview
This package provides the foundational database schema for storing and managing document embeddings using PostgreSQL and pgvector. It supports:
- Multiple embedding models per collection
- Semantic chunking strategies
- JSON/CSV import and export for migrations
Schema
$3
| Table | Description |
|-------|-------------|
| rag.embedding_model | Embedding model configurations (OpenAI, Cohere, local, etc.) |
| rag.collection | Document collections with chunking configuration |
| rag.collection_model | Links collections to embedding models (many-to-many) |
| rag.document | Source documents with content and metadata |
| rag.chunk | Document segments/chunks for embedding |
| rag.embedding | Vector embeddings (pgvector) linked to chunks |
$3
The chunk_config JSONB column on rag.collection supports:
``json`
{
"strategy": "semantic",
"max_tokens": 512,
"overlap_tokens": 50
}
Supported strategies: semantic, fixed, sentence, paragraph
Installation
`bash``
pgpm deploy @sf-ai/rag-core
Requirements
- PostgreSQL 17+
- pgvector extension