CAP LLM Plugin

CAP LLM Plugin helps developers create tailored Generative AI based CAP applications:

1. Without exposing confidential data to LLM by anonymizing sensitive data leveraging SAP HANA Cloud Data Anonymization.
2. Seamlessly generate vector embeddings via SAP AI Core.
3. Easily retrieve Chat Completion response via SAP AI Core.
4. Efforlessly perform similarity search via SAP HANA Cloud Vector engine.
5. Simplified single RAG (retrieval-augmented generation) retrieval method powered by SAP AI Core and SAP HANA Cloud Vector Engine.
6. Access the harmonized chat completion API of the SAP AI Core Orchestration service.

✔️ Anonymization Features

| Feature | Details
| :-------------------------------------------------------------------------------------------: | :------------------------------------------------------------------------------------------------------------------------------------
| Seamlessly anonymize sensitive data using a variety of SAP HANA Cloud's anonymization capabilities | Effortlessly anonymize sensitive data within a CAP application by employing a single @anonymize annotation using a diverse range of SAP HANA Cloud's anonymization algorithms, including but not limited to:

k-Anonymity

l-Diversity

Differential Privacy

|
| Effortlessly replace the anonymized data within the LLM response with genuine information| Given that the data provided to the LLM consists of anonymized information, the CAP LLM plugin ensures a seamless replacement of anonymized content within the LLM response with the corresponding authentic data.

🎯 LLM Access Layer Features

| Feature | Details
| :-------------------------------------------------------------------------------------------: | :------------------------------------------------------------------------------------------------------------------------------------
| Embedding generation via SAP AI Core | Easily connect to embedding models via SAP AI Core and generate embeddings seamlessly |
| Similarity search | Leverage the SAP HANA Cloud's Vector engine to perform similarity search via CAP LLM Plugin |
| Chat LLM Access via SAP AI Core | Simple access to LLM models via SAP AI Core with simplified method for chat completion |
| Streamlining RAG retrieval | Single method to streamline the entire RAG retrieval process leveraging SAP AI Core and SAP HANA Cloud Vector Engine |
| Orchestration Service Support | Support for SAP AI Core orchestration service's harmonized chat completion APIs

📚 Samples and documentation

For API documentation of CAP LLM Plugin, check refer to SAP Samples

For sample use cases leveraging CAP LLM Plugin, refer to SAP Samples.

❗ Version Upgrade Notice

From 1.3.* to 1.4.2 (function signature changed for following methods, version not recommended):
getEmbedding
getChatCompletion
getRagResponse

From 1.3.* to 1.4.4 and above(backwards compatible, new methods to support more models):
No change required unless you want to use new methods supporting new models as mentioned in API document:
(old)getEmbedding -> getEmbeddingWithConfig
(old)getChatCompletion -> getChatCompletionWithConfig
(old)getRagResponse -> getRagResponseWithConfig

👉 Guidance :

When to use the SAP Cloud SDK for AI vs. the CAP LLM Plugin?

| SAP Cloud SDK for AI | CAP LLM Plugin |
|--------------------------|---------------------|
| SAP Cloud SDK for AI is the official SDK for SAP AI Core, generative AI hub, and orchestration. This is the recommended AI SDK for customers and partners. | CAP LLM Plugin is an open source community plugin for simplifying the development of SAP CAP applications. |
| - This SDK offers enterprise-grade integration with SAP AI Core and related GenAI platform services.
- Use this for all productive use cases requiring LLM access, chat completion, content filtering, grounding, data masking, orchestration, type safety, and business AI compliance.
- SDK has support for Java, JavaScript/TypeScript and Python.
- Fully endorsed and maintained by SAP for integration with SAP AI Core and related services. | - The plugin is not an official SAP product. It is released under the Apache 2.0 license and is published as a community contribution.
- Targeted for RAG-based CAP applications leveraging SAP HANA Cloud's vector engine capabilities and anonymization capabilities through annotations in SAP CAP applications.
- Supports Node.js and can be used for rapid prototyping of CAP-based RAG applications. |
| Resource: SAP CLoud SDK for AI | Resource: CAP LLM Plugin |

🙌 Support and Contact

This project is provided "as-is" with no expectation for major changes or support.