OpenAI compatible gateway.
npm install @kluai/gatewayKlu gateway is an OpenAI compatible gateway package. With it you can self host your own OpenAI API that supports the following models:
* OpenAI
* Azure OpenAI
* Perplexity
* Anthropic
* Replicate
* Google (Gemini models)
To install the gateway simply run the following command:
``bash`
npm i @kluai/gateway
We support node versions 14+. If you are using an older version of node, please upgrade.
In order to run on edge (Cloudflare for example) you need to copy the cloudflare directory in this repository.
Once you've copied over the code into your own project, create a cloud worker and run the following command:
`bash`
npm i
wrangler dev
Now you will be able to run this locally. Feel free to tweak the implementation as much as you need and once you are ready to deploy run:
`bash`
wrangler deploy
If you'd like to run this into a cloud you can import the OpenAI gateway function like so.
`typescript
import { OpenAI } from "@kluai/gateway";
export default {
async fetch(request: Request, context: ExecutionContext) {
return OpenAI()(request, context);
},
};
``
You could run this in a lambda or any other server and you would be ready to handle any number of requests.
For feedback, suggestions or problems feel free to email stefan@klu.ai