Chimpbase Connect


Chimpbase Connect moves streaming data between services with minutes of setup. Declarative pipelines, a production-grade runtime, and first-class observability let product teams run webhooks, queue bridges, and data fan-out jobs without bespoke glue code.

Built for teams that need:
- Webhook ingestion with deterministic forwarding
- Queue-to-queue synchronization across services
- Lightweight streaming jobs that can be monitored and tuned from the CLI

Why teams choose Chimpbase Connect

- Declarative control – Describe connectors, processors, and outputs once in YAML; ship new pipelines via config changes, not rewrites.
- Operational safety – Backpressure, retries, DLQs, and circuit-breaking guard downstream systems automatically.
- Observability by default – Structured logs, correlation IDs, and metrics make every pipeline debuggable in production.
- Shipping velocity – Install with npm, reuse the component catalog, and push new pipelines without staging expensive stacks.

Run pipelines in minutes

$3

Use npx for zero-install runs or install the package globally for shared tooling.

``bash
npx @chimpbase/connect --help


or


npm install -g @chimpbase/connect
`

$3

Capture inputs, processors, and outputs in YAML. This example receives webhooks, adds metadata, and forwards to an API.

`yaml
input:
http:
port: 8080
path: "/webhook"

pipeline:
processors:
- metadata:
correlation_id_field: "correlationId"

output:
http:
url: "https://api.example.com/events"
method: POST
`

$3

Point the CLI at any config and add flags for debug logging or dry runs.

`bash
npx @chimpbase/connect run my-pipeline.yaml
chimpbase-connect run my-pipeline.yaml --debug # after global install
chimpbase-connect run my-pipeline.yaml --once # process available messages and exit
`

Use structured logs for quick triage or forward metrics/traces to your preferred observability stack via the built-in exporters.

Operationally ready out of the box

- Backpressure & flow control – Keep throughput high and latency predictable even under bursty load.
- Dead-letter handling – Automatic retries, error categorization, and DLQ fan-out preserve failed events.
- Extensible component model – Add new connectors or processors without touching the runtime.
- Testing harness – YAML assertions and programmable harnesses validate pipelines before rollout.

Component catalog

Inputs (HTTP, AWS SQS, Redis), processors (metadata, mapping, HTTP, logging), and outputs (HTTP, SQS, Redis) ship with the CLI. Browse the complete catalog and advanced guides in docs/ (soon mirrored on the public documentation site).

Resources & next steps

- docs/ – Local copies of detailed guides until the public docs site launches.
- configs/ – Ready-made pipeline examples to clone or adapt.
- docs/TESTING.md` – How to validate pipelines with YAML or programmable harnesses.

Contributing

Contributions are welcome—open an issue or pull request with context on the use case you are enabling.

License

MIT