@techolution-ai/computer-vision

A JavaScript/TypeScript library for computer vision applications, providing tools for image processing, scanning, and MQTT-based messaging.

Table of Contents

- Features
- Usage
- Stream the video
- Listening to MQTT messages
- Scanner with MQTT with UI
- API Reference
- Requirements
- License
- Contributing
- Support

Features

The library provides several modules:

- Core computer vision functionality
- Scanner module for image/video scanning
- MQTT-based messaging system for real-time communication

Usage

$3

Use scanner to stream the camera's live recording to web using useScanner hook.

``tsx
import { useMemo } from 'react';

import { useScanner } from '@techolution-ai/computer-vision/scanner';

function App(){

const scanner = useScanner({ baseUrl: 'http://0.0.0.0:8501' })
const streamUrl = useMemo(()=>scanner.getVideoStreamUrl(),[scanner]);

// render the image with steaming url
return
}
`

$3

We can use useMessages to connect, subscribe to certain topics, and can receive the messages on onMessage callback.

We must use the MessagesProvider to use useMessages hook. It can be put at the root (or one level up) to the component's tree.

`tsx
import {
MessagesProvider,
useMessages,
} from '@techolution-ai/computer-vision/messages'


function MessagesListener() {

const handleReceiveMessage = (topic: string, message: any) => {
// your topic and message is here
// add your logic here
}

useMessages({
topics: ['current_status', 'products_scanned'], // subscribing to topics
onMessage: handleReceiveMessage, // message handler
})
}

function App(){



}
`

$3

We can combine above examples to do a full integration.

We also have a ready-to-use react component: ScannerStatus with default options added which can be used to show different statuses in the UI.

`tsx

import { useMemo, useState } from 'react';

import {
ScanImage,
useScanner,
ScannerStatus,
defaultStatusMap,
TStatusMap,
} from '@techolution-ai/computer-vision/scanner';

import {
MessagesProvider,
useMessages,
} from '@techolution-ai/computer-vision/messages';

function MessagesListener() {
const scanner = useScanner({ baseUrl: 'http://0.0.0.0:8501' })

const streamUrl = useMemo(() => scanner.getVideoStreamUrl(), [scanner])

const [statusMap, setStatusMap] = useState(
defaultStatusMap,
)

const [status, setStatus] = useState('idle')
const [messages, setMessages] = useState<{ topic: string; message: any }[]>([])

const handleReceiveMessage = (topic: string, message: any) => {
try {
const value = JSON.parse(message.toString())

if (topic === 'current_status') {
setStatus(value.status)
} else if (topic === 'item_scanned') {
setMessages((prev) => [...prev, { topic, message: value }])
}
} catch (e) {
console.error('Error parsing message:', e)
}
}

useMessages({
topics: ['current_status', 'item_scanned'],
onMessage: handleReceiveMessage,
})

return (
style={{
border: '4px solid #4b2f99',
borderRadius: 16,
overflow: 'hidden',
position: 'relative',
width: '100%',
height: '100%',
}}
status={status}
statusMap={statusMap}
>
src={streamUrl}
style={{
width: '100%',
height: '100%',
objectFit: 'cover',
position: 'absolute',
top: 0,
left: 0,
bottom: 0,
right: 0,
}}
/>

)
}

function App() {
return (



)
}

`

API Reference

The library exposes three main modules:

1. Scanner Module
- Specialized scanning capabilities
- Import via @techolution-ai/computer-vision/scanner

2. Messages Module
- MQTT-based messaging system
- Import via @techolution-ai/computer-vision/messages

$3

Provider for MQTT connection.

#### Props

Prop | Type | Default Value | Remarks
| ----- | ----- | ----- | ----- |
| url | string | -- | Required |
| enableDebugging | boolean | false | Optional, to enable logging |

$3

Take all HTMLDivElement props as root

#### Props

Prop | Type | Default Value | Remarks
| ----- | ----- | ----- | ----- |
| status | string | -- | Required |
| statusMap | TStatusMap (Object) | null | Optional, uses default statusMap internally if not passed |

$3

#### Props

| Prop | Type | Default Value | Remarks |
| ----- | ----- | ----- | ----- |
| topics | string[] | -- | Required |
| onMessage | (topic:string,message:Buffer)=>void | callback to receive messages | Required |
| onError | function | Callback function when an error occurs. (not implemented) |
| onConnect | function | Callback function when the connection is established. (not implemented) |
| onClose | function |Callback function when the connection is closed. (not implemented) |

$3

#### Props

| Prop | Type | Default Value | Remarks |
| ----- | ----- | ----- | ----- |
| baseUrl | string | -- | Required | The base URL for the scanner API.
| endpoints | object | Custom endpoints for the scanner API. (Optional)

#### APIs (returned functions)

| API | Remarks |
| ----- | ----- |
| getVideoStreamUrl() | Returns the URL for the video stream. |
| startInference() | Starts the inference process. |
| stopInference() | Stops the inference process. |
| captureFrame()` | Captures the current frame. |

Requirements

- React ≥ 16.8.0
- Node.js ≥ 14.0.0
- TypeScript ≥ 4.0.0 (for TypeScript users)
- MQTT server running along with AI backend service

License

This project is licensed under the MIT License - see the LICENSE file for details.

---

Contributing

Please read our contributing guidelines and code of conduct before submitting pull requests.

Support

For issues and feature requests, please use the GitHub issue tracker.