Pretrained hand pose detection model
npm install @tensorflow-models/hand-pose-detectionThis package provides models for running real-time hand pose detection.
Currently, we provide 1 model option:
#### MediaPipe:
Demo
MediaPipe Hands can detect multiple hands, each hand contains 21 3D hand keypoints.
More background information about the model, as well as its performance characteristics on different datasets, can be found here: Model Card
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
You first create a detector by choosing one of the models from SupportedModels,
including MediaPipeHands.
For example:
``javascript`
const model = handPoseDetection.SupportedModels.MediaPipeHands;
const detectorConfig = {
runtime: 'mediapipe', // or 'tfjs',
solutionPath: 'https://cdn.jsdelivr.net/npm/@mediapipe/hands',
modelType: 'full'
}
const detector = await handPoseDetection.createDetector(model, detectorConfig);
Then you can use the detector to detect hands.
`
const hands = await detector.estimateHands(image);
The returned hands list contains detected hands for each hand in the image.
If the model cannot detect any hands, the list will be empty.
For each hand, it contains a prediction of the handedness (left or right), a confidence score of this prediction, as well as an array of keypoints.
MediaPipeHands returns 21 keypoints.
Each keypoint contains x and y, as well as a name. In addition, an array of 3D keypoints is returned.
Example output:
`
[
{
score: 0.8,
handedness: ‘Right’,
keypoints: [
{x: 105, y: 107, name: "wrist"},
{x: 108, y: 160, name: "pinky_finger_tip"},
...
],
keypoints3D: [
{x: 0.00388, y: -0.0205, z: 0.0217, name: "wrist"},
{x: -0.025138, y: -0.0255, z: -0.0051, name: "pinky_finger_tip"},
...
]
}
]
The score ranges from 0 to 1. It represents the model's confidence of the detected hand.
handedness is set to either 'Left' or 'Right', which is the model prediction of the detected hand's handedness.
For the keypoints, x and y represent the actual keypoint position in the image pixel space.
For the keypoints3D, x, y and z represent absolute distance in a metric scale, where the origin is formed as an average between the first knuckles of index, middle, ring and pinky fingers.
The name provides a label for each keypoint, such as 'wrist', 'pinky_finger_tip', etc.
Refer to each model's documentation for specific configurations for the model
and their performance.
MediaPipeHands MediaPipe Documentation
MediaPipeHands TFJS Documentation
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------