Introduction
In this tutorial we will show you how to use sly.GraphNodes class to create data annotation for pose estimation / keypoints detection task. The tutorial illustrates basic upload-download scenario:
create project and dataset on server
programmatically create annotation and upload it to image
download image and annotation
ℹ️ Everything you need to reproduce this tutorial is on GitHub : source code, Visual Studio Code configuration, and a shell script for creating virtual env.
How to debug this tutorial
Step 1. Prepare ~/supervisely.env file with credentials. Learn more here.
Step 2. Clone repository with source code and demo data and create Virtual Environment .
Copy git clone https://github.com/supervisely-ecosystem/keypoints-labeling-example
cd keypoints-labeling-example
./create_venv.sh Step 3. Open repository directory in Visual Studio Code
Step 4. Start debugging src/main.py
Python Code
Importing Necessary Libraries
Import necessary libraries:
Copy import supervisely as sly
from supervisely.geometry.graph import Node, KeypointsTemplate
import os
import json
from dotenv import load_dotenv Before we will start creating our project, let's learn how to create keypoints template - we are going to use it in our project.
Working With Keypoints Template
We will need an image to create and visualize our keypoints template.
Image for building keypoints template:
Create keypoints template:
Copy # initialize template
template = KeypointsTemplate()
# add nodes
template.add_point(label="nose", row=635, col=427)
template.add_point(label="left_eye", row=597, col=404)
template.add_point(label="right_eye", row=685, col=401)
template.add_point(label="left_ear", row=575, col=431)
template.add_point(label="right_ear", row=723, col=425)
template.add_point(label="left_shoulder", row=502, col=614)
template.add_point(label="right_shoulder", row=794, col=621)
template.add_point(label="left_elbow", row=456, col=867)
template.add_point(label="right_elbow", row=837, col=874)
template.add_point(label="left_wrist", row=446, col=1066)
template.add_point(label="right_wrist", row=845, col=1073)
template.add_point(label="left_hip", row=557, col=1035)
template.add_point(label="right_hip", row=743, col=1043)
template.add_point(label="left_knee", row=541, col=1406)
template.add_point(label="right_knee", row=751, col=1421)
template.add_point(label="left_ankle", row=501, col=1760)
template.add_point(label="right_ankle", row=774, col=1765)
# add edges
template.add_edge(src="left_ankle", dst="left_knee")
template.add_edge(src="left_knee", dst="left_hip")
template.add_edge(src="right_ankle", dst="right_knee")
template.add_edge(src="right_knee", dst="right_hip")
template.add_edge(src="left_hip", dst="right_hip")
template.add_edge(src="left_shoulder", dst="left_hip")
template.add_edge(src="right_shoulder", dst="right_hip")
template.add_edge(src="left_shoulder", dst="right_shoulder")
template.add_edge(src="left_shoulder", dst="left_elbow")
template.add_edge(src="right_shoulder", dst="right_elbow")
template.add_edge(src="left_elbow", dst="left_wrist")
template.add_edge(src="right_elbow", dst="right_wrist")
template.add_edge(src="left_eye", dst="right_eye")
template.add_edge(src="nose", dst="left_eye")
template.add_edge(src="nose", dst="right_eye")
template.add_edge(src="left_eye", dst="left_ear")
template.add_edge(src="right_eye", dst="right_ear")
template.add_edge(src="left_ear", dst="left_shoulder")
template.add_edge(src="right_ear", dst="right_shoulder") Visualize your keypoints template:
Copy template_img = sly.image.read("images/girl.jpg")
template.draw(image=template_img, thickness=7)
sly.image.write("images/template.jpg", template_img) Explore Keypoints Template in JSON Format
You can also transfer your template to json:
Copy template_json = template.to_json()
Copy {
"nodes": {
"nose": {
"label": "nose",
"loc": [635, 427],
"color": "#0000FF"
},
"left_eye": {
"label": "left_eye",
"loc": [597, 404],
"color": "#0000FF"
},
"right_eye": {
"label": "right_eye",
"loc": [685, 401],
"color": "#0000FF"
},
"left_ear": {
"label": "left_ear",
"loc": [575, 431],
"color": "#0000FF"
},
"right_ear": {
"label": "right_ear",
"loc": [723, 425],
"color": "#0000FF"
},
"left_shoulder": {
"label": "left_shoulder",
"loc": [502, 614],
"color": "#0000FF"
},
"right_shoulder": {
"label": "right_shoulder",
"loc": [794, 621],
"color": "#0000FF"
},
"left_elbow": {
"label": "left_elbow",
"loc": [456, 867],
"color": "#0000FF"
},
"right_elbow": {
"label": "right_elbow",
"loc": [837, 874],
"color": "#0000FF"
},
"left_wrist": {
"label": "left_wrist",
"loc": [446, 1066],
"color": "#0000FF"
},
"right_wrist": {
"label": "right_wrist",
"loc": [845, 1073],
"color": "#0000FF"
},
"left_hip": {
"label": "left_hip",
"loc": [557, 1035],
"color": "#0000FF"
},
"right_hip": {
"label": "right_hip",
"loc": [743, 1043],
"color": "#0000FF"
},
"left_knee": {
"label": "left_knee",
"loc": [541, 1406],
"color": "#0000FF"
},
"right_knee": {
"label": "right_knee",
"loc": [751, 1421],
"color": "#0000FF"
},
"left_ankle": {
"label": "left_ankle",
"loc": [501, 1760],
"color": "#0000FF"
},
"right_ankle": {
"label": "right_ankle",
"loc": [774, 1765],
"color": "#0000FF"
}
},
"edges": [
{
"src": "left_ankle",
"dst": "left_knee",
"color": "#00FF00"
},
{
"src": "left_knee",
"dst": "left_hip",
"color": "#00FF00"
},
{
"src": "right_ankle",
"dst": "right_knee",
"color": "#00FF00"
},
{
"src": "right_knee",
"dst": "right_hip",
"color": "#00FF00"
},
{
"src": "left_hip",
"dst": "right_hip",
"color": "#00FF00"
},
{
"src": "left_shoulder",
"dst": "left_hip",
"color": "#00FF00"
},
{
"src": "right_shoulder",
"dst": "right_hip",
"color": "#00FF00"
},
{
"src": "left_shoulder",
"dst": "right_shoulder",
"color": "#00FF00"
},
{
"src": "left_shoulder",
"dst": "left_elbow",
"color": "#00FF00"
},
{
"src": "right_shoulder",
"dst": "right_elbow",
"color": "#00FF00"
},
{
"src": "left_elbow",
"dst": "left_wrist",
"color": "#00FF00"
},
{
"src": "right_elbow",
"dst": "right_wrist",
"color": "#00FF00"
},
{
"src": "left_eye",
"dst": "right_eye",
"color": "#00FF00"
},
{
"src": "nose",
"dst": "left_eye",
"color": "#00FF00"
},
{
"src": "nose",
"dst": "right_eye",
"color": "#00FF00"
},
{
"src": "left_eye",
"dst": "left_ear",
"color": "#00FF00"
},
{
"src": "right_eye",
"dst": "right_ear",
"color": "#00FF00"
},
{
"src": "left_ear",
"dst": "left_shoulder",
"color": "#00FF00"
},
{
"src": "right_ear",
"dst": "right_shoulder",
"color": "#00FF00"
}
]
}
Now, when we have successfully created keypoints template, we can start creating keypoints annotation for our project.
Programmatically Create Keypoints Annotation
Authenticate (learn more here ):
Copy load_dotenv(os.path.expanduser('~/supervisely.env'))
api = sly.Api.from_env()
my_teams = api.team.get_list()
team = my_teams[0]
workspace = api.workspace.get_list(team.id)[0] Input image:
Create project and dataset:
Copy project = api.project.create(workspace.id, "Human Pose Estimation", change_name_if_conflict=True)
dataset = api.dataset.create(project.id, "Person with dog", change_name_if_conflict=True)
print(f"Project {project.id} with dataset {dataset.id} are created") Now let's create annotation class using our keypoints template as a geometry config (unlike other supervisely geometry classes, sly.GraphNodes requires geometry config to be passed - it is necessary for object class initialization):
Copy person = sly.ObjClass("person", geometry_type=sly.GraphNodes, geometry_config=template)
project_meta = sly.ProjectMeta(obj_classes=[person])
api.project.update_meta(project.id, project_meta.to_json()) You can also go to Supervisely platform and check that class with shape "Keypoints" was successfully added to your project:
Upload image:
Copy image_info = api.image.upload_path(
dataset.id, name="person_with_dog.jpg", path="images/person_with_dog.jpg"
) Build keypoints graph:
Copy nodes = [
sly.Node(label="nose", row=146, col=670),
sly.Node(label="left_eye", row=130, col=644),
sly.Node(label="right_eye", row=135, col=701),
sly.Node(label="left_ear", row=137, col=642),
sly.Node(label="right_ear", row=142, col=705),
sly.Node(label="left_shoulder", row=221, col=595),
sly.Node(label="right_shoulder", row=226, col=738),
sly.Node(label="left_elbow", row=335, col=564),
sly.Node(label="right_elbow", row=342, col=765),
sly.Node(label="left_wrist", row=429, col=555),
sly.Node(label="right_wrist", row=438, col=784),
sly.Node(label="left_hip", row=448, col=620),
sly.Node(label="right_hip", row=451, col=713),
sly.Node(label="left_knee", row=598, col=591),
sly.Node(label="right_knee", row=602, col=715),
sly.Node(label="left_ankle", row=761, col=573),
sly.Node(label="right_ankle", row=766, col=709),
] Label the image:
Copy input_image = sly.image.read("images/person_with_dog.jpg")
img_height, img_width = input_image.shape[:2]
label = sly.Label(sly.GraphNodes(nodes), person)
ann = sly.Annotation(img_size=[img_height, img_width], labels=[label])
api.annotation.upload_ann(image_info.id, ann) You can check that keypoints annotation was successfully created in Annotation Tool:
Download data:
Copy image = api.image.download_np(image_info.id)
ann_json = api.annotation.download_json(image_info.id) Draw annotation:
Copy ann = sly.Annotation.from_json(ann_json, project_meta)
output_path = "images/person_with_dog_labelled.jpg"
ann.draw_pretty(image, output_path=output_path, thickness=7) Result:
