Point Cloud Episodes and object tags

How to create and add tags, update and remove tags from Point Cloud Episode annotation objects and frames

Introduction

In this tutorial, you will learn how to create new tags and assign them, update its values or remove tags for selected annotation objects or frames (with these objects) in Point Cloud Episodes using the Supervisely SDK.

Supervisely supports different types of tags:

• NONE

• ANY_NUMBER

• ANY_STRING

• ONEOF_STRING

And could be applied to:

• ALL

• IMAGES_ONLY - PCD in our case

• OBJECTS_ONLY

You can find all the information about those types in the Tags in Annotations section and SDK documentation.

You can learn more about working with Point Cloud Episodes (PCE) using Supervisely SDK and what Annotations for PCE are.

Everything you need to reproduce this tutorial is on GitHub: source code, Visual Studio Code configuration, and a shell script for creating virtual env.

How to debug this tutorial

Step 1. Prepare ~/supervisely.env file with credentials. Learn more here.

Step 2. Clone repository with source code and create Virtual Environment.

git clone https://github.com/supervisely-ecosystem/how-to-work-with-pce-object-tags
cd how-to-work-with-pce-object-tags
./create_venv.sh

Step 3. Open repository directory in Visual Studio Code.

code -r .

Step 4. Get, for example, Demo KITTI pointcloud episodes annotated project from Ecosystem.

There you see project classes after Demo initialization

Project tags metadata after Demo initialization. This data is empty.

Visualization in Labeling Tool before we add tags

Step 5. Change Workspace ID in local.env file by copying the ID from the context menu of the workspace. Do the same for Project ID and Dataset ID .

WORKSPACE_ID=82841  # ⬅️ change value
PROJECT_ID=239385  # ⬅️ change value
DATASET_ID=774629  # ⬅️ change value

Step 6. Start debugging src/main.py

Python Code

Import libraries

import os
import supervisely as sly
from dotenv import load_dotenv

Init API client

Init api for communicating with Supervisely Instance. First, we load environment variables with credentials, Project and Dataset IDs:

load_dotenv("local.env")
load_dotenv(os.path.expanduser("~/supervisely.env"))
api = sly.Api.from_env()

With next lines we will get values from local.env.

project_id = sly.env.project_id()
dataset_id = sly.env.dataset_id()

By using these IDs, we can retrieve the project metadata and annotations, and define the values needed for the following operations.

project_meta_json = api.project.get_meta(project_id)
project_meta = sly.ProjectMeta.from_json(data=project_meta_json)

key_id_map = sly.KeyIdMap()

pcd_ep_ann_json = api.pointcloud_episode.annotation.download(dataset_id)

Create new tag metadata

To create a new tag, you need to first define a tag metadata. This includes specifying the tag name, type, the objects to which it can be added, and the possible values. This base information will be used to create the actual tags.

tag_name = "Car"
tag_values = ["car_1", "car_2"]

if not project_meta.tag_metas.has_key(tag_name):
    new_tag_meta = sly.TagMeta(
        tag_name,
        sly.TagValueType.ONEOF_STRING,
        applicable_to=sly.TagApplicableTo.OBJECTS_ONLY,
        possible_values=tag_values,
    )

Then recreate the source project metadata with new tag metadata.

    new_tags_collection = project_meta.tag_metas.add(new_tag_meta)
    new_project_meta = sly.ProjectMeta(
        tag_metas=new_tags_collection, obj_classes=project_meta.obj_classes
    )
    api.project.update_meta(project_id, new_project_meta)

New tag metas added

Right after updating the metadata, we need to obtain added metadata on the previous step to get the IDs in the next steps.

    new_prject_meta_json = api.project.get_meta(project_id)
    new_project_meta = sly.ProjectMeta.from_json(data=new_prject_meta_json)
    new_tag_meta = new_project_meta.tag_metas.get(new_tag_meta.name)

Or use existing tag metadata

If a tag with the tag_name already exists in the metadata, we could just use it if it fits our requirements.

else:
    new_tag_meta = project_meta.tag_metas.get(tag_name)
    if sorted(new_tag_meta.possible_values) != sorted(tag_values):
        sly.logger.warning(
            f"Tag [{new_tag_meta.name}] already exists, but with another values: {new_tag_meta.possible_values}"
        )

In case this tag doesn't meet our requirements, it would be better to create a new one with a different name. On the other hand, we could update the tag values.

Create new tag with value and add to objects

When you pass information from tag metadata using its ID to the object, a new tag is created and appended.

If you want to add a tag with value, you can define the value argument with possible values.

If you want to add a tag to frames, you can define the frame_range argument.

project_objects = pcd_ep_ann_json.get("objects")
tag_frames = [0, 26]
created_tag_ids = {}

for object in project_objects:
    if object["classTitle"] == "Car":
        tag_id = api.pointcloud_episode.object.tag.add(
            new_tag_meta.sly_id, object["id"], value="car_1", frame_range=tag_frames
        )
        created_tag_ids[object["id"]] = tag_id

created_tag_ids uses to store IDs for the following operations.

Visualization in Labeling Tool with new tags

You could more precisely define tag_frames in your dataset using the following example:

replace line number 46 of source code with this:

project_frames = pcd_ep_ann_json.get("frames")

insert on line number 51 of source code this:

        frame_range = []
        for frame in project_frames:
            for figure in frame["figures"]:
                if figure["objectId"] == object["id"]:
                    frame_range.append(frame["index"])
        frame_range = frame_range[0:1] + frame_range[-1:]

You will most likely need to modify this example to more accurately define the objects. It is only provided to make it faster and easier to understand where and with what information to interact.

Update tag value

Also, if you need to correct tag values, you can easily do so as follows:

tag_id_to_operate = created_tag_ids.get(project_objects[0]["id"])

api.pointcloud_episode.object.tag.update(tag_id_to_operate, "car_2")

In our example, we took the first annotated object and the tag assigned to it in the previous step.

You can use a different approach to obtain information about objects, their tags, and the values of those tags according to your goal.

Delete tag

To remove a tag, all you need is its ID.

api.pointcloud_episode.object.tag.remove(tag_id_to_operate)

Please note that you are only deleting the tag from the object. To remove a tag from the project (TagMeta), you need to use other SDK methods.