Supervisely
About SuperviselyEcosystemContact usSlack
  • 💻Supervisely Developer Portal
  • 🎉Getting Started
    • Installation
    • Basics of authentication
    • Intro to Python SDK
    • Environment variables
    • Supervisely annotation format
      • Project Structure
      • Project Meta: Classes, Tags, Settings
      • Objects
      • Tags
      • Image Annotation
      • Video Annotation
      • Point Clouds Annotation
      • Point Cloud Episode Annotation
      • Volumes Annotation
    • Python SDK tutorials
      • Images
        • Images
        • Image and object tags
        • Spatial labels on images
        • Keypoints (skeletons)
        • Multispectral images
        • Multiview images
        • Advanced: Optimized Import
        • Advanced: Export
      • Videos
        • Videos
        • Video and object tags
        • Spatial labels on videos
      • Point Clouds
        • Point Clouds (LiDAR)
        • Point Cloud Episodes and object tags
        • 3D point cloud object segmentation based on sensor fusion and 2D mask guidance
        • 3D segmentation masks projection on 2D photo context image
      • Volumes
        • Volumes (DICOM)
        • Spatial labels on volumes
      • Common
        • Iterate over a project
        • Iterate over a local project
        • Progress Bar tqdm
        • Cloning projects for development
    • Command Line Interface (CLI)
      • Enterprise CLI Tool
        • Instance administration
        • Workflow automation
      • Supervisely SDK CLI
    • Connect your computer
      • Linux
      • Windows WSL
      • Troubleshooting
  • 🔥App development
    • Basics
      • Create app from any py-script
      • Configuration file
        • config.json
        • Example 1. Headless
        • Example 2. App with GUI
        • v1 - Legacy
          • Example 1. v1 Modal Window
          • Example 2. v1 app with GUI
      • Add private app
      • Add public app
      • App Compatibility
    • Apps with GUI
      • Hello World!
      • App in the Image Labeling Tool
      • App in the Video Labeling Tool
      • In-browser app in the Labeling Tool
    • Custom import app
      • Overview
      • From template - simple
      • From scratch - simple
      • From scratch GUI - advanced
      • Finding directories with specific markers
    • Custom export app
      • Overview
      • From template - simple
      • From scratch - advanced
    • Neural Network integration
      • Overview
      • Serving App
        • Introduction
        • Instance segmentation
        • Object detection
        • Semantic segmentation
        • Pose estimation
        • Point tracking
        • Object tracking
        • Mask tracking
        • Image matting
        • How to customize model inference
        • Example: Custom model inference with probability maps
      • Serving App with GUI
        • Introduction
        • How to use default GUI template
        • Default GUI template customization
        • How to create custom user interface
      • Inference API
      • Training App
        • Overview
        • Tensorboard template
        • Object detection
      • High level scheme
      • Custom inference pipeline
      • Train and predict automation model pipeline
    • Advanced
      • Advanced debugging
      • How to make your own widget
      • Tutorial - App Engine v1
        • Chapter 1 Headless
          • Part 1 — Hello world! [From your Python script to Supervisely APP]
          • Part 2 — Errors handling [Catching all bugs]
          • Part 3 — Site Packages [Customize your app]
          • Part 4 — SDK Preview [Lemons counter app]
          • Part 5 — Integrate custom tracker into Videos Annotator tool [OpenCV Tracker]
        • Chapter 2 Modal Window
          • Part 1 — Modal window [What is it?]
          • Part 2 — States and Widgets [Customize modal window]
        • Chapter 3 UI
          • Part 1 — While True Script [It's all what you need]
          • Part 2 — UI Rendering [Simplest UI Application]
          • Part 3 — APP Handlers [Handle Events and Errors]
          • Part 4 — State and Data [Mutable Fields]
          • Part 5 — Styling your app [Customizing the UI]
        • Chapter 4 Additionals
          • Part 1 — Remote Developing with PyCharm [Docker SSH Server]
      • Custom Configuration
        • Fixing SSL Certificate Errors in Supervisely
        • Fixing 400 HTTP errors when using HTTP instead of HTTPS
      • Autostart
      • Coordinate System
      • MLOps Workflow integration
    • Widgets
      • Input
        • Input
        • InputNumber
        • InputTag
        • BindedInputNumber
        • DatePicker
        • DateTimePicker
        • ColorPicker
        • TimePicker
        • ClassesMapping
        • ClassesColorMapping
      • Controls
        • Button
        • Checkbox
        • RadioGroup
        • Switch
        • Slider
        • TrainValSplits
        • FileStorageUpload
        • Timeline
        • Pagination
      • Text Elements
        • Text
        • TextArea
        • Editor
        • Copy to Clipboard
        • Markdown
        • Tooltip
        • ElementTag
        • ElementTagsList
      • Media
        • Image
        • LabeledImage
        • GridGallery
        • Video
        • VideoPlayer
        • ImagePairSequence
        • Icons
        • ObjectClassView
        • ObjectClassesList
        • ImageSlider
        • Carousel
        • TagMetaView
        • TagMetasList
        • ImageAnnotationPreview
        • ClassesMappingPreview
        • ClassesListPreview
        • TagsListPreview
        • MembersListPreview
      • Selection
        • Select
        • SelectTeam
        • SelectWorkspace
        • SelectProject
        • SelectDataset
        • SelectItem
        • SelectTagMeta
        • SelectAppSession
        • SelectString
        • Transfer
        • DestinationProject
        • TeamFilesSelector
        • FileViewer
        • Dropdown
        • Cascader
        • ClassesListSelector
        • TagsListSelector
        • MembersListSelector
        • TreeSelect
        • SelectCudaDevice
      • Thumbnails
        • ProjectThumbnail
        • DatasetThumbnail
        • VideoThumbnail
        • FolderThumbnail
        • FileThumbnail
      • Status Elements
        • Progress
        • NotificationBox
        • DoneLabel
        • DialogMessage
        • TaskLogs
        • Badge
        • ModelInfo
        • Rate
        • CircleProgress
      • Layouts and Containers
        • Card
        • Container
        • Empty
        • Field
        • Flexbox
        • Grid
        • Menu
        • OneOf
        • Sidebar
        • Stepper
        • RadioTabs
        • Tabs
        • TabsDynamic
        • ReloadableArea
        • Collapse
        • Dialog
        • IFrame
      • Tables
        • Table
        • ClassicTable
        • RadioTable
        • ClassesTable
        • RandomSplitsTable
        • FastTable
      • Charts and Plots
        • LineChart
        • GridChart
        • HeatmapChart
        • ApexChart
        • ConfusionMatrix
        • LinePlot
        • GridPlot
        • ScatterChart
        • TreemapChart
        • PieChart
      • Compare Data
        • MatchDatasets
        • MatchTagMetas
        • MatchObjClasses
        • ClassBalance
        • CompareAnnotations
      • Widgets demos on github
  • 😎Advanced user guide
    • Objects binding
    • Automate with Python SDK & API
      • Start and stop app
      • User management
      • Labeling Jobs
  • 🖥️UI widgets
    • Element UI library
    • Supervisely UI widgets
    • Apexcharts - modern & interactive charts
    • Plotly graphing library
  • 📚API References
    • REST API Reference
    • Python SDK Reference
Powered by GitBook
On this page
  • Introduction
  • How to debug this tutorial
  • Import libraries
  • Enter your Team ID and Workspace ID
  • Init API client
  • Create a new project and dataset
  • Set multispectral settings for the project
  • How to upload multispectral images
  • Upload an image as channels
  • Upload a multichannel tiff image as channels
  • Upload nrrd image as channels
  • Upload a pair of RGB and thermal images without splitting them into channels
  • Upload RGB image, its channels, and depth image
  • Upload grayscale and UV images
  • Upload RGB image, thermal image, and channels of thermal image
  • Upload RGB image and two MRI images
  • Grouped view in the labeling interface
  • Summary
  • (Advanced) Use supervisely format for multispectral images

Was this helpful?

Edit on GitHub
  1. Getting Started
  2. Python SDK tutorials
  3. Images

Multispectral images

PreviousKeypoints (skeletons)NextMultiview images

Last updated 3 months ago

Was this helpful?

Introduction

In this tutorial, you will learn how to import multispectral images to Supervisely using Python SDK and get the advantage of the grouped view in the labeling interface, which allows you to synchronize the view, zooming, panning, and labeling of images in one group.

You can also import multispectral images using app from Supervisely Ecosystem.

You will learn how to:

For the , you can use supervisely annotation JSON format to download and upload projects with multispectral images.

Everything you need to reproduce : source code and additional app files.

How to debug this tutorial

git clone https://github.com/supervisely-ecosystem/import-multispectral-images-tutorial.git

cd import-multispectral-images-tutorial

sh create_venv.sh

Step 3. Open the repository directory in Visual Studio Code.

code -r .

Step 4. Change the Team ID and Workspace ID in the main.py file by copying the ID from the context menu.

team_id = 448
workspace_id = 690

Step 5. Start debugging src/main.py.

Supervisely instance version >= 6.8.54 Supervisely SDK version >= 6.72.201\

In the tutorial, Supervisely Python SDK version is not directly defined in the requirements.txt. But when developing your app, we recommend defining the SDK version in the requirements.txt.

Import libraries

import cv2
import os
import tifffile
import supervisely as sly
import nrrd
from dotenv import load_dotenv

Enter your Team ID and Workspace ID

team_id = 448
workspace_id = 690

Init API client

load_dotenv(os.path.expanduser("~/supervisely.env"))
api = sly.Api.from_env()

Create a new project and dataset

project = api.project.create(workspace_id, "Multispectral images", change_name_if_conflict=True)
dataset = api.dataset.create(project.id, "ds0")

Set multispectral settings for the project

To enable grouping of images, including view, zooming, panning and labeling, you need to set multispectral settings for the project. You can do it with just one line of code:

api.project.set_multispectral_settings(project.id)

And now we're ready to upload images.

How to upload multispectral images

In this tutorial, we'll be using the api.image.upload_multispectral method to upload images to Supervisely.

def upload_multispectral(
        dataset_id: int,
        image_name: str,
        channels: Optional[List[np.ndarray]] = None,
        rgb_images: Optional[List[str]] = None,
    ) -> List[ImageInfo]:
Parameters
Type
Description

dataset_id

int

ID of the dataset to upload

image_name

str

Name of the image will be used as name of the images group

channels

Optional[List[np.ndarray]] = None

List of channels as 2d numpy arrays.

rgb_images

Optional[List[str]] = None

List of paths to RGB images.

So, the method uploads images (which can be passed as channels or RGB images) to Supervisely and returns a list of ImageInfo objects. RGB images as paths or channels as NumPy arrays can be passed to the method or both at the same time. The result will be a group of images in both cases.

Below, you will find examples of different ways to load multispectral data. However, it is important to note that you can group the images in any way you like: split them into channels, load them as a whole, or both at the same time. It is entirely up to you.

Upload an image as channels

Input: 1 RGB image in PNG format. Output: 3 images in a group with the name demo1.png in Supervisely.\

image_name = "demo1.png"
image = cv2.imread(f"demo_data/{image_name}")

# Extract channels as 2d numpy arrays: channels = [a, b, c]
channels = [image[:, :, i] for i in range(image.shape[2])]

image_infos = api.image.upload_multispectral(dataset.id, image_name, channels)

We'll do this operation for other cases too, so let's take a closer look at the code:

  1. The image_name will be used as a group name in the labeling interface and it can be any string.

  2. Then we read the image from the disk using OpenCV.

  3. Now we're splitting the image into channels, and preparing a list of channels as NumPy arrays.

  4. Finally, we upload the channels to Supervisely using the api.image.upload_multispectral method.

So, those are the steps we'll be doing for all the other cases.

Upload a multichannel tiff image as channels

Input: 1 multichannel tiff image. Output: 7 images in a group with the name demo2.tif in Supervisely.\

image_name = "demo2.tif"
image = tifffile.imread(f"demo_data/{image_name}")

# Extract channels as 2d numpy arrays: channels = [a, b, c, d, e, f]
channels = [image[:, :, i] for i in range(image.shape[2])]

image_infos = api.image.upload_multispectral(dataset.id, image_name, channels)

Upload nrrd image as channels

In this tutorial, we'll be uploading the channels of nrrd image as separate images. But Supervisely supports high-dimensional nrrd images, so you can upload them as is without splitting them into channels.

Input: 1 nrrd image. Output: 7 images in a group with the name demo3.nrrd in Supervisely.\

image_name = "demo3.nrrd"
image, header = nrrd.read(f"demo_data/{image_name}")

# Extract channels as 2d numpy arrays: channels = [a, b, c, d, e, f]
channels = [image[:, :, i] for i in range(image.shape[2])]

image_infos = api.image.upload_multispectral(dataset.id, image_name, channels)

Upload a pair of RGB and thermal images without splitting them into channels

Input: 1 RGB image and 1 thermal image in PNG format. Output: 2 images in a group with the name demo4.png in Supervisely.\

image_name = "demo4.png"
images = ["demo_data/demo4-rgb.png", "demo_data/demo4-thermal.png"]

image_infos = api.image.upload_multispectral(dataset.id, image_name, rgb_images=images)

As you can see, in this case, we don't extract any channels since we need to upload only images, not channels. So, we pass the list of image paths to the rgb_images parameter.

Upload RGB image, its channels, and depth image

Input: 1 RGB image and 1 depth image in PNG format. Output: 5 images in a group with the name demo5.png in Supervisely.\

image_name = "demo5.png"
images = ["demo_data/demo5-rgb.png", "demo_data/demo5-depths.png"]

image = cv2.imread(images[0])

# Extract channels as 2d numpy arrays: channels = [a, b, c]
channels = [image[:, :, i] for i in range(image.shape[2])]

image_infos = api.image.upload_multispectral(dataset.id, image_name, channels, images)

Here, we uploaded one image both as channels and as an image. So, we pass the list of image paths to the rgb_images parameter and the list of channels to the channels parameter.

Upload grayscale and UV images

Input: 1 grayscale image and 1 UV image in PNG format. Output: 2 images in a group with the name demo6.png in Supervisely.\

image_name = "demo6.png"
images = ["demo_data/demo6-grayscale.png", "demo_data/demo6-uv.png"]

image_infos = api.image.upload_multispectral(dataset.id, image_name, rgb_images=images)

Upload RGB image, thermal image, and channels of thermal image

Input: 1 RGB image and 1 thermal image in PNG format. Output: 5 images in a group with the name demo7.png in Supervisely.\

image_name = "demo7.png"
images = ["demo_data/demo7-rgb.png", "demo_data/demo7-thermal.png"]

image = cv2.imread(images[1])

# Extract channels as 2d numpy arrays: channels = [a, b, c]
channels = [image[:, :, i] for i in range(image.shape[2])]

image_infos = api.image.upload_multispectral(dataset.id, image_name, channels, images)

Upload RGB image and two MRI images

Input: 1 RGB image and 2 MRI images in PNG format. Output: 3 images in a group with the name demo8.png in Supervisely.\

image_name = "demo8.png"
images = ["demo_data/demo8-rgb.png", "demo_data/demo8-mri1.png", "demo_data/demo8-mri2.png"]

image_infos = api.image.upload_multispectral(dataset.id, image_name, rgb_images=images)

Grouped view in the labeling interface

So now, that we've uploaded all the images, let's take a look at the labeling interface.

As you can see, all the images are grouped by the name of the group, which is the name of the image we passed to the image_name parameter. We can zoom, pan, and label images in one group at the same time. So, whenever you create a label on one image, it will be automatically created on all the other images in the group. You can edit label on another image in the group, and it will be automatically updated on all the other images in the group. Just a reminder: we set the multispectral settings for the project at the beginning of the tutorial with the api.project.set_multispectral_settings method, which enables this grouped view.

Summary

In this tutorial, you learned how to upload multispectral images to Supervisely using Python SDK and get the advantage of the grouped view in the labeling interface, which allows you to synchronize the view, zooming, panning, and labeling of images in one group. Let's recap the steps we did:

  1. Create a new project and dataset.

  2. Set multispectral settings for the project using the api.project.set_multispectral_settings method.

  3. Upload images using the api.image.upload_multispectral method.

And that's it! Now you can upload your multispectral images to Supervisely using Python SDK.

(Advanced) Use supervisely format for multispectral images

{
  "classes": [
    {
      "title": "leaf",
      "shape": "bitmap",
      "color": "#FB00ED",
      "geometry_config": {},
      "id": 6509759,
      "hotkey": ""
    }
  ],
  "tags": [
    {
      "name": "im_id",
      "value_type": "any_number",
      "color": "#0F8A2D",
      "id": 27855,
      "hotkey": "",
      "applicable_type": "all",
      "classes": []
    },
    {
      "name": "band",
      "value_type": "any_string",
      "color": "#9F7A2F",
      "id": 27856,
      "hotkey": "",
      "applicable_type": "all",
      "classes": []
    }    
  ],
  "projectType": "images",
  "projectSettings": {
    "multiView": {
      "enabled": true,
      "tagName": "im_id",
      "tagId": 27855,
      "isSynced": false
    }
  }
}

This is very easy to see, if you want to group your images by the band tag instead of im_id (sync mode on), simply change your projectSettings this way:

  "projectSettings": {
    "multiView": {
      "enabled": true,
      "tagName": "band",
      "tagId": 27856,
      "isSynced": true
    }
  }

To download and upload a project using Supervisely SDK, use the following code:

import supervisely as sly
from tqdm import tqdm

load_dotenv(os.path.expanduser("~/supervisely.env"))
api = sly.Api.from_env()
# id of your workspace and project from environmental variables
# see https://developer.supervisely.com/getting-started/environment-variables 
WORKSPACE_ID = sly.env.workspace_id()
PROJECT_ID = sly.env.project_id()

your_dir = "/your/multiview/project/dir"

project_info = api.project.get_info_by_id(PROJECT_ID)
pbar = tqdm(desc="Download Project", total=project_info.items_count)
sly.download_project(api, PROJECT_ID, your_dir, progress_cb=pbar)

project_fs = sly.read_project(your_dir)
pbar = tqdm(desc="Upload Project", total=project_fs.total_items)
sly.upload_project(project_fs.directory, api, WORKSPACE_ID, progress_cb=pbar)

Step 1. Prepare ~/supervisely.env file with credentials.

Step 2. Clone the with source code and demo data and create a .

RGB image as channels

The method returns a list of ImageInfo objects, which contain information about the uploaded images. Learn more about ImageInfo .

Multichannel tiff image as channels
Nrrd image as channels
RGB and thermal images without splitting them into channels
RGB image, its channels and depth image
Grayscale and UV images
RGB image, thermal image and channels of thermal image
RGB image and two MRI images
Grouped view in the labeling interface

You can always use to download to the local directory of your favorite multispectral project with the preserved multiview settings and then easily upload it as a new project to the

From the developer's point of view, the gives you easy access to the necessary parameters while grouping the images. To feel the power of this instrument, let's imagine the situation when you have already downloaded the project and opened the meta.json file:

Moreover, you can additionally enhance your tags with hotkey, or specify the tag classes. See the explanation of every field .

🎉
Learn more here.
repository
Virtual Environment
here
Export to Supervisely format
Import images in Supervisely format
Supervisely annotation JSON format
here
Import Multispectral Images
this tutorial is on GitHub
Upload an image as channels
Upload a multichannel tiff image as channels
Upload nrrd image as channels
Upload a pair of RGB and thermal images without splitting them into channels
Upload RGB image, its channels and depth image
Upload grayscale and UV images
Upload RGB image, thermal image and channels of thermal image
Upload RGB image and two MRI images
advanced level