Supervisely
About SuperviselyEcosystemContact usSlack
  • 💻Supervisely Developer Portal
  • 🎉Getting Started
    • Installation
    • Basics of authentication
    • Intro to Python SDK
    • Environment variables
    • Supervisely annotation format
      • Project Structure
      • Project Meta: Classes, Tags, Settings
      • Objects
      • Tags
      • Image Annotation
      • Video Annotation
      • Point Clouds Annotation
      • Point Cloud Episode Annotation
      • Volumes Annotation
    • Python SDK tutorials
      • Images
        • Images
        • Image and object tags
        • Spatial labels on images
        • Keypoints (skeletons)
        • Multispectral images
        • Multiview images
        • Advanced: Optimized Import
        • Advanced: Export
      • Videos
        • Videos
        • Video and object tags
        • Spatial labels on videos
      • Point Clouds
        • Point Clouds (LiDAR)
        • Point Cloud Episodes and object tags
        • 3D point cloud object segmentation based on sensor fusion and 2D mask guidance
        • 3D segmentation masks projection on 2D photo context image
      • Volumes
        • Volumes (DICOM)
        • Spatial labels on volumes
      • Common
        • Iterate over a project
        • Iterate over a local project
        • Progress Bar tqdm
        • Cloning projects for development
    • Command Line Interface (CLI)
      • Enterprise CLI Tool
        • Instance administration
        • Workflow automation
      • Supervisely SDK CLI
    • Connect your computer
      • Linux
      • Windows WSL
      • Troubleshooting
  • 🔥App development
    • Basics
      • Create app from any py-script
      • Configuration file
        • config.json
        • Example 1. Headless
        • Example 2. App with GUI
        • v1 - Legacy
          • Example 1. v1 Modal Window
          • Example 2. v1 app with GUI
      • Add private app
      • Add public app
      • App Compatibility
    • Apps with GUI
      • Hello World!
      • App in the Image Labeling Tool
      • App in the Video Labeling Tool
      • In-browser app in the Labeling Tool
    • Custom import app
      • Overview
      • From template - simple
      • From scratch - simple
      • From scratch GUI - advanced
      • Finding directories with specific markers
    • Custom export app
      • Overview
      • From template - simple
      • From scratch - advanced
    • Neural Network integration
      • Overview
      • Serving App
        • Introduction
        • Instance segmentation
        • Object detection
        • Semantic segmentation
        • Pose estimation
        • Point tracking
        • Object tracking
        • Mask tracking
        • Image matting
        • How to customize model inference
        • Example: Custom model inference with probability maps
      • Serving App with GUI
        • Introduction
        • How to use default GUI template
        • Default GUI template customization
        • How to create custom user interface
      • Inference API
      • Training App
        • Overview
        • Tensorboard template
        • Object detection
      • High level scheme
      • Custom inference pipeline
      • Train and predict automation model pipeline
    • Advanced
      • Advanced debugging
      • How to make your own widget
      • Tutorial - App Engine v1
        • Chapter 1 Headless
          • Part 1 — Hello world! [From your Python script to Supervisely APP]
          • Part 2 — Errors handling [Catching all bugs]
          • Part 3 — Site Packages [Customize your app]
          • Part 4 — SDK Preview [Lemons counter app]
          • Part 5 — Integrate custom tracker into Videos Annotator tool [OpenCV Tracker]
        • Chapter 2 Modal Window
          • Part 1 — Modal window [What is it?]
          • Part 2 — States and Widgets [Customize modal window]
        • Chapter 3 UI
          • Part 1 — While True Script [It's all what you need]
          • Part 2 — UI Rendering [Simplest UI Application]
          • Part 3 — APP Handlers [Handle Events and Errors]
          • Part 4 — State and Data [Mutable Fields]
          • Part 5 — Styling your app [Customizing the UI]
        • Chapter 4 Additionals
          • Part 1 — Remote Developing with PyCharm [Docker SSH Server]
      • Custom Configuration
        • Fixing SSL Certificate Errors in Supervisely
        • Fixing 400 HTTP errors when using HTTP instead of HTTPS
      • Autostart
      • Coordinate System
      • MLOps Workflow integration
    • Widgets
      • Input
        • Input
        • InputNumber
        • InputTag
        • BindedInputNumber
        • DatePicker
        • DateTimePicker
        • ColorPicker
        • TimePicker
        • ClassesMapping
        • ClassesColorMapping
      • Controls
        • Button
        • Checkbox
        • RadioGroup
        • Switch
        • Slider
        • TrainValSplits
        • FileStorageUpload
        • Timeline
        • Pagination
      • Text Elements
        • Text
        • TextArea
        • Editor
        • Copy to Clipboard
        • Markdown
        • Tooltip
        • ElementTag
        • ElementTagsList
      • Media
        • Image
        • LabeledImage
        • GridGallery
        • Video
        • VideoPlayer
        • ImagePairSequence
        • Icons
        • ObjectClassView
        • ObjectClassesList
        • ImageSlider
        • Carousel
        • TagMetaView
        • TagMetasList
        • ImageAnnotationPreview
        • ClassesMappingPreview
        • ClassesListPreview
        • TagsListPreview
        • MembersListPreview
      • Selection
        • Select
        • SelectTeam
        • SelectWorkspace
        • SelectProject
        • SelectDataset
        • SelectItem
        • SelectTagMeta
        • SelectAppSession
        • SelectString
        • Transfer
        • DestinationProject
        • TeamFilesSelector
        • FileViewer
        • Dropdown
        • Cascader
        • ClassesListSelector
        • TagsListSelector
        • MembersListSelector
        • TreeSelect
        • SelectCudaDevice
      • Thumbnails
        • ProjectThumbnail
        • DatasetThumbnail
        • VideoThumbnail
        • FolderThumbnail
        • FileThumbnail
      • Status Elements
        • Progress
        • NotificationBox
        • DoneLabel
        • DialogMessage
        • TaskLogs
        • Badge
        • ModelInfo
        • Rate
        • CircleProgress
      • Layouts and Containers
        • Card
        • Container
        • Empty
        • Field
        • Flexbox
        • Grid
        • Menu
        • OneOf
        • Sidebar
        • Stepper
        • RadioTabs
        • Tabs
        • TabsDynamic
        • ReloadableArea
        • Collapse
        • Dialog
        • IFrame
      • Tables
        • Table
        • ClassicTable
        • RadioTable
        • ClassesTable
        • RandomSplitsTable
        • FastTable
      • Charts and Plots
        • LineChart
        • GridChart
        • HeatmapChart
        • ApexChart
        • ConfusionMatrix
        • LinePlot
        • GridPlot
        • ScatterChart
        • TreemapChart
        • PieChart
      • Compare Data
        • MatchDatasets
        • MatchTagMetas
        • MatchObjClasses
        • ClassBalance
        • CompareAnnotations
      • Widgets demos on github
  • 😎Advanced user guide
    • Objects binding
    • Automate with Python SDK & API
      • Start and stop app
      • User management
      • Labeling Jobs
  • 🖥️UI widgets
    • Element UI library
    • Supervisely UI widgets
    • Apexcharts - modern & interactive charts
    • Plotly graphing library
  • 📚API References
    • REST API Reference
    • Python SDK Reference
Powered by GitBook
On this page
  • Introduction
  • How to debug this tutorial
  • Import libraries
  • Init API client
  • Get variables from environment
  • Create new project and dataset
  • Upload videos from local directory to Supervisely
  • Upload single video.
  • Upload list of videos.
  • Get information about videos
  • Single video.
  • Get all videos from dataset.
  • Download video
  • Get video metadata
  • Get video metadata from file
  • Get video metadata from server
  • Download video frames as images
  • Download video frames as RGB NumPy matrix
  • Remove videos from Supervisely
  • Remove one video.
  • Remove list of videos.
  • Information about available codecs, extensions, and containers.
  • How to exract frames from videos correctly using the OpenCV library.

Was this helpful?

Edit on GitHub
  1. Getting Started
  2. Python SDK tutorials
  3. Videos

Videos

PreviousVideosNextVideo and object tags

Last updated 2 years ago

Was this helpful?

Introduction

In this tutorial we will focus on working with videos using Supervisely SDK.

You will learn how to:

📗 Everything you need to reproduce : source code and demo data.

How to debug this tutorial

Step 1. Prepare ~/supervisely.env file with credentials.

git clone https://github.com/supervisely-ecosystem/tutorial-video.git

cd tutorial-video

./create_venv.sh

Step 3. Open repository directory in Visual Studio Code.

code -r .

Step 4. Change workspace ID in local.env file by copying the ID from the context menu of the workspace.

WORKSPACE_ID=654 # ⬅️ change value

Step 5. Start debugging src/main.py.

Import libraries

import os
from dotenv import load_dotenv
from pprint import pprint
import supervisely as sly

Init API client

First, we load environment variables with credentials and init API for communicating with Supervisely Instance.

if sly.is_development():
    load_dotenv("local.env")
    load_dotenv(os.path.expanduser("~/supervisely.env"))
api = sly.Api()

Get variables from environment

workspace_id = sly.env.workspace_id()

Create new project and dataset

Create new project.

Source code:

project = api.project.create(
    workspace_id, "Animals", type=ProjectType.VIDEOS, change_name_if_conflict=True
)

print(f"Project ID: {project.id}")

Output:

# Project ID: 15599

Create new dataset.

Source code:

dataset = api.dataset.create(project.id, "Birds")

print(f"Dataset ID: {dataset.id}")

Output:

# Dataset ID: 53465

Upload videos from local directory to Supervisely

Upload single video.

Source code:

original_dir = "src/videos/original"
path = os.path.join(original_dir, "Penguins.mp4")

video = api.video.upload_path(
    dataset.id,
    name="Penguins",
    path=path
)

print(f'Video "{video.name}" uploaded to Supervisely with ID:{video.id}')

Output:

# Video "Penguins" uploaded to Supervisely platform with ID:17539140

Upload list of videos.

✅ Supervisely API allows uploading multiple videos in a single request. The code sample below sends fewer requests and it leads to a significant speed-up of our original code.

Source code:

names = ["Flamingo.mp4", "Swans.mp4", "Toucan.mp4"]
paths = [os.path.join(original_dir, name) for name in names]

upload_info = api.video.upload_paths(dataset.id, names, paths)

print(f"{len(upload_info)} videos successfully uploaded to  Supervisely platform")

Output:

# 3 videos successfully uploaded to Supervisely platform

Get information about videos

Single video.

Get information about video from Supervisely by id.

Source code:

video_info = api.video.get_info_by_id(video.id)

print(video_info)

Output:

# VideoInfo(
#     id=17539140,
#     name="Penguins",
#     hash="6aTUVnuyfGqIuMxH8l1t1yvkmn/9iuWbHUOE7iebhYk=",
#     link=None,
#     team_id=435,
#     workspace_id=654,
#     project_id=15748,
#     dataset_id=53751,
#     path_original="/h5un6l2bnaz1vj8a9qgms4-public/videos/U/O/LF/T7tiPVoBUyFaM.mp4",
#     frames_to_timecodes=[],
#     frames_count=413,
#     frame_width=640,
#     frame_height=360,
#     created_at="2022-12-21T23:09:54.191Z",
#     updated_at="2022-12-21T23:09:54.191Z",
#     tags=[],
#     file_meta={},
#     custom_data={},
#     processing_path="1/1703793",
# )

You can also get information about video from Supervisely by name.

Source code:

video_info_by_name = api.video.get_info_by_name(dataset.id, video.name)

print(f"Video name - '{video_info_by_name.name}'")

Output:

# Video name - 'Penguins'

Get all videos from dataset.

Get information about video from Supervisely by id.

Source code:

video_info_list = api.video.get_list(dataset.id)

print(f"{len(video_info_list)} videos information received.")

Output:

# 4 videos information received.

Download video

Download video from Supervisely to local directory by id. Source code:

save_path = os.path.join(result_dir, f"{video_info.name}.mp4")

api.video.download_path(video_info.id, save_path)

print(f"Video has been successfully downloaded to '{save_path}'")

Output:

# Video has been successfully downloaded to 'src/videos/result/Penguins.mp4'

Get video metadata

Get video metadata from file

Source code:

video_path = "src/videos/result/Penguins.mp4"
file_info = sly.video.get_info(video_path)
pprint(file_info)

Output:

{
  "duration": 16.52,
  "formatName": "mov,mp4,m4a,3gp,3g2,mj2",
  "size": "1599101",
  "streams": [
    {
      "codecName": "h264",
      "codecType": "video",
      "duration": 16.52,
      "framesCount": 413,
      "framesToTimecodes": [],
      "height": 360,
      "index": 0,
      "rotation": 0,
      "startTime": 0,
      "width": 640
    }
  ]
}

Get video metadata from server

Source code:

video_info = api.video.get_info_by_id(video.id)
pprint(video_info.file_meta)

Output:

{
  "codecName": "h264",
  "codecType": "video",
  "duration": 16.52,
  "formatName": "mov,mp4,m4a,3gp,3g2,mj2",
  "framesCount": 413,
  "framesToTimecodes": [],
  "height": 360,
  "index": 0,
  "mime": "video/mp4",
  "rotation": 0,
  "size": "1599101",
  "startTime": 0,
  "streams": [],
  "width": 640
}

Download video frames as images

Download single frame of video as image from Supervisely to local directory.

Source code:

frame_idx = 15
file_name = "frame.png"
save_path = os.path.join(result_dir, file_name)

api.video.frame.download_path(video_info.id, frame_idx, save_path)

print(f"Video frame has been successfully downloaded as image to '{save_path}'")

Output:

# Video frame has been successfully downloaded as image to 'src/videos/result/frame.png'

Download multiple frames in a single requests from Supervisely and save as images to local directory.

Source code:

frame_indexes = [5, 10, 20, 30, 45]
save_paths = [os.path.join(result_dir, f"frame_{idx}.png") for idx in frame_indexes]

api.video.frame.download_paths(video_info.id, frame_indexes, save_paths)

print(f"{len(frame_indexes)} images has been successfully downloaded to '{save_path}'")

Output:

# 5 images has been successfully downloaded to 'src/videos/result/frame.png'

Download video frames as RGB NumPy matrix

You can also download video frame as RGB NumPy matrix.

Source code:

video_frame_np = api.video.frame.download_np(video_info.id, frame_idx)
print(f"Video frame downloaded as RGB NumPy matrix. Frame shape: {video_frame_np.shape}")

Output:

# Video frame downloaded as RGB NumPy matrix. Frame shape: (360, 640, 3)

Download multiple frames in a single requests from Supervisely as RGB NumPy matrix.

Source code:

video_frames_np = api.video.frame.download_nps(video_info.id, frame_indexes)

print(f"{len(video_frames_np)} video frames downloaded in RGB NumPy matrix.")

Output:

# 5 video frames downloaded as RGB NumPy matrix.

Remove videos from Supervisely

Remove one video.

Remove video from Supervisely by id

Source code:

api.video.remove(video_info.id)
print(f"Video (ID: {video_info.id}) successfully removed.")

Output:

# Video (ID: 17536607) has been successfully removed.

Remove list of videos.

Remove list of videos from Supervisely by ids.

Source code:

videos_to_remove = api.video.get_list(dataset.id)
remove_ids = [video.id for video in videos_to_remove]
api.video.remove_batch(remove_ids)
print(f"{len(videos_to_remove)} videos successfully removed.")

Output:

# 3 videos have been successfully removed.

Information about available codecs, extensions, and containers.

Note: Only basic video codecs are available in the Community Edition, for additional video codecs you can try the Enterprise Edition.

The Enterprise Edition allows you to use the full range of extensions, containers and codecs listed below without any limits.

  • extensions: .avi, .mp4, .3gp, .flv, .webm, .wmv, .mov, .mkv

  • containers: mp4, webm, ogg, ogv

  • codecs: h264, vp8, vp9

In the Community Edition, it is recommended to use vp9, h264 codecs with mp4 container.

How to exract frames from videos correctly using the OpenCV library.

cap = cv2.VideoCapture(filepath)

# set CAP_PROP_ORIENTATION_AUTO flag for VideoCapture
cap.set(cv2.CAP_PROP_ORIENTATION_AUTO, 1)

while cap.isOpened():
    success, frame = cap.read()
    if not success:
        print("Can't receive frame (stream end?). Exiting ...")
        break
    # cv2.imshow(f"{frame.shape[:2]}", frame)
    if cv2.waitKey(1) & 0xFF == ord("q"):
        break

# Release everything if job is finished
cap.release()
# cv2.destroyAllWindows()

Step 2. Clone with source code and demo data and create .

In this tutorial, you will need an workspace ID that you can get from environment variables.

In case you need to exract frames from videos, you should be aware of one important detail of the OpenCV library. According to , different versions of the OpenCV library have different values of the CAP_PROP_ORIENTATION_AUTO flag. This may cause that VideoCapture to ignore video orientation metadata. To avoid incorrect extraction of frames from videos, it is recommended to directly define flag CAP_PROP_ORIENTATION_AUTO:

🎉
repository
Virtual Environment
issue #15499
this tutorial is on GitHub
Learn more here.
upload one or more videos to Supervisely dataset.
get information about videos by id or name.
download video from Supervisely.
get video metadata
download one or more frames of video and save to local directory as images.
download one or more frames of video as RGB NumPy matrix.
remove videos from Supervisely.
choose from the available codecs, extensions and containers.
exract frames from videos correctly using the OpenCV library.
Learn more here