For video data, Kili Technology lets you decide if you want to import your data as native video files (Native Video projects) or extracted into lists of images (Frame projects). The labeling experience is very similar for both input types. Kili Technology allows to carry object detection with bounding boxes and points on Frame or Native video interface for video. It makes it easy to create, edit, modify objects on one frame/timestamp or across frames/timestamps. All combinations with classification and nested classification can be leveraged.
Two different intputs
Native Video project
For Native Video projects, videos are stored as video files. Inside the platform, the video is then sampled into timestamps separated by the same duration. The division is done such that there are 24 sampled timestamps within a second. Thus, annotation can be done for every 1/24th seconds of video
For Frame projects, videos are stored as lists of images. We extract images at the original frame rate of the video. You can then annotate frame by frame.
Native Video project
You can upload videos in a video input type project using a csv file with two columns : one for the external identifier, and the other containing a link to the video. For more information, see here.
You can also upload videos from the API.
In order to have a smooth interface, you should work on videos which contain less than 1000 frames. As performance also depends on the number of annotations, we recommend having a maximum of 10000 annotations accross all frames for each asset. You have three possibilities when it comes to importing frames in a frame project :
You can upload videos in mp4, mkv or any other format readable by ffmpeg. For more information, see here.
Uploading from a public cloud
You can upload videos using a csv file with two columns : one for the external identifier, and the other containing a link to the video. Videos are then converted to an array of images. For more information, see here.
Uploading a list of images from the API
We allow you to upload videos as lists of extracted images with the API (see the recipe). This can be useful for viewing 3D volumes of image data. We show you how to do that here.
This allows you to upload videos with up to 20000 images. If you want more, we advise to switch to the Native Video player.
Video or Frame play bar
The labeling interface is almost identical between frame projects and video projects. It includes a playbar below the video/frame display. The playbar is composed of
- a controller bar to control the video or to swith between frames.
- a speed selector. In FPS for Frame or in play rate speed for native Video.
- a frame bar (for Frame) or a timestamp bar (for Native Video)
The frame/timestamp bar allows you to display a specific frame/timestamp by clicking on it. When you hit pause on a video project, the video will be set to pause at the nearest sampled timestamp.
To select multiple frames/timestamps, select the start or the end frame/timestamp, hold Shift, and click on the final frame/timestamp. The whole section will be selected.
Create an object on one frame/timestamp
If you draw an object on one frame/timestamp with propagation activated, it is propagated to all following frames/timestamps, until the end of the video. To create an object only on one frame, deactivate the propagation on the tool bar by clicking on the propagation button.
Delete an object on one frame/timestamp
When you delete an object on a frame/timestamp, it will be deleted on the frame/timestamp and all following frames.
Create an object on multiple frame/timestamps
To create an object from the current frame to the end of the video, activate the propagation and draw the object on the current frame. To create an object on a specific range of frames/timestamps, select the start or the end frame/timestamp, click on Shift, and click on the final frame. The whole section will be selected. Then create the object on the section.
Edit an object on multiple frames
You can edit and modify an object on one frame/timestamp. Then:
- the modification will be applied to all following frames/timestamps if the propagation is activated
- the frame/timestamp will be turned into a "key frame/timestamp" and marked with a lock icon on the interface
When modified, bounding box and point annotations are interpolated between the preceding key frame/timestamp and the current key frame/timestamp, as well as between the current key frame/timestamp and the following key frame/timestamp. If there are no preceding/following key frame/timestamp, then there is no interpolation.
For bounding box jobs, you can use a tracker to help you. It automatically labels an object, by following it.
- Set it up by ticking the checkbox after creating the bounding box job, or add it to your existing job in the json interface.
- Use it in the application : click on a bounding box category, and press the tracking button before drawing the bounding box.
For Point, choose tool: Point.
For Bounding box, choose tool: Bounding box.
Shortcuts are provided to help you annotate quickly :
- A shortcut will be generated for each different class of object to detect : see shortcuts section
- On the frame interface, you can use PLAY and PAUSE buttons to play and pause the video, or their common shortcut SPACE.
- To scroll quickly in the frames, you can also use SPACE + wheel of the mouse (or 2 fingers on the pad).
- You also have zoom shortcuts available.
- You can also edit faster with this shortcut to correct the last segments drawn for the current object, or to escape the editing interface
For both Native Video and Frame projects, the exported data has the same structure. You can see this structure here. For a Frame project, the Json response is split into frames and for a Native Video project, the Json response is split into timestamps. For example, for a Native Video project, the annotation in the Json response at the key value 0 is the annotation at timestamp 00:00:00, then at the key 1 the timestamp 00:00:1/24 etc.