Open Images V6

Overview

Open Images is a dataset of around 9 million images annotated with image-level labels, object bounding boxes, object segmentation masks, visual relationships, and localized narratives. The dataset contains:

• 16 million bounding boxes for 600 object classes on 1.9 million images, making it the largest existing dataset with object location annotations;
• over 2.7 million instance segmentation on 350 categories;
• over 3.2 million relationship annotations on 1446 relationships;
• over 500 000 localized narratives;
• over 59 million image-level labels on over 19 000 categories.

The boxes have been largely manually drawn by professional annotators to ensure accuracy and consistency. The images are very diverse and often contain complex scenes with several objects (8.3 per image on average). Open Images also offers visual relationship annotations, indicating pairs of objects in particular relations (e.g. "woman playing guitar", "beer on table"), object properties (e.g. "table is wooden"), and human actions (e.g. "woman is jumping").

The authors believe that having a single dataset with unified annotations for image classification, object detection, visual relationship detection, instance segmentation, and multimodal image descriptions will enable to study these tasks jointly and stimulate progress towards genuine scene understanding.

The extended version of the dataset contians an additional 478 000 crowdsourced images with 6000 categories.

Associated Paper or Article

For more information on images themselves, please read The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale.

For more information on the annotation methodology, please read Large-scale interactive object segmentation with human annotators.

For more information on local narratives, please read Connecting Vision and Language with Localized Narratives.

Annotations

1. The bounding box annotations have the following structure:

2. The image segmentation masks are black and white images corresponding to the training images, in which the object appears as white.

3. The relationship annotations have the following structure:

4. The localized narratives have the following structure:

Download

You can download the dataset here.

Model

No official model has been provided for this dataset. However, you can check the models from the 2018 and 2019 competition leaderboards.

Benchmarks

No official benchmark has been provided for this dataset. However, you can check the leaderboards from the 2018 and 2019 competitions.

Associated Challenges

There are several challenges associated with this dataset: Open Images Challenge 2018, Open Images Challenge 2019 and Robust Vision Challenge 2020.

The challenges focus on object detection, visual relationship detection and instance segmentation.

License

Dataset licensed under the CC BY 4.0 license.