SYNSCAPES

Dataset layout, features and annotations

OVERVIEW

FEATURES

WHITE PAPER AND CODE

DOWNLOAD

 

How many cars are visible in a given image? Is the sky clear or cloudy? Synscapes provides a wide range of metadata which helps characterize each image.

Additionally, Synscapes offers information that is difficult for humans to annotate, such as accurate per-instance visibility.

dataset Layout

Synscapes is organized into the following directories:

|--  img
|   |-- class     [1-25000].png
|   |-- depth     [1-25000].exr
|   |-- instance  [1-25000].png
|   |-- rgb       [1-25000].png
|   |-- rgb-2k    [1-25000].png
|-- meta          [1-25000].json

Image resolution

Synscapes' native resolution is 1440x720, stored in the img/rgb folder. In order to best support training with architectures designed for Cityscapes, we also include an up-scaled version at 2048x1024 resolution in img/rgb-2k. Note that this up-scaling precedes the sensor simulation stage, ensuring pixel noise is present at the appropriate scale.
 

Annotations

RGB data

Class as single-channel PNG (visualized in color below)

The class annotations follows the Cityscapes convention.

Instance as PNG

The instance id can be found as R + G * 256 + B * 256^2.

Depth as floating point OpenEXR

Stores the planar depth (not distance) in meters.

 
 

Camera metadata

The camera's position and field of view is as follows:

"camera": {
    "extrinsic": {
      "pitch": 0.038,
      "roll": -0.0,
      "x": 1.7,
      "y": 0.1,
      "yaw": -0.0195,
      "z": 1.22
    },
    "intrinsic": {
      "fx": 1590.83437,
      "fy": 1592.79032,
      "resx": 1440,
      "resy": 720,
      "u0": 771.31406,
      "v0": 360.79945
    }
  }
 

Instance metadata

2D Bounding Boxes

3D Bounding Boxes in ego-vehicle coordinates

Occlusion (fraction of object hidden behind other objects)

Truncation (fraction of object outside field of view)

 

Note: 'class' is also recorded in the JSON file, to facilitate instance-to-class mapping without having to refer to the PNG file.
 

Scene Metadata

altitude_variation
The largest altitude difference in the scene in meters.

curb_height
The height of the sidewalk curb in meters.

dist_*_{mean,stddev}
For each actor class, contains the mean and standard deviation of distance for all visible instances.

ego_speed
The speed in m/s traveled by the ego vehicle at the time of image capture.

fence_{presence,height}
Indicates whether fences are present in the image. Note that due to occlusion, it may be hidden behind another object. Height is measured in meters.

median_presence
Whether the road median is present.

num_*
For each actor class, contains the number of visible instances.

parking_{presence,angle}
Whether a parking lane is present, and whether cars park at 0 (parallel), 45 or 90 degrees.

rel_dist_to_isect
Relative distance to nearest intersection. 0.0 indicates ego vehicle is inside the intersection, 1.0 indicates it is one city block away from the next intersection.

road_material_type
Integer representing the material used for the road surface.

sidewalk_width
The width of the sidewalk in meters

sky_contrast
Contains the logarithm of the sky's contrast, measured as max/mean. Values around 2-3 indicate fully overcast sky, 5-6 indicate direct sunlight.

sun_height
The normalized angular height of the sun. 0.0 indicates sunset/sunrise, 1.0 indicates zenith.

wall_{presence,height}
Whether the wall class is present, with height in meters.

 

Synscapes 2020