1. Preparation

Start by creating a LOOK folder.

Download the LOOK annotation file here and copy it inside the LOOK folder.

2. Download the images

The LOOK dataset is made of images from 3 different existing datasets where we annotated the pedestrians as looking (1) or not (0) at the camera.

Nuscenes

Nuscenes dataset contains mainly high resolution images taken from different places (US, Asia). Some scenarios are very crowded and common for self-driving cars.

nuScenes contains 1600x900 .jpg images.

Go to Nuscenes official website and download the samples and the CAM_BACK_LEFT sweeps from the server closest to your location (US or ASIA). Create a Nuscenesfolder inside the LOOK folder and extract the downloaded files there.

JRDB

JRDB is a dataset collected from a social mobile manipulator JackRabbot from Stanford University. The goal of JRDB is to provide a new source of data and a test-bench for research in the areas of autonomous robot navigation and all perceptual tasks related to social robotics in human environments.

JRDB contains 752x480 .jpg images.

Create a folder JRDB inside LOOK. Download the JRDB train dataset from here and extract it in the JRDB folder you just created.

KITTI

KITTI datset is captured by driving around the mid-size city of Karlsruhe, in rural areas and on highways. It is a common dataset and bechmark used for various tasks (3D object detection, depth estimation, ..)

KITTI contains .png images at different resolutions.

Inside the LOOKfolder, create a KITTI folder.

Training data

Create an account at the KITTI Benchmark website and move to the raw data section. Download the 2011_09_29_drive_0071 [synced+rectified data] folder.

Unzip the folder and copy the 2011_09_29/2011_09_29_drive_0071_sync/image_03/data folder into the KITTI folder you created. Rename it train.

Testing data

Move to the 2d Object detection benchmark here. Download the first folder called Download left color images of object data set (12 GB). Unzip the folder and copy the folder data_object_image_2/training/image_2 into the KITTI folder. Rename it test.

STIP

Follow the instructions from the STIP official website to get the STIP images.
To get the STIP annotations download everything from this link

3. Filter the dataset

To keep only the files with a LOOK annotation, you need to filter the datasets you previously downloaded. To do so, you can download the extract_look_dataset.py python script. This script allows you to extract only the annotated images and reorganize them in order the respect the paths of the LOOK dataset. To run this script you need to use python 3 running and the pandas library.
To be able to run the script you need to input the name of the dataset(s) you want to filter and the path to the annotation file. To run the script for all the datasets, you can navigate inside the LOOK folder in a terminal window and run the command
python extract_look_dataset.py -d Nuscenes,KITTI,JRDB -a LOOK_annotations.csv.