Data Specification

The collected datasets are each contained in a folder, named after a random hash, for example 71de12f9. A dataset folder has the following directory structure:

camera_matrix.csv
odometry.csv
imu.csv
depth/
  - 000000.npy
  - 000001.npy
  - ...
confidence/
  - 000000.png
  - 000001.png
  - ...
rgb.mp4

rgb.mp4 is an HEVC encoded video, which contains the recorded data from the iPhone's camera.

The depth/ directory contains the depth maps. One .npy file per rgb frame. Each of these is a Numpy matrix file containing uint16 values. They have a height of 192 elements and width of 256 elements. The values are the measured depth in millimeters, for that pixel position. These can be loaded using Numpy using the np.load function.

The confidence/ directory contains confidence maps corresponding to each depth map. They are grayscale png files encoding 192 x 256 element matrices. The values are either 0, 1 or 2. A higher value means a higher confidence.

The camera_matrix.csv is a 3 x 3 matrix containing the camera intrinsic parameters.

The odometry.csv file contains the camera positions for each frame. The first line is a header. The meaning of the fields are:

Field
Meaning
timestampTimestamp in seconds
frameFrame number to which this pose corresponds to e.g. 000005
xx coordinate in meters from when the session was started
yy coordinate in meters from when the session was started
zz coordinate in meters from when the session was started
qxx component of quaternion representing camera pose rotation
qyy component of quaternion representing camera pose rotation
qzz component of quaternion representing camera pose rotation
qww component of quaternion representing camera pose rotation

The imu.csv file contains timestamps, linear acceleration readings and angular rotation readings. The first line is a header. The meaning of the fields are:

Field
Meaning
timestampTimestamp in seconds
a_xAcceleration in m/s^2 in x direction
a_yAcceleration in m/s^2 in y direction
a_zAcceleration in m/s^2 in z direction
alpha_xRotation in rad/s around the x-axis
alpha_yRotation in rad/s around the y-axis
alpha_zRotation in rad/s around the z-axis