mola_input_kitti_dataset

CLI tool to evaluate the KITTI odometry benchmark metrics to arbitrary trajectory files, in both, kitti and tum formats. Better used together with evo.

• Build and install
• Documentation
• Examples of usage
  • Evaluate the KITTI metrics on a solution by your SLAM method
  • Evaluate the KITTI metrics on another dataset
  • Transform a KITTI solution file in TUM format to KITTI
• License

Build and install

Refer to the root MOLA repository for compilation instructions.

To install from the ROS repositories:

sudo apt install ros-${ROS_DISTRO}-mola-metrics-eval

This program is largely based on the public Kitti dataset evaluation C++ code. Rewritten to use Eigen instead of GNU GPL’d code, and to add additional features. Original source code notice:

###########################################################################
#   THE KITTI VISION BENCHMARK SUITE: VISUAL ODOMETRY / SLAM BENCHMARK    #
#              Andreas Geiger    Philip Lenz    Raquel Urtasun            #
#                    Karlsruhe Institute of Technology                    #
#                Toyota Technological Institute at Chicago                #
#                             www.cvlibs.net                              #
###########################################################################

Documentation

Basically, this CLI program can be used for evaluating the KITTI metrics in two ways:

• To estimated trajectories of the KITTI odometry sequences, stored in the TUM format (unlike the original KITTI dev kit, which uses the kitti format). In this case, the program reads the ground truth sequences and the calibration files from a user’s local copy of the KITTI dataset. Calibration data is used to first transform the user’s input trajectory to the cam0 frame, in which the KITTI ground truth paths are given.
• To any other pair of trajectory files for other datasets, i.e. a ground truth and an estimated trajectory file.

USAGE: 

   kitti-metrics-eval  [--no-figures] [--gt-tum-path <trajectory_gt.txt>]
                       [-s <01>] ...  [--save-as-kitti <result.kitti>] -r
                       <result.txt> [-k <>] [--] [--version] [-h]


Where: 

   --no-figures
     Skip generating the error figures

   --gt-tum-path <trajectory_gt.txt>
     If provided, the --sequence flag will be ignored and this particular
     file in TUM format will be read and used as ground truth to compare
     against the resulting odometry path.

   -s <01>,  --sequence <01>  (accepted multiple times)
     The sequence number of the path(s) file(s) to evaluate, used to find
     out GT and calibration files for the Kitti dataset.

   --save-as-kitti <result.kitti>
     If given, will transform the input path from the LIDAR frame to the
     cam0 frame and save the path to a TXT file in the format expected by
     KITTI dev kit.

   -r <result.txt>,  --result-tum-path <result.txt>
     (required)  File to evaluate, in TUM format

   -k <>,  --kitti-basedir <>
     Path to the kitti datasets. Overrides to the default, which is reading
     the env var `KITTI_BASE_DIR`.

   --,  --ignore_rest
     Ignores the rest of the labeled arguments following this flag.

   --version
     Displays version information and exits.

   -h,  --help
     Displays usage information and exits.

Examples of usage

Evaluate the KITTI metrics on a solution by your SLAM method

Evaluate on all KITTI test sequences 00-10.

kitti-metrics-eval \
    -r results/estim_%02i.txt \
    -s 00 -s 01 -s 02 -s 03 -s 04 -s 05 -s 06 -s 07 -s 08 -s 09 -s 10 \
    --kitti-basedir /path/to/your/local/kitti_dataset

Expected tree layout under the kitti_dataset directory:

/path/to/your/local/kitti_dataset
├── poses
└── sequences

Evaluate the KITTI metrics on another dataset

Evaluate on another arbitrary dataset, with ground truth trajectory given in the TUM format:

kitti-metrics-eval \
    -r results/my_solution.txt \
    --gt-tum-path /path/to/ground_truth.txt

Transform a KITTI solution file in TUM format to KITTI

kitti-metrics-eval \
    -r results/estim_%02i.txt \
    -s 00 \
    --save-as-kitti results/00.txt

License

This package is released under the BSD-3-clause license.

Changelog for package kitti_metrics_eval

1.2.1 (2024-09-29)

1.2.0 (2024-09-16)

1.1.3 (2024-08-28)

• Depend on new mrpt_lib packages (deprecate mrpt2)
• Contributors: Jose Luis Blanco-Claraco

1.1.2 (2024-08-26)

1.1.1 (2024-08-23)

1.1.0 (2024-08-18)

• Merge pull request #62 from MOLAorg/docs-fixes Docs fixes
• Fix ament_xmllint warnings in package.xml
• Contributors: Jose Luis Blanco-Claraco

1.0.8 (2024-07-29)

• ament_lint_cmake: clean warnings
• Contributors: Jose Luis Blanco-Claraco

1.0.7 (2024-07-24)

1.0.6 (2024-06-21)

1.0.5 (2024-05-28)

• Add new flag --result-in-kitti-format to read trajectories in KITTI format too, not only in TUM format.
• Clean all warnings and clang-tidy recommendations
• Contributors: Jose Luis Blanco-Claraco

1.0.4 (2024-05-14)

• bump cmake_minimum_required to 3.5
• Contributors: Jose Luis Blanco-Claraco

1.0.3 (2024-04-22)

• Fix package.xml website URL
• Contributors: Jose Luis Blanco-Claraco

1.0.2 (2024-04-04)

1.0.1 (2024-03-28)

1.0.0 (2024-03-19)

• Add option to use several groundtruth/result file pairs
• copyright update
• kitti eval cli moves to its own package
• Contributors: Jose Luis Blanco-Claraco

0.2.2 (2023-09-08)