Towards lifelong object recognition: A dataset and benchmark

Lifelong learning algorithms aim to enable robots to handle open-set and detrimental conditions, and yet there is a lack of adequate datasets with diverse factors for benchmarking. In this work, we constructed and released a lifelong learning robotic vision dataset, OpenLORIS-Object. This dataset was collected by RGB-D camera capturing dynamic environment in daily life scenarios with diverse factors, including illumination, occlusion, object pixel size and clutter, of quantified difficulty levels. To the best of our knowledge, this is an unique real-world dataset for robotic vision with independent and quantifiable environmental factors, which are currently unaccounted for in other lifelong learning datasets such as CORe50 and NICO. We tested 9 state-of-the-art algorithms with 4 evaluation metrics over the dataset in Domain Incremental Learning, Task Incremental Learning, and Class Incremental Learning scenarios. The results demonstrate that these existing algorithms are insufficient to handle lifelong learning task in dynamic environments. Our dataset and benchmarks are now publicly available at this website.2