---
configs:
  - config_name: train
    data_files: train.parquet
  - config_name: test
    data_files: test.parquet
---
## Overview

**RoboCerebra** is a large-scale benchmark designed to evaluate robotic manipulation in **long-horizon tasks**, shifting the focus of robotic imitation learning from fast, reactive **System 1** behavior to slow, deliberate **System 2** reasoning.

This dataset is tailored for advancing research in high-level robotic planning and sequential decision-making, providing rich and fine-grained demonstrations that span complex task sequences under dynamic and realistic conditions.

## Key Features

- **Long-Horizon Tasks**: Each trajectory in RoboCerebra consists of extended subtask sequences, enabling the study of long-term reasoning and planning in robotic systems.
- **6× Longer Trajectories**: On average, RoboCerebra trajectories are approximately **6 times longer** than those in existing robotic manipulation datasets.
- **Top-Down Dataset Generation Pipeline**:
  - A large language model generates **high-level task instructions**.
  - Tasks are automatically decomposed into **subtask sequences**.
  - Human operators execute the subtasks using a simulator to collect high-quality robotic trajectories.
  - Multiple **verification stages** ensure semantic consistency and data integrity.
- **Realistic Visual Conditions**: The dataset includes dynamically changing environments to mimic real-world scenarios, improving robustness and generalization.

## Installation

This project includes resources related to **LIBERO**. Please follow the instructions below for installation and a quick start.

---

### 1. Clone the RoboCerebra repository

``````bash
# Make sure git-lfs is installed (https://git-lfs.com)
git lfs install
git clone https://huggingface.co/datasets/qiukingballball/RoboCerebra
``````

### 2. Install OpenVLA-OFT

1. See [SETUP.md](https://github.com/moojink/openvla-oft/blob/main/SETUP.md) for instructions on setting up the conda environment.
2. Install the modified version of LIBERO and required packages:

```bash
pip install -e LIBERO # From robocerebra base dir
pip install -r experiments/robot/libero/libero_requirements.txt  # From openvla-oft base dir
```

---

## Evaluate OpenVLA-OFT model on RoboCerebra

Evaluate on RoboCerebra-Ideal using resume mode:

``````bash
python run_robocerebra_eval.py --resume True --root_dir RoboCerebra Bench/Ideal --pretrained_checkpoint <path_to_your_pretrained_openvla-oft_model>
``````

Evaluate on RoboCerebra-ObservationMismatching using resume mode:

``````bash
python run_robocerebra_eval.py --resume True --root_dir RoboCerebra Bench/Observation Mismatching --pretrained_checkpoint <path_to_your_pretrained_openvla-oft_model> --dynamic_shift_description True
``````

Evaluate on RoboCerebra-RandomDisturbance using resume mode:

``````bash
python run_robocerebra_eval.py --resume True --root_dir RoboCerebra Bench/Random Disturbance --pretrained_checkpoint <path_to_your_pretrained_openvla-oft_model> --dynamic True
``````

Evaluate on RoboCerebra-MemoryExecution using resume mode:

``````bash
python run_robocerebra_eval.py --resume True --root_dir RoboCerebra Bench/Memory Execution --pretrained_checkpoint <path_to_your_pretrained_openvla-oft_model>
``````

Evaluate on RoboCerebra-MemoryExploration using resume mode:

``````bash
python run_robocerebra_eval.py --resume True --root_dir RoboCerebra Bench/Memory Exploration --pretrained_checkpoint <path_to_your_pretrained_openvla-oft_model>
``````

Evaluate on RoboCerebra-Mix using resume mode:

``````bash
python run_robocerebra_eval.py --resume True --root_dir RoboCerebra Bench/Mix --pretrained_checkpoint <path_to_your_pretrained_openvla-oft_model> --dynamic True --dynamic_shift_description True
``````



