# Generalizing Experience for Language Agents with Hierarchical MetaFlows


**MetaFlowLLM** is a novel experience reuse framework that builds a hierarchical experience tree from past tasks to enhance multi-step LLM agents, enabling significant performance gains and cost reductions across complex environments like AppWorld and WorkBench.


---
The folder contents are as follows:
```
metaflow_neurips/
├── AppWorld/
├── WorkBench/
├── config/
├─── README.md
└── requirements.txt
```

## Descriptions

- **`AppWorld/`**: Experiments conducted on the [AppWorld dataset](https://github.com/StonyBrookNLP/appworld), as presented in ACL 2024.
- **`WorkBench/`**: Experiments conducted on the [WorkBench dataset](https://github.com/olly-styles/WorkBench), as presented in COLM 2024.
- **`config/`**: Contains DeepSpeed configuration files used for model training in this study.
- **`requirements.txt`**: Lists all the dependencies required to run the project.




## `AppWorld/` Directory Structure

```
AppWorld/
├── input/
│   ├── SFT_data.pkl  # Cold start data for SFT
│   └── GRPO_data.pkl  # Data for GRPO reinforcement learning training
├── output/
│   └── experience_tree.pkl  # Hierarchical MetaFlow tree constructed
├── inference_build_tree.py  # Implements Algorithm 1: Hierarchical MetaFlow Merging
├── inference_traverse_tree.py  # Implements Algorithm 2: Deep Matching Traversal
├── prompts.py  # Contains the prompt templates used in the experiments
├── train_metaflow_generator_GRPO.py  # Training script for the MetaFlow generator using GRPO
├── train_metaflow_generator_SFT.py  # Training script for the MetaFlow generator using SFT
└── utils.py  # Defines MetaFlow execution under ReAct and Reflexion frameworks

```


## `WorkBench/` Directory Structure


```
WorkBench/scripts/
├── input/
│   ├── SFT_data.pkl  # Cold start data for SFT
│   └── GRPO_data.pkl  # Data for GRPO reinforcement learning training
├── output/
│   └── experience_tree.pkl  # Hierarchical MetaFlow tree constructed
├── train/
│ ├── train_GRPO.py # Training script for the MetaFlow generator using GRPO
│ └── train_SFT.py # Training script for the MetaFlow generator using SFT
├── inference/
│ ├── metaflow_workbench.py # Defines MetaFlow execution within the ReAct framework
│ ├── inference_build_tree_workbench.py # Implements Algorithm 1: Hierarchical MetaFlow Merging
│ ├── inference_traverse_tree_workbench.py # Implements Algorithm 2: Deep Matching Traversal
│ └── prompts.py # Contains the prompt templates used in the experiments
```



