mujoco_rlhf: this directory contains the code for our Mujoco experiments. All code is ours

alignment_handbook: this is an adapted version of the alignment-handbook GitHub repository (https://github.com/huggingface/alignment-handbook). We use this to train our LLMs. To reproduce our experiments, set the relevant parameters at alignment-handbook/scripts/run_all.py . As an example, we have the code to train gemma-7b with sft and then simpo.

alpaca_eval: this is an adapted version of the alpaca_eval GitHub repository (https://github.com/tatsu-lab/alpaca_eval). To reproduce the way we evaluate our models, you can launch the script alpaca_eval_ready/scripts/run_all_comparisons.py .