# Supplementray of Generative-RLHF-V


Generally, you can find anything you want on our [Anonymous Web](https://generative-rlhf-v.github.io/).

We use the **Frequently Asked Questions** setting to guide the searching of supplementary materials.

1. Where is your open-sourced model?

We open-source the model on [Anonymous Hugging Face](https://huggingface.co/generative-rlhf-v).

2. Where is your open-sourced code?

Our implementation is mainly based on verl: https://github.com/volcengine/verl. Additional details has been included in the page `14-16` of our appendix.

3. What is the prompts of your generative reward model?

We provide the prompts in the line `426` of our appendix.

3. Where is the hyper-parameters of your experiments?

We provide the hyper-parameters in the line `433`, Table 4 of our appendix.

4. GRLHF-V can exhibit reward hacking behavior under overtraining conditions. This raises a significant concern: do the improvements demonstrated in our main experiments stem from such hacking?

We clarify this in the section 7 of our appendix in this supplementray folder.

5. Where is the training curve of GRLHF-V?

We provide the training curve of GRLHF-V in the section 8 of our appendix in this supplementray folder.
