Code
For convenience, the supplemental material contains the code submitted for review. However, for more recent code and additional documentation, please have a look at our GitHub: https://github.com/ir-lab/LanguagePolicies
The source code of our model is given the Code directory of our supplemental material.
The following tree outlines the major source code files.
- main.py $\rightarrow$ Used to train our model with the default hyperparameters.
- model_src
- attention.py $\rightarrow$ Our attention model
- basismodel.py $\rightarrow$ The implementation of our basis model used to generate the trajectories
- feedbackcontroller.py $\rightarrow$ This is our overall control model used as a cell in a recurrent architecture
- glove.py $\rightarrow$ Embedding lookup into GloVe
- model.py $\rightarrow$ Main file of the model, combining all components
- ros2
- src
- policy_translation $\rightarrow$ Defining a ROS 2 node to generate the ROS service files used by our model to communicate between CoppeliaSim and the network service.
- service.py $\rightarrow$ ROS 2 node that loads a pre-trained model and provides a service that can be used by CoppeliaSim to actuate the robot.
- utils
- collect_data.py $\rightarrow$ Code used to collect our training data from CoppeliaSim
- data_processing_rss.py $\rightarrow$ Data processing: Raw data to TFRecords
- graphs.py $\rightarrow$ Plotting functions for TensorBoard
- network.py $\rightarrow$ Wrapper class for our neural network. This class handles the training and validation loop.
- tf_util.py $\rightarrow$ Utility file to configure TensorFlow depending on different hardware requirements
- voice.py $\rightarrow$ Templating engine to automatically generate task instructions during the collection of training data
- val_model_vrep.py $\rightarrow$ CoppeliaSim interface to validate a pre-trained model (together with service.py as control backend)
- viz_val_vrep.py $\rightarrow$ Takes the val_results.json file generated by val_model_vrep.py to generate a single line of Latex for our main results table.