2NVIDIA; 3UT Austin; *Equal Contribution Corresponding authors: [email protected], [email protected] Abstract Transferring policies learned in simulation to the real world is a promising strategy for acquiring robot skills at scale. However, sim-to-real approaches typically rely on manual design and tuning of the task reward function as well as the simulation physics parameters, rendering the process slow and human-labor intensive. In this paper, we investigate using Large Language M...