While our method can show zero-shot generalization in completely new environments, we probe OK-Robot to better understand when and how it succeeds and fails. While we find a 58.5% success rate at completely novel homes, at a closer look, we also notice a long tail of failure causes, which is presented in the figure above. We see that the leading three cause of failures are failing to retrieve the right object to navigate to from the semantic memory (9.3%), getting a difficult pose from the manip...