Using State-of-the-Art Artificial Intelligence Detection for Retrieval of Sample Tubes on Mars

The Mars Sample Return Mission, a collaborative effort by NASA and the European Space Agency (ESA), aims to return Martian soil and rock samples to Earth for the first time. A key aspect of the mission involves retrieving sample tubes dropped by the Perseverance rover on the Martian surface, which are critical for scientific analysis. However, the harsh and dynamic Martian environment presents challenges in locating these tubes, as they can be partially buried or obscured by dust and debris. This paper investigates the use of artificial intelligence (AI) to enhance the detection and retrieval of these sample tubes. Specifically, we explore the effectiveness of two AI models: the Segment Anything Model (SAM) and its language-augmented variant, LangSAM, for detecting the sample tubes in images. SAM demonstrated good performance on the Earth-based training data, but struggled in real Martian conditions. In contrast, LangSAM, augmented with natural language prompts, showed superior accuracy, especially in scenarios where the tubes were obscured or partially buried. Our results highlight the potential of LangSAM to overcome the challenges of small datasets and visually complex environments, making it a promising tool for future Mars exploration missions.