Geolocating Astronaut Photography

Explore Georeferenced Photos overview

Astronauts have been taking breathtaking, and scientifically valuable photographs of Earth since the earliest days of human space flight. The photos help scientists better understand the changing Earth, respond to natural disaster events, and measure various daytime and nighttime phenomena.

where on earth

Yet, before being used in analyses, photos must be geolocated and registered. With an astronaut "in the loop" taking a photo with a handheld camera, the images do not inherently contain geographic metadata - they could depict any location within a 12.5M sq km circle centered on the ISS nadir point. We present methods to localize these images in both daytime and nighttime scenarios.

Find My Astronaut Photo: Localization and Georectification of Astronaut Photography

Alex Stoken1, Kenton Fisher2
1Jacobs Technology, NASA Johnson Space Center 2NASA, Johnson Space Center
Paper Presentation Video arXiv Dataset (AIMS-Day)

(Street) Lights Will Guide You: Georeferencing Nighttime Astronaut Photography of Earth

Alex Stoken1, Peter Ilhardt1, Mark Lambert1, Kenton Fisher2
1Jacobs Technology, NASA Johnson Space Center 2NASA Johnson Space Center
Paper Presentation arXiv Dataset (AIMS-Night)

Astronaut Image Matching Subsets (AIMS)

To evaluate geolocation techniques, we introduce the Astronaut Image Matching Subsets (AIMS).

AIMS-Day

 aims_day_distrib

AIMS-Day contains 323 daytime astronaut photographs with varying focal lengths, obliquity (tilt), orientation (angles), and cloud coverage. The original AIMS task uses preselected reference images at different scales to evaluate localization performance. Download reference imagery for each scale below:

Images + Metadata 1.00x (45GB) 1.25x (31GB) 1.50x (29GB) 1.75x (21GB) 2.00x (20GB)

AIMS-Night
aims_night_distrib

AIMS-Night contains 363 nighttime astronaut photographs with varying focal lengths, obliquity (tilt), orientation (angles), and acquisition year. They are distributed as follows:

aims_night_distrib

The AIMS-Night does not use reference images, and instead measures the number of images localized within 50km of their ground truth center point. Download the set and associated metadata here:

Images + Metadata

Cite

Please use the following BibTex to cite these works: 

@InProceedings{Stoken_2023_CVPR_FMAP,
        author    = {Stoken, Alex and Fisher, Kenton},
        title     = {Find My Astronaut Photo: Automated Localization and Georectification of Astronaut Photography},
        booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
        month     = {June},
        year      = {2023},
        pages     = {6196-6205}
        }
      
@InProceedings{Stoken_2024_CVPR_NightMatch,
        author    = {Stoken, Alex and Ilhardt, Peter and Lambert, Mark and Fisher, Kenton},
        title     = {(Street) Lights Will Guide You: Georeferencing Nighttime Astronaut Photography of Earth},
        booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
        month     = {June},
        year      = {2024},
        pages     = {492-501}
      }