How it works

Star trackers find their direction from star patterns. This solver does the same with craters: it finds crater patterns in the image, looks them up in a catalog of the Moon, and computes where the camera must have been.

1. Overview
2. Finding craters
3. Crater patterns
4. The index
5. Checking a match
6. The camera position
7. Tilted views
8. Accuracy
9. Limits
10. API
11. Data and credits

Overview

The input is one image. Nothing else is necessary: no time, no approximate position, no camera data. The output is the camera latitude, longitude and altitude, the direction the camera pointed, and the latitude and longitude of every pixel.

1. Imageany lighting
2. Find cratersneural network
3. Make patterns4 craters each
4. Look up8.6 million patterns
5. Check and fitcamera position

A typical solve takes a few seconds. The search stops when a match passes the statistical check, or when the time limit is reached.

1. Finding craters

A convolutional neural network, CraterNet, finds the centre and the radius of each crater. It is a U-Net with 6.2 million parameters. For each output pixel (half the input resolution) it gives three things: the probability of a crater centre, the sub-pixel offset of the centre, and the logarithm of the radius.

The main difficulty is the sun. The same crater looks completely different at sunrise, at noon, and from the other side. So the network is trained only on images with known craters under many kinds of lighting:

• the LOLA elevation model of the whole Moon, rendered with a random sun from 1.5° to 89° above the horizon, with cast shadows and real surface brightness (Clementine 750 nm albedo);
• crops of the LRO Wide Angle Camera mosaic (real images at 100 m per pixel);
• random noise, blur, contrast and camera tilt.

The training labels come from the crater catalog. One region of the Moon (longitude 330°–360°) is kept out of training, to test the network on terrain it has never seen. Large craters are found on reduced copies of the image (an image pyramid), so one network covers radii from 3 pixels to hundreds of pixels.

2. Crater patterns

One crater alone says almost nothing. Four craters together make a pattern that is rare enough to look up. The solver uses the idea of astrometry.net: describe each group of four by a short code that does not change when the image is moved, turned or scaled.

The four position numbers are \((x_C, y_C, x_D, y_D)\). The other four numbers use the crater radii, which the shape alone ignores. With \(m\) the mean of \(\ln r\) over the four craters:

The weights \(w_R = 0.30\) and \(w_S = 0.15\) scale the radius terms to the noise of the detector, so all 8 numbers have about the same uncertainty. Swapping A with B, or C with D, gives the same group, so the solver tries all the equivalent orders.

3. The index

The catalog is the Robbins (2019) database: 1.3 million craters of 1 km and larger, for the whole Moon. From it the solver builds an index of crater patterns, once, ahead of time:

• 15 size bands, from patterns about 8 km across to patterns of hundreds of km, each band \(\sqrt{2}\) larger than the one before;
• in each band, groups of four neighbouring craters of matching size, at evenly spaced points over the whole sphere;
• 8.6 million patterns in total, stored in a k-d tree for fast nearest-neighbour search in 8 dimensions.

To search, the solver makes patterns from the largest and clearest craters in the image, computes their codes, and asks the tree for catalog patterns with a code within a small distance (0.08). Every answer is a hypothesis: "these four image craters are those four catalog craters".

4. Checking a match

Most hypotheses are wrong: with millions of patterns, some codes agree by chance. A hypothesis is accepted only when the rest of the image agrees with it. The four matched craters give a map from the image to the Moon. That map predicts where every other catalog crater must appear in the image. Then the solver counts:

• predicted craters that are found in the image (each one is evidence for the match), and
• predicted craters that are missing (each one is evidence against it, unless it is too small to be detected).

The evidence is summed as log-odds, like in astrometry.net:

A match is accepted when \(\Lambda \ge 60\), which means odds of more than \(10^{26}\) to 1 against chance. A weaker match (\(\Lambda \ge 18\)–\(25\)) is accepted only when at least 10 craters agree and the largest craters of the image are among them. These rules came from tests on thousands of simulated views: the solver gives no answer rather than a wrong one.

5. The camera position

The matched craters are points on the Moon with known 3-D positions (latitude, longitude, and the height of the crater rim from the LOLA elevation model). Their image positions fix the camera. The solver fits a pinhole camera with 6 unknowns, position and pointing, by a robust least-squares fit (perspective-n-point).

Position along the ground comes from where the craters are. Altitude comes from how large they look: a crater of diameter \(D\) that is \(d\) pixels wide in an image with focal length \(f\) pixels is about \(f D / d\) away. If the field of view is unknown, the solver tries 10 values from 0.5° to 120°, keeps the best fit and refines it. Then the altitude is less certain, because a camera far away with a narrow lens gives almost the same image as a camera close by with a wide lens. Tilted views and long footprints separate the two.

The result has a 1-sigma uncertainty from the fit. In simulations, the true position is inside 2 sigma in about 9 of 10 cases.

6. Tilted views and the horizon

In a strongly tilted view, craters far away are squashed into thin ellipses, and the patterns change shape. When the edge of the Moon is in the image, the solver uses it: the edge is a circle on the sky, and its curvature and position give the camera tilt and altitude directly. The solver then re-projects the image to a straight-down view and solves that view as usual.

7. Accuracy

Tests on real spacecraft images and on simulated views (the solver had no position hint in any test):

* Compared with the Apollo spacecraft positions on record, which have their own errors of about a kilometre.

Most of the remaining error is in the catalog itself: crater positions in the catalog and in the maps disagree by a few hundred metres. An experimental refinement matches the image pixel by pixel to the WAC map. In simulations with the same lighting as the map it reduces the error to about 10 m, but for real photos the different sun angle moves the shadows and biases the match, so it is off by default.

8. Limits

• High sun. Near full Moon, craters show few shadows and the network finds fewer of them.
• Small footprints. The catalog has craters of 1 km and larger. An image needs about 15 of them, so views less than about 25 km across rarely solve.
• Compressed images. Strong JPEG compression creates false crater detections in flat areas. Use PNG or high-quality JPEG when you can.
• Renders and art. Many Moon images online are paintings or 3-D renders with invented craters. They cannot match the real Moon.

API

The website uses a public HTTP API. You can call it from any program.

curl -F image=@photo.png -F fov_deg=20 https://…/api/solve

Interactive documentation of every field: /docs on the API server.

Data and credits

• Crater catalog: S. J. Robbins (2019), A new global database of lunar impact craters >1–2 km, JGR Planets.
• Elevation: LRO Lunar Orbiter Laser Altimeter (LOLA), LDEM_64.
• Images: LRO Camera Wide Angle Camera global mosaic, 100 m per pixel (NASA/GSFC/Arizona State University).
• Albedo: Clementine UVVIS 750 nm mosaic (USGS).
• Apollo Metric Camera frames: NASA/JSC/Arizona State University.
• Feature names: IAU / USGS Gazetteer of Planetary Nomenclature.
• Method: after D. Lang et al. (2010), Astrometry.net: blind astrometric calibration of arbitrary astronomical images, AJ 139.