Finding Exoplanets in Noisy Data with Machine Learning

Detecting Earth-like planets from distant stars is challenging. NASA’s Kepler mission provides raw light curves, or brightness measurements over time, that contain tiny dips caused by planets crossing their star. To automatically identify these elusive signals, researchers developed Astrobit 1.0, an AI system that sifts through the noisy data.

Kepler’s Simple Aperture Photometry (SAP) flux method is complicated by instrumental systematics, cosmic rays, and quarter-boundary artifacts that resemble planetary signals. A simple threshold approach is unreliable, frequently missing genuine planets or flagging false positives.

The Astrobit architecture addresses these issues through a series of independent stages, each cacheable for efficient processing:

1. Cleaning: Mask bad cadences and sigma-clip outliers from the raw SAP flux to minimize the impact of noisy measurements.

2. Detrending: Apply a per-quarter Savitzky-Golay filter to remove long-term trends and recover approximately 90% of true transit depth compared to only 33% using a running median. This method also avoids edge artifacts caused by Kepler’s quarterly roll boundaries.

3. BLS Period Search: Conduct a 50k log-spaced coarse grid search followed by a 600-point fine refinement around each peak. Implement alias checking at 0.5x, 1x, 2x, and 3x periods to catch harmonics that could be misidentified as separate planets.

4. Feature Extraction: Extract key features such as the Signal-to-Data-Noise (SDE), transit depth, signal-to-noise ratio (SNR), odd/even depth ratio, and secondary eclipse depth. These features provide a comprehensive representation of each candidate transit.

5. Random Forest Classifier: Train a Random Forest Classifier on 269 labeled stars to differentiate between true planets and false positives. This multi-feature decision boundary surpasses mere SDE thresholding, resulting in significantly fewer false positives.

6. Platt Scaling: Calibrate the classifier’s raw scores to probabilities using Platt Scaling on the development set. This ensures that the confidence scores accurately reflect the likelihood of a candidate being a true planet.

7. Vetting: Perform additional checks to further filter out non-planetary signals. This includes a secondary eclipse check, odd/even depth ratio consistency, per-quarter recurrence analysis to rule out repeated false positives, and filtering based on known systematic periods that could mimic planetary signals.

The final ranked candidates are compiled into a submission.csv file, detailing the most promising exoplanet candidates for further investigation.

The development process highlighted several key lessons:

• Detrending plays a critical role in accurately recovering transit depths, far surpassing the performance of a simple classifier. Significant effort was dedicated to refining the Savitzky-Golay filter, recognizing its importance in preserving the true signal.

• Caching intermediate results, such as BLS results in CSV format, dramatically speeds up the iterative development process by enabling interruptions and resumptions without reprocessing data.

• Relying solely on the SDE (Signal-to-Data-Noise) metric is insufficient for distinguishing planetary signals from noise. Incorporating multiple features into the Random Forest Classifier drastically reduces false positive rates.

• Calibration of classifier scores through Platt Scaling transforms raw model outputs into meaningful probabilities, enhancing the reliability of candidate rankings.

• Vetting is essential to ensure the accuracy of the detected planetary signals. Secondary eclipse consistency and odd/even depth ratio checks are particularly valuable in eliminating false positives associated with eclipsing binaries.

Astrobit 1.0 leverages Python, scikit-learn, lightkurve, scipy, and numpy to create an efficient, transparent, and scalable exoplanet detection pipeline. By systematically addressing the challenges posed by Kepler’s noisy data, Astrobit 1.0 advances the search for Earth-like planets beyond our solar system.