Work Experience
During 2023 and 2024, I had the privilege of working alongside NASA engineers at Goddard Space Flight Center on the Distributed Systems Mission (DSM) team.
The goal of the DSM team was to produce a viable proof-of-concept for autonomous coordination and decision-making by a group of agents (such as drones and/or satellites) referred to as a constellation. This framework has a variety of applications, particularly in circumstances where operator input is impractical, such as when the round trip time of light creates a significant delay, or when prioritizing what data to send to the ground.
NASA OSTEM Intern
Goddard Space Flight Center
June 2023 – August 2023
Python
Linux
Ubuntu
Machine Learning
Tensorflow
Keras
TensorFlow Lite
ROS 2
Gazebo
Git
Gitlab
Docker
• Iterated on Keras anomaly detection model to optimize performance
This machine-learning algorithm was trained using data from the Orbiting Carbon Observatory-2 (OCO-2) satellite cross-referenced with the National Ecological Observatory Network (NEON). The model initially predicted CO2 isotope values at locations near NEON sites so that performance could be measured based on accuracy to actual values measured. Anomalies were classified as predictions of CO2 isotope values outside of a nominal range.
I improved the model’s performance by:
- Simplifying the model to a binary classifier that directly predicts anomalies
- Experimenting with different numbers of neurons at different stages of the model
- Creating an additional dataset incorporating data from the National Land Cover Database so that the model could consider the type of land (e.g., forest, grassland, city, etc.) to both improve accuracy and generalizability of the model.
• Converted anomaly detection model to TensorFlow Lite to simulate performance on embedded hardware
Software designed to run on flight hardware such as satellites must take up as little space as possible, and be able to run with limited processing power.
The model was successfully converted to Tensorflow Lite to simulate these conditions, and still retained much of its accuracy in detecting anomalies.
• Used ROS 2 and Gazebo to create a basic proof-of-concept demonstrating how the algorithm could be utilized in a constellation mission
The demo included two agents. The first collected data to be processed by the anomaly detection algorithm, and flagged any anomalies. The next was a theoretical camera satellite, which would autonomously receive the location of any flagged anomalies, and ensure that those locations are captured during the flyover.
Gazebo was used to visualize the two agents, which points were flagged as anomalous, and the camera satellite’s field of view as it travelled.
• Compiled results and deliverables into a poster and presented it alongside other OSTEM interns
NASA Software Engineering Contractor
Southeast Universities Research Association (SURA)
January 2024 – September 2024
Python
Linux
Ubuntu
Redis
Machine Learning
Tensorflow
Keras
Git
Gitlab
Docker
• Developed framework for data prioritization as proof-of-concept for autonomous decision-making
Continuing my work from the previous internship, the next step was to integrate the anomaly detection algorithm into a more complex demo that can better simulate a real-world scenario.
The chosen scenario was data prioritization, which can become necessary in situations where data is being collected faster than it can be sent to the ground.
The basic framework involved several agents communicating with one another, able to assign a ‘priority score’ to each observation based on whether it is anomalous, or its relationship to other observations.
This system was first implemented as each agent running as a thread, communicating with other agents via global variables. The next iteration used the pub/sub functionality of Redis to send and receive messages instead.
• Optimized scoring and sorting algorithms to balance between speed and accuracy
Because data was being collected, scored, and sorted in real time, several algorithms were tested to ensure the scoring and sorting process did not become a bottleneck. Maintaining a sorted list could be too slow depending on the size of the data buffer, however a simpler classification system, such as simply labeling data as low, medium, or high priority and maintaining separate unordered buffers for each was able to serve as a good approximation for perfect prioritization and run significantly faster.
• Integrated additional agents for cloud fraction and fire detection into proof-of-concept
In order to better demonstrate the system’s ability to prioritize data from a constellation of different satellites, additional agents were added to the demo, utilizing data from the MODIS satellite. Specifically, using data for fire detection, and cloud cover.
This allowed for more complex interactions between agents, such as ensuring observations near an active fire are higher priority, or deprioritizing observations when cloud cover is high enough to cause interference.
• Reformatted demo to function as plugin for the OnAIR platform
The OnAIR (On-board Artificial Intelligence Research) platform is a framework designed to enable AI algorithms to interact with a simulation of NASA’s core Flight System.
The platform is a hierarchal system with four layers. Only the first two layers directly receive a copy of the data on a given step, however each layer passes its conclusions, along with the conclusions of the previous layers on to the next.
My initial prototype of the plugin implemented everything on one layer, however once I was able to verify the functionality of the system, I was able to split it up across the different layers to better utilize the platform’s architecture.
Additionally, I implemented a listener program that could log how the data was being processed and scored. These logs were used to create a visual demo of how the plugin worked, which was presented by the Distributed Systems Mission team at the end of the fiscal year.