Finding Exoplanets with NASA TESS data

Refactoring Dataset Generation

Implementing dataset generation checkpoints, in case generation fails, I can just start from where I left off. Also employs synthetic dataset generation.

Basically I’m rewriting the compile_confirmed_dataset and same for negative dataset to include checkpoints and all, because last time when I tried to generate dataasets without this it led to the endpoint giving an error and my whole generation is wasted. Even though it took 12+ hours to generate it

Generating datasets…

I tweaked some stuff in the synthetic dataset maker, and added another one that makes synthetic datasets for the negative samples. Generating the datasets right now.

Synthetic dataset is live!

After many struggles, along with downgrading python to 3.11 for compatibility, I was able to generate synthetic datasets. This will significantly streamline the dataset creation process and hopefully speed up the path towards training the model until its nearly flawless

to-do:

• update README to be relevant
• run model training
• optimize model
• run the model to find actual candidates

A MASSIVE dataset

everyone probably knows from my previous devlog that my CNN had massive overfitting. I wanted to fix that.

You know what I did?

I downloaded every single TESS project candidate. Every. Single. One.

Although not all of these candidates are proven to be exoplanets, the algorithms used to detect it market it as very similar to an exoplanet, if not a real one. This will expand my boundaries of my training data, giving me access to ~8k vectors for training.

What I want to do in the future, is to implement an synthetic data generator, Probably using batman. I’ll have to implement things like guassian noise to add some randomenss and whatnot, along with a LOT of RNGs. But we’ll get to that when we get to it.

Wish me luck! My laptop’s about to get fried.

I’ll need to change things around significantly.

Currently, I’m training the CNN with only 2000 vector datasets.

If I really want the CNN to stop overfitting, I’ll have to significantly increase the dataset size.

So here is the todo:

• Use the NASA exoplanet archive to create a list of existing, confirmed exoplanets. Should get over 6000 candidates
• Generate synthetic data using something like batman or someting.

I keep on seeing signs of significant overfitting, unable to be solved. The dataset size must be the problem.

Changes to my CNN & training.

I tried to reduce overfitting on a small dataset. I increased dropout to 0.5, added L2 regularization, and simplified the architecture by removing a convolutional block. Should improve generalization.

Examining the results of training the CNN model show that there is still significant overfitting. I will need to improve this.

Performance evaluation:

• Training loss drops but val loss remains igher showing overfitting
• Training accuracy reaches very high but val acc lags behind. Shows that model is learning patterns specific to dataset.

To-Do:

• Add time-series data augmentation to increase effective dataset size
  -Reduce model capacity
• Tune regularization
  -Tune optimizer/LR

Improvements & a LOT of waiting.
Optimized the 1D CNN

• changed optimizer from default ADAM to tf.keras.optimizers.Adam(learning_rate=1e-4)
• increased batch_size from 32 to 64
• Added ReduceLROnPlateu callback

Why: The training curves showed early-epoch instability and chaos. Lowering to 1e^-4 should smooth out the convergence.

Larger batches provide a more stable gradient estimade, reducing noise

ReduceLROnPlateu allows the model to automatically lower the learning rate further if the validation loss plateus. Help to settle in a more optimal local minimum

Now I’m just waiting, it’s been 3 hours, but I have only downloaded ~2/3 of both Confirmed and Negative datasets.

Training the Neural Network went OK. It was able to learn, it did generalize, but early on it was unstable and then had some mild overfitting. The Train loss shows that it had good optimization, but the Val loss showed that it had unstabel learning. It did converge downard though, so that is good. The way the Train Accuracy shot up so quickly , but the Val acc climbed steadily shows that it isn’t memorizing but actually learning the patterns.

Generating the datasets

You all might be wondering why my progress started from here. Basically, I started this project before I discovered Stardance, by then I had completed, or at least created the basics of the cnn_model, and the generating datasets.