✨ Precaching got a turbo boost! 🚀

Hi hi~ today the caching step stopped being such a lazy little snail! 🐌➡️🐇

• Before, we loaded one image, encoded it, saved it… one… by… one… zzz 😴 The GPU was basically napping while the disk did all the reading.

• Now we have a squad of worker threads (cache_workers) fetching images in parallel while the GPU chomps through them in batches (cache_batch_size). I/O and compute finally hold hands and work together! 🤝

• Bonus: a cute little progress log tells you img/s and ETA, so you know exactly when snack time ends. 🍪⏱️

• Also added a safety hug for resumes!
  If you try to resume a checkpoint with a different config, the trainer now gently stops and asks “are you suuure? type ‘yes’~” 🥺 No more silently mixing old weights with new settings and crying later. In non-interactive mode it just aborts to keep you safe. 💖

Commit