Bitsletter #4: ML Debugging Secrets, Concurrent Promises, and Video Creation with React
🧠 ML Tip: Debug Models With Overfitting
Debugging machine learning models can be hard and time-consuming. Many things can go wrong:
- Data loading
- Training, …
Fortunately there is a simple trick to detect bugs early: overfitting. Overfitting happens when a model is over-specialized on the training set and unable to generalize on new data.
As bad as it sounds, overfitting can be your ally to find bugs easily in your code: if your model overfits, at least the training process worked on the training dataset. It is a clue that you don’t have a bug in you code.
The idea is to create artificially the condition for overfitting:
- Take a few batches of your training data
- Train your model on it for many steps
- If the model overfits (loss going to almost 0 quickly), you likely have a bug-free code
- If the model can’t overfit, you must have a bug somewhere in the code
🌐 Web Tip: Await Promises Concurrently With Promise.all
Instead create an array of promises and call Promise.all(arrayOfPromises) to await for all the promises at the same time, resulting in the concurrent execution of all the promises. Using this technique you can make several call to APIs concurrently, saving your script lots of execution time.
👩🔬 Research Paper: Large Language Models are Zero-Shot Reasoners
There is a reasoning potential in large language models that you can unlock with prompt engineering (zero-shot learning). Large language models are great few-shot learners:
Use a pre-trained large model and fine-tune on a few examples describing the new tasks. Using few-shot learning, these models achieved great performances in reasoning tasks like arithmetic or symbolic reasoning.
What if we don't even need fine-tuning?
Large Language Models are Zero-Shot Reasoners is a recent paper exploring zero-shot learning for reasoning tasks. Here the model is not fine-tuned on tasks examples. Only the prompts are engineered so that the model can handle the task, without explicit training.
💡 Simply adding “Let’s think step by step” before each model answers increased the accuracy significantly on reasoning tasks:
👉🏽 from 17.7% to 78.7% on MultiArith
👉🏽 from 10.4% to 40.7% GSM8K
These results show the huge zero-shot learning potential of large language models. ⇒ High-level reasoning tasks can be extracted via clever prompting.
🛠 Tool: Remotion, A Tool To Create Videos With React
Whether you're an individual doing it for fun, or a company with a content strategy, video is one of the most consumed content on the internet.
Remotion unlocks the power of data driven videos:
👉🏽 Build a template with React components
👉🏽 Use input data to drive the content
👉🏽 Build infinitely many videos by just changing the data
With Remotion you can also render videos in the cloud:
👉🏽 Build your script describing the video
👉🏽 Run it in the cloud and get back an mp4 video file
💡 Finally Remotion also have a great video player with live reloading to inspect your video as a web application. => You can smoothly play animations to get your video "frame perfect". Definitely give it a try if you are into video production/editing and web development.
Hugging Face Releases “Evaluate”, Metrics For ML
Hugging Face released a library for model evaluation. It contains many metrics, compatible with Numpy/Pandas/PyTorch/TensorFlow/JAX. Bonus: the input are type checked to avoid bugs and each metrics comes with a card describing the values, limitations and ranges.
GitHub Now Supports Math In Markdown
Math equations contains many information if a compressed format ⇒ a great way to communicate. GitHub now supports Markdown equation using MathJax. Just use $ for inline equations and $$ for block equations (similar to LaTeX).