Bitsletter #7: Accelerating ML Training & Diving into Geometry-aware 3D GANs
🧠 ML Tip: Pay attention to data loading
Data loading is an important part of any machine learning project. Models need data, and data must be carefully prepared and fed to them. Training time is another important factor in machine learning: the quicker is your training loop, the most examples you can show to your model for the same time budget. Nowadays, access to accelerated hardware such as GPUs is ubiquitous, it’s also mandatory to train big models on big datasets in a decent amount of time.
However, it’s not as easy as attaching a GPU to your virtual machine. What’s the point of using a GPU if your utilization rate is low? Indeed, GPUs are so fast at matrix multiplications that your data loading process must transfer data to them as efficiently as possible. Otherwise the GPU will spend most of its time waiting for data. If the data fits into your memory, you should not have much trouble. The CPU should be able to transfer the data quickly to the GPU (except if you do a lot of CPU bound preprocessing before sending the data).
However, in most cases, the data doesn’t fit entirely in your memory, and your data loading becomes IO-bound: you spend time reading data from the disk every time you build a batch for the GPU. Most of the time, if you perform data loading naively, you won’t optimize these IO calls, resulting in a poor utilization of your GPU. Techniques such as moving preprocessing inside the GPU, pre-loading batches of data in advance or using memory mapped files can greatly improve your performance. Fortunately there exists libraries to help you implementing these tricks in you project, such as mmap.ninja or FFCV.
🌐 Web Tip: Web Tip: You should use TypeScript
However you quickly miss some features of statically types languages: great autocompletion, tons of bug detected at dev/build time, code quality**, improved developer experience …
It’s not that hard to pick up, and you will gain much more confidence in your code. Stop sweating when your continuous delivery triggers the deployment of your new release 😅.
👩🔬 Research Paper: Efficient Geometry-aware 3d Generative Adversarial Network (EG3D)
NVlabs released publicly the official PyTorch implementation of EG3D (CVPR 2022 Paper). Efficient Geometry-aware 3d Generative Adversarial Network (EG3D) is a recent paper about using GANs to generate multiple consistent view of images and 3D shapes. They only use 2D pictures has an input, which makes the problem particularly challenging.
Existing 3d GANs models are computationally intensive or sacrifice consistency by making approximations to reduce the computation time: it alters the multi-view consistency, meaning that a same object generated at different angles will clearly exhibits geometry or texture flaws. A key process to achieve their impressive results was to decouple feature generation and neural rendering to leverage state-of-the-art 2D CNN such as StyleGAN2. Definitely worth reading. 💡 It's insightful to have access to the code produced by top AI labs. Reading others' code, is the best way to learn new techniques, tricks, useful libraries, ...
🛠 Tool: Plasmo, build browser extensions easily
"It's like NextJS but for browser extensions".
Who doesn't have any browser extensions? You likely have at least an ad-blocker. Browser extensions add features to the browsers. They allow you to integrate your products directly inside the Chrome, Firefox, Safari, ... of your users. Plasmo is a framework to ease the development process of browser extensions.
👉🏽 First-class React + TypeScript Support
👉🏽 Declarative development with manifest.json auto-generation
👉🏽 .env* files
👉🏽 Remote code bundling
👉🏽 Automated deployment
Do not hesitate anymore. If you application can benefit from a browser extension, use Plasmo and build it in a glimpse.
Lightning AI is the startup behind the PyTorch Lightning framework which dramatically reduces the boilerplate of PyTorch projects.
The startup also provide a SaaS to easily scale your machine learning workloads in the cloud and leverage distributed training using many GPUs.
Push API: Web Push Notifications coming to iOS and Safari
Apple is working on the Web Push Notifications for iOS and Safari. As part of the iOS 16 release, Safari will support the Push API. This is a great new for web developers, since the Push API is becoming a standard for sending notifications to your users.