Parallelizing neural networks on one GPU with JAX

Since the launch of ChatGPT, explorations in two directions kicked into high gear.

Where simulation is good enough, it is transforming robotics. But it’s nearly impossible to engineer simulators that can accurately model complex physical and visual properties.

Cross-posted from the CILVR blog. In the last few years, there's been an explosion of work on learning good representations of data. From NLP123 to computer vision456 to reinforcement learning789, the field has never been hotter. However, defining precisely what we mean by a good representation can be tricky. This has led to a somewhat ad-hoc...