First off, thank you for your hard work on this package! It's proven to be very helpful for fully leveraging multi-GPU training of large-scale language models. Is your feature request related to a ...

Pytorch is one of the popular and the fastest growing deep learning framework. The flexibility of this framework allows academicians and researchers to quickly prototype research ideas. Moreover, the ...