Article
Near-Optimal Scaling of Large Deep Network Training on Public Cloud
Rating:
0.0
Views:
77
Likes:
1
Library:
1
A recently published study, MiCS, provides experimental evidence that the infrastructure used to carry out model training should be taken into account, especially for large deep neural networks trained on the public cloud. The article shows distributing the model weights unevenly between GPUs decreases inter-node communication overhead on AWS V100 and A100 instances.
Rate This Post
-
Education
-
Communication
-
Entertainment
Rate The Educational Value
Rate The Ease of Understanding and Presentation
Interesting or Boring? Rate the Entertainment Value