Skip to content

Booksellers & Trade Customers: Sign up for online bulk buying at trade.atlanticbooks.com for wholesale discounts

Booksellers: Create Account on our B2B Portal for wholesale discounts

Parallel Computing Cluster for Solving Computational Problems in Data

by Chavala Mutyala Rao
Save 12% Save 12%
Current price ₹4,231.00
Original price ₹4,832.00
Original price ₹4,832.00
Original price ₹4,832.00
(-12%)
₹4,231.00
Current price ₹4,231.00

Imported Edition - Ships in 18-21 Days

Free Shipping in India on orders above Rs. 500

Request Bulk Quantity Quote
+91
Book cover type: Paperback
  • ISBN13: 9786209340734
  • Binding: Paperback
  • Subject: N/A
  • Publisher: LAP Lambert Academic Publishing
  • Publisher Imprint: LAP Lambert Academic Publishing
  • Publication Date:
  • Pages: 56
  • Original Price: GBP 38.19
  • Language: English
  • Edition: N/A
  • Item Weight: 87 grams
  • BISAC Subject(s): General

Thisbook tackles a critical bottleneck in large-scale AI: the slow and communication-heavy training of massive Deep Neural Networks (DNNs) on multi-GPU systems. It addresses the trade-off between two main parallelization methods. Data parallelism suffers from severe communication overhead for large models, while pipelined model parallelism (like PipeDream) offers up to 8.91x speedup for large Fully Connected/Recurrent Neural Networks but causes "weight staleness," degrading model accuracy. To resolve this, the paper introduces SpecTrain, a novel technique. SpecTrain uses the momentum from optimizers to predict future weight updates, allowing pipelined computation with accurate, non-stale weights. This enables the high GPU utilization and speed of pipelining while maintaining the training robustness and final accuracy of synchronous methods.

Trusted for over 49 years

Family Owned Company

Secure Payment

All Major Credit Cards/Debit Cards/UPI & More Accepted

New & Authentic Products

India's Largest Distributor

Need Support?

Whatsapp Us