Local Updating

Local updating mechanisms emerge as crucial strategies for minimizing communication overhead between nodes, thereby optimizing network utilization. Local SGD\cite{stich2018local} enables independent execution of stochastic gradient descent on multiple worker nodes in parallel, and achieves the convergence rate comparable to traditional mini-batch methods\cite{dekel2012optimal}.