SVRG++ is a recent randomized optimization algorithm designed to solve non-
strongly convex smooth composite optimization problems in the large data regime.
In this paper we combine SVRG++ with non-uniform sampling of the data points
(already present in the original SVRG algorithm), leading to an algorithm with the
best sample complexity to date and state-of-the art empirical performance. While
the combination and the analysis of the algorithm is admittedly straightforward,
our experimental results show significant improvement over the original SVRG++
method with the new method outperforming all competitors on datasets where the
smoothness of the components varies. This demonstrates that, despite its simplicity
and limited novelty, this extension is important in practice.