We study the growth of concentration fluctuations of weakly inertial particles in the turbulent channel flow starting with a smooth initial distribution. The steady-state concentration is singular and multifractal so the growth describes the increasingly rugged structure of the distribution. We demonstrate that inhomogeneity influences the growth of concentration fluctuations profoundly. For homogeneous turbulence the growth is exponential and is fully determined by Kolmogorov scale eddies.We derive lognormality of the statistics in this case. The growth exponents of the moments are proportional to the sum of Lyapunov exponents, which is quadratic in the small inertia of the particles. In contrast, for inhomogeneous turbulence the growth is linear in inertia. It involves correlations of inertial range and viscous scale eddies that turn the growth into a stretched exponential law with exponent three halves. We demonstrate using direct numerical simulations that the resulting growth rate can differ by orders of magnitude over channel height. This strong variation might have relevance in the planetary boundary layer.