Many authors have tried to explain the shape of the species abundance distribution (SAD). Some of them have suggested that sampling spatial scale is an important factor shaping SADs. These suggestions, however, did not consider the indirect and well-known effect of sample size, which increases as samples are combined to generate SADs at larger spatial scales. Here, we separate the effects of sample size and sampling scale on the shape of the SAD for three groups of organisms (trees, beetles and birds) sampled in the Brazilian Atlantic Forest. We compared the observed SADs at different sampling scales with simulated SADs having the same richness, relative abundances but comparable sample sizes, to show that the main effect shaping SADs is sample size and not sampling spatial scale. The effect of scale was minor and deviations between observed and simulated SADs were present only for beetles. For trees, the match between observed and simulated SADs was improved at all spatial scales when we accounted for conspecific aggregation, which was even more important than the sampling scale effect. We build on these results to propose a conceptual framework where observed SADs are shaped by three main factors, in decreasing order of importance: sample size, conspecific aggregation and beta diversity. Therefore, studies comparing SADs across sites or scales should use sampling and/or statistical approaches capable of disentangling these three effects on the shape of SADs.