In future full-duplex communications, the cancellation of self-interference (SI) arising from hardware nonidealities will play an important role in the design of mobile-scale devices. To this end, we introduce an optimal digital SI cancellation solution for shared-antenna-based direct-conversion transceivers. To establish that the underlying widely linear signal model is not adequate for strong transmit signals, the impact of various circuit imperfections, including power amplifier distortion, frequency-dependent I/Q imbalance, quantization noise, and thermal noise, on the performance of the conventional augmented least mean square (LMS) based SI canceller, is analyzed. In order to achieve a sufficient signal-to-interference-plus-noise ratio when the nonlinear SI components are not negligible, we propose an augmented nonlinear LMS based SI canceller for a joint cancellation of both the linear and nonlinear SI components by virtue of a widely nonlinear model fit. A rigorous mean and mean square performance evaluation is conducted to justify the performance advantages of the proposed scheme over the conventional augmented LMS solution. Simulations on orthogonal frequency division multiplexing-based wireless local area network standard compliant waveforms support the analysis.