In recent years, local field potentials (LFPs) haveemerged as a promising alternative input signal for brain-machine interfaces (BMIs). Several studies have demonstratedthat LFP-based BMIs could provide long-term recording stabilityand comparable decoding performance to their spike counter-parts. Despite the compelling results, however, most LFP-basedBMIs still make use of hand-crafted features which can betime-consuming and suboptimal. In this paper, we propose anend-to-end system approach based on temporal convolutionalnetwork (TCN) to automatically extract features and decodekinematics of hand movements directly from raw LFP signals.We benchmark its decoding performance against traditionalapproach incorporating long short-term memory (LSTM) de-coders driven by hand-crafted LFP features. Experimental re-sults demonstrate significant performance improvement of theproposed approach compared to the traditional approach. Thissuggests the suitability of TCN-based end-to-end system and itspotential for providng stable and high decoding performanceLFP-based BMIs.