We investigate the numerical implementation of functionally graded properties in the context of the finite element method. The macroscopic variation of elastic properties inherent to functionally graded materials (FGMs) is introduced at the element level by means of the two most commonly used schemes: (i) nodal based gradation, often via an auxiliary (non-physical) temperature-dependence, and (ii) Gauss integration point based gradation. These formulations are extensively compared by solving a number of paradigmatic boundary value problems for which analytical solutions can be obtained. The nature of the notable differences revealed by the results is investigated in detail. We provide a user subroutine for the finite element package ABAQUS to overcome the limitations of the most popular approach for implementing FGMs in commercial software. The use of reliable, element-based formulations to define the material property variation could be key in fracture assessment of FGMs and other non-homogeneous materials.