With the increasing demands on quality healthcare and the raising cost of care, pervasive healthcare is considered as a technological solutions to address the global health issues. In particular, the recent advances in Internet of Things have led to the development of Internet of Medical Things (IoMT). Although such low cost and pervasive sensing devices could potentially transform the current reactive care to preventative care, the security and privacy issues of such sensing system are often overlooked. As the medical devices capture and process very sensitive personal health data, the devices and their associated communications have to be very secured to protect the user's privacy. However, the miniaturized IoMT devices have very limited computation power and fairly limited security schemes can be implemented in such devices. In addition, with the widespread use of IoMT devices, managing and ensuring the security of IoMT systems are very challenging and which are the major issues hindering the adoption of IoMT for clinical applications. In this paper, the security and privacy challenges, requirements, threats, and future research directions in the domain of IoMT are reviewed providing a general overview of the state-of-the-art approaches.