The popularity of current communication technologies has boosted the spread of polarization and radical ideologies, which can be exploited by terrorist organizations. Building upon previous research, this work focuses on the task of automatic radicalization detection in texts using natural language processing and machine learning techniques. In this way, we investigate the effectiveness of integrating moral values through the Moral Foundations Theory (MFT). Moral values play a crucial role in identifying ideological inclinations and can have a significant impact on the radicalization detection task. Our approach distinguishes itself in the feature extraction stage, leveraging moral values, emotions, and similarity-based features that utilize word embeddings. Additionally, we thoroughly evaluate the proposed representations with three distinct datasets that model radicalization and use the SHAP method to gain relevant insight into the models’ reasoning.