In this study, we present an analysis of the relationship between the linguistic profile of a text and the physiological and acoustic characteristics of the reader to improve the emotion recognition systems. To this aim, we recorded the speech and electrodermal activity (EDA) signals from 33 healthy volunteers reading neutral and affective texts aloud. We used the BioVoice toolbox and cvxEDA algorithm to estimate some of the main speech and EDA features, respectively. The selected texts were analyzed to quantify their lexical, morpho-syntactic, and syntactic properties. Correlation and Support Vector Regression analyses between linguistic and speech and EDA features have shown a significant bidirectional association between the morpho-syntactic structure of the text and both sympathetic markers and voice acoustic properties. Specifically, significant relationships were observed between linguistic properties and certain EDA and speech features commonly used to evaluate human emotional state (e.g., edaSymp, mean tonic, F0). These findings suggest that lexical, morpho-syntactic, and syntactic properties may have a significant impact on an individual’s emotional dynamics.