Quantitative psychology under scrutiny: measurement requires not result-dependent but traceable data generation

Various lines of critique of quantitative psychology, well-established and new, are used to trace along the field's typical steps of research a complex network of misconceptions and fallacies codified in psychological jargon. The article explores what constructs actually are, why they are needed in psychology, fallacies and challenges in construct research, and the crucial role of language. It shows how common misconceptions of language and concepts mislead psychologists to conflate phenomena, qualities, quantities and constructs with one another and with their semiotic encodings in terms, variables and scores. The article clarifies the conceptual relations between nomological networks, representation theorems and psychometric modelling. It reveals conflations of disparate notions of causality and unobservability, and erroneous equations of nomological networks with semantic networks, description with explanation, and measurement theories with explanatory theories. Instead of establishing causal measurand-result relations, common practices match data generation to the results rather than the phenomena and properties studied. Mathematical meaning for scores is often created from differences between individuals and between different phenomena and properties, which constitute mere conceptual entities and cannot reflect magnitudes attributable to individuals. This entails biased inferences on the actual study phenomena and shows that replicability problems may be even larger than assumed.