A theory-driven testing methodology for developing scientific software

Computer implementations of theoretical concepts play an ever-increasing role in the development and application of scientific ideas. As the scale of such implementations increases from relatively small models and empirical setups to overarching frameworks from which many kinds of results may be obtained, it is important to consider the methodology by which these implementations are developed. Using cognitive architectures as an example, we discuss the relation between an implementation of an architecture and its underlying theory, a relation between a computer program and its description. We argue for the use of an agile development methodology, based around a three-layer scientific test harness and continuous refactoring, as most suitable for developing scientific software. The ideas are illustrated with extended examples of implementing unified theories of human learning, taken from the chunking and template theories.