Can static type systems speed up programming? An experimental evaluation of static and dynamic type

Programming languages that use the object-oriented approach have been around for quite a while now. Most of them use either a static or a dynamic type system. However, both types are very common in the industry. But, in spite of their common use in science and practice, only very few scientific studies have tried to evaluate the two type systems' usefulness in certain scenarios. There are arguments for both systems. For example, static type systems are said to aid the programmer in the prevention of type errors, and further, they provide documentation help for, there is an explicit need to annotate variables and methods with their respective types.



This book describes a controlled experiment that was conducted to shed some light into the presented matter. Which of the type systems can live up to its promises? Is one of these better suited for a particular task? And which type system is the most supportive in a problem solving? The main hypothesis claims that a static type system is faster in a problem solving in use of an undocumented API. Thus, in the study, the participants need to solve different programming tasks in an undocumented API environment with the help of the static type system (Java), and the dynamic type system (Groovy).



The author starts with a short introduction to the topic, the experimentation, and the motivation. Then, he describes a list of related works, and proceeds to the description of the experiment, its evaluation, and finally, the discussion of the results.

This book should prove interesting reading for anyone who is interested in the mechanics that drive programmer productivity and performance that depend on the kind of technology used, as well as for anyone who might be interested in empirical research in software engineering, in general.



Text Sample:

Chapter 4.3, Questionnaire:

The questionnaire that was handed out to all participants consisted of two parts: The first was the Big Five questionnaire, also called the NEO-FFI [Costa 1992] and the second part was made of a few questions about the participant’s programming experience. The Big Five questionnaire was chosen because it is one of the most popular and widely used psychological tests. It tries to measures five personality dimensions called 'Openness to Experience', 'Conscientiousness', 'Extraversion', 'Agreeableness' and 'Neuroticism', each with its own set of questions that altogether sum up to about 60 questions.

Evaluation of the questionnaire data was not planned to be part of the experiment but it made a lot of sense to gather as much additional data as possible for future research and analysis. The NEO-FFI part was included for a possible exploratory analysis of personality traits/types and experiment performance, and the programming experience part was mainly included to serve as additional data for possible grouping and a meta-analysis of participant questionnaires from different experiments. The latter is a continuation of work already done by this author and Hanenberg [Kleinschmager and Hanenberg 2011], where participant questionnaire data was correlated to participants’ performance in the experiments to try to find meaningful connections.

4.4, Hard- and Software Environment:

4.4.1, Environment:

All participants completed the study on the same Lenovo R60 Thinkpad Computers which were provided by the University, along with a mouse for every laptop.

The prepared software environment was installed on an 8 GB USB Stick. It was a Ubuntu Live Installation of Version 11.04, configured and intended to run and boot only from the stick. The only applications/libraries that were installed apart from the experiment application were XVidCapture (a tool used to record screencasts of the whole experiment for each participant) and the Sun Java Runtime and SDK of Version 1.6_25. The videos from the screen logging application were used as backup for potential problems during the experiment and as a source of information that could give answers to questions which an analysis of the log files alone could not provide.

The experiment IDE (integrated development environment) itself was called Emperior, an editor specifically designed for empirical programming experiments (it was originally created by Steinberg for his master thesis [Steinberg 2011], where it is also explained in much more detail). Emperior provides very simple editing features like a search function or syntax highlighting and also logs different types of information into log files which can be analyzed later.

When the participants clicked on either the 'Test' or the 'Run' buttons, Unix bash (command-line) and Ant (a Java library for batch and command-line processing, [Apache Foundation]) scripts worked behind the scenes. Both were tasked with setting correct paths variables, compile the application, test projects and return possible run output to the console or call JUnit (a unit testing library for Java, see [JUnit]) to show the test results.



Sebastian Kleinschmager is a software engineer from Germany, and has a special interest in creating a scientific foundation for his field. During his studies of applied computer science (Bachelor’s degree), and business information systems (Master), his research focused on the conduction of empirical experiments in order to evaluate programming techniques.

During his day-to-day job, he specializes in software development where he uses the .NET Framework and the newest web technologies, and therefore, has the chance to put theory into practice.