I am emphatically not anti-empiricist

Yesterday, I gave a talk at SPLASH Onward in Vancouver introducing my essay “Concept analysis in programming language research: Done well it is all right” [ACM DL] [Author’s PDF] [presentation slides (PDF)]. Both reader comments earlier and some of the questions after my presentation leads me to think that I was not quite clear enough in my essay and my presentation:

I am not advocating abandoning empirical research. Quite the contrary! I believe it should be used wherever it makes sense. I just happen to think that conceptual questions cannot be resolved based on empirical stuff (alone – though empirical research can function as reasons in a philosophical argument quite fine).

A related question is, is nonempirical research science? I believe it is. If we restricted science to empirical stuff only, we would have to reject mathematics from the halls of science. I believe the key characteristic that separates science from non-science is intellectual honesty combined with the use of best available methodology. Concept analysis, done well, fits that bill.

Doctoral defense approaching, dissertation publicly available

I will be defending my doctoral dissertation “Evidence-based programming language design: a philosophical and methodological exploration” on December 4, 2015 at noon, in the Seminarium building, auditorium S212, of the University of Jyväskylä. My opponent will be Professor Lutz Prechelt (Freie Universität Berlin, Germany), and the custos is Professor Tommi Kärkkäinen (University of Jyväskylä).

The defense is public; anyone may come. Dress code for the audience is whatever one would wear to any lecture or regular academic activity at the university (no formal dress required). There is a Facebook event page.

The dissertation manuscript was reviewed (for a permission to publish and defend) by Professor Matthias Felleisen (Northeastern University, USA) and Professor Andreas Stefik (University of Nevada, Las Vegas, USA). The dissertation incorporates most of my licentiate thesis, which was examined last year by Doctor Stefan Hanenberg (University of Duisburg-Essen, Germany) and Professor Stein Krogdahl (University of Oslo, Norway).

The dissertation is now publicly available as a PDF.

The dissertation mentions Haskell in several places, although that is not its main focus.

ABSTRACT

Kaijanaho, Antti-Juhani
Evidence-Based Programming Language Design. A Philosophical and Methodological Exploration.
Jyväskylä: University of Jyväskylä, 2015, 256 p.
(Jyväskylä Studies in Computing
ISSN 1456-5390; 222)
ISBN 978-951-39-6387-3 (nid.)
ISBN 978-951-39-6388-0 (PDF)
Finnish summary
Diss.

Background: Programming language design is not usually informed by empirical studies. In other fields similar problems have inspired an evidence-based paradigm of practice. Such a paradigm is practically inevitable in language design, as well. Aims: The content of evidence-based programming design (EB-PLD) is explored, as is the concept of evidence in general. Additionally, the extent of evidence potentially useful for EB-PLD is mapped, and the appropriateness of Cohen’s kappa for evaluating coder agreement in a secondary study is evaluated. Method: Philosophical analysis and explication are used to clarify the unclear. A systematic mapping study was conducted to map out the existing body of evidence. Results: Evidence is a report of observations that affects the strength of an argument. There is some but not much evidence. EB-PLD is a five-step process for resolving uncertainty about design problems. Cohen’s kappa is inappropriate for coder agreement evaluation in systematic secondary studies. Conclusions: Coder agreement evaluation should use Scott’s pi, Fleiss’ kappa, or Krippendorff’s alpha. EB-PLD is worthy of further research, although its usefulness was out of scope here.

Keywords: programming languages, programming language design, evidence-based paradigm, philosophical analysis, evidence, systematic mapping study, coder agreement analysis

How a wrong model can lead you astray

The other day I got on a bus at the downtown main bus terminal. Behind me, a woman started to interrogate the driver.

“Do you go to Pohjantie?”

When the bus driver did not respond (likely, he does not remember all the names of the roads on his route), she changed tactics:

“Your sign says Kuokkala. Which route do you take?”

There are three roads to Kuokkala, which is on the other side of Lake Jyväsjärvi: two drive around the like on opposite sides, and one takes a bridge over the lake. One of the drivearounds in fact goes through Pohjantie (as well as a neighbourhood called Tikka), and several buses take that route. Buses also use the bridge; to my knowledge, no bus uses the other driveraound route.

“On the way back I’ll go through Tikka.”

“So you end up at Viherlandia?” Viherlandia is the terminus of one of the bus routes that drives through Pohjantie.

“No.”

“But Kuokkala, how do you get there?”

“I’ll take the bridge, then drive around the Kuokkala centre and then turn right …”

“… toward Nenäinniemi. You’re not my bus, thanks.”

In fact, she was wrong; it would have been her bus had she waited to hear the bus driver’s reaction to Nenäinniemi; he wasn’t going there, instead, he would have just turned to Tikka and from there through Pohjantie back to downtown.

But I suspect she had a mental model: all buses in Jyväskylä run (so she probably thought) pendulum routes, going back the same route they take. So, once she had established that the bus took the bridge, she had all the information she thought she needed.

In fact, that particular line runs a mixed pendulum and ring model: going Northeast from the downtown terminal, it runs to a particular suburb and retraces its steps back to downtown; however, southbound, it goes over the bridge to Kuokkala and drives a semiring route in Kuokkala, existing through the Pohjantie drivearound and making its way back to the downtown terminal.

Route of Bus 20
Route of Bus 20 (Map © OpenStreetMap contributors)

People sometimes say that science is objective and empicial, and that the data speak for themselves. This sort of a statement forgets that data mean nothing by themselves, and your conclusions are no better than your model.