"A lot of research in software engineering strikes me as hopelessly naive in one of two ways. Most of it fails entirely to account for the social and belief aspects altogether. It looks at its object of inquiry as if it was entirely material and inert; as if 'software' was some kind of naturally occurring substance, the properties of which can be revealed in the equivalent of a test tube." (Laurent Bossavit, "The Leprechauns of Software Engineering", 2015)
"As practitioners, it is both in our interest and within our responsibility to pay attention to research. This includes not just the findings of such research, but also its processes and its institutions. Read research papers; find out what’s happening in that world and why it’s not more relevant to your work; weigh in; make your voice heard." (Laurent Bossavit, "The Leprechauns of Software Engineering", 2015)
"As software professionals, we should be interested in knowing at least the basics of our own history, for just the same reasons that as citizens we are expected to know about our national history and about world history: so that we will be able to make informed decisions and know who to trust, who to listen to; so that we are not deceived by lies. Untrue histories generally have an agenda - 'someone trying to sell you something', as the saying goes." (Laurent Bossavit, "The Leprechauns of Software Engineering", 2015)
"But 'average cost to fix one defect' is a stupid metric […] It makes bad projects look good, and good projects look bad. How? By failing to divide the costs of fixing into two categories: fixed costs of detecting and fixing defects - costs which are the same no matter how buggy or how good the product is - and variable costs, those which you pay for each defect." (Laurent Bossavit, "The Leprechauns of Software Engineering", 2015)
"But if you want to count defects, you first have to decide
what (literally) counts as one. We have a hard time even agreeing among a single
team on what counts as a defect - in fact I’m pretty sure my own thinking has
changed over the years, so I’m not even agreeing with myself."
"But we have now reached the most pressing problem in
software engineering: low standards for research publications. Most of what
passes for 'research' in the discipline is ridiculously careless with respect
to examining the 'terms of inquiry'."
"Creating mockups to communicate is not intrinsically a bad idea.
But, as we are subject to confirmation bias, there’s always a risk that we will
stop at our first design attempt and become reluctant to ask if there are
better ways to achieve the same goals. Making these first ideas very detailed; putting
them into a document; and especially blessing that document with the label 'requirements' are all moves which make further revision less likely, and put
us more at risk from confirmation bias."
"Debugging is known as an open-ended sort of activity, and
even seasoned programmers expect variable completion times when faced with this type of task."
"I do not dispute that there are large variations in reported measurements of programmer productivity; however, close examination of the evidence suggests that this observed variability originates in vague definitions of the term, in unreliable instruments of measurement, or in uncontrolled environmental factors, much more than it does in the intrinsic capabilities of programmers at comparable levels of training and experience." (Laurent Bossavit, "The Leprechauns of Software Engineering", 2015)
"Often, the hard work in research isn’t in performing the experiment and collecting data (that tends to be grunt work, in fact, in most disciplines). The hard work consists of designing the experiment that will rule out the most alternative explanations for what you see happening (or think you see happening)." (Laurent Bossavit, "The Leprechauns of Software Engineering", 2015)
"Science journalism is a fine and important thing, but it has a well-known failure mode: sensationalism, where the lure of an attention-grabbing headline causes writers to toss caution to the wind and radically misrepresent a claim." (Laurent Bossavit, "The Leprechauns of Software Engineering", 2015)
"Software development […] needs to be studied with tools that borrow as much from the social and cognitive sciences as they do from the mathematical theories of computation." (Laurent Bossavit, "The Leprechauns of Software Engineering", 2015)
"Software engineering is a social process, not a naturally
occurring one- it therefore has the property that what we believe about
software engineering has causal impacts on what is real about software
engineering.
"The problem is that we cannot infer variations in individual productivity from data collected at the team level: we do not have an adequate theory of how a team’s productivity results from the aggregation of individual abilities, and in particular we cannot assume that a team’s output is a linear sum of individual 'productivities'." (Laurent Bossavit, "The Leprechauns of Software Engineering", 2015)
"The problem, as we know, is that projects are very different from each other: there are big projects and large projects, there are projects with lots of defects and projects with… even greater numbers of defects. How do we make these comparable with each other?" (Laurent Bossavit, "The Leprechauns of Software Engineering", 2015)
"The wonderful thing about over-precise statistics is that
they are the easiest to expose as being Leprechauns, as you can trace their
spread from one dubious source to another."
"We cannot leave this enterprise to academia: that system’s inertia is too great. I believe that many of us will have to become scientists, and study software development even as we practice it." (Laurent Bossavit, "The Leprechauns of Software Engineering", 2015)
"When looking at a graph or chart, purported to be backed by
empirical data, remember to ask: what specific measurement does each of the
points I’m looking at represent? If the picture is a curve, and it has many
more data points than were actually measured, ask: was the method for
interpolating the missing points actually valid?"