Why and How to Profit from Software Engineering Research?

Software engineers are inherently biased. We can see this by looking at the many ideological flame wars concerning whether to use spaces or tabs, where to put the curly braces, which operating system to use, which development environment/text editor to use, and the list can go on and on.

However, there are critical choices to be made, which require more reasoning and objectivity. This is especially true when designing or defining things which influence more than one project but instead many projects or a whole organization, e.g., guidelines for modelling and coding, the structure of requirements, ways of documenting design choices, general properties of architectural styles like microservices. However, our understanding of these things is limited, and we are trapped in subjective “evaluations.”

Do we really know that microservices are “better”? And if yes in what regard? What does agile software development improve? How can we benefit from UML models? Or do we not? Should we define service contracts? If yes, how? What are efficient ways of testing distributed applications?

The sad truth is that we do not know the answers - although everyone pretends to. Large amounts of money are spent based on assumptions but not facts. As such, good (empirical) research should be desirable for every professional software engineer to better understand the things, which we are using. By this I do not mean that everyone should become a researcher, but everyone should be interested in empirical research results and learn from them. Of course, if you can afford, collaborating with research organizations can improve yourself, your organization, and our field as a whole.

Good research applicable in practice is hard to find. In my opinion this has to do with missing collaboration. Research institutions are good at doing research designs, applying research methods, and hopefully staying objective. However, they usually (and naturally) lack extensive experience in applying the researched technologies in complex software development projects. This has led to bad research, which I find frustrating, because the research addresses problems which are important to my daily work, but often after reading the first page of an academic article, I put it to the side: the authors have demonstrated that they don’t know important technical aspects, thus did not consider them, and all the results are only of theoretical interest and the resulting interpretations are too often wrong.

I tried to summarize the different sides in a keynote at ZEUS back in 2017:

To advance our field, we must establish a culture of research and industry cooperation. Professional software engineers need to provide knowledge to researchers. We must establish a feedback cycle between research and industry who are ideally suited to criticize each other and help improving the state of the art collectively.

I have set up some of such collaborations because I was a) lucky to work in a project that is open and keen to learn and improve, and b) still have connections to the academic field including being an external researcher at my former research group. This way I and my projects learnt a lot, which helped to being successful and helped me getting better: We learnt how to better create tests. We learnt to focus more on data-flow than on process-flow in service orchestrations. And, We can prove the correctness of the underlying assumptions!

However, not everyone wants to do or has the time to do research. And (young) researchers ask themselves how to get in touch with professionals. What can everyone of us do?

As a software development professional: I think the action, which you can take that requires the least effort, is to participate in surveys. No, not these marketing surveys, which are only designed to get your emails and send you offers without that you have profited from the survey itself. I do mean academic surveys, which have a solid research design and will send you a report (or other publications) with the results, which you can use in your daily work.

As a researcher: Contact companies or professionals to validate and/or participate in your research. Be very careful with their time because it is limited and precious (you won’t be able to pay their hourly rates so honor their time if they volunteer to work with you). Also tell them what they will get back from you and at least give them an option to leave their email address and remember to send them the final evaluation (at least the papers that you write; an industrial summary would be better).

I like to improve the current state of software development and thus want to help you: Professionals can register to my mailing list for empirical studies to get informed about studies in which I am interested or involved or which I have checked and considered to be no waste of my and your time. No more than one email a month – guaranteed! As a researcher you can send me surveys together with the survey design and the promise to email the research results to me and all participants (Broken promises will get you a life-time ban.)

As an example, you can see my quiz about the usage of different BPMN constructs. It is simple, quick to do (5 minutes), and will teach you about BPMN options and show what you prefer. After the research is done, I will send you the results, if you have left an email address. Feel free to try it out! The idea of quizzes being used in research originated as part of a Tobias Scholz' Master's thesis, which I supervised. The results for subjective preferences of BPMN layouts are available here. Engaging in Bachelor's and Master's thesis to begin to cooperate with universities might also be a consideration for you?!

If you have interesting and larger problems for which you require more objective answers, you should also think about contacting a university or another research institution to evaluate options or to set up a research project. This will likely require financial investment but might be worth it if your problem is solved. Very often researchers have access to other research means. For example, eye-tracking is a remarkably interesting way of analyzing the usage of models, documents, or user interfaces.

I think both sides - professionals and researchers - need to improve and concentrate on establishing more collaborations to advance the knowledge in our field. I would like to see this happen and just for reference I present you two books that present empirical research useful in day-to-day software development activities (Warning: I am editor of the second one.)

• A. Oram, G. Wilson: Making Software: What Really Works, and Why We Believe It, O'Reilly and Associates, 2010.
• D. Lübke, C. Pautasso: Empirical Studies on the Development of Executable Business Processes, Springer, 2019.