Author Archives: mauricioaniche

MSR 2017 – To Mock or Not To Mock? An Empirical Study on Mocking Practices

Davide Spadini, M. Finavaro Aniche, Magiel Bruntink, Alberto Bacchelli

When writing automated unit tests, developers often deal with software artifacts that have several dependencies. In these cases, one has the possibility of either instantiating the dependencies or using mock objects to simulate the dependencies’ expected behavior. Even though recent quantitative studies showed that mock objects are widely used in OSS projects, scientific knowledge is still lacking on how and why practitioners use mocks. Such a knowledge is fundamental to guide further research on this widespread practice and inform the design of tools and processes to improve it. The objective of this paper is to increase our understanding of which test dependencies developers (do not) mock and why, as well as what challenges developers face with this practice. To this aim, we create MOCKEXTRACTOR, a tool to mine the usage of mock objects in testing code and employ it to collect data from three OSS projects and one industrial system. Sampling from this data, we manually analyze how more than 2,000 test dependencies are treated. Subsequently, we discuss our findings with developers from these systems, identifying practices, rationales, and challenges. These results are supported by a structured survey with more than 100 professionals. The study reveals that the usage of mocks is highly dependent on the responsibility and the architectural concern of the class. Developers report to frequently mock dependencies that make testing difficult and prefer to not mock classes that encapsulate domain concepts/rules of the system. Among the key challenges, developers report that maintaining the behavior of the mock compatible with the behavior of original class is hard and that mocking increases the coupling between the test and the production code.


SIGCSE 2017 – A Collaborative Approach to Teaching Software Architecture

Arie van Deursen, Maurício Aniche, Joop Aué, Rogier Slag, Michael de Jong, Alex Nederlof, Eric Bouwers


Teaching software architecture is hard. The topic is abstract and is best understood by experiencing it, which requires proper scale to fully grasp its complexity. Furthermore, students need to practice both technical and social skills to become good software architects. To overcome these teaching challenges, we developed the Collaborative Software Architecture Course. In this course, participants work together to study and document a large, open source software system of their own choice. In the process, all communication is transparent in order to foster an open learning environment, and the end-result is published as an online book to benefit the larger open source community.

We have taught this course during the past four years to classes of 50-100 students each. Our experience suggests that: (1) open source systems can be successfully used to let students gain experience with key software architecture concepts, (2) students are capable of making code contributions to the open source projects, (3) integrators (architects) from open source systems are willing to interact with students about their contributions, (4) working together on a joint book helps teams to look beyond their own work, and study the architectural descriptions produced by the other teams.

Download: Paper

SCAM 2016 – SATT: Tailoring Code Metric Thresholds for Different Software Architectures

Maurício Aniche, Christoph Treude, Andy Zaidman, Arie van Deursen, Marco Aurélio Gerosa
16th IEEE International Working Conference on Source Code Analysis and Manipulation

Code metric analysis is a well-known approach for assessing the quality of a software system. However, current tools and techniques do not take the system architecture (e.g., MVC, Android) into account. This means that all classes are assessed similarly, regardless of their specific responsibilities. In this paper, we propose SATT (Software Architecture Tailored Thresholds), an approach that detects whether an architectural role is considerably different from others in the system in terms of code metrics, and provides a specific threshold for that role. We evaluated our approach on 2 different architectures (MVC and Android) in more than 400 projects. We also interviewed 6 experts in order to explain why some architectural roles are different from others. Our results shows that SATT can overcome issues that traditional approaches have, especially when some architectural role presents very different metric values than others.

Download the paper

Maus cheiros em códigos de teste… Eles existem!

Acabei de ler um paper muito legal sobre code smells em código de testes [1]. Resumindo aqui os resultados:

  1. Maus cheiros em testes não são percebidos pelos desenvolvedores como problemas reais de código. Ou seja, desenvolvedores não costumam enxergá-los como problemas.
  2. Na maioria dos casos, as classes de teste são afetadas pelos smells desde o momento da sua criação. Ou seja, a maioria dos testes mal escritos foram mal escritos desde o começo.
  3. Os maus cheiros tem uma taxa de sobrevivência altíssima. Ou seja, o mau cheiro fica lá no código por muito tempo, até alguém decidir refatorá-lo.
  4. Existe uma relação entre maus cheiros no código de teste e maus cheiros no código de produção. Ou seja, código de produção ruim faz com que desenvolvedores escrevam testes problemáticos também.

Você quer ler mais sobre possíveis maus cheiros em código de teste? Então veja o excelente paper do Arie van Deursen et al. [2], onde eles listam diversos maus cheiros que podem acontecer nesse tipo de classe.

[1] Tufano, M., Palomba, F., Bavota, G., Di Penta, M., Oliveto, R., De Lucia, A., Poshyvanyk, D.Towards Automated Tools for Detecting Test Smells: An Empirical Investigation into the Nature of Test Smells. In 31st Automated Software Engineering, 2016. Link aberto pro paper.

[2] A. van Deursen, L. Moonen, A. Bergh, and G. Kok. Refactoring test code. In Proceedings of the 2nd International Conference on Extreme Programming and Flexible Processes in Software Engineering (XP), pages 92–95, 2001. Link aberto pro paper.

ICSME 2016 – A Validated Set of Smells in Model-View-Controller Architectures

Maurício Aniche, Gabriele Bavota, Christoph Treude, Arie van Deursen, Marco Aurélio Gerosa
32nd IEEE International Conference on Software Maintenance and Evolution

Code smells are symptoms of poor design and implementation choices that may hinder code comprehension, and possibly increase change- and defect-proneness. A vast catalogue of smells has been defined in the literature, and it includes smells that can be found in any kind of system (e.g., God Classes), regardless of their architecture. On the other hand, software systems adopting specific architectures (e.g., the Model-View-Controller pattern) can be also affected by other types of poor practices.

We surveyed and interviewed 53 MVC developers to collect bad practices to avoid while working on Web MVC applications. Then, we followed an open coding procedure on the collected answers to define a catalogue of six Web MVC smells, namely \textsc{Smart Repository, Fat Repository, Promiscuous Controller, Smart Controller, Laborious Repository Method}, and \textsc{Meddling Service}. Then, we ran a study on 100 MVC projects to assess the impact of these smells on code change- and defect-proneness. In addition, we surveyed 21 developers to verify their perception of the defined smells. The achieved results show that the Web MVC smells (i) more often than not, increase change- and defect-proneness of classes, and (ii) are perceived by developers as severe problems.

Download the paper

SBES 2016 – Developers’ Perceptions on Object-Oriented Design and Architectural Roles

Maurício Aniche, Christoph Treude, Marco Aurélio Gerosa
30th Brazilian Conference on Software Engineering

Software developers commonly rely on well-known software architecture patterns, such as MVC, to build their applications. In many of these patterns, classes play specific roles in the system, such as Controllers or Entities, which means that each of these classes has specific characteristics in terms of object-oriented class design and implementation. Indeed, as we have shown in a previous study, architectural roles are different from each other in terms of code metrics. In this paper, we present a study in a software development company in which we captured developers’ perceptions on object-oriented design aspects of the architectural roles in their system and whether these perceptions match the source code metric analysis. We found that their developers do not have a common perception of how their architectural roles behave in terms of object-oriented design aspects, and that their perceptions also do not match the results of the source code metric analysis. This phenomenon also does not seem to be related to developers’ experience. We find these results alarming, and thus, we suggest software development teams to invest in education and knowledge sharing about how their system’s architectural roles behave.

Download the paper

Is TDD magic?

Will I create a better class design if I use Test-Driven Development? Well, yes and no; TDD doesn’t do magic. In this article, we discuss the effects of the practice, and how the practice can help developers during class design. We do it by interviewing developers from different companies.

Developers commonly argue that TDD helps software design, improving internal code quality. For example, Kent Beck [4], Robert Martin [5], Steve Freeman [6], and Dave Astels [7], state in their books (without scientific evidence) that the writing of unit tests in a TDD fashion promotes significant improvement in class design, helping developers to create simpler, more cohesive, and less coupled classes. They even consider TDD as a class design technique [8], [9]. Nevertheless, the way in which TDD actually guides developers during class design is not yet clear.

We ask: What are the developers’ perceptions on the effects of Test-Driven Development in class design?

We invited developers from the Brazilian software development industry to be part of this study. We had 14 participants from 6 different companies.

TDD does not drive to a better design by itself

Participants affirmed that the practice of TDD did not change their class design during the experiment. The main justification was that their experience and previous knowledge regarding object-orientation and design principles guided them during class design. They also affirmed that a developer with no knowledge in object-oriented design would not create a good class design just by practicing TDD or writing unit tests.

The participants gave two good examples reinforcing the point. One of them said that he made use of a design pattern [30] he learned a few days before. Another participant mentioned that his studies on SOLID principles helped him during the exercises.

The only participant who had never practiced TDD before stated that he did not feel any improvement in the class design when practicing the technique. Curiously, this participant said that he considered TDD a design technique. It somehow indicates that the popularity of the effects of TDD in class design is high. That opinion was slightly different from that of experienced participants, who affirmed that TDD was not only about design, but also about testing.

However, different from the idea that TDD and unit tests do not guide developers directly to a good class design, all participants said that TDD has positive effects on class design. Many of them mentioned the difficulty of trying to stop using TDD or thinking about tests, what can be one reason for not having significant difference in terms of design quality in the code produced with and without TDD.

“When you are about to implement something, you end up thinking about the tests that you’ll do. It is hard to think “write code without thinking about tests!”. As soon as you get used to it, you just don’t know another way to write code… ”

According to them, TDD can help during class design process, but in order to achieve that, the developer should have certain experience in software development. Most participants affirmed that their class designs were based on their experiences and past learning processes. In their opinion, the best option is to link the practice of TDD and experience.

Baby steps and simplicity

TDD suggests developers to work in small (baby) steps; one should define the smallest possible functionality, write the simplest code that makes the test green, and do one refactoring at a time [4].

In the interviews, participants commented about this. One of them mentioned that, when not writing tests, a developer thinks about the class design at once, creating a more complex structure than needed.

One of the participants clearly stated how he makes use of baby steps, and how it helps him think better about his class design:

“Because we start to think of the small and not the whole. When I do TDD, I write a simple rule (…), and then I write the method. If the test passes, it passes! As you go step by step, the architecture gets nice. (…) I used to think about the whole (…). I think our brain works better when you think small. If you think big, it is clear, at least for me, that you will end up forgetting something. ”

Refactoring confidence

Participants affirmed that, during the process of class design, changing minds is constant. After all, there is still a small knowledge about the problem, and about how the class should be built. This was the most mentioned point by the participants. According to them, an intrinsic advantage of TDD is the generated test suite. It allows developers to change their minds and refactor all the class design safely. Confidence, according to them, is an important factor when changing class design or even implementation.

“It gives me the opportunity to learn along the way and make things differently. (…). The test gives you confidence. ”

A participant even mentioned a real experience, in which TDD made the difference. According to him, he changed his mind about the class design many times and trusted the test suite to guarantee the expected behavior.

A safe space to think

In an analogy done by one of the participants, tests are like draft paper, in which they can try different approaches and change their minds about it frequently. According to them, when starting by the test, developers are, for the first time, using their own class. It makes developers look for a better and clearer way to invoke the class’ behaviors, and facilitate its use:

“Tests help you on that. They are a draft for you to try to model it the best way you can. If you had to model the class only once, it is like if you have only one chance. Or if you make it wrong, fixing it would give you a lot of work. The idea of you having tests and thinking about them, it is like if you have an empty draft sheet, in which you can put and remove stuff because that stuff doesn’t even exist.”

We asked their reasons for not thinking on the class design even when they were not practicing TDD or writing tests. According to them, when a developer does not practice TDD, they get too focused on the code they are writing, and thus, they end up not thinking about the class design they were creating. They believe tests make them think of how the class being created will interact with the other classes and of the easiness of using it.

One of the participants was even more precise in his statement. According to him, developers that do not practice TDD, as they do not think about the class design they are building, they end up not doing good use of OOP. TDD forces developers to speed down, allowing them to think better about what they are doing.

Rapid feedback

Participants also commented that one difference they perceived when they practiced TDD was the constant feedback. In traditional testing, the time between the production code writing and test code writing is too long. When practicing TDD, developers are forced to write the test first, and thus receive the feedback a test can provide sooner.

One participant commented that, from the test, developers observe and criticize the code they are designing. As the tests are done constantly, developers end up continuously thinking about the code and its quality:

“When you write the test, you soon perceive what you don’t like in it (…). You don’t perceive that until you start using tests.”

Reducing the time between the code writing and test writing also helps developers to create code that effectively solves the problem. According to the participants, in traditional testing, developers write too much code before actually knowing if it works.

The search for testability

Maybe the main reason for the practice of TDD helping developers in their class design is the constant search for testability. It is possible to infer that, when starting to produce code by its test, the production code should be, necessarily, testable.

When it is not easy to test a specific piece of code, developers understand it as a class design smell. When this happens, developers usually try to refactor the code to make it easier to test. A participant also affirmed that he takes it as a rule; if it is hard to test, then the code may be improved.

Feathers [33] raised this point: the harder it is to write the test, the higher the chance of a class design problem. According to him, there is a strong synergy between a highly testable class and a good class design; if developers are looking for testability, they end up creating good class design; if they are looking for good class design, they end up writing testable code.

Participants went even further. During the interviews, many of them mentioned patterns that made (and make) them think about possible design problems in the class they build. As an example, they told us that when they feel the need to write many different unit tests to a single method, this may be a sign of a non-cohesive method. They also said that when a developer feels the need to write a big scenario for a single class or method, it is possible to infer that this need emerges in classes dealing with too many objects or containing too many responsibilities, and thus, it is not cohesive. They also mentioned how they detect coupling issues. According to them, the abusive use of mocking objects indicates that the class under testing has coupling issues.

What did we learn from it?

The first interesting myth contested by the participants was the idea that the practice of TDD would drive developers towards a better design by itself. As they explained, the previous experience and knowledge in good design is what makes the difference; however, TDD helps developers by giving feedback by means of the unit tests that they are writing constantly. As they also mentioned, the search for testability also makes them rethink about the class design many times during the day — if a class is not easy to be tested, then they refactor it.

We agree with the rationale. In fact, when comparing to test-last approaches, developers do not have the constant feedback or the need to write testable code. They will have the same feedback only at the end, when all the production code is already written. That may be too late (or expensive) to make a big refactoring in the class design.


We also agree with the confidence when refactoring. As TDD forces developers to write unit tests frequently, those tests end up working as a safety net. Any broken change in the code is quickly identified. This safety net makes developers more confident to try and experiment new design changes — after all, if the changes break anything, tests will warn developers about it. That is why they also believe the tests are a safe space to think.

Therefore, we believe that is is not the practice by itself that helps developers to improve their class design; but it is the consequences of the constant act of writing a unit test, make that class testable, and refactor the code, that drives developers through a better design.


To answer the research question: What are the developers’ perceptions on the effects of Test-Driven Development in class design?

Developers believe that the practice of test-driven development helps them to improve their class design, as the constant need of writing a unit test for each piece of the software forces them to create testable classes. These small feedbacks — is your test easy to be tested or not? — makes them think and rethink about the class and improve it. Also, as the number of tests grow, they act as a safety net, allowing developers to refactor freely, and also try and experiment different approaches to that code. Based on that, we suggest developers to experiment the practice of test-driven development, as its effects look positive to software developers.

If you wanna read the full journal paper or need the references:

O que são testes de mutantes?

Como garantir que nossa bateria de testes está boa? Essa é uma pergunta importante, afinal de nada adianta termos testes automatizados se eles não forem bons o suficiente. Uma maneira bem conhecida é calcular a cobertura do código. Ferramentas como o Cobertura dizem pra você quais linhas do seu código de produção não são executadas pela sua bateria de teste. Assim, você pode facilmente ver qual if você esqueceu de testar, e escrever um teste pra ele.

Mas essa não é a única maneira. Uma delas, bastante interessante e muito popular na academia (mas não tão popular na indústria) é o teste de mutante. A ideia é genial. Imagine que tenhamos testes verdes para o seguinte trecho de código:

Veja que o if verifica se a é MAIOR que 10. O que deveria acontecer com os testes se mudássemos o sinal de MAIOR para MENOR? Nossa bateria de testes deveria quebrar, certo? Pois é exatamente isso que testes de mutantes fazem. Um mutante é simplesmente uma versão alterada do seu código de produção. Alterar o sinal de maior para menor é uma maneira de gerar um mutante. Mudar o sinal de + pra -, * por /, e assim por diante. E a ideia é que, se rodarmos os testes trocando a classe por um mutante, algum teste deve falhar. Afinal, o mutante faz coisa errada!

Ferramentas como o Pitest pegam seu código de produção, geram diferentes mutantes, e executam a bateria de testes. Se algum teste falha, dizemos então que o mutante foi morto. Agora, se algum mutante sobrevive, então você precisa melhorar seus testes.

Para exemplificar, veja meu repositório pitest-fizzbuzz. Ele tem uma simples implementação do jogo Fizz Buzz. É o mesmo exemplo que usei na pequena série de começando com testes, aqui no meu blog. Você entenderá o jogo pelo código:

Veja os testes dele (que fizemos juntos no blog):

A bateria de testes parece bem completa. Veja o resultado do relatório emitido pelo Pitest. Tudo verde, ou seja todos os mutantes foram mortos (ahá, eu mandei bem quando escrevi os testes! 🙂


Agora, vamos supor que tivéssemos esquecido o teste deveRetornarFizzBuzzQuandoMultiploDe5e7(). Olha só o relatório. Ele nos mostra bem a linha do código de produção onde ele fez a mutação (nesse caso, as mutações), a descrição, e mostra que o mutante sobreviveu. Nesse caso, ele mudou o % por *, e nenhum teste pegou.


Até aí, parece que o cobertura também pegaria isso. Afinal, essa linha não foi coberta. No entanto, testes de mutantes vão além. Mas imagine uma nova regra no jogo: Números maiores ou iguais a 500 imprimem “big”. Implementamos:

E adicionamos o teste:

Agora, a cobertura desse código é 100%.


No entanto, veja o teste de mutante. Ele pegou! Afinal, não testamos o caso do igual no maior-ou-igual. Uma mutação ali (trocar por exemplo para só maior), e o mutante sobrevive!


Testes de mutantes são bastante interessantes, e ainda muito desafiadores para a academia. Afinal, em um sistema grande, o número de mutantes pode ser muito grande. Como encontrar os melhores mutantes? Como remover mutantes redundantes? No site do pitest, você pode ler um pouco mais sobre mutantes e ver as operações de mutação que ele implementa.

A ferramenta tem plugin para o Maven, então é fácil de executar. No repositório de exemplo, o pom.xml está configurado.

Como você usa pré-processadores CSS?

Pré-processadores CSS estão na moda. Afinal, eles dão poder para CSS que, por natureza são simples, mas nada fáceis de manter. O pessoal da Universidade de Concórdia acabou de fazer um estudo empírico bem legal sobre o uso de SASS e LESS.

De forma resumida, os resultados encontrados foram:

  • Desenvolvedores declaram principalmente variáveis globais (89% dos casos), especialmente para armazenar cores (45% dos casos).
  • Aninhamento (nesting) é bastante popular. 78% dos seletores são aninhados.
  • 2/3 dos mixins são reusados duas ou mais vezes. Interessantemente, 42% deles são usados para lidar com propriedades específicas de browsers (-webkit-*, etc).
  • Desenvolvedores preferem mixins sem parâmetros do que usar herança.

O artigo vai mais a fundo, sugerindo razões e possíveis ferramentas focadas para ajudar desenvolvedores nessas situações (refatorações, por exemplo). Você pode ler mais no preprint do artigo que será apresentado no SANER 2016:

E aí? Os números são o que você esperava? Ou você faz diferente?