Giulia Sellitto

Early and Realistic Exploitability Prediction of Just-Disclosed Sotware Vulnerabilities: How Reliable Can It Be?

Emanuele Iannone, Giulia Sellitto, Emanuele Iaccarino, Filomena Ferrucci, Andrea De Lucia, Fabio Palomba

ACM Transactions on Software Engineering and Methodology

With the rate of discovered and disclosed vulnerabilities escalating, researchers have been experimenting with machine learning to predict whether a vulnerability will be exploited. Existing solutions leverage information unavailable when a CVE is created, making them unsuitable just after the disclosure. This paper experiments with early exploitability prediction models driven exclusively by the initial CVE record, i.e., the original description and the linked online discussions. Leveraging NVD and Exploit Database, we evaluate 72 prediction models trained using six traditional machine learning classiiers, four feature representation schemas, and three data balancing algorithms. We also experiment with ive pre-trained large language models (LLMs). The models leverage seven different corpora made by combining three data sources, i.e., CVE description, Security Focus, and BugTraq. The models are evaluated in a realistic, time-aware fashion by removing the training and test instances that cannot be labeled neutral with sufficient confidence. The validation reveals that CVE descriptions and Security Focus discussions are the best data to train on. Pre-trained LLMs do not show the expected performance, requiring further pre-training in the security domain. We distill new research directions, identify possible room for improvement, and envision automated systems assisting security experts in assessing the exploitability.

Citation • PDF • Online Library

Toward the Deployment of a Chatbot to Augment Computer Science Education

Giusy Annunziata, Giulia Sellitto, Stefano Lambiase, Emanuele Bruno, Gabriele De Vito, Filomena Ferrucci

1st Workshop on Artificial Intelligence with and for Learning Sciences: Past, Present, and Future Horizons (WAILS 2024)

Over the last few years, human life has been significantly improved by many technological innovations, especially due to considerable advances in the field of Artificial Intelligence. In particular, the most noticeable impact of such progress on everyday life is given by Conversational Agents, also known as Chatterbots, and later shortened to Chatbots. Chatbots in education are becoming more and more popular due to the advantages they guarantee toward personalized learning, ease of use, and accessibility. Such benefits have been observed in various contexts, from primary and secondary education to university and vocational training. One of the most recent chatbots proposed in the literature is Hermias, presented by Petousi et al., which is aimed at helping high school students in the learning of History. We aim to follow the path traced by Petousi et al., by developing a chatbot with three-fold intentions, i.e., we are interested in (1) encouraging the engagement of high school students in the field of Computer Science, particularly about the topic related to networks, (2) improving their interest toward the history of Computer Science and the progress of this discipline over the decades, and (3) raising students’ awareness on women in Science, escaping the common belief that STEM subjects are mainly for males. Therefore, we propose CAIHL, a chatbot envisioned to help students understand the concepts of computer networks, with an eye on the history of the subject and its pioneers. In particular, CAIHL impersonates Hedy Lamarr, a leading figure in Computer Science who made substantial contributions to the field of networks. Through the persona of Hedy Lamarr, students can be captured in the learning of computer science, as the curiosity in her personal history acts as a Trojan horse, which initially engages students by means of history and trivia, and finally brings them to learn. We hypothesize that the employment of CAIHL in a real education environment can boost the effectiveness of lectures, by stimulating the curiosity of students and making them the protagonists of the learning.

Citation • PDF • Online Library

An Empirical Study on the Performance of Vulnerability Prediction Models Evaluated Applying Real-world Labelling

Giulia Sellitto, Alexandra Sheykina, Fabio Palomba, Andrea De Lucia

Joint Conference of the 32nd International Workshop on Software Measurement (IWSM) and the 17th International Conference on Software Process and Product Measurement (MENSURA) 2023

Software vulnerabilities are infamous threats to the security of computing systems, and it is vital to detect and correct them before releasing any piece of software to the public. Many approaches for the detection of vulnerabilities have been proposed in the literature; in particular, those leveraging machine learning techniques, i.e., vulnerability prediction models, seem quite promising. However, recent work has warned that most models have only been evaluated in in-vitro settings, under certain assumptions that do not resemble the real scenarios in which such approaches are supposed to be employed. This observation ignites the risk that the encouraging results obtained in previous literature may be not as well convenient in practice. Recognizing the dangerousness of biased and unrealistic evaluations, we aim to dive deep into the problem, by investigating whether and to what extent vulnerability prediction models' performance changes when measured in realistic settings. To do this, we perform an empirical study evaluating the performance of a vulnerability prediction model, configured with three data balancing techniques, executed at three different degrees of realism, leveraging two datasets. Our findings highlight that the outcome of any measurement strictly depends on the experiment setting, calling researchers to take into account the actuality and applicability in practice of the approaches they propose and evaluate.

Citation • PDF • Online Library

Fairness-aware Machine Learning Engineering: How Far Are We?

Carmine Ferrara, Giulia Sellitto, Filomena Ferrucci, Fabio Palomba, Andrea De Lucia

Empirical Software Engineering

Machine learning is part of the daily life of people and companies worldwide. Unfortunately, bias in machine learning algorithms risks unfairly influencing the decision-making process and reiterating possible discrimination. While the interest of the software engineering community in software fairness is rapidly increasing, there is still a lack of understanding of various aspects connected to fair machine learning engineering, i.e., the software engineering process involved in developing fairness-critical machine learning systems. Questions connected to the practitioners’ awareness and maturity about fairness, the skills required to deal with the matter, and the best development phase(s) where fairness should be faced more are just some examples of the knowledge gaps currently open. In this paper, we provide insights into how fairness is perceived and managed in practice, to shed light on the instruments and approaches that practitioners might employ to properly handle fairness. We conducted a survey with 117 professionals who shared their knowledge and experience highlighting the relevance of fairness in practice, and the skills and tools required to handle it. The key results of our study show that fairness is still considered a second-class quality aspect in the development of artificial intelligence systems. The building of specific methods and development environments, other than automated validation tools, might help developers to treat fairness throughout the software lifecycle and revert this trend.

Citation • PDF • Online Library

The Yin and Yang of Software Quality: On the Relationship between Design Patterns and Code Smells

Giammaria Giordano, Giulia Sellitto, Aurelio Sepe, Fabio Palomba, Filomena Ferrucci

49th Euromicro Conference Series on Software Engineering and Advanced Applications (SEAA 2023)

Software reuse is considered the silver bullet of software engineering. It has been largely demonstrated that the proper implementation of design and reuse principles can substantially reduce the effort, time, and costs required to develop software systems. Design patterns are one of the most affirmed techniques for source code reuse. While previous work pointed out their benefits in terms of maintainability and understandability, some seem to raise the opposite concern, suggesting that they can negatively impact code quality from the developers' perspectives. We recognize such discrepancy in the literature, and we aim to fill this gap by investigating whether and how design patterns are related to the emergence of issues compromising code understandability, namely the Complex Class, God Class, and Spaghetti Code smells, which have been also shown to increase the change- and fault-proneness of code. We perform an empirical evaluation on 15 Java projects evolving over 542 releases, and we find that, although design patterns are supposed to improve code quality without prejudice, they can be related to dangerous issues, as we observe the emergence of code smells in the classes participating in their implementation. From our findings, we distil a number of implications for developers and project managers to support them in dealing with design patterns.

Citation • PDF • Online Library

QuantuMoonLight: A low-code platform to experiment with quantum machine learning

Francesco Amato, Matteo Cicalese, Luca Contrasto, Giacomo Cubicciotti, Gerardo D’Ambola, Antonio La Marca, Giuseppe Pagano, Fiorentino Tomeo, Gennaro Alessio Robertazzi, Gabriele Vassallo, Giovanni Acampora, Autilia Vitiello, Gemma Catolino, Giammaria Giordano, Stefano Lambiase, Valeria Pontillo, Giulia Sellitto, Filomena Ferrucci, Fabio Palomba

SoftwareX

Nowadays, machine learning is being used to address multiple problems in various research fields, with software engineering researchers being among the most active users of machine learning mechanisms. Recent advances revolve around the use of quantum machine learning, which promises to revolutionize program computation and boost software systems’ problem-solving capabilities. However, using quantum computing technologies is not trivial and requires interdisciplinary skills and expertise. For such a reason, we propose QuantuMoonLight, a community-based low-code platform that allows researchers and practitioners to configure and experiment with quantum machine learning pipelines, compare them with classic machine learning algorithms, and share lessons learned and experience reports. We showcase the architecture and main features of QuantuMoonLight, other than discussing its envisioned impact on research and practice.

Citation • PDF • Online Library

Toward Understanding the Impact of Refactoring on Program Comprehension

Giulia Sellitto, Emanuele Iannone, Zadia Codabux, Valentina Lenarduzzi, Andrea De Lucia, Fabio Palomba, Filomena Ferrucci

29th International Conference on Software Analysis, Evolution, and Reengineering (SANER 2022)

Distinguished Paper Award

Software refactoring is the activity associated with developers changing the internal structure of source code without modifying its external behavior. The literature argues that refactoring might have beneficial and harmful implications for software maintainability, primarily when performed without the support of automated tools. This paper continues the narrative on the effects of refactoring by exploring the dimension of program comprehension, namely the property that describes how easy it is for developers to understand source code. We start our investigation by assessing the basic unit of program comprehension, namely program readability. Next, we set up a large-scale empirical investigation – conducted on 156 open-source projects – to quantify the impact of refactoring on program readability. First, we mine refactoring data and, for each commit involving a refactoring, we compute (i) the amount and type(s) of refactoring actions performed and (ii) eight state-of-the-art program comprehension metrics. Afterwards, we build statistical models relating the various refactoring operations to each of the readability metrics considered to quantify the extent to which each refactoring impacts the metrics in either a positive or negative manner. The key results are that refactoring has a notable impact on most of the readability metrics considered.

Citation • PDF • Online Library