Martin Weyssow's Homepage

Short Bio

I am currently a Ph.D. Candidate in the Department of Computer Science and Operations Research at the University of Montreal. I am fortunate to be advised by Prof. Houari Sahraoui. From October 2022 to January 2023, I was a visiting Ph.D. researcher at Singapore Management University, working with Prof. David Lo. Prior to this, I obtained my master’s degree in data science and a bachelor’s degree in computer science and mathematics from the University of Namur.

My research lies at the intersection of deep learning, natural language processing, and software engineering, focusing on large language models (LLMs) for code intelligence. Specifically, my thesis revolves around understanding and efficiently adapting LLMs for software coding tasks. My first research objective is to investigate the intricate training dynamics of LLMs when exposed to code, aiming to enhance our comprehension of phenomena such as emergent capabilities in LLMs and generalization. The second aspect of my thesis involves exploring data-centric applications related to out-of-distribution generalization and lifelong learning for LLMs applied to code. This encompasses studying scenarios where important distribution shifts of the data may impede the effectiveness of LLMs and how to adapt LLMs to new codebases over time without forgetting prior knowledge.

News

[May. 2023] Our paper on continual learning and out-of-distribution generalization for PLMs of code is accepted at FSE 2023.
[Oct. 2022] I am starting a new visiting Ph.D. researcher position at SCIS, Singapore Management University under the supervision of Prof. David Lo.
[Jul. 2022] Our paper on syntax probing for pre-trained language models of code is accepted at ASE 2022.
[May 2022] I was awarded a prestigious FRQ Merit Scholarship for Foreign Students (PBEEE) issued by the Government of Quebec to fund my Ph.D.
[Mar. 2022] I am serving as a PC member of the Deep Learning for Code workshop at ICLR 2022.
[Dec. 2021] Our paper about multimodal learning with concept graphs for code is accepted at ICSE-NIER 2022.
[Dec. 2021] Our paper about pre-trained language models for MDE is accepted to SOSYM journal AI-MDE theme issue.
[May 2021] I am recipient of a Google Scholarship for Excellence offered by DIRO, University of Montreal funding 1-year of full-time research.
[Sep. 2020] I am officially starting my Ph.D. program under the supervision of Prof. Houari Sahraoui.
Show more

Selected Publications [Google Scholar]

Under review

Exploring Parameter-Efficient Fine-Tuning Techniques for Code Generation with Large Language Models

Martin Weyssow*, Xin Zhou, Kisub Kim, David Lo, Houari Sahraoui

Under review

PDF

ESEC/FSE

On the Usage of Continual Learning for Out-of-Distribution Generalization in Pre-trained Language Models of Code

Martin Weyssow*, Xin Zhou, Kisub Kim, David Lo, Houari Sahraoui

ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 2023.

PDF Code

ASE

AST-Probe: Recovering abstract syntax trees from hidden representations of pre-trained language models

José Antonio Hernández López*, Martin Weyssow*, Jesús Sánchez Cuadrado, Houari Sahraoui (*Equal contributions)

International Conference on Automated Software Engineering, 2022.

PDF Code

ICSE-NIER

Better modeling the programming world with code concept graphs-augmented multi-modal learning

Martin Weyssow, Houari Sahraoui, Bang Liu

International Conference on Software Engineering, New Ideas and Emerging Results, 2022.

PDF

SOSYM

Recommending metamodel concepts during modeling activities with pre-trained language models

Martin Weyssow, Houari Sahraoui, Eugene Syriani

Software and Systems Modeling, 2022.

PDF Code

Students Supervision

Aton Kamanda, MSc. at the University of Montreal, 2021-2023.
Adaptive computation strategies for deep neural networks
Lucas Maes, MSc. at the University of Montreal, 2021-2023.
Self-supervised learning for code representation learning.

Services

Program Committee

Workshop on Deep Learning for Code at ICLR 2022

Conference Reviewers

Journal Reviewers

Contact

Email: martin.weyssow (at) umontreal.ca