Master's thesis; Adversarial Purification for Large Audio-Language Models

Background
Large audio-language models (LALMs) are moving from demos to production for assistants, meetings, and accessibility. Yet they remain vulnerable to adversarial audio that corrupts transcription, reasoning, and safety layers. Recent benchmarks show that seemingly benign edits, including content tweaks, emotional style shifts, and common acoustic transformations such as reverberation or whisper-like effects, can degrade multiple models’ performance. Multilingual and multi-accent jailbreaks further expose weaknesses that text prompts fail to capture. Existing defenses are either attack/model-specific or computationally heavy when operating directly on the audio signal during inference. To tackle this gap, we aim to propose an efficient purification pipeline for LALMs that maintains task-relevant signal while removing attack artifacts, where generative models (e.g. diffusion or autoregressive models) are applied for signal-space denoising and latent-space enhancing.

Objectives
This thesis develops an inference-time and attack/model-agnostic adversarial purification framework for LALMs. The project includes:

• Review existing threat models, benchmarks, and defense methods for LALMs, including audio jailbreaks and perturbation-based settings.
• Evaluate the adversarial robustness of open and commercial LALMs (e.g., Gemini-1.5-Pro, GPT-4o) with standardized safety/utility metrics.
• Implement signal-space denoising (e.g., DSP filtering) and generative purification (e.g., diffusion resynthesis) mechanisms.
• Design adaptive purification strategies and compare with other defense baselines, analyze robustness–latency–fidelity trade-offs.
• Deliver an open-source repository suitable for real-world audio applications and a written report on the findings.

Development on clusters with the NVIDIA GPUs. A successful project could lead to an academic publication presented at a prestigious conference or workshop such as ICASSP.

Candidate profile
We expect you are strong programmer in Python (PyTorch preferred) and have:

• Knowledge of speech/audio deep learning and digital signal processing (DSP).
• Experience on adversarial robustness and LLM toolchains (e.g., Hugging Face stacks).
• Familiarity with reproducible software Git/Docker and cloud-based training.
• Solid oral and written English skills.
  

Terms

• Recruiting manager: Sepideh Pashami, PhD (Sepideh.pashami@ri.se)
• Supervisor: Jia Fu, PhD candidate (jia.fu@ri.se)
• Examiner needs to be contacted by master student at the university
• Company: RISE Research Institutes of Sweden
• Location: RISE Computer Science Department, Kista
• Application deadline: November 1^st, 2025
• Starting date: As soon as possible, no later than December 1^st, 2025
• Credits: 30 HP
• Compensation: 30,000 SEK upon a successful completion of a high-quality thesis.
  

Applications will be reviewed on a rolling basis.
Send in your application with the following materials:

• Your CV with education background, professional experience, and specific skills.
• A written report you authored or co-authored for a second-cycle course.
• Demos of previous project or samples of relevant code.
• Recent grades on the academic transcript.