Open-Set Intrusion Detection and Semantic Analysis of Zero-Day Network Attacks Using Deep Reinforcement Learning and Large Language Models

Thursday, July 02, 2026 at 12:30pm to 1:30pm

Zoom - Online
Gokhan Kul
gkul@umassd.edu
https://teams.microsoft.com/meet/258262216614270?p=5gSDREaPiTObpT1eqG

Thesis Advisor: Gokhan Kul, Department of Computer and Information Science

Committee Members:

• Ashokkumar Patel, Department of Computer and Information Science
• Adnan El-Nasan, Department of Computer and Information Science

Abstract:

Zero-day cyberattacks pose a major challenge to traditional Intrusion Detection Systems (IDS) because previously unseen attacks are not represented in training data and are often misclassified. While recent open-set recognition methods can identify unknown traffic, they typically provide limited insight into the nature of detected anomalies. This thesis presents a unified framework for open-set intrusion detection and semantic analysis of unknown network traffic by integrating deep learning, reinforcement learning, and large language models (LLMs). The proposed approach uses a Convolutional Neural Network (CNN) to learn traffic representations and a Deep Q-Network (DQN) to distinguish known from unknown traffic using uncertainty-based metrics without manually defined thresholds. An LLM reasoning module is selectively applied to traffic identified as unknown to generate interpretable behavioral explanations. Experiments on the CICIDS-2017 and UNSW-NB15 datasets demonstrate that the CNN-DQN framework achieves a binary F1-score of 97.83% for known-versus-unknown traffic classification while effectively identifying previously unseen attacks. The LLM-assisted analysis further provides meaningful behavioral interpretations of suspicious network activity, improving the explainability of intrusion detection outcomes. The proposed framework contributes to the development of adaptive and explainable intrusion detection systems capable of identifying and interpreting emerging cyber threats, supporting faster incident response and enhanced cybersecurity decision-making.

For further information please contact Dr. Gokhan Kul at gkul@umassd.edu