Boosting URL detection with syntactic features in spam emails

Friday 26 September 11:00 - 11:30, Red room

Antonia Scherz (Net at Work)

Traditional URL blocklisting methods often miss automated techniques used in spam URL generation. For example, attackers may use URLs with newly registered domains and similar paths or query parameters that often differ only by the domain and a random string. This dynamic variation of domains frequently evades detection. However, using syntactic patterns to identify spam URLs directly addresses this issue and classifies many spam domains at once.

In this work, we present a practical approach to identifying structural patterns in URLs generated by automation. Our goal is to enhance URL detection systems and reduce the risk of phishing, malware, and spam threats. The presented technique complements existing detection methods, which mainly focus on character-level (e.g. frequencies of special characters) or content-based features (e.g. suspicious domain names, email body text). Our multi-step approach enables us to link seemingly dissimilar URLs to the same spam campaign origin, even when traditional detection techniques fail.

We propose a four-step procedure to include syntactic pattern recognition:

1. To extract syntactic patterns from the URLs using regex representation
2. To pre-filter syntactic patterns based on similarity in n-gram usage
3. To cluster prefiltered patterns based on Locality Sensitive Hashing and DBSCAN (Density-Based Spatial Clustering of Applications with Noise)
4. To classify URLs in each cluster as malicious or benign based on cluster information and additional (traditional) email metadata.

We apply this technique to a large corpus of URLs extracted from email bodies, demonstrating where syntactic features excel and fail. Our results show that combining URL structural analysis with contextual email metadata – such as sender reputation and email headers – improves the robustness of our current spam email detection systems, as validated on a dataset of over 1 million emails. This approach also provides additional signals to identify suspicious campaigns that evade traditional methods. By leveraging these insights, organizations can more effectively detect, mitigate, and adapt to evolving spam and phishing threats.

Back to VB2025 Programme page

Back to VB2025 conference page

Other VB2025 papers