Detecting DLL hijacking

To evade detection by security solutions, cybercriminals employ various techniques that mask their malicious activity. One of the methods increasingly seen in recent years in attacks on Windows systems is DLL hijacking: replacing dynamic-link libraries (DLLs) with malicious ones. And traditional security tools often don’t detect use of this technique. To solve this problem, our colleagues from the Kaspersky AI Technology Research Center developed a machine-learning model that can detect DLL hijacking with high accuracy. This model has already been implemented in the latest release of our SIEM system, the Kaspersky Unified Monitoring and Analysis Platform . In this post, we explain the challenges of detecting DLL hijacking, and how our technology addresses them.

How DLL hijacking works and why it’s hard to detect

The sudden launch of an unknown file in a Windows environment inevitably draws the attention of security tools — or is simply blocked. Essentially, DLL hijacking is an attempt to pass off a malicious file as a known and trusted one. There are several variations of DLL hijacking: one is when attackers distribute a malicious library along with legitimate software (DLL sideloading) so that the software executes it; another is when they replace standard DLLs that are called by already-installed programs on the computer; and there’s also when they manipulate system mechanisms that determine the location of the library that a process loads and executes. As a result, the malicious DLL file is launched by a legitimate process within its own address space and with its own privileges; thus the usual endpoint protection systems view this activity as looking legitimate. That’s why our experts decided to counter this threat with the use of AI technologies.

Detecting DLL hijacking with ML

AI Technology Research Center experts trained an ML model to detect DLL hijacking based on indirect information about the library and the process that called it. They identified key indicators of an attempt to manipulate a library: whether the executable file and the library are located in standard paths, whether the file was renamed, whether the library’s size and structure have changed, whether its digital signature is intact, and so on. They initially trained the model on data about dynamic link libraries’ being loaded — sourced from both internal automatic analysis systems and anonymized telemetry from the Kaspersky Security Network (KSN) voluntarily provided by our users. For labeling, our experts used data from our file reputation databases.

The first model was rather inaccurate, so before adding it to the solution our experts experimented through multiple iterations, refining both the labeling of the training dataset and the features that indicate DLL hijacking. As a result, the model now detects this technique with high accuracy. On Securelist, our colleagues published a detailed article about how they developed this technology — from the initial hypothesis, through testing in Kaspersky Managed Detection and Response, and finally to the practical application in our SIEM platform.

DLL hijacking detection in Kaspersky SIEM

In the SIEM system, the model analyzes the metadata of loaded DLLs and processes that called them from the telemetry, flags suspicious cases, and then cross-checks its verdict against KSN cloud data. This not only improves the accuracy of DLL hijacking detection, but also reduces false positives. The model can operate in both the correlation subsystem and the event collection subsystem.

In the first case, it checks only the events that have already triggered correlation rules. This allows for a more precise threat assessment and faster alert generation if needed. Since not all events are checked, the volume of cloud queries doesn’t significantly impact the model’s response speed.

In the second case, the model processes all library loading events that meet certain conditions. This method consumes more resources but is invaluable for retrospective threat hunting.

In another Securelist blog post, colleagues from the Anti-Malware Research group described in detail how the DLL hijacking detection model helps Kaspersky SIEM catch targeted attacks, with real examples of early incident detection.

Most importantly, the model’s accuracy will only continue to improve as more data on threats and legitimate processes accumulates and KSN algorithms evolve.