Binary Application Attacks

What we’ll be learning

In this article, we’ll be discussing attacks that target binary applications. We’ll talk about the basics of how computer memory works, and how it can be exploited to get programs to misbehave.

Memory 101

Random Access Memory (RAM) is where a computer stores everything that it’s currently “thinking” about, in the form of a sequence of bytes. The text you’re reading right now is somewhere in your computer’s RAM, but so are the processor instructions that make up the browser you’re using to read it, and the operating system the browser is running on.

Each of these bytes has a unique address, and in most cases, there isn’t a distinction made between what bytes are data and what bytes are instructions for the computer to follow. When you create a variable in a program, you are telling the computer to allocate a little part of its memory for that variable when the program runs.

Pointers

A pointer is a variable that stores the address of another variable. Most programming languages don’t require you to worry about pointers, but some, like C and C++, do. However, just because you don’t need to worry about pointers while programming doesn’t mean they aren’t used “under the hood” of the language.

Pointers can be dereferenced, which is the process of reading the value at the address the pointer points to. For example, imagine you have a row of boxes, numbered zero to seven. Box one contains the value 7, and box seven contains the number 36. Reading the contents of box one gives you 7, but dereferencing box one gives you 36.

A null pointer is a pointer that points to nothing. This is not the same as a pointer pointing to a variable with a value of 0: a pointer to a value of 0 is like being handed a paper with 0 written on it, and a null pointer is like being handed nothing. Attempting to dereference a null pointer will usually cause a crash, but can sometimes allow for arbitrary code execution.

Messing with memory

Memory leaks

Computers have a finite amount of memory, so it’s important to be efficient. If everything is working correctly, programs will deallocate memory that they no longer need, but sometimes things go wrong. A memory leak occurs when allocated memory is not released when it’s no longer being used. This can cause a program to consume more and more memory over time, until no more are available, causing a crash.

Memory leaks are an example of resource exhaustion, where a computer runs out of finite resources like memory, disk capacity, network capacity, etc. Attackers can use resource exhaustion for denial of service attacks.

Overflow attacks

Not all of our data fits neatly into a single byte, which is why it’s important to be able to handle larger quantities of data. One method used for this is a buffer: a section of memory allocated by a program to store data in. If a program knows it’s going to receive 10 bytes of data, it can allocate a 10-byte buffer to store that data.

What happens if a buffer receives more data than it can handle? In the worst-case scenario, a buffer overflow happens, where data continues to be written outside of where it should be. Recall that computer memory contains not only data but also the instructions executed by the processor. If a program writes outside its buffer, it may unintentionally overwrite its own code, leading to unexpected behavior.

This can be used for a buffer overflow attack, where an attacker intentionally passes data that overflows a buffer, allowing an attacker to overwrite program instructions with malicious instructions of their own choosing.

Another type of overflow attack is an integer overflow attack. Unlike buffers, integers are a finite size, but this doesn’t mean they can’t overflow. Integers have upper and lower limits on the numbers they can store, based on how many bytes they use and how they are interpreted. Trying to store a number larger than the maximum will cause the number to ‘wrap around’ and start again from the lowest value. A clever attacker can exploit this to cause unintended behavior in programs.

DLL injection

A dynamic link library (DLL) is a file that contains instructions that other programs can use. This is very useful for programmers, as it means they don’t have to write all their programs from scratch: They can use DLLs that implement the functionality they need.

Unfortunately, not all DLLs are trustworthy. DLL injection is when an attacker is able to attach a malicious DLL to a legitimate program. As far as the program knows, the DLL is harmless, when in reality it’s doing the attackers’ bidding while masquerading as part of the legitimate program.

Antivirus software is well aware of malicious DLLs, so attackers will attempt to evade detection by refactoring their DLLs so that they don’t match the antivirus’s signatures for known malicious files.

DLLs are also used to maintain compatibility with legacy applications via a process known as shimming, where a DLL is injected into a program to ‘translate’ outdated function calls into ones supported by the current operating system. This is a popular way of injecting malicious DLLs into a program.

Conclusion

Attackers are able to directly target binary applications by abusing bugs that allow for maliciously crafted input to modify the code of the program. Attackers can also use malicious DLLs to infect legitimate programs with malicious code.