Binary Exploitation

Calling convention

Assembly Calling Conventions For Reverse Engineers [Patreon Unlocked]

A practical look at x86 calling conventions from a reverse engineering perspective. We take a look at __cdecl __stdcall __fastcall __thiscall Full Patreon tutorial (with examples): https://www.patreon.com/posts/62676631 ----- OALABS DISCORD https://discord.gg/6h5Bh5AMDU OALABS PATREON https://www.patreon.com/oalabs Twitch https://www.twitch.tv/oalabslive OALABS GITHUB https://github.com/OALabs UNPACME - AUTOMATED MALWARE UNPACKING https://www.unpac.me/#/ -----

www.youtube.com

Assembly Calling Conventions For Reverse Engineers [Patreon Unlocked]

In computer science, a calling convention is an implementation-level (low-level) scheme for how subroutines receive parameters from their caller and how they return a result. Differences in various implementations include where parameters, return values, return addresses and scope links are placed (registers, stack or memory etc.), and how the tasks of preparing for a function call and restoring the environment afterwards are divided between the caller and the callee.

The default calling convention for C/C++ is [ __cdecl ]

Arguments are passed on the stack (pushed right-to-left)

Figure shows that arguments are pushed to the stack from l

Caller is responsible for stack cleanup
Supports vararg (variadic) functions

makes it hard to pass vararg undetermined set of arguments [variadic function]

X86 Calling convention

All arguments are widened to 32 bits (DWORD)
Return value is widened to 32 bits (DWORD)
Return values up to 32 bits are returned in EAX
Return values of 64 bit size (QWORD) are returned in EDX:EAX. EDX is most significant byte.
Return structures returned by reference with a pointer in EAX
Registers ESI,EDI,EBX, and EBP are restored (non-volatile)
MSDN is your friend!

String format vulnerabilities:

It can leak stack memory
it can arbitrary write

#include <stdio.h>
#include <unistd.h>

int main() {
    int secret_num = 0x8badf00d;

    char name[64] = {0};
    read(0, name, 64);
    printf("Hello ");
    printf(name);
    printf("! You'll never get my secret!\n");
    return 0;
}

/*
$ ./fmt_string
%7$llx
Hello 8badf00d3ea43eef
! You'll never get my secret!
*/

erflow_example.c

#include <stdio.h> #include <string.h>

int main(int argc, char *argv[]) {

int value = 5; char buffer_one[8], buffer_two[8];

strcpy(buffer_one, "one"); /* Put "one" into buffer_one. */ strcpy(buffer_two, "two"); /* Put "two" into buffer_two. */

printf("[BEFORE] buffer_two is at %p and contains \'%s\'\n", buffer_two, buffer_two); printf("[BEFORE] buffer_one is at %p and contains \'%s\'\n", buffer_one, buffer_one); printf("[BEFORE] value is at %p and is %d (0x%08x)\n", &value, value, value);

printf("\n[STRCPY] copying %d bytes into buffer_two\n\n", strlen(argv[1])); strcpy(buffer_two, argv[1]); /* Copy first argument into buffer_two. */

printf("[AFTER] buffer_two is at %p and contains \'%s\'\n", buffer_two, buffer_two); printf("[AFTER] buffer_one is at %p and contains \'%s\'\n", buffer_one, buffer_one); printf("[AFTER] value is at %p and is %d (0x%08x)\n", &value, value, value);

}