Reverse Engineering: Tips and Tricks
by Nikolaos Tsapakis
When reverse engineering, time spent is very important. Especially in our days where people are more and more occupied and busy. In this small guide, I will provide some tips that may accelerate this process and mature your skills. Let us now begin.
Functions
It is very common for coders to write a function in order to perform a specific task and then reuse it.
In many cases, such a function would return just a success flag and code will validate the outcome based on that flag. Modify that flag and you may be able to switch from "bad boy" into "good boy." In most cases, that flag would reside in EAX/RAX after the function exits. But in other circumstances, a function may not only return a flag but also an object.
Such an object can be in the form of a structure in memory with a valid pointer. If you try to modify the return value, that would most probably result in application crush due to memory access violation, since it would try to still use a non-existing object. In such a case, emulation is a possibility. That would mean trying to craft function return data which are similar to the expected object and its structures. In such a case, you would need to understand that function in order to emulate its behavior.
Anti-Debugs
Check the application behavior running under debugger versus not running under debugger.
If the application terminates unexpectedly under debugger, then debug tools have probably been detected. There are so many anti-debug tricks which can be used to prevent debugging and reverse engineering. Some of them detect the modifications a debugger performs on the process when being debugged. Those modifications may include larger execution time between different parts of code (since a debugger pauses in between), changes in internal process structures (like the BeingDebugged flag in the PEB), changes in machine environment (like the presence of processes related to debug tools), environment checks (like running on a Virtual Machine [VM]) and much more.
The best approach against these checks would initially be to use ready-made plugins which disable anti-debug mechanisms in a massive way. An example of such plugins could be x64dbg anti-anti-debug plugins1. You can go through the list of anti-anti debugs in the plugins and look for more information using search engines.
If you still suspect that application detects the debugger then you should go step-by-step and trace up to the part of code which changes its flow to process termination or other unusual code exits.
0xEBFE
The 0xEBFE trick is a two byte sequence which is assembled into JMP EIP.
That would mean the program will execute that instruction and indefinitely pause its execution (self-jump). It is a nice trick in order to make the program stop where you wish without the need for attaching a debugger from the beginning of the application execution. The trick would be the ability to bypass some anti-debugging detections.
For example, any code that would run prior to self jump instruction would not detect debugging since it makes sense to attach the debugger after the application executes the self-jump instruction and pauses in memory.
File Tampering
Avoid modifying the file and try modifying memory only.
For example, you may write a program (which is called a loader) which runs the application and then injects code in memory depending on the program manipulation you wish to achieve. The reason for that is you may bypass file tampering detections.
File tampering can be detected using various methods like checksums and digital signature checking. Python is a great language for writing such a loader2. Moreover, there may be other tools which can generate loaders based on execution conditions and input. One such example is the following reference3.
Automation and Plugins
Try using existing plugins1 for your reverse engineering actions.
For example, if you are using x64dbg, then try using supported plugins which bypass anti-debugs or perform other actions like dumping memory regions or fixing imports. By studying the plugins and their code or configuration, you may learn lots of things for their functionality and use this knowledge for your future quests.
Try to automate stuff by using any tool your debugger provides. Most debuggers have scripting capabilities to interact with the debugged application. Discover and try using those capabilities.
Online Versus Offline
Find out how an application behaves when being online versus being offline.
In general, it is always a good idea to notice high-level application behavior before going deeper into assembly code. That could save much time that would result in going through tons of code, probably wasting lots of energy and time for nothing.
Recon
Try accumulating public information for older version(s) of the same application.
Such research could reveal interesting high-level functions and is a must to know before going deeper into assembly code. Application evolution may also reveal interesting information on how certain functions evolved and got more complicated through time. Places such as developer forums, vendor sites, GitHub repos, or personal accounts may be a gold mine in regards to the application history and evolution.
Testing Environment
Make sure you have a dedicated environment for conducting your experiments.
Setting up virtual machines like VirtualBox4 would be ideal in order to take snapshots for tracing and restoring the snapshots if tracing was not successful. That will save much time to reach a debug point. You may create many different environments like a snapshot with no debugging tools at all (fresh) and another with debugging tools. You may try to run the application on fresh versus non-fresh snapshots and notice differences in behavior. Make sure you have all necessary tools like IDA dasm5, x64dbg debugger6, Process Explorer7, Process Monitor8, a hex editor, and an API Monitor9.
Keep in mind that applications may behave differently when running under a virtual machine.
For example, some apps may detect running inside a VM and abnormally terminate. Applications like games may also not run correctly in a VM since proper 3D capabilities may not be available.
Security Products
In regards to the testing environment, having a network connection and a security product installed (like an anti-virus) on the test machine could be a challenge depending on the type of research.
Security products may leak lots of information about your research. They could send out telemetry related to filesystems, registry, running processes, network connections, and more. Perhaps it is optimal to do all research and testing on a clean machine except if research does not mandate such a need.
High-Level Process Info
Before going deeper into your analysis, try to find out as much high-level information for your running application as possible.
For example, you may check for open handles, remote communication hosts, command line input arguments, parent and children processes, process environment variables, loaded modules, and much more. Try to spend some time interacting with the application and observe changes in its behavior. That would generate important knowledge which you can use it in the next steps of your analysis.
Debug Mode
A couple of important traits that may assist your analysis are strings and debug messages.
It may be possible that you would execute the application in some sort of debug mode which could dump logs with valuable information. Logs may be dumped into a file or on the debugger itself. Always check public available information and binary/memory strings in order to discover input options that would allow the application to run in development or debug mode.
Last Words
I hope this short guide provides you with some helpful information in order to increase your analytical skills. But the most important properties for you to have throughout the journey are curiosity and imagination.
Good luck!
References