Log in

View Full Version : _fastcall with VC++


alexanderlukas
December 11th, 2006, 05:12
According to the documentation _fastcall in VC++ uses the ECX and EDX registers. A function (compiled with VC++) that I'm reversing uses the EAX and EDX registers for two of the arguments. So my question is: Does anybody know in which cases VC++ uses the EAX register instead of the ECX register?

TBone
December 11th, 2006, 10:56
Sorry to ask this, but are you sure it was compiled with VC++? PEID can be wrong sometimes. Borland and Watcom both use EAX and EDX for the first two arguments in __fastcall, according to Wikipedia

http://en.wikipedia.org/wiki/X86_calling_conventions

If it really is VC++, I have no idea. Perhaps EAX contains the address of a global or static var instead of an actual argument?

alexanderlukas
December 11th, 2006, 11:25
Quote:
[Originally Posted by TBone;63034]Sorry to ask this, but are you sure it was compiled with VC++? PEID can be wrong sometimes. Borland and Watcom both use EAX and EDX for the first two arguments in __fastcall, according to Wikipedia

http://en.wikipedia.org/wiki/X86_calling_conventions

If it really is VC++, I have no idea. Perhaps EAX contains the address of a global or static var instead of an actual argument?


Yes, I'm 100% sure it was compiled with VC++. For a while I thought it was compiled with the /GL option, but the rest of the code doesn't look like it is compiled with /GL. So my best guess is __fastcall even though it doesn't really fit that one either.

blabberer
December 11th, 2006, 13:32
I dont know if this makes any sense but if you have ida (ida free would suffice) it understands compilers better and if it was vc++ it would apply the right flirt libraries should give you some clues

Maximus
December 11th, 2006, 15:37
eax can contain the object parameter (the self/this/whatever). If you are examining an object call, of course. Compiler might simply have kept it across calls.
...and if it is really VC++, of course.

TBone
December 11th, 2006, 16:13
Are you sure? Everything I've found says that VC++ uses ECX for "this" pointer.

Maximus
December 11th, 2006, 16:55
Very possible, indeed. Admittely, I usually remember the register's order only. I only noted that the register that holds This sometime changes, but I honestly forgot on which kind of apps it happens, where and why... I usually recognize This by the kind of indirect call/access performed, and apply the structure to it -if I am a good boy naming structures...
Indeed, eax as a standing parameter is a bit silly suggestion, as it is usually used for the return value...

LLXX
December 12th, 2006, 00:12
Does it really matter what registers are used? As long as the caller and callee used the same ones, nothing is wrong.

Optimising compilers also like to move around with the registers used, i.e.
Code:
mov eax, [...]
mov ebx, [...]
mov ecx, [...]
mov edx, [...]
mov [...], eax
mov [...], ebx
mov [...], ecx
mov [...], edx


Take a look at a disassembled compiled MD5 or some other routine that has a lot of repetition and you'll see that it uses different registers for each rep, something that helps speed since the CPU can execute two or more independent instructions at once.

alexanderlukas
December 12th, 2006, 02:41
Quote:
[Originally Posted by Maximus;63047]eax can contain the object parameter (the self/this/whatever). If you are examining an object call, of course. Compiler might simply have kept it across calls.
...and if it is really VC++, of course.


It is compiled with VC++ but I'm 99.9% sure that it is pure C code, not C++, so there should be no this pointer passed around. The EAX register contains a pointer to a local buffer in this case. I have fully reversed the functions involved and the only strange thing is that the first parameter is passed in EAX. (As a sidenote the return value from the function is passed in EAX as usual when it returns.)

alexanderlukas
December 12th, 2006, 02:45
Quote:
[Originally Posted by LLXX;63076]Does it really matter what registers are used? As long as the caller and callee used the same ones, nothing is wrong.

Optimising compilers also like to move around with the registers used, i.e.
Code:
mov eax, [...]
mov ebx, [...]
mov ecx, [...]
mov edx, [...]
mov [...], eax
mov [...], ebx
mov [...], ecx
mov [...], edx


Take a look at a disassembled compiled MD5 or some other routine that has a lot of repetition and you'll see that it uses different registers for each rep, something that helps speed since the CPU can execute two or more independent instructions at once.


As I interpret the VC++ documentation, this kind of optimization only happens *between* functions when the /GL option is used for the compilation. But perhaps I have misunderstood something?

reverser
December 12th, 2006, 17:10
Yes, that's the result of /GL switch. With it, the compiler doesn't put machine code in the .obj files but intermediate bytecode (results of compiler's first pass). Later, linker launches the second pass of the compiler, and this time the compiler knows which functions are not actually called outside the final module, and thus can use registers to pass arguments between themselves. This usually results in getting rid of stack parameters for most of the functions.
See Matt Pietrek[1] for details.
[1]http://msdn.microsoft.com/msdnmag/issues/02/05/Hood/