Two posts ago I talked about NOP in 64bits. It was unclear whether 0x90 acts as a true NOP. Because if you really think about it, the way 64bits processors work when it executes 32bits operations is to do the exchange between EAX and EAX, and then zero the high dword of touched registered. This is why XOR EAX, EAX is similar to XOR RAX, RAX… Only in 64bits!
So in that posts some guy (or girl?) wrote in a comment that exchanging two registers using 0x90 is possible when you use a REX prefix. REX prefix is like the 0x66 of the new 64bits code. It lets you access more registers and indicate the instruction may run with operation size of 64 bits rather than 32 bits (which is the default, hence 0x90 in 64bits is supposedly xchg eax, eax).
The documentation is not readable in shit. But that might be cause I’m the shock here… But let’s put it this way, after I wrote a disassembler and read the docs so many times regarding instructions etc – then if I don’t understand it, who should? FFS
Talking with Stefan And Peter (the guy behind YASM) we got the issue cleared on both processors (Intel/AMD). I just want to state before that Stefan came up with the whole problem and then Peter did some tests too on his end. So thank you both :) we now know that XCHG R8, RAX can be decoded in two ways, but we will focus on the one with the byte code of 0x90. The byte code for that instruction is: 0x49 0x90. 49 for Width (64bits operand size) and Base (access R8) and 0x90 for the XCHG. Together it really works! Same as that 0x41 0x90 works as XCHG R8D, EAX (clearing high dword though…).
diStorm treated this errornously by giving an output of:
DB 0x49
NOP
to indicate that the 0x49 prefix wasn’t used it was (what I call) DB’ed. So I had to change the behavior of this one and it wasn’t so trivial because 0x90 can’t just say “Hey, I’m NOP from now on and always was”. Now it’s up to the prefixes to decide whether 0x90 is XCHG or NOP – runtime detection. The static DB of instructions can’t help it. In addition, don’t forget 0xf3 0x90 is PAUSE which is a completely different instruction for the sake of example when mentioning prefixes of 0x90…