4

Introduction

I compiled a simple executable with Visual Studio in x64 Windows. Source code:

long test(int a, int b, int c, int d, int e, int f, int g, int h, int i, int j) {
  printf("%d %d %d", a, b, c);
  return 0x0123456789acdef;
}

int main() { test(1,2,3,4,5,6,7,8,9,10); }

If you compile the executable with GCC, then you get what you'd expect: one function call in main (it calls test) and one function call in test (it calls printf).

However, if you compile the executable with Visual Studio on an x64 Windows machine, then in both main and test, an additional call is made to some function. Let's call that function mystery. I'd like to know what the mystery function is for.

Code

Here is the disassembly of the main function. I have added a comment to show where the mystery function is called.

int main (int argc, char **argv, char **envp);
; var int64_t var_c8h @ rbp+0xc8
; var int64_t var_20h @ rsp+0x20
; var int64_t var_28h @ rsp+0x28
; var int64_t var_30h @ rsp+0x30
; var int64_t var_38h @ rsp+0x38
; var int64_t var_40h @ rsp+0x40
; var int64_t var_48h @ rsp+0x48
; var int64_t var_50h @ rsp+0x50

push rbp push rdi sub rsp, 0x118 lea rbp, [var_50h] lea rcx, [0x1400c1003] call fcn.140034cf1 ; Mystery function here! mov dword [var_48h], 0xa mov dword [var_40h], 9 mov dword [var_38h], 8 mov dword [var_30h], 7 mov dword [var_28h], 6 mov dword [var_20h], 5 mov r9d, 4 mov r8d, 3 mov edx, 2 mov ecx, 1 call fcn.140033e73 xor eax, eax lea rsp, [var_c8h] pop rdi pop rbp ret

The same mystery function is also called in the test function:

test (int64_t arg1, int64_t arg2, int64_t arg3, int64_t arg4);
; var int64_t var_c8h @ rbp+0xc8
; var int64_t var_20h_2 @ rsp+0x20
; var int64_t var_8h @ rsp+0x100
; var int64_t var_10h @ rsp+0x108
; var int64_t var_18h @ rsp+0x110
; var int64_t var_20h @ rsp+0x118
; arg int64_t arg1 @ rcx
; arg int64_t arg2 @ rdx
; arg int64_t arg3 @ r8
; arg int64_t arg4 @ r9

mov dword [var_20h], r9d ; arg4 mov dword [var_18h], r8d ; arg3 mov dword [var_10h], edx ; arg2 mov dword [var_8h], ecx ; arg1 push rbp push rdi sub rsp, 0xe8 lea rbp, [var_20h_2] lea rcx, [0x1400c1003] call fcn.140034cf1 ; Mystery function here!!! mov r9d, dword [var_18h] mov r8d, dword [var_10h] mov edx, dword [var_8h] lea rcx, str._d__d__d ; 0x14009ff98 ; "%d %d %d"
call fcn.1400335b3 mov eax, 0x789acdef lea rsp, [var_c8h] pop rdi pop rbp ret

Here are the contents of the mystery function:

├ 60: mystery (int64_t arg1);
│           ; var int64_t var_20h @ rsp+0x20
│           ; var int64_t var_8h @ rsp+0x40
│           ; arg int64_t arg1 @ rcx
│           0x1400387d8      48894c2408     mov qword [var_8h], rcx    ; arg1
│           0x1400387dd      4883ec38       sub rsp, 0x38
│           0x1400387e1      488b442440     mov rax, qword [var_8h]
│           0x1400387e6      4889442420     mov qword [var_20h], rax
│           0x1400387eb      488b442440     mov rax, qword [var_8h]
│           0x1400387f0      0fb600         movzx eax, byte [rax]
│           0x1400387f3      85c0           test eax, eax
│       ┌─< 0x1400387f5      7418           je 0x14003880f
│       │   0x1400387f7      833d06fa0700.  cmp dword [0x1400b8204], 0 ; [0x1400b8204:4]=0
│      ┌──< 0x1400387fe      740f           je 0x14003880f
│      ││   0x140038800      ff15fa770800   call qword [sym.imp.KERNEL32.dll_GetCurrentThreadId] ; [0x1400c0000:8]=0xc0738 reloc.KERNEL32.dll_GetCurrentThreadId ; "8\a\f" ; DWORD GetCurrentThreadId(void)
│      ││   0x140038806      3905f8f90700   cmp dword [0x1400b8204], eax ; [0x1400b8204:4]=0
│     ┌───< 0x14003880c      7501           jne 0x14003880f
│     │││   0x14003880e      90             nop
│     │││   ; CODE XREFS from mystery @ 0x1400387f5, 0x1400387fe, 0x14003880c
│     └└└─> 0x14003880f      4883c438       add rsp, 0x38
└           0x140038813      c3             ret

Notes

The call to KERNEL32.dll_GetCurrentThreadId in the mystery function seemed interesting, but I couldn't find any information about functions, which get added by the compiler and call GetCurrentThreadId.

The question arose because I was studying the Microsoft x64 calling convention and I noticed that the mystery function didn't seem to use "home space" (also known as "shadow space", 32 bytes of space on the stack that's assigned to each function). I wanted to know what that function is and why home space wasn't assigned to it.

So the question is, what is the mystery function?

Nopslide__
  • 239
  • 2
  • 8

2 Answers2

10

The "mystery" function you refer to is called __CheckForDebuggerJustMyCode and is inserted by MSVC when /JMC (Just My Code debugging) option is enabled. From docs:

/JMC (Just My Code debugging)

Specifies compiler support for native Just My Code debugging in the Visual Studio debugger. This option supports the user settings that allow Visual Studio to step over system, framework, library, and other non-user calls, and to collapse those calls in the call stack window.

[...]

The Visual Studio Just My Code settings specify whether the Visual Studio debugger steps over system, framework, library, and other non-user calls. The /JMC compiler option enables support for Just My Code debugging in your native C++ code. When /JMC is enabled, the compiler inserts calls to a helper function, __CheckForDebuggerJustMyCode, in the function prolog. The helper function provides hooks that support Visual Studio debugger Just My Code step operations.

To verify that the "mystery" function indeed corresponds to __CheckForDebuggerJustMyCode, you can put the breakpoint in Visual Studio in the main's first line, press F5 to start debugging, and then ctrl + alt + d to show window with the assembly generated by the compiler. You will get something like this:

main's disassembly

bart1e
  • 3,369
  • 2
  • 8
  • 24
3

provide full code compiler version and compiler / linker arguments

here i checked it with msvc 2017 x64 and i don't find any mystery function

also i ran it on Compiler Explorer with several versions of msvc x64 and they dont spit it either

:\>cl 2>&1 | head -n 1
Microsoft (R) C/C++ Optimizing Compiler Version 19.16.27045 for x64

:&gt;cat myst.cpp #include <stdio.h> long test(int a, int b, int c, int d, int e, int f, int g, int h, int i, int j) { printf("%d %d %d", a, b, c); return 0x0123456789acdef; }

int main() { test(1,2,3,4,5,6,7,8,9,10); }

:&gt;cl /Zi /W4 /analyze /Od /EHsc /nologo myst.cpp /link /release myst.cpp myst.cpp(4): warning C4305: 'return': truncation from '__int64' to 'long' myst.cpp(4): warning C4309: 'return': truncation of constant value myst.cpp(2): warning C4100: 'j': unreferenced formal parameter myst.cpp(2): warning C4100: 'i': unreferenced formal parameter myst.cpp(2): warning C4100: 'h': unreferenced formal parameter myst.cpp(2): warning C4100: 'g': unreferenced formal parameter myst.cpp(2): warning C4100: 'f': unreferenced formal parameter myst.cpp(2): warning C4100: 'e': unreferenced formal parameter myst.cpp(2): warning C4100: 'd': unreferenced formal parameter

:&gt;cdb -c "uf myst!main;q" myst.exe | awk /Reading/,/quit/ 0:000> cdb: Reading initial command 'uf myst!main;q' myst!main: 00007ff6538a1040 4883ec58 sub rsp,58h 00007ff6538a1044 c74424480a000000 mov dword ptr [rsp+48h],0Ah 00007ff6538a104c c744244009000000 mov dword ptr [rsp+40h],9 00007ff6538a1054 c744243808000000 mov dword ptr [rsp+38h],8 00007ff6538a105c c744243007000000 mov dword ptr [rsp+30h],7 00007ff6538a1064 c744242806000000 mov dword ptr [rsp+28h],6 00007ff6538a106c c744242005000000 mov dword ptr [rsp+20h],5 00007ff6538a1074 41b904000000 mov r9d,4 00007ff6538a107a 41b803000000 mov r8d,3 00007ff6538a1080 ba02000000 mov edx,2 00007ff6538a1085 b901000000 mov ecx,1 00007ff6538a108a e871ffffff call myst!test (00007ff6538a1000) 00007ff6538a108f 33c0 xor eax,eax 00007ff6538a1091 4883c458 add rsp,58h 00007ff6538a1095 c3 ret quit:

:&gt;cdb -c "uf myst!test;q" myst.exe | awk /Reading/,/quit/ 0:000> cdb: Reading initial command 'uf myst!test;q' myst!test: 00007ff6538a1000 44894c2420 mov dword ptr [rsp+20h],r9d 00007ff6538a1005 4489442418 mov dword ptr [rsp+18h],r8d 00007ff6538a100a 89542410 mov dword ptr [rsp+10h],edx 00007ff6538a100e 894c2408 mov dword ptr [rsp+8],ecx 00007ff6538a1012 4883ec28 sub rsp,28h 00007ff6538a1016 448b4c2440 mov r9d,dword ptr [rsp+40h] 00007ff6538a101b 448b442438 mov r8d,dword ptr [rsp+38h] 00007ff6538a1020 8b542430 mov edx,dword ptr [rsp+30h] 00007ff6538a1024 488d0d15c30400 lea rcx,[myst!__xt_z+0x8 (00007ff6538ed340)] 00007ff6538a102b e8d0000000 call myst!printf (00007ff6538a1100) 00007ff6538a1030 b8efcd9a78 mov eax,789ACDEFh 00007ff6538a1035 4883c428 add rsp,28h 00007ff6`538a1039 c3 ret quit:

edit as bart1e pointed out in his answer it was debug_just_my_code the argument /jmc wasn't documented in cl /? in vs 2017 but has been implemented

:\>cl /? >args.txt
Microsoft (R) C/C++ Optimizing Compiler Version 19.16.27045 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

:&gt;grep -ir jmc args.txt

:&gt;grep -ir jm args.txt

:&gt;grep -ir j args.txt /Fm[file] name map file /Fo<file> name object file /Fm: <file> name map file /Fo: <file> name object file /P preprocess to file /Fx merge injected code to file c++latest - latest draft standard (feature set subject to change) /permissive[-] enable some nonconforming code to compile (feature set subject to change) (on by default) alignedNew[-] enable C++17 alignment of dynamically allocated objects (on by default) /Zp[n] pack structs on n-byte boundary /Zl omit default library name in .OBJ /bigobj generate extended object format /c compile only, no link /J default char type is unsigned /Yd put debug info in every .OBJ /Yl[sym] inject .PCH ref for debug lib

:&gt;

passing jmc and resulting just my code function call addition

:\>cdb -c "uf myst!main;uf myst!__CheckForDebuggerJustMyCode;q" myst.exe | awk /Reading/,/quit/
0:000> cdb: Reading initial command 'uf myst!main;uf myst!__CheckForDebuggerJustMyCode;q'

myst!main: 00007ff73c031050 4883ec58 sub rsp,58h 00007ff73c031054 488d0da84f0600 lea rcx,[myst!_DebuggerCurrentSteppingThreadId <PERF> (myst+0x66003) (00007ff7`3c096003)] 00007ff73c03105b e844010000 call myst!__CheckForDebuggerJustMyCode (00007ff73c0311a4) 00007ff73c031060 c74424480a000000 mov dword ptr [rsp+48h],0Ah 00007ff73c031068 c744244009000000 mov dword ptr [rsp+40h],9 00007ff73c031070 c744243808000000 mov dword ptr [rsp+38h],8 00007ff73c031078 c744243007000000 mov dword ptr [rsp+30h],7 00007ff73c031080 c744242806000000 mov dword ptr [rsp+28h],6 00007ff73c031088 c744242005000000 mov dword ptr [rsp+20h],5 00007ff73c031090 41b904000000 mov r9d,4 00007ff73c031096 41b803000000 mov r8d,3 00007ff73c03109c ba02000000 mov edx,2 00007ff73c0310a1 b901000000 mov ecx,1 00007ff73c0310a6 e855ffffff call myst!test (00007ff73c031000) 00007ff73c0310ab 33c0 xor eax,eax 00007ff73c0310ad 4883c458 add rsp,58h 00007ff7`3c0310b1 c3 ret

myst!__CheckForDebuggerJustMyCode: 00007ff73c0311a4 48894c2408 mov qword ptr [rsp+8],rcx 00007ff73c0311a9 4883ec38 sub rsp,38h 00007ff73c0311ad 488b442440 mov rax,qword ptr [rsp+40h] 00007ff73c0311b2 4889442420 mov qword ptr [rsp+20h],rax 00007ff73c0311b7 488b442440 mov rax,qword ptr [rsp+40h] 00007ff73c0311bc 0fb600 movzx eax,byte ptr [rax] 00007ff73c0311bf 85c0 test eax,eax 00007ff73c0311c1 7418 je myst!__CheckForDebuggerJustMyCode+0x37 (00007ff7`3c0311db)

myst!__CheckForDebuggerJustMyCode+0x1f: 00007ff73c0311c3 833d7ef4050000 cmp dword ptr [myst!__DebuggerCurrentSteppingThreadId (00007ff73c090648)],0 00007ff73c0311ca 740f je myst!__CheckForDebuggerJustMyCode+0x37 (00007ff73c0311db)

myst!__CheckForDebuggerJustMyCode+0x28: 00007ff73c0311cc ff152ebe0400 call qword ptr [myst!_imp_GetCurrentThreadId (00007ff73c07d000)] 00007ff73c0311d2 390570f40500 cmp dword ptr [myst!__DebuggerCurrentSteppingThreadId (00007ff73c090648)],eax 00007ff73c0311d8 7501 jne myst!__CheckForDebuggerJustMyCode+0x37 (00007ff73c0311db)

myst!__CheckForDebuggerJustMyCode+0x36: 00007ff7`3c0311da 90 nop

myst!__CheckForDebuggerJustMyCode+0x37: 00007ff73c0311db 4883c438 add rsp,38h 00007ff73c0311df c3 ret quit:

blabb
  • 16,376
  • 1
  • 15
  • 30