[SOLVED] DllCall problem

Get help with using AutoHotkey and its commands and hotkeys
User avatar
Soft
Posts: 174
Joined: 07 Jan 2015, 13:18
GitHub: visionary1
Location: Seoul
Contact:

[SOLVED] DllCall problem

03 Feb 2015, 13:52

Hello, I just watched the anti-reverse methods using Windows API in C++ forum

I'm trying to use those methods in AutoHotkey but have a problem.

CheckRemoteDebuggerPresent function C++ syntax is
BOOL WINAPI CheckRemoteDebuggerPresent(
_In_ HANDLE hProcess,
_Inout_ PBOOL pbDebuggerPresent
);

Parameters
hProcess [in] - A handle to the process.
pbDebuggerPresent [in, out] - A pointer to a variable that the function sets to TRUE if the specified process is being debugged, or FALSE otherwise.


So I translated this to AHK as
aModule := DllCall("CheckRemoteDebuggerPresent", UInt, Handleme, IntP, pbDebuggerPresent)

However, 'pbDebuggerPresent' doesnt return FALSE or TRUE. Instead, it returns 0

here is my test script, you have to manually write process handle on edit control.

Code: Select all

#SingleInstance force
#NoEnv
SetTitleMatchMode, 3
Gui, +LastFound +hwndMainF
Gui, add, button, w100 h70, button
Gui, add, edit, w100 vhandleme, edit
Gui, add, listbox, w100, listbox
Gui, add, listview, w100, listview
Gui, add, groupbox, w100, groupbox
Gui, show,, Test Script
pbDebuggerPresent := ""
return

Buttonbutton:
Gui, Submit, NoHide
MsgBox % handleme
WinGet, which, PID, Test Script
aModule := DllCall("CheckRemoteDebuggerPresent", UInt, handleme, IntP, pbDebuggerPresent)
hModule := DllCall("IsDebuggerPresent")
MsgBox, % "IsDebuggerPresent : " hModule "`nCheckRemoteDebugger : " aModule "`nDebugpresent : " boolean
/*
if hModule <> 0
{
	aModule := DllCall("DebugActiveProcessStop", "Uint", which)
	MsgBox % aModule
}
*/
return

GuiClose:
ExitApp
Last edited by Soft on 03 Feb 2015, 22:49, edited 1 time in total.
AutoHotkey & AutoHotkey_H v1.1.22.07
HotKeyIt
Posts: 2228
Joined: 29 Sep 2013, 18:35
Contact:

Re: DllCall problem

03 Feb 2015, 15:29

Code: Select all

If DllCall("CheckRemoteDebuggerPresent", "PTR", DllCall("GetCurrentProcess","PTR"), "IntP", DebuggerPresent)
MsgBox, % "CheckRemoteDebugger : " DebuggerPresent "`nDebugpresent : " DllCall("IsDebuggerPresent")
User avatar
trismarck
Posts: 506
Joined: 30 Sep 2013, 01:48
Location: Poland

Re: DllCall problem

03 Feb 2015, 15:30

For the first parameter of CheckRemoteDebuggerPresent(): I guess UInt will fail if you're running Autohotkey in 64bit mode (UInt is always 32bits).
Instead of doing pbDebuggerPresent := "" perhaps it would be cleared to write: pbDebuggerPresent := false or := 0.
Displaying the value of pbDebuggerPresent will return either 0 or 1, which translates to either false or true.

Code: Select all

Run, notepad.exe, , , PID
Run, windbg -p %PID% -c "g"
sleep 1000
dwDesiredAccess := 0x400	; PROCESS_QUERY_INFORMATION
bInheritHandle := false
hProcess := DllCall("OpenProcess", "UInt", dwDesiredAccess, "Int", bInheritHandle, "UInt", PID, "Ptr")
ret := DllCall("CheckRemoteDebuggerPresent", "Ptr", hProcess, "Int*", pbDebuggerPresent, "Int")
msgbox % pbDebuggerPresent
lexikos
Posts: 7088
Joined: 30 Sep 2013, 04:07
GitHub: Lexikos

Re: DllCall problem

03 Feb 2015, 16:33

Each parameter is always a multiple of the pointer/register size, so UInt vs Ptr doesn't matter for input parameters. It only matters if the */p suffix is present, or for return values or NumPut/NumGet. Of course, it's clearer to use the right type.
User avatar
Soft
Posts: 174
Joined: 07 Jan 2015, 13:18
GitHub: visionary1
Location: Seoul
Contact:

Re: DllCall problem

03 Feb 2015, 22:59

@HotKeyIt, @trismarck and @lexikos / thank you very much for the reply.
lexikos wrote:so UInt vs Ptr doesn't matter for input parameters.
so both of them work fine

DllCall("CheckRemoteDebuggerPresent", "PTR", DllCall("GetCurrentProcess","PTR"), "IntP", DebuggerPresent)
DllCall("CheckRemoteDebuggerPresent", "UInt", DllCall("GetCurrentProcess","PTR"), "IntP", DebuggerPresent)

according to help file,
UInt is for only 32-bit, and PTR is equivalent to Int or Int64 depending on whether the exe running the script is 32-bit or 64-bit!
An Int should also be used for each BOOL argument expected by a function (a BOOL value should be either 1 or 0).

therefore, 'DebuggerPresent' retruns 1 or 0 instead of TRUE or FALSE!

did I understand correctly?
AutoHotkey & AutoHotkey_H v1.1.22.07
lexikos
Posts: 7088
Joined: 30 Sep 2013, 04:07
GitHub: Lexikos

Re: [SOLVED] DllCall problem

03 Feb 2015, 23:21

TRUE and FALSE are the names of constants which have the values 1 and 0. Obviously, a function with integer return type is not going to return the string value "TRUE".

Code: Select all

MsgBox % true  ; 1
User avatar
Soft
Posts: 174
Joined: 07 Jan 2015, 13:18
GitHub: visionary1
Location: Seoul
Contact:

Re: [SOLVED] DllCall problem

04 Feb 2015, 00:02

@lexikos / I got it! thank you
AutoHotkey & AutoHotkey_H v1.1.22.07
User avatar
trismarck
Posts: 506
Joined: 30 Sep 2013, 01:48
Location: Poland

Re: DllCall problem

06 Feb 2015, 09:29

I'm gonna stick to this thread a little more.
Lexikos wrote:Each parameter is always a multiple of the pointer/register size, so UInt vs Ptr doesn't matter for input parameters.
Thanks for pointing this out Lexikos.

Ok, so in the context of data alignment, in case of x64, all parameters are aligned to the 8 byte boundary. Thus UInt, even though 4 bytes in length, is still aligned to 8 byte boundary.

In the context of the interpretation of the value, UInt and Ptr also seem to be equivalent on x64 - doing DllCall(addr, "UInt", 4.3 * 10**9) passes 4300000000 instead of the lower DWORD part of the number. I wonder if this was intended. (can't debug x64 from x86 host OS).

//edit:
It looks like this check doesn't appear to do anything to a >32bit/<32bit positive/negative number, if the type of DllCall param is UInt, at least on x86. Not sure if this is some sort of a prevention check for compilers that cast int64 to int weirdly.

Code: Select all

			if (this_dyna_param.type != DLL_ARG_INT64) // Shift the 32-bit value into the high-order DWORD of the 64-bit value for later use by DynaCall().
				this_dyna_param.value_int = (int)this_dyna_param.value_int64; // Force a failure if compiler generates code for this that corrupts the union (since the same method is used for the more obscure float vs. double below).
lexikos
Posts: 7088
Joined: 30 Sep 2013, 04:07
GitHub: Lexikos

Re: DllCall problem

06 Feb 2015, 19:42

trismarck wrote:Ok, so in the context of data alignment, in case of x64, all parameters are aligned to the 8 byte boundary.
In the context of the x64 calling convention, all parameters are exactly 8 bytes. Larger structs are implicitly passed by address, even if it doesn't appear that way in C source code.
Thus UInt, even though 4 bytes in length, is still aligned to 8 byte boundary.
What happens when you store a 32-bit value in an AutoHotkey variable? Putting aside strings, AutoHotkey uses 64-bit integers. So when you put a 32-bit value in, it gets padded out to 64 bits when it's stored.

With x64 parameters, each parameter value is stored in a register, in an 8-byte slot on the stack, or both. Since DllCall has 8 bytes to work with and the source integer is always 8 bytes (because once again, AutoHotkey uses 64-bit integers), DllCall does the lazy thing and just copies the whole integer into place. For x86-32, it has to either copy a 64-bit integer or truncate it to 32 bits.

When the called function executes, if its parameter is int (for example), it will retrieve only a 32-bit value from the stack or register even on x64. According to the x64 calling convention, the upper 32 bits of the stack slot or register are garbage (i.e. the function is supposed to ignore them). It doesn't matter what the upper 32 bits are; they could be 0, 1, or random/indeterminate.
In the context of the interpretation of the value, UInt and Ptr also seem to be equivalent on x64
What do you mean by "also"? Isn't that we were already talking about? As I said, "UInt vs Ptr doesn't matter for input parameters".
It looks like this check doesn't appear to do anything
It probably doesn't, but I'm inclined not to fix that which isn't broken. The result of casting __int64 to int is just taking the lower 32 bits as they are, and assigning to the value_int member of the union will only affect the lower 32 bits. The compiler probably knows that it's doing nothing, and so does nothing. The comment is mistaken about it affecting the high-order DWORD (it actually assigns to the low-order DWORD, because x86-32/64 is little-endian), so that code might be intended for some other architecture or the author might simply be mistaken.
just me
Posts: 7612
Joined: 02 Oct 2013, 08:51
Location: Germany

Re: [SOLVED] DllCall problem

07 Feb 2015, 03:10

lexikos wrote:As I said, "UInt vs Ptr doesn't matter for input parameters".
As far as I understand the source code the same is true for "U/Char" and "U/Short", isn't it?
lexikos
Posts: 7088
Joined: 30 Sep 2013, 04:07
GitHub: Lexikos

Re: [SOLVED] DllCall problem

07 Feb 2015, 17:28

Yes, that's right. All integer types are equivalent for input parameters, except int vs int64 on 32-bit. So uint vs int is also unimportant, except to make the code self-documenting.

Return to “Ask For Help”

Who is online

Users browsing this forum: AHKStudent, Bing [Bot] and 48 guests