AHK v2: converting/optimizing scripts Topic is solved

Get help with using AutoHotkey and its commands and hotkeys
oif2003
Posts: 214
Joined: 17 Oct 2018, 11:43
GitHub: oif2003

Re: AHK v2: converting/optimizing scripts

18 Dec 2018, 16:21

You uploaded the .o file so I figure I can maybe just load part (or all) of it to get it to work, but apparently I have failed!
User avatar
vvhitevvizard
Posts: 304
Joined: 25 Nov 2018, 10:15
Location: Russia

Re: AHK v2: converting/optimizing scripts

18 Dec 2018, 16:24

oif2003 wrote:
18 Dec 2018, 16:21
You uploaded the .o file so I figure I can maybe just load part (or all) of it to get it to work, but apparently I have failed!
omg. I made a botch again. =) ok, I re-uploaded the right file. in the same post. .o was output made by ur first script posted concerning TCC combined with AHK
oif2003
Posts: 214
Joined: 17 Oct 2018, 11:43
GitHub: oif2003

Re: AHK v2: converting/optimizing scripts

18 Dec 2018, 16:29

vvhitevvizard wrote:
18 Dec 2018, 16:24
oif2003 wrote:
18 Dec 2018, 16:21
You uploaded the .o file so I figure I can maybe just load part (or all) of it to get it to work, but apparently I have failed!
omg. I made a botch again. =) ok, I re-uploaded the right file. in the same post. .o was output made by ur first script posted concerning TCC combined with AHK
All good man, happens to the best of us :D
That's some really nice work, btw. This worked for me:

Code: Select all

b:=FileRead(A_Desktop "\otest.bin", "RAW"), n:=StrLen(b)*2
VarSetCapacity(p, n)
Loop(n//8) ;write program data to memory in 8 byte chunks
	i:=A_Index-1, NumPut(NumGet(b,i*8, "int64"), p,i*8, "int64")
DllCall("VirtualProtect", "ptr",&p, "uint",n, "uint",0x40, "uintp",op)
msgbox(DllCall(&p))
msgbox(DllCall(&p))
msgbox(DllCall(&p))
User avatar
vvhitevvizard
Posts: 304
Joined: 25 Nov 2018, 10:15
Location: Russia

Re: AHK v2: converting/optimizing scripts

18 Dec 2018, 16:35

oif2003 wrote:
18 Dec 2018, 16:29
That's some really nice work, btw. This worked for me:
good to know. ;) but the query is could we get rid of redundant copying from 1 buffer to another? can we just assign 1st buffer address obtained from FileRead?

I believe execution shared with write/read will be prohibited in the future versions of Windows: thats a weak point for security. Malicious code exploiting stack overflows works b/c of that.
Last edited by vvhitevvizard on 18 Dec 2018, 16:38, edited 1 time in total.
oif2003
Posts: 214
Joined: 17 Oct 2018, 11:43
GitHub: oif2003

Re: AHK v2: converting/optimizing scripts

18 Dec 2018, 16:38

vvhitevvizard wrote:
18 Dec 2018, 16:35
oif2003 wrote:
18 Dec 2018, 16:29
That's some really nice work, btw. This worked for me:
good to know. ;) but the query is could we get rid of redundant copying from 1 buffer to another? can we just assign 1st buffer address obtained from FileRead?
Very good point, I don't know why that never occurred to me :facepalm:

Code: Select all

p:=FileRead(A_Desktop "\otest.bin", "RAW"), n:=StrLen(p)*2
DllCall("VirtualProtect", "ptr",&p, "uint",n, "uint",0x40, "uintp",op)
msgbox(DllCall(&p))
msgbox(DllCall(&p))
msgbox(DllCall(&p))
User avatar
vvhitevvizard
Posts: 304
Joined: 25 Nov 2018, 10:15
Location: Russia

Re: AHK v2: converting/optimizing scripts

18 Dec 2018, 16:41

oif2003 wrote:
18 Dec 2018, 16:38
it works! :thumbup: btw I didnt know StrLen works with binary data :D *2 is to pretend the binary data is widechars array? resulting length was 26 (original file: 25) I wonder how come StrLen skipped some zero DWORDs and made it to the file's end?
it should quit at offset 5-6 (Strlen=3)
Image
Last edited by vvhitevvizard on 18 Dec 2018, 16:49, edited 5 times in total.
oif2003
Posts: 214
Joined: 17 Oct 2018, 11:43
GitHub: oif2003

Re: AHK v2: converting/optimizing scripts

18 Dec 2018, 16:43

And since you like this sort of thing:

Code: Select all

DllCall("VirtualProtect",'ptr',&(p:=FileRead(f:=A_Desktop "\otest.bin","RAW")),'uint',FileGetSize(f),'uint',0x40)
loop 3
	msgbox(DllCall(&p))
:lol:

Anyway, I copied and pasted old code I used and didn't think much of it. I am not sure why it works here. Anyway, maybe it's best to use FileGetSize(f) instead here. Also it works here as long as that parameter is greater than 0.
The size of the region whose access protection attributes are to be changed, in bytes. The region of affected pages includes all pages containing one or more bytes in the range from the lpAddress parameter to (lpAddress+dwSize). This means that a 2-byte range straddling a page boundary causes the protection attributes of both pages to be changed.
https://msdn.microsoft.com/en-us/librar ... s.85).aspx

so we only need:

Code: Select all

DllCall("VirtualProtect",ptr,&(p:=FileRead(A_Desktop "\otest.bin","RAW")),uint,1,uint,0x40)
User avatar
vvhitevvizard
Posts: 304
Joined: 25 Nov 2018, 10:15
Location: Russia

Re: AHK v2: converting/optimizing scripts

18 Dec 2018, 17:16

oif2003 wrote:
18 Dec 2018, 16:43
so we only need:

Code: Select all

DllCall("VirtualProtect",ptr,&(p:=FileRead(A_Desktop "\otest.bin","RAW")),uint,1,uint,0x40)
Indeed. :thumbup: That works for mcodes < 4k bytes (fitting to 1 page). Maybe even more bytes. GetSystemInfo DllCall returns SYSTEM_INFO.dwPageSize about current page size.

EDIT: hmm I think we have to calculate size still. depending on where our code chunk is located, we might be close to the page boundary and fail having mapped only 1 of 2 code parts. its a very dangerous lurking error.


but about the last parameter u removed, msdn info says: lpflOldProtect [out]
A pointer to a variable that receives the previous access protection value of the first page in the specified region of pages. If this parameter is NULL or does not point to a valid variable, the function fails.

I tried to replace real pointer with NULL before, it worked but was giving me a distressing feeling of chaos so I kept it intact.

2.
It occurred to me instead of creating a buffer/calling GlobalAlloc and calling VirtualProtect, we can just call VirtualAlloc - the latter just combines both methods.
Last edited by vvhitevvizard on 18 Dec 2018, 17:51, edited 1 time in total.
oif2003
Posts: 214
Joined: 17 Oct 2018, 11:43
GitHub: oif2003

Re: AHK v2: converting/optimizing scripts

18 Dec 2018, 17:47

BTW, have you seen this? AHK H v2 comes with MemoryModule (by the same person that wrote the tutorial I linked earlier) that allows you to load DLL directly into memory...
MemoryModule was developed by Joachim Bauch and is released under MPL 2.0.
The default windows API functions to load external libraries into a program (LoadLibrary, LoadLibraryEx) only work with files on the filesystem. MemoryModule is a library that can be used to load a DLL completely from memory - without storing on the disk first.
https://hotkeyit.github.io/v2/docs/MemoryModule.htm

Also, AHK H v2's AutoHotkey.dll looks promising. If I can use another language and only use AHK as a frontend things could get quite interesting!
User avatar
vvhitevvizard
Posts: 304
Joined: 25 Nov 2018, 10:15
Location: Russia

Re: AHK v2: converting/optimizing scripts

18 Dec 2018, 17:54

oif2003 wrote:
18 Dec 2018, 17:47
BTW, have you seen this? AHK H v2 comes with MemoryModule (by the same person that wrote the tutorial I linked earlier) that allows you to load DLL directly into memory...
haha I was right in the middle of digesting his tutorial. btw I figured out the logic loads the whole DLL even if only 1 imported function is requested by DLL caller
User avatar
vvhitevvizard
Posts: 304
Joined: 25 Nov 2018, 10:15
Location: Russia

Re: AHK v2: converting/optimizing scripts

18 Dec 2018, 23:55

btw, in ur 2nd code:

Code: Select all

haystack := "The quick brown fox jumps over the lazy dog."
needle   := "fly"
tcc.__StrPutVar(haystack, _haystack)
tcc.__StrPutVar(needle, _needle)
string2 := new tcc('
	(
		#include <string.h>
		int match(char *haystack, char *needle) {
			int r = strpbrk(haystack, needle)-haystack+1;
			return r > 0 ? r : 0;
		}
	)', path)
msgbox(	'Found a char from: "' needle '" in "' haystack '" at position: ' 
		. DllCall( string2["match"], "Str", _haystack, "Str", _needle ) )
remove tcc.__StrPutVar lines and just pass string parameters as "astr" -> DllCall( string2["match"], "astr", haystack, "astr", needle)). This way Unicode AHK will convert the strings to ANSI automagically.
Spoiler
here goes a new query: trying to enable Unicode support. SOLVED BELOW
Last edited by vvhitevvizard on 19 Dec 2018, 00:29, edited 3 times in total.
oif2003
Posts: 214
Joined: 17 Oct 2018, 11:43
GitHub: oif2003

Re: AHK v2: converting/optimizing scripts

19 Dec 2018, 00:04

vvhitevvizard wrote:
18 Dec 2018, 23:55
That was one of those things I was unsure of, and that's why I left them there. I think AStr was also faster, too on some of my tests. Here's a more slim version I was messing around with last night. I changed the Str inside __Get to AStr, removed a few things, and updated some examples. I managed to return wchar_t* in the first example, but I had to use wchar.h

Code: Select all

/*
	AHK v2 (64-bit) LIBTCC by oif2003 (16 Dec 2018)
	This script allows the use of C code within AHK. C code is compiled and loaded into 
	memory using libtcc.dll.  Function pointers are used for DllCall.
	Requires TCC: https://download.savannah.gnu.org/releases/tinycc/tcc-0.9.27-win64-bin.zip
	Place script in TCC folder or locate it when prompted. (or set tcc.__defPath)
	For more information on TCC: https://bellard.org/tcc/
*/
#singleinstance force

;----------------------------
;define C functions with strings
	string1 := new tcc('
		(
			#include <wchar.h>
			
			wchar_t* hello() {
				return L"hello world";
			}
		)'
	)
	MsgBox(DllCall(string1["hello"], 'Str'))

	haystack := "The quick brown fox jumps over the lazy dog."	,StrPutVar(haystack, _haystack)
	needles  := "high"											,StrPutVar(needles, _needles)
	string2 := new tcc('
		(
			#include <string.h>
			
			int match(char *haystack, char *needles) {
				int r = strpbrk(haystack, needles) - haystack + 1;
				return r > 0 ? r : 0;
			}
		)'
	)
	pos := DllCall(string2["match"], 'Str', _haystack, 'Str', _needles)
	MsgBox('Found "' SubStr(haystack, pos, 1) '" at position ' pos ' in haystack: "' haystack '" using needles: "' needles '"')

;---------------------------
;define C functions in comment blocks
	comment1 := new tcc(parseC("Block 1")) 
	;block tags ("Block 1" here) must not be followed by anything other than a new line
	/*_C Block 1
		#include <math.h>
		
		int inc() {
			static int i = 0;
			return ++i;
		}
		
		double Csqrt(double x) {
			return sqrt(x);
		}
	*/
	MsgBox("Csqrt(" format("{:.3g}", x := 3.14**2) ")"
			. " ~ " format("{:.3g}", DllCall(comment1["Csqrt"], 'Double', x, 'Double'))
	)
	
	;save function pointer to variable for speed
	fp := comment1["inc"], t := A_TickCount
		loop 1000000
			x := DllCall(fp)
	MsgBox("x = " x " (Time elapsed: " A_TickCount - t " ms)" )

	;explicitly include User32.dll so MessageBox will work
	comment2 := new tcc(parseC("Block 2"),,, "C:\Windows\System32\User32.dll") 
	/*_C Block 2
		#include <stdio.h>
		#include <windows.h>
		
		int hello(char name[]) {
			char buf[256];
			snprintf(buf, sizeof buf, "Hello %s!", name);
			MessageBox(0, buf, "Greeting", 0);
		}
	*/
	DllCall(comment2["hello"], 'AStr', A_UserName)


StrPutVar(ByRef s, ByRef v) => (VarSetCapacity(v, StrPut(s, "UTF-8")), StrPut(s, &v, "UTF-8"))
;===============================================================================================================
parseC(tag := "", path:="") {	;reads C code between labels
	static script, lastpos := 1
	return (
		path := path ? path : A_ScriptFullPath
		,startLabel := "/*_" "C " tag "`r"
		,endLabel   := "*/"
		,script || script := fileRead(A_ScriptFullPath) 
		,cStart  := InStr(script, startLabel, ,lastpos) + StrLen(startLabel)
		,lastpos++ := cEnd := InStr(script, endLabel, , cStart + 1)
		,SubStr(script, cStart, cEnd - cStart)
	)
}

class tcc {	
	static 	 __defPath := A_ScriptDir
			,__instanceCount := 0
			,__hModule := 0
			
	__New(cStr, tccPath := "", opts := "", addfiles*) {	;opts: literal option string, addfiles := variadic array of paths
		static	 _path := "", libtcc
				,tcc_add_file := 0x1B2B4
				,tcc_compile_string := 0x1CF83
				,tcc_free := 0x2FC4
				,tcc_new := 0xF703
				,tcc_relocate := 0x1C71C
				,tcc_set_options := 0xD8E5
				,tcc_set_output_type := 0xF5A9
				,TCC_OUTPUT_MEMORY := 1
				,TCC_RELOCATE_AUTO := 1
		tccPath := _path ? _path : (tccPath ? tccPath : tcc.__defPath)
		while !FileExist(tccPath "\libtcc.dll")
			_path := (tccPath := DirSelect(, , "Locate TCC folder.  Close to exit.")) ? tccPath : ExitApp()
		(tcc.__hModule) || libtcc := tcc.__hModule := DllCall("LoadLibrary", 'Str', tccPath "\libtcc.dll", 'Ptr')
		,this.__context := context := DllCall(libtcc + tcc_new, 'Ptr') 
		,(opts) && (StrPutVar(opts, options), DllCall(libtcc + tcc_set_options, 'Ptr', context, 'Str', options))
		for _, v in addfiles
			StrPutVar(v, filename), DllCall(libtcc + tcc_add_file, 'Ptr', context, 'Str', filename)
		 DllCall(libtcc + tcc_set_output_type, 'Ptr', context, 'UInt', TCC_OUTPUT_MEMORY, 'Int')
		,StrPutVar(cStr, _cStr), DllCall(libtcc + tcc_compile_string, 'Ptr', context, 'Str', _cStr, 'Int')
		,DllCall(libtcc + tcc_relocate, 'Ptr', context, 'Ptr', TCC_RELOCATE_AUTO, 'Int')
		,tcc.__instanceCount++
	
		StrPutVar(ByRef s, ByRef v) => (VarSetCapacity(v, StrPut(s, "UTF-8")), StrPut(s, &v, "UTF-8"))
	}
	
	__Get(f) {
		static tcc_get_symbol
		return DllCall(tcc_get_symbol ? tcc_get_symbol : (tcc_get_symbol := tcc.__hModule + 0x80E2), 'Ptr', this.__context, 'AStr', f, 'Ptr')
	}
	
	__Delete() {
		static tcc_delete := 0xD543
		DllCall(tcc.__hModule + tcc_delete, 'Ptr', this.__context), (--tcc.__instanceCount == 0) && DllCall("FreeLibrary", 'Ptr', tcc.__hModule)
	}
}

BTW, AHK H v2 is looking pretty awesome. I've been playing with AutoHotkey.dll, and I might use that for simple GUI's in another language. Still need to test the MemoryModule's Dll loading capabilities.

Update: It works! I loaded AutoHotkey.dll inside another language and used its MemoryModule to manually load a Dll. It worked!
Last edited by oif2003 on 19 Dec 2018, 00:25, edited 1 time in total.
User avatar
vvhitevvizard
Posts: 304
Joined: 25 Nov 2018, 10:15
Location: Russia

Re: AHK v2: converting/optimizing scripts

19 Dec 2018, 00:24

Code: Select all

#include <wchar.h>
		int match(const wchar_t *haystack, const wchar_t *needle) {
			int r = wcspbrk(haystack, needle)-haystack+1;
ok I figured out it requires another sub-function. This one works

Now my last question: how to inline (expand inside the function) this lib call? 8-)
oif2003
Posts: 214
Joined: 17 Oct 2018, 11:43
GitHub: oif2003

Re: AHK v2: converting/optimizing scripts

19 Dec 2018, 00:28

vvhitevvizard wrote:
19 Dec 2018, 00:24
A quick look inside wchar.h :shock:
Spoiler
and there's also : #include <_mingw.h>
User avatar
vvhitevvizard
Posts: 304
Joined: 25 Nov 2018, 10:15
Location: Russia

Re: AHK v2: converting/optimizing scripts

19 Dec 2018, 00:40

these headers is a definition of hell =)

I just found a source for wcspbrk

Code: Select all

/* Find the first occurrence in WCS of any wide-character in ACCEPT.  */
wchar_t *
wcspbrk (const wchar_t *wcs, const wchar_t *accept)
{
  while (*wcs != L'\0')
    if (wcschr (accept, *wcs) == NULL)
      ++wcs;
    else
      return (wchar_t *) wcs;
  return NULL;
}
Last edited by vvhitevvizard on 19 Dec 2018, 00:53, edited 1 time in total.
oif2003
Posts: 214
Joined: 17 Oct 2018, 11:43
GitHub: oif2003

Re: AHK v2: converting/optimizing scripts

19 Dec 2018, 00:47

Hey, I think I will switch over to AutoHotkey H v2 for a little bit. Being able to use AHK features from other programs is pretty awesome and that MemoryModule is pretty slick, too:

Code: Select all

msgbox(DllCall(MemoryGetProcAddress(MemoryLoadLibrary(A_Desktop "\\tcc64\\test2.dll"),"hello"), "AStr", A_UserName))
8-)

I didn't see the Fat Arrow there though, so I will miss that for sure!
Edit: Maybe I missed it in the docs, but Fat Arrow does work!
Last edited by oif2003 on 19 Dec 2018, 15:15, edited 1 time in total.
User avatar
vvhitevvizard
Posts: 304
Joined: 25 Nov 2018, 10:15
Location: Russia

Re: AHK v2: converting/optimizing scripts

19 Dec 2018, 01:04

oif2003 wrote:
19 Dec 2018, 00:47

Code: Select all

msgbox(DllCall(MemoryGetProcAddress(MemoryLoadLibrary(A_Desktop "\\tcc64\\test2.dll"),"hello"), "AStr", A_UserName))
8-)
Slick 1-liner of urs as usual ;)

I managed to inline this lib function. now its time to tinker with it in assembly )
LOL it has a call to another function wcschr. :D
pic
so fully expanded inlined C source looks like: No wonder its performance is middling

Code: Select all

		#include <string.h>
		#include <wchar.h>

inline wchar_t * wcspbrk2 (const wchar_t *wcs, const wchar_t *accept);
inline wchar_t * wcschr (const wchar_t *wcs, const wchar_t wc);

		int match(const wchar_t *haystack, const wchar_t *needle) {
			int r = wcspbrk2(haystack, needle)-haystack+1;
			return r > 0 ? r : 0;
		}
//Find the first occurrence in WCS of any wide-character in ACCEPT.
inline wchar_t * wcspbrk2 (const wchar_t *wcs, const wchar_t *accept){
	while (*wcs != L'\0')
		if (wcschr (accept, *wcs) == NULL)
		++wcs;
	else
		return (wchar_t *) wcs;
	return NULL;
}

inline wchar_t * wcschr (const wchar_t *wcs, const wchar_t wc){
  do
    if (*wcs == wc)
      return (wchar_t *) wcs;
  while (*wcs++ != L'\0');
  return NULL;
}
oif2003
Posts: 214
Joined: 17 Oct 2018, 11:43
GitHub: oif2003

Re: AHK v2: converting/optimizing scripts

19 Dec 2018, 01:21

Thanks to HotKeyIt. He really has done some wonderful things with AHK H.
vvhitevvizard wrote:
19 Dec 2018, 01:04
LOL it has a call to another function wcschr. :D
so the rabbit hole continues ... Anyhow, what tools are you using to map those functions? It looks pretty awesome :P
User avatar
vvhitevvizard
Posts: 304
Joined: 25 Nov 2018, 10:15
Location: Russia

Re: AHK v2: converting/optimizing scripts

19 Dec 2018, 01:32

oif2003 wrote:
19 Dec 2018, 01:21
so the rabbit hole continues ... Anyhow, what tools are you using to map those functions? It looks pretty awesome :P
old version of IDA. Interactive Debugger. btw it says import table is broken for DLL output of make_c("btest.dll"). ELF works fine

I cant make TCC to inline functions. It accepts inline keyword but ignores it. :(

Check C listing above - it works in TCC. But in order to tinker with it in assembly I need to move all the 2 lib functions into the caller function
oif2003
Posts: 214
Joined: 17 Oct 2018, 11:43
GitHub: oif2003

Re: AHK v2: converting/optimizing scripts

19 Dec 2018, 01:37

vvhitevvizard wrote:
19 Dec 2018, 01:32
Yeah, I think it's time to find a new compiler! Maybe give GCC a shot? joedf has a MCode generator that you can use, too, as a bonus
https://www.autohotkey.com/boards/viewt ... =mcode+gcc

Return to “Ask For Help”

Who is online

Users browsing this forum: Bing [Bot], DaveF, hasantr, Helgef, MSN [Bot], Odlanir, Rohwedder, swagfag and 369 guests