AHK v2: converting/optimizing scripts Topic is solved

Get help with using AutoHotkey (v2 or newer) and its commands and hotkeys
oif2003
Posts: 214
Joined: 17 Oct 2018, 11:43
Contact:

Re: AHK v2: converting/optimizing scripts

15 Dec 2018, 22:14

vvhitevvizard wrote:
15 Dec 2018, 19:52
I like those ideas! It's a good way to get our feet wet. I have taken a few C courses in college in the past but I don't remember much from it aside from doing a lot of pointer arithmetic (not fun :crazy: )

As it turns out, it was a configuration issue. I was able to call MessageBox after explicitly including User32.dll using "libtcc\tcc_add_file". The command line option variant does not work. Here's the latest working copy I have that takes options(opts) and includes(addfiles*) as parameters (addfiles is variadic)

Edit: fixed typo in tcc_set_options
Spoiler
Last edited by oif2003 on 15 Dec 2018, 22:49, edited 1 time in total.
User avatar
vvhitevvizard
Posts: 454
Joined: 25 Nov 2018, 10:15
Location: Russia

Re: AHK v2: converting/optimizing scripts

15 Dec 2018, 22:26

Nice. So TCC updates DLL functions addresses for code snippets at compile time. (for included DLLs)

BTW, having started parsing AHK functions' source code, we eventually could end up with full C working projects. But thats not the point I guess - I do like interpreted scripting with mixed in compiled patches for the best result. :)

:thumbup: It does work (Message and "InStr" function).
Next step is making .obj out of it. If it can create x64 one - there was something in its documentation stating it can't. And force inlining of the lib function strpbrk. And make sure to comment out all the debug lines =)
Last edited by vvhitevvizard on 15 Dec 2018, 22:38, edited 1 time in total.
User avatar
vvhitevvizard
Posts: 454
Joined: 25 Nov 2018, 10:15
Location: Russia

Re: AHK v2: converting/optimizing scripts

16 Dec 2018, 00:39

meanwhile back to Floats pretty-printing. U bumped me up against revision of Format("{:g}" potential and unlike other float formatting modes for the function (e, f, a), this one is what I needed: Round to N significant digits.
It turned out it was .4g logic of Format function that I recreated in parallel earlier. Example of Input->Output for 4 significant digits:
.9337806789 -> .9338
.009337806789 -> .009338
.00009337806789 -> .00009338
7.7640098 -> 7.764
7724.76400 -> 7725

U got the idea, right? :)

It works faster than my AHK function. But it adds adverse effect of scientific notation sometimes, e.g.:
55500.76400989 -> -5.55e+0.4. Its intolerable for me cuz I rather consider floats as decimal floats with _finite_ number of decimals after the point.
Also there is an additional condition in my version: comply with limiting number for decimals d at the same time
, e.g. for d=6:
".0000093378080" -> 0.000009

SmartR(a) vs Format("{:.4g}", a), score || output results:
"-000.9337808" ->
454|343||-0.9338|-0.9338 mine is 32% slower

".0093378080" ->
546|391||0.009338|0.009338 40% slower

"000.0000093378080" ->
687|407||0.00000934|9.338e-06 69% slower
.

I broke down the logic of my original SmartR and added more functionality (to work with any kind of messy stringified float numbers like -0000.0100).
The logic works on string level and actually doesn't change the string at all but calculates, according to 2 settings, number of decimals for finalizing Round call. The same approach Ive done for 1-liner SmartT.
Could u look at it and optimize logic somehow? :D It might later be m-coded to greatly reduce overhead of all these SubStr/InStr loops.

updated to v1.2.1

Code: Select all

;v1.2.1 round down to p significant digits,limit decimals to m:
SmartR(_a){
	static p:=4, d:=8 ;precision (significant digits)=4, max decimals=8
	if((i:=InStr(a:=LTrim(_a, "-0"),".",1))=1){ ;>1
		r:=p
		while(SubStr(a,++i,1)=="0")
			r++ ;".93979" > -0.94, ".0093379" > 0.0094
	}
	else
		(i>p) ? r:=0:r:=p-i+1 ;"724.764" > 724.8, "5550.764" > 5551
	return(Round(_a, r>d ? d:r))
}


;////////////////////tests
a:="-000.9337808" ; > -0.9338
;a:=".0093378080" ; > 0.009338
;a:=".0000093378080" ; > 0.009338 | 9.338e-06

;a:="00724.76400989" ; > 724.8
;a:="007.76400989" ; > 724.8
;a:="5550.76400989" ; > 5551 | 5551
;a:="-55500.76400989" ; > -55501 | -5.55e+0.4


n:=200*1000
t:=A_TickCount
loop(n)
	s1:=SmartR(a)
a1:=A_TickCount-t

t:=A_TickCount
loop(n)
	s2:=Format("{:.4g}", a)
a2:=A_TickCount-t

msgbox(clipboard:=a1 "|" a2 "||" s1 "|" s2)
Last edited by vvhitevvizard on 16 Dec 2018, 04:41, edited 7 times in total.
oif2003
Posts: 214
Joined: 17 Oct 2018, 11:43
Contact:

Re: AHK v2: converting/optimizing scripts

16 Dec 2018, 01:32

vvhitevvizard wrote:
16 Dec 2018, 00:39
Would this work instead?

Code: Select all

format8(x) {
	if InStr(LTrim(x, "-0"), ".") > 4
		return Round(x += 0)
	else {
		s := Format("{:.4g}", x += 0)
		if InStr(s, "e") 
			return Format("{:.8f}", x)
		return s
	}
}
or its variant

Code: Select all

format8(x) =>	(InStr(LTrim(x, "-0"), ".") > 4) ? (
					Round(x += 0)
				):(
					s := Format("{:.4g}", x += 0)
					,(InStr(s, "e") ? Format("{:.8f}", x) : s)
				)
User avatar
vvhitevvizard
Posts: 454
Joined: 25 Nov 2018, 10:15
Location: Russia

Re: AHK v2: converting/optimizing scripts

16 Dec 2018, 01:37

It does! =) First I checked "-55500.76400989" > "-55501": works as expected
It doesn't respect max decimals (2nd condition) tho. :D

score is (urs/mine/vanilla format): 454 vs 546 vs 391 for ".0093378080" string
17% improvement


Try the following:
"000.93378080" should be "0.9" for p=2, d=1. decimals limit overriding the first parameter

Code: Select all

format8(x) {
	static p:=2, d:=1 ;precision (significant digits)=4, max decimals=1
	if InStr(LTrim(x, "-0"), ".") > p
		return Round(x += 0)
	else {
		s := Format("{:." p "g}", x += 0)
		if InStr(s, "e")
			return Format("{:." d "f}", x)
		return s
	}
}

a:="000.93378080" 
msgbox(format8(a))
Last edited by vvhitevvizard on 16 Dec 2018, 04:19, edited 3 times in total.
oif2003
Posts: 214
Joined: 17 Oct 2018, 11:43
Contact:

Re: AHK v2: converting/optimizing scripts

16 Dec 2018, 01:54

vvhitevvizard wrote:
16 Dec 2018, 01:37

Code: Select all

format8(x) {
	static p:=2, d:=5 ;precision (significant digits)=4, max decimals=8
	if InStr(LTrim(x, "-0"), ".") > p
		return Round(x += 0)
	else {
		s := Format("{:." p "g}", x += 0)
		if InStr(s, "e") || StrLen(s) - InStr(s, ".", , -1) > d
			return Format("{:." d "f}", x)
		return s
	}
}
what about this?

Code: Select all

formatR(x,p:=4,d:=8) =>	(InStr(LTrim(x, "-0"), ".") > p) ? (
							Round(x += 0)
						):(
							s := Format("{:." p "g}", x += 0)
							,(InStr(s, "e") || (StrLen(s) - InStr(s, ".", , -1) > d) ? Format("{:." d "f}", x) : s)
						)
User avatar
vvhitevvizard
Posts: 454
Joined: 25 Nov 2018, 10:15
Location: Russia

Re: AHK v2: converting/optimizing scripts

16 Dec 2018, 02:09

what about this?
it fails on this string: ".0000093378080" with p:=2,d:=8:
-> 0.00000934. expected is 0.0000093 cuz p=2 ;) U might want to check decimals limit right before calling Format

that part in ur code looks so so. 2 Format calls. Trying to replace the exponent output - and this will happen on every number >9999 for precision=4. btw if u call f-mode Format("{:.f}", there will be no exponent output at all.

Code: Select all

s := Format("{:." p "g}", x += 0)
	if InStr(s, "e")
		return Format("{:." d "f}", x)
But can u change ur path of thinking and make ur version of N decimals calculation for Round? The latter function simply works faster. My original variant with 1 Round call fails no tests but looks unoptimized. tho it has a pretty final check for d limit that works with any conditions: Round(a,r>d ? d:r) where r is calculated N digitals

btw, u dont have to convert string to number before calling Round. Conversion will be done automatically.
Round(x += 0) -> Round(x).
oif2003
Posts: 214
Joined: 17 Oct 2018, 11:43
Contact:

Re: AHK v2: converting/optimizing scripts

16 Dec 2018, 03:16

tried a slightly different approach:

Code: Select all

format8(x, p:=4, d:=8) => (InStr(LTrim(x, "-0"), ".") > p) ? Round(x)
	:(n := InStr(s := Format("{:." p "g}", x), "e")) 
	? RTrim(Format("{:." d "f}", SubStr(s, 1, n-1)*10**SubStr(s, n+1)), "0") : s
it's slow for very small numbers, but seems fast otherwise(?) Maybe a separate routine can be used for very close to zero numbers
User avatar
nnnik
Posts: 4500
Joined: 30 Sep 2013, 01:01
Location: Germany

Re: AHK v2: converting/optimizing scripts

16 Dec 2018, 03:40

vvhitevvizard wrote:
15 Dec 2018, 15:13
nnnik wrote:
15 Dec 2018, 08:40
When you need performance the first step is to avoid AHK as much as possible.
Therefore calling external functions using MCode is a valid strategy and there is no reason not to.
I beg to differ. We don't optimize performance-wise the whole program structure. We pinpoint time-critical cycles in it which take 90% of CPU time at run time. In case of json parsing we try to optimize within the last 5 pages, it might be just a few simple cycles comparing 2 string buffers. No external DLL calls needed.
Ofc usage-case depends...
Yeah Imagine a few hundred thousand DllCalls passing along some OpenGL data.
The overhead of each DllCall would be greater by a few magnitudes if you used AHK to do them.
Recommends AHK Studio
User avatar
vvhitevvizard
Posts: 454
Joined: 25 Nov 2018, 10:15
Location: Russia

Re: AHK v2: converting/optimizing scripts

16 Dec 2018, 03:46

nnnik wrote:
16 Dec 2018, 03:40
Yeah Imagine a few hundred thousand DllCalls passing along some OpenGL data.
The overhead of each DllCall would be greater by a few magnitudes if you used AHK to do them.
OpenGL is a bit different story - GPU works in parallel with CPU. Ok ok. Maybe GDI work wasn't a glaring example.
Ok, other examples: function with some dllcalls for networking or for file manipulations. Do u think optimizing these dllcall work would make a difference? Or optimizing GUI calls? I bet the majority of AHK scripts with hotkeys/clipboard/dialog boxes/downloading from url/reading-writing files/etc doesn't require mcode optimizations.


Meanwhile I cleaned up my variant and borrowed 1st branch from urs :)
score vs my prev version
500 vs 546: +8.5% for ".0093378080" string
218 vs 297: +26.5% for "5550.76400989" string

Code: Select all

SmartR(_a){ ;v1.2.1 round down to p significant digits (precision), limit decimals to m
	static p:=4, d:=8
	if((i:=InStr(a:=LTrim(_a, "-0"),".",1))=1){ ;>1
		r:=p
		while(SubStr(a,++i,1)=="0")
			r++ ;".93979" > -0.94, ".0093379" > 0.0094
	}
	else
		(i>p) ? r:=0:r:=p-i+1 ;"724.764" > 724.8, "5550.764" > 5551
	return(Round(_a, r>d ? d:r))
}
oif2003 wrote:
16 Dec 2018, 03:16
tried a slightly different approach:
using power of 10? very interesting.
this one is much faster for second branch, for "5550.76400989" string
157 vs 218 of my 1.2.1: +28% :thumbup:
but at the same time its slower for "007.76400989" string. 484 vs 360 of my 1.2.1 -34%. so its slower if left side digits<p and that would be the majority of numbers. for p=4 its all between 1 and 999, for p=6 its 1..99999
User avatar
vvhitevvizard
Posts: 454
Joined: 25 Nov 2018, 10:15
Location: Russia

Re: AHK v2: converting/optimizing scripts

16 Dec 2018, 05:56

v1.3
438 vs 500 of v2.1.1: +13.5% for ".0093378080" string

Code: Select all

SmartR(_a){ ;v1.3 round down to p significant digits (precision), limit decimals to m
;"-000.93379" > -0.934, ".0093379" > 0.00934, "724.764" > 724.8, "5550.764" > 5551
	static p:=4, d:=8
	r:=((i:=InStr(a:=LTrim(_a, "-0"),".",1))=1) ? (a ~= "[^.0]")-2+p : (i>p) ? 0:p-i+1
	return(Round(_a, r>d ? d:r))
}
Last edited by vvhitevvizard on 16 Dec 2018, 06:07, edited 1 time in total.
oif2003
Posts: 214
Joined: 17 Oct 2018, 11:43
Contact:

Re: AHK v2: converting/optimizing scripts

16 Dec 2018, 06:02

vvhitevvizard wrote:
16 Dec 2018, 03:46
Too tired to think...here's your smartR with some tweaks

Code: Select all

SmartR2(_a, p:=4, d:=8){
	return(
		(i:=InStr(a:=LTrim(_a,"-0"),".",1)) ? (
			(i==1) ? (
				Round(_a,(c:=StrLen(s:=SubStr(a,i+1))-StrLen(LTrim(s,"0"))+p)>d ? d : c)
			):(
				Round(_a,(r:=(i>p) ? 0 : p-i+1)>d ? d : r)
			)
		):(
			Round(_a, 0)
		)
	)
}
User avatar
vvhitevvizard
Posts: 454
Joined: 25 Nov 2018, 10:15
Location: Russia

Re: AHK v2: converting/optimizing scripts

16 Dec 2018, 06:10

oif2003 wrote:
16 Dec 2018, 06:02
v1.3.1
ok. I 1-lined it eventually :) a tad long line tho.
422 vs 438 of v1.3: +3.7% for ".0093378080" string. So now its close to Format("{:.4g}", but with 2 additional features.
Deleted it. This variant (unlike 1.2.1) fails on integers.
oif2003 wrote:
16 Dec 2018, 06:02
here's your smartR with some tweaks
ur variant moves even further. 390 vs 422 of my last! +7.5% for ".0093378080" string :thumbup:
And this one starts beating vanilla Format version! :D
winning line: (a~="[^.0]")-2+p:(i>p) -> StrLen(s:=SubStr(a,2))-StrLen(LTrim(s,"0"))+p

I combined urs and mine, fixing my error + small tweak of urs.
There is only 1 Round call and if the need arises, its replaceable with Format call using F format mode with respective decimals N. The latter can add paddings etc

v1.4:

Code: Select all

;v1.4 round down to p significant digits (precision), limit decimals to m
;for p=4: "-000.93379" > -0.934, ".0093379" > 0.00934, "724.76" > 724.8, "5550.76" > 5551
SmartR(_n,p:=4,d:=8)=>Round(_n,(r:=(i:=InStr(a:=LTrim(_n,"-0"),".",1)) 
	? ((i=1) ? StrLen(s:=SubStr(a,2))-StrLen(LTrim(s,"0"))+p : (i>p) ? 0:p-i+1):0)>d ? d:r)

btw, about Format - its counterpart in AHK LIB has a huge regex:
while i:=RegExMatch(f, O_ "\{((\w+)(?::([^*`%{}]*([scCdiouxXeEfgGaAp])))?|[{}])\}", m, j) ; For each {placeholder}.
no wonder its slow
Last edited by vvhitevvizard on 16 Dec 2018, 06:34, edited 1 time in total.
User avatar
vvhitevvizard
Posts: 454
Joined: 25 Nov 2018, 10:15
Location: Russia

Re: AHK v2: converting/optimizing scripts

17 Dec 2018, 04:50

BTW, like TCC, but for Assembly:
Fasm into a simple DLL which allows to assemble just inside the memory. The package contains short documentation, demonstration program:
https://board.flatassembler.net/topic.php?p=48376#48376

Also, FASM can create not only DLL but ELF object files as well.
FASM code snippet example Im just messing around here. Removed prologue/epilogue for size, it just returns 7 with 1 CPU opcode.

Code: Select all

format ELF64

section '.text' executable

 public cwstr
cwstr:
	mov rax,7
	retn
It might be useful cuz uve posted load_cObject AHK routine for exporting MCode from .o ELF object files:

Code: Select all

load_cObject(oName, ByRef code) {
	;parse ELF (Executable and Linkable Format) file
	o := FileRead(A_ScriptDir "\" oName, "RAW")
	,tableAdd := NumGet(o, 0x28, "Ptr")
	,toffset := 0x40
	,padd := NumGet(o, tableAdd + toffset +0x18, "UInt")
	,plen := NumGet(o, tableAdd + toffset +0x20, "UInt")
	,VarSetCapacity(code, plen)
	Loop plen//4	;write Program data to memory in 4 byte chuncks
		NumPut(NumGet(o, padd + (A_Index - 1)*4, "UInt"), code, (A_Index-1)*4, "UInt")
	return DllCall("VirtualProtect", "Ptr", &code, "UInt", plen, "UInt", 0x40, "Ptr*" , 0)
}	
User avatar
jeeswg
Posts: 6902
Joined: 19 Dec 2016, 01:58
Location: UK

Re: AHK v2: converting/optimizing scripts

17 Dec 2018, 04:59

- @vvhitevvizard: Here are some links re. FASM in case they're of interest. Some interesting things were done with AutoIt and FASM (mentioned in the 'resources' link), although I haven't had a chance to investigate them yet.
ASM: ASM (assembly language) to machine code via FASM (flat assembler) - AutoHotkey Community
https://autohotkey.com/boards/viewtopic ... 23&t=49638
resources for learning assembly language - AutoHotkey Community
https://autohotkey.com/boards/viewtopic ... 17&t=57651
- Can someone give me a summary of what people are trying to achieve in this thread? Thanks.
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
User avatar
vvhitevvizard
Posts: 454
Joined: 25 Nov 2018, 10:15
Location: Russia

Re: AHK v2: converting/optimizing scripts

17 Dec 2018, 05:03

jeeswg wrote:
17 Dec 2018, 04:59
- Can someone give me a summary of what people are trying to achieve in this thread? Thanks.
General optimizations for performance and code size (small but effective functions-inliners) for AHK v2. And AHK v1.1 -> v2 conversion issues. And solving some other issues in the bargain.
Last challenge was to replace some logic of json parser with optimized machine code snippets.

thank u for ur links! :D

Some interesting things were done with AutoIt and FASM (mentioned in the 'resources' link), although I haven't had a chance to investigate them yet.
yeah, mixing 2 extremes (interpreting script and optimized machine code) would yield the best of both worlds if used properly. )
this one is very handy in general but changing it to x64 is a nightmare. Its better to start from the scratch by writing algorithm in c and optimizing its x64 machine code after disassembling.
User avatar
vvhitevvizard
Posts: 454
Joined: 25 Nov 2018, 10:15
Location: Russia

Re: AHK v2: converting/optimizing scripts

17 Dec 2018, 10:26

oif2003 wrote:
15 Dec 2018, 03:40
You can test to see that static variables do not work correctly under MCode.
I figured out the issue.

As u can see in the function u used (I modified it and added comments), in order to initialize MCode from the ELF object file, it loads ONLY first section, .text:

Code: Select all

f:=o_load("wstrings.o")
...
o_load(_s){ ;parse ELF64
	b:=FileRead(_s, "RAW")
	h:=NumGet(b,0x28) ;start of program header
	a:=NumGet(b,h+0x58), n:=NumGet(b,h+0x60) ;physical address & size
	p:=DllCall("GlobalAlloc", 'uint',0, 'ptr',n, "PTR")
	;msgbox(Format("0x{:x}", NumGet(o, padd, "uchar")))
	Loop(n//4) ;write program data to memory in 4 byte chuncks
		i:=A_Index-1, NumPut(NumGet(b,a+i*4, "uint"), p,i*4, "uint")
	DllCall("VirtualProtect", "ptr",p, "uint",n, "uint",0x40, "uintp",op)
	return p
;Changes the protection on a region of committed pages in the virtual addr space
;	of the calling process. 0x40: PAGE_EXECUTE_READWRITE. retval 0=FAIL
}	
Static initialized data reside in .data segment, uninitialized data in .bss. So theoretically if we change that loader to load and initialize .data segment at least, we shall have it functioning. Mcode tries to access its data (relative address, no fixups needed) and since data's address is not mapped, we get virtual protection error.
I probably could write Mcode that initializes a few segments for the snippet. But that would be a support for badly designed mcodes.
example: sections for ur file otest.o
Image

what is the relative addr of our initialized value? just a few bytes ahead actually. but within initialized segment .data
(at 0x11 is ur i++, at 0x28 sits ur int32 initialized with 0).
Image

2.
Btw I tested this function with ELF files generated by Fasm. .text section gets imported and mcode works.
So BOTH TCC and FASM can compile text from memory via DLL calls and BOTH can produce the same ELF object file. Thats something )
Last edited by vvhitevvizard on 17 Dec 2018, 12:12, edited 3 times in total.
User avatar
nnnik
Posts: 4500
Joined: 30 Sep 2013, 01:01
Location: Germany

Re: AHK v2: converting/optimizing scripts

17 Dec 2018, 11:47

If you could combine this with the existing framework of:
https://github.com/nnnik/MCode-Ex That would be of great benefit to me :)
Recommends AHK Studio
User avatar
vvhitevvizard
Posts: 454
Joined: 25 Nov 2018, 10:15
Location: Russia

Re: AHK v2: converting/optimizing scripts

17 Dec 2018, 12:37

nnnik wrote:
17 Dec 2018, 11:47
If you could combine this with the existing framework of:
https://github.com/nnnik/MCode-Ex That would be of great benefit to me :)
wow nice framework!

Alas, I don't have installed VC at the moment. :(
We have 2 different object formats here - Linux ELF (used by GCC, Fasm, TCC) and MS COFF .obj (used by VC).
User avatar
nnnik
Posts: 4500
Joined: 30 Sep 2013, 01:01
Location: Germany

Re: AHK v2: converting/optimizing scripts

17 Dec 2018, 12:53

Yeah well the VCC compiling I use is a mess and I had many issues with it - but it eventually worked out for most compiles.
One didnt want to no matter what I did - though I made sure that the resulting MCode is read correctly and my CPU can handle all the instructions and that all the pointers are correct etc. etc. just didnt want to work :(.
I wanted to further use this tool, however I suddenly had issues in the only project I use it in. Which made me doubt its usefulness.
99.9999% of all error messages Windows gives you are non-descriptive 0xC0000005s, incredibly helpful for debugging.
Recommends AHK Studio

Return to “Ask for Help (v2)”

Who is online

Users browsing this forum: Holarctic and 40 guests