AHK v2: converting/optimizing scripts Topic is solved

Get help with using AutoHotkey and its commands and hotkeys
oif2003
Posts: 214
Joined: 17 Oct 2018, 11:43
GitHub: oif2003

Re: Problems with objects and legacy syntax

11 Dec 2018, 18:13

Small modification: use memoization to avoid unneeded string/array manipulation
version1: saving only last string and array

Code: Select all

class _deref {
	static ls := "", la := []
	__Get(s) {
		return ((s != _deref.ls) && _deref.parse(s), %_deref.la[1]%[_deref.la[2]*])
	}	
	__Set(s, v) {
		return ((s != _deref.ls) && _deref.parse(s), %_deref.la[1]%[_deref.la[2]*] := v)
	}
	parse(s) {
		_deref.ls := s, _deref.la := [(m := StrSplit(s, ".")).RemoveAt(1), [m*]]	
	}
} deref := new _deref()
version2: add each entry to dictionary (it's about the same speed as obj.str, which is not very impressive)

Code: Select all

class _deref {
	static dict := {}
	__Get(s) {
		return (_deref.dict.haskey(s) || this.parse(s), %_deref.dict[s][1]%[_deref.dict[s][2]*])
	}	
	__Set(s, v) {
		return (_deref.dict.haskey(s) || this.parse(s), %_deref.dict[s][1]%[_deref.dict[s][2]*] := v)
	}
	parse(s) {
		_deref.dict[s] := [(m := StrSplit(s, ".")).RemoveAt(1), [m*]]	
	}
} deref := new _deref()
User avatar
vvhitevvizard
Posts: 405
Joined: 25 Nov 2018, 10:15
Location: Russia

Re: Problems with objects and legacy syntax

11 Dec 2018, 19:00

and unlike prototype class definition last version's class instance deref has to be initiated with-in auto-execute code area and declared as global

Code: Select all

global deref:=new {__Set:(_,s,v)=>%(m:=StrSplit(s,".")).RemoveAt(1)%[m*]:=v
	, __Get:(_,s)=>%(m:=StrSplit(s,".")).RemoveAt(1)%[m*]}
otherwise deref becomes an uninitialized var (deref=""), Type(deref)="String" instead of "Object"
oif2003
Posts: 214
Joined: 17 Oct 2018, 11:43
GitHub: oif2003

Re: Problems with objects and legacy syntax

11 Dec 2018, 19:07

Good point. I prefer the self initializing version so I can put it anywhere. Maybe I can combine the two versions above without incurring much penalty. I'll test it out later.
User avatar
vvhitevvizard
Posts: 405
Joined: 25 Nov 2018, 10:15
Location: Russia

Re: Problems with objects and legacy syntax

11 Dec 2018, 19:15

oif2003 wrote:
11 Dec 2018, 18:13
Small modification: use memoization to avoid unneeded string/array manipulation
version1: saving only last string and array
I like the way it goes. Results for a.b.c.key:
2406 | 1781 | 2047 : 3000000
But now u try to hone it for the benchmark where we use only 1 address :D 2nd variant with dictionary use would be fair one but speed is slower than prototype method (Testing Case 3): 2360 | 2109 | 2031 : 3000000
Last edited by vvhitevvizard on 11 Dec 2018, 19:49, edited 3 times in total.
oif2003
Posts: 214
Joined: 17 Oct 2018, 11:43
GitHub: oif2003

Re: Problems with objects and legacy syntax

11 Dec 2018, 19:24

Another way to do this is to cache the object address. I will try it later when I get a chance. We should start benchmarking it against the plain a.test.key. I think we are only about 8x away
User avatar
vvhitevvizard
Posts: 405
Joined: 25 Nov 2018, 10:15
Location: Russia

Re: Problems with objects and legacy syntax

11 Dec 2018, 19:26

we might increase performance by setting capacity for the array used: m.SetCapacity(1000) oh nvm it doesn't work with StrSplit().
well, another way is to use Loop Parse and move thru it instead of creating an additional array with StrSplit(). But usual level of nestedness in the case shouldn't be high so I don't think high of changing the way we parse it.
User avatar
vvhitevvizard
Posts: 405
Joined: 25 Nov 2018, 10:15
Location: Russia

Re: Problems with objects and legacy syntax

11 Dec 2018, 19:43

offtopic, right now I'm trying to optimize json.set "json.stringify" functionality (mine from the scratch) and json.get "json.parse" (heavily changed coco's version from this forum).
the code below takes a sample of json-string j:='{"r1":{"r2":{"type1":11,"type2":11},"misc":12}, "type":"ask","price":104.2,"amount":101, "a":[111,"aa"]}', converts it to AHK object and then serializes it back to string form. Lines starting from j:=FileRead("1678982546") read a json-file from disk, convert it from and back to json and compare as 2 strings. it also benchmarks the methods (json.get and json.set)

For coco's routine, I re-structured it, optimized, removed all the unnecessary stuff for my task (like support for escape chars) and honed it for AHK v2 only, it expects json input as just
4 fundamental types

main goal is to optimize it for speed (to parse MB-size files) and code size (80 lines, 2Kb for now). Do u think u could optimize json.get method any further? :D
json in/out procs (AHK v2)
Output (for Compact:=0 mode) looks like:
Image
oif2003
Posts: 214
Joined: 17 Oct 2018, 11:43
GitHub: oif2003

Re: Problems with objects and legacy syntax

11 Dec 2018, 21:45

I'm all AHKed out for the day. I tested a few implementations against a.test.key where key is a 20 element array initialized at 0. obj.str is hard to beat, but I still prefer the better looking version that performs within 10% of obj.str and is usually less than 10x slower than operating directly on a.test.key.

Code: Select all

class _deref {
	__New() {
		static autoload := new _deref
		global deref := this
	}
	__Get(s) {
		return %(m := StrSplit(s, ".")).RemoveAt(1)%[m*]
	}	
	__Set(s, v) {
		return %(m := StrSplit(s, ".")).RemoveAt(1)%[m*] := v
	}
}
That JSON class looks good, it is way better than what I came up with on my own, so I don't think I will be of much help there ;)
User avatar
vvhitevvizard
Posts: 405
Joined: 25 Nov 2018, 10:15
Location: Russia

Re: Problems with objects and legacy syntax

12 Dec 2018, 14:58

btw, I noticed u like to use "inline if" construction (condition) && (expression). I used another: (condition) ? (expression):0 (ternary operator with 0 as 3rd operand)

a ? k:="value":0 vs a && k:="value"
Urs seems to be 10% as fast: 344 vs 313 :D

And concerning optimization of json.set method in the listing above - replacing s:="" with VarSetCapacity(s,1024*8) improves performance by 10% for small and medium json strings. 8k is x2 of CPU page size and seems to be the best value.

I do miss some built-in AHK function for cutting off strings by changing its length w.o completely rewriting old|new var's content.
To cut off 2 chars from the string's end, AHK copies the old string's var (not reference but its content, char by char) to a new var: newholder:=SubStr(oldholder, 1, -2) or oldholder:=SubStr(oldholder, 1, -2), in both cases the whole string is copied to retval.
Now imagine this string is 128MB and we have to truncate/concatenate it many times...
Alternative is forcing zero-termination with NumPut+VarSetCapacity:

Code: Select all

s:="1234567"
NumPut(0,s,(StrLen(s)-2)*(A_IsUnicode*2), "uchar") ;terminating zero byte in new place, -2 chars
VarSetCapacity(s, -1) ;for further string manipulations: -1=update the var's internally-stored string length to the length of its current contents
MsgBox(s)
Last edited by vvhitevvizard on 12 Dec 2018, 15:28, edited 2 times in total.
oif2003
Posts: 214
Joined: 17 Oct 2018, 11:43
GitHub: oif2003

Re: Problems with objects and legacy syntax

12 Dec 2018, 15:25

Those are nice tweaks! Please do post the complete class somewhere when your done.

If A then B === A && B
If Not A then B === A || B
are pretty standard ways of inlining IF without ELSE in some languages that short circuit && and ||
User avatar
vvhitevvizard
Posts: 405
Joined: 25 Nov 2018, 10:15
Location: Russia

Re: Problems with objects and legacy syntax

12 Dec 2018, 15:39

oif2003 wrote:
12 Dec 2018, 15:25
Those are nice tweaks! Please do post the complete class somewhere when your done.
json.get is unchanged,
json.set (AHK v2)
oh I forgot: it also requires my SmartR8() routine for pretty-printing Floats (get rid of trailing 0s and ., round to 8 decimals).
e.g. 80.->80, 0005->5, 0.000234567878999->0.00023457):
SmartR8()
I use it for debugging my objects as well:
2nd mentioned tweak is not included yet cuz it seems to be unstable for big strings. I can't figure out why.

btw, return((a ? "[":"{") (s ? (n RTrim(s, "," n) n _d):"") (a ? "]":"}")) ;wrap to brackets
instead of s:=(a ? "[":"{") (s ? (n RTrim(s, "," n) n _d):"") (a ? "]":"}")
return s

saves around 10% CPU time as well. every damned string [re] assignment has a drastic performance hit. meanwhile, concatenate .= operations r quite optimized in AHK.

PS: I wrote SmartR() some time ago as a dirty hack and now I see it can be optimized by speed and size. :D
Last edited by vvhitevvizard on 12 Dec 2018, 16:21, edited 1 time in total.
User avatar
vvhitevvizard
Posts: 405
Joined: 25 Nov 2018, 10:15
Location: Russia

Re: Problems with objects and legacy syntax

12 Dec 2018, 16:16

oif2003 wrote:
12 Dec 2018, 15:25
are pretty standard ways of inlining IF without ELSE in some languages that short circuit && and ||
Yeah! it depends on interpreter optimization. I recall Basic language in old times (~1990) that tried to evaluate the right side expression every time. :)
oif2003
Posts: 214
Joined: 17 Oct 2018, 11:43
GitHub: oif2003

Re: Problems with objects and legacy syntax

12 Dec 2018, 16:42

Just out of curiosity, is there a reason you have to use SmartR8() instead of say a RTrim(_a, ".") and format(..., _a) ?

Not really an improvement, but if you remove the this.space option and opt for fixed spacing, (ie: the default 4), you can write that if else block in ternary form.
User avatar
vvhitevvizard
Posts: 405
Joined: 25 Nov 2018, 10:15
Location: Russia

Re: Problems with objects and legacy syntax

12 Dec 2018, 16:48

oif2003 wrote:
12 Dec 2018, 16:42
Just out of curiosity, is there a reason you have to use SmartR8() instead of say a RTrim(_a, ".") and format(..., _a) ?
Actually it could be replaced with AHK sub-functions and thus improve performance. :D Hehe I told u I wrote this some time ago in a hurry and its this very moment when I looked back to the overall working code )) Feel free to start optimizing it!
the logic consists of 2 more functions SmartKMG(), SmartR(). One is adding K, M, G, effectively reducing numbers by 3,6,9 digits respectively, e.g. 125670.897 -> 125.7K
other removes insignificant digits _intellectually_, e.g. 5676.12345 -> 5676.1 or 0.0234789 -> 0.0235.
These routines r not a part of critical code, they used for output like this:
Image

sorry for some russian comments in the code below. I'm not a native english speaker:

Code: Select all

;если >0: округлить до 3-х значимых знаков после запятой
;если <1: округлить максимум до 3 значимых знаков после запятой
SmartR(_a){
	a:="" _a ;Float(_a)
	if(SubStr(a,1,2)="0."){
		i:=3
		while(SubStr(a,i,1)="0")
			i++
;0.00040989 > 40989
;		 if(i>5)
;			 a:="m" Floor(a*10**8) ;умножить на 10^8 и привести к целому
;		 else
			a:=Round(a,i) ;0.7257889 > 0.726
;the result is a numeric string with exactly N decimal places.
;	If a pure number is needed, simply perform another math operation on Round's return value
;	eg: Round(3.333, 1)+0
	}
	else{ ;724.76400989 > 724.8
		i:=InStr(a, ".")
		if(i<5)
			a:=(Mod(a,1)) ? Round(a,5-i):Round(a) ;не равна ли дробная часть нулю
		else
			a:=Round(a)
	}
	return SmartT(a)
}
SmartKMG(_a){
	return (_a<1000) ? SmartR(_a)
		: (_a<10**6) ? SmartR(_a/1000) "K" ;power of 10: =1000000
		: (_a<10**9) ? SmartR(_a/10**6) "M"
		: (_a<10**12) ? SmartR(_a/10**6) "G":_a
}
oif2003 wrote:
12 Dec 2018, 16:42
Not really an improvement, but if you remove the this.space option and opt for fixed spacing, (ie: the default 4), you can write that if else block in ternary form.
thank u for feedback!
this.Space is a key adjustable feature here for human-readable output. Along with this.Compact=1 which enables compressed output w/o any linefeeds and spaces.
Also there is a block for built-in objects handling which is not directly related to pass-thru converting to/from json data but usable for debugging.
I let this functional to be while sacrificing some performance hit. Well, an alternative solution could be 2 get/set pairs, the second pair as bare-bone, optimized just for speed. but again, after having the optimal algorithm settled down, I could re-write it in C with SSE optimized assembler snippets and use compiled DLL calls or insert the machine code in AHK script with Mcode(). Maybe later I'm not that desperate atm.

I could replace if(expression)
(expression)

with
(expression) && (expression) inliners
in many places (especially within json.get()). :D But I meant logic optimization, removing unneeded steps, etc
oif2003
Posts: 214
Joined: 17 Oct 2018, 11:43
GitHub: oif2003

Re: Problems with objects and legacy syntax

12 Dec 2018, 18:15

vvhitevvizard wrote:
12 Dec 2018, 16:48
I could replace if(expression)
(expression)

with
(expression) && (expression) inliners
in many places (especially within json.get()). :D But I meant logic optimization, removing unneeded steps, etc
Funny you said that, cuz I got bored and that's exactly what I did lol (among other trivial things)
I did make the indentation (this.Space) loop run only when !_d and gave it a static variable, other than that, I made fthrow method cuz I hate commands!
Also replaced SmartR8 with format("{:g}", v), but I imagine you probably want something more robust?

Code: Select all

class json {
;json.set by vvhitevvizard. json.get by coco and modified by vvhitevvizard
	get(ByRef _s) { ;obj:=json.get(str)
		static q:=Chr(34), p:={"__Arr":1} ,x:=q "{[0123456789-" ;json allowed chars outside strings
		n:=0, k:="",kf:=0, s:=[root:={}], z:="{"

		while (c:=SubStr(_s, ++n, 1))!=="" {
			if InStr(" `t`n`r", c)
				continue
			InStr(z, c) || E(c, n), a:=(b:=s.1).__Arr

			if InStr("}]", c) {
				(s.1=root) && E(c, n), s.RemoveAt(1), z:=s.1.__Arr ? ",]" : ",}"
			} else if InStr(",:", c) {
				z:=(kf:=!a && c==",") ? q : x
			} else if InStr("{[", c) {
				(c=="{") ? (kf:=1, v:={}, z:=q "}") : (v:=new p, z:=x "]"), s.InsertAt(1,v)
				,a ? k:=b.Push(v) : b[k]:=v
			} else {
				if c==q { ;string literals
					v:=SubStr(_s, n+1, InStr(_s,q,, n+1)-n-1), n += StrLen(v)+1
					if kf {
						k:=v, z:=":"
						continue
					}
				} else { ;number
					v:=SubStr(_s, n, (SubStr(_s, n) ~= "[\]\},\s]|$")-1), n += StrLen(v)-1
					,(v is "Number") ? v += 0 : this.E(c, n)
				} 
				a ? k:=b.Push(v) : b[k]:=v, z := a ? ",]" : ",}"
			}
		} 
		return (s.1==root || this.E(c, n), b) ;s.Count()!=1 ;unpaired {}, []
		
		E(_c,_n) => this.fthrow(Exception("Unexpected char <" _c "> in pos " _n, -1))
	}

	;"stringify": format [associative] array (object) as JSON string: str:=json.set(obj)
	;by vvhitevvizard
	;0=enable indentation and line feed, 1=the most compact representation:
	static Compact:=0
	;spaces per level, similar to JavaScript's JSON.stringify() 'space' parameter
	;	JSON array elements and object members will be pretty-printed with the indent level:
	static Space:=4
	;0=suppress error on built-in object types, just output a class name, e.g.: "File Object":
	static ErrObjType:=0
	;0=supress error if the root is not an object:
	static ErrObj:=0

	set(_o, _d:="") { ;_d:current indent
		if !IsObject(_o) ;"string" | number
			return this.ErrObj && this.fthrow(Exception("Not an object.", -1))
		static q:= Chr(34), ind
		(_d) || ind := indent(this.Space), (this.Compact) ? n:=d:="" : n:="`n", d:=_d ind
		,VarSetCapacity(s,1024*8)

		;Check object type, skipping of non-serializable objects such as COM, Func, BoundFunc, File, etc
		;	otherwise we get "Unknown method _NewEnum" for loop:
		try a:=_o.__Arr ;trying to get a unique key
		catch ;"Unknown property" exception means its a built-in inenumerable object
			return (t:=Type(_o) " Object"
					, (this.ErrObjType) && this.fthrow(Exception(t " type is not supported.", -1))
					, "{" q t q "}")
		;due to the use of a for-loop, arrays such as '[1,,3]' are detected as objects({})
		for i in _o
			if !(a:=i=A_Index) ;a=0:associative or sparse array ([1,,3])
				break
		for k, v in _o ;recursive
		;JSON standard: keys are always "strings", values r: number|"string"|{object}|[array]
			s .=  d (a ? "" : q k q ":") (isObject(v) ? this.set(v, d)
				: (Type(v)=="String") ? q v q
				: (Type(v)=="Float") ? format("{:g}", v)
				: v) "," n
		return (a ? "[" : "{") (s ? (n RTrim(s, "," n) n _d) : "") (a ? "]" : "}") ;wrap to brackets
		
		indent(n) {
			loop(n)
				_ .= " "
			return _
		} 
	}
	
	fthrow(e) {
		throw e
	}
}

Dump(_s, _z:="") { ;dump text to file with random name
	f:=FileOpen(_z ? _z:A_TickCount,"w"), n:=f.Write(_s)
	f.Close() ;also flushes any data in the cache to disk and releases the share locks
	return n
}


j:='{"r1":{"r2":{"type1":11,"type2":11},"misc":12}, "type":"ask","price":104.2,"amount":101, "a":[111,"aa"]}'

if(0) {
j:=FileRead("1678982546")

	b:=json.get(j)
	s:=json.set(b)
	if(j==s)
		msgbox("in=out")
	else
		Dump(s, "out")
}
else {
	loop(5000)
		b:=json.get(j)

	t:=A_TickCount
		loop(20000)
			b:=json.get(j)
	a1:=A_TickCount-t

	t:=A_TickCount
	loop(20000)
		s:=json.set(b)
	a2:=A_TickCount-t
		
	msgbox(clipboard:=a1 "|" a2 "`n`n" s)
}
User avatar
vvhitevvizard
Posts: 405
Joined: 25 Nov 2018, 10:15
Location: Russia

Re: Problems with objects and legacy syntax

12 Dec 2018, 18:21

I could use a closure (sub-func) and move away all the settings/1-time checks to the outer func:
Spoiler
but looks like it doesn't win much performance gain: 469 vs 500. +6% And there is a new task of pushing.popping back 1 more var (stack of indent levels) cuz a closure has an access to its parent's locals - that's an issue for our recursion. Or to subtract the level back in closure itself - and here we return to the question of the reasonability. :D
Last edited by vvhitevvizard on 12 Dec 2018, 19:21, edited 1 time in total.
oif2003
Posts: 214
Joined: 17 Oct 2018, 11:43
GitHub: oif2003

Re: Problems with objects and legacy syntax

12 Dec 2018, 18:37

I knew there's a better way to write that indent function:

Code: Select all

indent(n) => n ? indent(--n) " " : "" 
User avatar
vvhitevvizard
Posts: 405
Joined: 25 Nov 2018, 10:15
Location: Russia

Re: Problems with objects and legacy syntax

12 Dec 2018, 18:45

oif2003 wrote:
12 Dec 2018, 18:15
I did make the indentation (this.Space) loop run only when !_d and gave it a static variable, other than that, I made fthrow method cuz I hate commands!
Also replaced SmartR8 with format("{:g}", v), but I imagine you probably want something more robust?
First I benchmarked it. 425 vs 500 +15% Well done, sir! :D

SmartR8() could be just Round(v, 8) (or any n of decimals suitable) just for a performance sake in here.
format("{:g}", 0.1234567) vs Round(8, 0.1234567). 1516 vs 140. Round() is x10 as fast.
Last edited by vvhitevvizard on 12 Dec 2018, 19:22, edited 1 time in total.
oif2003
Posts: 214
Joined: 17 Oct 2018, 11:43
GitHub: oif2003

Re: Problems with objects and legacy syntax

12 Dec 2018, 18:46

oif2003 wrote:
12 Dec 2018, 18:37
I knew there's a better way to write that indent function:

Code: Select all

indent(n) => n ? indent(--n) " " : ""
Disregard this change, it's too slow! Didn't think there'd be a perceptible difference, but there is!
oif2003
Posts: 214
Joined: 17 Oct 2018, 11:43
GitHub: oif2003

Re: Problems with objects and legacy syntax

12 Dec 2018, 18:51

vvhitevvizard wrote:
12 Dec 2018, 18:45
SmartR8() could be just Round(v, 8) (or any n of decimals suitable) just for a performance sake in here.
format("{:g}", 0.1234567) vs Round(8, 0.1234567). 1516 vs 140. Round() is x10 as fast.
But I don't think round(v, 8) handles trailing zeros?

Return to “Ask For Help”

Who is online

Users browsing this forum: electrone77, Pid16, RubbeH, tmplinshi and 72 guests