Multiple scripts with same variables
-
- Posts: 44
- Joined: 08 Nov 2021, 23:40
Multiple scripts with same variables
Hi all,
I am using multiple scripts which refer to huge variable sets.
Rather than running the scripts and saving variables for each instance (which seems really inefficient), is there a way to store the variables in memory and have all scripts refer to their values?
Note: ini's/file storage is too slow for my use case.
thanks everyone!
I am using multiple scripts which refer to huge variable sets.
Rather than running the scripts and saving variables for each instance (which seems really inefficient), is there a way to store the variables in memory and have all scripts refer to their values?
Note: ini's/file storage is too slow for my use case.
thanks everyone!
- flyingDman
- Posts: 2817
- Joined: 29 Sep 2013, 19:01
-
- Posts: 44
- Joined: 08 Nov 2021, 23:40
Re: Multiple scripts with same variables
probably 60k variables per script...I would rather have 60k total that 3 scripts refer to than 180k.
Not sure how I would go about posting examples... just heaps of var := val essentially.
Not sure how I would go about posting examples... just heaps of var := val essentially.
Re: Multiple scripts with same variables
- write the variables' contents to a common region in memory https://docs.microsoft.com/en-us/windows/win32/memory/creating-named-shared-memory
- write the variables' contents to an object, then use viewtopic.php?t=6148
-
- Posts: 44
- Joined: 08 Nov 2021, 23:40
Re: Multiple scripts with same variables
Yeah I probably am doing something wrong.
I'm using an api to save price data and other information for thousands of companies. I use arrays where I can, but found that searching arrays was a little too slow for my needs. So I put all data in variables so I can at least access them quickly.
I'm using an api to save price data and other information for thousands of companies. I use arrays where I can, but found that searching arrays was a little too slow for my needs. So I put all data in variables so I can at least access them quickly.
Re: Multiple scripts with same variables
I’m not seeing why you would have to search arrays but you can instantly access one of thousands of simple variables. How (and why) were you searching your array? Were you looping through it or something? How are you referencing the desired variable out of 60,000 variables? Does each variable have the company name as well as the name of property (such as price) that it contains? Did you know that you can use an associative array to identify the desired element directly with that same information instead of searching through a simple array?
-
- Posts: 44
- Joined: 08 Nov 2021, 23:40
Re: Multiple scripts with same variables
I definitely don't have to, think I'm just not familiar with what a better option is.boiler wrote: ↑16 Jan 2022, 07:31I’m not seeing why you would have to search arrays but you can instantly access one of thousands of simple variables. How (and why) were you searching your array? Were you looping through it or something? How are you referencing the desired variable out of 60,000 variables? Does each variable have the company name as well as the name of property (such as price) that it contains? Did you know that you can use an associative array to identify the desired element directly with that same information instead of searching through a simple array?
Currently I use a script that builds arrays/simple variable lists and saves to file.
These are loaded by a 'processor' script, which reads these variables at startup and then handles decision logic.
Code: Select all
;============================================================================================
GetCoinIDs() { ; get coingecko ids
;============================================================================================
global
Loop 6
{
command=\K -H ""Content-Type: application/json"" -X GET https://api.coingecko.com/api/v3/coins/markets?vs_currency=usd&order=market_cap_desc&per_page=250&page=1&sparkline=false"" ",," hide
stringreplace, command, command, page=1, page=%A_index%
;msgbox command: %command%
out := RunGetOutput("curl",command)
;clipboard := out
;pause
stringreplace, out, out, ",,A
stringreplace, out, out, },,A
stringreplace, out, out, {,,A
stringreplace, out, out, ],,A
stringreplace, out, out, [,,A
stringsplit, spl, out, `,
coincount=1
StartTime := A_TickCount
loop %spl0%
{
thisspl := spl%A_index%
stringsplit, subspl, thisspl, :
label := subspl1
value := subspl2
if instr(label,"symbol")
{
symbolarray=%symbolarray%,%value%
symbolcount++
lastsymbol=%value%
bigarray=%bigarray%,[%lastsymbol%]%lastid%
}
if instr(label,"id")
{
idarray=%idarray%,%value%
idcount++
lastid=%value%
}
if instr(label,"name")
{
namearray=%namearray%,%value%
CoinList1=%coinlist1%%lastsymbol%`r`n%value%`r`n
namecount++
lastname=%value%
}
}
}
;======================================
; housekeeping name array
;======================================
stringreplace, namearray, namearray,%A_space%Protocol,,A
stringreplace, namearray, namearray,%A_space%Token,,A
stringreplace, namearray, namearray,%A_space%Network,,A
stringreplace, namearray, namearray,%A_space%Coin,,A
stringreplace, coinlist1, coinlist1,%A_space%Protocol,,A
stringreplace, coinlist1, coinlist1,%A_space%Token,,A
stringreplace, coinlist1, coinlist1,%A_space%Network,,A
stringreplace, coinlist1, coinlist1,%A_space%Coin,,A
stringreplace, coinlist1, coinlist1,the-,,A
stringreplace, coinlist1, coinlist1, -network,,A
stringreplace, coinlist1, coinlist1, -usd,,A
stringreplace, coinlist1, coinlist1, -protocol,,A
stringreplace, coinlist1, coinlist1, -coin,,A
stringreplace, coinlist1, coinlist1, -token,,A
stringreplace, coinlist1, coinlist1, -standard,,A
stringreplace, coinlist1, coinlist1, -governance,,A
stringreplace, coinlist1, coinlist1, -,%A_space%,A
fileappend,%symbolarray%,E:\Trade\ref\symbols.txt
fileappend,%namearray%,E:\Trade\ref\names.txt
fileappend,%idarray%,E:\TradeBot\ref\ids.txt
fileappend,%bigarray%,E:\Trade\ref\bigarray.txt
fileappend,%coinlist1%,E:\Trade\ref\CoinListNEW.txt
symbolarray=
namearray=
idarray=
bigarray=
}
clipboard := coinlist1
msgbox idcount: %idcount%`nsymbolcount: %symbolcount%
return
Re: Multiple scripts with same variables
OK, well you're basically building "pseudo-arrays" in a sense, but it looks even worse in the sense that apparently you're making a huge string that has a ton of keys and values strung together. You would be somewhat better off using psuedo-arrays, where at least they're separate variables that have different names that you could key off of like you would with a true array's keys. But the best would be to use real object-based arrays. Like I was saying, there would be no searching involved if you used associative arrays where you could instantly obtain the value of any element based on the key(s).
-
- Posts: 44
- Joined: 08 Nov 2021, 23:40
Re: Multiple scripts with same variables
thanks boiler, i'll look into this and wb!
-
- Posts: 44
- Joined: 08 Nov 2021, 23:40
Re: Multiple scripts with same variables
@boiler one question - would it be better/is there a need to create associative arrays when building datasets, and having child scripts that can refer to their location (rather than building text files and then using associative arrays from child scripts like I am currently thinking of doing)?
Objects/Classes etc have
Objects/Classes etc have
Re: Multiple scripts with same variables
It is easy to share your main script's array object with your other scripts using ObjRegisterActive as swagfag recommended. Once you've shared it, accessing the object from the child scripts would be exactly the same (with some relatively unimportant limitations) as you would access it from the main script. And in general, using objects is a better approach than strings of text.
- flyingDman
- Posts: 2817
- Joined: 29 Sep 2013, 19:01
Re: Multiple scripts with same variables
So I decided to test this for speed. I made a look-up application. I created a 60,000 row text file of 2 columns. First column contains strings of 8 random letters and the second contains random 8 digit integers. The 2 columns are tab separated. Looks like this:
I changed one random line to AAAAAAAA 99999999. Goal is to find AAAAAAAA and return 99999999.
I have a server script:
and a receiving script:
It gets the result in a lousy ~3.2 seconds
Compared with a fileread script like this:
which does it 100 times faster, i.e. in ~32 ms!
What am I doing wrong? Or is this it? (in which case ObjRegisterActive should not be used for this type of application....)
Code: Select all
IGPGHHTT 50323898
WOOXKZFB 86657532
NYSJVTMB 56869807
FTGEELLG 04950696
ABRHWNHX 51560344
FBNZMAXR 88955001
SWJSETSL 87320915
GJSFYBWU 50882448
I have a server script:
Code: Select all
#Persistent
OnExit Revoke
fileread, var, twocolumn.txt
AnObject := strsplit(var,"`n")
ObjRegisterActive(AnObject, "{072736AA-BC5F-4AB3-8E6A-1311B2ED444F}")
return
Revoke:
ObjRegisterActive(ActiveObject, "")
ExitApp
ObjRegisterActive(Object, CLSID, Flags:=0) {
static cookieJar := {}
if (!CLSID) {
if (cookie := cookieJar.Remove(Object)) != ""
DllCall("oleaut32\RevokeActiveObject", "uint", cookie, "ptr", 0)
return
}
if cookieJar[Object]
throw Exception("Object is already registered", -1)
VarSetCapacity(_clsid, 16, 0)
if (hr := DllCall("ole32\CLSIDFromString", "wstr", CLSID, "ptr", &_clsid)) < 0
throw Exception("Invalid CLSID", -1, CLSID)
hr := DllCall("oleaut32\RegisterActiveObject"
, "ptr", &Object, "ptr", &_clsid, "uint", Flags, "uint*", cookie
, "uint")
if hr < 0
throw Exception(format("Error 0x{:x}", hr), -1)
cookieJar[Object] := cookie
}
Code: Select all
starttime := a_tickcount
x := ComObjActive("{072736AA-BC5F-4AB3-8E6A-1311B2ED444F}")
loop, % x.maxindex()
if (strsplit(x[a_index],"`t")[1] = "AAAAAAAA")
msgbox % strsplit(x[a_index],"`t")[2] "`n" a_tickcount - starttime
Compared with a fileread script like this:
Code: Select all
starttime := a_tickcount
fileread, var, twocolumn.txt
for x,y in strsplit(var,"`n")
if (strsplit(y,"`t").1 = "AAAAAAAA")
msgbox % strsplit(y,"`t").2 "`n" a_tickcount - starttime
What am I doing wrong? Or is this it? (in which case ObjRegisterActive should not be used for this type of application....)
14.3 & 1.3.7
Re: Multiple scripts with same variables
But what if with the object approach, you made it an associative array like I was recommending so it would eliminate the need to do any searching? Then instead of looping until you find it, it would just be simply:
Code: Select all
MsgBox, % MyArray["AAAAAAAA"]
- flyingDman
- Posts: 2817
- Joined: 29 Sep 2013, 19:01
Re: Multiple scripts with same variables
Thank you. Big difference! and faster than fileread.
Serving script:
receiving script:
Serving script:
Code: Select all
#Persistent
OnExit Revoke
fileread, var, twocolumn.txt
AnObject := {}
for x,y in strsplit(var,"`n")
AnObject[strsplit(y,"`t").1] := strsplit(y,"`t").2
ObjRegisterActive(AnObject, "{072736AA-BC5F-4AB3-8E6A-1311B2ED444F}")
return
Revoke:
ObjRegisterActive(ActiveObject, "")
ExitApp
ObjRegisterActive(Object, CLSID, Flags:=0) {
static cookieJar := {}
if (!CLSID) {
if (cookie := cookieJar.Remove(Object)) != ""
DllCall("oleaut32\RevokeActiveObject", "uint", cookie, "ptr", 0)
return
}
if cookieJar[Object]
throw Exception("Object is already registered", -1)
VarSetCapacity(_clsid, 16, 0)
if (hr := DllCall("ole32\CLSIDFromString", "wstr", CLSID, "ptr", &_clsid)) < 0
throw Exception("Invalid CLSID", -1, CLSID)
hr := DllCall("oleaut32\RegisterActiveObject"
, "ptr", &Object, "ptr", &_clsid, "uint", Flags, "uint*", cookie
, "uint")
if hr < 0
throw Exception(format("Error 0x{:x}", hr), -1)
cookieJar[Object] := cookie
}
Code: Select all
starttime := a_tickcount
x := ComObjActive("{072736AA-BC5F-4AB3-8E6A-1311B2ED444F}")
msgbox % x["AAAAAAAA"] "`n" a_tickcount - starttime
14.3 & 1.3.7
Re: Multiple scripts with same variables
Nice. Thanks for running that.flyingDman wrote: ↑ Big difference! and faster than fileread.