Forking a script execution without adding code to each thread separatedly.

Post AHK_H specific scripts & libraries and discuss the usage and development of HotKeyIt's fork/branch
letacio
Posts: 8
Joined: 08 Mar 2018, 16:05

Forking a script execution without adding code to each thread separatedly.

31 Oct 2018, 09:50

Hello everybody!

I have a question, which i looked a lot and couldn't find any way for it to work, and even if it is possible.

The thing is:

I have a script, where i have a lots of variables.
After i load all this variables, i want to process them in separate threads, at the same time.

For instance, i will have 4 subroutines (each one makes a REST api call, then process the retrieved data), and each one takes around 2 seconds to complete. So, the report i want to generate takes around 8 seconds to be ready. I wish to use 4 threads at least, so the time goes to 2 or 3 seconds total time, instead of 8.

After that, i display the report on a GUI.

The thing is: i DO NOT want to write all data to a file, then access this file with the threads. It looks too much time consuming. I wanted to do it in a more elegant way.

The closer i got to what i wanted was using "ahklabel" (https://hotkeyit.github.io/v2/docs/comm ... kLabel.htm), but the problem is that it creates a thread, executes it, then creates another one, execute it, and so on (which means, it does not multithread in reality, and takes the same time as if there were no threads).

Plus, i dont want to copy tons of code inside the thread code again. I wish it would just gosub to the labels from my running script, and execute it separatedly.

The example "code" i put here is very simple, and takes no time to execute.

But the my real application is done with REST API calls. As statedI call 4 rest api calls to get some json data, then do some processing afterwards. Each of this call/processing takes around 2 seconds, so my report is generated in around 8 seconds.


[code] ::show report:: ;example pseudocode

workfile = october work logs ;this sets a variable before starting the new threads. some of the new threads might need this value to continue. for instance, the "readtime" label right ahead will need this value to know which .ini file to read from.

mondaytime = 2 ;number of hours worked in each week day. in here, the data is hardcoded to make the example easier. in real application, the data is read from other source, like a gui, or a REST call to a site.

tuesdaytime = 3
thursdaytime = 1

gosub, readtime ; i want to call THIS sub as another thread.
gosub, sumweektime ; i want to call THIS sub as another thread.
gosub, RESTapicallA ; i want to call THIS sub as another thread.
gosub, RESTapicallB ; i want to call THIS sub as another thread.

;here i check if the both threads are already finished, and the i display data on a gui interface.

while (threads are running) ;waits until threads finish
{
sleep 100
}

msgbox, worktime: %worktime% , sumtime: %sumtime% ;after thread finish, display results on screen.

return

;subroutines

readtime:

IniRead, worktime, %A_ScriptFullPath%, %workfile%.ini, worktime ;gets the worktime from the october logs.

return

sumweektime:

sumtime = mondaytime + tuesdaytime + thursdaytime

return

RESTapicallA:
(...) ; do rest api call, and process data
return

RESTapicallB:
(...) ; do rest api call, and process data
return [/code]
HotKeyIt
Posts: 1705
Joined: 29 Sep 2013, 18:35
Contact:

Re: Forking a script execution without adding code to each thread separatedly.

31 Oct 2018, 18:25

That is a way to do it, as you described.
[code]threads:=[]
Loop 4
threads[A_Index]:=AhkThread("
(
#Persistent
Alias(mondaytime," getvar(mondaytime) ")
Alias(tuesdaytime," getvar(tuesdaytime) ")
Alias(thursdaytime," getvar(thursdaytime) ")
Alias(worktime," getvar(worktime) ")
Alias(sumtime," getvar(sumtime) ")
Alias(testvar," getvar(testvar) ")
Alias(testvar2," getvar(testvar2) ")
" CreateScript("<subroutines:subroutines>") "
)")
::show report:: ;example pseudocode

workfile = october work logs ;this sets a variable before starting the new threads. some of the new threads might need this value to continue. for instance, the "readtime" label right ahead will need this value to know which .ini file to read from.

mondaytime = 2 ;number of hours worked in each week day. in here, the data is hardcoded to make the example easier. in real application, the data is read from other source, like a gui, or a REST call to a site.

tuesdaytime = 3
thursdaytime = 1

threads.1.ahkLabel("readtime")
threads.2.ahkLabel("sumweektime")
threads.3.ahkLabel("RESTapicallA")
threads.4.ahkLabel("RESTapicallB")

;here i check if the both threads are already finished, and the i display data on a gui interface.

while !threads.1.ahkgetvar.done || !threads.2.ahkgetvar.done || !threads.3.ahkgetvar.done || !threads.4.ahkgetvar.done ;waits until all threads finish
{
sleep 100
}

msgbox, worktime: %worktime%`nsumtime: %sumtime%`n testvar: %testvar%`n testvar2: %testvar2% ;after thread finish, display results on screen.

return

<subroutines:

readtime:

IniRead, worktime, %A_ScriptFullPath%, %workfile%.ini, worktime ;gets the worktime from the october logs.
done:=1
return

sumweektime:

sumtime := mondaytime + tuesdaytime + thursdaytime
done:=1
return

RESTapicallA:
testvar:=A_TickCount
done:=1
return

RESTapicallB:
testvar2:=A_Now
done:=1
return

subroutines>:
Return[/code]
letacio
Posts: 8
Joined: 08 Mar 2018, 16:05

Re: Forking a script execution without adding code to each thread separatedly.

31 Oct 2018, 20:57

Thanks a lot!

I'll study your code and implement in my application!

In the meantime, could you explain some of your code?

a) do i need to use the array to create the threads? i could create them manually for each task i wanted, and you did in an elegant way (not to paste the code 4 times), or is mandatory?

b) what does this code stands for? I think i've seen it while looking for my answers, but didnt make sense at the time, and now i can't find it anymore, nor documentation about it.

<subroutines:

;some code

subroutines>:


C) for what i understand, using the alias makes all threads point to the same memory space? this means, each time someone modifies the variables, the data will be already available for everyone, including the main thread?

Thanks again!
HotKeyIt
Posts: 1705
Joined: 29 Sep 2013, 18:35
Contact:

Re: Forking a script execution without adding code to each thread separatedly.

01 Nov 2018, 06:12

a) No you don't need them, it makes the code cleaner.
b) This is how CreateScript parses it, from label to label, see CreateScript.
c) No, they point to the other variable which is the only that points to the memory space, Alias is not multi-thread safe, consider using CriticalObject if you need to access same data from multiple threads.
letacio
Posts: 8
Joined: 08 Mar 2018, 16:05

Re: Forking a script execution without adding code to each thread separatedly.

01 Nov 2018, 14:30

Hello again!

I've done a bug bunch of testing, but i got some issues, and it does not work nicely yet. Im not sure if im writing code correctly.

First, questions to sintax and code meaning:

A) About "createscript": there is no mention to this sintax (with the little arrow, like <subroutine : subroutine >: ) in the create script page (https://hotkeyit.github.io/v2/docs/comm ... Script.htm)

A.1) Does all the code need to be inside the same ahk.file? i made a #include inside the thread code, as i would need other functions and subs for data processing. can i do it with no issues? i looks like it works.

threads:=[]
Loop 4
threads[A_Index]:=AhkThread("
(
# include subroutines.ahk ; <---------------

#Persistent
Alias(mondaytime," getvar(mondaytime) ")
Alias(tuesdaytime," getvar(tuesdaytime) ")
Alias(thursdaytime," getvar(thursdaytime) ")
Alias(worktime," getvar(worktime) ")
Alias(sumtime," getvar(sumtime) ")
Alias(testvar," getvar(testvar) ")
Alias(testvar2," getvar(testvar2) ")
" CreateScript("<subroutines:subroutines>") "
)")

B) About the "#persistent" : why is it there? and why is this necessary?

i've couldnt note a difference by removing this. after i run scripts, they wont close anymore. maybe it's because my main script has already the "persistent" parameter?

C) The sintax you sent me looks like it opens a copy of the currently running script, is it correct? For instance, when i run it, it displayes 4 extra "main.ahk" icons on my system tray

D) Instead of using the "done:=1" variable to check if the thread is over, can i use ahkReady()? i assume not, as they are persistent, so they never finish. but that was a question i had.

E)
In this code, you wrote:
threads[A_Index]:=AhkThread

so, to "call" the functions, should you not write this way?

threads[1].ahkLabel("readtime")
threads[2].ahkLabel("sumweektime")
threads[3].ahkLabel("RESTapicallA")
threads[4].ahkLabel("RESTapicallB")

Instead of the way you used, with the dots?

threads.1.ahkLabel("readtime")
threads.2.ahkLabel("sumweektime")
threads.3.ahkLabel("RESTapicallA")
threads.4.ahkLabel("RESTapicallB")

Still about this sintax, in the docs. (https://hotkeyit.github.io/v2/docs/comm ... kLabel.htm), you used this sintax:

dll.ahkLabel["MyLabel"] (with the [])

But in you answer to me, u used () instead, like this -> dll.ahkLabel("MyLabel").
Are they equivalent?

F) About the Alias function: i should do the running of the threads, with the alias before using my main program variables? or does not matter? you started the thread before the "actual code". is it necessary?

F.1) If i get it right, when i use the alias inside a thread, like you did, for instance, if inside my thread i create a msgbox with the value of the variable alias, it would display it correctly? i am passing the value from my main thread, to the newly created threads, am i right? or not? because i notice when i try to use the variables inside the threads, they are empty. I've seen a post from someone saying he/she couldnt ever use the alias function with success. maybe its not working or am i wrong in how using it?

Still about it, in documentation (https://hotkeyit.github.io/v2/docs/commands/Alias.htm) the sintax is done without " " , like this:

Alias(refvar, getvar(var)) <- documentation
Alias(refvar, " getvar(var) ") <- your example to me

which one is correct?

G) About this : " CreateScript("<subroutines:subroutines>")

I understand you are adding some code with this comand, right?
But the thing is: the labels inside this are already in my code. when i include my files, it gives me "duplicate labels" error.
so i removed this comand, and only added the #include in the initiation of the threads. it worked i think, but i would like to know if you have any thoughts on it.

ABOUT EXECUTION

H) Now there is an issue i'm not really sure why it's happening, and not sure how to solve it as well.

When i run the script, for some reason it looks like one of the threads runs a lot of times, then the other threads run one time each.

I'm not sure actually if the following is working correctly.

threads.1.ahkLabel("timereport") <-- actual code of my script

For instance, i have 4 labels. each one displays a msg when starting (so i can know if they are starting at the same time or almost).

label1:
msgbox 1
....
return

label2:
msgbox 2
....
return

label3:
msgbox 3
....
return

label4:
msgbox 4
....
return

so, when i run script, i get a lot of messageboxes with 1, at the same time.
Then the 2, 3 and 4 comes one each time, when i click "OK" on the already existing msgboxes.

the curious things:

1: it happens even when i dont have the threads.1.ahkLabel("timereport") code. in my understanding, im not calling anything inside the start of the thread. so no msgbox should show. but it does, like it was doing the subroutine. but i didnt call it.

2: it get a particular label to execute a lot of times, and i dont know why. if there is a value somewhere in memory that is "static", so it gets always the same label to execute

3: looks like that ALL threads execute this label first, then the start to get the other labels to execute.

for instance:

create thread 1, thread 2, thread 3, thread 4

FIRST: all of them execute label 1. (so i get 4 msgboxes with "1" text). looks like when there are 4 "main.ahk" running, and i call a label, it calls the label on the 4 main.ahk processess. but the same does not happen to other labels, which i don't understand.

then, i dont know why, ONE of the threads get responsible for the other numbers.

So it displays a 2 on the screen, and wait until i hit "OK". then displays 3, and wait. then 4. then wait. just then the script moves forward.

my actual code:

threadD:= AhkThread(" <- this is the fourth thread. there are more 3 of them, being threadA, B, C.
(
#Include %A_ScriptDir%/Jira.ahk

Alias(project1," getvar(project1) ")
Alias(version1," getvar(version1) ")
Alias(datainicio," getvar(datainicio) ")
Alias(horacorrente," getvar(horacorrente) ")
Alias(jirapassword," getvar(jirapassword) ")
)")

threadD.ahkLabel("timereport") ; <--- here i "call" the label

so, im not sure what i am doing wrong, but does not feel (or work) right.
inside my labels, i make http calls (post, get, and stuff). dont know if changes anything.

if it got too complicated i can make a video, so its easier to explain!

thanks again for your time!
HotKeyIt
Posts: 1705
Joined: 29 Sep 2013, 18:35
Contact:

Re: Forking a script execution without adding code to each thread separatedly.

01 Nov 2018, 15:40

a.1 Inside the thread #include will work just fine.
B. Without #Persistent script would exit, see help file. If you have #Persistent in your #included code then you won't need it obviously.
C. each Thread has a tray icon, you can disable it using #NoTrayIcon
D. Correct, as long as thread is running ahkReady will return 1/true
E. you can use either [] or . or (), all do the same currently, [] is the fastest way AFAIK.
F. No, it does not matter, variables exist before script starts execution.
F.1. In example it is not used inside thread, getvar(var) needs to run in main script so needs to be an expression, so " getvar(var) " is correct.
G. You can do either way, CreateScript will create the script (text) for the thread, try MsgBox % CreateScript(...)
H. This happens because you use #include and there is no Return at the top of your code, so it is executed straight away. You can do AhkThread("#Persistent`nReturn`n#Include myscript.ahk").
letacio
Posts: 8
Joined: 08 Mar 2018, 16:05

Re: Forking a script execution without adding code to each thread separatedly.

01 Nov 2018, 16:34

Thanks again.
About F.1 i didnt understand which example you are talking about (yours or documentation).

Anyway, i cant seem to access the variables inside the threads. i pull the alias code out from thread "initializing", and still does nothing!
Could you give me further advance?
HotKeyIt
Posts: 1705
Joined: 29 Sep 2013, 18:35
Contact:

Re: Forking a script execution without adding code to each thread separatedly.

02 Nov 2018, 16:51

Docs are not showing code for thread. My example is for a thread.

Code: Select all

project1:="this is a test"
threadD:= AhkThread("
(
Alias(project1," getvar(project1) ")
MsgBox `% project1
)")
While threadD.ahkReady()
	Sleep 100

Return to “AutoHotkey_H”

Who is online

Users browsing this forum: No registered users and 5 guests