Multithread - using all CPU cores

Get help with using AutoHotkey and its commands and hotkeys
Davidb
Posts: 6
Joined: 18 Sep 2018, 10:56

Multithread - using all CPU cores

12 Apr 2019, 11:30

Hi,

I'm running a AHK script which opens an IE window, click on some buttons and download a file.
The script downloads in this way hundred of files.

I want to run it in multi-thread manner, meaning use all my 8 CPU cores, then the time to download all files will be shorter.
I'm putting here only a partial code in order you can understand and help me to change it.
I downloaded Autohotkey_H, but I'm not sure how to use it.
Can you help me?
Thanks,

Code: Select all


; Create the array, initially empty:
   Array := Object()

; Write to the array:
   Loop, Read, %Broker% ; This loop retrieves each line from the file, one at a time.
   {
      Array.Insert(A_LoopReadLine) ; Append this line to the array.
   }
   excel := Array.MaxIndex()
   


   for index, element in Array ; Recommended approach in most cases.
   {
      Inter_cnt := 0
      
      FileAppend,%element%>>,log.txt
      StartHere1:
      
      ifwinnotexist, ahk_class IEFrame
      {
         sleep, 100
         FileAppend,2>,log.txt
         pwb	:= ComObjCreate( "InternetExplorer.Application" )	; Creates the ComObj
         pwb.Visible 		:= True			; Shows the IE app
         Sleep, 100
      }
      
      WinMaximize, ahk_class IEFrame 
      Sleep, 100
      FileAppend,5>,log.txt
      pwb.Navigate("https .........= "  Broken Link for safety . element)

      cnt := 0
      FileAppend,6>,log.txt
      
      Sleep 6000
      cnt := cnt + 1
      FileAppend,7>,log.txt
      pwb.document.GetElementsByTagName("INPUT")[44].checked :=1 ; dates history
      Sleep 500
      pwb.document.GetElementsByTagName("INPUT")[48].InnerText := "01/01/2012" ; 1 year back
      Sleep 400
      pwb.document.GetElementsByTagName("INPUT")[70].Click() ; Get history data
      pwb.document.GetElementsByTagName("INPUT")[70].fireEvent("onchange") ; Get history data


      FileAppend,8>,log.txt
     
      Sleep 8000
      links := pwb.document.getElementsByTagName("TD")
      FileAppend,9>,log.txt

      linkElement := LoopElements(links, " ******  ", "innertext")
      FileAppend,10>,log.txt

      if (linkElement)
      {
         FileAppend,11>,log.txt
         linkElement.click()
      }
      else
      {
         Inter_cnt := Inter_cnt + 1
         FileAppend,12>,log.txt
         goto StartHere1
      }
   
      Sleep, 5000
      links := pwb.document.getElementsByTagName("A")

      linkElement := LoopElements(links, "CSV", "innertext")
       
      if (linkElement)
      {
         FileAppend,13>,log.txt
         linkElement.click()
      }
      else
      {
         Inter_cnt := Inter_cnt + 1
         pwb.quit()
         Sleep, 100
         FileAppend,14>,log.txt
         goto StartHere1
      }
FanaticGuru
Posts: 1349
Joined: 30 Sep 2013, 22:25

Re: Multithread - using all CPU cores

12 Apr 2019, 13:38

Davidb wrote:
12 Apr 2019, 11:30
I'm running a AHK script which opens an IE window, click on some buttons and download a file.
The script downloads in this way hundred of files.

I want to run it in multi-thread manner, meaning use all my 8 CPU cores, then the time to download all files will be shorter.
I'm putting here only a partial code in order you can understand and help me to change it.
I downloaded Autohotkey_H, but I'm not sure how to use it.
It seems like the download speed of your network is going to be the limiting factor more than your CPU but since AHK does not have true multi-thread, you could have 1 script that starts 8 other scripts and give them each a different download task. They would each open a different instant of IE and download a different "block" of items. Your computer should then handle spreading the multi-processes out over different CPUs as it does for all the individual processes you have running at any given time.

Maybe the limiting factor is the download site that only gives each downloader so much bandwidth and having 8 downloaders might fool the site in to giving each of them their share of bandwidth.

FG
Hotkey Help - Help Dialog for Currently Running AHK Scripts

AHK Startup - Consolidate Multiply AHK Scripts with one Tray Icon

[Function] Timer - Create and Manage Timers
Davidb
Posts: 6
Joined: 18 Sep 2018, 10:56

Re: Multithread - using all CPU cores

13 Apr 2019, 06:34

Hi FG,

Thanks for your response.

I'm trying to implement your suggestion without success.

I'm using this master ahk script which runs 5 individual scripts. Each of them run OK but the master doesn't run they at all.

>"C:\Program Files\AutoHotkey\AutoHotkey.exe" /ErrorStdOut "F:\AHK\master.ahk"
>Exit code: 0 Time: 0.4216

Code: Select all

Run, F:\AHK\thread_1.ahk
Run, F:\AHK\thread_2.ahk
Run, F:\AHK\thread_3.ahk
Run, F:\AHK\thread_4.ahk
Run, F:\AHK\thread_5.ahk
ExitApp
User avatar
gregster
Posts: 2133
Joined: 30 Sep 2013, 06:48

Re: Multithread - using all CPU cores

13 Apr 2019, 10:46

Are you sure that it doesn't start the scripts?
It is clear that this master script will execute quite fast and then immediately exit; and I am not surprised that it would only need 0.42 seconds to complete...

But you can check if run was succesful by checking errorlevel:

Code: Select all

Run, F:\AHK\thread_1.ahk, , UseErrorLevel
msgbox % Errorlevel
Run, F:\AHK\thread_2.ahk, , UseErrorLevel
msgbox % Errorlevel
; [...]
ExitApp
ErrorLevel is set to the word ERROR upon failure or 0 upon success.
Davidb
Posts: 6
Joined: 18 Sep 2018, 10:56

Re: Multithread - using all CPU cores

13 Apr 2019, 12:16

I run the master script with UseErrorLevel and I got Errorlevel 0. The thread_*.ahk scripts don't run at all.
User avatar
gregster
Posts: 2133
Joined: 30 Sep 2013, 06:48

Re: Multithread - using all CPU cores

13 Apr 2019, 12:22

That doesn't make sense. If Errorlevel is 0, they are run.
What code is in them? Perhaps they exit quickly, too (perhaps you need to add #persistent).
Davidb
Posts: 6
Joined: 18 Sep 2018, 10:56

Re: Multithread - using all CPU cores

13 Apr 2019, 12:33

Hi Greg,
You can see the thread_*.ahk content (partially) in my first post (yesterday 19:30).
The script opens an IE window click on some buttons and download a file.
I can run each of the thread.ahk separately, but they do not run inside the master script.
User avatar
gregster
Posts: 2133
Joined: 30 Sep 2013, 06:48

Re: Multithread - using all CPU cores

13 Apr 2019, 23:55

I still can only assume that the scripts get really started, like the Errorlevel indicates (perhaps try adding a msgbox here and there to check that they are really running and to check the contents of their variables).

But probably the log.txt files just end up in a "wrong" directory that you don't expect, because you don't specify an absolute file path for them.

If you run one of the "thread"-scripts directly, the working directory of this specific script will be used - but if you use another script to run it, like in this case, the working directory of the starter/master/main script might be used instead. So, try to set a working directory that you like, either via the second parameter of the run command:

Code: Select all

Run, Target [ , WorkingDir, Options, OutputVarPID ]
Or, by doing this:
https://autohotkey.com/docs/commands/SetWorkingDir.htm#Remarks wrote:The script's working directory is the default directory that is used to access files and folders when an absolute path has not been specified. In the following example, the file My Filename.txt is assumed to be in %A_WorkingDir%: FileAppend, A Line of Text, My Filename.txt.

A script's initial working directory is determined by how it was launched. For example, if it was run via shortcut -- such as on the Start Menu -- its working directory is determined by the "Start in" field within the shortcut's properties.

To make a script unconditionally use its own folder as its working directory, make its first line the following:

SetWorkingDir %A_ScriptDir%


Once changed, the new working directory is instantly and globally in effect throughout the script.[...]
Or, use absolute paths to save the log files...
Davidb
Posts: 6
Joined: 18 Sep 2018, 10:56

Re: Multithread - using all CPU cores

14 Apr 2019, 06:25

Hi Greg,
Thanks for your help.
The problem was that I need to add %A_AhkPath% to the Run script:

Code: Select all

Run %A_AhkPath% F:\AHK\thread_1.ahk
I found it here:
https://autohotkey.com/board/topic/54296-script-for-people-who-like-to-run-portable/

So now, the master script is running but I got too many problems.
For now I'll run each script in a distinguished PC,
it seems that this specific script will be very hard to run in parallel with my limited knowledge in AutoHotkey.
SOTE
Posts: 546
Joined: 15 Jun 2015, 06:21

Re: Multithread - using all CPU cores

14 Apr 2019, 07:55

You might also want to check out AutoHotkey_H
https://www.autohotkey.com/boards/viewtopic.php?f=65&t=46105&p=208275&hilit
(Help me generate script run with real multithread)
User avatar
Tigerlily
Posts: 277
Joined: 04 Oct 2018, 22:31

Re: Multithread - using all CPU cores

14 Apr 2019, 20:03

Here is another AHK_H thread where I got help with multi-threading (multiple examples), it may be a little tricky at first:

https://www.autohotkey.com/boards/viewtopic.php?f=65&t=63227
-TL
FanaticGuru
Posts: 1349
Joined: 30 Sep 2013, 22:25

Re: Multithread - using all CPU cores

15 Apr 2019, 14:12

Davidb wrote:
14 Apr 2019, 06:25
Hi Greg,
Thanks for your help.
The problem was that I need to add %A_AhkPath% to the Run script:

Code: Select all

Run %A_AhkPath% F:\AHK\thread_1.ahk
I found it here:
https://autohotkey.com/board/topic/54296-script-for-people-who-like-to-run-portable/

So now, the master script is running but I got too many problems.
For now I'll run each script in a distinguished PC,
it seems that this specific script will be very hard to run in parallel with my limited knowledge in AutoHotkey.

This script will let you easily let you have several scripts start and stop together kind of like one script.

I imagine in your case, there could still be timing problems with having 5 scripts all trying to start 5 copies of Explorer and download things at the same time. It seems doable though. Just got to make sure each script keeps track of its version of Explorer and communicates with its version correctly.

FG
Hotkey Help - Help Dialog for Currently Running AHK Scripts

AHK Startup - Consolidate Multiply AHK Scripts with one Tray Icon

[Function] Timer - Create and Manage Timers
Davidb
Posts: 6
Joined: 18 Sep 2018, 10:56

Re: Multithread - using all CPU cores

17 Apr 2019, 11:25

Hi FG,
I imagine in your case, there could still be timing problems with having 5 scripts all trying to start 5 copies of Explorer and download things at the same time. It seems doable though. Just got to make sure each script keeps track of its version of Explorer and communicates with its version correctly.
You correct, I must "make sure each script keeps track of its version of Explorer and communicates with its version correctly", but I have no idea how to reach that.
I think in my specific case it would be must to run each script (= IE window) in a different Desktop window in order they do not interfere each other.

Any suggestions?
FanaticGuru
Posts: 1349
Joined: 30 Sep 2013, 22:25

Re: Multithread - using all CPU cores

17 Apr 2019, 15:33

Davidb wrote:
17 Apr 2019, 11:25
Hi FG,
I imagine in your case, there could still be timing problems with having 5 scripts all trying to start 5 copies of Explorer and download things at the same time. It seems doable though. Just got to make sure each script keeps track of its version of Explorer and communicates with its version correctly.
You correct, I must "make sure each script keeps track of its version of Explorer and communicates with its version correctly", but I have no idea how to reach that.
I think in my specific case it would be must to run each script (= IE window) in a different Desktop window in order they do not interfere each other.

Any suggestions?

You just need to have each script do it's own:

Code: Select all

pwb	:= ComObjCreate( "InternetExplorer.Application" )	; Creates the ComObj
pwb.Visible	:= True			; Shows the IE app
and forget the ifwinnotexist, ahk_class IEFrame to check if an IE is already going.

You start 5 scripts, each with this, and you will have 5 windows of IE open up.

Each script them needs to only use direct COM controls. Stuff like pwb.document.GetElementsByTagName("INPUT")[70].Click() which you are already doing. It is fine to use the same pwb name in each script.

Each script should have its own uniquely named log file also.

It should all be very doable.

FG
Hotkey Help - Help Dialog for Currently Running AHK Scripts

AHK Startup - Consolidate Multiply AHK Scripts with one Tray Icon

[Function] Timer - Create and Manage Timers

Return to “Ask For Help”

Who is online

Users browsing this forum: Ataricx, Bing [Bot], Google [Bot], Odlanir, swagfag, vsub and 30 guests