RunCMD() v0.94 : Capture stdout to variable. Non-blocking version. Pre-process/omit individual lines.

Post your working scripts, libraries and tools
gwarble
Posts: 461
Joined: 30 Sep 2013, 15:01

Re: RunCMD() v0.94 : Capture stdout to variable. Non-blocking version. Pre-process/omit individual lines.

16 Jul 2020, 10:52

SyncToy and many other folder syncing utilities appear to use it in the background if I'm not mistaken
EitherMouse - Multiple mice, individual settings . . . . www.EitherMouse.com . . . . forum . . . .
Sam_
Posts: 125
Joined: 20 Mar 2014, 20:24

Re: RunCMD() v0.94 : Capture stdout to variable. Non-blocking version. Pre-process/omit individual lines.

30 Jul 2020, 16:48

Magnificent! Thank you for the streaming version SKAN.
User avatar
TheArkive
Posts: 385
Joined: 05 Aug 2016, 08:06
GitHub: TheArkive

Re: RunCMD() v0.94 : Capture stdout to variable. Non-blocking version. Pre-process/omit individual lines.

03 Aug 2020, 13:50

@SKAN

I can't thank you enough for this function!

Here is a short re-write compatible with AHK v2-a119. Haven't tested streaming yet. Just basic input / output is working on latest v2 alpha.

Code: Select all

RunCMD(CmdLine, WorkingDir:="", Codepage:="CP0", Fn:="RunCMD_Output") {  ;         RunCMD v0.94        
    Local         ; RunCMD v0.94 by SKAN on D34E/D37C @ autohotkey.com/boards/viewtopic.php?t=74647                                                             
    Global A_Args ; Based on StdOutToVar.ahk by Sean @ autohotkey.com/board/topic/15455-stdouttovar

      Fn := IsFunc(Fn) ? Func(Fn) : 0
    , DllCall("CreatePipe", "PtrP",hPipeR:=0, "PtrP",hPipeW:=0, "Ptr",0, "Int",0)
    , DllCall("SetHandleInformation", "Ptr",hPipeW, "Int",1, "Int",1)
    , DllCall("SetNamedPipeHandleState","Ptr",hPipeR, "UIntP",PIPE_NOWAIT:=1, "Ptr",0, "Ptr",0)

    , P8 := (A_PtrSize=8)
    , SI := BufferAlloc(P8 ? 104 : 68, 0)                          ; STARTUPINFO structure      
    , NumPut("UInt",P8 ? 104 : 68, SI)                                     ; size of STARTUPINFO
    , NumPut("UInt",STARTF_USESTDHANDLES:=0x100, SI, P8 ? 60 : 44)  ; dwFlags
    , NumPut("Ptr",hPipeW, SI, P8 ? 88 : 60)                              ; hStdOutput
    , NumPut("Ptr",hPipeW, SI, P8 ? 96 : 64)                              ; hStdError
    , PI := BufferAlloc(P8 ? 24 : 16)                              ; PROCESS_INFORMATION structure

      If not DllCall("CreateProcess", "Ptr",0, "Str",CmdLine, "Ptr",0, "Int",0, "Int",True
                    ,"Int",0x08000000 | DllCall("GetPriorityClass", "Ptr",-1, "UInt"), "Int",0
                    ,"Ptr",WorkingDir ? StrPtr(WorkingDir) : 0, "Ptr",SI.ptr, "Ptr",PI.ptr)  
         Return Format("{1:}", "", ErrorLevel := -1
                       ,DllCall("CloseHandle", "Ptr",hPipeW), DllCall("CloseHandle", "Ptr",hPipeR))

      DllCall("CloseHandle", "Ptr",hPipeW)
    , A_Args.RunCMD := { PID: NumGet(PI, P8? 16 : 8, "UInt") }      
    , File := FileOpen(hPipeR, "h", Codepage)

    , LineNum := 1,  sOutput := ""
      While (A_Args.RunCMD.PID + DllCall("Sleep", "Int",0))
        and DllCall("PeekNamedPipe", "Ptr",hPipeR, "Ptr",0, "Int",0, "Ptr",0, "Ptr",0, "Ptr",0)
            While A_Args.RunCMD.PID and (Line := File.ReadLine())
              sOutput .= Fn ? Fn.Call(Line, LineNum++) : Line

      A_Args.RunCMD.PID := 0
    , hProcess := NumGet(PI, 0, "Ptr")
    , hThread  := NumGet(PI, A_PtrSize, "Ptr")

    , DllCall("GetExitCodeProcess", "Ptr",hProcess, "PtrP",ExitCode:=0)
    , DllCall("CloseHandle", "Ptr",hProcess)
    , DllCall("CloseHandle", "Ptr",hThread)
    , DllCall("CloseHandle", "Ptr",hPipeR)

    , ErrorLevel := ExitCode

    Return sOutput  
}
User avatar
SKAN
Posts: 853
Joined: 29 Sep 2013, 16:58

Re: RunCMD() v0.94 : Capture stdout to variable. Non-blocking version. Pre-process/omit individual lines.

04 Aug 2020, 10:33

TheArkive wrote:
03 Aug 2020, 13:50
I can't thank you enough for this function!
 
I'm happy too. It works fine at everything I so far have tested!
I guess I have to thank @Sam_ for motivating me to write this streaming version. :)
 
Here is a short re-write compatible with AHK v2-a119. Haven't tested streaming yet. Just basic input / output is working on latest v2 alpha.
 
Thank you. I will try and let know :) :thumbup:
User avatar
hasantr
Posts: 677
Joined: 05 Apr 2016, 14:18
Location: İstanbul

Re: RunCMD()

16 Aug 2020, 09:43

SKAN wrote:
02 Jul 2020, 17:45
Sam_ wrote:This function is awesome! Thank you for sharing it.
Thank you. :)
Sam_ wrote:Will you pretty please also provide a streaming version?
Quickly written.. not well tested. I will try to post a proper demo later. (streaming to an edit control)

Code: Select all

RunCmd_Stream(CmdLine, WorkingDir:="", Cp:="CP0") {           ; v0.90 by SKAN on D34E/D373  
  Local                                                       ;           @ tiny.cc/runcmd
  Global A_Args

  DllCall("CreatePipe", "PtrP",hPipeR:=0, "PtrP",hPipeW:=0, "Ptr",0, "UInt",0)
, DllCall("SetHandleInformation", "Ptr",hPipeW, "UInt",1, "UInt",1)

, P8 := (A_PtrSize=8),            VarSetCapacity(SI, P8? 104:68,0)      
, NumPut(P8? 104:68, SI),         NumPut(0x100, SI,  P8? 60:44,"UInt")  
, NumPut(hPipeW, SI, P8? 88:60),  NumPut(hPipeW, SI, P8? 96:64)   

, VarSetCapacity(PI, P8? 24:16),  pWorkingDir := (WorkingDir ? &WorkingDir : 0)               

  If not DllCall("CreateProcess", "Ptr",0, "Str",CmdLine, "Ptr",0, "UInt",0, "UInt",True
              , "UInt",0x08000000 | DllCall("GetPriorityClass", "Ptr",-1,"UInt"), "UInt",0
              , "Ptr",pWorkingDir, "Ptr",&SI, "Ptr",&PI )  
     Return Format( "{1:}", "", ErrorLevel := -1 
          , DllCall("CloseHandle", "Ptr",hPipeW)
          , DllCall("CloseHandle", "Ptr",hPipeR) )
 
  DllCall("CloseHandle", "Ptr",hPipeW), A_Args.RunCMD := {"PID":NumGet(PI,P8?16:8,"UInt")}
  File := FileOpen(hPipeR, "h", Cp), Line := ""

  While (Line := File.ReadLine())
    If not RunCmd_Output(Line, A_Index)
      Break  

  A_Args.RunCMD.PID := 0,  hProcess := NumGet(PI,0),  hThread := NumGet(PI,4)
, DllCall("GetExitCodeProcess", "Ptr",hProcess, "PtrP",ExitCode:=0)
, DllCall("CloseHandle", "Ptr",hProcess),    DllCall("CloseHandle", "Ptr",hThread)
, DllCall("CloseHandle", "Ptr",hPipeR)
Return ExitCode  
}




; Quick test follows:

#NoEnv
#Warn
#SingleInstance, Force
SetWorkingDir %A_ScriptDir%
SetBatchLines -1
Process, Priority,,High

RunCmd_Stream(A_Comspec . " /c Dir *.*",  A_AhkPath . "\..\")
SoundBeep
Return                          ; end of auto-execute section

RunCmd_Output(Line, LineNum) {
  Sleep 1
  MsgBox % Line . "#" LineNum
;Return False ; Cancel
Return True  ; Continue 
}
My English is not good and I find it difficult to follow the event. Do we use this version as the streaming version. Is there a different version currently?My English is not good and I find it difficult to follow the event. Do we use this version as the streaming version. Is there a different version currently?
User avatar
SKAN
Posts: 853
Joined: 29 Sep 2013, 16:58

Re: RunCMD()

16 Aug 2020, 10:26

hasantr wrote:
16 Aug 2020, 09:43
SKAN wrote:
02 Jul 2020, 17:45
Sam_ wrote:This function is awesome! Thank you for sharing it.
Thank you. :)
Sam_ wrote:Will you pretty please also provide a streaming version?
Quickly written.. not well tested. I will try to post a proper demo later. (streaming to an edit control)
Obsolete version
 
My English is not good and I find it difficult to follow the event. Do we use this version as the streaming version. Is there a different version currently?My English is not good and I find it difficult to follow the event.
Do we use this version as the streaming version. Is there a different version currently?

No!.
Sorry for the confusion. I've marked this temp code (a blocking version) as Obsolete.
Please use the new RunCMD() posted at the beginning of this topic and also try the examples.
https://www.autohotkey.com/boards/viewtopic.php?t=74647

The new version is non-blocking and all the examples posted throughout this topic should work with it without any errors.
User avatar
hasantr
Posts: 677
Joined: 05 Apr 2016, 14:18
Location: İstanbul

Re: RunCMD()

17 Aug 2020, 01:33

SKAN wrote:
16 Aug 2020, 10:26
hasantr wrote:
16 Aug 2020, 09:43
SKAN wrote:
02 Jul 2020, 17:45
Sam_ wrote:This function is awesome! Thank you for sharing it.
Thank you. :)
Sam_ wrote:Will you pretty please also provide a streaming version?
Quickly written.. not well tested. I will try to post a proper demo later. (streaming to an edit control)
Obsolete version
 
My English is not good and I find it difficult to follow the event. Do we use this version as the streaming version. Is there a different version currently?My English is not good and I find it difficult to follow the event.
Do we use this version as the streaming version. Is there a different version currently?

No!.
Sorry for the confusion. I've marked this temp code (a blocking version) as Obsolete.
Please use the new RunCMD() posted at the beginning of this topic and also try the examples.
https://www.autohotkey.com/boards/viewtopic.php?t=74647

The new version is non-blocking and all the examples posted throughout this topic should work with it without any errors.
Thank you for the explanation. You are so kind.

(a blocking version) What exactly does it mean. It shows if it's streaming.

I apologize for learning so hard. :)
User avatar
SKAN
Posts: 853
Joined: 29 Sep 2013, 16:58

Re: RunCMD()

17 Aug 2020, 06:40

hasantr wrote:
17 Aug 2020, 01:33
(a blocking version) What exactly does it mean. It shows if it's streaming.
URLDownloadToFile is a blocking function (command).
When you call this function from a GUI application, and say your net connection is too slow or the server is too busy,
then the GUI will become unresponsive (cannot be moved/will not redraw etc.) until the requested data is returned (or refused).
This is not a a big problem nowadays as internet has become very fast.

Here is an example showing how a GUI becomes nonresponsive when a synchronous function is in wait state.
 

Code: Select all

#NoEnv
#SingleInstance, Force

Gui, Add, Text,, This window will NOT move/minimize for 5 seconds
Gui, Show, w480 h240
DllCall("Sleep", "Int",5000) ; <== This is a blocking function
; Sleep, 5000                ; <== This is a non-blocking function (command)
GuiControl,,Static1, It's okay now!
 
 
If you wondering why I mentioned about URLDownloadToFile:
URLDownloadToFile uses InternetReadFile
and StdOutToVar variants use ReadFile
Both functions wait for data and during the wait period they block a GUI from receiving messages.

Whats new in RunCMD() is that I've flagged the pipe to not wait.
That is, when ReadFile is called, the pipe would return immediately whatever data it has, which most of the time would be empty.
For example: Say, something.exe outputs 6 lines
A blocking version RunCMD() would loop only 6 times (or less) but will block a GUI during the period it waits for data to arrive.
A non-blocking version of RunCMD() will loop thousands of times to check for data but will never block GUI messages.
Within Loop, I have used smallest possible Sleep: DllCall("Sleep", "Int",0) which does spike CPU to 25% on my Quad core machine.
If one needs to run it on a single core machine, change it to DllCall("Sleep", "Int",1)

:)
User avatar
hasantr
Posts: 677
Joined: 05 Apr 2016, 14:18
Location: İstanbul

Re: RunCMD()

17 Aug 2020, 07:40

SKAN wrote:
17 Aug 2020, 06:40
hasantr wrote:
17 Aug 2020, 01:33
(a blocking version) What exactly does it mean. It shows if it's streaming.
URLDownloadToFile is a blocking function (command).
When you call this function from a GUI application, and say your net connection is too slow or the server is too busy,
then the GUI will become unresponsive (cannot be moved/will not redraw etc.) until the requested data is returned (or refused).
This is not a a big problem nowadays as internet has become very fast.

Here is an example showing how a GUI becomes nonresponsive when a synchronous function is in wait state.
 

Code: Select all

#NoEnv
#SingleInstance, Force

Gui, Add, Text,, This window will NOT move/minimize for 5 seconds
Gui, Show, w480 h240
DllCall("Sleep", "Int",5000) ; <== This is a blocking function
; Sleep, 5000                ; <== This is a non-blocking function (command)
GuiControl,,Static1, It's okay now!
 
 
If you wondering why I mentioned about URLDownloadToFile:
URLDownloadToFile uses InternetReadFile
and StdOutToVar variants use ReadFile
Both functions wait for data and during the wait period they block a GUI from receiving messages.

Whats new in RunCMD() is that I've flagged the pipe to not wait.
That is, when ReadFile is called, the pipe would return immediately whatever data it has, which most of the time would be empty.
For example: Say, something.exe outputs 6 lines
A blocking version RunCMD() would loop only 6 times (or less) but will block a GUI during the period it waits for data to arrive.
A non-blocking version of RunCMD() will loop thousands of times to check for data but will never block GUI messages.
Within Loop, I have used smallest possible Sleep: DllCall("Sleep", "Int",0) which does spike CPU to 25% on my Quad core machine.
If one needs to run it on a single core machine, change it to DllCall("Sleep", "Int",1)

:)
Thank you very much for the lighting. I am grateful to you. The Autohotkey team is really great. :bravo:
User avatar
hasantr
Posts: 677
Joined: 05 Apr 2016, 14:18
Location: İstanbul

Re: RunCMD() v0.94 : Capture stdout to variable. Non-blocking version. Pre-process/omit individual lines.

22 Sep 2020, 07:44

Even though I have set it as UTF-8, the characters still sound corrupt.
What's wrong?

Code: Select all

RunCmd( A_Comspec . " /c dir /S /B /A:-D *.doc | findstr /V /I /C:""""\\Microsoft\\"""" /C:""""$Recycle.Bin"""" /C:""""\\Windows\\""""" , "D:\","UTF-8")
User avatar
SKAN
Posts: 853
Joined: 29 Sep 2013, 16:58

Re: RunCMD() v0.94 : Capture stdout to variable. Non-blocking version. Pre-process/omit individual lines.

23 Sep 2020, 03:02

hasantr wrote:
22 Sep 2020, 07:44
Even though I have set it as UTF-8, the characters still sound corrupt.
What's wrong?

Code: Select all

RunCmd( A_Comspec . " /c dir /S /B /A:-D *.doc | findstr /V /I /C:""""\\Microsoft\\"""" /C:""""$Recycle.Bin"""" /C:""""\\Windows\\""""" , "D:\","UTF-8")
Hi @hasantr
I don't understand that line, but cmd.exe supports unicode with /U parameter.
Ref: https://www.autohotkey.com/boards/viewtopic.php?p=323980#p323980
User avatar
hasantr
Posts: 677
Joined: 05 Apr 2016, 14:18
Location: İstanbul

Re: RunCMD() v0.94 : Capture stdout to variable. Non-blocking version. Pre-process/omit individual lines.

23 Sep 2020, 06:03

SKAN wrote:
23 Sep 2020, 03:02
hasantr wrote:
22 Sep 2020, 07:44
Even though I have set it as UTF-8, the characters still sound corrupt.
What's wrong?

Code: Select all

RunCmd( A_Comspec . " /c dir /S /B /A:-D *.doc | findstr /V /I /C:""""\\Microsoft\\"""" /C:""""$Recycle.Bin"""" /C:""""\\Windows\\""""" , "D:\","UTF-8")
Hi @hasantr
I don't understand that line, but cmd.exe supports unicode with /U parameter.
Ref: https://www.autohotkey.com/boards/viewtopic.php?p=323980#p323980
Thanks SKAN for the answer.
This cmd line lists the exe files on the disk.
CMD doesn't have a character problem anyway. When I call it with Runcmd, there is a character problem. When I open cmd and execute the code with cmd everything seems to be fine.

I think I ran into an unanswered problem. :)
User avatar
SKAN
Posts: 853
Joined: 29 Sep 2013, 16:58

Re: RunCMD() v0.94 : Capture stdout to variable. Non-blocking version. Pre-process/omit individual lines.

23 Sep 2020, 06:09

hasantr wrote:CMD doesn't have a character problem anyway
From command prompt execute CHCP command.
What is the active code page?
User avatar
hasantr
Posts: 677
Joined: 05 Apr 2016, 14:18
Location: İstanbul

Re: RunCMD() v0.94 : Capture stdout to variable. Non-blocking version. Pre-process/omit individual lines.

25 Sep 2020, 07:17

SKAN wrote:
23 Sep 2020, 06:09
hasantr wrote:CMD doesn't have a character problem anyway
From command prompt execute CHCP command.
What is the active code page?
UTF-8 Instead @just me found a solution.
https://www.autohotkey.com/boards/viewtopic.php?f=76&t=81309&start=20#p354492
Using it is definitely a solution.
CP1
User avatar
rommmcek
Posts: 1123
Joined: 15 Aug 2014, 15:18

Re: RunCMD()

21 Oct 2020, 15:48

rommmcek wrote:
13 May 2020, 19:13
… Purely theoretically, this would perform more efficient:

Code: Select all

While A_Args.RunCMD.PID
     * DllCall("ReadFile",  "Ptr",hPipeR, "Ptr",&Buff, "UInt",4094, "PtrP",nSz, "UInt",0)
… I'll probably do the test in days to come.
SKAN wrote:
14 May 2020, 00:20
... Please do. ...
Not in days... it took months to get at least to some extent consistent results.
Test should be run with zero or with as little as possible running apps and certainly not inside RunCmd().

Code: Select all

SetBatchLines, -1
DllCall("QueryPerformanceFrequency", "Int64P", Freq), fact:=1000/Freq

a:= b:= 1, c:= dif:= 0
loop, 10100000 {
   if (Mod(A_Index,2)>0) {
        DllCall("QueryPerformanceCounter", "Int64P", cBf)
      , a&&b? c++: ""
      , DllCall("QueryPerformanceCounter", "Int64P", cAf)
      , A_Index>100000? dif+= cAf-cBf: ""
   } else {
        DllCall("QueryPerformanceCounter", "Int64P", cBf)
      , a*b? c++: ""
      , DllCall("QueryPerformanceCounter", "Int64P", cAf)
      , A_Index>100000? dif-= cAf-cBf: ""
   }
}
MsgBox % c "`n" dif*fact

c:= dif:= 0
loop, 10100000 {
    Mod(A_Index,2)>0
      ? (DllCall("QueryPerformanceCounter", "Int64P", cBf)
         , a&&b? c++: ""
         , DllCall("QueryPerformanceCounter", "Int64P", cAf)
         , A_Index>100000? dif+= cAf-cBf: "")
      : (DllCall("QueryPerformanceCounter", "Int64P", cBf)
         , a*b? c++: ""
         , DllCall("QueryPerformanceCounter", "Int64P", cAf)
         , A_Index>100000? dif-= cAf-cBf: "")
}
MsgBox % c "`n" dif*fact
Test takes 2x 10 sec. (on my PC). Make your own conclusion!
User avatar
SKAN
Posts: 853
Joined: 29 Sep 2013, 16:58

Re: RunCMD()

22 Oct 2020, 19:06

rommmcek wrote:
21 Oct 2020, 15:48
Test takes 2x 10 sec. (on my PC). Make your own conclusion!
Wonderful. Thanks @rommmcek :) :thumbup:
william_ahk
Posts: 26
Joined: 03 Dec 2018, 20:02

Re: RunCMD() v0.94 : Capture stdout to variable. Non-blocking version. Pre-process/omit individual lines.

20 Nov 2020, 03:53

Hi SKAN, do you know why the output string of RunCMD does not equal to the same string content?

Code: Select all

str1 := RunCMD(A_ComSpec . " /c echo SKAN")
msgbox % str1
msgbox % (str1 = "SKAN") ;0
burque505
Posts: 1397
Joined: 22 Jan 2017, 19:37

Re: RunCMD() v0.94 : Capture stdout to variable. Non-blocking version. Pre-process/omit individual lines.

20 Nov 2020, 09:03

@william_ahk, try this code. There's apparently a carriage return in the stdout from 'echo'.

Code: Select all

#Include RunCmd.ahk

str1 := RunCMD(A_ComSpec . " /c echo SKAN")
str1 := RegExReplace(str1, "`r`n")
msgbox % str1
If (str1 == "SKAN") 
	msgbox Equal
else
	msgbox %str1% Is Not Equal to SKAN
Regards, burque505

Return to “Scripts and Functions”

Who is online

Users browsing this forum: No registered users and 15 guests