CliSAK class - 2022/11/12 - beta.15

Post your working scripts, libraries and tools.
User avatar
TheArkive
Posts: 1027
Joined: 05 Aug 2016, 08:06
Location: The Construct
Contact:

CliSAK class - 2022/11/12 - beta.15

31 Jul 2020, 06:17

Cli - Swiss Army Knife v4 (CliSAK)


AHK v1 version/discussion moved to this thread.

Usage:
  • For one-time data collection: var := CliData(inCommand)
  • For streaming collection: obj := cli(inCommand, options := "", environment := "cmd", params := "/Q")
The "wget example":
Image

Please see example script for detailed examples, and the main class script for detailed docs.

Highlights:
  • Wrapper function CliData(inCommand) for easy inline one-time data collection from a program.
  • Easy collection of streaming output with callback functions, can handle multiple CLI sessions simultaneously.
  • Easy live background CLI interaction with PromptCallback() event.
  • Creates a hidden console window so that CTRL+C and CTRL+Break can still be passed to halt execution with hotkey, button, etc.
  • Capture StdErr separately, if desired. Best if checked in PromptCallback()
  • Pass dynamically created multi-line batch commands executed in succession.
  • Define a "Quit String" to automatically halt execution, clean up processes and handles, and trigger a callback.
  • Capture console animations like incrementing percent, or progress bar.
The following CLI environment prompts (shells) can be detected:
  • Windows batch
  • netsh
  • Android ADB --- finally fixed the prompt display issue!
  • SSH
==================================================================================
Important Update
Please Read
==================================================================================

If you would like support for a new shell prompt, post a msg with details so I can research and add the new environment.

NOTE: Some command line programs don't like it when you try to redirect stdout away from the console it was started with. All you can do is try! Sadly... TELNET is one of these, but it won't even tell you, it just doesn't work.

=======================================================================
UPDATES:
=======================================================================

:arrow: Download on GitHub

Huge thanks to the following users:
@segalion , @Sweeet, @sean , @maraksan_user, @SKAN , @HotKeyIt , @maz_1, @lexikos , @TheGood, @just me, @joedf
Details of their contributions are in the comments of the CLI class library. They have done a LOT. Without their contributions, I would not have gained the knowledge or the means to make this.

I'm happy to hear any commets/critiques to make this library more useful, fast, and memory-efficient.
Last edited by TheArkive on 12 Nov 2022, 04:38, edited 30 times in total.
User avatar
joedf
Posts: 8940
Joined: 29 Sep 2013, 17:08
Location: Canada
Contact:

Re: CliSAK class - 2020/07/31 - a119

06 Aug 2020, 16:52

Neat-o! :+1:
Image Image Image Image Image
Windows 10 x64 Professional, Intel i5-8500, NVIDIA GTX 1060 6GB, 2x16GB Kingston FURY Beast - DDR4 3200 MHz | [About Me] | [About the AHK Foundation] | [Courses on AutoHotkey]
[ASPDM - StdLib Distribution] | [Qonsole - Quake-like console emulator] | [LibCon - Autohotkey Console Library]
User avatar
TheArkive
Posts: 1027
Joined: 05 Aug 2016, 08:06
Location: The Construct
Contact:

Re: CliSAK class - 2020/07/31 - a119

07 Aug 2020, 13:30

@joedf
thanks for the encouragement man :) ... if you do any in-depth testing and find any issues please let me know
burque505
Posts: 1731
Joined: 22 Jan 2017, 19:37

Re: CliSAK class - 2020/07/31 - a119

07 Aug 2020, 13:46

:D Better and better!
The examples are really helpful.
Regards,
burque505
User avatar
TheArkive
Posts: 1027
Joined: 05 Aug 2016, 08:06
Location: The Construct
Contact:

Re: CliSAK class - 2020/07/31 - a119

07 Aug 2020, 13:55

Thanks @burque505
Fortunately not much as changed with all the alpha updates (that i know of) ... i haven't managed to test much more than the examples lately. I have some bigger projects that will put this library to the test.
User avatar
TheArkive
Posts: 1027
Joined: 05 Aug 2016, 08:06
Location: The Construct
Contact:

Re: CliSAK class - 2020/09/28 - a122

28 Sep 2020, 07:43

Update:2020/09/28
  • Improved consistency of operation across modes, optimized code
  • Added QuitStringCallback() for user defined quit string.
  • Fixed handling of ADB environment. Prompt consistently shown after command execution.
  • Finished documentation comments (top of the script).
  • Added ConsoleSend() method for proper usage of StdIn using mode "m" (thanks @lexikos!)
  • Added GetLastLine(str) method for usage in callbacks to filter output.
User avatar
TheArkive
Posts: 1027
Joined: 05 Aug 2016, 08:06
Location: The Construct
Contact:

Re: CliSAK class - 2020/11/29 - a122

29 Nov 2020, 07:45

Update:2020/11/29
  • Fixed potential security vulernability using mode "m" (WriteConsoleInput) - now StdIn is secure, and it can only communicate with the script, much more difficult to hijack.
  • Removed .ConsoleSend() method, because this contributed to a less secure background CLI session.
  • Added parameters to separate the environment from the commands, when calling cli.New().
  • Pure PowerShell now works with CliSAK.
    - Known issue: multi-line commands not currently possible.
    - Once you enter multi-line command mode you can't get out of it unless you terminate the session.
    - Multi-line commands might be possible in mode "m" but still with some limitations.
  • Default environment is CMD window with /Q for ECHO OFF.
  • Removed most of the modes - now this cli class is "streaming only".
  • Updated CliData() wrapper function for one-time easy "run and collect". This is the equivalent of the old "w" (wait) mode.
  • Now using a callback only requires that the callback function exists.
  • Improved timing and execution of PromptCallback() - StdErr and StdOut easily line up within this callback function.
  • Added CliObj.ready property to indicate when the environment is finished loading and the first prompt is displayed.
    - Best used in PromptCallback().
  • Added CliObj.BatchProgress property so the user can check which command in a multi-line set of commands is finished executing.
    - Best used in PromptCallback().
  • Mode "m" now allows StdErr to be a separate stream (using mode "x").
    - This can cause unwanted side effects with some programs, ie. the program may appear to hang.
  • Updated and simplified examples.
  • Added example GIF of wget Example #8.
CappuccinoCake
Posts: 3
Joined: 01 Dec 2020, 02:01

Re: CliSAK class - 2020/11/29 - a122

01 Dec 2020, 02:13

I have a question, i have to made a telnet session to a device, how i can get it to work with your class? If i insert "telnet" into the environment param i get the command line input field as a multi line input so i cannot send the commands to the console. And it also does not load the telnet cleint. I installed the telnet via windows features. It would be nice if somebody could help me out.

Kind reguards
User avatar
TheArkive
Posts: 1027
Joined: 05 Aug 2016, 08:06
Location: The Construct
Contact:

Re: CliSAK class - 2020/11/29 - a122

01 Dec 2020, 05:49

@CappuccinoCake
please post your script so I can see what you are working with. How you use the PromptCallback() and StdOutCallback() functions is critical to making this work.

I'll also do some testing on my end.
User avatar
TheArkive
Posts: 1027
Joined: 05 Aug 2016, 08:06
Location: The Construct
Contact:

Re: CliSAK class - 2020/11/29 - a122

01 Dec 2020, 06:20

@CappuccinoCake
I have seen something like this before with ssh.exe on Windows 10, but you actually get an error message. I'm afraid I'll have to do more testing.

I tried using mode "m", and i saw that telnet does load, but immediately exits. There are strange side effects with some command line programs when redirecting stdout/stdin away from the console.

What kind of automation are you trying to accomplish? Maybe there is another way?

I have done more testing with putty/plink. That works quite well with CliSAK.

Please let me know if you find a success method. I may need to modify some settings when using CreateProcess DllCall().

I might actually add a mode for redirecting to file, so that the same interface (this script) can be used. I've tried file redirect with ssh.exe on Win10 and it does work, so that may be the solution for telnet (I've been trying to avoid using a file as much as possible).
User avatar
TheArkive
Posts: 1027
Joined: 05 Aug 2016, 08:06
Location: The Construct
Contact:

Re: CliSAK class - 2020/11/29 - a122

01 Dec 2020, 11:13

This post on another forum is related to exactly what we are experiencing: http://www.delphigroups.info/3/1/116676.html

Either I need to use a specific security descriptor or this isn't meant to work without a console. In which case, redirecting to a file would probably be the only option, but then your harddrive is constantly churning as the script keeps checking the file... definitely not ideal.

Or you can check out my socket script class, and write a telnet script. Main problem though, it's just sockets, no encryption.

EDIT:

Or you can use a modified version of mode "m", which i removed because it is a bit unsecure and could be hijacked by another application. The AHK v1 version currently uses this other method to execute mode "m". Basically it a typical hidden console that you can scrape, and send keystrokes to. But still not ideal because for decent performance you need to shrink the console size, which also reduces your ability to scroll back.

I think for the time being, I'm just going to take off telnet as a capability since I have found a few posts already of people being unable to use telnet with CreateProcess. I had added it as a capability only because i added the "prompt recognition" so it could be used with CliSAK. Never thought it would actually fail though.

Sorry to say, I think telnet and CliSAK won't be getting along any time soon.

EDIT2:

@CappuccinoCake
I might dabble with making my own telnet script using sockets. If i get one working I'll post it in the AHK v2 Scripts forum.
CappuccinoCake
Posts: 3
Joined: 01 Dec 2020, 02:01

Re: CliSAK class - 2020/11/29 - a122

04 Dec 2020, 07:29

Sorry, i was not respomnding here because I had to do other stuff. I changed the line where you are sending multiple command to the normal telnet command only and the console isn't showing anithing...

Code: Select all

If (IsObject(c))
		c.close(), c:=""
	oGui["CmdOutput"].Value := ""
	
	batch := "ECHO. & telnet`r`n"
	
	; Mode "r" removes prompt and command from data, so only output is visible.
	c := cli.New(batch,"mode:|ID:Console_Streaming") ; Only using StdOutCallback()
I am not sure if this is the correct way.

Kind reguards.
User avatar
TheArkive
Posts: 1027
Joined: 05 Aug 2016, 08:06
Location: The Construct
Contact:

Re: CliSAK class - 2020/11/29 - a122

04 Dec 2020, 13:59

@CappuccinoCake
Please read my previous post more carefully, as well as the comments in the main class script, specifically the option showWindow. Normally the console screen will be blank, because StdOut / StdIn / StdErr are redirected.

I'm afraid I mentioned Telnet as a possibility prematurely (I apologize for this). Other scripters / programmers have run into issues trying to run telnet and redirecting std handles. It would be more effective to rewrite Telnet in AHK using the socket class. Again, please read my last post. All is explained there.

EDIT:

I'm am planning to attempt to write a Telnet script that will recreate the protocol of the Telnet program, but I can't give a proposed ETA at this time.
CappuccinoCake
Posts: 3
Joined: 01 Dec 2020, 02:01

Re: CliSAK class - 2020/11/29 - a122

10 Dec 2020, 04:35

I tried it now with the socket class, with this it worked fine. Thank you for the help.
User avatar
TheArkive
Posts: 1027
Joined: 05 Aug 2016, 08:06
Location: The Construct
Contact:

Re: CliSAK class - 2021/03/09 - a128

09 Mar 2021, 11:18

Update:2021/03/09
  • updated for a128 - same functionality
Larkal
Posts: 21
Joined: 08 Mar 2019, 00:36

Re: CliSAK class - 2021/03/09 - a128

29 Apr 2021, 21:45

Any chance of an update soon for v133? I really love this and use it on a few of my scripts!
User avatar
TheArkive
Posts: 1027
Joined: 05 Aug 2016, 08:06
Location: The Construct
Contact:

Re: CliSAK class - 2021/03/09 - a128

29 Apr 2021, 23:07

Yes but more likely for a134. This script should be compatible with a131 currently. In a132-3 there were some issues that make several of my scripts non functional.

I'll officially update this post to a131 in a few hours.

If you really still want a133 just changing BufferAlloc to Buffer should do it. I normally upgrade all my scripts all at once.
User avatar
TheArkive
Posts: 1027
Joined: 05 Aug 2016, 08:06
Location: The Construct
Contact:

Re: CliSAK class - 2021/05/28 - a136

28 May 2021, 06:47

Update:2021/05/28
  • tested and working on a136
  • Now cmd output is untouched, all trailing spaces are left intact (except in mode "m").
  • ADB Shell is no longer "supported" but can still be easily managed by using 'ADB SHELL cmd params', which will always return to the windows prompt.
  • Fixed parsing of options to properly set WorkingDir if specified.
  • Updated examples and docs.
  • Fixed default env params for PowerShell (which is none, instead of "params" default).
  • Removed mode "r". Now the prompt is always removed from stdout. You can still capture the prompt with a prompt event.
  • Fixed mode "f" to finally properly remove control codes from the SSH prompt.
  • Added option "AutoClean" to RTrim spaces from line endings. But generally this should be done in client code, if at all.
  • Added obj.clean_lines() method to docs for RTrimming spaces from data, line by line.
  • Improved prompt detection regex.

Please read Important Info section in the updated OP, especially if you use this tool with Android Debug Bridge or PowerShell.
User avatar
TheArkive
Posts: 1027
Joined: 05 Aug 2016, 08:06
Location: The Construct
Contact:

Re: CliSAK class - 2021/06/21 - a137

21 Jun 2021, 07:37

Update:2021/06/21
  • Improved environment loading so that no commands will be fired until the initial prompt has loaded.
  • Added "adb" as an option for the env parameter on object creation. This improves handling of the ADB environment.
  • Improved detection of whether or not a prompt is displayed in selected environment (for ADB only at the moment). This type of functionality has to be added per-environment, as far as I can tell.
  • Removed "obj.ready" check in obj.Write() method.
  • Fixed default for "params" to be blank when specifying "powershell" as env, since /Q /K params are not valid for powershell.
  • Expanded detection of the command being output to stdout and truncated/wrapped to 80 chars. These echos are now automatically removed.
  • Updated regex detection of ADB prompt.
  • Improved false prompt construction for ADB when no prompt is returned.
  • Mode "f" is now automatically applied when env = "adb".
  • Added a general .Wait() method, mostly used with mode "c", for delayed execution and setting up a more complex environment.
  • Updated docs in the lib.

Return to “Scripts and Functions (v2)”

Who is online

Users browsing this forum: mprost and 23 guests