AHK with Chrome extension (No Selenium, no debug mode, no web connectors)

Put simple Tips and Tricks that are not entire Tutorials in this forum
burque505
Posts: 1731
Joined: 22 Jan 2017, 19:37

Re: AHK with Chrome extension (No Selenium, no debug mode, no web connectors)

04 Nov 2020, 08:42

I'd like to see that, @william_ahk, you might have a look at this thread.
Regards,
burque505
User avatar
tadamm
Posts: 40
Joined: 27 Sep 2020, 11:41

Re: AHK with Chrome extension (No Selenium, no debug mode, no web connectors)

04 Nov 2020, 19:53

william_ahk wrote:
04 Nov 2020, 04:06
Has anyone considered using Tampermonkey to communicate with AHK? If not I'll write one.
This sounds interesting never used it before how does it listen for AHK?
william_ahk
Posts: 481
Joined: 03 Dec 2018, 20:02

Re: AHK with Chrome extension (No Selenium, no debug mode, no web connectors)

04 Nov 2020, 21:07

tadamm wrote:
04 Nov 2020, 19:53
william_ahk wrote:
04 Nov 2020, 04:06
Has anyone considered using Tampermonkey to communicate with AHK? If not I'll write one.
This sounds interesting never used it before how does it listen for AHK?
I'm thinking about setting an HTTP server using G33kDude's Socket.ahk and use Tampermonkey to send requests to AHK via HTTP.
User avatar
tadamm
Posts: 40
Joined: 27 Sep 2020, 11:41

Re: AHK with Chrome extension (No Selenium, no debug mode, no web connectors)

05 Nov 2020, 09:41

william_ahk wrote:
04 Nov 2020, 21:07
I'm thinking about setting an HTTP server using G33kDude's Socket.ahk and use Tampermonkey to send requests to AHK via HTTP.
Would it be able to send req from AHK to tampermonkey?
SOTE
Posts: 1426
Joined: 15 Jun 2015, 06:21

Re: AHK with Chrome (No Selenium, no debug mode, no web connectors)

06 Nov 2020, 16:54

swagfag wrote:
01 Nov 2020, 08:27
:arrow: RunJsFromChromeAddressBar()
I agree with swagfag, that teadrinker's script is a very viable solution. AHKStudent gives a simplified example of how to use it for clicking links. This will work with both the new Edge and Chrome.

The only way I think you can make this script better or easier, is to kind of hide the JavaScript code (which you will need to control the browser). Have "pre-written" JavaScript code to do specific tasks. So that you have task specific functions like- ClickLink(LinkName, exe := "chrome.exe"), ClickButtton(ButtonName, exe := "msedge.exe"), or NextTab(HowManyTimes, exe := "chrome.exe").

A lot of what the AutoControl extension is doing, is giving pre-written JavaScript to do an action, allowing you to add in your own JavaScript code, or choose an external script. Teadrinker's script can be modified to do much the same, without being an extension.

Beyond that, maybe create an AHK specific browser extension. Various RPA offerings do this (UiPath, OpenRPA, etc...). The problem with TamperMonkey, ViolentMonkey, etc... To include AutoControl... You will still need to know JavaScript. Where with an AHK specific browser extension, it could be made to simplify controlling the browser with easy to understand/use functions (library of functions) made for AHK users. ClickLink(LinkName), ClickButton(ButtonName), CopyScreen(), ListLinks(), Wait4Page(), etc...
User avatar
tadamm
Posts: 40
Joined: 27 Sep 2020, 11:41

Re: AHK with Chrome extension (No Selenium, no debug mode, no web connectors)

07 Nov 2020, 09:59

can the teadrinker's script fill out forms and grab info from fields or only press buttons and tabs?
SOTE
Posts: 1426
Joined: 15 Jun 2015, 06:21

Re: AHK with Chrome extension (No Selenium, no debug mode, no web connectors)

07 Nov 2020, 18:12

tadamm wrote:
07 Nov 2020, 09:59
can the teadrinker's script fill out forms and grab info from fields or only press buttons and tabs?
In theory it can, but requires the JavaScript code to do such operations. Remember that JavaScript is the core language of browsers. This is kind of like how the "guts" of the AutoHotkey interpreter is C++, but the AHK community benefits from having the easier to use scripting language of AutoHotkey. To make teadrinker's script useful for the majority and a useful library of functions, it would require strong study into JavaScript or an AHKer who is also a JavaScript wiz. JavaScript syntax is also C family (like AHK), and should not be too hard for advanced AHKers to get into if they apply themselves or want to bother learning it.

To more clearly understand what I mean. You can have a function called ClickLink (one in a library of functions). It would contain "pre-written" JavaScript code to make browser automation easier. Each "ease of use" function would have different JavaScript code. The AHKer would just need to know the ClickLink function and add the library with #Include.

Looking at this issue, which repeatedly appears to come up (browser automation) I believe there are enough libraries, DllCalls, and knowledge of JavaScript floating around the AHK community to make a more AutoHotkey centric solution that doesn't need 3rd party add-ons/extensions, executables, or dlls. Though if somebody goes down the path of making an AutoHotkey extension for Chrome to do browser automation (like what UiPath and OpenRPA does) that would be cool too. Tadamm, we might also be able to dig into the source code of such add-ons/extensions, like AutoControl (that you have found and explain), or the OpenRPA extension.

Seems like expanding the idea of teadrinker's script, to make it even more user friendly, could go a long way. And as for AutoHotkey browser automation functions, of course those that know JavaScript can make more changes to suit their needs.

Code: Select all

ClickLink("Gmail")
Return

ClickLink(LinkText, exe := "msedge.exe")
{
	js =
	(LTrim
	(() => 
	{
		const links = document.links;
		for (let i = 0; i < links.length; i++) 
		{
			if (links[i].innerText == '%LinkText%') 
			{
				links[i].click();
				break
			}
		}
	})();
	)
	RunJsFromChromeAddressBar(js, exe)
}
   
RunJsFromChromeAddressBar(js, exe := "msedge.exe") 
{
   static WM_GETOBJECT := 0x3D
        , ROLE_SYSTEM_TEXT := 0x2A
        , STATE_SYSTEM_FOCUSABLE := 0x100000
        , SELFLAG_TAKEFOCUS := 0x1
        , AccAddrBar
   if !AccAddrBar {
      window := "ahk_class Chrome_WidgetWin_1 ahk_exe " . exe
      SendMessage, WM_GETOBJECT, 0, 1, Chrome_RenderWidgetHostHWND1, % window
      AccChrome := AccObjectFromWindow( WinExist(window) )
      AccAddrBar := SearchElement(AccChrome, {Role: ROLE_SYSTEM_TEXT, State: STATE_SYSTEM_FOCUSABLE})
   }
   AccAddrBar.accValue(0) := "javascript:" . js
   AccAddrBar.accSelect(SELFLAG_TAKEFOCUS, 0)
   ControlSend,, {Enter}, % window, Chrome Legacy Window
}

SearchElement(parentElement, params)
{
   found := true
   for k, v in params {
      try {
         if (k = "ChildCount")
            (parentElement.accChildCount != v && found := false)
         else if (k = "State")
            (!(parentElement.accState(0) & v) && found := false)
         else
            (parentElement["acc" . k](0) != v && found := false)
      }
      catch 
         found := false
   } until !found
   if found
      Return parentElement
   
   for k, v in AccChildren(parentElement)
      if obj := SearchElement(v, params)
         Return obj
}

AccObjectFromWindow(hWnd, idObject = 0) 
{
   static IID_IDispatch   := "{00020400-0000-0000-C000-000000000046}"
        , IID_IAccessible := "{618736E0-3C3D-11CF-810C-00AA00389B71}"
        , OBJID_NATIVEOM  := 0xFFFFFFF0, VT_DISPATCH := 9, F_OWNVALUE := 1
        , h := DllCall("LoadLibrary", "Str", "oleacc", "Ptr")
        
   VarSetCapacity(IID, 16), idObject &= 0xFFFFFFFF
   DllCall("ole32\CLSIDFromString", "Str", idObject = OBJID_NATIVEOM ? IID_IDispatch : IID_IAccessible, "Ptr", &IID)
   if DllCall("oleacc\AccessibleObjectFromWindow", "Ptr", hWnd, "UInt", idObject, "Ptr", &IID, "PtrP", pAcc) = 0
      Return ComObject(VT_DISPATCH, pAcc, F_OWNVALUE)
}

AccChildren(Acc) 
{
   static VT_DISPATCH := 9
   Loop 1  {
      if ComObjType(Acc, "Name") != "IAccessible"  {
         error := "Invalid IAccessible Object"
         break
      }
      try cChildren := Acc.accChildCount
      catch
         Return ""
      Children := []
      VarSetCapacity(varChildren, cChildren*(8 + A_PtrSize*2), 0)
      res := DllCall("oleacc\AccessibleChildren", "Ptr", ComObjValue(Acc), "Int", 0
                                                , "Int", cChildren, "Ptr", &varChildren, "IntP", cChildren)
      if (res != 0) {
         error := "AccessibleChildren DllCall Failed"
         break
      }
      Loop % cChildren  {
         i := (A_Index - 1)*(A_PtrSize*2 + 8)
         child := NumGet(varChildren, i + 8)
         Children.Push( (b := NumGet(varChildren, i) = VT_DISPATCH) ? AccQuery(child) : child )
         ( b && ObjRelease(child) )
      }
   }
   if error
      ErrorLevel := error
   else
      Return Children.MaxIndex() ? Children : ""
}

AccQuery(Acc) 
{
   static IAccessible := "{618736e0-3c3d-11cf-810c-00aa00389b71}", VT_DISPATCH := 9, F_OWNVALUE := 1
   try Return ComObject(VT_DISPATCH, ComObjQuery(Acc, IAccessible), F_OWNVALUE)
}
User avatar
tadamm
Posts: 40
Joined: 27 Sep 2020, 11:41

Re: AHK with Chrome extension (No Selenium, no debug mode, no web connectors)

07 Nov 2020, 19:53

The way I have it working only uses one line of JS and a few of AHK
gregster
Posts: 8916
Joined: 30 Sep 2013, 06:48

Re: AHK with Chrome extension (No Selenium, no debug mode, no web connectors)

07 Nov 2020, 19:56

That's apples and oranges.
You use a third-party extension - do you think that the extension is only running on fumes? :D

The javascript part can probably be the same with your and teadrinker's approach (for the same goal)... also the hotkey code, at least length-wise.
The rest are library functions - comparable to an extension. For a fair comparison, you need to compare the whole shebang.
User avatar
tadamm
Posts: 40
Joined: 27 Sep 2020, 11:41

Re: AHK with Chrome extension (No Selenium, no debug mode, no web connectors)

07 Nov 2020, 20:03

ok i thought the code above was for a single button click.
gregster
Posts: 8916
Joined: 30 Sep 2013, 06:48

Re: AHK with Chrome extension (No Selenium, no debug mode, no web connectors)

07 Nov 2020, 20:04

I used a single line of js when I clicked a button with teadrinker's code, iirc.
SOTE
Posts: 1426
Joined: 15 Jun 2015, 06:21

Re: AHK with Chrome extension (No Selenium, no debug mode, no web connectors)

07 Nov 2020, 20:19

tadamm wrote:
07 Nov 2020, 19:53
The way I have it working only uses one line of JS and a few of AHK
If the "ease of use" functions for browser automation were already created, you would just need to do:
Note- Could add an URL option to the function, but to keep things simple, AHK can get to the page with other commands.

Code: Select all

#Include BrowserAutomation

Run, http://www.google.com
ClickLink("Gmail", "Chrome.exe")
Return
The more difficult code would be in the BrowserAutomation.ahk file (if it were created).
ok i thought the code above was for a single button click.
The JavaScript code for each function would be different, based on the operation that the function would do.

Return to “Tips and Tricks (v1)”

Who is online

Users browsing this forum: No registered users and 14 guests