How to send text to a location in a window? Topic is solved

Get help with using AutoHotkey (v1.1 and older) and its commands and hotkeys
Newer2AHK
Posts: 59
Joined: 24 Jun 2020, 10:32

How to send text to a location in a window?

27 Jun 2020, 18:13

I need to enter text in a textbox on a webpage while the browser window is not active.

When the browser window is active, I can do this with:

Code: Select all

click 100, 890, 2		; Double-click in text box to select a value already there.
sleep 250
send %sTextValue%
So when the window is not active, I thought I could do this by analogy with ControlClick and ControlSend. The first command would become:

Code: Select all

SetControlDelay -1
ControlClick x100 y890, ahk_id %nAHK_ID_webpage%,,, 2, pos NA
This works by not specifying a control to click on, but by specifying the x-y coordinates at which to click. But ControlSend seems to require me to specify a control and does not allow me to specify coordinates.

When I make the browser window active and run the following on it:

Code: Select all

WinGet sControlList, ControlList, A
WinGet sControlIDList, ControlListHwnd, A
the variables sControlList and sControlIDList are empty. So I don't see how I can use controls to tell AHK where to enter the text. Is there a way to tell it to enter the text at a specific x-y location in the window?

In case it helps, the HTML for the text box on the page is:

Code: Select all

<h3>ZIP Code Search</h3>
                <div style="padding-left: 10px;">
                    ZIP Code*:<br />
                    <input type="text" id="OZipCode" maxlength="5" style="width: 65px" />&nbsp;<input type="button" value="Search" onclick="doZipSearch()" />
                </div>
Newer2AHK
Posts: 59
Joined: 24 Jun 2020, 10:32

Re: How to send text to a location in a window?

29 Jun 2020, 13:03

I see this question has dropped to the second page with 1,000 views and no answers. Does that mean there is no way to enter text in a text box on a webpage while the browser window is not active?
gregster
Posts: 8919
Joined: 30 Sep 2013, 06:48

Re: How to send text to a location in a window?

29 Jun 2020, 13:11

Did you search the forums? I think it's not a super-rare question.
All forum members and contributing guests,including the forum staff, are pure volunteers, with limited resources - just AHK users like you, helping each other out. So it can happen that some questions don't get answered (right away - or at all). But you were right to bump it, not having found a solution yet.

Website elements are no standard Windows controls. So selecting them or sending to them will often require other techniques or workarounds (luckily, browsers are often well-suited for automation).
The most reliable way is usually some kind of dedicated browser automation method - using a specific interface and/or library: IE COM, Chrome.ahk, Selenium, Acc.ahk - to name some methods. The best choice depends, among other factors and personal preferences, on which you browser you are planning (or allowed) to use. Basic knowledge of HTML and javascript is usually a plus.

I can probably give you examples for IE COM and Chrome.ahk (which can also automate some other Chromium-based browsers).
gregster
Posts: 8919
Joined: 30 Sep 2013, 06:48

Re: How to send text to a location in a window?  Topic is solved

29 Jun 2020, 13:29

I mean, using ControlClick and ControlSend might also work, to a certain extent, if the browser is just in the background, but not minimized. I just tested this on a Firefox instance with a typical Google results page - and it filled the search field at the top:

Code: Select all

F1::
SetControlDelay -1
ControlClick x370 y145, ahk_exe firefox.exe ,,, 2, NA	; adjust coords!
ControlSend, , test, ahk_exe firefox.exe
return
The Controlclick chose the website control based on position and then, leaving the control name out, ControlSend just sent to the topmost control of the chosen window (like described for the first parameter in the docs). No idea, if that will work on any browser type or how reliable it really is. Of course, you can't really account for slight layout changes on the website or the the zoom level this way. Browser automation would give you more options, even for "headless" (not visible) browsers. (Although changes in the website's source code might also break your script.) But perhaps this is already sufficient for your specific use case.

Edit:
Example for Internet Explorer's COM interface (see, for example, Joe Gline's videos; and many examples on this forum):

Code: Select all

ie := ComObjCreate("InternetExplorer.Application") 
ie.visible := true 
ie.Navigate("https://82187.csb.app/")		; mockup page for testing

; wait for the browser to load
while (ie.ReadyState != 4 or ie.Document.ReadyState != "complete" or ie.busy)
{
	sleep 100
}
msgbox Click 'OK' to fill in zip code...

ie.document.getElementById("OZipCode").value := 44304
Please note that I put the HTML code you provided above into a "codesandbox" (https://82187.csb.app), a website mockup, to be able to demonstrate the effect. You don't have to use it - you can try "your" website instead (but the code might fail, if there are frames involved - there is no way to tell, based on the snippet you provided).
zip search.png
zip search.png (2 KiB) Viewed 12261 times
(Currently, the button won't do anything.)


Edit2:
The same in Chrome (in debug mode), using Chrome.ahk by GeekDude:

Code: Select all

#NoEnv
SetBatchLines, -1
SetTitleMatchMode 2
#Include Chrome.ahk											; https://www.autohotkey.com/boards/viewtopic.php?f=6&t=42890

url := "https://82187.csb.app/"										; mockup page for testing
zip := 44304

; --- Create a new Chrome instance ---
FileCreateDir, ChromeProfile										; creates a profile subdirectory in current folder (on first start)
ChromeInst := new Chrome("ChromeProfile")
winwait, - Google Chrome	

; --- Connect to the page ---
if !(Page := ChromeInst.GetPage(  ))		 
{
	MsgBox, Could not retrieve page!
	ChromeInst.Kill()
}
else
	Page.WaitForLoad()

Page.Call("Page.navigate", {"url": url})			; Navigate to url
Page.WaitForLoad()
msgbox Click 'OK' to fill in zip code...

Page.Evaluate("document.getElementById('OZipCode').value = " zip) 
return

Esc::				; press Esc to exit script and Chrome instance
try ChromeInst.Kill()
ExitApp
Disclaimer: ;) Like always, use browser automation techniques in a reasonable manner; and at your own risk.
Newer2AHK
Posts: 59
Joined: 24 Jun 2020, 10:32

Re: How to send text to a location in a window?

15 Jul 2020, 22:25

Hi @gregster,

Thank you very much for your detailed answer with two revisions. I've been busy with another project, so it's been a while, but I'm now back on this.

I tried your suggested code using ControlSend in Firefox. The ControlClick worked, and I tried all kinds of variations and never saw any action from ControlSend. Too bad, that wold have been the easiest way to go.

Of the other two solutions you posted, I delved into Chrome.ahk . Interesting, although it requires me to install Chrome, which I could otherwise live without. I've been watching Joe Glines' video series of GeekDude explaining Chrome.ahk, which appears to be the only detailed documentation of it. I'm in the middle of watching video number 3, and I notice two things:
  • The tools it works with in Chrome look a lot like the Developer Tools in Firefox, which is not surprising because I've heard something about Firefox being built on the same platform as Chrome. (Chromium?)
  • Looking back at your Chrome.ahk code above, I see that all of the code exists to get one line to execute:

    Code: Select all

    Page.Evaluate("document.getElementById('OZipCode').value = " zip)
    and from the video (#3, t=17:00 et seq.) I understand that what that is doing is executing the javascript command:

    Code: Select all

    document.getElementById('OZipCode').value = "44304"
    on the code of the webpage in order to enter "44304" into the zip code text box, which is exactly what I asked how to do.
I went to the Firefox DevTools console and entered that javascript command, and voila, "44304" appeared in the zip code text box. So executing that code could do what I need, if I can only get AHK to execute it!

So the question this raises for me is:
  • Is there a simple way to get AHK to execute that javascript command in Firefox, or would that require translating Chrome.ahk into Firefox.ahk ?
I found a post on Reddit, where @GeekDude and another guy are discussing a couple dozen lines of AHK code to execute javascript in Firefox. But I don't understand that code and can't see how I could use it to set the value of a document element. If I could, that could be a simple solution to my problem.

On the other hand, your first response above says that Chrome.ahk can automate other Chromium-based browsers. Does that include Firefox? I searched for information on using Chrome.ahk with Firefox and didn't find anything. Can I do that? Is there information available on how to do it?

Do I need to go ahead and install Chrome and finish getting up to speed on Chrome.ahk, or is there a non-Herculean way to do this in Firefox?
Newer2AHK
Posts: 59
Joined: 24 Jun 2020, 10:32

Re: How to send text to a location in a window?

17 Jul 2020, 15:46

I posted a much shorter version of the above in the thread about Chrome.ahk. I got two answers there. I am responding further to those here because I don't want to hijack that thread away from its intended focus on Chrome.ahk.

What I thought should be a simple task of simulating the typing of text in a text box in a browser window that is not active is turning out to be not at all simple, but is going to be a major project.

Reviewing what I have learned so far:
  • Sending text to an active window. AHK can easily simulate typing at any location in an active window using "click %x%, %y%" to put the cursor at the desired location, followed by "send %text%".
  • Cannot send text to a non-control location in an inactive window. In an inactive window, AHK can simulate clicking at any location using "ControlClick". But there does not appear to be a native AHK command that can reliably put text at an arbitrary position. If the desired location is a window element called a "control," then text can be sent to it with "ControlSend". And @gregster said above that he succeeded in typing in a text box in Firefox by first clicking in the box with "ControlClick" and then using "ControlSend" to send the text. But he said it was of doubtful reliability, and it didn't work for me. I wish I knew why. If I could get that to work, it would solve my problem.
  • Non-native ways to put text in an HTML text box. If the desired location is an HTML text box (as it is for me), then typing in it can be simulated by executing a javascript command on the text box. But apparently, AHK does not have a native way to do that. The ways I have seen to do it in AHK are:
    • Use Chrome.ahk to get AHK to communicate with the browser. Then use "page.evaluate()" to execute javascript commands. This can be done when the browser is:
    • Use COM in Internet Explorer. (I'm not sure it would be worthwhile to invest time getting up to speed in IE, which is deprecated by MS in favor of Edge.)
    • Use a webdriver with ahkWebDrive. Then use "WDsession.execute()" to execute javascript commands.
    • As I mentioned above, there's a post on Reddit, where @GeekDude and another guy discuss a couple dozen lines of AHK code to execute javascript in Firefox. I don't understand the code. If I could understand that code, it might solve my problem. Can someone explain that code or point to something that would help me understand it?
    • @malcev suggested using IAccessible or IUIAutomation. When I Google those along with "AutoHotkey", I get a lot of stuff, but it's pretty confusing and I haven't made sense out of it or figured out how to do anything with it.
    • @malcev also said I can run javascript from the address bar. I tried that and it didn't work. I found some discussion on about it on StackOverflow and also an explanation of why it used to work but doesn't now. But even if it did work, I don't think that would help because then I face the problem of how to get AHK to simulate typing the javascript command in the address bar, which doesn't seem to be any different from getting AHK to simulate typing the text I need in the text box.
Of the possibilities listed above, I like the idea of the WebDriver and UI automation, but without decent documentation, I'm left scratching my head over how to use them. So I think the best option might be Chrome.ahk with Chrome because the videos of @GeekDude explaining it appear to lay out a path to implementation. Text documentation would be better than videos, but videos are better than nothing. So I guess I'll go back to watching the videos to get up to speed on that method.

If I'm making this more complicated than it needs to be, I hope someone will tell me how I can do this simply. It seems strange that AHK allows me to easily click anywhere on an inactive window (with "ControlClick") but doesn't give me any direct way to put text there. If I'm wrong about that and there is a simple way to do it, please say what it is.
Newer2AHK
Posts: 59
Joined: 24 Jun 2020, 10:32

Re: How to send text to a location in a window?

18 Jul 2020, 13:21

If one of the expert users here has time, it could be helpful if you would say if it looks like I'm on the right track, or if there is something naive or wrong about the approach described.

Thanks.
malcev
Posts: 1769
Joined: 12 Aug 2014, 12:37

Re: How to send text to a location in a window?

19 Jul 2020, 10:44

What is the site?
And what do You want?
Newer2AHK
Posts: 59
Joined: 24 Jun 2020, 10:32

Re: How to send text to a location in a window?

19 Jul 2020, 11:32

@malcev wrote:
> What is the site?

The issue is not relevant to any particular site. In my first post in this topic, I gave the HTML of a text box on an example site where I want to enter a zip code. Gregster made a prototype website from that code, which demonstrates the issue, and which he used to test his two solutions using DOM in IE and Chrome.ahk in Chrome. He also tested the issue in Google.com in Firefox and said he was able to enter text in the the search bar there using ControlClick followed by ControlSend. But he also doubted the reliability of that method and I was not able to get it to work.

> And what do you want?

What I need to do is simulate the typing of text in a text box in a browser window that is not the active window (but also is not hidden). Doing that is trivially easy in an active window using "click" followed by "send". But when the window is not active, there does not seem to be a way to do it natively in AHK.

If it's correct that that cannot be done natively, I'd be really interested in knowing why. ControlClick is able to simulate a click anywhere in a webpage in an inactive window. Why can't ControlSend put keystrokes there?

I've installed Chrome and have been studying Chrome.ahk, making progress in understanding it. I'd still be interested if people have any comments on my research notes posted above on this issue.
malcev
Posts: 1769
Joined: 12 Aug 2014, 12:37

Re: How to send text to a location in a window?

19 Jul 2020, 11:46

I do not see any issue:

Code: Select all

F1::
SetControlDelay -1
SetKeyDelay, -1
ControlClick x75 y213, ahk_exe firefox.exe,,,, NA	; adjust coords!
ControlSend, ahk_parent, test, ahk_exe firefox.exe
return
Newer2AHK
Posts: 59
Joined: 24 Jun 2020, 10:32

Re: How to send text to a location in a window?

22 Jul 2020, 08:36

@malcev:

I tried your code, and I was excited to see that it worked. Looking into why, I found that you had included "ahk_parent" as the "Control" parameter in the "ControlSend" command. That was not in @gregster's code, and it seemed to make the difference in getting it to work.

However, it did not work anymore when I modified the code to put the target window in the background.

Here is my code, adapted from yours, that demonstrates the difference from using "ahk_parent".

Code: Select all

; "put text in a background window - 1.ahk"
; From www.autohotkey.com/boards/viewtopic.php?p=338929#p338929 (post by gregster, 2020 06 29)
; and www.autohotkey.com/boards/viewtopic.php?p=342763#p342763 (post by malcev, 2020 07 19)

; SetControlDelay -1			These two commands don't seem to be necessary.
; SetKeyDelay -1

run https://82187.csb.app		; gregster's prototype webpage with just a text box.
sleep 1000
WinGet nAHK_ID_ZipSearch, ID, Parcel Sandbox

ControlClick x100 y325, ahk_id %nAHK_ID_ZipSearch%,,,, pos NA
ControlSend , 11111, ahk_id %nAHK_ID_ZipSearch%
MsgBox % "A zip code did NOT get entered. ControlSend was used WITHOUT ""ahk_parent""."

ControlSend ahk_parent, 22222, ahk_id %nAHK_ID_ZipSearch%
MsgBox % "A zip code DID get entered. ControlSend was used WITH ""ahk_parent""."

return
The message boxes report what is happening, at least on my computer (Windows 10 Pro 64, Firefox 78, AutoHotkey 1.1.32). The ControlSend command has no effect when used without "ahk_parent", but when used with it, the command works, writing the zip code in the text box.

Image

But here is the same code with the insertion of a line that opens another browser window, so the window I want to write to is in the background:

Code: Select all

; "put text in a background window - 2.ahk"
; From www.autohotkey.com/boards/viewtopic.php?p=338929#p338929 (post by gregster, 2020 06 29)
; and www.autohotkey.com/boards/viewtopic.php?p=342763#p342763 (post by malcev, 2020 07 19)

; SetControlDelay -1			These two commands don't seem to be necessary.
; SetKeyDelay -1

run https://82187.csb.app		; gregster's prototype webpage with just a text box.
sleep 1000
WinGet nAHK_ID_ZipSearch, ID, Parcel Sandbox
run https://www.Google.com
sleep 1000

ControlClick x100 y325, ahk_id %nAHK_ID_ZipSearch%,,,, pos NA
ControlSend , 11111, ahk_id %nAHK_ID_ZipSearch%
MsgBox % "A zip code did NOT get entered. ControlSend was used WITHOUT ""ahk_parent""."

ControlSend ahk_parent, 22222, ahk_id %nAHK_ID_ZipSearch%
MsgBox % "A zip code DID get entered. ControlSend was used WITH ""ahk_parent"".`n"
		. "But, oops, it got entered on the Google page, the active page."

return
This time, the zip code gets written, but in the wrong window. Even though the ControlSend command uses the AHK ID of the zip code search page, the text gets written on the Google page.

Image

So it appears that ControlSend is not working correctly. Am I doing something wrong?
Last edited by Newer2AHK on 23 Jul 2020, 06:39, edited 1 time in total.
malcev
Posts: 1769
Joined: 12 Aug 2014, 12:37

Re: How to send text to a location in a window?

22 Jul 2020, 12:29

I think that You cannot Controlsend to another firefox window while You have active document of firefox.
You can activate url or search bar of active firefox and then controlsend will work.
You cannot explain what do You want and therefore You will get such results.
Newer2AHK
Posts: 59
Joined: 24 Jun 2020, 10:32

Re: How to send text to a location in a window?

22 Jul 2020, 15:07

As far as I understand, the purpose of ControlSend is to send keystrokes to a window that may not be the active window. That's why it has the parameter "WinTitle", to allow one to select which window to send to. If I have to make the window active first, then I could just use "send". In the second sample script in my previous post, the script tells ControlSend to send the keystrokes to the zip search window, but instead they are sent to the active window. Isn't that incorrect behavior by ControlSend? Is there a reason for it? Should I file a bug report?

I don't know why you say that I am not saying what I want. I think I've been very clear about what I want. I want to send keystrokes to a text box in a browser window in the background. Can I be more clear than that?
malcev
Posts: 1769
Joined: 12 Aug 2014, 12:37

Re: How to send text to a location in a window?

22 Jul 2020, 15:37

As far as I understand, the purpose of ControlSend is to send keystrokes to a window that may not be the active window
It depends on target program.
I don't know why you say that I am not saying what I want. I think I've been very clear about what I want.
You did not tell that You want use firefox while sending to another instance of firefox.
Newer2AHK
Posts: 59
Joined: 24 Jun 2020, 10:32

Re: How to send text to a location in a window?

22 Jul 2020, 16:59

Is there a way to know how the capability of ControlSend depends on the target program? I don't see anything about that in the documentation.
malcev
Posts: 1769
Joined: 12 Aug 2014, 12:37

Re: How to send text to a location in a window?

23 Jul 2020, 00:09

No, You have to test it by Yourself.
ControlSend is just wrapper of PostMessage, WM_CHAR.
Also as I said before there are a lot of different approaches to automate tasks with Your browser.
WinHttpRequest, IAccessible, UIA, webdriver, selenium, chrome wrapper, tagui, imacros, run js from address bar, special browsers extensions...
Newer2AHK
Posts: 59
Joined: 24 Jun 2020, 10:32

Re: How to send text to a location in a window?

23 Jul 2020, 08:45

"ControlSend is just wrapper of PostMessage, WM_CHAR."

Interesting. I'm not experienced in GitHub, but I went there to see if I could see what you mean. If I understand correctly, it's not *just* a wrapper, but is one massively complex wrapper!

For future reference, if I got this right, a function ControlSend() is defined in script2.cpp. It calls a function SendKeys(), which is defined in keyboard_mouse.cpp. It has 855 lines of code and calls a function PostMessage() in two places with parameter WM_CHAR, as well as doing God (i.e., Lexikos)-only-knows what else.

There's a post on StackOverflow that says of SendKeys(): "All kinds of checks and magic is done in there, but in the end, you'll more often than not arrive at KeyEvent(), where the key is sent by posting a WM_KEYDOWN and WM_KEYUP message to the target window. In some special cases (if the the key doesn't seem to have a VK or SC), it is sent by posting a WM_CHAR message." If that is correct, it seems to be saying that PostMessage is not the primary method used by ControlSend to do its sending.
malcev
Posts: 1769
Joined: 12 Aug 2014, 12:37

Re: How to send text to a location in a window?

23 Jul 2020, 11:54

You can see what controlsend sends with spy++.
No magic.
wm_keydown, wm_keyup, wm_char for every letter.
User avatar
Xtra
Posts: 2744
Joined: 02 Oct 2015, 12:15

Re: How to send text to a location in a window?

23 Jul 2020, 12:37

change:
ControlSend , 11111, ahk_id %nAHK_ID_ZipSearch%
to
ControlSend, , 11111, ahk_id %nAHK_ID_ZipSearch%

Simple test example:

Code: Select all

MsgBox , is this the title?, test 1, 3
MsgBox , , is this the title?, test 2, 3
MsgBox 0, is this the title?, test 3, 3
HTH

Edit: added help file reference commands
The comma separating the command name from its parameters is optional, except in the following cases:
#2 When the first parameter is blank.
Newer2AHK
Posts: 59
Joined: 24 Jun 2020, 10:32

Re: How to send text to a location in a window?

23 Jul 2020, 13:28

@Xtra: Thank you for that correction. I read in the doc that the comma separating the command name from its parameters is optional, except in three cases. I didn't realize at first that the second case is really important, which is when the first parameter is omitted, but followed by other parameters. I've been omitting that comma because that made the code look more like other languages. But I’m putting it back in from now on to keep my AHK syntax consistent and to prevent errors like the one you just pointed out.

However, when I make that correction in my sample scripts above, it does not change the behavior. The text is still not sent when "ahk_parent" is omitted and is sent when "ahk_parent" is included. Have you found differently?

Edit: I see that you were adding that edit to cite the doc at the same time that I was writing my response that cited it.

Return to “Ask for Help (v1)”

Who is online

Users browsing this forum: catquas, Descolada, garry, Google [Bot], jarhead, mikeyww, Ralf_Reddings200244, sofista, thelegend and 148 guests