Page 2 of 31
Re: Rufaydium WebDriver (no selenium/websocket)
Posted: 18 Apr 2022, 12:25
by joedf
This is pretty neat!
Question, how can I set the user-agent?
Something with
ChromeDriver.exe --user-agent ?
Re: Rufaydium WebDriver (no selenium/websocket)
Posted: 18 Apr 2022, 12:53
by Xeo786
joedf wrote: ↑18 Apr 2022, 12:25
This is pretty neat!
Question, how can I set the user-agent?
Something with
ChromeDriver.exe --user-agent ?
@joedf
I just able to run chrome with user agent,
You need to add a custom profile into Capabilities class
Code: Select all
static ChromeCustomUserAgent =
( LTrim Join
{
"capabilities": {
"alwaysMatch": {
"browserName": "chrome",
"platformName": "windows",
"goog:chromeOptions": {
"w3c": json.true,
"args" :["--user-agent=Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.127 Safari/537.36"],
"excludeSwitches": ["enable-automation"]
}
},
"firstMatch": [{}]
},
"desiredCapabilities": {
"browserName": "chrome"
}
}
)
then code should be
Code: Select all
f1::
Driver := new RunDriver(ChromeDriver) ; running driver
Chrome := new Rufaydium(Driver) ; this will return control over Browser
Chrome.capabilities := Capabilities.ChromeCustomUserAgent ; loading custom capabilities
; msgbox, % json.dump(Chrome.capabilities)
Page := Chrome.NewSession()
Page.Navigate("https://www.google.com/")
Page := ""
return
Re: Rufaydium WebDriver (no selenium/websocket)
Posted: 19 Apr 2022, 08:16
by joedf
Great, thanks!
Re: Rufaydium WebDriver (no selenium/websocket)
Posted: 19 Apr 2022, 10:01
by Milchmann
Milchmann wrote: ↑14 Apr 2022, 07:42
Thank you, that is already very good.
I will try it next week, but I can say that various things already work.
Some things do not work yet, but I'll write that down and ask.
Hello,
I am trying to recreate the example code.
The code with the F1 key is ok. But as soon as I press F3 after pressing F1, Autohotkey hangs. The reason is
Listline displays
Code: Select all
0: Sessions := this.send(this.DriverUrl "/sessions","GET")
034: if !instr(url,"HTTP")
036: r := Json.load(Request(url,Method,Payload,WaitForResponse)).value
---- C:\TEMP\Rufaydium\lib\WDM.ahk
050: WebRequest.Open(Method, url, false)
051: WebRequest.SetRequestHeader("Content-Type","application/json")
052: if Payload
055: WebRequest.Send() (0.03)
056: if WaitForResponse
057: WebRequest.WaitForResponse()
058: Return,WebRequest.responseText
---- C:\TEMP\Rufaydium\lib\JSON.ahk
261: this._init()
065: if (this.lib)
066: Return
263: _json := " " json
264: VarSetCapacity(pJson, A_PtrSize)
265: NumPut(&_json, &pJson, 0, "Ptr")
267: VarSetCapacity(pResult, 24)
269: if (r := DllCall(this.lib.loads, "Ptr", &pJson, "Ptr", &pResult , "CDecl Int")) || ErrorLevel
276: result := ComObject(0x400C, &pResult)[]
277: if (IsObject(result))
278: ObjRelease(&result)
279: Return,result
---- C:\TEMP\Rufaydium\lib\Rufaydium.ahk
037: if r
038: Return,r
071: windows := []
072: For k,se in Sessions
074: chromeOptions := Se["capabilities","goog:chromeOptions"]
075: s := []
076: s.id := Se.id
077: s.debuggerAddress := StrReplace(chromeOptions.debuggerAddress,"localhost","http://127.0.0.1")
078: s.address := this.DriverUrl "/session/" s.id
079: windows[k] := new Session(s)
118: this.id := i.id
119: this.Address := i.address
120: this.debuggerAddress := i.debuggerAddress
121: this.currentTab := this.Send("window","GET")
034: if !instr(url,"HTTP")
035: url := this.address "/" url
036: r := Json.load(Request(url,Method,Payload,WaitForResponse)).value
---- C:\TEMP\Rufaydium\lib\WDM.ahk
050: WebRequest.Open(Method, url, false)
051: WebRequest.SetRequestHeader("Content-Type","application/json")
052: if Payload
055: WebRequest.Send() (60.19)
Other conspicuous features:
-I can no longer get into the developer tools in Chrome by pressing F12 done, my mistake
- querySelectorAll("a").length returns no value on google.com
My Chrome version is 100.0.4896.60 and Autohotkey version 1.1.33.10
I tried it with 2 different computers with the same result.
Do you have any idea why this is occurring?
Thanks
Re: Rufaydium WebDriver (no selenium/websocket)
Posted: 19 Apr 2022, 14:16
by ahk7
@Xeo786 I've submitted a pull request to fix some typos, you can see the changes here
https://github.com/Xeo786/Rufaydium-Webdriver/pull/1/files
Just as note: you can also add images to your GitHub repository so you don't have to rely on a third party image hosting provider.
Re: Rufaydium WebDriver (no selenium/websocket)
Posted: 19 Apr 2022, 14:31
by ahk7
Input text, checkbox, clicking buttons all OK but one thing that eludes me at the moment is selection options in a
<select> - I can get the innertext so it does see the html but setting a value .selectByIndex .selectByVisibleText or .selectByValue doesn't seem to do much. Anybody have an idea?
Code: Select all
/*
html code:
<select id='selform' size=4>
<option>
<option value='ca'>Canada
<option value='fr'>France
<option value='uk'>United Kingdom
</select>
*/
dropdown := GS.GetelementById("selform")
MsgBox % dropdown.innertext ; works
dropdown.selectByIndex(1)
dropdown.selectByVisibleText("Canada")
dropdown.selectByValue("fr")
Re: Rufaydium WebDriver (no selenium/websocket)
Posted: 19 Apr 2022, 14:33
by ahk7
Milchmann wrote: ↑19 Apr 2022, 10:01
- When the code runs ( f1 executed ) , I can no longer get into the developer tools in Chrome by pressing F12
If you use the posted example script
f12 is a hotkey at the bottom of the script
Code: Select all
...
f12:: ; close all session first then exit driver
Driver := new RunDriver(ChromeDriver)
Chrome := new Rufaydium(Driver)
Chrome.QuitAllSessions()
driver.exit()
return
Re: Rufaydium WebDriver (no selenium/websocket)
Posted: 20 Apr 2022, 00:21
by Milchmann
ahk7 wrote: ↑19 Apr 2022, 14:33
Milchmann wrote: ↑19 Apr 2022, 10:01
- When the code runs ( f1 executed ) , I can no longer get into the developer tools in Chrome by pressing F12
If you use the posted example script
f12 is a hotkey at the bottom of the script
Code: Select all
...
f12:: ; close all session first then exit driver
Driver := new RunDriver(ChromeDriver)
Chrome := new Rufaydium(Driver)
Chrome.QuitAllSessions()
driver.exit()
return
Thanks, I had overlooked the hotkey
Re: Rufaydium WebDriver (no selenium/websocket)
Posted: 20 Apr 2022, 02:03
by Xeo786
Just Approved and merged your pull request I am exploring github and learning stuff. thank you
, When it comes spell I am awfull
Re: Rufaydium WebDriver (no selenium/websocket)
Posted: 20 Apr 2022, 02:56
by Xeo786
ahk7 wrote: ↑19 Apr 2022, 14:31
Input text, checkbox, clicking buttons all OK but one thing that eludes me at the moment is selection options in a
<select> - I can get the innertext so it does see the html but setting a value .selectByIndex .selectByVisibleText or .selectByValue doesn't seem to do much. Anybody have an idea?
Code: Select all
/*
html code:
<select id='selform' size=4>
<option>
<option value='ca'>Canada
<option value='fr'>France
<option value='uk'>United Kingdom
</select>
*/
dropdown := GS.GetelementById("selform")
MsgBox % dropdown.innertext ; works
dropdown.selectByIndex(1)
dropdown.selectByVisibleText("Canada")
dropdown.selectByValue("fr")
by using this technique I can easily implement
selectByIndex(),
selectByVisibleText()and
selectByValue()
but there are two ways to select dropdown list, and construction of dropdown lists not same everytime
Following is working example for recent type of list box uses ul>il tag names
Code: Select all
Driver := new RunDriver(ChromeDriver) ; running driver
Chrome := new Rufaydium(Driver) ; this will return control over Browser
Chrome.capabilities := Capabilities.ChromeCustomUserAgent ; loading custom capabilities
; msgbox, % json.dump(Chrome.capabilities)
Page := Chrome.getSessionByUrl("https://www.w3.org/TR/wai-aria-practices-1.1/examples/listbox/listbox-collapsible.html")
; we can also do getSessionByTitle(title)
if !IsObject(Page) ; GS >> google session
{
Page := Chrome.NewSession()
Page.Navigate("https://www.w3.org/TR/wai-aria-practices-1.1/examples/listbox/listbox-collapsible.html")
}
msgbox, Fast Way
Page.Getelementbyid("exp_button").click()
Slist := Page.querySelector("#exp_elem_list")
for k, item in StrSplit(Slist.innerText,"`n") ; using strsplit to get to location this is fast
{
if instr(item,"Curium")
{
Slist.QuerySelectorAll("li")[k].click()
}
}
Page.Getelementbyid("exp_wrapper").click() ; clicking somewhere else to collapse dropdown
msgbox, slow way
Page.Getelementbyid("exp_button").click()
for k, li in Slist.QuerySelectorAll("li")
{
if instr(li.innerText,"Californium") ; asking for every element innerText take time
{
li.click()
}
}
Page.Getelementbyid("exp_wrapper").click() ; clicking somewhere else to collapse dropdown
Msgbox, done
Re: Rufaydium WebDriver (no selenium/websocket)
Posted: 20 Apr 2022, 03:32
by Xeo786
Milchmann wrote: ↑19 Apr 2022, 10:01
The code with the F1 key is ok. But as soon as I press F3 after pressing F1, Autohotkey hangs. The reason is
If you feel leg while accessing session, close all session using Session.exit(), and Driver.exit() and reload driver
the issue is when you manually close session created by driver, its not properly closed. and webdriver take time to query non exiting session
so make sure always close session using session.exit()
Re: Rufaydium WebDriver (no selenium/websocket)
Posted: 20 Apr 2022, 09:04
by Tre4shunter
So, at the end of the day, Rufaydium is CDP(chrome.ahk - Minus the IE websocket) being controlled via chromedriver.exe? Obviously IE(Websocket for chrome.ahk) has its issues, but its like...the devil you know vs the one you don't.
Are there any reliability concerns here? I'm thinking about moving my Chrome.ahk apps over to this, but don't want to cause myself unnecessary headaches down the line.
Again, great work though! Thoroughly impressed!
-Tre4
Re: Rufaydium WebDriver (no selenium/websocket)
Posted: 20 Apr 2022, 10:59
by Tre4shunter
Actually, I dont see a way to emulate Callbacks like we could in chrome.ahk.
Is this something that would be possible with Rufaydium? Looking online i see some examples for python/selenium related to monitoring console messages etc, but havent dug in too far yet.
What im referencing would be like so via chrome.ahk:
Code: Select all
BoundCallback := Func("Callback").Bind()
PageInstCreate := ChromeInst.GetPageByURL("file:///" Path, "exact",,BoundCallBack)
if !PageInstCreate
{
ChromeInst.Kill()
MsgBox, Could not Connect to Quote Page!
ExitFunc()
}else{
PageInstCreate.Call("Runtime.enable")
}
;Listen for messages here {console.logs()]
Callback(event){
GLOBAL
if(Event.Method = "Runtime.consoleAPICalled"){
}
}
Re: Rufaydium WebDriver (no selenium/websocket)
Posted: 20 Apr 2022, 11:13
by Xeo786
Tre4shunter wrote: ↑20 Apr 2022, 10:59
Actually, I dont see a way to emulate Callbacks like we could in chrome.ahk.
Is this something that would be possible with Rufaydium? Looking online i see some examples for python/selenium related to monitoring console messages etc, but havent dug in too far yet.
What im referencing would be like so via chrome.ahk:
Code: Select all
BoundCallback := Func("Callback").Bind()
PageInstCreate := ChromeInst.GetPageByURL("file:///" Path, "exact",,BoundCallBack)
if !PageInstCreate
{
ChromeInst.Kill()
MsgBox, Could not Connect to Quote Page!
ExitFunc()
}else{
PageInstCreate.Call("Runtime.enable")
}
;Listen for messages here {console.logs()]
Callback(event){
GLOBAL
if(Event.Method = "Runtime.consoleAPICalled"){
}
}
There is a way you can access event using logs files generated by webdriver you can use these cmd argument like --log-file-path or --enable-chrome-logs
https://github.com/Xeo786/Rufaydium-Webdriver#rundriver
I don't know any callback support for webdriver but I think callback can be setup using these log files
Re: Rufaydium WebDriver (no selenium/websocket)
Posted: 20 Apr 2022, 14:57
by ahk7
Xeo786 wrote: ↑20 Apr 2022, 02:56
Following is working example for recent type of list box uses ul>il tag names ...
works for me, thanks.
Re: Rufaydium WebDriver (no selenium/websocket)
Posted: 21 Apr 2022, 01:51
by Milchmann
Hello,
I am trying to recreate the example code.
The code with the F1 key is ok. But as soon as I press F3 after pressing F1, Autohotkey hangs. The reason is
Code: Select all
GS := Chrome.getSessionByUrl("https://www.google.com/")
If you feel leg while accessing session, close all session using Session.exit(), and Driver.exit() and reload driver
the issue is when you manually close session created by driver, its not properly closed. and webdriver take time to query non exiting session
so make sure always close session using session.exit()
Hi,
thank you for the answer.
to the points:
- With your example, if I kill the driver at F1 with Driver.Exit(), then when I start F3 the following comes:
Code: Select all
if !IsObject(GS) ; GS >> google session
{
msgbox, Session with google tab no found
return
}
What should the code be then if it should work without Autohotkey hanging? Please an example.
- Session.exit() ( Page.exit() in the example ) does not exist.
- There is a problem with AutoHotkeyU64.exe and your code. Here I can partly not access the driver at all. Might be explainable, because Chromedriver.exe is 32 bit.
- querySelectorAll("a").length returns no value on google.com --> is there another way to get the length?
Thanks for your effort and I hope the project will run properly.
Bert
Re: Rufaydium WebDriver (no selenium/websocket)
Posted: 21 Apr 2022, 03:54
by Xeo786
Milchmann wrote: ↑21 Apr 2022, 01:51
to the points:
- With your example, if I kill the driver at F1 with Driver.Exit(), then when I start F3 the following comes:
Code: Select all
if !IsObject(GS) ; GS >> google session
{
msgbox, Session with google tab no found
return
}
What should the code be then if it should work without Autohotkey hanging? Please an example.
- Session.exit() ( Page.exit() in the example ) does not exist.
- There is a problem with AutoHotkeyU64.exe and your code. Here I can partly not access the driver at all. Might be explainable, because Chromedriver.exe is 32 bit.
- querySelectorAll("a").length returns no value on google.com --> is there another way to get the length?
Thanks for your effort and I hope the project will run properly.
Bert
This has been discussed see link
https://github.com/Xeo786/Rufaydium-Webdriver#script-reloading
1) never close automated session manually it will cause leg, in case you manually close session then driver will response with delay solution is to driver.exit()
2) Its better to Session.exit() then Driver.Exit()
3) Yah exiting driver without closing session will cause no longer access to any previous session created through WebDriver
Re: Rufaydium WebDriver (no selenium/websocket)
Posted: 21 Apr 2022, 10:10
by Tre4shunter
using "--log-PATH=c:\myfile.log --enable-chrome-logs" does work to log the console messages from a web page...but whenever I add those two switches, for some reason it continuously pops up multiple cmd windows for chrome.exe which are blank.
I've not yet figured out how to prevent those cmd windows from appearing wen using the above switches...
Re: Rufaydium WebDriver (no selenium/websocket)
Posted: 21 Apr 2022, 12:16
by ahk7
@Tre4shunter fyi driver.show() or driver.hide() has been added -
https://github.com/Xeo786/Rufaydium-Webdriver/blob/main/WDM.ahk#L53 - perhaps c:\ is write protected (often is).
Milchmann wrote:There is a problem with AutoHotkeyU64.exe and your code. Here I can partly not access the driver at all. Might be explainable, because Chromedriver.exe is 32 bit.
No such problems here, runs on Win7 32bit and Win10 64bit here. Try adding
https://www.autohotkey.com/docs/commands/ListVars.htm after the
GS := Chrome.getSessionByUrl("https://www.google.com/") line - if it doesn't show up as 'session' there might be something wrong indeed.
Try stripping down the script to the bare minimum and only add a new thing if it works.
Re: Rufaydium WebDriver (no selenium/websocket)
Posted: 21 Apr 2022, 12:20
by ahk7
.innertext works, but how do you get the .OuterHTML? Not that I directly need it but would be useful to have to be able to inspect and debug I think (I do see OuterHTML in CDP.ahk)