Rufaydium WebDriver 1.7.2 (no selenium/websocket)

Post your working scripts, libraries and tools for AHK v1.1 and older
User avatar
Xeo786
Posts: 759
Joined: 09 Nov 2015, 02:43
Location: Karachi, Pakistan

Re: Rufaydium WebDriver 1.7.2 (no selenium/websocket)

Post by Xeo786 » 08 Dec 2022, 00:48

anhnha wrote:
07 Dec 2022, 16:14

Code: Select all

This will not create blank page
I'm not sure why you keep saying that it does not create blank page.
I did try that many times and it didn't work for me. I also I captured result that it still opened with a page called data; in my previous post.
image.png

Code: Select all

/* ; this is wrong
session:= Chrome.getSession(1)
Page.close(session)
*/
Session:= Chrome.getSession(1)
Session.close()
And this again does not work as well. I want to delete about:blank#blocked but all the suggestions so far sometimes delete the wrong tab or not delete any tab at all.
please Read documentation for basic tasks
I would be more than happy if this work but normally it does not as you don't give a complete example to show how it work so many times I get confused between pages/tabs/sessions and even sessions and Session.
I have been trying to read definitions in function definition as well.

So far none of the methods you said above works.
Does it work on your PC or just me?
I tried it on two PC but none of them work.
Here is the image showing that it does not work and error.

image.png

image.png

And another time, with exactly same code above, it deleted the wrong page. The blank page is still there.

Code: Select all

Session:= Chrome.getSessionByUrl("about:Blank")
Session.close()
Chrome.getSessionByUrl("about:Blank") works but I'm looing for a solution to not generate the blank page instead of just create and then remove.
That is why I asked if that blank page is required.
You can disable that line but no help, it created a different blank page.
didn't that code work for you ?

Code: Select all

Chrome := new Rufaydium()
urls := ["https://www.autohotkey.com/boards/viewforum.php?f=4","https://chromedevtools.github.io/devtools-protocol","https://github.com/Xeo786/Rufaydium-Webdriver"]
Page := Chrome.NewSession()
; Page.url := "about:Blank" ; this line has been disabled
for i, url  in urls
    Page.CDP.CreateTarget(url)
"When there is no gravity, there is absolute vacuum and light travel with no time" -Game changer theory

anhnha
Posts: 116
Joined: 08 Aug 2018, 02:46

Re: Rufaydium WebDriver 1.7.2 (no selenium/websocket)

Post by anhnha » 08 Dec 2022, 01:02

Code: Select all

didn't that code work for you ?
No, and I have been saying this for several posts above. I also took pictures to show as well. You can check all images in my previous posts.

From the document:

Code: Select all

msgbox, % Chrome1.status()
Do you have any example to show how to use this information to check?

User avatar
Xeo786
Posts: 759
Joined: 09 Nov 2015, 02:43
Location: Karachi, Pakistan

Re: Rufaydium WebDriver 1.7.2 (no selenium/websocket)

Post by Xeo786 » 08 Dec 2022, 01:07

mora145 wrote:
07 Dec 2022, 18:26
Xeo786 wrote:
07 Dec 2022, 02:25
mora145 wrote:
07 Dec 2022, 01:14
I am trying to do some scrolling in this window. I can't get it to scroll correctly but it's the screen behind me that moves. How can I do it?
Screenshot_81.jpg
Get all elements having innertext unfollow, now you can use element[x].focus it will automatically scroll to the element and focus that element
if you use New Scroll() it will scroll the stuff according to the mouse pointer location, therefore you also need to use New Mouse() to place the pointer to such a location so it gets the desired scrollbar act upon.
Thanks for your answer. I don't quite understand what element[x].focus should be used for, if New Mouse is going to be used to then click. New Mouse will click on the focus element? But you are supposed to be suggesting to use innertext in unfollow.
Just like JavaScript methods you can get element/elements using Rufaydium, Every element is Webdriver Class Object has it own methods
Spoiler
"When there is no gravity, there is absolute vacuum and light travel with no time" -Game changer theory

User avatar
Xeo786
Posts: 759
Joined: 09 Nov 2015, 02:43
Location: Karachi, Pakistan

Re: Rufaydium WebDriver 1.7.2 (no selenium/websocket)

Post by Xeo786 » 08 Dec 2022, 01:32

anhnha wrote:
08 Dec 2022, 01:02
No, and I have been saying this for several posts above. I also took pictures to show as well. You can check all images in my previous posts.
try this, I didn't get it until I literally tried it. so I have made some changes.

Code: Select all

#include Rufaydium.ahk 
Chrome := new Rufaydium()
urls := ["https://www.autohotkey.com/boards/viewforum.php?f=4","https://chromedevtools.github.io/devtools-protocol","https://github.com/Xeo786/Rufaydium-Webdriver"]
Page := Chrome.NewSession()
for i, url  in urls
{
	if i = 1
	{	
		Page.url := url
	}
	else
    	Page.CDP.CreateTarget(url)
}	
msgbox, done.....
anhnha wrote:
08 Dec 2022, 01:02

Code: Select all

msgbox, % Chrome1.status()
Do you have any example to show how to use this information to check?
The following msg is proof driver is ready for work
Proof.PNG
Proof.PNG (9.85 KiB) Viewed 2750 times

Code: Select all

#include Rufaydium.ahk 
Chrome1 := new Rufaydium()
msgbox, % Chrome1.status() ; this is the proof driver is running
Chrome1.Driver.Exit()
msgbox, % Chrome1.status() ; this throw error because driver is found running 
"When there is no gravity, there is absolute vacuum and light travel with no time" -Game changer theory

anhnha
Posts: 116
Joined: 08 Aug 2018, 02:46

Re: Rufaydium WebDriver 1.7.2 (no selenium/websocket)

Post by anhnha » 08 Dec 2022, 01:38

Code: Select all

#include Rufaydium.ahk 
Chrome := new Rufaydium()
urls := ["https://www.autohotkey.com/boards/viewforum.php?f=4","https://chromedevtools.github.io/devtools-protocol","https://github.com/Xeo786/Rufaydium-Webdriver"]
Page := Chrome.NewSession()
for i, url  in urls
{
	if i = 1
	{	
		Page.url := url
	}
	else
    	Page.CDP.CreateTarget(url)
}	
msgbox, done.....
Ok, this works great now. That looks simple but I didn't know how. One problem I asked last time. You can use handle to move to any tab you want.
But can you make the first tab always active here while the second and third urls are loading?

Code: Select all

Chrome.Driver.Exit()
xx := % Chrome.Status()
object := JSON.load(xx).value.ready
msgbox, % object
Can we make it return a number for example 0 when it fails instead of just giving a bunch of error message popups?

User avatar
Xeo786
Posts: 759
Joined: 09 Nov 2015, 02:43
Location: Karachi, Pakistan

Re: Rufaydium WebDriver 1.7.2 (no selenium/websocket)

Post by Xeo786 » 08 Dec 2022, 02:05

anhnha wrote:
08 Dec 2022, 01:38

Code: Select all

Chrome.Driver.Exit()
xx := % Chrome.Status()
object := JSON.load(xx).value.ready
msgbox, % object
Can we make it return a number for example 0 when it fails instead of just giving a bunch of error message popups?
Updated Rufaydium!, You can use Chrome1.Ready

Code: Select all

Chrome1 := new Rufaydium()
msgbox, % "OS Name : " 	Chrome1.OS.Name 
.    "`n.OS.Arch : " 	Chrome1.OS.Arch
.    "`n.OS.Version : " Chrome1.OS.Version
. 	 "`n.Build.Version : " Chrome1.Build.Version
. 	 "`n.Message : " Chrome1.Message
. 	 "`n.Ready : "	 Chrome1.Ready
"When there is no gravity, there is absolute vacuum and light travel with no time" -Game changer theory

anhnha
Posts: 116
Joined: 08 Aug 2018, 02:46

Re: Rufaydium WebDriver 1.7.2 (no selenium/websocket)

Post by anhnha » 08 Dec 2022, 05:22

It's better now.
Is there any way to to automatically close chrome driver whenever user press X button on chrome (the button near minimize/maximize)?
I tried If (!Page.Detail() && Chrome.ready) with SetTimer but this makes the program can't even start.

Code: Select all

#SingleInstance FORCE
SetBatchLines, -1
#Include Rufaydium.ahk
SetTimer, closeChrome, 500
return

Chrome := new Rufaydium()
urls := ["https://www.autohotkey.com/boards/viewforum.php?f=4","https://chromedevtools.github.io/devtools-protocol","https://github.com/Xeo786/Rufaydium-Webdriver"]
Page := Chrome.NewSession()
for i, url  in urls
{
	if i = 1
	{
		Page.url := url
	}
	else
    	Page.CDP.CreateTarget(url)
}

^q:
;do something with javascript here


closeChrome:
If (!Page.Detail() && Chrome.ready)
{

 Chrome.QuitAllSessions() ; closing all session one by one
 Chrome.driver.exit() ; exitting driver

}

return

User avatar
Xeo786
Posts: 759
Joined: 09 Nov 2015, 02:43
Location: Karachi, Pakistan

Re: Rufaydium WebDriver 1.7.2 (no selenium/websocket)

Post by Xeo786 » 08 Dec 2022, 05:27

anhnha wrote:
08 Dec 2022, 05:22
It's better now.
Is there any way to to automatically close chrome driver whenever user press X button on chrome (the button near minimize/maximize)?
I tried If (!Page.Detail() && Chrome.ready) with SetTimer but this makes the program can't even start.

Code: Select all

#SingleInstance FORCE
SetBatchLines, -1
#Include Rufaydium.ahk
SetTimer, closeChrome, 500
return

Chrome := new Rufaydium()
urls := ["https://www.autohotkey.com/boards/viewforum.php?f=4","https://chromedevtools.github.io/devtools-protocol","https://github.com/Xeo786/Rufaydium-Webdriver"]
Page := Chrome.NewSession()
for i, url  in urls
{
	if i = 1
	{
		Page.url := url
	}
	else
    	Page.CDP.CreateTarget(url)
}

^q:
;do something with javascript here


closeChrome:
If (!Page.Detail() && Chrome.ready)
{

 Chrome.QuitAllSessions() ; closing all session one by one
 Chrome.driver.exit() ; exitting driver

}

return
well, that would need a WebSocket client connection only with that we can track what is going on with sessions, I am working on it.
W3C is working on Webdriver Bidi (a Websocket based webdriver), why?
Spoiler
Last edited by Xeo786 on 08 Dec 2022, 05:52, edited 3 times in total.
"When there is no gravity, there is absolute vacuum and light travel with no time" -Game changer theory

anhnha
Posts: 116
Joined: 08 Aug 2018, 02:46

Re: Rufaydium WebDriver 1.7.2 (no selenium/websocket)

Post by anhnha » 08 Dec 2022, 05:29

Thanks, then I'll just make a hotkey to close it for now.

User avatar
Xeo786
Posts: 759
Joined: 09 Nov 2015, 02:43
Location: Karachi, Pakistan

Re: Rufaydium WebDriver 1.7.2 (no selenium/websocket)

Post by Xeo786 » 08 Dec 2022, 07:06

anhnha wrote:
08 Dec 2022, 05:29
Thanks, then I'll just make a hotkey to close it for now.
What I have just achieved so far. not implemented to Rufaydium yet,
This message appeared after closing tab manually. if pressed yes it will close all sessions then exit webdriver and exitapp in the end
Capture.PNG
Capture.PNG (70.29 KiB) Viewed 2622 times
and you can use this code separately, you will need this wonderful Websocket cleint lib by Shadownrun to make it work.
Spoiler
"When there is no gravity, there is absolute vacuum and light travel with no time" -Game changer theory

anhnha
Posts: 116
Joined: 08 Aug 2018, 02:46

Re: Rufaydium WebDriver 1.7.2 (no selenium/websocket)

Post by anhnha » 08 Dec 2022, 09:07

Nice, the popup is exactly what I was thinking.

mora145
Posts: 57
Joined: 25 Jun 2022, 15:31

Re: Rufaydium WebDriver 1.7.2 (no selenium/websocket)

Post by mora145 » 08 Dec 2022, 09:50

Xeo786 wrote:
08 Dec 2022, 01:07
mora145 wrote:
07 Dec 2022, 18:26
Xeo786 wrote:
07 Dec 2022, 02:25
mora145 wrote:
07 Dec 2022, 01:14
I am trying to do some scrolling in this window. I can't get it to scroll correctly but it's the screen behind me that moves. How can I do it?
Screenshot_81.jpg
Get all elements having innertext unfollow, now you can use element[x].focus it will automatically scroll to the element and focus that element
if you use New Scroll() it will scroll the stuff according to the mouse pointer location, therefore you also need to use New Mouse() to place the pointer to such a location so it gets the desired scrollbar act upon.
Thanks for your answer. I don't quite understand what element[x].focus should be used for, if New Mouse is going to be used to then click. New Mouse will click on the focus element? But you are supposed to be suggesting to use innertext in unfollow.
Just like JavaScript methods you can get element/elements using Rufaydium, Every element is Webdriver Class Object has it own methods
Spoiler
it will need scroll. The windows is a list of elements and the scroll load the news ones buttons and that buttons need be clicked. In the code, first I get all the 'Follow' buttons and I place them in a "followButton" after that I make the clicks with for k, v in followButton but then there must be a scroll, of about 2 or 3 seconds, that will make the list go down and load new buttons in followButton. The idea is to make the scroll using the driver, and not the mouse pointer.

Code: Select all

Chrome := new Rufaydium()
WheelEvent := new Scroll() 
Page := Chrome.capabilities.setUserProfile("Academia")
Page := Chrome.NewSession()

Page.Navigate("https://oxford.academia.edu/JohannesZachhuber")

;Click in Followers
texto := Page.getElementsbyXpath("//*[@id='content']/div[3]/div/div/div[2]/div[3]/a[2]")[0]
texto.Click()
Sleep, 3000
texto := Page.getElementsbyXpath("//*[@id='content']/div[3]/div/div/div[2]/div[3]/a[2]")

followButton := Page.getElementsbyXpath("//*[text()='Follow']")

Suma:=0
Loop, 10 {

	for k, v in followButton
	{
	followButton[k].Click()
	S := k
	}
	;HERE IS NEED THE SCROLL
	Sleep, 1000
	followButton := Page.getElementsbyXpath("//*[text()='Follow']")
}
Return

anhnha
Posts: 116
Joined: 08 Aug 2018, 02:46

Re: Rufaydium WebDriver 1.7.2 (no selenium/websocket)

Post by anhnha » 08 Dec 2022, 22:29

Code: Select all

#SingleInstance FORCE
SetBatchLines, -1
#Include Rufaydium.ahk

Chrome := new Rufaydium()
Page := Chrome.NewSession()

^1::
; the urls doesn't need to be fixed. They could change by input selection from user as well
urls := ["https://www.autohotkey.com/boards/viewforum.php?f=4","https://chromedevtools.github.io/devtools-protocol","https://github.com/Xeo786/Rufaydium-Webdriver"]

for i, url  in urls
{  
	if i = 1
	{
		Page.url := url
	}
	else
    	Page.CDP.CreateTarget(url)
}

; do some java script here

return
Is there some trick to make these urls always in the first three tabs (in the example, the urls are fixed but they could be changed every time hotkey is pressed as well).
In the code above, if you press the hotkey Ctrl + 1 for the first time, these urls are on three tabs 1-3 from left.
However, from second time, if you not close chrome and continue to press Ctrl + 1 it will open 3 more tabs right to the right of the previous tabs.
Is there any way to force them always on the third 3 tabs?
Session.close() or SwitchTab() or Chrome.QuitAllSessions() would probably works but they're slow.

User avatar
Xeo786
Posts: 759
Joined: 09 Nov 2015, 02:43
Location: Karachi, Pakistan

Re: Rufaydium WebDriver 1.7.2 (no selenium/websocket)

Post by Xeo786 » 09 Dec 2022, 02:30

mora145 wrote:
08 Dec 2022, 09:50
it will need scroll. The windows is a list of elements and the scroll load the news ones buttons and that buttons need be clicked. In the code, first I get all the 'Follow' buttons and I place them in a "followButton" after that I make the clicks with for k, v in followButton but then there must be a scroll, of about 2 or 3 seconds, that will make the list go down and load new buttons in following Button. The idea is to make the scroll using the driver, and not the mouse pointer.
There are a Few ways,
  • You can use Element[x].click(0), click(0) is webdriver click, it will bring that element into view and click
  • You look for event lister in devtools, create that event and dispatch it using javascript Session.ExecuteSync(JS) so it can load expected data
anhnha wrote:
08 Dec 2022, 22:29

Code: Select all

; do some java script here
Use Session.ExecuteSync(JS) there, you can find examples here in previous posts.
anhnha wrote:
08 Dec 2022, 22:29
Is there some trick to make these urls always in the first three tabs
What you have to do is get existing sessions. and look for a specific URL if it is available then create no tab/session just gives you access.
"When there is no gravity, there is absolute vacuum and light travel with no time" -Game changer theory

Dec
Posts: 2
Joined: 09 Dec 2022, 07:17

Re: Rufaydium WebDriver 1.7.2 (no selenium/websocket)

Post by Dec » 09 Dec 2022, 07:34

Hello,

Sorry if this is a bit off-topic but I'm a complete newbie to AHK and to programming in general.

I just wanted to ask if it's possible to retrieve certain fields from a webpage for later use. For more background information, I'm trying to write out a script that will store the information from a couple of fields from one of the webpages I'm currently viewing and I was wondering if Rufaydium would do the trick, I'm also wondering if it needs a new session each time the script is being run or if it can either run in the background to retrieve those values or use the same browser instance after it's been started via the .AHK file. I'm basically using this to automatically unzip some archives that were just downloaded, create new folders according to those two fields, scan the folder containing the newly created folders and have a drop down menu automatically start up Notepad++ Workspaces depending on my drop-down menu choice (everything else is basically out of scope but I just wanted to offer a bit of insight into what I'm trying to do). Thanks in advance :)

User avatar
Xeo786
Posts: 759
Joined: 09 Nov 2015, 02:43
Location: Karachi, Pakistan

Re: Rufaydium WebDriver 1.7.2 (no selenium/websocket)

Post by Xeo786 » 09 Dec 2022, 08:10

Dec wrote:
09 Dec 2022, 07:34
Hello,

Sorry if this is a bit off-topic but I'm a complete newbie to AHK and to programming in general.

I just wanted to ask if it's possible to retrieve certain fields from a webpage for later use. For more background information, I'm trying to write out a script that will store the information from a couple of fields from one of the webpages I'm currently viewing and I was wondering if Rufaydium would do the trick, I'm also wondering if it needs a new session each time the script is being run or if it can either run in the background to retrieve those values or use the same browser instance after it's been started via the .AHK file. I'm basically using this to automatically unzip some archives that were just downloaded, create new folders according to those two fields, scan the folder containing the newly created folders and have a drop-down menu automatically start up Notepad++ Workspaces depending on my drop-down menu choice (everything else is basically out of scope but I just wanted to offer a bit of insight into what I'm trying to do). Thanks in advance :)
Your task seems simple you can use winhttp request, Winhttp will work in background no browser is needed.
look into this URLDownloadtoFile,Webscraping, WinHTTPrequests and more
"When there is no gravity, there is absolute vacuum and light travel with no time" -Game changer theory

Flowgun
Posts: 74
Joined: 25 Aug 2022, 09:42

Re: Rufaydium WebDriver 1.7.2 (no selenium/websocket)

Post by Flowgun » 09 Dec 2022, 08:13

hello everyone. Rufaydium is great!
I'm new to web scraping. can anyone help me access this button that has this "data-elem" thingy? I want to select the button based on the text inside of it (Access Email & Phone Number), but I've been stuck for hours trying all sorts of stuff :/

<div class="zp_2cHnw" data-elem="button-label">Access Email &amp; Phone Number</div>

User avatar
Xeo786
Posts: 759
Joined: 09 Nov 2015, 02:43
Location: Karachi, Pakistan

Re: Rufaydium WebDriver 1.7.2 (no selenium/websocket)

Post by Xeo786 » 09 Dec 2022, 08:24

Flowgun wrote:
09 Dec 2022, 08:13
hello everyone. Rufaydium is great!
I'm new to web scraping. can anyone help me access this button that has this "data-elem" thingy? I want to select the button based on the text inside of it (Access Email & Phone Number), but I've been stuck for hours trying all sorts of stuff :/

<div class="zp_2cHnw" data-elem="button-label">Access Email &amp; Phone Number</div>
try something like this,

Code: Select all

for i, element in page.QuerySelectorAll("div")
{
	if instr(element.innerText,"Phone Number")
		msgbox, % element.outerHTML
}
but there would be so many div would be so many, check if the class name is not unique for every phone number then you can do something like this

Code: Select all

for i, element in page.QuerySelectorAll("div.zp_2cHnw")
{
	if instr(element.innerText,"Phone Number")
		msgbox, % element.outerHTML
}
you can write some JS function that searches data for you at once and pass it using ExecuteSync('return' JS) and this would work lighting fast.
"When there is no gravity, there is absolute vacuum and light travel with no time" -Game changer theory

automater
Posts: 20
Joined: 01 Jul 2022, 08:51

Re: Rufaydium WebDriver 1.7.2 (no selenium/websocket)

Post by automater » 09 Dec 2022, 08:35

Hey @Xeo786, my code works perfectly except when I run headless mode. I've searched this thread and learned that it might have to do with cookies but I'm not sure how to load cookies.

I tried adding this below my capabilities but it didn't work:

Code: Select all

Cookies := Session.GetCookies()
I'm not surprised since that's just a variable assignment but I'm not sure what to do with the Cookies variable after assignment.

How do I activate cookies when using headless mode?

Flowgun
Posts: 74
Joined: 25 Aug 2022, 09:42

Re: Rufaydium WebDriver 1.7.2 (no selenium/websocket)

Post by Flowgun » 09 Dec 2022, 09:08

@Xeo786
Thank you a lot for the quick reply. I will try your suggestions =D

Post Reply

Return to “Scripts and Functions (v1)”