Getting Web Content After JavaScript Loaded Topic is solved

[hasantr]

https://nodle.subscan.io/account/4jbHZrRFMM7gUw2zVjh2S7RBbDeeNraFYsnEkTeCd8VL63h8

Loads the above web page content with java script. So I am not getting all the text on the page. I'm trying to get the "balance-text" data. In this example balance is "1.1163816". With ahk the existing methods are getting the page source directly. Unable to receive generated content. What other method should I try? Thank you.

Note: Solutions with Internet Explorer are not valid.

[AHKStudent]

Try this, I learned this technique from @malcev so you can thank him and the jsontoahk function is by @teadrinker

Code: Select all

HTTP := ComObjCreate("WinHttp.WinHttpRequest.5.1")
HTTP.Open("POST", "https://nodle.webapi.subscan.io/api/v2/scan/search", true)
HTTP.SetRequestHeader("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko")
HTTP.SetRequestHeader("Pragma", "no-cache")
HTTP.SetRequestHeader("Cache-Control", "no-cache, no-store")
HTTP.SetRequestHeader("If-Modified-Since", "Sat, 1 Jan 2000 00:00:00 GMT")
HTTP.SetRequestHeader("Content-Type", "application/json")
json = {"key":"4jbHZrRFMM7gUw2zVjh2S7RBbDeeNraFYsnEkTeCd8VL63h8","row":"1","page":"0"}
HTTP.Send(json)
HTTP.WaitForResponse()
;MsgBox, % HTTP.ResponseText ; uncomment to see everything it returns
obj := JsonToAHK(HTTP.ResponseText)
MsgBox, % obj.data.account.balance
ExitApp

JsonToAHK(json, rec := false) {
   static doc := ComObjCreate("htmlfile")
         , __ := doc.write("<meta http-equiv=""X-UA-Compatible"" content=""IE=9"">")
         , JS := doc.parentWindow
   if !rec
      obj := JsonToAHK(JS.eval("(" . json . ")"), true)
   else if !IsObject(json)
      obj := json
   else if json.toString() != "[object Object]" {
      obj := []
      Loop % json.length
         obj.Push( JsonToAHK(json[A_Index - 1], true) )
   }
   else {
      obj := {}
      keys := JS.Object.keys(json)
      Loop % keys.length {
         k := keys[A_Index - 1]
         obj[k] := JsonToAHK(json[k], true)
      }
   }
   Return obj
}

Here is a regex option

Code: Select all

HTTP := ComObjCreate("WinHttp.WinHttpRequest.5.1")
HTTP.Open("POST", "https://nodle.webapi.subscan.io/api/v2/scan/search", true)
HTTP.SetRequestHeader("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko")
HTTP.SetRequestHeader("Pragma", "no-cache")
HTTP.SetRequestHeader("Cache-Control", "no-cache, no-store")
HTTP.SetRequestHeader("If-Modified-Since", "Sat, 1 Jan 2000 00:00:00 GMT")
HTTP.SetRequestHeader("Content-Type", "application/json")
json = {"key":"4jbHZrRFMM7gUw2zVjh2S7RBbDeeNraFYsnEkTeCd8VL63h8","row":"1","page":"0"}
HTTP.Send(json)
HTTP.WaitForResponse()
;MsgBox, % HTTP.ResponseText ; uncomment to see everything it returns
RegExMatch(HTTP.ResponseText, "balance"":""(.*?)""",ba)
MsgBox, % ba1
ExitApp

[hasantr]

Try this, I learned this technique from @malcev so you can thank him and the jsontoahk function is by @teadrinker

Code: Select all

HTTP := ComObjCreate("WinHttp.WinHttpRequest.5.1")
HTTP.Open("POST", "https://nodle.webapi.subscan.io/api/v2/scan/search", true)
HTTP.SetRequestHeader("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko")
HTTP.SetRequestHeader("Pragma", "no-cache")
HTTP.SetRequestHeader("Cache-Control", "no-cache, no-store")
HTTP.SetRequestHeader("If-Modified-Since", "Sat, 1 Jan 2000 00:00:00 GMT")
HTTP.SetRequestHeader("Content-Type", "application/json")
json = {"key":"4jbHZrRFMM7gUw2zVjh2S7RBbDeeNraFYsnEkTeCd8VL63h8","row":"1","page":"0"}
HTTP.Send(json)
HTTP.WaitForResponse()
;MsgBox, % HTTP.ResponseText ; uncomment to see everything it returns
obj := JsonToAHK(HTTP.ResponseText)
MsgBox, % obj.data.account.balance
ExitApp

JsonToAHK(json, rec := false) {
   static doc := ComObjCreate("htmlfile")
         , __ := doc.write("<meta http-equiv=""X-UA-Compatible"" content=""IE=9"">")
         , JS := doc.parentWindow
   if !rec
      obj := JsonToAHK(JS.eval("(" . json . ")"), true)
   else if !IsObject(json)
      obj := json
   else if json.toString() != "[object Object]" {
      obj := []
      Loop % json.length
         obj.Push( JsonToAHK(json[A_Index - 1], true) )
   }
   else {
      obj := {}
      keys := JS.Object.keys(json)
      Loop % keys.length {
         k := keys[A_Index - 1]
         obj[k] := JsonToAHK(json[k], true)
      }
   }
   Return obj
}

Here is a regex option

Code: Select all

HTTP := ComObjCreate("WinHttp.WinHttpRequest.5.1")
HTTP.Open("POST", "https://nodle.webapi.subscan.io/api/v2/scan/search", true)
HTTP.SetRequestHeader("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko")
HTTP.SetRequestHeader("Pragma", "no-cache")
HTTP.SetRequestHeader("Cache-Control", "no-cache, no-store")
HTTP.SetRequestHeader("If-Modified-Since", "Sat, 1 Jan 2000 00:00:00 GMT")
HTTP.SetRequestHeader("Content-Type", "application/json")
json = {"key":"4jbHZrRFMM7gUw2zVjh2S7RBbDeeNraFYsnEkTeCd8VL63h8","row":"1","page":"0"}
HTTP.Send(json)
HTTP.WaitForResponse()
;MsgBox, % HTTP.ResponseText ; uncomment to see everything it returns
RegExMatch(HTTP.ResponseText, "balance"":""(.*?)""",ba)
MsgBox, % ba1
ExitApp

Dude, what you're doing is amazing.
I certainly did not expect such a good solution. I had no knowledge of getting information from the web. This has been very gratifying for me.
Thank you so much.