15 replies to this topic

[Bruttosozialprodukt]

I just realized that the WinHttpRequest object has built in cookie handling. And since so many people had trouble doing logins using the WinHttpRequest COM and we always thought that this would be because of the cookies, I'm now giving you two working, very well explained login examples, as well as some tips on how to get started reverseengineering HTML / HTTP-Requests.

If you want to do complicated things like logins, then you should really learn some HTML and the basics about the HTTP protocol. Fiddler and SetProxy(2,"localhost:8888") will help you A LOT with the debugging. I also recommend using an add on for your browser to quickly clean your cookies. 

To reverse engineer the AHK forum login I simply analyzed the browsers HTTP requests to autohotkey.com and by some trial and error I was able to minimize it to the basics. We need exactly two requests and the login needs one request header and 3 POST data parameters.

So let's do this login to the AHK forums. 
Step 1. Do a simple GET request on http://www.autohotke...l&section=login
Step 2. Extract the auth_key parameter form the login form from the response body (ResponseText)
Step 3. Create the POST data string containing the auth_key parameter as well as the username, password and rememberMe parameter for the login
Step 4. Set the Content-Type header for the next request
Step 5. Send  the POST data string to http://www.autohotke...ogin&do=process
Step 6. Analyze the response body checking if the HTML documents title starts with the words "Sign In". If so, then you're obviously not signed in (the login failed/wrong login data). If the title is different, then the login was successfull.

;Prepare our WinHttpRequest object
HttpObj := ComObjCreate("WinHttp.WinHttpRequest.5.1")
;HttpObj.SetProxy(2,"localhost:8888") ;Send data through Fiddler
HttpObj.SetTimeouts(6000,6000,6000,6000) ;Set timeouts to 6 seconds
;HttpObj.Option(6) := False ;disable location-header rediects

;Set our URLs
loginSiteURL := "http://www.autohotkey.com/board/index.php?app=core&module=global&section=login"
loginURL := "http://www.autohotkey.com/board/index.php?app=core&module=global&section=login&do=process"

;Set our login data
username := "Brutosozialprodukt"
password := "xxxxxxxxxxxxxx"
rememberMe := "1"

;Step 1
HttpObj.Open("GET",loginSiteURL)
HttpObj.Send()

;Step 2
RegExMatch(HttpObj.ResponseText,"<input\stype='hidden'\sname='auth_key'\svalue='(\w+)'\s/>",match)
auth_key := match1

;Step 3
loginBody := "auth_key=" auth_key "&ips_username=" username "&ips_password=" password "&rememberMe=" rememberMe

;Step 4/5
HttpObj.Open("POST",loginURL)
HttpObj.SetRequestHeader("Content-Type","application/x-www-form-urlencoded")
HttpObj.Send(loginBody)

;Step 6
If (InStr(HttpObj.ResponseText,"<title>Sign In"))
    MsgBox, The login failed!
Else
    MsgBox, Login was successfull!

This will probably work for most IPB forums if change the URLs properly. For other sites this will be probably look very different.

But okay, let's do another login to the new/other AHK forum (this will be much easier).
Step 1. Create the POST data containing username, password and the autologin parameter
Step 2. Set the Content-Type header
Step 3. Send the POST data to http://ahkscript.org....php?mode=login
Step 4. Analyze the response body checking if the HTML documents title starts with the word "Login". If so, then you're obviously not logged in yet (the login failed/wrong login data). If the title is different, then the login was successfull.

;Prepare our WinHttpRequest object
HttpObj := ComObjCreate("WinHttp.WinHttpRequest.5.1")
;HttpObj.SetProxy(2,"localhost:8888") ;Send data through Fiddler
HttpObj.SetTimeouts(6000,6000,6000,6000) ;Set timeouts to 6 seconds
;HttpObj.Option(6) := False ;disable location-header rediects

;Set our URLs
loginURL := "http://ahkscript.org/boards/ucp.php?mode=login"

;Set our login data
username := "Brutosozialprodukt"
password := "xxxxxxxxxxxxxx"
autologin := "on"

;Step 1
loginBody := "username=" username "&password=" password "&autologin=" autologin "&login=Login"

;Step 2/3
HttpObj.Open("POST",loginURL)
HttpObj.SetRequestHeader("Content-Type","application/x-www-form-urlencoded")
HttpObj.Send(loginBody)

;Step 4
If (InStr(HttpObj.ResponseText,"<title>Login"))
    MsgBox, The login failed!
Else
    MsgBox, Login was successfull!

Any questions? I will try to answer them the next time I'm here.

You may also read the existing answers in this thread or the one on the other forum.

[script_me]

hey ..

 

 

this is a great post .

 

Now I am able to do logins .

 

note : As per the website you will need to post different data in different format .This can be done pretty well by fiddler as told by Brutosozialprodukt.

 

Supposing the site runs on iis server , you will need to add a request header of type "content length " . this can be checked in fiddler using breakpoints .

 

Also I want to ask ...

 

Suppose I want to run many instances of winhhttprequest (it handles session independently )  .  But some request can take long time to complete .

 

So there is any asynchronous method which will run the request but will not wait for response . Maybe like a worker thread . Or i will have to run multiple script at once 

 

each containing one instance of winhttprequest .

 

thnx ur examples are nice and very simple to understand. 

[Bruttosozialprodukt]

I'm not 100% sure, but I think WinHttpRequest actually generates the Content-Length header automatically. (tank told me this, yesterday)

You can only do one request at a time for each created WinHttpRequest object and yes you can make them work asynchronously.

HttpObj1 := ComObjCreate("WinHttp.WinHttpRequest.5.1")
HttpObj2 := ComObjCreate("WinHttp.WinHttpRequest.5.1")

HttpObj1.Open("GET","http://google.com",true) ;the third parameter means, that it will be working asynchronously
HttpObj1.Send()
HttpObj2.Open("GET","http://microsoft.com",true)
HttpObj2.Send()

I'm not sure if there is a good way of checking if the request has finished...
You could try something like this:

Loop {
    If (HttpObj1.Status) { ;The first request returned a status code...
        MsgBox % HttpObj1.ResponseText
    }
    If (HttpObj2.Status) { ;The second request returned a status code...
        MsgBox % HttpObj2.ResponseText
    }
}

[script_me]

I think this is not a good way to check for response . The correct method can be seen here  

http://msdn.microsoft.com/en-us/library/aa384071(VS.85).aspx

We should do this after HttpObj1.Send() like 

HttpObj1.Send()
HttpObj1.WaitForResponse();

It confirms that the whole response ir received from the server before using it. Otherwise it will result in a error saying " The data necessary  to complete this operation

 

is not yet available " as with your example above to check for response .

 

Thnx

[Bruttosozialprodukt]

If you use HttpObj1.WaitForResponse() then the script will wait until the first request has finished, so you can't work on HttpObj2 which might finish much earlier. 
Asynch requests + WaitForResponse() are in my opinion only necessary if you don't want your GUI to freeze during the request.

 

For a cleaner solution, you could build your own callback function using timers:

HttpObj1 := ComObjCreate("WinHttp.WinHttpRequest.5.1")
HttpObj2 := ComObjCreate("WinHttp.WinHttpRequest.5.1")

HttpObj1.Open("GET","http://google.com",true) ;the third parameter means, that it will be working asynchronously
HttpObj1.Send()
SetTimer, HttpObj1AsynchHandler, -1
HttpObj2.Open("GET","http://microsoft.com",true)
HttpObj2.Send()
SetTimer, HttpObj1AsynchHandler, -1
Return

OnHttpResponse(HttpObj) {
    MsgBox % HttpObj.ResponseText
}

HttpObj1AsynchHandler:
    HttpObj1.WaitForResponse()
    OnHttpResponse(HttpObj1)
Return
HttpObj2AsynchHandler:
    HttpObj2.WaitForResponse()
    OnHttpResponse(HttpObj2)
Return

With SetTimerF() you could make this even cleaner because you would only need one timer.

[script_me]

ya this is better approach . 

 

I can see that httpwebrequest bypasses the fiddler . Is there any way to set up ahk script or fiddler so that 

 

fiddler captures the request made by the script .It would be very nice for debugging errors . 

[Bruttosozialprodukt]

I can see that httpwebrequest bypasses the fiddler . Is there any way to set up ahk script or fiddler so that 

 

fiddler captures the request made by the script .It would be very nice for debugging errors . 

Yes, this is indeed possible. Look at the examples form my first post, the line is actually in there, it is just commented out:

HttpObj.SetProxy(2,"localhost:8888") ;Send data through Fiddler

(You have to set the proxy before doing any requests, afaik.)

[script_me]

yeah its there ,,,  I have a confusion ..

 

Suppose we have a form which I want to the server . But I don,t want to recieve the response because I dont need it . Plus it will also increase 

 

the speed to send the request  . And save some broadband data . Is there any way to do this . Or if it is not possible then how can I minimize the response 

 

to block images and js . I think it is possible through request header . But rejecting the whole response will be good (just knowing that data has been received by

 

server by checking the status of response header but rejecting the content of the body ).  

 

Any way it would be helpful .

 

thnx  

[Bruttosozialprodukt]

You could try to do a "HEAD" request. This way the server will only give you the response-line and the headers as answer, but not the body (ResponseText). 
But I'm not sure if that's going to work since forms are usually sent using the "POST" method, so the server might get confused... 

You could probably try to do an asynchronous request and cancel it immediately. Though this might cancel both, the request and the response-receiving if done too quickly. 

HttpObj := ComObjCreate("WinHttp.WinHttpRequest.5.1")
HttpObj.Open("POST",myUrl,True)
HttpObj.Send(postData)
HttpObj.Abort()

I guess you could wait until the response line has been received and then cancel it:

HttpObj := ComObjCreate("WinHttp.WinHttpRequest.5.1")
HttpObj.Open("POST",myUrl,True)
HttpObj.Send(postData)
Loop {
    If (HttpObj.status)
        Break
}
HttpObj.Abort()

Another good idea might be to set the range header and ask for 0 or maybe 1 byte:

HttpObj := ComObjCreate("WinHttp.WinHttpRequest.5.1")
HttpObj.Open("POST",myUrl)
HttpObj.SetRequestHeader("Range","bytes=0-1")
HttpObj.Send(postData)

[Nachoman21]

This is very interesting, but one small question. How would one set the useragent that is being used via all this?

[Bruttosozialprodukt]

The user-agent is a request header. It can look like this:

myUserAgent := "Mozilla/5.0 (Windows NT 6.2; WOW64; rv:31.0) Gecko/20100101 Firefox/31.0"
HttpObj.SetRequestHeader("User-Agent",myUserAgent)

You can define headers between the HttpObj.Open() and the HttpObj.Send() Method.

So this would request the google.com site using a Firefox user-agent:

HttpObj := ComObjCreate("WinHttp.WinHttpRequest.5.1")
HttpObj.SetTimeouts(6000,6000,6000,6000) ;Set timeouts to 6 seconds

HttpObj.Open("GET","https://google.com/")
HttpObj.SetRequestHeader("User-Agent","Mozilla/5.0 (Windows NT 6.2; WOW64; rv:31.0) Gecko/20100101 Firefox/31.0")
HttpObj.Send()

[Edgard]

winhttprequest proxy When to set it? And Why? Is it changing my IP?

[Edgard]

HttpObj.SetProxy(2,"localhost:8888") ;Send data through Fiddler When to set it? And Why? Is it changing my IP?

[Bruttosozialprodukt]

No, this won't change your IP and you certainly don't have to set it.

If you want to do a request with a different IP, then you have to find an anonymous or elite proxy.

For example from here: http://www.samair.ru/proxy/ (just google for proxylist)

The "localhost:8888" is only valid if you have the HTTP-Debugging tool called "Fiddler" installed. "localhost:8888" then acts as a transparent proxy to log your requests. This is useful if you want to see if your requests are actually sent or not.

 

The proxy can be set anywhere between the ComObjCreate() function and the HttpObj.Open() method. For example:

HttpObj := ComObjCreate("WinHttp.WinHttpRequest.5.1")
HttpObj.SetTimeouts(6000,6000,6000,6000) ;Set timeouts to 6 seconds
HttpObj.SetProxy(2,"64.31.22.131:3127") ;Set 64.31.22.131:3127 as our proxy

HttpObj.Open("GET","https://google.com/")
HttpObj.SetRequestHeader("User-Agent","Mozilla/5.0 (Windows NT 6.2; WOW64; rv:31.0) Gecko/20100101 Firefox/31.0")
HttpObj.Send()

[Johnny R]

Can you help me writing a script to expand a short url with the goo.gl-api?

 

I have posted in the german forum, but there is no help. There you can find a script, which shortens a long url.#

 

EDIT:

Solved! http://goo.gl/vLbrcT