| View previous topic :: View next topic |
| Author |
Message |
WhiteCloud
Joined: 19 Jun 2004 Posts: 68
|
Posted: Sun Jul 04, 2004 1:02 am Post subject: URLDownloadToFile to not hold up the computer |
|
|
while it waits for the html to come, the system won't respond when i try to open textpad or internet explorer or any program
the code i use isn't anything exotic or else i'd paste it here
so while all the urls are downloading i can't do much on the computer _________________ AHK = Hella fun |
|
| Back to top |
|
 |
Chris Site Admin
Joined: 02 Mar 2004 Posts: 10467
|
Posted: Sun Jul 04, 2004 2:03 am Post subject: |
|
|
This might be explained if the download speed is very slow and the keyboard or mouse hook is installed. To work around this, don't download from a script that has one of the hooks installed. Call a separate script.
Has anyone else noticed this? If so, perhaps the program can be changed to use a dedicated thread for the download. |
|
| Back to top |
|
 |
WhiteCloud
Joined: 19 Jun 2004 Posts: 68
|
Posted: Sun Jul 04, 2004 5:28 pm Post subject: |
|
|
the site takes about 20 seconds to load. It's not getting hardly any data, it's just takes forever to respond. (that's not ahk's fault at all)
there aren't any hooks installed. I only have 3 keys assigned and they are basic (F1, F2, F3). When I "view hotkeys and their methods" it says they are of type "reg" not "hook".
i can work with an already open application just fine but if i want to refresh a directory or run textpad or another application it waits until the page responds. _________________ AHK = Hella fun |
|
| Back to top |
|
 |
Chris Site Admin
Joined: 02 Mar 2004 Posts: 10467
|
Posted: Sun Jul 04, 2004 10:54 pm Post subject: |
|
|
If you're sure the hook isn't installed, this appears to be a quirk of Windows. Are you using Windows 9x? If so, it's possible that this is purely an issue related to that OS and not to XP/2k/NT.
The reason I blame it on the OS is that the only thing URLDownloadToFile does is call standard Windows API functions to download a file from the Internet. It doesn't do anything that should cause such widespread problems with the system (except the hooks, but you said the script doesn't use them). You can double check the hook status at the top of the "View > Key history" menu item in the main window. |
|
| Back to top |
|
 |
WhiteCloud
Joined: 19 Jun 2004 Posts: 68
|
Posted: Mon Jul 05, 2004 12:22 am Post subject: |
|
|
I'm on windows 2000 pro with all the updates installed
there aren't any keyboard hooks installed since all it is is some parsing loops, file appends, 3 basic keys and the download url command
i'll try to test and post some simplified/cleaned up code if i get some free time. _________________ AHK = Hella fun |
|
| Back to top |
|
 |
WhiteCloud
Joined: 19 Jun 2004 Posts: 68
|
Posted: Mon Jul 19, 2004 1:45 am Post subject: |
|
|
alright i got the code whittled down to about a screensworth
you can't do much while it's downloading a page. Plus if you're in the middle of highlighting some text, the highlighting will stop prematurely when a page gets done downloading. Maybe that will tip you off to what in the world is going on with this command.
btw: These pages go a bit quicker than the 20-second ones i mentioned before but i still do those with a different version of this script.
You'll need to put a text file in the same directory as this script with this in it:
A01820153
A01820132
A01820131
A01820109
A01820108
A01820052
A01820084
A01820022
A01819991
A01819990
A01819989
A01819978
A01819970
A01819934
A01819925
A01819912
A01819913
A01819905
A01819904
A01819903
A01819902
A01819901
A01819900
A01819899
A01819898
A01819897
A01819896
A01819895
A01819894
A01819893
A01819892
A01819891
A01819890
A01819889
A01819888
A01819887
A01819886
A01819791
A01819753
A01819752
A01819745
A01819742
A01819710
A01819658
A01819657
A01819605
A01819604
A01819603
A01819571
A01819570
A01819558
A01819530
A01819523
A01819513
A01819441
A01819440
A01819433
A01819404
A01819381
A01819379
A01819372
A01819371
A01819364
A01819365
A01819369
A01819286
A01819285
| Code: |
F2::
setbatchlines 30ms
sleep 100
loop read, straight_doc_numbers.txt
{
number_of_docs++
If line_to_start_on Is Space
{
InputBox, line_to_start_on, line_to_start_on, line_to_start_on
}
if number_of_docs < %line_to_start_on%
{
Continue
}
ifnotexist line_%number_of_docs%_straight.html
{
URLDownloadToFile, https://www.stclaircountyrecorder.com/cgi-bin/recorder.cgi?mode=view&document=%A_Loopreadline%, line_%number_of_docs%_straight.html
}
doc_number_on_line_%number_of_docs% = %a_loopreadline%
results_variable_1 =
loop read, line_%number_of_docs%_straight.html
{
results_variable_1 = %results_variable_1%%A_loopreadline%`n
}
StringGetPos, this_doc_number_position, results_variable_1, References%A_Space%A01
this_doc_number_position += 15
StringMid this_doc_number, results_variable_1, %this_doc_number_position%, 6
If this_doc_number Is Not Digit
{
fileappend %number_of_docs%%A_Tab%%a_loopreadline%--->No references.`n, legals_for_straight_doc_numbers.txt
}
else
{
URLDownloadToFile, https://www.stclaircountyrecorder.com/cgi-bin/recorder.cgi?mode=view&document=A01%this_doc_number%, line_%number_of_docs%_reference.html
ifexist line_%number_of_docs%_straight.html
{
;now to get the legal into a file
ifexist line_%number_of_docs%_reference.html
{
results_variable_2 = ;it's called two just because it's the 2nd webpage parsed for this original query of a parcel number
Loop, Read, line_%number_of_docs%_reference.html ;this loop puts the whole webpage into one searchable variable
{
results_variable_2 = %results_variable_2%%A_loopreadline%`n
}
StringGetPos, legal_start_position, results_variable_2, &tab=legal"%A_Space%class="textlink"> ;Position 0 is the first character.
left_occurance = 1
loop 100 ;this loop is to find the end position of the legal description
{
StringGetPos, legal_end_position, results_variable_2, </a></td>, L%left_occurance% ;Position 0 is the first character.
if legal_end_position < %legal_start_position%
left_occurance++
else
break
}
legal_start_position += 30
characters_to_extract = %legal_end_position%
characters_to_extract -= %legal_start_position%
characters_to_extract += 1
stringmid legal, results_variable_2, %legal_start_position%, %characters_to_extract%
}
else
{
legal = line_%number_of_docs%_reference.html was not found.
}
file_line = %number_of_docs%%A_Tab%%a_loopreadline%--->A01%this_doc_number%%A_Tab%%legal%
fileappend %file_line%`n, legals_for_straight_doc_numbers.txt
}
else
{
fileappend line_%number_of_docs%_1_returned.html does not exist.`n, legals_for_straight_doc_numbers.txt
}
}
}
msgbox Finished. (%number_of_docs% Lines)
Return |
_________________ AHK = Hella fun |
|
| Back to top |
|
 |
WhiteCloud
Joined: 19 Jun 2004 Posts: 68
|
Posted: Mon Jul 19, 2004 2:06 am Post subject: |
|
|
whoa, i forgot to mention that if i make a script with only the urldownload command in it activated by a keypress it works fine. No hold-ups at all. _________________ AHK = Hella fun |
|
| Back to top |
|
 |
Chris Site Admin
Joined: 02 Mar 2004 Posts: 10467
|
Posted: Mon Jul 19, 2004 2:40 am Post subject: |
|
|
Thanks for posting the script. I ran it exactly as-is on my XP system. I received 124 files over the course of about 3 minutes I would say. During that time, launching programs was a little slower, but I didn't have any problem holding the mouse button down to select text (even when holding the button down for a long time and changing the selection area). The script's main window was a little sluggish during that time, I suspect because the site sending the data is a little slow so while AHK waits for an 8K block, it's thread is stuck until the block is finished.
So I think this situation is probably caused by one or more of the following: 1) The site responds even slower for you than it does for me; 2) Win2k's HTTP downloading smoothness is inferior to WinXP's (which might not be true); 3) other differences in network or hardware performance between our two systems.
You might try using #InstallKeybdHook at the top of the script because as a side-effect, the block size is reduced to 1K vs. 8K when a hook is installed. This is done to reduce keyboard lag during the download. However, the hook might make it worse rather than better. Even if it does help, it might not work in future versions because I will probably increase the block size back up to 8K in a future version if I can get a dedicated thread working for the keyboard/mouse hooks. The expectation is that 8K blocks perform better than 1K blocks, but that it purely a guess and it might depend on the nature of the connection. |
|
| Back to top |
|
 |
WhiteCloud
Joined: 19 Jun 2004 Posts: 68
|
Posted: Mon Jul 19, 2004 5:03 am Post subject: |
|
|
I appreciate it a lot that you went thru and took the whole thing seriously and wrote up a useful reply as always ...despite the fact that no one else has posted about encountering this problem.
I thought for sure you'd still have your text selection prematurely stopped. With or without #InstallKeybdHook, it still disengages my mouse from the text selection even though I've still got the button down.
Once difference I notice with the #InstallKeybdHook in the script is a very noticeible sluggishness while typing this message (and running the script). Without #InstallKeybdHook my computer runs perfectly smoothly but it hangs when trying certain things.
My cpu is an athlon 2100. My internet connection is Charter.net cable in St. Louis. 1mb downstream. Your download rate is the same as mine is.
Anybody else care to try out the script? Test it by holding your mouse cursor down in a text document and moving it in a circle the whole time.
With my other scripts that do roughly the same thing as this one but to different webpages, some of them take up to half an hour to download a single page. (the website is at fault not ahk) During that time, the only way to do things like refresh a directory in explorer, open a new program, maximize ahk or interrupt the url download is to use the task manager to end the autohotkey process.
Thanks _________________ AHK = Hella fun |
|
| Back to top |
|
 |
Chris Site Admin
Joined: 02 Mar 2004 Posts: 10467
|
Posted: Mon Jul 19, 2004 8:26 pm Post subject: |
|
|
I made a change to the Installer that might alleviate this. Please download and give it a try so that I can determine whether the change is worth keeping. http://www.autohotkey.com/download/
Thanks. |
|
| Back to top |
|
 |
WhiteCloud
Joined: 19 Jun 2004 Posts: 68
|
Posted: Tue Jul 20, 2004 6:08 am Post subject: |
|
|
I tried it. I couldn't detect any differences. For all we know, i could have some adware interfering or something. I'll get by. Since no one else has this problem, i'd rather we move on. _________________ AHK = Hella fun |
|
| Back to top |
|
 |
Chris Site Admin
Joined: 02 Mar 2004 Posts: 10467
|
Posted: Tue Jul 20, 2004 12:54 pm Post subject: |
|
|
I read quite a lot about how Internet downloading works. It appears there is an asynchronous mode that is sometimes recommended but somewhat difficult to implement, and there are also concerns about cross-platform support and reliability with that method. In addition, since I had considerable doubt that it would fix the issue you're experiencing, I haven't implemented it yet.
However, since URLDownloadToFile uses an API function that comes with MSIE: if you're using an old version of MSIE, you might try upgrading to the latest.
If that doesn't help, I might like to e-mail you a test version that contains a Sleep(10) inside the download loop. Although this might reduce the download speed (or maybe not if OS "reads ahead"), it would be interesting to see if that solves it. If it does, there is a good chance that the async method above would solve it too.
Finally, as an alternative to downloading the file directly, perhaps it would be better to use the web browser and check the status bar (as mentioned in the FAQ) to detect when the download is finished. At that time, you could use "Save As". |
|
| Back to top |
|
 |
|