Something with the download page changed recently?

TheArkive · 12 Aug 2023, 13:18

I have my own script to parse the ahk download pages (1.0, 1.1, and 2.0), and as of today it gets a "Just a moment..." page with what looks like a bunch of Base64 text.

Is parsing the download page more complex than normal?

I tried just using the normal download command and here's what I get:

Code: Select all

<!DOCTYPE html><html lang="en-US"><head><title>Just a moment...</title><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta http-equiv="X-UA-Compatible" content="IE=Edge"><meta name="robots" content="noindex,nofollow"><meta name="viewport" content="width=device-width,initial-scale=1"><link href="/cdn-cgi/styles/challenges.css" rel="stylesheet"></head><body class="no-js"><div class="main-wrapper" role="main"><div class="main-content"><noscript><div id="challenge-error-title"><div class="h2"><span class="icon-wrapper"><div class="heading-icon warning-icon"></div></span><span id="challenge-error-text">Enable JavaScript and cookies to continue</span></div></div></noscript></div></div><script>(function(){window._cf_chl_opt={cvId: '2',cZone: 'www.autohotkey.com',cType: 'managed',cNounce: '20453',cRay: '7f5a9e813a432c7e',cHash: '250e6aeb5fd1c16',cUPMDTk: "\/download\/1.1?__cf_chl_tk=aI4FZvcw.k3BvuyP9D0my2O91OcNdloVC5jCVhju3RU-1691863239-0-gaNycGzNCbs",cFPWv: 'b',cTTimeMs: '1000',cMTimeMs: '0',cTplV: 5,cTplB: 'cf',cK: "",fa: "/download/1.1?__cf_chl_f_tk=aI4FZvcw.k3BvuyP9D0my2O91OcNdloVC5jCVhju3RU-1691863239-0-gaNycGzNCbs",md: "5RArXUr_ULtVs7jAZXBpzkz3PsdqZH0T_8L05UyCMcU-1691863239-0-AaOfTRRLPg_IJspAnTwcGo1eoLwsiRK_cxY-Pa6dlBqRF0TpEdKnd3g5E_-Y-DR1w-mwYyImXAY6FoKV3SKmtpJl7cMT4MW5872D4MlEGgAfDG-xrR_yvmkInH_54bu1VlknYm3XFXgIZpnVKAXij-YUPJOXbCCtrl0DQZWvC2aGiWU6QZtE2iwrtRlJGc8OTAlqgAZPbvRnjtfPPXqqbZeERQUEjri6F66M3xS0QzJUEbw3JZ8HK1H3oX5SQhyR9nCe9w3JZAH0-oUDbTmpxXoSNdwGZ0lA5Z0MBEHvnJs2RhZw8b5uxX5MNnTczdfYkjosczIR4DA4JpfinipEdFvU8BHsOudi7BTWl7aG0I5pxQDBtCeZCpqeneADWBuPz6NIph5Eblq__E1yXmqJuxRVtTc0oeV9g5PC617pAKZBDqY5nfciPi0U400PPoZemXWA3Za8QTXw9aAYRfgtw39wpBcAyDBhhz_9U-iY88AnEwBTgb3U_75piy2TDjU6q5NMNDmEt9C6kBswan5-LaCgFwmb1He16m3K6l-ViJ73V9g0gynhyGFJVIanjFnof2CxbFy68WAABf_tYn_vTsUDhfE0ncRj0G0oM0MXHIrrz-85NVKcLJz5tXLickVafoNRl245lciL6h04EiE7rLWHTG5s2yx2xdZPGldk4_koLeJGOyD5KykOCoDPcmEVE8_1w370zD2_SCDkHdcm7Zod5jtxhYLHM-oTEy5eb045fxyFCB-S3-LcutfIV65Tgo49AYX7NzWrK_8GRr4GOqjGJG4BF22q4NM90m6jYXEzA5Wr9qyclh_W0bvtmdvKIIgM6zO9QiWvtrmU3yotlEQMPe8kaK9M_iMhedqagcAyYSQRj4Z_vzTlpZS8LFqatsVvLUF4Oyt6JgPavpDbGr-5ptsZKCtZ1veWiGjNW78bGv3jrV7Ik4k9sMxOs7IbGqVCUdMWNWpsD3XN4AIg1KKsYHuqJJUxm4tdZd8_yLB1X14GgvAccI7bgB9tlIAdaXcx3Z5hG9hbIRweQWcTXnaYoJ2r5cjYyb68y7pgX3GxjfAPPcEvNR27ZmJMTC4HqCafs-sVsbs7PPSV8pi5uCxrfBW--4vTfbHD2G4yxOgRD34SbmQOTda-aMCWUZorYTy7oDTmY19JdlhkG5bIYUcsnvmNlOShtrAhttDPAy2C_n4ZUh2S-dU8C0H5N7MDvWlbN10feFevGRO92dSBdOTQYyXDF_D9kRr1ROs_hkxwD7qjf0E8PX-4E3X51ml9nMCsyzivW5KeFvT0M9VkVYAzKytuhHnNQqQzEGVvTv6xD3nhFB1AMIEoVwRpjYLH3-mQPsT_MtrH-GkAypu-iOQP2Uf4W_gQAgtyLytp29_dCjOFYVeTyFwSvNzNDAAri5K_JMNIntq3vtaBdoPtDRgwOFHcIp9AONd3MEEaGP-_HK4fOmtTXggOMAVvW3UsT4JR8oyhNT9jy5UioHxf2TDQwVjgbcIxMuFKVpytbCxva6QwXQ0mqgQQCspP_OtYg1N9JPpG8wJ1qFS-AEU0XU-4JdwwGPdC2XlEUYt2whloFdX9yJ20H6EZEUpAhGaFWMohNR0A5dMjUUvR0rfSSyeqPFlDMUTVTRxTlWTL5Z_TwSVd3laBdjkG2JllT5M5QKUQXWMq5BhHZImpBlvbQ-CAWhrFoaTzWkRAO9N6z0g2S_c1VCRYA7Fp_0CRoxCP_VJdrIaMfdjZXqUUOV04fjF-LzcZcdbGMgTeInaDikVSuuA7YZjWM2nKrhf9eTfoMB1bdqbDYUACce6O4lpF0wJSBhLsuxjlQH0KT0VNhiLN5PkI3okRhZk9pO8CwqBvMKB8u6xkSsn-YJY1LBqwc-tVyaFtOmMQbzZ1bhTRM2ZFAPBWycTnv5kemC3Ka5GUNug7UKmgsLG7hqiGh5OtOP083P_dclvCTQ9QTArytMlufrn0_7AL4Xgbf8CncjXC8GuosoHvO01Pg7JOrQtRudJEiWTjKwtxGvoWv_blKDf1seco3ziZpOibCbmzw2snLqBlmqHbeUufdB2NSlGKr5wGIEfvS2lYvM4trx9I_dOg4vqvnZYudo679O8HDUd0q4SMdnTC65fyeXnL3pyqBMbmysVQ5hCgFNks4grQjn8QFbw-Rc-mRqsDx02r7fe71RBebV6vM4c0121y_1ReUkK4CfTcR_S8F_YXNa3mr6pGtOpQTZv5pxJsdU0ykAKeKlGGT22_b7CBQL0aGlTKr189bi7qYxpOpxop3pZmvQK3CuZHFOlNXrKu9ilqmiaUQo7mL9PLfZEGRT_FILrN-awrXYTC4D68YtN5suw1OxSrMLj-aZ_MQcBcP11v5yeiVpkWPwpNJrAyvh-5Dr0MrMFiP5rvnb2H7ottUZrZnzRLqPmsBFSl0vxNnxxVxUbcwQ",cRq: {ru: 'aHR0cHM6Ly93d3cuYXV0b2hvdGtleS5jb20vZG93bmxvYWQvMS4x',ra: 'QXV0b0hvdGtleQ==',rm: 'R0VU',d: 'DSVwNaT1mVaThGN4O+OcROzpcI4DOwpqunoRqxRrV3nU3QViieHHuBM7HoS5B3XfVaYFea8AQq8LYlkHbrRSUlC0Gjc4aFaiA+WmXKad0X5sT10Yt4/sCWS6IQMQKBwIrM+0Ds1YeuEhGW4vnuh1ePSeeqFRMH6MkUxshY3iwqVjQdR2QzJ7o0Rw06QsPR0U0MbGkuE08TiwhInitAfglfnf/Ve1E1HhbN1nsx9qY9UTG5oklHL/SQKOtlvW1mFG4WiyesPIa4b2Rb6S9wIp2g1gKngc4HthXL+5CI5Y7AoyhgQHtHsG2vfMhPyBR7cdBBqetUGO8amgRVBupgQySX/9BxrKNSCe0QBbSirUNGbtoVU2+/pqy2LjkIGHC68P7kl9tEoq0ntma8fb7a22Bt1rWcJtPdeefIE/boUOgVgs5/BklIzhge5abmCpNvQzXTolEeNOh6nZ0lJa/i80Og==',t: 'MTY5MTg2MzIzOS44NzkwMDA=',cT: Math.floor(Date.now() / 1000),m: 'oG3HcZk2ewTNPvuehdtmHfYmLYobQmeM7SkGji0pW8Q=',i1: 'aZa1UVcPRFJaiHQRbn0ucw==',i2: '5sWBWAAE+ULKlsptTSwZuQ==',zh: '7HQcobp3K6H09pr/nGTI9oxIXUjIKSZbLz0il7CAMSA=',uh: 'dc45Sze+MYtsCSkSWrWLhuDert1j/6N8/Msc0J4BPyI=',hh: '23nUAWHWE3jYr1IhewWxvDEyd/FxBpLUcEnW/thlDQ4=',}};var cpo = document.createElement('script');cpo.src = '/cdn-cgi/challenge-platform/h/b/orchestrate/chl_page/v1?ray=7f5a9e813a432c7e';window._cf_chl_opt.cOgUHash = location.hash === '' && location.href.indexOf('#') !== -1 ? '#' : location.hash;window._cf_chl_opt.cOgUQuery = location.search === '' && location.href.slice(0, location.href.length - window._cf_chl_opt.cOgUHash.length).indexOf('?') !== -1 ? '?' : location.search;if (window.history && window.history.replaceState) {var ogU = location.pathname + window._cf_chl_opt.cOgUQuery + window._cf_chl_opt.cOgUHash;history.replaceState(null, null, "\/download\/1.1?__cf_chl_rt_tk=aI4FZvcw.k3BvuyP9D0my2O91OcNdloVC5jCVhju3RU-1691863239-0-gaNycGzNCbs" + window._cf_chl_opt.cOgUHash);cpo.onload = function() {history.replaceState(null, null, ogU);};}document.getElementsByTagName('head')[0].appendChild(cpo);}());</script></body></html>

I'm guessing this might be a cloudflare change.

Any tips would be greatly appreciated.

12 Aug 2023, 18:22

That might be related to the (temporary) cloudflare check against automated attacks which has been activated a few days ago.
It seems the recent slow forum problem was solved when tank purged the server cache.
If there are currently no signs for an attack, perhaps the cloudflare "under attack" mode can be lifted again. We should ask @joedf.
But it's not unlikely that this check will come back some day (as long as it is needed).

But do you really need to parse the download page regularly? To check for new versions, you could perhaps parse the AHK github releases page, or smth simpler like
https://www.autohotkey.com/download/1.1/version.txt
https://www.autohotkey.com/download/2.0/version.txt
Or are they currently blocked for you as well?

12 Aug 2023, 23:04

Indeed Cloudflare... I am still getting some slow downs, but I can definitely turn back off if you think it makes sense now.
I was gonna suggest the *.txt links as well. In fact, I'd say that's the recommended source for the latest versions.

TheArkive · 13 Aug 2023, 05:32

Thanks guys. I'll take a look at doing this better soon.

@joedf Which txt links are you referring to. Sorry to say I'm not aware of these links.

I'd gladly use the github api over parsing the download page, but only the installer is offered for download, not the zip. Is it possible to use the installer like a zip file and just decompress the files to a specified location?

This is all part of my "AHK Portable Installer" since a while ago. Updates are only checked at max once per day, if the user even opens the program. I don't usually open it unless I'm installing/uninstalling a new version, and I usually wait for forum announcements to do that.

13 Aug 2023, 12:24

@TheArkive Oh, I meant the links gregster shared

https://www.autohotkey.com/download/1.1/version.txt
https://www.autohotkey.com/download/2.0/version.txt

kczx3 · 13 Aug 2023, 12:49

There’s also https://www.autohotkey.com/download/versions.txt which you can use to generate the version specific urls

TheArkive · 13 Aug 2023, 13:51

Interesting. I had not thought of generating the URLs from the version.txt files. Though that is the first thing that my script checks before parsing the download page.

And i think that is the part that failed. I'll double check that.

Thanks to everyone for the clarification.

13 Aug 2023, 21:44

The v2 installer is a upx-compressed compiled script. I don't think I implemented a switch for "extracting", but it could be done.

You could determine the available versions from GitHub and build a URL from that, but if you can't automate the version.txt download, I suppose you'd have the same problem with the zip.

14 Aug 2023, 00:40

Yes right, you could use this handy link, it will redirect to the latest: https://github.com/AutoHotkey/AutoHotkey/releases/latest

14 Aug 2023, 00:51

The GitHub API has better ways to get information about releases, without scraping/parsing HTML.

neogna2 · 14 Aug 2023, 06:44

If not a big hassle I request that github releases also include the .zip version.

TheArkive · 15 Aug 2023, 13:34

@lexikos Yes I certainly would prefer to use the GibHub API. As neogna2 said, it would be nice if the zip files could be included on GitHub, if possible. Or if there could be a switch on the installer to only extract, that would work equally well for my uses, then the zip would not be necessary.

EDIT: And if extraction could be done via the command line, that would be ideal for my uses.

neogna2 · 30 Aug 2023, 08:59

I noticed that with v2.0.6 today the GitHub release now includes the .zip version. Thank you Lexikos!

Something with the download page changed recently?

Something with the download page changed recently?

Re: Something with the download page changed recently?

Re: Something with the download page changed recently?

Re: Something with the download page changed recently?

Re: Something with the download page changed recently?

Re: Something with the download page changed recently?

Re: Something with the download page changed recently?

Re: Something with the download page changed recently?

Re: Something with the download page changed recently?

Re: Something with the download page changed recently?

Re: Something with the download page changed recently?

Re: Something with the download page changed recently?

Re: Something with the download page changed recently?

Who is online