AutoHotkey Homepage AutoHotkey Community
Let's help each other out
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Use Autohotkey as an interface to Amazon S3

 
Reply to topic    AutoHotkey Community Forum Index -> Ask for Help
View previous topic :: View next topic  
Author Message
airjrdn



Joined: 25 Feb 2008
Posts: 37

PostPosted: Fri May 23, 2008 4:50 pm    Post subject: Use Autohotkey as an interface to Amazon S3 Reply with quote

Has anyone done any work using AutoHotkey to connect to Amazon S3? Would it be possible?
Back to top
View user's profile Send private message
n-l-i-d
Guest





PostPosted: Fri May 23, 2008 5:20 pm    Post subject: Reply with quote

Amazon S3 apparently uses SOAP or XML/REST, and the last one can be done with AHK + curl/libcurl.

SOAP via AHK?

And here an external site that explains the steps to be taken:

Introduction to Amazon S3 with Java and REST

HTH

PS: No, I haven't, yes it is... Wink
Back to top
Roger Matthews



Joined: 21 Apr 2004
Posts: 4

PostPosted: Mon Jun 16, 2008 10:17 pm    Post subject: Reply with quote

On the Amazon forums, there is this link, http://developer.amazonwebservices.com/connect/entry.jspa?externalID=739&categoryID=47 which describes a "Simple Command-Line Access to Amazon EC2 and Amazon S3". The article on Amazon mentioned above links to the following site, http://www.timkay.com/aws/, and especially, http://www.timkay.com/aws/#windows, which is the Windows-oriented versions of his command line utility.

It looks like the user will need to install a Windows-oriented version of "curl" and "Perl", for which downloads are provided on the site. Then one gets a command-line utility to completely control their access to Amazon S3. When I read the capabilities on this site, the FIRST THING I THOUGHT OF WAS AUTOHOTKEY. So I came to the forum, and found this post. I think this command line capability is perfect for a marriage with AutoHotKey, and I will be spending some time working on that. But I wanted to share the find with the AutoHotKey forum Smile. It does not require SOAP or REST or JAVA or any other protocols.

Here is a quote from one of the reviewers who has used this utility: (Dec 2007) "I have been using aws for approximately 2 months. This program makes access to Amazon S3 as easy as it gets. From the simple installation to the command line shortcuts, I never have to rely on any other way of accessing my S3 files. It is as if I have S3 mounted as a file system on my local machine - it's that easy. I wrote the author when I first started using aws to congratulate him on a very useful piece of code, and my favorable opinion of this little program has only grown stronger. "

Here is another review: (Jan 2008) "I have to say a big thanks to Tim for developing such a great way to interact with S3 and EC2. Installation takes about 5 seconds and afterwards you can easily interact with your S3 account from the command line. I use this mainly for moving our backups offsite and haven't had a single problem sending multiple gigabyte files to S3 each night. I've tried a lot of S3 scripts but none comes close the easy of use and reliability this script. "
Back to top
View user's profile Send private message
airjrdn



Joined: 25 Feb 2008
Posts: 37

PostPosted: Mon Jun 16, 2008 10:33 pm    Post subject: Reply with quote

I'd still love to see this happen, but honestly, I didn't have time to wait. I did quite a bit of research and settled on WebDrive which now supports S3. It mounts S3 as a drive letter in windows, and it really doesn't get much easier than that. At that point, both GUI & command line access are available, which makes automating backups (SyncBack, etc.) extremely easy.

I've uploaded about 80G so far, and it's been solid. Failed uploads are auto-retried X times, it backs up files that didn't make it after the set number of retries. It's also backed by some decent support, as I've worked with one of their devs on fixing some minor issues.
Back to top
View user's profile Send private message
JoeSchmoe



Joined: 17 Feb 2008
Posts: 303

PostPosted: Fri Jul 24, 2009 5:40 pm    Post subject: Reply with quote

Hi all,

Has anyone ever managed to get AHK to work with S3? I'd really love any help that anyone could provide.

My overarching goal is to use S3 to store and retrieve [/url]the contents of several variables in order to coordinate the activities of scripts running on various machines. In the end, while there will probably be various supporting functions, I'm looking for a pair of functions:

Code:
Put(Name, Value) {
Puts "value" into a file named "name" in my Amazon S3 account
}
and
Get(Name) {
Returns the contents of the file named "name" from my Amazon S3 account
}


I'm not too worried about security as the code will only be running on my own machine and I don't really have anything worth stealing. Smile

Do you think that this sounds feasible?

One thought I had was using a command line program and pipes of some sort. libs3 and aws seemed like great candidates. Ideally, though, it would be terrific to be able to use httpQuery directly, as I'd rather not have things written to a file to slow things down. However, while I've written several thousands of lines of AHK code, I've never used a library like httpQuery, let alone dealt with a webservice before, so I thought that I would inquire about feasibility first.

Thanks in advance
Back to top
View user's profile Send private message
wakewatcher



Joined: 15 Jul 2006
Posts: 253

PostPosted: Fri Jul 24, 2009 6:43 pm    Post subject: Reply with quote

@JoeSchmoe

Thanks for the wakeup call. If you are not worried about security, i.e. Its only you on your PC accessing S3 then I'd recommend you using John Spurlock's rescmd.exe. You can get it here. http://resourceful.codeplex.com/

However if you want to share your ahk code with others and want to share your S3 account with others then you need some security. I've mashed up a solution using S3's Form Post upload capability and cURL. You can see it here including a sample of ahk showing how I used it. So its a combination of ahk/cURL on the client side and PHP on the server side. http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1434
Back to top
View user's profile Send private message
JoeSchmoe



Joined: 17 Feb 2008
Posts: 303

PostPosted: Fri Jul 24, 2009 7:29 pm    Post subject: Reply with quote

Thanks for your ideas, wakewatcher! I will definitely both of those out.

In the meantime, I also had another idea that I'm really excited about. I remember when I was an undergrad, one of my who knew more about computers showed me something that really stuck with me (I studied math with a minor in CS, while he majored in CS and was older). He showed me how to telnet to various TCP ports, and how easy it was to replicate various protocols. This was before HTTP was big, but it looks like it would be quite easy to do the same thing with HTTP, as demonstrated in the following screen shot:
http://upload.wikimedia.org/wikipedia/commons/c/c6/Http_request_telnet_ubuntu.png
You just telnet to the host and issue the appropriate HTTP commands. Then it send the document right back at you through the TCP channel.

Based on the following Amazon API document:
http://docs.amazonwebservices.com/AmazonS3/2006-03-01/index.html?RESTAPI.html
It looks like S3's "REST interface" basically means doing this, following what looks like a fairly simple protocol. For example, here are the references for the Put and Get commands:
http://docs.amazonwebservices.com/AmazonS3/2006-03-01/RESTObjectPUT.html
http://docs.amazonwebservices.com/AmazonS3/2006-03-01/RESTObjectGET.html

Could I go through the code for httpQuery to figure out which libaries I need to form and use this sort of TCPIP connection and just make the requests myself? Or am I missing something important? Since all that I need to store and retreive are short strings (most about 10 BYTES) long, this approach would be extremely attractive to me, especially if I could avoid writing temporary files.

If this worked, I could wrap it up in a nice library, which I bet others might find very useful as well. I can't be the only person who wants variable synchronized between instances of AHK scripts possibly running on different computers.... With Amazon's pricing being so cheap as to practically be free (I'd pay them about 46 cents per month, I estimate) this really seems like a terrific solution to a dilemna I've been frustrated with for months.

.... that is, IF it works...
Does it seem feasible to you folks?
Back to top
View user's profile Send private message
wakewatcher



Joined: 15 Jul 2006
Posts: 253

PostPosted: Fri Jul 24, 2009 8:19 pm    Post subject: Reply with quote

Or you could go here -> https://www.getdropbox.com/referrals/NTEzNDExMTk and get 2Gigs of Dropbox for free and save 46 cents a month. Very Happy I think the back end of Dropbox is S3 and its a great service.
Back to top
View user's profile Send private message
JoeSchmoe



Joined: 17 Feb 2008
Posts: 303

PostPosted: Fri Jul 24, 2009 8:23 pm    Post subject: Reply with quote

Okay, another related question for anyone who knows about REST.

If I want to learn some code to make a well defined REST query, should I study httpQuery or the TCP/IP client server script?
DerRaphael's httpquery: http://www.autohotkey.com/forum/topic33506.html
Zed Gecko's tcp/ip: http://www.autohotkey.com/forum/viewtopic.php?t=13829

For a look at the API I would be trying to match (don't know if you need to know this), see the following three links for quick examples:
http://docs.amazonwebservices.com/AmazonS3/2006-03-01/index.html?RESTAPI.html
http://docs.amazonwebservices.com/AmazonS3/2006-03-01/RESTObjectPUT.html
http://docs.amazonwebservices.com/AmazonS3/2006-03-01/RESTObjectGET.html
Back to top
View user's profile Send private message
JoeSchmoe



Joined: 17 Feb 2008
Posts: 303

PostPosted: Fri Jul 24, 2009 8:28 pm    Post subject: Reply with quote

wakewatcher wrote:
Or you could go here -> https://www.getdropbox.com/referrals/NTEzNDExMTk and get 2Gigs of Dropbox for free and save 46 cents a month. Very Happy I think the back end of Dropbox is S3 and its a great service.

Great minds think alike. I tried building the script on that, but had no end of problems because files didn't get synchronized when they were supposed to. I think that DropBox doesn't synchronize files that are frequently changed very often (most likely because it wants to save its 46 cents Very Happy ). Because files weren't synchronized, different computers would both make changes and cause a synchronization crash and I'd have to sort it out by hand. Evil or Very Mad Overall, I love Dropbox and think that it is great at what it is designed for, but I just don't think that it was designed for what I was using it for.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    AutoHotkey Community Forum Index -> Ask for Help All times are GMT
Page 1 of 1

 
Jump to:  
You can post new topics in this forum
You can reply to topics in this forum


Powered by phpBB © 2001, 2005 phpBB Group