Jump to content


Photo

Sync BIG files over the internet?


  • Please log in to reply
2 replies to this topic

#1 TheLeO

TheLeO
  • Members
  • 264 posts

Posted 28 January 2010 - 10:10 AM

Hey guys,

I do a lot of work at home, and I need to keep my home and work environment synced up, unfortunately a laptop won't get the job done, I got 2+ Tb of stuff. :-/


I use <!-- w -->www.mesh.com<!-- w --> to sync notebooks/programs/files/documents/pictures/scripts/exc

But I'm not sure how I should sync big stuff
e.g like my Virtual lab
( contains 550 gb's worth of VM's, xp/vista/win7/2003/2008/2000 with Lots of snap shots and in multiple Languages).

I've got a home server with a static ip/dns and a 24/24 conneciton.

I've had a go at DC++, set up my own hub, and copied the lab across the internet, but the difficulty here is to keep the two copies synced up.

I tried ftp Sync, but that doesn't work well for files over 4gb.

I tried iScsi over internet, but that doesn't seem to work smoothly either. sometimes works sometimes not. (good for listening to music thouhg).

Has anyone done that before? Any reccomendtaions?

(I'm not so good with Linux, but I know my way round windows quite well).

#2 SoLong&Thx4AllTheFish

SoLong&Thx4AllTheFish
  • Members
  • 4999 posts

Posted 28 January 2010 - 11:06 AM

It is not easy to use and setup BUT if you do manage to use rsync it is a hoot, it will sync large files by only up/downloading the changed sections of a file, so rather than FTP 500 mb just because you added two seconds of data to a file at the end it will use an algorithm to calculate the changes

<!-- m -->http://en.wikipedia.org/wiki/Rsync<!-- m -->

The rsync utility uses an algorithm (invented by the Australian computer programmer Andrew Tridgell) for efficiently transmitting a structure (such as a file) across a communications link when the receiving computer already has a different version of the same structure.


A good starting point:

<!-- m -->http://lifehacker.co... ... 196122.php<!-- m -->

Edit: a bit more user friendly and also using the same algorirthm is <!-- m -->http://www.cis.upenn...cpierce/unison/<!-- m -->

#3 TheLeO

TheLeO
  • Members
  • 264 posts

Posted 19 July 2010 - 08:26 PM

done it over gBridge btw, works very well.