Transfer large amount of files

Zack12

New Member
I have about 3 million files and folders sized (240G) on a local PC . Once the main job is synced only a thousand or so files change every day.

What I am trying to do is to sync the entire contents to a webhost via ftp and keep it updated as the files change on my local PC.

So what I need , is to transfer only the changes that happens on the files in the same time it changed on the destination .

I tried using GoodSync for this but it takes 15 hrs to analyses and by that time a number of files have changed.

Is there any software or string of batch files that would do this for me and utilize multiple connections for a ftp username?

- Greetings
Zack
 

Neemobeer

Principal Cybersecurity Architect
Staff member
I would just shift to a backup solution. Even Onedrive, Google Drive or any of the big name offerings will automatically just sync the changed files.
 

Zack12

New Member
I would just shift to a backup solution. Even Onedrive, Google Drive or any of the big name offerings will automatically just sync the changed files.
I was thinking about that but will transfer 240G to OneDrive and the daily changes be free?
 

amandawhisen

New Member
I have a laptop and a desktop, and a USB 3.0 stick. I work with big project files regularly and I sometimes have to copy work from my desktop to my laptop whenever I work outside my home. I copy the project to my USB to transfer it between my PCs (and to keep a backup).

From what I've observed, when you start the copying process (with many files), Windows runs through all files while copying, and then eventually asks you if you want to replace the files in the destination folder (which I always want to do, since I update files as I work). What I don't understand, is why Windows doesn't ask me this before it starts copying. At least if I have a large quantity of files to transfer (I know Windows knows how many files it is).
I've many times before started a large file-transfer either from or to the USB (which usually would take ~30 minutes or more), and then left the PC to copy on its own, only to return 30 minutes later and find the process paused at 15%, with Windows waiting to know if it should replace the files or not.

Would not it be more efficient to just ask me in the beginning, regardless of whether there's any files with the same name in the destination folder? Or does Microsoft not want Windows to ask unnecessary questions in any given case? (In the case there wasn't any files to replace.)

I've searched on Google to see if anyone had had the same problem, or if it turned out there was a way to change this in some settings somewhere. But to no avail. Is this fixable?

I'm on Windows 10, but I had this problem back in 8.1, and I believe the other Windows versions might be doing somewhat the same thing (so I'm posting it here and not on the Windows 10 subreddit) bam margera
 

Zack12

New Member
OneDrive would be a good choice and I think GoodSync can work fine and quick with OneDrive, also while searching I found alternatives to Goodsync that I can try, the tools are Carbonite and Gs Richcopy360 , I am checking every scenario now to sync all my data easy, quick and without any loss. and as Neemobeer said my issue will be with the cost of storage !
@amandawhisen I saw your comment and I think any of the mentioned backup tools will help
 

ussnorway

Windows Forum Team
Staff member
Premium Supporter
Windows doesn't ask for your input before copying because the guy who did the script designed it to scan each file first, then copy the files that don't have a existing folder next and finally ask you what to do with the files that need to replace old data

As a user Microsoft assumes you have limited understanding and zero permission so doesn't offer alternative workflows

Sent from my Nokia X20 using Windows Forums mobile app
 

ussnorway

Windows Forum Team
Staff member
Premium Supporter
Zack be careful using OneDrive on active files because it's designed for storage not realtime sync... If a user tries to access/change files that OneDrive is currently syncing then the file gets locked or corrupted

OneDrive should not be used in Lu of network devices as active file sharing between multiple users but if you want to sync between multiple stations for a single user i.e from desktop to laptop and back for taking work home it's very good

Sent from my Nokia X20 using Windows Forums mobile app
 

Zack12

New Member
I ended up transferring the data with the backup tool Gs Richcopy 360 and using OneDrive as a destination. This seems to have worked
Thanks all for your helpful information
 

wawayaya

New Member
You could try a free backup and transfer third-party tool. Here I strongly recommend you to use AOMEI MBackupper, with which you could transfer or backup a large amount of files and there's no need to worry about the space. Have a look.https://www.ubackup.com/phone/
 
Top