Transfer large amount of files

Zack12

New Member
Joined
Jan 15, 2022
Messages
8
I have about 3 million files and folders sized (240G) on a local PC . Once the main job is synced only a thousand or so files change every day.

What I am trying to do is to sync the entire contents to a webhost via ftp and keep it updated as the files change on my local PC.

So what I need , is to transfer only the changes that happens on the files in the same time it changed on the destination .

I tried using GoodSync for this but it takes 15 hrs to analyses and by that time a number of files have changed.

Is there any software or string of batch files that would do this for me and utilize multiple connections for a ftp username?

- Greetings
Zack
 
Solution
I ended up transferring the data with the backup tool Gs Richcopy 360 and using OneDrive as a destination. This seems to have worked
Thanks all for your helpful information
Sort by date Sort by votes
I would just shift to a backup solution. Even Onedrive, Google Drive or any of the big name offerings will automatically just sync the changed files.
 
Upvote 0 Downvote
Neemobeer said:
I would just shift to a backup solution. Even Onedrive, Google Drive or any of the big name offerings will automatically just sync the changed files.
Click to expand...
I was thinking about that but will transfer 240G to OneDrive and the daily changes be free?
 
Upvote 0 Downvote
I have a laptop and a desktop, and a USB 3.0 stick. I work with big project files regularly and I sometimes have to copy work from my desktop to my laptop whenever I work outside my home. I copy the project to my USB to transfer it between my PCs (and to keep a backup).

From what I've observed, when you start the copying process (with many files), Windows runs through all files while copying, and then eventually asks you if you want to replace the files in the destination folder (which I always want to do, since I update files as I work). What I don't understand, is why Windows doesn't ask me this before it starts copying. At least if I have a large quantity of files to transfer (I know Windows knows how many files it is).
I've many times before started a large file-transfer either from or to the USB (which usually would take ~30 minutes or more), and then left the PC to copy on its own, only to return 30 minutes later and find the process paused at 15%, with Windows waiting to know if it should replace the files or not.

Would not it be more efficient to just ask me in the beginning, regardless of whether there's any files with the same name in the destination folder? Or does Microsoft not want Windows to ask unnecessary questions in any given case? (In the case there wasn't any files to replace.)

I've searched on Google to see if anyone had had the same problem, or if it turned out there was a way to change this in some settings somewhere. But to no avail. Is this fixable?

I'm on Windows 10, but I had this problem back in 8.1, and I believe the other Windows versions might be doing somewhat the same thing (so I'm posting it here and not on the Windows 10 subreddit) bam margera
 
Upvote 0 Downvote
OneDrive would be a good choice and I think GoodSync can work fine and quick with OneDrive, also while searching I found alternatives to Goodsync that I can try, the tools are Carbonite and Gs Richcopy360 , I am checking every scenario now to sync all my data easy, quick and without any loss. and as Neemobeer said my issue will be with the cost of storage !
@amandawhisen I saw your comment and I think any of the mentioned backup tools will help
 
Upvote 0 Downvote
Windows doesn't ask for your input before copying because the guy who did the script designed it to scan each file first, then copy the files that don't have a existing folder next and finally ask you what to do with the files that need to replace old data

As a user Microsoft assumes you have limited understanding and zero permission so doesn't offer alternative workflows

Sent from my Nokia X20 using Windows Forums mobile app
 
Upvote 0 Downvote
Zack be careful using OneDrive on active files because it's designed for storage not realtime sync... If a user tries to access/change files that OneDrive is currently syncing then the file gets locked or corrupted

OneDrive should not be used in Lu of network devices as active file sharing between multiple users but if you want to sync between multiple stations for a single user i.e from desktop to laptop and back for taking work home it's very good

Sent from my Nokia X20 using Windows Forums mobile app
 
Upvote 0 Downvote
I ended up transferring the data with the backup tool Gs Richcopy 360 and using OneDrive as a destination. This seems to have worked
Thanks all for your helpful information
 
Upvote 1 Downvote
Solution
You could try a free backup and transfer third-party tool. Here I strongly recommend you to use AOMEI MBackupper, with which you could transfer or backup a large amount of files and there's no need to worry about the space. Have a look.https://www.ubackup.com/phone/
 
Upvote 0 Downvote
Hi wawayaya
Thanks for the suggestion and would say Gs Richcopy360 was a perfect choice for my situation
 
Upvote 0 Downvote
Zack12 said:
Hi wawayaya
Thanks for the suggestion and would say Gs Richcopy360 was a perfect choice for my situation
Click to expand...
@Zack12
As a big fan of Teracopy, for sorry, I found that there is no feature to limit the transfer speed! My question: Does Gs Richcopy have a feature to limit the transfer speed?
 
Upvote 0 Downvote
can I transfer all data files of wordpress website to another platform without losing the sequence and any virus error?
I need to shift it to other running porgram.
 
Upvote 0 Downvote
You must log in or register to reply here.

Similar threads

  • Featured
  • Article Article
WSL2: The Right Amount of Linux on Windows for Developers
Replies
0
Views
256
ChatGPT
  • Featured
  • Article Article
Reliable Large-Scale Windows File Transfers: Robocopy, Compression & Tools
Replies
0
Views
26
ChatGPT
  • Featured
  • Article Article
FastCopy: Speed Up Large File Transfers on Windows with Direct I/O and Verification
Replies
0
Views
28
ChatGPT
  • Featured
  • Article Article
Windows 11 Compression Guide: Built‑in ZIP vs 7‑Zip and Robocopy for Large Transfers
Replies
0
Views
50
ChatGPT
  • Featured
  • Article Article
Reclaim Disk Space on Windows 11: Find Large Files and Clean Up Safely
Replies
0
Views
83
ChatGPT