I have tried to generate an automated system of file transfer for my solar and deep sky astrophotography setup since I have not much time for the hobby, and not so much patience with unpleasant things like waiting for copy processes to end, and also tend to make errors when shuffling many data .
I have a network of Windows 10 and Stellarmate (Raspberry Pi 4, Linux) image acquisition computers, and Windows 10, Linux (Ubuntu), and Mac post-processing computers. The two acquisition computers at the telescopes are connected via Mesh network to the rest of the network. If I acquire large data quickly (typically in solar imaging with 1TB or more in a single session) I use an additional CAT6a-network cable.
On my central windows 10 post-processing computer I run http://Syncovery.com software. It is available for Windows, Mac, and Linux, and it connects via network (ethernet or wifi) to the acquisition computers' acquisition directory and automatically moves the files from there to a central acquisition directory on the central post-processing PC. Syncovery allows one to set up profiles for a variety of tasks with optional bandwidth limitations (e.g. to not interfere with an existing remote desktop connection ). Image download is initiated automatically every few minutes.
I also have SYNCOVERY profiles to automatically copy image and video data during the night from the post-processing PC to a local NAS, as well as results of the post processing directory. After post-processing I can just delete the acquisition files because everything is backed-up already.
SYNCOVERY is excellent wrt to the copy-profile flexibility (other software may be as well) but it takes a bit of time to optimize everything before it runs smoothly. This optimization is not so much of an issue with deep sky image data but video data in solar imaging (10GB ser-files with rapid acquisition of images at 165 2megapixel images/sec for typically 20sec intervals, seeing triggered, and sometimes saturating/overheating the 2TB PCie3.0 NVMe SSD at the telescope's NUC because of the high data rate) . Since the NUC only has 1GB ethernet I use a USB3 to 5GB ethernet converter connected to my 10GB switch to accelerate downloads if needed. This converter has a lot of overhead but gets me about 2Gb/sec throughput for download. For deep sky images the wifi connection (typically somewhere between 54 and 200Mb/sec) is sufficient, however .
I hope there are some ideas in this which may be of interest to the OP.
Edited by LauraMS, 29 October 2021 - 05:18 PM.