Skip to content

Utility to download only missing files from remote server based on file list

License

Notifications You must be signed in to change notification settings

shoobyban/gwget

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

7 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Wget based on file list

Documentation Go Report Card Build Status

The tool can (for example) mirror a media folder to a staging or developer server from the current live, over standard http requests, when scp is way too slow, creating tarball is not an option because of space requirements, VPN makes it hard to run tar | tar or rsync.

Usage

source$ cd ~/web/media
source$ find . -type f | grep -v '.thumb' | grep -v product/cache | sed 's#^./##' > ~/media.txt

scp media.txt onto the target server with a gwget binary

target$ cd ~/web/media
target$ ~/gwget ~/media.txt https://www.somesite.com/media/

About

Utility to download only missing files from remote server based on file list

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages