Fixes issues #16, #17, #18 and #19

[kranzky]

These fixes introduce the following changes:

• manifest was added to configuration to specify the full path to the manifest.yml file generated by the asset pipeline
  • this allows the list of local files to upload to be read from the manifest file
• if manifest is not given, the list of files is generated by recursing into public/assets as before
  • however, the list of local files is now cached to prevent this from happening more than once
• the assets sub-directory is now specified when locating files in the S3 bucket, meaning that the bucket may safely contain non-asset files in other sub-directories
• the full list of asset files already on S3 is now retrieved, rather than the first 1000, preventing unnecessary re-uploads and ensuring all old assets are deleted (if that configuration option was given)
• the list of files to upload is now correctly calculated:
  • it is now simply the local files minus the remote files; previously it was the xor of the two collections
• the list of remote files to delete (if that configuration option was specified) is now correctly calculated:
  • it is now simply the remote files minus the local files; previously it was the cor of the two collections
• new files are uploaded before old files are deleted.

[davidjrice]

This all sounds great

I will be reviewing everything this week.

On 11 Nov 2011, at 01:49, Jason Hutchens wrote:

These fixes introduce the following changes:

• manifest was added to configuration to specify the full path to the manifest.yml file generated by the asset pipeline
  • this allows the list of local files to upload to be read from the manifest file
• if manifest is not given, the list of files is generated by recursing into public/assets as before
  • however, the list of local files is now cached to prevent this from happening more than once
• the assets sub-directory is now specified when locating files in the S3 bucket, meaning that the bucket may safely contain non-asset files in other sub-directories
• the full list of asset files already on S3 is now retrieved, rather than the first 1000, preventing unnecessary re-uploads and ensuring all old assets are deleted (if that configuration option was given)
• the list of files to upload is now correctly calculated:
  • it is now simply the local files minus the remote files; previously it was the xor of the two collections
• the list of remote files to delete (if that configuration option was specified) is now correctly calculated:
  • it is now simply the remote files minus the local files; previously it was the cor of the two collections
• new files are uploaded before old files are deleted.

You can merge this Pull Request by running:

git pull https://github.com/agworld/asset_sync e26f5ca

Or you can view, comment on it, or merge it online at:

#20

-- Commit Summary --

• fixes Files that already exist on S3 are still uploaded #16
• fixes Storage::get_remote_files returns non-asset files #18
• Implements Local files should perhaps be built from the manifest.yml #17
• fixes Calculation of local_files_to_upload is incorrect #19

-- File Changes --

M lib/asset_sync/config.rb (3)
M lib/asset_sync/storage.rb (33)

-- Patch Links --

https://github.com/rumblelabs/asset_sync/pull/20.patch
https://github.com/rumblelabs/asset_sync/pull/20.diff

---

Reply to this email directly or view it on GitHub:
#20

---

David Rice
+44 (0)75 905 38303
http://davidjrice.co.uk

[davidjrice]

I've improved this to detect the configured manifest path from Rails.application.config.assets.manifest if set. So it's only a boolean option on AssetSync. Added some extra logging for visibility as to which process it is using to determine assets to upload. Gave it a test run there and it looks good. Think am ready to cut a new version.

Using: Manifest /app/public/assets/manifest.yml
Uploading: /app/public/assets/application-f8f0ab290a5f852498c17bff3fd4d2d5.css.gz in place of assets/application-f8f0ab290a5f852498c17bff3fd4d2d5.css saving 80.88%
Done.