Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Recursively put files sometimes #10

Merged
merged 8 commits into from
Feb 21, 2018
Merged

Recursively put files sometimes #10

merged 8 commits into from
Feb 21, 2018

Conversation

garyluu
Copy link
Contributor

@garyluu garyluu commented Feb 13, 2018

Allows uploading directories for issue #9

Only allowed if the destination ends with a slash. S3cmd would complain about uploading otherwise.

@denis-yuen
Copy link
Member

FYI, as discussed, this could use testing.
Feel free to take inspiration/copy the tests for event consumer in the main dockstore repo to use localstack for testing.

Copy link
Member

@denis-yuen denis-yuen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

testing

* Run localstack

* Add s3cmd config file

* Test enum

* Install s3cmd

* Add test

* Install s3cmd with pip

* Change s3cmd client location

* Wait for localstack to start
Copy link
Member

@denis-yuen denis-yuen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

with comment

.travis.yml Outdated
jdk:
- oraclejdk8
script:
- mvn clean install
before_install:
- pip install --upgrade pip
- pip install --user s3cmd
Copy link
Member

@denis-yuen denis-yuen Feb 14, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets get versions on these so they do not break unexpectedly

@denis-yuen denis-yuen self-requested a review February 20, 2018 16:42
Copy link
Member

@denis-yuen denis-yuen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor question

private void setConfiguration() throws Exception {
HashMap hm = new HashMap();
hm.put("client", "/home/travis/.local/bin/s3cmd");
// hm.put("client", "/usr/local/bin/s3cmd");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this have to be manually uncommented to run locally?
If yes, just use https://docs.oracle.com/javase/7/docs/api/java/nio/file/Files.html#exists(java.nio.file.Path,%20java.nio.file.LinkOption...)
If no (i.e. this is superfluous, just delete it)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It depends on where s3cmd is installed. I happen to have mine installed in /usr/local/bin/s3cmd. Travis by default installs it in /home/travis/.local/bin/s3cmd instead. If the local user doesn't even have s3cmd installed, the tests would fail also.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I would just check the two locations using Files.exists and fail otherwise indicating just that

@garyluu garyluu merged commit 3fa4995 into develop Feb 21, 2018
@garyluu garyluu deleted the feature/directory branch February 21, 2018 20:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants