Multiple tiny AWS related fixes #1083
Conversation
oavdeev
requested review from
davidheryanto,
khorshuheng,
pyalex,
woop and
zhilingc
as code owners
|
/kind bug |
| @@ -912,7 +912,7 @@ def get_historical_features( | |||
| else entity_staging_uri.netloc | |||
| ) | |||
| staging_client.upload_file( | |||
| df_export_path.name, bucket, entity_staging_uri.path | |||
| df_export_path.name, bucket, entity_staging_uri.path.lstrip("/") | |||
There was a problem hiding this comment.
shouldn't that strip be part of implementation (GS, s3)?
There was a problem hiding this comment.
We can put strip into both implementations too (gcp actually has it already). It just goes a bit against S3 (and i suspect GCP) semantics where /a and a are separate objects
There was a problem hiding this comment.
I will look into the GCP part again.
oavdeev
force-pushed
the
aws-more-fixes
branch
2 times, most recently
from
a8a4eff
to
2b79209
Compare
|
/retest |
oavdeev
force-pushed
the
aws-more-fixes
branch
from
ea71985
to
4420fce
Compare
|
/retest |
|
/lgtm |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: khorshuheng, oavdeev The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
* Fix object path math in client staging uploader * Add random suffix to the historical retrieval output location * EMR job now actually checks for status * add boto3 dep Signed-off-by: Oleg Avdeev <oleg.v.avdeev@gmail.com>
Signed-off-by: Oleg Avdeev <oleg.v.avdeev@gmail.com>
oavdeev
force-pushed
the
aws-more-fixes
branch
from
4420fce
to
1102f04
Compare
|
/lgtm |
|
/test test-end-to-end-auth |
What this PR does / why we need it:
Which issue(s) this PR fixes:
Fixes #
Does this PR introduce a user-facing change?: