Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement the "source" side of profile parsing #594

Merged
merged 18 commits into from
Jul 21, 2021
Merged

Implement the "source" side of profile parsing #594

merged 18 commits into from
Jul 21, 2021

Conversation

rcoh
Copy link
Collaborator

@rcoh rcoh commented Jul 20, 2021

Issue #, if available: #210
Description of changes: This PR makes several steps towards support of profile file parsing and is probably best reviewed commit-by-commit for clarity.

  • Establish common abstraction for environment and fs
  • Update region provider to use that abstraction
  • update credentials provider to use that abstraction
  • Implement the "source" side of profile parsing (which includes windows specific tests)
  • add github action to run tests on windows

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@rcoh rcoh requested a review from jdisanti July 20, 2021 22:45
don't run rustfmt on windows
}
}

/// Process Environment Abstraction
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recommend adding more details on what this means. I had to read the code to realize it's an abstraction over environment variables.

@@ -9,6 +9,12 @@ license = "Apache-2.0"

[dependencies]
lazy_static = "1"
tracing = "0.1.26"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be less specific on the version number?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah good call, when we use tracing as a library 0.1 is probably what we want

Comment on lines 6 to 9
//! Abstractions for testing code that interacts with the process environment:
//! - Determining the current time
//! - Reading environment variables
//! - Reading from the file system
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does this look in the module summary line in the generated docs when it has bullet points like this?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just renders as bullets on the module page

aws/rust-runtime/aws-types/src/environment.rs Outdated Show resolved Hide resolved
aws/rust-runtime/aws-types/src/environment.rs Outdated Show resolved Hide resolved
aws/rust-runtime/aws-types/src/environment.rs Outdated Show resolved Hide resolved
}
}

pub fn read(&self, path: impl AsRef<Path>) -> std::io::Result<Vec<u8>> {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be called read_to_end to allow for room for others in the future?

}
}

impl From<HashMap<String, String>> for ProcessEnvironment {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this implement From<Iterator<(String, String)>>?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe? although then you would need to convert back to an iterator if you had a hashmap, so I think HashMap is probably the right thing here

/// Load a [Source](Source) from a given environment and filesystem.
pub fn load(proc_env: &environment::ProcessEnvironment, fs: &environment::Fs) -> Source {
let config = tracing::info_span!("load_config_file")
.in_scope(|| read(&fs, &proc_env, "~/.aws/config", "AWS_CONFIG_FILE"));
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wasn't obvious that read was replacing ~ on different OSes. Should we just pass in the .aws/config part and just always look in the user directory for it? Or have a lower-level read function that takes a full path, and a higher level one that resolves it?

Better yet, should we have an enum to represent the config and credentials files with a path() function?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there's some wrinkles here that we also need to resolve the home directory from what the user might give us in an environment variable

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so having implicit home directory searching is probably going to create more complexity (since we still need generic home directory replacement)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but I can split up the path resolution and the home directory resolution for clarity.

@@ -4,6 +4,8 @@
*/

pub mod build_metadata;
pub mod environment;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we make these doc hidden and name this something like internal_env to avoid requiring it to have a stable interface?

@tschuett
Copy link

LLVM talks about virtual filesystems. There is a virtual FileSystem, an InMemoryFileSystem and a RealFileSystem:
https://github.com/llvm/llvm-project/blob/main/llvm/include/llvm/Support/VirtualFileSystem.h

@rcoh rcoh requested a review from jdisanti July 21, 2021 19:59
Copy link
Collaborator

@jdisanti jdisanti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Minor comment touch ups.

aws/rust-runtime/aws-types/src/os_shim_internal.rs Outdated Show resolved Hide resolved
.github/workflows/ci.yaml Outdated Show resolved Hide resolved
rcoh and others added 2 commits July 21, 2021 17:12
@rcoh rcoh enabled auto-merge (squash) July 21, 2021 21:13
@rcoh rcoh disabled auto-merge July 21, 2021 21:36
@rcoh rcoh merged commit 6e3ebaa into main Jul 21, 2021
@rcoh rcoh deleted the profile-source branch July 21, 2021 21:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants