-
Notifications
You must be signed in to change notification settings - Fork 2.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Sftp remote file support #4750
Merged
Merged
Sftp remote file support #4750
Changes from 4 commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
8c8060e
Adding the provider and spec files for sftp
jk185160 360d506
Adding base implementation and the start of unit test coverage
jk185160 c0d0e1c
Finishing off the sftp unit tests. Added some checks to the sftp prov…
jk185160 2420f2e
fixing rubocop violations, mostly string format, in the sftp provider…
jk185160 da54509
removing zombied references and migrating validation logic directly i…
jk185160 88e2e05
updating copyright info
jk185160 3c5c75e
removing an unnecessary comment with an incorrect reference
jk185160 5865d46
removing a superflous assignment in the sftp provider
jk185160 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,118 @@ | ||
# | ||
# Author:: Jesse Campbell (<hikeit@gmail.com>) | ||
# Copyright:: Copyright 2013-2016, Jesse Campbell | ||
# License:: Apache License, Version 2.0 | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
# | ||
|
||
require "uri" | ||
require "tempfile" | ||
require "net/sftp" | ||
require "chef/provider/remote_file" | ||
require "chef/file_content_management/tempfile" | ||
|
||
class Chef | ||
class Provider | ||
class RemoteFile | ||
class SFTP | ||
attr_reader :uri | ||
attr_reader :new_resource | ||
attr_reader :current_resource | ||
|
||
def initialize(uri, new_resource, current_resource) | ||
@uri = uri | ||
@new_resource = new_resource | ||
@current_resource = current_resource | ||
validate_path! | ||
validate_userinfo! | ||
end | ||
|
||
def hostname | ||
@uri.host | ||
end | ||
|
||
def port | ||
@uri.port | ||
end | ||
|
||
def user | ||
URI.unescape(uri.user) | ||
end | ||
|
||
def filename | ||
parse_path if @filename.nil? | ||
@filename | ||
end | ||
|
||
def fetch | ||
get | ||
end | ||
|
||
private | ||
|
||
def sftp | ||
host = port ? "#{hostname}:#{port}" : hostname | ||
@sftp ||= Net::SFTP.start(host, user, :password => pass) | ||
end | ||
|
||
def pass | ||
URI.unescape(uri.password) | ||
end | ||
|
||
def validate_path! | ||
parse_path | ||
end | ||
|
||
def validate_userinfo! | ||
if uri.userinfo | ||
unless uri.user | ||
raise ArgumentError, "no user name provided in the sftp URI" | ||
end | ||
unless uri.password | ||
raise ArgumentError, "no password provided in the sftp URI" | ||
end | ||
else | ||
raise ArgumentError, "no userinfo provided in the sftp URI" | ||
end | ||
end | ||
|
||
# Fetches using Net::FTP, returns a Tempfile with the content | ||
def get | ||
tempfile = | ||
Chef::FileContentManagement::Tempfile.new(@new_resource).tempfile | ||
sftp.download!(uri.path, tempfile.path) | ||
tempfile.close if tempfile | ||
tempfile | ||
end | ||
|
||
def parse_path | ||
path = uri.path.sub(%r{\A/}, "%2F") # re-encode the beginning slash because uri library decodes it. | ||
directories = path.split(%r{/}, -1) | ||
directories.each {|d| | ||
d.gsub!(/%([0-9A-Fa-f][0-9A-Fa-f])/) { [$1].pack("H2") } | ||
} | ||
unless filename = directories.pop | ||
raise ArgumentError, "no filename: #{path.inspect}" | ||
end | ||
if filename.length == 0 || filename.end_with?( "/" ) | ||
raise ArgumentError, "no filename: #{path.inspect}" | ||
end | ||
|
||
@directories, @filename = directories, filename | ||
end | ||
|
||
end | ||
end | ||
end | ||
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,150 @@ | ||
# | ||
# Author:: John Kerry (<john@kerryhouse.net>) | ||
# Copyright:: Copyright 2013-2016, John Kerry | ||
# License:: Apache License, Version 2.0 | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
# | ||
|
||
require "spec_helper" | ||
|
||
describe Chef::Provider::RemoteFile::SFTP do | ||
#built out dependencies | ||
let(:enclosing_directory) { | ||
canonicalize_path(File.expand_path(File.join(CHEF_SPEC_DATA, "templates"))) | ||
} | ||
let(:resource_path) { | ||
canonicalize_path(File.expand_path(File.join(enclosing_directory, "seattle.txt"))) | ||
} | ||
|
||
let(:new_resource) do | ||
r = Chef::Resource::RemoteFile.new("remote file sftp backend test (new resource)") | ||
r.path(resource_path) | ||
r | ||
end | ||
|
||
let(:current_resource) do | ||
Chef::Resource::RemoteFile.new("remote file sftp backend test (current resource)'") | ||
end | ||
|
||
let(:uri) { URI.parse("sftp://conan:cthu1hu@opscode.com/seattle.txt") } | ||
|
||
let(:sftp) do | ||
sftp = double(Net::SFTP, {}) | ||
allow(sftp).to receive(:download!) | ||
sftp | ||
end | ||
|
||
let(:tempfile_path) { "/tmp/somedir/remote-file-sftp-backend-spec-test" } | ||
|
||
let(:tempfile) do | ||
t = StringIO.new | ||
allow(t).to receive(:path).and_return(tempfile_path) | ||
t | ||
end | ||
|
||
before(:each) do | ||
allow(Net::SFTP).to receive(:start).with(any_args).and_return(sftp) | ||
allow(Tempfile).to receive(:new).and_return(tempfile) | ||
end | ||
describe "on initialization without user and password provided in the URI" do | ||
it "throws an argument exception with no userinfo is given" do | ||
uri.userinfo = nil | ||
uri.password = nil | ||
uri.user = nil | ||
expect { Chef::Provider::RemoteFile::SFTP.new(uri, new_resource, current_resource) }.to raise_error(ArgumentError) | ||
end | ||
|
||
it "throws an argument exception with no user name is given" do | ||
uri.userinfo = ":cthu1hu" | ||
uri.password = "cthu1hu" | ||
uri.user = nil | ||
expect { Chef::Provider::RemoteFile::SFTP.new(uri, new_resource, current_resource) }.to raise_error(ArgumentError) | ||
end | ||
|
||
it "throws an argument exception with no password is given" do | ||
uri.userinfo = "conan:" | ||
uri.password = nil | ||
uri.user = "conan" | ||
expect { Chef::Provider::RemoteFile::SFTP.new(uri, new_resource, current_resource) }.to raise_error(ArgumentError) | ||
end | ||
|
||
end | ||
|
||
describe "on initialization with user and password provided in the URI" do | ||
|
||
it "throws an argument exception when no path is given" do | ||
uri.path = "" | ||
expect { Chef::Provider::RemoteFile::SFTP.new(uri, new_resource, current_resource) }.to raise_error(ArgumentError) | ||
end | ||
|
||
it "throws an argument exception when only a / is given" do | ||
uri.path = "/" | ||
expect { Chef::Provider::RemoteFile::SFTP.new(uri, new_resource, current_resource) }.to raise_error(ArgumentError) | ||
end | ||
|
||
it "throws an argument exception when no filename is given" do | ||
uri.path = "/the/whole/path/" | ||
expect { Chef::Provider::RemoteFile::SFTP.new(uri, new_resource, current_resource) }.to raise_error(ArgumentError) | ||
end | ||
|
||
end | ||
|
||
describe "when fetching the object" do | ||
|
||
let(:cache_control_data) { Chef::Provider::RemoteFile::CacheControlData.new(uri) } | ||
let(:current_resource_checksum) { "e2a8938cc31754f6c067b35aab1d0d4864272e9bf8504536ef3e79ebf8432305" } | ||
|
||
subject(:fetcher) { Chef::Provider::RemoteFile::SFTP.new(uri, new_resource, current_resource) } | ||
|
||
before do | ||
current_resource.checksum(current_resource_checksum) | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this looks copied from the ftp class
the thing is that in
#get
you've removed the one reference to this method. thefilename
inparse_path
is actually a local variable that gets constructed (hella confusingly -- the ftp class could use a bit of cleanup to make it clearer) and therefore both this#filename
code and the#parse_path
code are unused in this class that you've written and could simply be deleted.