Skip to content

Commit

Permalink
[match] prevent directory download from s3 (#20975)
Browse files Browse the repository at this point in the history
* fix: prevent directory download from s3

* chore: add test to verify file-like s3 downloads

* Update match/lib/match/storage/s3_storage.rb

Co-authored-by: Roger Oba <rogerluan.oba@gmail.com>

---------

Co-authored-by: Roger Oba <rogerluan.oba@gmail.com>
  • Loading branch information
markhomoki and rogerluan committed Aug 30, 2023
1 parent 65db20a commit ab9fb32
Show file tree
Hide file tree
Showing 2 changed files with 18 additions and 0 deletions.
6 changes: 6 additions & 0 deletions match/lib/match/storage/s3_storage.rb
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,12 @@ def download
self.working_directory = Dir.mktmpdir

s3_client.find_bucket!(s3_bucket).objects(prefix: s3_object_prefix).each do |object|

# Prevent download if the file path is a directory.
# We need to check if string ends with "/" instead of using `File.directory?` because
# the string represent a remote location, not a local file in disk.
next if object.key.end_with?("/")

file_path = strip_s3_object_prefix(object.key) # :s3_object_prefix:team_id/path/to/file

# strip s3_prefix from file_path
Expand Down
12 changes: 12 additions & 0 deletions match/spec/storage/s3_storage_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,18 @@

subject.download
end

it 'downloads only file-like objects and skips folder-like objects' do
valid_object = files_to_download[0]
invalid_object = instance_double('Aws::S3::Object', key: 'ABCDEFG/certs/development/')

allow(s3_client).to receive_message_chain(:find_bucket!, :objects).and_return([valid_object, invalid_object])

expect(valid_object).to receive(:download_file).with("#{working_directory}/#{valid_object.key}")
expect(invalid_object).not_to receive(:download_file).with("#{working_directory}/#{invalid_object.key}")

subject.download
end
end
end
end

0 comments on commit ab9fb32

Please sign in to comment.