New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[supply] parallel uploads for meta per language #21474
Changes from 6 commits
5cca70f
1530c8f
4c6dbd9
8ef972f
72e2fb6
5d4c89a
2e0ca94
548b293
3307aa3
f5b6c1b
eec2964
f71a7c1
e5a4030
ed3b77d
31e9866
fd7568c
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,6 +1,10 @@ | ||
require 'fastlane_core' | ||
|
||
module Supply | ||
# rubocop:disable Metrics/ClassLength | ||
class Uploader | ||
UploadJob = Struct.new(:language, :version_code) | ||
|
||
def perform_upload | ||
FastlaneCore::PrintTable.print_values(config: Supply.config, hide_keys: [:issuer], mask_keys: [:json_key_data], title: "Summary for supply #{Fastlane::VERSION}") | ||
|
||
|
@@ -81,25 +85,32 @@ def perform_upload_meta(version_codes, track_name) | |
version_codes.to_s == "" | ||
end | ||
|
||
release_notes = [] | ||
upload_worker = FastlaneCore::QueueWorker.new do |job| | ||
UI.message("Preparing uploads for language '#{job.language}'...") | ||
start_time = Time.now | ||
listing = client.listing_for_language(job.language) | ||
upload_metadata(job.language, listing) unless Supply.config[:skip_upload_metadata] | ||
upload_images(job.language) unless Supply.config[:skip_upload_images] | ||
upload_screenshots(job.language) unless Supply.config[:skip_upload_screenshots] | ||
release_notes << upload_changelog(job.language, job.version_code) unless Supply.config[:skip_upload_changelogs] | ||
UI.message("Uploaded all items for language '#{job.language}'... (#{Time.now - start_time} secs)") | ||
rescue => error | ||
UI.abort_with_message!("#{job.language} - #{error}") | ||
end | ||
|
||
version_codes.each do |version_code| | ||
UI.user_error!("Could not find folder #{metadata_path}") unless File.directory?(metadata_path) | ||
|
||
track, release = fetch_track_and_release!(track_name, version_code) | ||
UI.user_error!("Unable to find the requested track - '#{Supply.config[:track]}'") unless track | ||
UI.user_error!("Could not find release for version code '#{version_code}' to update changelog") unless release | ||
|
||
release_notes = [] | ||
all_languages.each do |language| | ||
next if language.start_with?('.') # e.g. . or .. or hidden folders | ||
UI.message("Preparing to upload for language '#{language}'...") | ||
|
||
listing = client.listing_for_language(language) | ||
|
||
upload_metadata(language, listing) unless Supply.config[:skip_upload_metadata] | ||
upload_images(language) unless Supply.config[:skip_upload_images] | ||
upload_screenshots(language) unless Supply.config[:skip_upload_screenshots] | ||
release_notes << upload_changelog(language, version_code) unless Supply.config[:skip_upload_changelogs] | ||
end | ||
upload_worker.batch_enqueue( | ||
# skip . or .. or hidden folders | ||
all_languages.reject { |lang| lang.start_with?('.') }.map { |lang| UploadJob.new(lang, version_code) } | ||
) | ||
upload_worker.start | ||
|
||
upload_changelogs(release_notes, release, track, track_name) unless release_notes.empty? | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. When reviewing this I initially wondered if I'm telling you this because I feel like this PR should definitively benefit some additional unit tests (aka specs) to help us guarantee the behavior. Especially given how tricky multi-threaded code can sometimes be, with hidden edge cases and all, better get some additional assurance 🙂 Of course, the fact that There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Writing tests is always a good idea. I'll have a look at your suggestions above and take some time this week make that happen 🧪 |
||
end | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One difference in behavior before/after this change is that before,
release_notes
was reset to[]
on each iteration of theversion_codes
loop, while now it's not, and thus is common to allversion_codes
.That might break the logic (and potentially mix up the release notes published when there are different version codes with different
changelogs/*.txt
files.You might thus want to keep this reset of
release_notes = []
inside the loop to cover for this case.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nicely observed 😄 I haven't been testing the multi version codes scenario. Will fix.