Skip to content

Commit

Permalink
Merge pull request #3095 from fluent/enable-timkey-comparison-for-all…
Browse files Browse the repository at this point in the history
…-platform

buffer: Enable optimization for metadata comparison in all platform
  • Loading branch information
repeatedly committed Nov 13, 2020
2 parents 7e88922 + f78c873 commit 2088f16
Showing 1 changed file with 2 additions and 21 deletions.
23 changes: 2 additions & 21 deletions lib/fluent/plugin/buffer.rb
Original file line number Diff line number Diff line change
Expand Up @@ -143,33 +143,14 @@ def <=>(o)
end
end

# timekey should be unixtime as usual.
# So, unixtime should be bigger than 2^30 - 1 (= 1073741823) nowadays.
# We should check object_id stability to use object_id as optimization for comparing operations.
# e.g.)
# irb> Time.parse("2020/07/31 18:30:00+09:00").to_i
# => 1596187800
# irb> Time.parse("2020/07/31 18:30:00+09:00").to_i > 2**30 -1
# => true
def self.enable_optimize?
a1 = 2**30 - 1
a2 = 2**30 - 1
b1 = 2**62 - 1
b2 = 2**62 - 1
(a1.object_id == a2.object_id) && (b1.object_id == b2.object_id)
end

# This is an optimization code. Current Struct's implementation is comparing all data.
# https://github.com/ruby/ruby/blob/0623e2b7cc621b1733a760b72af246b06c30cf96/struct.c#L1200-L1203
# Actually this overhead is very small but this class is generated *per chunk* (and used in hash object).
# This means that this class is one of the most called object in Fluentd.
# See https://github.com/fluent/fluentd/pull/2560
# But, this optimization has a side effect on Windows and 32bit environment(s) due to differing object_id.
# This difference causes flood of buffer files.
# So, this optimization should be enabled on `enable_optimize?` as true platforms.
def hash
timekey.object_id
end if enable_optimize?
timekey.hash
end
end

# for tests
Expand Down

0 comments on commit 2088f16

Please sign in to comment.