Segfault on simple query (when GC runs ?) #226
Comments
|
Note that we tried the latest google-protobuf 3.18.0.rc.1 (which has moved the DSL to pure ruby in protocolbuffers/protobuf#8850), it still segfaults |
|
Something is very odd. pg_query/lib/pg_query/parse.rb Line 103 in ccf4e29 The issue description points to a subselect item also being invalid in pg_query/lib/pg_query/parse.rb Line 238 in ccf4e29 This makes me wonder if this is actually a google-protobuf memory issue dealing with repeated fields, perhaps similar to protocolbuffers/protobuf#8559 If I hack pg_query to use google-protobuf v3.14.0, I don't see a seg fault, but for some reason the loop runs 15 times instead of 2. |
|
The sample reproduction script runs fine, without segfault in |
|
It still crashes If I take the C extension diff --git a/lib/pg_query/parse.rb b/lib/pg_query/parse.rb
index 810c0f4..2015587 100644
--- a/lib/pg_query/parse.rb
+++ b/lib/pg_query/parse.rb
@@ -1,18 +1,24 @@
module PgQuery
def self.parse(query)
- result, stderr = parse_protobuf(query)
+ result = File.open('/tmp/proto.bin', 'rb').read
begin
result = PgQuery::ParseResult.decode(result)
rescue Google::Protobuf::ParseError => e
raise PgQuery::ParseError.new(format('Failed to parse tree: %s', e.message), __FILE__, __LINE__, -1)
end
warnings = []
- stderr.each_line do |line|
- next unless line[/^WARNING/]
- warnings << line.strip
- end
+# stderr.each_line do |line|
+# next unless line[/^WARNING/]
+# warnings << line.strip
+# end
PgQuery::ParserResult.new(query, result, warnings)
endI'm also seeing seg faults due to: This suggests the garbage collector is visiting some Ruby pointer that's marked as I'm trying to reproduce this in the Ruby protobuf tests, but I'm not getting a crash yet. |
|
I suspect this is an issue with google-protobuf rather than the |
|
Closing in favor of the google-protobuf issue since there is a reproduction step that doesn't even involve pg_query: protocolbuffers/protobuf#8938 (comment) |
|
BTW, I was able to workaround this issue in the meantime in our CI by disabling GC temporarily : begin
GC.disable
tables = PgQuery.parse(sql).tables
ensure
GC.enable
end |
|
I submitted #227 to workaround the Ruby bug described in https://bugs.ruby-lang.org/issues/18140#note-2. |
We at GitLab are using PgQuery to parse sql queries (to extract table name information) in our CI suite. Frequently, this segfaults. Please see https://gitlab.com/gitlab-org/gitlab/-/issues/339475 for full details
https://gitlab.com/gitlab-org/gitlab/blob/40f9110d0803f84d639889c14cf619a126581359/spec/support/database/prevent_cross_joins.rb#L33 is how we use it.
We suspect that garbage collection during our CI runs might be causing these segfaults. I managed to reproduce a segfault with this small script below.
It seems any query with three or more
ANDin theWHEREcan cause this segfault underGC.stressconditions./cc @stanhu
Segfault:
The text was updated successfully, but these errors were encountered: