Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Syslog parser works with any time format #2886

Merged
merged 12 commits into from
Mar 24, 2020
Merged

Syslog parser works with any time format #2886

merged 12 commits into from
Mar 24, 2020
+156 −71

Commits on Mar 16, 2020

  1. Add comment 

    Signed-off-by: Yuta Iwama <ganmacs@gmail.com>
    @ganmacs
    ganmacs committed Mar 16, 2020
    Configuration menu
    Copy the full SHA
    c45e457 View commit details
    Browse the repository at this point in the history

Commits on Mar 17, 2020

  1. split paring time and pri and others 

    Signed-off-by: Yuta Iwama <ganmacs@gmail.com>
    @ganmacs
    ganmacs committed Mar 17, 2020
    Configuration menu
    Copy the full SHA
    096805d View commit details
    Browse the repository at this point in the history

  2. work with time_format 

    Signed-off-by: Yuta Iwama <ganmacs@gmail.com>
    @ganmacs
    ganmacs committed Mar 17, 2020
    Configuration menu
    Copy the full SHA
    325a5b9 View commit details
    Browse the repository at this point in the history

  3. freeze for performance 

    total allocated object 1430 -> 1190

```
require 'fluent/env'
require 'fluent/engine'
require_relative '../lib/fluent/plugin/parser_syslog'
require 'memory_profiler'

parser = Fluent::Plugin::SyslogParser.new
parser.configure(Fluent::Config::Element.new('ROOT', '', { 'keep_time_key' => true }, []))
log = 'Feb 28 12:00:00 192.168.0.1 fluentd[11111]: [error] Syslog test'

GC.start

report = MemoryProfiler.report do
  parser.parse_rfc3164_regex(log) { |t, r|  }
end

report.pretty_print
```

Signed-off-by: Yuta Iwama <ganmacs@gmail.com>
    @ganmacs
    ganmacs committed Mar 17, 2020
    Configuration menu
    Copy the full SHA
    c6c6dd0 View commit details
    Browse the repository at this point in the history

  4. Do not create string object 

    Total allocated: 1190 -> 1070 bytes

```
require 'fluent/env'
require 'fluent/engine'
require_relative '../lib/fluent/plugin/parser_syslog'
require 'memory_profiler'

parser = Fluent::Plugin::SyslogParser.new
parser.configure(Fluent::Config::Element.new('ROOT', '', { 'keep_time_key' => true }, []))
log = 'Feb 28 12:00:00 192.168.0.1 fluentd[11111]: [error] Syslog test'

GC.start

report = MemoryProfiler.report do
  parser.parse_rfc3164_regex(log) { |t, r|  }
end

report.pretty_print

```

Signed-off-by: Yuta Iwama <ganmacs@gmail.com>
    @ganmacs
    ganmacs committed Mar 17, 2020
    Configuration menu
    Copy the full SHA
    09f5d11 View commit details
    Browse the repository at this point in the history

  5. squeeze must be invoke before time_parser.parse 

    Signed-off-by: Yuta Iwama <ganmacs@gmail.com>
    @ganmacs
    ganmacs committed Mar 17, 2020
    Configuration menu
    Copy the full SHA
    abbf77a View commit details
    Browse the repository at this point in the history

  6. m.names returns always same result 

    so make it const to avoid string creation

Total allocated 1030 -> 798

```

require 'fluent/env'
require 'fluent/engine'
require_relative '../lib/fluent/plugin/parser_syslog'
require 'memory_profiler'

parser = Fluent::Plugin::SyslogParser.new
parser.configure(Fluent::Config::Element.new('ROOT', '', { 'keep_time_key' => true }, []))
log = 'Feb 28 12:00:00 192.168.0.1 fluentd[11111]: [error] Syslog test'

GC.start

report = MemoryProfiler.report do
  parser.parse_rfc3164_regex(log) { |t, r|  }
end

report.pretty_print
```

Signed-off-by: Yuta Iwama <ganmacs@gmail.com>
    @ganmacs
    ganmacs committed Mar 17, 2020
    Configuration menu
    Copy the full SHA
    86e67dc View commit details
    Browse the repository at this point in the history

  7. write directory to avoid string creation 

    Signed-off-by: Yuta Iwama <ganmacs@gmail.com>
    @ganmacs
    ganmacs committed Mar 17, 2020
    Configuration menu
    Copy the full SHA
    8ed7dde View commit details
    Browse the repository at this point in the history

  8. Avoid to create MatachData object 

    Signed-off-by: Yuta Iwama <ganmacs@gmail.com>
    @ganmacs
    ganmacs committed Mar 17, 2020
    Configuration menu
    Copy the full SHA
    2fb9979 View commit details
    Browse the repository at this point in the history

Commits on Mar 18, 2020

  1. Use Regexp#names instead of write it as is 

    Signed-off-by: Yuta Iwama <ganmacs@gmail.com>
    @ganmacs
    ganmacs committed Mar 18, 2020
    Configuration menu
    Copy the full SHA
    f000aa2 View commit details
    Browse the repository at this point in the history

  2. remove useless insertion and deletion 

    Signed-off-by: Yuta Iwama <ganmacs@gmail.com>
    @ganmacs
    ganmacs committed Mar 18, 2020
    Configuration menu
    Copy the full SHA
    5510b8f View commit details
    Browse the repository at this point in the history

Commits on Mar 23, 2020

  1. Remove unused capture 

    Signed-off-by: Yuta Iwama <ganmacs@gmail.com>
    @ganmacs
    ganmacs committed Mar 23, 2020
    Configuration menu
    Copy the full SHA
    8c6b8bd View commit details
    Browse the repository at this point in the history