Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make App::about accept Cow #2504

Closed
wants to merge 2 commits into from
Closed

Make App::about accept Cow #2504

wants to merge 2 commits into from

Conversation

d-e-s-o
Copy link

@d-e-s-o d-e-s-o commented May 27, 2021

Draft only. See #2150

Copy link
Member

@pksunkara pksunkara left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before agreeing to merge this, was there any performance issues?

src/output/help.rs Outdated Show resolved Hide resolved
@d-e-s-o
Copy link
Author

d-e-s-o commented May 28, 2021

This pull request is not meant to be merged. The example changes make no sense to be in the tree and are just there to show what was not possible beforehand. This is only one API adjusted, I suppose others remain with similar problems. It's meant to show the ergonomics of the API, as you asked for.

was there any performance issues?

I don't know. I haven't done an analysis. But then, it's a cold method being adjusted, right?

@pksunkara
Copy link
Member

The ergonomics are definitely more complicated now. But the API is more useful now. I was wondering what the performance would say.

But then, it's a cold method being adjusted, right?

This line is not cold and is part of the parsing logic.

@d-e-s-o
Copy link
Author

d-e-s-o commented Jun 2, 2021

What's the performance measurement setup you folks use?

@pksunkara
Copy link
Member

We use criterion, and we have benches setup already. You don't need to test the dynamic string. Just provide a static string to about and let's see if the dereferencing effects the performance.

@d-e-s-o
Copy link
Author

d-e-s-o commented Jun 3, 2021

Seems as if there were a couple of regressions detected:

$ cargo bench -- --baseline at-fbd338ce2
    Finished bench [optimized] target(s) in 0.05s
     Running unittests (clap/target/release/deps/01_default-77d3cfc757e6614a)
build_empty             time:   [260.15 ns 260.18 ns 260.21 ns]
                        change: [+1.0894% +1.6513% +2.6404%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 20 outliers among 100 measurements (20.00%)
  2 (2.00%) high mild
  18 (18.00%) high severe

parse_empty             time:   [649.15 ns 649.26 ns 649.42 ns]
                        change: [-0.7453% -0.6482% -0.5350%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 7 outliers among 100 measurements (7.00%)
  1 (1.00%) high mild
  6 (6.00%) high severe

     Running unittests (clap/target/release/deps/02_simple-0c31f7d3583bef15)
parse_simple_with_complex
                        time:   [2.6822 us 2.6892 us 2.6962 us]
                        change: [+4.6288% +4.7907% +4.9561%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 18 outliers among 100 measurements (18.00%)
  2 (2.00%) high mild
  16 (16.00%) high severe

parse_simple_with_pos   time:   [2.0311 us 2.0315 us 2.0319 us]
                        change: [+3.9027% +4.1918% +4.5737%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 11 outliers among 100 measurements (11.00%)
  6 (6.00%) high mild
  5 (5.00%) high severe

parse_simple_with_opt   time:   [2.1295 us 2.1339 us 2.1389 us]
                        change: [+2.1138% +2.2645% +2.4165%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 15 outliers among 100 measurements (15.00%)
  2 (2.00%) high mild
  13 (13.00%) high severe

parse_simple_with_flag  time:   [1.8876 us 1.8878 us 1.8881 us]
                        change: [+1.2972% +1.3533% +1.4085%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 6 outliers among 100 measurements (6.00%)
  1 (1.00%) low mild
  3 (3.00%) high mild
  2 (2.00%) high severe

build_with_pos_ref      time:   [408.66 ns 408.75 ns 408.88 ns]
                        change: [+1.0168% +1.1128% +1.2479%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 15 outliers among 100 measurements (15.00%)
  4 (4.00%) high mild
  11 (11.00%) high severe

build_with_pos          time:   [342.98 ns 343.11 ns 343.25 ns]
                        change: [+2.1880% +2.2076% +2.2288%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 16 outliers among 100 measurements (16.00%)
  1 (1.00%) low severe
  1 (1.00%) low mild
  2 (2.00%) high mild
  12 (12.00%) high severe

build_with_opt_ref      time:   [468.80 ns 468.85 ns 468.91 ns]
                        change: [-0.7866% -0.5859% -0.2462%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 18 outliers among 100 measurements (18.00%)
  2 (2.00%) low severe
  3 (3.00%) low mild
  1 (1.00%) high mild
  12 (12.00%) high severe

build_with_opt          time:   [390.07 ns 390.09 ns 390.11 ns]
                        change: [-0.1618% -0.1501% -0.1419%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 7 outliers among 100 measurements (7.00%)
  1 (1.00%) low severe
  6 (6.00%) high severe

build_with_flag_ref     time:   [431.91 ns 432.17 ns 432.54 ns]
                        change: [+0.5768% +0.7203% +0.9231%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 12 outliers among 100 measurements (12.00%)
  1 (1.00%) low mild
  11 (11.00%) high severe

build_with_flag         time:   [363.11 ns 363.20 ns 363.35 ns]
                        change: [+0.6765% +0.7096% +0.7509%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 8 outliers among 100 measurements (8.00%)
  1 (1.00%) low mild
  3 (3.00%) high mild
  4 (4.00%) high severe

build_simple            time:   [651.95 ns 652.47 ns 653.16 ns]
                        change: [+9.5159% +9.8012% +10.163%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 15 outliers among 100 measurements (15.00%)
  5 (5.00%) high mild
  10 (10.00%) high severe

     Running unittests (clap/target/release/deps/03_complex-9da7565caa16701b)
build_from_usage        time:   [3.6927 us 3.9319 us 4.2065 us]
                        change: [+5.1766% +8.3475% +11.987%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 11 outliers among 100 measurements (11.00%)
  1 (1.00%) high mild
  10 (10.00%) high severe

build_from_builder      time:   [4.0159 us 4.2768 us 4.5435 us]
                        change: [+102.98% +108.69% +115.01%] (p = 0.00 < 0.05)
                        Performance has regressed.

build_from_macros       time:   [2.6897 us 2.7240 us 2.7699 us]
                        change: [+24.183% +28.464% +32.539%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild

parse_complex           time:   [6.1918 us 6.2066 us 6.2254 us]
                        change: [-0.6136% -0.3390% -0.0222%] (p = 0.02 < 0.05)
                        Change within noise threshold.

parse_complex_with_flag time:   [7.5305 us 7.5473 us 7.5653 us]
                        change: [-0.5571% -0.3308% -0.1173%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 9 outliers among 100 measurements (9.00%)
  5 (5.00%) high mild
  4 (4.00%) high severe

parse_complex_with_opt  time:   [7.5846 us 7.5906 us 7.5966 us]
                        change: [+1.2550% +1.3919% +1.5391%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
  2 (2.00%) high mild
  2 (2.00%) high severe

parse_complex_with_pos  time:   [7.4088 us 7.4130 us 7.4172 us]
                        change: [-0.2846% +0.0114% +0.4165%] (p = 0.95 > 0.05)
                        No change in performance detected.
Found 7 outliers among 100 measurements (7.00%)
  7 (7.00%) high severe

parse_complex_with_sc   time:   [7.9733 us 7.9745 us 7.9760 us]
                        change: [+2.2805% +2.4391% +2.7329%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) low mild
  1 (1.00%) high mild
  1 (1.00%) high severe

parse_complex_with_sc_flag
                        time:   [9.0959 us 9.1003 us 9.1050 us]
                        change: [+3.6017% +3.9207% +4.2550%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 16 outliers among 100 measurements (16.00%)
  1 (1.00%) high mild
  15 (15.00%) high severe

parse_complex_with_sc_opt
                        time:   [8.8688 us 8.8771 us 8.8858 us]
                        change: [+0.8658% +0.9505% +1.0380%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 6 outliers among 100 measurements (6.00%)
  3 (3.00%) low mild
  3 (3.00%) high mild

parse_complex_with_sc_pos
                        time:   [8.8681 us 8.8712 us 8.8741 us]
                        change: [+0.1477% +0.2060% +0.2771%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high severe

parse_complex1          time:   [10.940 us 10.947 us 10.957 us]
                        change: [+0.2229% +0.2901% +0.3634%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) high mild
  1 (1.00%) high severe

parse_complex2          time:   [12.313 us 12.318 us 12.325 us]
                        change: [+3.0372% +3.1324% +3.2616%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
  2 (2.00%) high mild
  2 (2.00%) high severe

parse_args_negate_scs   time:   [12.101 us 12.105 us 12.109 us]
                        change: [+2.6315% +2.6703% +2.7095%] (p = 0.00 < 0.05)
                        Performance has regressed.

parse_complex_with_sc_complex
                        time:   [9.9415 us 9.9457 us 9.9501 us]
                        change: [+0.6359% +0.8181% +0.9368%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild

     Running unittests (clap/target/release/deps/04_new_help-e61208e356f9c7dc)
example1                time:   [12.980 us 12.993 us 13.010 us]
                        change: [+1.1847% +1.4643% +1.8111%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 6 outliers among 100 measurements (6.00%)
  1 (1.00%) high mild
  5 (5.00%) high severe

example2                time:   [5.3614 us 5.3668 us 5.3736 us]
                        change: [-0.0713% +0.1152% +0.3330%] (p = 0.30 > 0.05)
                        No change in performance detected.
Found 17 outliers among 100 measurements (17.00%)
  5 (5.00%) high mild
  12 (12.00%) high severe

example3                time:   [10.580 us 10.585 us 10.592 us]
                        change: [-1.2469% -0.5805% +0.0011%] (p = 0.07 > 0.05)
                        No change in performance detected.
Found 6 outliers among 100 measurements (6.00%)
  2 (2.00%) high mild
  4 (4.00%) high severe

example4                time:   [7.6307 us 7.6333 us 7.6366 us]
                        change: [-0.1877% -0.1135% -0.0263%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 9 outliers among 100 measurements (9.00%)
  5 (5.00%) high mild
  4 (4.00%) high severe

example5                time:   [4.7847 us 4.7865 us 4.7881 us]
                        change: [-0.5919% -0.4821% -0.2988%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high severe

example6                time:   [5.5932 us 5.5947 us 5.5964 us]
                        change: [-0.7377% -0.6823% -0.6235%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) high mild
  2 (2.00%) high severe

example7                time:   [8.1238 us 8.1261 us 8.1284 us]
                        change: [-1.5458% -1.4443% -1.2693%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe

example8                time:   [8.1694 us 8.1709 us 8.1725 us]
                        change: [-0.0585% -0.0216% +0.0230%] (p = 0.28 > 0.05)
                        No change in performance detected.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) low mild
  2 (2.00%) high severe

example10               time:   [7.3464 us 7.3474 us 7.3487 us]
                        change: [+0.8680% +0.9185% +0.9718%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 8 outliers among 100 measurements (8.00%)
  5 (5.00%) high mild
  3 (3.00%) high severe

example4_template       time:   [6.9083 us 6.9090 us 6.9097 us]
                        change: [+0.3629% +0.3903% +0.4153%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) low mild
  1 (1.00%) high mild

     Running unittests (clap/target/release/deps/05_ripgrep-4ec51a11f09e1cbe)
build_rg_with_short_help
                        time:   [8.3229 us 8.3244 us 8.3262 us]
                        change: [+0.8276% +0.9280% +1.0679%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 13 outliers among 100 measurements (13.00%)
  8 (8.00%) high mild
  5 (5.00%) high severe

build_rg_with_long_help time:   [8.8455 us 8.8492 us 8.8538 us]
                        change: [+1.7421% +1.8539% +2.0244%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
  1 (1.00%) high mild
  4 (4.00%) high severe

write_rg_short_help     time:   [57.595 us 57.614 us 57.641 us]
                        change: [+2.4537% +2.5307% +2.6830%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
  3 (3.00%) high mild
  2 (2.00%) high severe

write_rg_long_help      time:   [222.31 us 222.32 us 222.34 us]
                        change: [+0.2648% +0.7561% +1.1537%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 9 outliers among 100 measurements (9.00%)
  1 (1.00%) low mild
  4 (4.00%) high mild
  4 (4.00%) high severe

parse_rg                time:   [18.185 us 18.320 us 18.493 us]
                        change: [+5.7548% +6.9850% +8.3303%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
  2 (2.00%) high mild
  2 (2.00%) high severe

parse_rg_with_complex   time:   [26.137 us 26.147 us 26.161 us]
                        change: [-0.2344% +0.2280% +0.6156%] (p = 0.33 > 0.05)
                        No change in performance detected.
Found 6 outliers among 100 measurements (6.00%)
  1 (1.00%) high mild
  5 (5.00%) high severe

parse_rg_with_lots      time:   [409.12 us 409.23 us 409.38 us]
                        change: [-1.3728% -1.2670% -1.1632%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 31 outliers among 100 measurements (31.00%)
  20 (20.00%) low severe
  1 (1.00%) high mild
  10 (10.00%) high severe

     Running unittests (clap/target/release/deps/06_rustup-12dd5d65cc5a02b6)
build_rustup            time:   [17.487 us 17.491 us 17.495 us]
                        change: [-3.3073% -3.2013% -3.1154%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
  3 (3.00%) high mild
  3 (3.00%) high severe

parse_rustup            time:   [25.826 us 25.829 us 25.833 us]
                        change: [+1.9486% +2.0375% +2.1692%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
  1 (1.00%) high mild
  3 (3.00%) high severe

parse_rustup_with_sc    time:   [25.979 us 25.986 us 26.000 us]
                        change: [+2.6127% +2.8944% +3.1164%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 10 outliers among 100 measurements (10.00%)
  4 (4.00%) high mild
  6 (6.00%) high severe

@pksunkara
Copy link
Member

Some of it might just be margin of error.

} else {
self.parser.app.about
self.parser.app.about.as_ref().map(|a| a.as_ref())
Copy link

@miDeb miDeb Jun 18, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quick suggestion: This could be written simpler:

Suggested change
self.parser.app.about.as_ref().map(|a| a.as_ref())
self.parser.app.about.as_deref()

(applies to other occurences of .as_ref().map(|a| a.as_ref()) as well, they can all be .as_deref())

@pksunkara
Copy link
Member

Closing this. Thanks for the work @d-e-s-o.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants