xxx regex cache

[melvinw]

not ready for review. needs some cleanup

[melvinw]

let's see if this blows anything up.

i want to compare this to std::list, too

[andychu]

This looks reasonable, I would like to see the before/after numbers on the 3 workloads we have (parse_help, synthetic with #regexes > cache size, and #regexes < cache size)

And I think we should have unit tests in libc_test.cc, which means moving the declaration into the header

I made a comment about the data layout ... reducing indirection by making CacheEntry a value.

That does make shifting more expensive, though now I kinda wonder about make_heap and operator >

The struct size on 64-bit machines would be 8+8+4 = 20 * (N=100) = 2000 bytes, which doesn't seem horrible

[andychu]

I wonder if it can be vector<CacheEntry>

And then the CacheEntry contains a pointer to string, pointer to regex_t, and int hash

Then I feel like there's more locality when doing the find_if.

But then it costs more when we do erase() I suppose, because instead of a pointer, it's a struct with 2 pointers and an int.

[andychu]

I think it's better to pull out hash(pat) from the loop

Even though it doesn't hash every time, it still checks if it's hashed every time

[andychu]

Overall I think this is fine, I think the main reason to look at any more optimizations (Dict<K, V> or make_heap()) would be if the (# regex > cache size) case exhibits a big slowdown, like 50% slower than not having a cache at all