New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Store graph edges in SharedArrayBuffer #6922
Commits on Apr 12, 2021
Commits on Apr 13, 2021
Commits on Apr 15, 2021
Commits on Apr 20, 2021
Commits on Apr 21, 2021
Commits on Apr 22, 2021
Commits on Apr 24, 2021
Commits on Apr 26, 2021
Commits on Apr 27, 2021
Commits on May 4, 2021
-
This is not likely to stay here; it is probably better integrated into some test or benchmarking utilities (like `dumpGraphToGraphviz`).
Commits on May 6, 2021
-
-
Add 1 to edge hashes and type values
This is to allow truthy tests to pass when checking to see if edges exist.
-
-
-
use EfficientGraph getNodes functions, update graph tests, change def…
…ault edge type to 1
-
Merge remote-tracking branch 'origin/lettertwo/buffer-backed-graph' i…
…nto bdo/buffer-backed-graph
Commits on May 10, 2021
Commits on May 11, 2021
Commits on May 12, 2021
-
-
Merge branch 'bdo/buffer-backed-graph' into lettertwo/buffer-backed-g…
…raph * bdo/buffer-backed-graph: Add generic TEdgeType to EfficientGraph change NullEdgeType back to 1, enforce edge types to be non-zero in Graph fix traversal test Use new EfficientGraph functions in Graph, update tests, fix edge type in removeEdge use EfficientGraph getNodes functions, update graph tests, change default edge type to 1
-
-
This adds `nodeAt` and `indexOfNode` utils to abstract away the bookkeeping for storing and retrieving nodes in the byte array. Also fixes a bug where subsequently added nodes would partially overlap previous nodes in the byte array.
-
-
-
Commits on May 13, 2021
-
-
-
-
-
Merge remote-tracking branch 'origin/lettertwo/buffer-backed-graph' i…
…nto bdo/buffer-backed-graph
-
-
preserve edge insertion order in getAllEdges
Lots of test cases seem to expect this. I'm not sure if that is intentional or not, but it seems reasonable to do this, regardless.
Commits on May 14, 2021
-
Fix removeEdge for edges that have hash collisions
Also attempt to disambiguate some of the naming for hash and index functions.
Commits on May 17, 2021
-
Remove old AdjacencyList, serialize EfficientGraph in Graph, update B…
…undleGraph to use new functions
Commits on May 23, 2021
Commits on May 26, 2021
-
Merge branch 'bdo/buffer-backed-graph' into lettertwo/buffer-backed-g…
…raph * bdo/buffer-backed-graph: (37 commits) Improve resolver performance (#6328) v2.0.0-beta.3.1 Add it to the nightly release workflow as well Configure jemalloc page size for M1 (#6314) Update swc (#6307) Fix parcelDependencies v2.0.0-beta.3 Update swc (#6289) Remove old AdjacencyList, serialize EfficientGraph in Graph, update BundleGraph to use new functions Babel ast location, diagnostic, and source location remapping (#6238) update contributing guide (#6293) Update sourcemap to rc-1.0 (#6279) Serve nearest index.html in case the applications has multiple index files (#6250) Allow bundling browserify bundles by replacing free requires with undefined (#6260) Upgrade flow to 0.151.0 (#6281) Use local require for `@babel/core` and `postcss` (#6264) Resolve helpers relative to JS transformer (#6278) Pure comment for $parcel$interopDefault (#6271) Don't transpile spread in JSX with modern targets (#6274) Update TS validator assertions (#6272) ...
-
-
Commits on May 30, 2021
-
Merge branch 'v2' of github.com:parcel-bundler/parcel into bdo/number…
…-edgetypes # Conflicts: # packages/core/core/src/BundleGraph.js # packages/core/core/src/dumpGraphToGraphViz.js # yarn.lock
Commits on Jun 3, 2021
-
-
Fix resize loop when initial graph size is very small
We resize the graph edges when the available capacity is <30%, but we were indexing a new edge _before_ expanding capacity, which was causing a loop when the the total capacity was too small (i.e., `4`) to trigger the 30% threshold in a previous add _and_ there was no available capcity for the next add.
-
Commits on Jun 4, 2021
-
Add simple clustering and uniformity stats
This could help us optimize our hash function.
-
-
-
Commits on Jun 8, 2021
-
Serialize node and edge capacities
Since these values are used in hashing, we serialize them along with the edges and nodes to ensure that the hashes don't change when a new graph is deserialized from a previous graph.
Commits on Jun 9, 2021
-
-
-
Fix lookups after edges have been removed
This addes a DELETED sentinel that preserves contiguity in the hash table so that scanning forward to find edges that match a hash continues to work after other edges with a matching hash have been removed.
Commits on Jun 15, 2021
-
-
-
-
Remove unused removeNode method
Not only is it unused, but it might not make any sense for it to exist, given that adding a node is basically just incrementing a counter.
Commits on Jun 17, 2021
-
Use stored node/edge capacity instead of recomputing
Also fix collision stat and add node/edge capacity and load stats
Commits on Jun 21, 2021
Commits on Jun 22, 2021
Commits on Jun 24, 2021
-
-
-
The common use case here is iterating over all the edges to copy them to a new graph. Since a graph may have _a lot_ of edges, we can conserve memory by avoiding collecting all edges up front.
-
-
-
-
Commits on Jun 28, 2021
Commits on Jun 30, 2021
Commits on Jul 1, 2021
-
-
Refactor AdjacencyList to Node and Edge views
These views are meant to simplify the bookkeeping operations in AdjacencyList, though they may be introducing some slowdowns. However, this should provide some optimization opportunity, for example, caching/memoizing of computed values, and also implementing a doubly linked edge list to improve scan / iteration times.
Commits on Jul 7, 2021
Commits on Jul 8, 2021
-
Add inspection methods to Node, Edge
This makes console logging a little more readable during debugging
-
Move edge linking out of Edge statics
We'll try to make AdjacencyList be responsible for all linking, while Edge/Node are just responsible for reading/writing to/from the node and edge arrays.
-
Update indexOf/indexFor to default to null edge type
Aslo remove some old code
Commits on Jul 9, 2021
-
-
Merge branch 'v2' into lettertwo/buffer-backed-graph
* v2: (34 commits) Wrap assets recursively when any incoming dependency is wrapped (#6572) Improvements for library targets (#6570) Diagnostic for undeclared external dependencies in library builds (#6564) More bugs (#6567) Don't require `url:` for image transformer (#6565) Remove 'Name already registered with serializer' error (#6566) Fix live bindings and `this` of external CommonJS modules (#6548) JS runtime improvements (#6531) Make sure the absolute path isn't contained in the cache (#5900) Adds '@parcel/diagnostic' to dependencies (#6563) Disable workers with string literals and improve diagnostics (#6536) Bug fixes (#6541) Don't attempt to resolve URLs starting with '#' (#6504) Correctly set worker's output format if not support by environment (#6534) Babel: Recognize peerDependencies in isJSX (#6497) fix setHeaders ordering on dev server (#6500) Graph: Remove Node interface (#6530) Fix TS build script for old Node versions (#6526) Improve library targets (#6517) Fix TypeScript and other sourcemaps by always creating an initial sourcemap (#6472) ...
Commits on Jul 13, 2021
-
Speed up traversal to/from a node by edge type.
This adds a lookup table by type to or from a given node, so that it is trivial to frequently get a given node's connected nodes of a given type.
-
-
-
Merge Edge and Node into AdjacencyList
The overhead associated with these two data structures was not negligible, so a better choice is to use methods on AdjacencyList instead of creating Edge and Node views.
-
-
Commits on Jul 14, 2021
-
Implement XOR doubly linked edge list
This is a space optimization; there may be some performance impact due to the extra time spent scanning the list when removing an edge. It saves space by eliminating the need to explicitly store the next and previous edges in each edge object. Instead, it maintains an XOR of the next and previous edges on each edge, which means following the links between edges requires knowing the edge that came before or after.
Commits on Jul 15, 2021
Commits on Jul 16, 2021
-
Refactor generator methods to return arrays
They're all just getting immediately exhausted by Graph anyway.
-
Revert to singly linked edge list
XOR links are about 25% slower than the reference link implementation, but use about 20-25% less space. The singly linked version is even slower than XOR, but has the same space savings. This commit restores a singly linked list, but adds a caching mechanism for improving the perf when finding a previous edge.
Commits on Jul 20, 2021
Commits on Jul 28, 2021
Commits on Jul 29, 2021
-
Bookkeep empty/deleted edges for constant time lookup when we search …
…for an available slot to add an edge to
Commits on Aug 3, 2021
-
Refactor hashing and index functions to use coalesced chaining
Each edge now has a pointer to the next edge in the list of collisions indexFor will find an empty edge and update the last edge in the chain to point to the newly added edge indexOf will now only traverse through the list of edges with the same hash TODO: * reuse deleted edges * move NEXT_HASH pointer to another structure to save on memory * the cellar gets exhausted pretty quickly, look into increasing the amount of space the cellar occupies
Commits on Aug 5, 2021
Commits on Aug 6, 2021
-
Keep NEXT_HASH reference when removing edges in the adressable space …
…that point to edges in the cellar
Commits on Aug 12, 2021
-
Implement a close addressed hash table
Also added a header sequence to the arrays, which eliminates the need to serialize other data separately from the arrays.
Commits on Sep 2, 2021
-
-
Defer reclaiming of deleted space until the next add
We were reclaiming space when removing edges, but it turns out that doing so was quite expensive in practice. Instead, we defer reclaiming that space until the sum total of adds and deletes exceeds the capacity by the load factor.
Commits on Sep 3, 2021
-
-
We don't need this any more because we eliminated linear scanning.
Commits on Sep 7, 2021
Commits on Sep 9, 2021
-
-
-
-
Ease edge growth rate as capacity increases
This is intended to strike a balance between growing the edge capacity in too small increments, which causes a lot of resizing, and growing the edge capacity in too large increments, which results in a lot of wasted memory.
Commits on Sep 10, 2021
-
It turns out prepopulating the typemap before doing any traversal slows down the graph building process significantly, but does not make traversal any faster.
Commits on Sep 13, 2021
-
Merge branch 'v2' into bdo/number-edgetypes
* v2: (42 commits) Remove dead link to docs in diagnostic (#6913) Fix engines.parcel in SVG packager (#6911) Use imported Readable flow type instead of global (#6910) Use non-deprecated SVGO options in HTMLNanoOptimizer (#6785) Use yarnpkg registry for all the things (#6908) Fix issue about loading configs from tsconfig.json (#6881) Add `@section` to Compressor type (#6885) Parcel API improvements (#6866) Use stream-browserify for polyfilling instead (#6863) Add support for compressor plugins (#6776) Support for inline style attributes and inline scripts in SVG (#6797) Bump less (#6852) Make sure (non-React) SVGs are in separate bundles (#6757) Add support for injecting manifest into service workers (#6798) Fix 'does not export default' error with scope hoisting and url/worklet pipeline (#6803) Remaining cargo clippy fixes (#6829) Lazily install sharp only when needed (#6816) More cargo clippy suggestions (#6811) Don't emit runtime manifest for inline child bundles (#6807) Apply suggestions from rust-clippy (#6256) ...
-
Merge branch 'bdo/number-edgetypes' into lettertwo/buffer-backed-graph
* bdo/number-edgetypes: (74 commits) Remove dead link to docs in diagnostic (#6913) Fix engines.parcel in SVG packager (#6911) Use imported Readable flow type instead of global (#6910) Use non-deprecated SVGO options in HTMLNanoOptimizer (#6785) Use yarnpkg registry for all the things (#6908) Fix issue about loading configs from tsconfig.json (#6881) Add `@section` to Compressor type (#6885) Parcel API improvements (#6866) Use stream-browserify for polyfilling instead (#6863) Add support for compressor plugins (#6776) Support for inline style attributes and inline scripts in SVG (#6797) Bump less (#6852) Make sure (non-React) SVGs are in separate bundles (#6757) Add support for injecting manifest into service workers (#6798) Fix 'does not export default' error with scope hoisting and url/worklet pipeline (#6803) Remaining cargo clippy fixes (#6829) Lazily install sharp only when needed (#6816) More cargo clippy suggestions (#6811) Don't emit runtime manifest for inline child bundles (#6807) Apply suggestions from rust-clippy (#6256) ...
Commits on Sep 14, 2021
-
Rename getNodesConnected{To,From} to getNodeIdsConnected{To,From}
This matches the naming of the corresponding methods on Graph.
-
-
Replace string conversion with prime multiplication in hash
It's benchmarking about 67% faster without any degradation in uniformity. Also fixed some tests that were assuming some things about hashing.
Commits on Sep 22, 2021
-
Merge branch 'v2' into lettertwo/buffer-backed-graph
* v2: Upgrade Flow to 0.160.1 (#6964) Only use error overlay if there's a document (#6960) Don't fail when HTML tags are implied (#6752) Reorder resolveOptions() env priority (#6904) Change edge types to numbers (#6126) Bump swc (#6848) Print diagnostics for scope hoisting bailouts at verbose log level (#6918)
-
Refactor nodes and edges into type maps
This looks like a big change because a lot of stuff moved around. The edges are currently stored the same way they were before, but the hash table implementation has been extracted for use in the new node map. The node map is where the real change is; instead of just linking nodes to addresses in the edge map, it now stores multiple sets of links per node, one for each type of edge. This allows us to eliminate the need for a type map cache, as we are now able to look up edges by type in constant time.
Commits on Sep 23, 2021
Commits on Sep 27, 2021
Commits on Sep 28, 2021
Commits on Nov 15, 2021
-
Merge branch 'v2' into lettertwo/buffer-backed-graph
* v2: (68 commits) Fix RangeError in `not export` error with other file type (#7295) Apply sourcemap in @parcel/transformer-typescript-tsc (#7287) Fix side effects glob matching (#7288) Fix changelog headings v2.0.1 Changelog for v2.0.1 Resolve GLSL relative to the importer, not the asset (#7263) fix: add @parcel/diagnostic as dependency of @parcel/transformer-typescript-types (#7248) Fixed missing "Parcel" export member in Module "@parcel/core" (#7250) Add script to sync engines with core version (#7207) Bump swc (#7216) Make Webpack loader detection regex dramatically faster (#7226) swc optimizer (#7212) Update esbuild in optimizer (#7233) Properly visit member expressions (#7228) Update to prettier 2 (#7209) Fix serve mode with target override and target source fields (#7187) Update package.json to include the repository (#7184) fix #6730: add transformer-raw as dependency of config-webextension (#7193) Log warning instead of crash if image optimizer fails (#7119) ...