Skip to content

Releases: alibaba/GraphScope

Release v0.27.0

29 Mar 13:46
@lidongze0629 lidongze0629
v0.27.0
b2db3bd
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
Release v0.27.0 Latest
Latest

We are delighted to bring a number of improvements to GraphScope, alongside the GraphScope 0.27.0 release. This release contains many important features and improvements to GraphScope FLEX、gsctl command-line utility、Graph Interactive Engine (GIE) and Groot persistent storage.

We highlight the following improvements included in this release:

1. gsctl command-line utility

gsctl is a command-line utility designed to simplify the usage of GraphScope. With gsctl, users can easily interact with GraphScope Flex product, such as Interactive(for high QPS queries) and Insight(for online BI analysis), build binaries and docker images, and perform various utility tasks related to GraphScope. See more detailed informaton from doc.

Key Features of gsctl

  • Utility Scripts: gsctl provides a collection of utility scripts that simplify common tasks such as installing dependencies, building binaries and docker images, and running test suites.
  • Client/Server Mode: With the client/server mode, users can connect to a launched coordinator and interact with specific GraphScope products behind it. This mode enables managing sessions, resources, and other product-specific functionalities.
  • Scope Management: gsctl allows users to switch between global and local scopes, enabling them to work with graphs and resources at different levels. This flexibility enhances productivity and enables efficient management of graph computation tasks.
  • Configuration Management: gsctl includes configuration options that can be customized to suit specific requirements. Users can specify a custom coordinator endpoint and manage other configuration settings as needed.

2. GraphScope FLEX

  • Support build docker images through gsctl flexbuild command.
$ git clone https://github.com/alibaba/GraphScope.git && cd GraphScope
$ python3 gsctl.py flexbuild insight/interactive --app docker
  • Support deploy Interactive and Insight product through docker image.
$ docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.27.0-flex
# launch engine and coordinator service: 12312 gremlin endpoint, 55556 grpc endpoint, 8080 coordinator endpoint
$ docker run -p 12312:12312 -p 55556:55556 -p 8080:8080 registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.27.0-flex /usr/local/groot/bin/start_local_cluster.sh
  • Connect to a launched coordinator and interact with the products
graphscope@host:/work/python$ gsctl connect --coordinator-endpoint http://127.0.0.1:8081
[SUCCESS] Connected to http://127.0.0.1:8081, coordinator is serving with GRAPHSCOPE_INSIGHT mode.

Try 'gsctl --help' for help.

graphscope@host:/work/python$ gsctl ls
schema
├── vertex types
│   ├── person
│   │   ├── Property(name: id, type: LONG, is_primary_key: True)
│   │   ├── Property(name: name, type: STRING, is_primary_key: False)
│   │   └── Property(name: age, type: LONG, is_primary_key: False)
│   └── software
│       ├── Property(name: id, type: LONG, is_primary_key: True)
│       ├── Property(name: name, type: STRING, is_primary_key: False)
│       └── Property(name: lang, type: STRING, is_primary_key: False)
└── edge types
    ├── (person) -[created]-> (software)
    │   ├── Property(name: edge_id, type: LONG, is_primary_key: False)
    │   └── Property(name: weight, type: DOUBLE, is_primary_key: False)
    └── (person) -[knows]-> (person)
        ├── Property(name: edge_id, type: LONG, is_primary_key: False)
        └── Property(name: weight, type: DOUBLE, is_primary_key: False)

3. Enhancements for Groot

  • Integrate a prototype of OpenTelemetry into Groot, users could switch on option trace.enabled=true, which will have a jaeger container that collect traces. And it will serve a website on POD_IP:16686.

Docker Images

# Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.27.0

# Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.27.0

# Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.27.0
# Executor component of Graph Interactive Engine(GIE) image 
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.27.0

# Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.27.0

# GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.27.0

# Develop Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-dev:latest

What's Changed

  • feat(flex): refactor memory management of mmap_array to support Hugepage. by @luoxiaojian in #3505
  • fix(interactive): Fix install_dependency.sh by @zhanglei1949 in #3500
  • fix(flex): Fixed the usage of storage strategy. by @luoxiaojian in #3506
  • fix(interactive): Fix building flex with -DBUILD_HQPS=OFF by @zhanglei1949 in #3497
  • fix(interactive): 离线全量导入时, 新增wait参数, 在download全部完成后, wait指定时间(单位ms), 再进行提交 by @bufapiqi in #3509
  • ci: Fix ci failure and pre-release v0.27.0 by @lidongze0629 in #3512
  • feat(flex): Configure the memory strategy with memory-level. by @luoxiaojian in #3511
  • refactor: coordinator and gsctl under FLEX architecture by @lidongze0629 in #3481
  • chore(deps): bump werkzeug from 0.16.1 to 2.3.8 in /flex/coordinator by @dependabot in #3518
  • chore(deps): bump flask from 2.1.1 to 2.2.5 in /flex/coordinator by @dependabot in #3519
  • docs(interactive): Update doc by @zhanglei1949 in #3514
  • ci: fixes a syntax error in CI by @sighingnow in #3521
  • fix(interactive): Fix argument passing in hqps_app by @zhanglei1949 in #3522
  • chore: change manylinux2014-ci to manylinux2014 by @siyuan0322 in #3529
  • feat(flex): Implemented Immutable Csrs. by @luoxiaojian in #3527
  • fix(interactive): Fix interactive building Dockerfile by @zhanglei1949 in #3528
  • fix(interactive): Fix error when loading edge with string properties by @zhanglei1949 in #3523
  • feat(interactive): Update graph graph_algo by @zhanglei1949 in #3486
  • fix(interactive): support non-blocking data loading interface in coordinator and fix failure during flexbuild process by @lidongze0629 in #3530
  • feat(interactive): Decouple ingestor and store, remote direct RPC connections. by @siyuan0322 in #3525
  • Update README.md for SIGMOD 2024 paper by @wenyuanyu in #3534
  • fix(interactive): Integrate Gremlin Into New Compilation Stack by @shirly121 in #3332
  • fix(k8s): Bump up vineyard to v0.20.3 by @dashanji in #3532
  • fix(interactive): Fix sinking ElementID for TwoLabelVertexSet by @zhanglei1949 in #3535
  • fix(interactive): Fix PathExpand related bugs by @zhanglei1949 in #3538
  • fix(interactive): fix install_dependencies by @liulx20 in #3539
  • fix(flex): Fix warning when compiling with AOCC compilers by @zhanglei1949 in #3510
  • fix(interactive): Fix test_gremlin_timeout in Python Unit Tests by @shirly121 in #3536
  • fix(interactive): Fix Bugs of Group Returning Results from Compiler by @shirly121 in #3533
  • ci: fixes some dependencies issues by @sighingnow in #3553
  • chore: Spelling comments by @jsoref in #3549
  • docs: fixes typos in documentation by @jsoref in #3554
  • fix(analytical): Fix Louvain algorithm's move on const message by @songqing in #3551
  • docs: fixes typos in YAML configurations by @jsoref in #3555
  • feat: Impl the file upload interface for FLEX dataloading by @lidongze0629 in #3557
  • fix(interactive): support src and dst of an edge with different type by @liulx20 in #3552
  • feat(interactive): Support PTHash indexer by @liulx20 in #3558
  • fix(interactive): Fix secondary catchup error by reopening secondary by @siyuan0322 in #3567
  • fix(interactive): Fix a bug in pkscan with expected values given in Within by @BingqingLyu in #3540
  • fix(analytical): fix bug that flash app can not run on string oid graph by @acezen in #3570
  • fix: Spelling python graphscope tests by @jsoref in https://gi...
Read more

Contributors

LiSu, jsoref, and 15 other contributors
Assets 2

Release v0.26.0

25 Jan 03:10
@lidongze0629 lidongze0629
v0.26.0
6db110e
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
Release v0.26.0

We are delighted to bring a number of improvements to GraphScope, alongside the GraphScope 0.26.0 release. This release contains many important features and improvements to GraphScope FLEX and Graph Interactive Engine (GIE), including secondary instance and type inference in GIE, loading from distributed data source and Macs deployment in FLEX Interactive.

We highlight the following improvements included in this release:

1. FLEX Interactive

  • Support use GraphScope Interactive on macos. Download the latest release to give it a shot interactive-latest
  • Support bulk loading with odps table as data source. Here is an example
graph: dd_graph
loading_config:
  data_source:
    scheme: odps  # file, odps
  import_option: init # init, overwrite
  format:
    type: arrow
vertex_mappings:
  - type_name: Person  # must align with the schema
    inputs:
      - your_proj_name/table_name/partition_col_name=paritition_name
  column_mappings:
      - column:
          index: 0  # can be omitted if the index is the same as the property index
          name: user_id  # can be omitted if the name is not known
        property: id
edge_mappings:
  - type_triplet:
      edge: Knows
      source_vertex: Person
      destination_vertex: Person
    inputs:
      - your_proj_name/table_name/partition_col_name=paritition_name
    source_vertex_mappings:
      - column:
          index: 0
          name: src_user_id
        property: id
    destination_vertex_mappings:
      - column:
          index: 1
          name: dst_user_id
        property: id

2. Enhancements for GIE

  • Support secondary instance in Groot. It enables user open another read-only instance along with the original instance, providing additional read qps to users.

    • How to use: Set secondary.enabled=true in helm charts to enable secondary mode. Other options could remain unchanges with the primary instance.
    • Note in secondary instance it could only be queried upon, the modification of schema and data should be routed to primary instance.
    • See more details in the doc

  • Support automatic type inference in graph queries, which can affect query execution in two ways:

    • Checking if vertex and edge types in the graph conform to user-defined schemas 
      # Now compiler can infer that 'kno' is a nonexistent edge type and returns error
Match (a:person)-[b:kno]->(c) Return a, b, c;
    • Inferring vertex and edge types in the graph. 
      # Now compiler can infer a and b must both be person types, and execution can be more efficient
Match (a)-[:knows]->(b) Return labels(a), labels(b)
=> Match (a:person)-[:knows]->(b:person) Return labels(a), labels(b)

See more details in the doc

3. Other enhancements and bug fixes

  • Introduce graph_algo as another example graph. https://github.com/alibaba/GraphScope/tree/main/flex/interactive/examples/graph_algo.
  • Support String DT_STRING as edge property type.
  • Refactor Flex/CMakeLists.txt to support use installed flex via find_package(flex).
  • Fix decode error in Groot
  • Fix memory leak in Pegasus and data race in Groot
  • Fix GIE peers_contains bugs in count operator in subtasks
  • Fix Pegasus unexpect result of operator aggregate() + iterate()
  • Refine Log Printing Messages in Compiler
  • Fix bugs of Groot bulk loading when it retry to download source files
  • Fix bugs in GIE for edge filtering

Docker Images

# Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.26.0

# Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.26.0

# Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.26.0
# Executor component of Graph Interactive Engine(GIE) image 
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.26.0

# Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.26.0

# GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.26.0

# Develop Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-dev:latest

What's Changed

  • ci: fixes failure during pushing image and pre-release v0.26.0 by @lidongze0629 in #3339
  • fix(interactive): Fix decode error in groot by @siyuan0322 in #3340
  • fix(interactive): Check vertex existence whenever queried by @siyuan0322 in #3343
  • feat(interactive): Support more data types for vertex/edge property by @zhanglei1949 in #3346
  • feat(interactive): Support fold_partiton_by_key by @lnfjpt in #3324
  • fix(interactive): fix a memory leak bug in IR Core by @BingqingLyu in #3349
  • fix(interactive): replace ok_or() by ok_or_else() for lazy evaluation by @BingqingLyu in #3350
  • feat(flex): support LFIndexer resize by @liulx20 in #3351
  • refactor(interactive): Refactor CSVFragmentLoader to remove redundant code by @zhanglei1949 in #3352
  • fix(interactive): Fix support for PathExpand for multiple triplets. by @zhanglei1949 in #3335
  • fix(interactive): Remove Unnecessary JNA Structure Creation to Save Memory Consumption by @shirly121 in #3354
  • fix(interactive): extension use hash(srcId, dstId, edgeLabelId, edgePks) instead of increase eid by @bufapiqi in #3357
  • fix(interactive): Comparison between std::shared_ptr<arrow::DataType> should use Equals() by @zhanglei1949 in #3361
  • chore: fixes the missing tzdata issue in graphscope ubuntu-22.04 based images by @sighingnow in #3363
  • chore: make sure tzdata can be installed by @sighingnow in #3365
  • fix(interactive): fix memory leak in pegasus by @lnfjpt in #3367
  • fix(interactive): add some optimizer microbenchmark queries by @BingqingLyu in #3369
  • fix(interactive): Fix memory leak in pegasus and data race in groot by @siyuan0322 in #3371
  • chore: Add writer interface for groot by @siyuan0322 in #3373
  • fix(interactive): Add a new release-with-debug-info profile by @siyuan0322 in #3372
  • fix(interactive): fix the bug in filtering after intersection in GIE Runtime in distributed computation by @BingqingLyu in #3359
  • feat(interactive): Add Rate Limiter for Control Flow by @shirly121 in #3368
  • refactor(interactive): Improve the code of checking if a given label sets contain all labels in the schema by @longbinlai in #3374
  • feat(interactive): Implement admin http service for hqps engine by @zhanglei1949 in #3322
  • fix(interactive): Support Start Tag for Graph Operators in Compiler by @shirly121 in #3376
  • fix(interactive): Fix documentation for interactive. by @zhanglei1949 in #3384
  • docs: Fix GIE Benchmark Readme bug by @JackyYangPassion in #3389
  • feat(interactive): Enable replay WAL records from offset of timestamp in groot by @siyuan0322 in #3385
  • refactor(interactive): flex storage with mmap array by @liulx20 in #3356
  • fix(interactive): event message error by @lnfjpt in #3380
  • fix(interactive): fix peers_contains bugs in count operator in subtasks by @lnfjpt in #3390
  • fix(interactive): fix unexpect result of operator aggregate() + iterate() by @lnfjpt in #3391
  • fix(interactive): Fix Start Time Metric in Compiler Log by @shirly121 in #3393
  • fix(interactive): Refine Log Printing Messages in Compiler by @shirly121 in #3381
  • feat(interactive): Introduce explicit barrier for actors when switching query service to a different graph by @zhanglei1949 in #3395
  • feat(interactive): Add support for loading graph with odps table as data source by @zhanglei1949 in #3305
  • refactor(interactive): Support Logical Operator within in Index Predicate by @shirly121 in #3398
  • feat(interactive): support string as edge property type by @liulx20 in #3403
  • fix(interactive): Pass acid by @liulx20 in #3415
  • fix(python): fixes the result validation for edge sets by @sighingnow in https://github.com/alibaba/GraphScope/pull...
Read more

Contributors

longbinlai, sighingnow, and 12 other contributors
Assets 4

Release v0.25.0

10 Nov 07:13
@lidongze0629 lidongze0629
v0.25.0
8235b29
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
Release v0.25.0

We are thrilled to introduce a range of enhancements to GraphScope, with the GraphScope 0.25.0 release. This release encompasses significant features and improvements in Interactive Engine (GIE), FLEX Interactive, and bug fix.

1. Interactive under GraphScope Flex Architecture

  • Release GraphScope Interactive.
  • Enrich the movie graph schema and example query statements.
  • Implement a query cache to avoid recompiling the same adhoc query.
  • Support using the property of string type as primary key for vertex.

2. Enhancements for GIE

  • Support get disk status in groot， user could retrieve the total space and free space of the store pod via Python/Java client. Example API is here
  • Enhancements to count(): We've implemented two primary optimizations to accelerate the count() operation:
    • Optimized Counting of Vertices/Edges: Operations like g.V().count() that previously required a complete enumeration of vertices followed by a count, are now streamlined into a single fused operator. This operator efficiently counts vertices or edges in tandem with their iteration.
    • Improved Worker Utilization: The prior count() implementation was limited to using a single worker per machine for counting tasks, irrespective of the available configured workers. Now, count() operations are distributed across all configured workers, leveraging parallel processing to boost performance.
  • Users can now interact with GIE using natural language. We've integrated the capability to call OpenAI-compatible APIs within GIE, enabling the translation of human language inputs into Cypher queries. This advancement allows for a more intuitive and accessible query experience. For instance, users can utilize our Python API to execute the following code:
from graphscope.langchain_prompt.query import query_to_cypher
endpoint = "https://xxx" # use your endpoint 
api_key = "xxx" # replace to your own api key
question = "贾宝玉是谁的儿子?"
cypher_sentence = query_to_cypher(graph, question, endpoint=endpoint, api_key=api_key)
print(cypher_sentence)

Docker Image

# Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.25.0

# Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.25.0

# Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.25.0
# Executor component of Graph Interactive Engine(GIE) image 
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.25.0

# Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.25.0

# GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.25.0

# Develop Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-dev:latest

What's Changed

  • refactor(interactive): Generate Stored Procedure Config Automatically in Compiler by @shirly121 in #3140
  • feat(interactive): Support length() Operator in Cypher Queries by @shirly121 in #3166
  • fix(interactive): Fix bug in computing path length in GIE by @BingqingLyu in #3172
  • ci: Support to build GraphScope Images(both x86 and arm64) in CI workflow by @FLYLX in #3152
  • ci: fixes failure of building images by adding actions/checkout step by @lidongze0629 in #3174
  • ci: Pre-release v0.25.0 by @lidongze0629 in #3175
  • chore: fixes the link for the flex directory by @sighingnow in #3176
  • fix(interactive): Fix the bug in the RexNode to Proto Conversion by @shirly121 in #3171
  • feat(interactive): Support [] Operator in Cypher Queries by @shirly121 in #3168
  • refactor(interactive): Parse Graph Schema from Yaml Configuration by @shirly121 in #3145
  • feat(python): Refactor coordinator for operator by @siyuan0322 in #3148
  • fix(interactive): don't require openssl-sys during building GIE by @sighingnow in #3181
  • feat(interactive): Enable clear properties for groot node by @siyuan0322 in #3180
  • refactor(interactive): Support Nested Branch Logical Plan by @MeloYang05 in #3151
  • refactor(interactive): Support head(collect(XX)) in Cypher Queries by @shirly121 in #3178
  • fix(python): fix default field in py311 by @siyuan0322 in #3183
  • feat(interactive): Support IS_NULL and IS_NOT_NULL in Cypher Queries by @shirly121 in #3167
  • feat(interactive): Support Unfold step in GIE by @Louyk14 in #3143
  • remove google-java-format.jar by @Louyk14 in #3196
  • feat(interactive): Add schema management ability for groot java sdk by @siyuan0322 in #3193
  • refactor(interactive): Support DateFormats in GIE Runtime by @BingqingLyu in #3197
  • fix(interactive): Fix bugs of Chinese Encoding by @shirly121 in #3195
  • refactor(flex): Replace the Adhoc csv reader with Arrow CSV reader by @zhanglei1949 in #3154
  • refactor(interactive): Support Date Type and EXTRACT Operator in Cypher Queries by @shirly121 in #3169
  • ci: Update pr-check.yml by @yecol in #3207
  • feat(interactive): support project properties of a path by @BingqingLyu in #3213
  • feat(interactive): Enhancement pieces for groot by @siyuan0322 in #3217
  • fix(interactive): Fix bugs in Label Name to Id Conversion by @shirly121 in #3221
  • feat(interactive): Support LDBC SNB IC1,4,7,10 for HQPS Engine by @zhanglei1949 in #3147
  • chore: don't reinstall (may trigger upgrade) if libarrow-dev has already been installed by @sighingnow in #3241
  • ci: fixes the Python dependency error in CI by @sighingnow in #3242
  • fix(interactive): Not requiring eid in update/delete edges in groot by @siyuan0322 in #3240
  • fix(python): default session needs to be aware of g.set_option(num_workers=...) by @sighingnow in #3246
  • refactor(interactive): Refactor the user interface of Interactive by @zhanglei1949 in #3201
  • feat(interactive): support unfold path entry in GIE Runtime by @BingqingLyu in #3236
  • fix(interactive): Fix several bugs in data-load-tools by @siyuan0322 in #3249
  • fix(interactive): Fix NPE when update/delete vertices by @siyuan0322 in #3250
  • build(interactive): Reduce image size by @zhanglei1949 in #3247
  • fix(interactive): Support fusion of scan+count in GIE Runtime for optimization by @BingqingLyu in #3233
  • fix(flex): Support set edge strategy on each vertex_type_pair_relation by @zhanglei1949 in #3257
  • feat(learning): Integrate GLTorch into GraphScope by @LiSu in #3230
  • feat(interactive): Support labels and type Functions in Cypher Queries by @shirly121 in #3255
  • build(learning): bump torch from 1.13 to 1.13.1 in /python by @dependabot in #3264
  • refactor(interactive): Introduce a new test set for Interactive by @zhanglei1949 in #3251
  • docs(interactive): Add docs for GraphScope Interactive by @zhanglei1949 in #3267
  • fix: Flex add rt_bench and fix some bug by @liulx20 in #3268
  • chore: Enable retrieve disk status of store node from client by @siyuan0322 in #3274
  • fix(interactive): clear download path when start store service by @siyuan0322 in #3276
  • ci: requires macos 12 as homebrew (and apple) do not provide support for this old version by @sighingnow in #3278
  • fix(interactive): Fix Implicit Type Conversion Issues in HQPS by @shirly121 in #3256
  • fix(interactive): block service until graph schema is synced by @siyuan0322 in #3280
  • chore: Increase the disk size of minikube cluster by @siyuan0322 in #3283
  • fix(intera...
Read more

Contributors

yecol, LiSu, and 15 other contributors
Assets 2

Release v0.24.0

01 Sep 10:14
@lidongze0629 lidongze0629
v0.24.0
8950371
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
Release v0.24.0

We are thrilled to introduce a range of enhancements to GraphScope, with the GraphScope 0.24.0 release. This release encompasses significant features and improvements in Interactive Engine (GIE), Learning Engine(GLE), and Deployment.

We highlight the following improvements included in this release:

1. Enhancements for GIE

Key Features:

  • Support the sample() step of Gremlin to randomly pick up a given number of traversers from a Gremlin traversal. Additionally, coin() step can now follow any step in a Gremlin traversal, instead of just V() and E(). For example,
g.V().sample(10)   // Sample 10 traversers from `g.V()`
g.V().out().coin(0.2)  // Randomly pick up 20% among all vertices' out neighbors
  • Support the union step of Gremlin to combine the traversers from multiple traversals. For example,
g.V().union(out(), out().out())  // the results would now be one-hop (out once) and two-hop (out twice) neighbors of all vertices.
  • Support the unfold step of Gremlin, which is a reversed operator of fold that flattens a collection of traversers into discrete elements. For example,
g.V().fold().unfold()  // must obtain `g.V()` again
  • Support the identity() step of Gremlin, which simply returns the current traversers, and is often used together with union step. For example,
g.V().union(identity(), out())  // return `V()` and its one-hop neighbors
  • Support isNull in the expr() for verifying wether a certain property has NULL value. For example,
g.V().where(expr("@.age isNull"))  // the vertices without the field of `age` will be pruned

But fixes:

  • Fix a bug of count() step followed by order() and limit() that can cause the gaia engine to abort unexpectedly. Now the following Gremlin query can be executed smoothly:
g.V().order().by('id', asc).limit(1).out().count()

2. Enhancements for Deployment

  • All GraphScope Images support both x86 and arm64 platform
  • This release includes the gsctl command-line utility for building and testing GraphScope. The gsctl tool provides several commands and options to streamline the development and deployment process. You can use the following command to install the graphscope-client package and then use gsctl:
$ pip3 install graphscope-client
$ gsctl --help

3. Other enhancements and bug fixes

  • Reduced the size of GRAPE-java shaded jar,reduce shaded jars' total size from 94MB to 58MB.
  • Introduce Hiactor-based High-QPS Engine for Flex.
  • Refine the schema definition for flex rt_mutable_graph
  • Fix the Py_None reference count issue in GLE
  • Fix the bug that GLE client only connects with a single server in distributed training
  • Update the GLE-related tutorials

Docker Image

# Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.24.0

# Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.24.0

# Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.24.0
# Executor component of Graph Interactive Engine(GIE) image 
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.24.0

# Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.24.0

# GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.24.0

# Develop Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-dev:latest

What's Changed

Read more

Contributors

yecol, LiSu, and 20 other contributors
Assets 2

Release v0.23.0

04 Jul 12:26
@lidongze0629 lidongze0629
v0.23.0
7dcf195
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
Release v0.23.0

We are thrilled to introduce a range of enhancements to GraphScope, with the GraphScope 0.23.0 release. This release encompasses significant features and improvements in Interactive Engine (GIE), GraphScope Flex, and Deployment.

We highlight the following improvements included in this release:

1. GraphScope Flex Technical Preview

GraphScope Flex represents the ongoing evolution of GraphScope. In this release, we're excited to introduce a technical preview of GraphScope Flex. It highlights a modular design that reduces resource and cost requirements while providing a seamless, user-friendly experience for flexible deployment. It's currently under active development, and we look forward to your feedback.

Key Features:

  • Modular Design: Assemble your stack much like LEGO blocks to customize your graph computing deployments.
  • Three-Layer Architecture: Components are organized into an application layer, execution layer, and storage layer.
  • Flexible Builds and Deployments: Use the flexbuild script to build a custom deployment tailored for your specific use case.

You can explore GraphScope Flex in the flex/ directory or through the released artifacts

  • graphscope_flex_db_cppsp_hiactor_mcsr,
  • graphscope_flex_olap_builtin_grape-cpu, and
  • graphscope_flex_gnn_gnnmodels_graphlearn_tensorflow_vineyard.so

packages for high-QPS interactive queries, graph analytics, and graph learning task use cases, respectively. Dive in and discover what GraphScope Flex has to offer!

2. Enhancements for GIE

  • Support the recording of both vertices and edges during path expansion. In the past, path expansion had the option to yield either all vertices or the end vertex of the path. However, due to the requirements of both Gremlin and our users, it's essential to also retrieve the edges of the path, besides the vertices. To activate this feature, use with('RESULT_OPT', 'ALL_V_E') in the path expansion syntactic sugar. Here is an example: 
    gremlin> g.V().out("1..3", "knows").with('RESULT_OPT', 'ALL_V_E')
     ==>[v[1], e[0][1-knows->2], v[2]]
     ==>[v[1], e[2][1-knows->4], v[4]]
  • We are now happy to introduce the capability of querying Cypher in GIE, by integrating Neo4j's bolt service in our system. Please follow the guide to enable bolt service (only on local) for querying with Cypher. We also attempt to make the syntax of Cypher as close as openCypher, and the details of our support for Cypher can be found here.

3. Other enhancements and bug fixes

  • GAE Java
    • Fix VertexSet's problem of supporting vertex_id in java long.
    • Add Grape-GraphX performance report
    • Fix the problem of installing grape-jdk locally.

Docker Image

# Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.23.0

# Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.23.0

# Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.23.0
# Executor component of Graph Interactive Engine(GIE) image 
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.23.0

# Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.23.0

# GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.23.0

# Develop Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-dev:latest

What's Changed

  • [7dcf195]: Change the version to v0.23.0 (#2969) (Dongze Li)
  • [7316297]: Introduce flexbuild and a readme to flex. (#2954) (luoxiaojian)
  • [d8ad6ce]: Install jdk 11 on MacOS CI and fixes the GAE link failure in nightly CI on MacOS (#2965) (Tao He)
  • [c27f459]: Fixes two compliation warnings (#2968) (Tao He)
  • [bb1921e]: Update the elementMap-step's doc of GIE (#2966) (Dongze Li)
  • [cb27df4]: Fixes nightly CI error for Arm Linux wheels (#2963) (Tao He)
  • [69258d9]: Fixes the "illegal thread local variable reference to regular symbol" error on macOS for GLE (#2962) (Tao He)
  • [1e2c8f6]: [GIE Compiler] Unify Gremlin Timeout Configurations (#2953) (Xiaoli Zhou)
  • [4694963]: Add setuptools to client requirement in order to use pyproject based build (#2958) (Siyuan Zhang)
  • [7fc14b5]: [GIE Compiler] Support Case When Expression in Logical and Physical Plan (#2918) (Xiaoli Zhou)
  • [b82b748]: Avoid implicit int to enum conversion, causing the -fpermissive error (#2957) (Siyuan Zhang)
  • [ad4aea1]: Use the correct delete in free_graph_handle (#2955) (Tao He)
  • [21fd644]: Sort the server list before iterate to keep consistent between workers (#2956) (Tao He)
  • [502e410]: Parallelize the PrepareToRun implementation in projected fragment (#2949) (Tao He)
  • [5ffad66]: Add two more pagerank variants (#2950) (Siyuan Zhang)
  • [a222479]: Use pagerank_local_parallel rather than pagerank_local for better performance (#2945) (Tao He)
  • [92da638]: Refine CI process (#2948) (Siyuan Zhang)
  • [555b13d]: add figures for GraphScope Flex. (#2946) (Jingbo Xu)
  • [7035c0a]: ElementMap step support (#2942) (Dongze Li)
  • [b6402e7]: cluster_builder.py: add missing copyright header and remove unused import lines (#2944) (Tao He)
  • [5b852ff]: Docs preview is only available for pull requests from the orignal repo, not forks (#2943) (Tao He)
  • [cab8cde]: Add GLE quick start example in the overview getting started page (#2939) (LiSu)
  • [badf3f3]: hotfix: pull_request_target issue. (#2941) (Jingbo Xu)
  • [08f8d0f]: Disable JAVA SDK (with a warning) when JNI not found (#2937) (Tao He)
  • [f543cd2]: Improve the deploy_with_existing_vineyard_cluster doc for easy understanding (#2930) (Ye Cao)
  • [ae8d96c]: Show explicit error message for SSSP run over a graph with no edge weight (#2929) (Lei Wang)
  • [50c7a1e]: Pin vineyard python version to 0.15.0 (#2933) (Siyuan Zhang)
  • [25496f8]: Connect to frontend directly, bypass coordinator for GIE query (#2923) (Siyuan Zhang)
  • [96d5093]: [GIE Compiler] Parse Label Id to Name in label Operator (#2928) (Xiaoli Zhou)
  • [34b3650]: [GAE-Java] Use FFI-bitset rather than java BitSet for VertexSet backend (#2926) (Zhang Lei)
  • [f0a779e]: [good-first-issue]Move save_to/load_from from client side to coordinator (#2917) (Zhang Lei)
  • [c1007cf]: [GIE Compiler] Introduce cypher service to accept queries from neo4j ecosystem (#2848) (Xiaoli Zhou)
  • [7637aaf]: [GIE] Refine FFI interface build_physical_plan (#2911) (Xiaoli Zhou)
  • [d0dc2b7]: Print metrics of application running time (#2908) (Siyuan Zhang)
  • [628773d]: [Bug Fix] Fix bug of null pointer exception in distinct operator (#2901) (Xiaoli Zhou)
  • [0a44e71]: Fixes the misuse of "continue-on-error" and "fail_ci_if_error". (#2904) (Tao He)
  • [612bd7f]: Upload coverage: continue on error to avoid false negative on main branch (#2903) (Tao He)
  • [513155c]: Upgrade the graphlearn submodule to fixes nightly CI, refactor the Makefile for early error exit (#2902) (Tao He)
  • [bd7680f]: Use CONFIG mode to find protobuf to address the absl dependency issue. (#2899) (Tao He)
  • [cf81c7b]: [GAE-Java] Relax version enforce for some mvn dependencies (#2894) (Zhang Lei)
  • [b37ab90]: [GIE Compiler] Add data type of algebra operator output to physical plan (#2870) (Xiaoli Zhou)
  • [79cd577]: Use local metadata backend to avoid the requirements of etcd for local sessions (#2889) (Tao He)
  • [7ef0e4d]: fix munew line are not rendered correclty when multiple params are given (#2891) (Siyuan Zhang)
  • [fbf40f6]: Update the pre-genereated unload_op when the op itself been deepcopied (#2888) (Tao He)
  • [32089f2]: [Grape-Java] Add grape-graphx performance.md (#2886) (Zhang Lei)
  • [e4f1f23]: Allow pass params to gie instance (#2885) (Siyuan Zhang)
  • [17819c8]: [BugFix] Fix bug in endV() after PathExpand (#2881) (BingqingLyu)
  • [a18836d]: [GIE Compiler] Support dynamic query params in cypher queries (#2874) (Xiaoli Zhou)
  • [ce3e16b]: [GAE/tests] Add GAE tests for FLASH algorithms of bfs and cc (#2869) (Longbin Lai)
  • [5a63faf]: upload log should not fail the CI (#2879) (Siyuan Zhang)
  • [559b084]: Direct the content of log files to stdout of pods in kubernetes deployment (#2871) (Siyuan Zhang)
  • [4af5689]: add initial version of command-line tool gsctl. (#2868) (Jingbo Xu)
  • [7a3e06b]: [GIE Doc] Update Doc for PathExpand Step Usage in GIE (#2877) (BingqingLyu)
  • [387f915]: Fixes the docs generation for tagged releases (#2878) (Tao He)
  • [c489c63]: [BugFix] Make the error message truncated not so aggressive (#2876) (Weibin Zeng)
  • [e9af91a]: [GIE] Support PathExpand with OPT=ALL_V_E in GIE (#2841) (BingqingLyu)
  • [3773889]: Add a initializeCommand to devcontainer.json: always pull newer image before create container (#2873) (Weibin Zeng)
  • [086fa63]: [GIE/docs] reorganize gremlin docs (#2862) (Longbin Lai)
  • [bd89520]: Allow triggering workflow dispatch event for CIs (#2865) (Ke Meng)
  • [a0b36ee]: [GIE Doc] Add examples to collect gremlin results in a streaming way (#2842) (Xiaoli Zhou)
  • [5fd6cc1]: Fixes the inconsistent usage of msgpack/json in graph reporter and avoid protobuf message creation in del (#2843) (Tao He)
  • [434acdf]: Fix dns search error in groot (#2850) (Siyuan Zhang)
  • [9bf071c]: auto-gen docs of flex (#2851) (ds-ssj)
  • [6851cd9]: allow null value in data loading (#2845) (Siyuan Zhang)
  • [1ba6102]: [GIE Compiler] Introduce GraphPlanner to unify logical and physical plan b...
Read more
Assets 3

Release v0.22.0

06 Jun 10:08
@lidongze0629 lidongze0629
v0.22.0
99c9d15
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
Release v0.22.0

We are delighted to bring a number of improvements to GraphScope, alongside the GraphScope 0.22.0 release. This release contains many important features and enhancements to Interactive Engine (GIE) and Deployment. Now the user can try GraphScope under the Linux Arm64 platform by pip3 install graphscope.

We highlight the following improvements included in this release:

1. Enhancements for GIE

  • We have provided a utility for users to easily develop and test GIE on a local machine. Basically, users can run three parts of integration tests using the gs command line tool, including the official gremlin test, auxiliary pattern match test, and ldbc complex interactive workload on ldbc social network scale factor 1. In addition, users are allowed to locally start a frontend and an executor, where the former exposes an endpoint to easily integrate with the official gremlin console, and the latter is responsible for executing queries. Details can be found in the documentation.

2. Enhancements for Deployment

  • User can try GraphScope under the Linux Arm64 platform by pip3 install graphscope
  • All images are now switched to ubuntu-based.
  • Add a devcontainer.json for users to quickly get started.
  • Use vineyardctl to install vineyard as the sidecar container.
  • Support to deploy the engines of GraphScope on demand in the lazy mode.

3. Other enhancements and bug fixes

  • Support to store and restore graphs from/to persistent storage on the kubernetes cluster. Details can be found in the guide: https://graphscope.io/docs/latest/deployment/persistent_storage_of_graphs_on_k8s.
  • Add an end-to-end example about training over vineyard graphs using GLE
  • Fixes the incorrect problem of results in python algorithm with PIE model under the session's parameter num_workers=1
  • We have fixed a bug of GIE related to using match step with a not-existed edge, for which it should be executed via anti join, but it returned unexpected error "the first sentence of MergedSentence must have InnerJoin" instead.
g.V().match(
    __.as("a").in().as("b"),
    __.as("b").out().as("c"),
    __.not(__.as("c").out().as("a")) # `not`-exsited edge
)

Now such kind of query can be executed without issue.

  • Fixes the bug of incorrect results when dedup edges on a partitioned graph in GIE, e.g.,
g.V().outE().dedup()

Now it can output expected results.

Docker Image

# Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.22.0

# Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.22.0

# Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.22.0
# Executor component of Graph Interactive Engine(GIE) image 
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.22.0

# Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.22.0

# GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.22.0

# Develop Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-dev:latest

What's Changed

  • [99c9d15]: Fixes the GAE fork issue with openmpi (#2820) (Tao He)
  • [ba35996]: [GLE Doc] complete the build and install details of learning engine (#2815) (LiSu)
  • [7af9255]: Fixes some failures in nightly CI (#2817) (Tao He)
  • [0b22053]: Support to install deps for Alibaba Cloud Linux in the gs bash utility (#2816) (Ye Cao)
  • [28a3605]: Cleanup space in CI (#2814) (Siyuan Zhang)
  • [3f50d7d]: Update dev_guide.md: change the link to learning engine to a correct one. (#2813) (Taoshu)
  • [035036b]: disable debug-assert in groot (#2811) (Siyuan Zhang)
  • [57b22fe]: Groot schema now supports add comment (#2802) (Siyuan Zhang)
  • [43493ce]: Fixes bug when building docs from pull requests (#2810) (Tao He)
  • [302b028]: Support to backup and recover the specific graphs via vineyardctl (#2771) (Ye Cao)
  • [d49d0ed]: docs: update rst-versions style, defaults default version to latest, and fixes the zh/en indexing (#2803) (Tao He)
  • [9efd4f6]: Add local python script path to PATH (#2804) (Tao He)
  • [9d5d968]: [Docs/GIE] add tinkerpop ecosystem (#2784) (Longbin Lai)
  • [278a24c]: Adapt to latest vineyard for varint encoding in CSR (#2791) (Tao He)
  • [f1b216c]: Support users to only input the kubeconfig path for connecting the k8s cluster (#2799) (Ye Cao)
  • [33e3007]: Correct the version of built wheel (#2798) (Siyuan Zhang)
  • [191949b]: Fixes the graphdef value when constructing graphs from vineyard_id. (#2793) (Tao He)
  • [699ffbe]: [k8s]Support deploying vineyard cluster independently and deploy the engines on called (#2710) (Ye Cao)
  • [c1ef885]: refine docs latest tag to make it ready as default. (#2788) (Jingbo Xu)
  • [e9090db]: Refine doc for dev and test of analytical engine. (#2787) (Jingbo Xu)
  • [1e57650]: Read version number from the package in setup.cfg (#2785) (Dongze Li)
  • [09ead30]: [GLE docs] minor fixes (#2783) (LiSu)
  • [b014dad]: use kubectl cp --retries (#2781) (Siyuan Zhang)
  • [d2c7d2e]: Fix typo in Makefile (#2782) (Siyuan Zhang)
  • [3606b85]: Fixes the building failure of client wheel on Linux arm64 platform (#2780) (Dongze Li)
  • [fdc626e]: Use vineyardctl API to inject the vineyard sidecar (#2612) (Ye Cao)
  • [7fd430b]: [GraphProxy] refine implementation of LazyDetails (#2766) (BingqingLyu)
  • [4b24146]: [Bug Fix] fix the bug in implementation of PartialEq for Edge in GIE Runtime (#2740) (BingqingLyu)
  • [5b46a68]: Improve GetInnerVertex() by querying local fragment only (#2772) (Songqing Zhang)
  • [983a41e]: Enable deploy groot on local environment (#2769) (Siyuan Zhang)
  • [2509958]: Make grape_engine help message more friendly (#2774) (Siyuan Zhang)
  • [e294377]: [GIE Dev Test] Refine GIE Dev and Test (#2767) (Xiaoli Zhou)
  • [bf8d7b0]: Generate pyi stubs for protobuf modules for better developer experience (#2768) (Tao He)
  • [6fc7db3]: [Bug Fix] Fix the alias of GetV of ExpandBase in PathExpand, an… (#2763) (BingqingLyu)
  • [a119c23]: Correct the python sssp result by setting generate_eid to false in load_p2p_network dataset (#2762) (Dongze Li)
  • [e4bb069]: Make sure grape-engine exit when CTRL-C without crash (#2761) (Tao He)
  • [df16284]: Store address offset of string properties with 48-bit unsigned intege… (#2760) (luoxiaojian)
  • [747df41]: Drop the environment switch in dev and test, and update gs (#2754) (Siyuan Zhang)
  • [46d7ed8]: Fixes duplicated run and upload actions for k8s-ci (#2751) (Tao He)
  • [3cf0798]: Propogate the k8s exception to avoid waiting forever (#2747) (Tao He)
  • [e6af69b]: Ensure use string as log-level, even for logging.XXX (#2745) (Tao He)
  • [e969d0e]: [GIE Dev Test] Add docs for interactive dev test (#2733) (Xiaoli Zhou)
  • [ac7308e]: [BugFix] add Metadata for PathExpand (#2720) (BingqingLyu)
  • [94ec2d8]: Updated license and notice. (#2725) (Jingbo Xu)
  • [8ab7b1b]: Overhaul data-load-tools (#2715) (Siyuan Zhang)
  • [b03bc60]: Publish wheel packages on Linux arm64 platform nightly (#2732) (Dongze Li)
  • [1e63e87]: [GIE/bug] fix a bug of matching with anti join (#2730) (Longbin Lai)
  • [1a81620]: use javac to find the version of java (#2728) (Siyuan Zhang)
  • [51f92ac]: Support to build wheel package on Linux aarch64 platform (#2723) (Dongze Li)
  • [1d3cd4a]: refine retry logic to exponential backoff (#2729) (Siyuan Zhang)
  • [06f8769]: Fixes the installation prefix for nightly linux CI (#2727) (Tao He)
  • [662014f]: Opt-out zstd in features when requiring rocksdb (#2722) (Tao He)
  • [0a5c144]: set timeout of rt_admin to 1 minute. (#2719) (luoxiaojian)
  • [fb79c62]: Bump up graph-learn submodule version (#2716) (Tao He)
  • [7a5ed13]: add docs for using dev container and fix typo (#2711) (Siyuan Zhang)
  • [5bd90ba]: Requires correct version of graphscope_client in coordinator to avoid diverage (#2708) (Tao He)
  • [10a6488]: Introduce the new version of kubernetes log exporter for dumping logs during a period of time such as pytest (#2706) (Ye Cao)
  • [e447cd5]: Fixes the linux wheel install prefix (#2707) (Tao He)
  • [f476c57]: Fixes the 'sudo' in dockerfile for gss image (#2704) (Tao He)
  • [3b11b77]: Build wheels for macOS m1 (#2701) (Tao He)
  • [446b660]: [LICENSE] add dependency licenses for calcite and antlr (#2699) (Xiaoli Zhou)
  • [52b1ee2]: Fixes the initialization of paramters in load_from() (#2698) (Tao He)
  • [d4b72fd]: Display stderr logs as expected on K8s (#2696) (Tao He)
  • [6da5c31]: Add version info for making interactive (#2685) (Siyuan Zhang)
  • [6dd3af2]: Implemented 3-retries, added detailed logs for insert queries, and unified bulk-load and recovery. (#2691) (luoxiaojian)
  • [03165eb]: [FEAT][GraphAr] Add API to python client to enable loading graph from gar files or archive graph to gar files (#2588) (Weibin Zeng)
  • [c2cb286]: Export the k8s logs of graphscope helm test and upload to artifact when the helm test failed (#2683) (Ye Cao)
  • [81230c5]: Add prepare-commit-msg (#2688) (Siyuan Zhang)
  • [90b7e00]: Update install-hook.sh (Wenyuan Yu)
  • [ff9582e]: Create install-hook.sh (#2687) (Wenyuan Yu)
  • [909b3c4]: [GIE/Runtime] Remove the dependency of GIE Runtime on GlobalQueryStore to accelerate the compilation (#2681) (BingqingLyu)
  • [a46663e]: Clearify the requirements for MacOS (#2684) (Tao He)
  • [25a70f1]: Preload searchindex.js in background to optimize the search experiences (#2682) (Tao He)
  • [b258999]: Bump up vineyard to v0.14.5 (#2676) (Siyu...
Read more

Contributors

TaoLbr1993 and wenyuanyu
Assets 2

Release v0.21.0

25 Apr 13:55
@lidongze0629 lidongze0629
v0.21.0
2657bec
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
Release v0.21.0

We are delighted to bring a number of improvements to GraphScope, alongside the GraphScope 0.21.0 release. This release contains many important features and improvements to GraphScope. The Graph Learning Engine (GLE) now supports the 'SubGraphSampler' and 'RandomWalk' operators and adds a node labeling-based SEAL and node2vec model. Also, GLE supports node cache for accelerating distributed feature lookup. This release also adds a 'devcontainer.json' for users to easily set up a developing environment using VSCode.

We highlight the following improvements included in this release:

  • Enhancements for GLE:
    a. Add operators: SubGraphSampler, RandomWalk
    b. Add node labeling based SEAL model, node2vec model.
    c. Support temporal graph with the timestamp, and add TokTimestampSampler operator.
    d. Add temporal models: TGN and TGAT.
    e. Support node cache for accelerating distributed feature lookup.

  • Enhancements for Dev:
    a. Add a devcontainer.json for users to easily set up a developing environment using vscode.

  • Other enhancements and bug fixes:
    a. 'helm install graphscope/graphscope' would now launch all pods upon the start.
    b. Allowing more fine-grained control over Groot deployment
    c. Fixed an error of 'lgraph' that could cause the building failure in M1 Mac

Docker Image

# Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.21.0

# Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.21.0
# Graph Analytical Engine(GAE) Image with java SDK, which supports running Giraph and GraphX algorithms on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical-java:0.21.0

# Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.21.0
# Executor component of Graph Interactive Engine(GIE) image 
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.21.0

# Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.21.0

# GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.21.0

# Develop Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-dev:latest

What's Changed

Read more

Contributors

yecol, LiSu, and 18 other contributors
Assets 6

Release v0.20.0

16 Mar 07:07
@lidongze0629 lidongze0629
v0.20.0
8c98bc5
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
Release v0.20.0

We are delighted to bring a number of improvements to GraphScope, alongside the GraphScope 0.20.0 release. This release contains many important features and improvements to the persistent graph store Groot, including load balance and failover mechanism. In addition, Graph Learning Engine (GLE) now supports the Aliyun ACK cluster.

We highlight the following improvements included in this release:

1. Enhancements for GLE:

  • GLE could be launched and used to training on ACK cluster.

2. Enhancements for Groot:

  • Chart is refactored and has more customizable fields.
  • LoadBalancher service is supported.
  • Optimization of failover mechanism.

3. Enhancement for images:

  • We release the first dev image (graphscope-dev) for arm64 platform.

Docker Image

# Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.20.0

# Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.20.0
# Graph Analytical Engine(GAE) Image with java SDK, which supports running Giraph and GraphX algorithms on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical-java:0.20.0

# Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.20.0
# Executor component of Graph Interactive Engine(GIE) image 
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.20.0

# Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.20.0

# GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.20.0

What's Changed

Full Changelog: v0.19.0...v0.20.0

Contributors

yecol, LiSu, and 8 other contributors
Assets 2

Release v0.19.0

18 Jan 07:27
@lidongze0629 lidongze0629
v0.19.0
22a3b42
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
Release v0.19.0

We are glad to present the release of GraphScope 0.19.0, which brings a number of new features as well as performance and stability improvements to GraphScope. Specially, users can try GraphScope in python3.10 and python3.11 environments of Linux and MacOS by pip3 install graphscope; This release also optimizes the memory usage for loading graphs, which allows users to load larger graphs on their machines. In addition, we improve Graph Interactive Engine (GIE) via a couple of bug fixes and improvements.

We highlight the following improvements included in this release:

1. Optimized memory usage for loading graphs:

  • Improves memory usage during loading and projecting graphs, especially for graphs with string as OID type.
  • Improves the performance of project_to_simple.

2. Bug fix and enhancements for GIE:

  • Fix a bug in Pegasus that may block the execution of interactive queries in certain case.
  • Fix a bug that fails to configure the server number for the GIE engine.
  • Fix multiple dependencies in both Java and Rust codes.
  • Resolve a issue that JNA could not map bool correctly.

3. Enhancements for GAE Java:

  • Pack JNI lib into the grape-runtime jar and fix some dependency issues.

4. Standalone deployment:

  • Components now have their own dedicated image, and user could configure a subset of engines to use when launching session.

5. Other bug fixes:

  • Fix helm deployment could not delete role and rolebindings when uninstalling GraphScope.
  • Fix compile error under the newer toolchain and arrow 10
  • Rename maxgraph to graphscope and other refactors
  • Cleanup overlapping rust dependencies

Docker Image

# Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.19.0

# Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.19.0
# Graph Analytical Engine(GAE) Image with java SDK, which supports running Giraph and GraphX algorithms on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical-java:0.19.0

# Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.19.0
# Executor component of Graph Interactive Engine(GIE) image 
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.19.0

# Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.19.0

# GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.19.0

What's Changed

Read more

Contributors

yecol, LiSu, and 11 other contributors
Assets 3

Release v0.18.0

02 Dec 02:28
@lidongze0629 lidongze0629
v0.18.0
6775658
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
Release v0.18.0

We are delighted to bring a number of improvements to GraphScope, alongside the GraphScope 0.18.0 release. This release contains many important features and enhancements to Graph Interactive Engine (GIE), including introducing a new strategy for pattern-matching queries and supporting canceling running queries. The Graph Learning Engine (GLE) now supports PyTorch and is also compatible with PyG. In addition, we take a first step towards modularized deployment for different components of GraphScope in Kubernetes.

We highlight the following improvements included in this release:

1. Enhancements for GIE

  • A new execution strategy based on worst-case optimal join is introduced to GIE engine, potentially improving the performance of match step by orders of magnitude.
  • The query can be canceled after its execution exceeds a pre-given overtime parameter (10min by default).
  • GIE engine supports a failover mechanism: if an executor pod fails, but the data is not missing, it can be restarted by k8s. Existing queries cannot recover, but the engine can serve the following.

2. Enhancements for GAE

  • Add more variants of WCC algorithm.
  • Supports local vertex map to make it could scale to larger graphs given more workers.

3. Enhancements for GLE

  • Add support for PyTorch and PyG
  • Add heterogeneous graph support for subgraph-based GNN, add HeteroSubGraph and HeteroConv, bipartite GraphSAGE and UltraGCN.
  • Add edge feature support in both EgoGraph and SubGraph.
  • Add recommendation metrics: Recall, NDCG and HitRate.
  • Add hiactor-based graph engine.

4. Standalone deployment

  • Supports a version of GIE standalone deployment in Kubernetes.
  • Redesigned docker files from the bottom to the top, making it more clear and more concise.

5. GAE Java Enhancement

  • Introduce @GrapeSkip for better user experience in template method overloading.
  • Speedup Java App runimte codegen.

Docker Image

docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope:0.18.0

# GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.18.0

Commits

  • [6775658]: Push jupyter image nightly with the latest client (#2266) (DongZe Li)
  • [df9f6eb]: Fix nightly CI (#2262) (Siyuan Zhang)
  • [4f17777]: Fixes CI failure both on Linux and MacOS (#2244) (DongZe Li)
  • [07efc74]: [GIE/engine] Bug fix; (#2250) (bmmcq)
  • [5ea5b87]: [GIE] Support parallel scan on ExpStore (#2253) (BingqingLyu)
  • [432e65c]: [GIE] Make the version of GIE compiler consistent with the default value in interactive engine pom (#2249) (shirly121)
  • [2b7cf00]: [GIE POM] Unify GIE version by the global flag 'revision' (#2243) (shirly121)
  • [71d61e3]: Reorganize for Makefiles and dockerfiles (#2246) (Siyuan Zhang)
  • [84f449c]: [GIE Runtime] Debug runtime op info on in a more clearly way (#2226) (BingqingLyu)
  • [f77627d]: Update the learning model to align with the latest graphlearn (#2235) (Tao He)
  • [1c4c1cd]: [GIE/Store] Reduce memory footprint in exp_store (#2245) (Longbin Lai)
  • [e375f8d]: [GIE/Exp-Store] Optimize memory usage in loading graph data into MutTopo. (#2242) (Longbin Lai)
  • [8844e90]: Fixes release graphscope-store image (#2241) (Siyuan Zhang)
  • [428c785]: [GIE/Engine]: add direct executor implementation; (#2234) (bmmcq)
  • [e071ce4]: add docs revision preview in CI (#2237) (Jingbo Xu)
  • [f524550]: Enable cpplint in CI (#2233) (Tao He)
  • [330fe14]: Makes GAE compatible with latest arrow release (#2232) (Tao He)
  • [ed86040]: Revise docs. (#2231) (Jingbo Xu)
  • [b4d7038]: Fixes compilation error of graphx runner (#2228) (Siyuan Zhang)
  • [a11b453]: Reorganize Dockerfile according to specification of standalone deployment (#2227) (Siyuan Zhang)
  • [dcaa38a]: [GIE/GLogue] rename Catalogue as GLogue (#2224) (BingqingLyu)
  • [aed1f13]: [GIE/IR] Introduce a new strategy for pattern matching (#2159) (BingqingLyu)
  • [92f7da3]: Add two other version of wcc for compatiblity (#2223) (Siyuan Zhang)
  • [2fbc4ad]: Add local vertex map to arrow fragment and arrow projected fragment. (#2212) (Siyuan Zhang)
  • [cc51704]: Fixes nightly CI on macOS (#2219) (DongZe Li)
  • [f21754f]: [GAE-Java] GRAPE JDK support Spark local mode for graphx (#2216) (Zhang Lei)
  • [ed33b08]: [BugFix] GAE JavaDoc not generated (#2206) (Zhang Lei)
  • [4405721]: Update the DefaultTag to .Chart.AppVersion (#2210) (Ye Cao)
  • [3ee05f7]: [new FAQ] add network proxy related FAQ (#2211) (Yifan Yuan)
  • [09995bf]: Delegate parquet/orc files to vineyard io adaptors (#2209) (Tao He)
  • [7ecd75a]: Delete all deployed resources without a extra rbac role when using helm unintsall (#2208) (Ye Cao)
  • [094e81d]: [Bug Fix] Add default tag (.Chart.AppVersion) to graphscope-store (#2205) (Ye Cao)
  • [4279b8f]: [BugFix] make ResultCode in ir core ffi consistent with Compiler (#2188) (BingqingLyu)
  • [7cfe9a3]: [Bug Fix] Avoid throwing exception after processing grpc error from executor in compiler (#2182) (shirly121)
  • [abdd5c9]: [GAE-Java] Update doc (#2199) (Zhang Lei)
  • [8de000f]: [GAE-Java] Remove String-specialized FFI classes and Simplify Java MessageManager interface (#2195) (Zhang Lei)
  • [5a4312d]: Fix failure in network forward CI (#2204) (Siyuan Zhang)
  • [7735451]: Fixes failures in nightly CI (#2201) (Tao He)
  • [264cbc8]: Log level can accept int and string ignore case (#2200) (Siyuan Zhang)
  • [fb12c7b]: Fix slf4j version conflicts (#2196) (Siyuan Zhang)
  • [87b9602]: [GIE Deploy] Deploy GIE with exp or v6d in standalone mode by helm configuration management (#2143) (shirly121)
  • [ecd58a2]: Fix possible java sandbox protection. (#2191) (Siyuan Zhang)
  • [38a3781]: Enable HDFS tests in CI. (#2175) (Tao He)
  • [7fdcd23]: [GAE-Java] Support grape-graphx running on yarn cluster and support multiple executor on one host (#2185) (Zhang Lei)
  • [f983bd6]: [GIE pegasus] Support cancelling query. (#2168) (Neng Li)
  • [26f0382]: gss ci use lesser image size (#2186) (Siyuan Zhang)
  • [2b00169]: [GAE-Java] Fix GAE-Java driver app's message strategy specification (#2178) (Zhang Lei)
  • [f4dea3e]: [OSPP] Spark RDD Reader for GraphScope (#2103) (Issac-Newton)
  • [cfb707d]: Enhance grpc finding in GAE cmake (#2184) (Tao He)
  • [25ab466]: Use pre-built llvm instead of build when making analytical docker (#2183) (Siyuan Zhang)
  • [79a08d3]: [GIE Compiler] update ResultCode to be consistent with ir core interface (#2158) (shirly121)
  • [74af74f]: [GAE-Java] GraphX-on-GraphScope : remove GraphXFragment and related components (#2123) (Zhang Lei)
  • [49ae814]: [GAE-Java] Add necessary configurations in pom.xml to publish GRAPE jdk to maven central repo (#2166) (Zhang Lei)
  • [2821169]: Fix create interactive engine failure on helm installed graphscope (#2171) (Siyuan Zhang)
  • [89a4a21]: [GAE-Java] GRAPE jdk submodules naming to same pattern (#2163) (Zhang Lei)
  • [321d938]: Upgrade GitHub actions versions (#2160) (Siyuan Zhang)
  • [695cb9c]: Reorganized coordinator and redesigned the logic to create engines. (#2152) (Siyuan Zhang)
  • [ea4ca1d]: fix src label and dst label of edge kinds may be empty (#2153) (Siyuan Zhang)
  • [1b312af]: Move the tolerances of graphscope-store charts to the global scope (#2151) (DongZe Li)
  • [37fa85a]: [GAE-Java] Speed up Java App runtime codegen& compiling (#2150) (Zhang Lei)
  • [733fda3]: [GIE IR] Add docker files to build images of GIE (using exp store) (#2142) (shirly121)
  • [0c5a4b5]: [GAE-Java] Fix : can not compile the submodule of GAE Java SDK (#2146) (Zhang Lei)
  • [3220a7f]: Fixes the conditions for GAE CI in PR and on main, add timeout for test_java_app.py (#2148) (Tao He)
  • [998e361]: Rename pegasus.hosts to network.servers for executor (#2147) (Siyuan Zhang)
  • [97eec70]: Add pytest-timeout to requirements-dev.txt and fixes a deprecated warnings when running tests (#2145) (Tao He)
  • [3038b89]: [GAE-Java] Introduce @GrapeSkip to support generated method overloading (#2144) (Zhang Lei)
  • [77b4ec5]: [GAE-Java] Fix log4j jar missing in grape-runtime and java pie driver app's message strategy settable. (#2140) (Zhang Lei)
  • [7d351a2]: [IR Compiler] make gremlin server listen to 0.0.0.0 instead of localhost (#2141) (shirly121)
  • [759b7a5]: Server now use aks provided by client side (#2132) (Siyuan Zhang)
  • [5c963f5]: Update cluster.py to fix the bug that etcd selector could not work (#2139) (changyi1223)
  • [6d94c3f]: [Bug Fix] Fix bug in index_scan on Groot/Vineyard (#2129) (BingqingLyu)
  • [d4d942c]: [Assembly] wait_servers_ready() instead of sleep 3s when start engine on vineyard (#2136) (BingqingLyu)
  • [2aa3cfc]: Update commons-codec:commons-codec 1.11 to 1.13 (#2119) (zhoumengyks)
  • [6d3d6aa]: Update org.apache.kafka:kafka-clients 2.5.0 to 2.7.2 (#2120) (zhoumengyks)
  • [6d5d288]: Fixes the docstring rendering for GAE apps (#2131) (Tao He)
  • [950ed7a]: App now can carry cmake options, and enabled wcc on string oid. (#2130) (Siyuan Zhang)
  • [31829fd]: [GIE/Runtime] Refine some error infos in Runtime (#2113) (BingqingLyu)
  • [d16b1b4]: Fixes configuration in dockerignore (#2126) (Tao He)
  • [71c37fe]: Upgrade com.fasterxml.jackson.core:jackson-databind to version 2.14.0-rc1 (#2122) (Siyuan Zhang)
  • [ace93c5]: [Pegasus] minor fix: avoid iter.next() when it is None in release (#2112) (BingqingLyu)
  • [5756de3]: Make gae CI invoked when CMakeLists.template changes (#2101) (Zhang Lei)
  • [b6d68bc]: New predicate support (#2086) (waruto)
  • [eb393da]: Bump protobuf-java from 3.18.2 to 3.19.6 in /interactive_engine (#2109) (dependabot[bot])
  • [0b10f5c]: Bump protobuf-java from 3.18.2 to 3.19.6 in /interactive_engine/compiler (#2111) (dependabot[bot])
  • [b1132cc]: [ASoC] report for subtask (#21...
Read more

Contributors

Issac-Newton, changyi1223, and 3 other contributors
Assets 2