ocparse - the openCypher parser written in Erlang

ocparse is a production-ready openCypher parser written in pure Erlang. ocparse is closely aligned to the openCypher project and in future will be adapted on a regular basis as the openCypher project evolves. The openCypher project aims to deliver a full and open specification of the industry’s most widely adopted graph database query language: Cypher. And, with the EBNF file the project provides the basis for the definition of the LALR grammar.

1. Usage

Example code:

MATCH (m:Movie)
WHERE m.title = 'The Matrix'
RETURN m

Parsing the example code:

1> {ok, {ParseTree, Tokens}} = ocparse:source_to_pt("MATCH (m:Movie) WHERE m.title = 'The Matrix' RETURN m").
{ok,
 {{cypher,
   {statement,
    {query,
     {regularQuery,
      {singleQuery,
       [{clause,
         {match,[],
          {pattern,
           [{patternPart,[],
             {anonymousPatternPart,{patternElement,{...},...}}}]},
          {where,
           {expression,
            {orExpression,{xorExpression,{andExpression,...},[]},[]}}}}},
        {clause,
         {return,[],
          {returnBody,
           {returnItems,[],[],[{returnItem,{...},...}]},
           [],[],[]}}}]},
      []}}},
   []},
  [{'MATCH',1},
   {'(',1},
   {'UNESCAPED_SYMBOLIC_NAME',1,"m"},
   {':',1},
   {'UNESCAPED_SYMBOLIC_NAME',5,"Movie"},
   {')',1},
   {'WHERE',1},
   {'UNESCAPED_SYMBOLIC_NAME',1,"m"},
   {'.',1},
   {'UNESCAPED_SYMBOLIC_NAME',5,"title"},
   {'=',1},
   {'STRING_LITERAL',1,"'The Matrix'"},
   {'RETURN',1},
   {'UNESCAPED_SYMBOLIC_NAME',1,"m"}]}}

Access the parse tree of the example code:

2> ParseTree.
{cypher,
 {statement,
  {query,
   {regularQuery,
    {singleQuery,
     [{clause,
       {match,[],
        {pattern,
         [{patternPart,[],
           {anonymousPatternPart,
            {patternElement,
             {nodePattern,{variable,...},{...},...},
             []}}}]},
        {where,
         {expression,
          {orExpression,
           {xorExpression,
            {andExpression,{notExpression,{...},...},[]},
            []},
           []}}}}},
      {clause,
       {return,[],
        {returnBody,
         {returnItems,[],[],
          [{returnItem,{expression,{orExpression,...}},[]}]},
         [],[],[]}}}]},
    []}}},
 []}

Access the token list of the example code:

3> Tokens.
[{'MATCH',1},
 {'(',1},
 {'UNESCAPED_SYMBOLIC_NAME',1,"m"},
 {':',1},
 {'UNESCAPED_SYMBOLIC_NAME',5,"Movie"},
 {')',1},
 {'WHERE',1},
 {'UNESCAPED_SYMBOLIC_NAME',1,"m"},
 {'.',1},
 {'UNESCAPED_SYMBOLIC_NAME',5,"title"},
 {'=',1},
 {'STRING_LITERAL',1,"'The Matrix'"},
 {'RETURN',1},
 {'UNESCAPED_SYMBOLIC_NAME',1,"m"}]

Compile the code from a parse tree:

4> ocparse:pt_to_source_td(ParseTree).
<<"match (m :Movie) where m .title = 'The Matrix' return m">>

5> ocparse:pt_to_source_bu(ParseTree).
<<"match (m :Movie) where m .title = 'The Matrix' return m">>

Complete parse tree:

The output of the parse tree in the Erlang shell is shortened (cause not known). The complete parse tree of the example code looks as follows:

{cypher,
 {statement,
  {query,
   {regularQuery,
    {singleQuery,
     [{clause,
       {match,[],
        {pattern,
         [{patternPart,[],
           {anonymousPatternPart,
            {patternElement,
             {nodePattern,
              {variable,{symbolicName,"m"}},
              {nodeLabels,
               [{nodeLabel,
                 {labelName,
                  {schemaName,{symbolicName,"Movie"}}}}]},
              []},
             []}}}]},
        {where,
         {expression,
          {orExpression,
           {xorExpression,
            {andExpression,
             {notExpression,
              {comparisonExpression,
               {addOrSubtractExpression,
                {multiplyDivideModuloExpression,
                 {powerOfExpression,
                  {unaryAddOrSubtractExpression,
                   {stringListNullOperatorExpression,
                    {propertyOrLabelsExpression,
                     {atom,{variable,{symbolicName,"m"}}},
                     [{propertyLookup,
                       {propertyKeyName,
                        {schemaName,{symbolicName,"title"}}}}]},
                    []},
                   []},
                  []},
                 []},
                []},
               [{partialComparisonExpression,
                 {addOrSubtractExpression,
                  {multiplyDivideModuloExpression,
                   {powerOfExpression,
                    {unaryAddOrSubtractExpression,
                     {stringListNullOperatorExpression,
                      {propertyOrLabelsExpression,
                       {atom,
                        {literal,{stringLiteral,"'The Matrix'"}}},
                       []},
                      []},
                     []},
                    []},
                   []},
                  []},
                 "="}]},
              []},
             []},
            []},
           []}}}}},
      {clause,
       {return,[],
        {returnBody,
         {returnItems,[],[],
          [{returnItem,
            {expression,
             {orExpression,
              {xorExpression,
               {andExpression,
                {notExpression,
                 {comparisonExpression,
                  {addOrSubtractExpression,
                   {multiplyDivideModuloExpression,
                    {powerOfExpression,
                     {unaryAddOrSubtractExpression,
                      {stringListNullOperatorExpression,
                       {propertyOrLabelsExpression,
                        {atom,{variable,{symbolicName,"m"}}},
                        []},
                       []},
                      []},
                     []},
                    []},
                   []},
                  []},
                 []},
                []},
               []},
              []}},
            []}]},
         [],[],[]}}}]},
    []}}},
 []}

2. Documentation

The documentation for ocparse is available here: Wiki.

3. Known issues of grammar support

Comment

The number of block comments (/* ... */) is limted to one per line.

Properties / Literal

The rule Properties has a higher precedence than the rule Literal.

SymbolicName

The following tokens may not be used as SymbolicName:

  ALL AND ANY AS ASC ASCENDING BY CONTAINS COUNT CREATE DECIMAL_INTEGER DELETE
  DESC DESCENDING DETACH DISTINCT ENDS ESCAPED_SYMBOLIC_NAME EXPONENT_DECIMAL_REAL
  EXTRACT FALSE FILTER HEX_INTEGER IN IS LIMIT MATCH MERGE NONE NOT NULL
  OCTAL_INTEGER ON OPTIONAL OR ORDER REGULAR_DECIMAL_REAL REMOVE RETURN SET
  SINGLE SKIP STARTS STRING_LITERAL TRUE UNESCAPED_SYMBOLIC_NAME UNION UNWIND
  WHERE WITH XOR

An exception is the use of the token COUNT as FunctionName.

Unicode

Unicode is not supported with Dash, LeftArrowHead, RightArrowHerad or UnescapedSymbolicName. Hence Dash is limited to the hyphen (-), LeftArrowHead is limited to '<' and RightArrowHead is limited to '>'.

4. Limitations of the test data generator

In the scripts test\gen_test.bat and test\gen_test_and_run.bat, the heap size has been changed to speed up test data generation. If necessary, you are welcome to make suitable adjustments for your purposes.

No test data is generated for the following rules:

FunctionInvocation

FunctionInvocation = FunctionName, [SP], '(', [SP], (D,I,S,T,I,N,C,T), ')' ;

MultiPartQuery

Instead of

MultiPartQuery = (ReadPart | (UpdatingStartClause, [SP], UpdatingPart)), With, [SP], { ReadPart, UpdatingPart, With, [SP] }, SinglePartQuery ;

it is only used

MultiPartQuery = (ReadPart | (UpdatingStartClause, [SP], UpdatingPart)), With, [SP], { ReadPart, With, [SP] }, SinglePartQuery ;

SchemaName

SchemaName = ... | ReservedWord ;

SymbolicName

SymbolicName = ... | (C,O,U,N,T) | (F,I,L,T,E,R) | (E,X,T,R,A,C,T) | (A,N,Y) | (N,O,N,E) | (S,I,N,G,L,E) ;

5. Acknowledgement

This project was inspired by the sqlparse project of the company K2 Informatics GmbH.