Need a Linter in this extension

[ChrisNiklasLucka]

Hi @Serhioromano,
to get nice and structured text, I need a linter in this extension. So in best case I want some text in the VS console, when naming of a variable is not right. Don't know if you can implement it? If you can do it I can put all linting-Tasks in a tabular here.

Best regards and stay safe
Chris

[Serhioromano]

I can do that. That would be nice to have all those rules.

But that will take time. It is done through LSP with is not a trivial task. On the other hand, if I do it, it will auto indent, with [CTRL] go to definition, intellisence will suggest correct properties of struct and function blocks and so on. It will full blown ST editor even better that any currently existing one like CoDeSys 3.5.

[ChrisNiklasLucka]

Hi @Serhioromano thats great. Here you'll have a documents with requirements from my side - about 25 requirements (not all for linter I think).
BR from Hamburg
Requirements_Linter.pdf

[Serhioromano]

@ChrisNiklasLucka this is very well written document. Thank you. Right now I am working on new Git Flow extension, but as soon as I finish it, I'll try to improve this one.

[ChrisNiklasLucka]

@Serhioromano thanks! Do you have any "timeline" for your programming goals? Do you think that this linter is ready in ~1 month or is it more a task about a half year? I ask, just du get an idea how long it whould take.
I have not that much knowledge in this programming language, but in other languages like python I have enought knowlegde, so if you can create a task line or something like this i could support you with my manpower.

[Serhioromano]

Right now, half year sounds more realistic since I am doing it alone. You see the point with lint is not only lint. In order to implement it LSP has to be created with full code parse capability. Lint will be just a byproduct of that. I mean to have lint a full blown code parser have to be created.

[ThirtySomething]

@Serhioromano Probably you want to use ATNLR4 for this task. I understand that this will be a complete re-write of the code, but a parser/lexer will work much better than a REGEXP based parser. You define the grammar for ST and generate the parser/lexer with ANTLR4. Unfortunately it will be in Javascript and not TypeScript, so I don't know if this will work as part of the VSCode extension which is based on TypeScript.

[Serhioromano]

@Serhioromano Probably you want to use ATNLR4 for this task. I understand that this will be a complete re-write of the code, but a parser/lexer will work much better than a REGEXP based parser. You define the grammar for ST and generate the parser/lexer with ANTLR4. Unfortunately it will be in Javascript and not TypeScript, so I don't know if this will work as part of the VSCode extension which is based on TypeScript.

Thank you. I was looking into this lib for quite a while already. Unfortunately I cannot find a guide how to do that. Docs I've found was not good enough. It is written as for someone who already understand parser and lexer just need to know how ATNLR works. I could not get a clear picture. Although I've created ATNLR file for ST.

[ThirtySomething]

@Serhioromano Probably you want to use ATNLR4 for this task. I understand that this will be a complete re-write of the code, but a parser/lexer will work much better than a REGEXP based parser. You define the grammar for ST and generate the parser/lexer with ANTLR4. Unfortunately it will be in Javascript and not TypeScript, so I don't know if this will work as part of the VSCode extension which is based on TypeScript.

Thank you. I was looking into this lib for quite a while already. Unfortunately I cannot find a guide how to do that. Docs I've found was not good enough. It is written as for someone who already understand parser and lexer just need to know how ATNLR works. I could not get a clear picture. Although I've created ATNLR file for ST.

I agree with you about the documentation. As far as I understood the process consists of a few step. First of all you have to define your grammar. Then you have to create the lexer/parser based on this grammar. Then you have to hook into the generated code (or vice versa) to work with the lexer/parser result.

I'll try to figure out how to work with this, too. I want to use it for a simple project of mine and also later on for a much more complex project.

[ThirtySomething]

You also need some kind of runtime. For JavaScript I suppose this is a good point to start.

[Serhioromano]

Ш was successful at creating grammar and then generate lexer and parser. The thing I do not completely understand how to turn it into LSP.

[ThirtySomething]

Ш was successful at creating grammar and then generate lexer and parser. The thing I do not completely understand how to turn it into LSP.

Oh, I understand. For this I'm not able to help. I'm struggling with my own project because it's in C++ and the ANTLR4 runtime is not compiling at the moment. But maybe this repository might throw some light on the topic. As far as I understood it's a plugin for VSCode using the LSP and supports ANTLR parse trees.

[Serhioromano]

Ш was successful at creating grammar and then generate lexer and parser. The thing I do not completely understand how to turn it into LSP.

Oh, I understand. For this I'm not able to help. I'm struggling with my own project because it's in C++ and the ANTLR4 runtime is not compiling at the moment. But maybe this repository might throw some light on the topic. As far as I understood it's a plugin for VSCode using the LSP and supports ANTLR parse trees.

Thank you for the link. That di not make anything clear as it is in C#. Although I used to code in C# decade ago and can read it partially, there are still lot of questions.

I also read this article
https://neuroning.com/post/implementing-code-completion-for-vscode-with-antlr/

After which I fill like stupid. How come he explained everything and I still cannot make it.

[ThirtySomething]

Okay, as far as I undrestand either the LSP itself or the interaction between LSP and ANTLR is not clear. This is also true for me - how to integrate own logic to the ANTLR generated code. Some googling came up with:

• LSP description
• Step By Step development
• Editor with ANTLR and LSP
  Unfortunately I got only basic skills in JS development and hope this will help.

[Serhioromano]

Those are useful links thank you. That would be a nice start.

I'm also looking toward golang gocc. This library is pure magic. It is the same concept you create BNF files and based on it parse it into parser, lexer and AST but as an outcome you can translate your piece of code to another language. Gor example convert ST file to C++ or use it as compiler to create executable directly.

My final dream is to make alternative to CoDeSys, so in VS Code you can write PLC program (but only one language ST) and then upload it to PLC. For that a small RT will be needed for different chips.

But unfortunately, although I consider myself a senior developer, this is not the area of my expertise.

[JohnnyOpcode]

Have a look at this repo. There was an IEC compiler written quite some time ago.

https://github.com/thiagoralves/OpenPLC_Editor/tree/master/matiec

[AndersHogqvist]

I'm also looking toward golang gocc. This library is pure magic. It is the same concept you create BNF files and based on it parse it into parser, lexer and AST but as an outcome you can translate your piece of code to another language. Gor example convert ST file to C++ or use it as compiler to create executable directly.

Sorry for highjacking this issue but that last sentence sounds very interesting for another reason. I'm looking into how to do an input filter for Doxygen in order to generate documentation from comments etc. I've read an article describing this process for turning structured text into C# pseudo code that Doxygen can interpret. If what you're saying is true, the part about converting ST to C++, that would be absolutely awesome since that is something Doxygen can read!

[AndersHogqvist]

This might come in handy. It contains the EBNF för ST that I suppose can be used as input to gocc:
https://www.researchgate.net/publication/228971719_A_syntactic_specification_for_the_programming_languages_of_the_IEC_61131-3_standard

[Serhioromano]

This might come in handy. It contains the EBNF för ST that I suppose can be used as input to gocc: https://www.researchgate.net/publication/228971719_A_syntactic_specification_for_the_programming_languages_of_the_IEC_61131-3_standard

Thank you. That article indeed very useful. It will not work for GOCC, there is slightly different variation of BNF but it might serve as a base for sure.

[AndersHogqvist]

I noticed that some language servers use tree-sitter so I looked into that. I have managed to create a grammar that successfully parses our source code and I'm currently looking into the syntax highlighting. It looks promising but it's still in a very very early experimental stage 🙂

[rempAut]

I also managed to implement the rules for parsing structured text using chevrotain. (example of what it is here)
I see that @AndersHogqvist is doing something similiar. I'm not sure which would be better, but I would like to help with your linter effort.

EDIT: you can try this with the chevrotain playground. It's the rules I created (only for the ST statements, it doesn't include var declarations and many other things... I was just playing around to see what can be done with the EBNF rules provided here).
I think I will eventually start a repo for it, to keep it organized and hopefully get some support.

[AndersHogqvist]

Both my tree-sitter grammar and the language server extension I'm working on is very specific to the way things are done in OpenPCS, I have no clue whether it would work with other environments as well. The grammar doesn't contain CONFIGURATION for instance since that's not something we use and I don't think it's covered in OpenPCS.

I'm also doing this as a consultant at company so the IP belongs to them, but once I'm done I will try to get an OK to publish both on GitHub (I think they will accept that actually).

[Serhioromano]

I also managed to implement the rules for parsing structured text using chevrotain. (example of what it is here) I see that @AndersHogqvist is doing something similiar. I'm not sure which would be better, but I would like to help with your linter effort.

This project looks awesome. Not only it is pure JS parser not a parser generator it can be used to create compiler too as long as LSP. Looks promising. I played with it but I stack with some issues which I cannot resolve. I posted an issue.

[Serhioromano]

@rempAut @AndersHogqvist Look at this https://github.com/langium/langium

Edited:
My goodness!!! It is that simple!

https://langium.org/playground/

Edited:

In aplay ground place this to grammar

grammar StructuredText

entry Document:
    (programs+=Program | functions+=Function | functionb+=FunctionBlock)*;

Program:
    'PROGRAM' name=ID 'END_PROGRAM';

FunctionBlock:
    'FUNCTION_BLOCK' name=ID 'END_FUNCTION_BLOCK';

Function:
    'FUNCTION' name=ID ':' dtype=Dtypes 'END_FUNCTION';

Dtypes:
    ('BOOL' | 'SINT' | 'USINT' | 'INT' | 'UINT' | 'DINT' | 'UDINT' | 'LINT' 
    | 'ULINT' | 'REAL' | 'LREAL' | 'TIME' | 'TOD' | 'DT' | 'BYTE' | 'WORD' 
    | 'DWORD' | functionb=[FunctionBlock:ID])*;

terminal ID: /[_a-zA-Z][\w_]*/;
terminal DIGIT returns number: /[0-9]+/;

hidden terminal WS: /\s+/;
hidden terminal ML_COMMENT1: /\/\*[\s\S]*?\*\//;
hidden terminal ML_COMMENT2: /\(\*[\s\S]*?\*\)/;
hidden terminal SL_COMMENT: /\/\/[^\n\r]*/;

and this to example

FUNCTION_BLOCK MyBlock

END_FUNCTION_BLOCK

FUNCTION MyFunc: MyBlock

END_FUNCTION

The types suggestions for a function return already working and go to definition too with Ctrl.

[rempAut]

@rempAut @AndersHogqvist Look at this https://github.com/langium/langium

Edited: My goodness!!! It is that simple!

It really looks like magic! I tried expressions in the playground too, it's mostly only re-writing the EBNF rules, and it works. This is very promising.

[Serhioromano]

If you fork this repo and run npm install then you can see LSP folder. There is langium, xtext and chevrotain tests. If you run node LSP/chevrotain.js you will see result in a terminal and html file with diagram will be generated.

[Serhioromano]

I was recently playing with langium and was manage do make running LSP you can see it in branch features/lsp. This works fine for may little syntax file in src/language-server/st.langium. All that is needed now if only make that file right.

Unfortunately, it is a complicated process of testing it. It takes too long. There is a generator and if that passes ok, then it should work. I'll be working on that file, but your help will be appreciated. As I noticed some of you (@rempAut ) are familiar with EBNF file format.

Anyway that BNF file provided by reseachlab linked by @AndersHogqvist will be quite useful as a start.