Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] manyclangs: Add preliminary support for compiler libraries #4600

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

veselink1
Copy link

This PR is the first step of manyclangs integration into CE and introduces some UI and code-behind changes.

In order to support something like manyclangs, which packages thousands of revisions of clang+some LLVM tools, I've added the concept of compiler libraries.

A compiler library is any instance of CompilerBase which has isCompilerLibrary set to true and exposes a getRevisions method, listing all the available revisions. Likely not the prettiest way to integrate something like this, happy to hear feedback.

If a compiler library is selected via the UI, a new "revision picker" is displayed, which provides search and pagination of the available compiler revisions, in addition to prev/next buttons to browser through revisions one-by-one.

Pagination is implemented by the /compilerLibrary/:compiler/search endpoint, which accepts an optional query, offset and limit.

This change also lays the groundwork for including manyclangs as a compiler library. Currently, I've made the compiler definition bind to /usr/bin/yes, because I'm not sure how we would want to process the compiler args in the backend.

  • Add a way for users to interact with manyclangs from the UI
  • Implement getRevisions() for manyclangs (via elfshaker list or similar)
  • Handle arguments for compiler libraries in complie() (we likely don't need to extend the command line as we currently do) and make it call manyclangs-run or some custom wrapper
2023-01-15.14-27-59.mp4

@RubenRBS
Copy link
Member

This is extremelly cool. Wow. Great job with this :)

@partouf
Copy link
Contributor

partouf commented Jan 15, 2023

I wonder if the name compiler-library is maybe confusing with normal libraries. Maybe compiler-repository would be more suitable?

@mattgodbolt
Copy link
Member

Hi @veselink1 ! Lovely to see this. Sorry you're seeing so many conflicts : unfortunately it became high priority to move everything to typescript (which has been our primary focus this year so far). We're nearly there now, and hopefully it won't be too painful. Looking great so far: will skim over the PR as-is shortly

Copy link
Member

@mattgodbolt mattgodbolt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have not done a thorough review but man this looks great as a start!

lib/handlers/api.js Outdated Show resolved Hide resolved
lib/compilers/manyclangs.ts Outdated Show resolved Hide resolved
lib/handlers/compile.js Outdated Show resolved Hide resolved
lib/handlers/compile.js Outdated Show resolved Hide resolved
views/templates/panes/compiler.pug Outdated Show resolved Hide resolved
views/templates/panes/compiler.pug Outdated Show resolved Hide resolved
views/templates/panes/compiler.pug Outdated Show resolved Hide resolved
static/compiler-revision-picker.ts Outdated Show resolved Hide resolved
static/compiler-revision-picker.ts Outdated Show resolved Hide resolved
Compiler repositories are compiler toolchains which provide access to a
large number of compiler revisions.

A compiler repository is any instance of `ICompiler` which has
`isCompilerLibrary` set to `true`, which means it also implements the
`ICompilerRepository` interface.

If a compiler repository is selected via the UI, a new "revision picker" is
displayed, which provides search and pagination of the available
compiler revisions, in addition to prev/next buttons to browser through
revisions one-by-one.

Pagination is implemented by the `/compilerRevision/:compiler/search`
endpoint, which accepts an optional query, offset and limit.

This change also lays the groundwork for including manyclangs as a
compiler repository.

TODO:

- Implement getRevisions() for manyclangs (via `elfshaker list` or
similar)
- Handle arguments for compiler repository in complie() (we likely
don'ot need to extend the command line as we currently do) and make it
call manyclangs-run or some custom wrapper
@mattgodbolt
Copy link
Member

Hey @veselink1 do let us know if you need a hand; this seems to have fallen by the wayside. I spoke with @pwaller the other day and he implied that you might have been blocked on us: let me know if that's the case please!

@veselink1
Copy link
Author

Hey @mattgodbolt, sorry for taking so long to reply. The UI portion of the work is pretty much complete. It provides a revision picker and a way for the backend to expose manyclangs-style compiler repositories through it. I was pretty much hoping either you or someone more familiar could take care of the remaining two tasks - exposing the list of revisions to the UI and doing the actual compilation :)

@mattgodbolt
Copy link
Member

Thanks @veselink1 ! Understood. I think we're all suffering from "not enough time". I know I'm drowning just trying to keep the bitrot/entropy in check...

@partouf partouf marked this pull request as draft May 29, 2024 16:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants