Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extension breaking after upgrading from 0.35.5 to 1.0.0a8 #6560

Closed
tugceakin opened this issue Jun 13, 2019 · 22 comments
Closed

Extension breaking after upgrading from 0.35.5 to 1.0.0a8 #6560

tugceakin opened this issue Jun 13, 2019 · 22 comments
Labels
status:resolved-locked Closed issues are locked after 30 days inactivity. Please open a new issue for related discussion.
Milestone

Comments

@tugceakin
Copy link

tugceakin commented Jun 13, 2019

In our extension we provide our own layout restorer by disabling the default restorer and activating ours.

The main difference in our custom layout restorer is the use of IDocumentManager to open widgets during restoration process. We built this to save custom layouts in a DB and load them during initialization.

This was working fine in 0.35.x but it's breaking in 1.0.0a8 with Uncaught RangeError: Maximum call stack size exceeded so I think there is a circular dependency somewhere. I don't get this error if I remove IDocumentManager from requires array. I would love to hear your suggestions if you think this problem could be solved differently.

To Reproduce
https://github.com/tugceakin/jupyterlab-dependency-bug

1 - Disable the default restorer:
jupyter labextension disable @jupyterlab/application-extension:layout

2 - Then activate the custom extension (I refactored this to accommodateapp.shell(0.35) -> ILabShell(1.0.0x) changes) :

export const layoutExtension: JupyterFrontEndPlugin<ILayoutRestorer> = {
    id: '@<my-id>/application-extension:layout',
    //id: '@jupyterlab/application:ILayoutRestorer',
    requires: [IStateDB, ILabShell, IDocumentManager],
    activate: (app: JupyterFrontEnd, state: IStateDB, shell: ILabShell, docmanager: DocumentManager) => {
        // This part is copied from @jupyterlab/application-extension:layout plugin
      const initLayoutRestorer = (layout: any) => {
        restorer.fetch().then(saved => {
          shell.restoreLayout(layout || saved);
          shell.layoutModified.connect(() => {
            restorer.save(shell.saveLayout());
          });
        });
      }
      // Do things and call initLayoutRestorer at the end with the new layout
      return restorer;
    },
    autoStart: true,
    provides: ILayoutRestorer

3 - Install the extension jupyter labextension install .
4 - Open JupyterLab

Expected behavior
To be able to use IDocumentManager in ILayoutRestorer extension.

Screenshots
Screen Shot 2019-06-12 at 6 33 37 PM

Desktop

  • OS: macOS Mojave 10.14.1
  • Browser [e.g. chrome 69.0, firefox 62.0]
  • JupyterLab [e.g. 0.34.2]

** Additional Info**

Updated package.json dependencies
```
  "dependencies": {
    "@jupyterlab/application": "^1.0.0-alpha.11",
    "@jupyterlab/apputils": "^1.0.0-alpha.11",
    "@jupyterlab/coreutils": "^3.0.0-alpha.11",
    "@jupyterlab/docmanager": "^1.0.0-alpha.11", 
    "@jupyterlab/filebrowser": "^1.0.0-alpha.11",
    "@jupyterlab/fileeditor": "^1.0.0-alpha.11",
    "@jupyterlab/launcher": "^1.0.0-alpha.11",
    "@jupyterlab/mainmenu": "^1.0.0-alpha.11",
    "@jupyterlab/notebook": "^1.0.0-alpha.12",
    "@jupyterlab/terminal": "^1.0.0-alpha.11",
    "@jupyterlab/theme-dark-extension": "^1.0.0-alpha.12",
    "@phosphor/coreutils": "^1.3.0",
    "@phosphor/messaging": "^1.2.2",
    "@phosphor/widgets": "^1.6.0",
    "@types/lodash": "^4.14.123",
    "@types/react-dom": "^16.8.2",
    "classnames": "^2.2.6",
    "downshift": "^3.2.10"
  }
Old package.json dependencies
```
  "dependencies": {
    "@jupyterlab/application": "^0.19.1",
    "@jupyterlab/apputils": "^0.19.1",
    "@jupyterlab/coreutils": "^2.2.1",
    "@jupyterlab/docmanager": "^0.19.1",
    "@jupyterlab/filebrowser": "^0.19.3",
    "@jupyterlab/fileeditor": "^0.19.1",
    "@jupyterlab/launcher": "^0.19.1",
    "@jupyterlab/mainmenu": "^0.8.1",
    "@jupyterlab/notebook": "^0.19.2",
    "@jupyterlab/terminal": "^0.19.1",
    "@jupyterlab/theme-dark-extension": "^0.19.1",
    "@phosphor/coreutils": "^1.3.0",
    "@phosphor/messaging": "^1.2.2",
    "@phosphor/widgets": "^1.6.0",
    "@types/lodash": "^4.14.123",
    "@types/react-dom": "^16.8.2",
    "classnames": "^2.2.6",
    "downshift": "^3.2.10"
  },
```
Troubleshoot Output
You are using pip version 10.0.1, however version 19.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
$PATH:
        /usr/local/Cellar/jupyter/1.0.0_4/libexec/bin
        /Users/tugce.akin/.nvm/versions/node/v11.15.0/bin
        /Users/tugce.akin/google-cloud-sdk/bin
        /usr/local/bin
        /usr/bin
        /bin
        /usr/sbin
        /sbin
        /Applications/Postgres.app/Contents/Versions/latest/bin
        /miniconda3/bin
        /miniconda3/condabin
        /Users/tugce.akin/google-cloud-sdk/bin

sys.path:
/usr/local/Cellar/jupyter/1.0.0_4/libexec/bin
/usr/local/Cellar/jupyter/1.0.0_4/libexec/lib/python3.6/site-packages
/usr/local/Cellar/jupyter/1.0.0_4/libexec/vendor/lib/python3.6/site-packages
/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python36.zip
/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6
/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/lib-dynload
/Users/tugce.akin/Library/Python/3.6/lib/python/site-packages
/usr/local/lib/python3.6/site-packages

sys.executable:
/usr/local/opt/python/bin/python3.6

sys.version:
3.6.5 (default, Apr 25 2018, 14:23:58)
[GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.1)]

platform.platform():
Darwin-18.2.0-x86_64-i386-64bit

which -a jupyter:
/usr/local/Cellar/jupyter/1.0.0_4/libexec/bin/jupyter
/usr/local/bin/jupyter
/miniconda3/bin/jupyter

pip list:
Package Version
---------------------------------- ---------
appnope 0.1.0
astroid 2.0.4
backcall 0.1.0
backports-abc 0.5
backports.shutil-get-terminal-size 1.0.0
bleach 2.1.3
certifi 2018.4.16
configparser 3.5.0
decorator 4.3.0
entrypoints 0.2.3
html5lib 1.0.1
ipykernel 4.8.2
ipython 6.4.0
ipython-genutils 0.2.0
ipywidgets 7.2.1
isort 4.3.4
jedi 0.12.0
Jinja2 2.10
jsonschema 2.6.0
jupyter-client 5.2.3
jupyter-console 5.2.0
jupyter-core 4.4.0
lazy-object-proxy 1.3.1
MarkupSafe 1.0
mccabe 0.6.1
mistune 0.8.3
nbconvert 5.3.1
nbformat 4.4.0
notebook 5.5.0
pandocfilters 1.4.2
parso 0.2.0
pathlib2 2.3.2
pexpect 4.5.0
pickleshare 0.7.4
pip 10.0.1
prompt-toolkit 1.0.15
ptyprocess 0.5.2
Pygments 2.2.0
pylint 2.1.1
python-dateutil 2.7.3
pyzmq 17.0.0
scandir 1.7
Send2Trash 1.5.0
setuptools 39.0.1
simplegeneric 0.8.1
singledispatch 3.4.0.3
six 1.11.0
terminado 0.8.1
testpath 0.3.1
tornado 5.0.2
traitlets 4.3.2
typed-ast 1.1.0
wcwidth 0.1.7
webencodings 0.5.1
wheel 0.31.0
widgetsnbextension 3.2.1
wrapt 1.10.11

conda list:
# packages in environment at /miniconda3:
#
# Name Version Build Channel
appnope 0.1.0 pypi_0 pypi
asn1crypto 0.24.0 py37_0
attrs 18.2.0 py_0 conda-forge
backcall 0.1.0 py_0 conda-forge
bleach 3.1.0 py_0 conda-forge
ca-certificates 2018.11.29 ha4d7672_0 conda-forge
certifi 2018.11.29 py37_1000 conda-forge
cffi 1.11.5 py37h6174b99_1
chardet 3.0.4 py37_1
conda 4.6.4 py37_0 conda-forge
conda-env 2.6.0 1
cryptography 2.5 py37ha12b0ac_0
decorator 4.3.0 pypi_0 pypi
entrypoints 0.3 py37_1000 conda-forge
idna 2.7 py37_0
ipykernel 5.1.0 py37h24bf2e0_1002 conda-forge
ipython 7.1.1 pypi_0 pypi
ipython-genutils 0.2.0 pypi_0 pypi
ipython_genutils 0.2.0 py_1 conda-forge
jedi 0.13.1 pypi_0 pypi
jinja2 2.10 py_1 conda-forge
jsonschema 3.0.1 pypi_0 pypi
jupyter-client 5.2.3 pypi_0 pypi
jupyter_client 5.2.4 py_1 conda-forge
jupyter_core 4.4.0 py_0 conda-forge
jupyterlab 1.0.0a8 pypi_0 pypi
jupyterlab-server 1.0.0rc0 pypi_0 pypi
libcxx 4.0.1 h579ed51_0
libcxxabi 4.0.1 hebd6815_0
libedit 3.1.20170329 hb402a30_2
libffi 3.2.1 h475c297_4
libsodium 1.0.16 h1de35cc_1001 conda-forge
markupsafe 1.1.0 py37h1de35cc_1000 conda-forge
mistune 0.8.4 py37h1de35cc_1000 conda-forge
nbconvert 5.3.1 py_1 conda-forge
nbformat 4.4.0 py_1 conda-forge
ncurses 6.1 h0a44026_0
notebook 5.7.4 py37_1000 conda-forge
openssl 1.1.1a h1de35cc_1000 conda-forge
pandoc 2.6 1 conda-forge
pandocfilters 1.4.2 py_1 conda-forge
parso 0.3.1 pypi_0 pypi
pexpect 4.6.0 pypi_0 pypi
pickleshare 0.7.5 pypi_0 pypi
pip 10.0.1 py37_0
prometheus_client 0.6.0 py_0 conda-forge
prompt-toolkit 2.0.7 pypi_0 pypi
prompt_toolkit 2.0.9 py_0 conda-forge
ptyprocess 0.6.0 pypi_0 pypi
pycosat 0.6.3 py37h1de35cc_0
pycparser 2.18 py37_1
pygments 2.3.0 pypi_0 pypi
pyopenssl 18.0.0 py37_0
pyrsistent 0.14.10 py37h1de35cc_0 conda-forge
pysocks 1.6.8 py37_0
python 3.7.2 haf84260_0
python-dateutil 2.7.5 pypi_0 pypi
python.app 2 py37_8
pyzmq 17.1.2 pypi_0 pypi
readline 7.0 h1de35cc_5
requests 2.19.1 py37_0
ruamel_yaml 0.15.46 py37h1de35cc_0
send2trash 1.5.0 py_0 conda-forge
setuptools 40.2.0 py37_0
six 1.11.0 py37_1
sqlite 3.26.0 h1765d9f_1000 conda-forge
terminado 0.8.1 py37_1001 conda-forge
testpath 0.4.2 py37_1000 conda-forge
tk 8.6.8 ha441bb4_0
tornado 5.1.1 pypi_0 pypi
traitlets 4.3.2 pypi_0 pypi
urllib3 1.23 py37_0
wcwidth 0.1.7 pypi_0 pypi
webencodings 0.5.1 py_1 conda-forge
wheel 0.31.1 py37_0
xz 5.2.4 h1de35cc_4
yaml 0.1.7 hc338f04_2
zeromq 4.2.5 h0a44026_1006 conda-forge
zlib 1.2.11 hf3cbc9b_2

Browser Output
index.js:140 Uncaught RangeError: Maximum call stack size exceeded
    at index.js:140
    at Array.map ()
    at JupyterLab.push.2R+v.Application.activatePlugin (index.js:140)
    at JupyterLab.push.2R+v.Application.resolveOptionalService (index.js:223)
    at index.js:142
    at Array.map ()
    at JupyterLab.push.2R+v.Application.activatePlugin (index.js:142)
    at JupyterLab.push.2R+v.Application.resolveRequiredService (index.js:190)
    at index.js:140
    at Array.map ()
@blink1073
Copy link
Member

Hi @tugceakin, it sounds like duplicate dependencies. Is your code available in a public repo?

@tugceakin
Copy link
Author

tugceakin commented Jun 13, 2019

Hi @blink1073, I created a simplified public repo to reproduce the issue: https://github.com/tugceakin/jupyterlab-dependency-bug

I also added updated and previous version of my package.json dependencies to the issue comment.

@blink1073
Copy link
Member

Found it! Tokens are meant to be singletons, and your extension was returning ILayoutRestorer when that was already provided by the application extension here. The DI system was getting stuck trying to resolve.

Here is a working diff:

diff --git a/src/index.ts b/src/index.ts
index 5949fbc..b9479c6 100644
--- a/src/index.ts
+++ b/src/index.ts
@@ -14,7 +14,7 @@ import {
   IStateDB
 } from '@jupyterlab/coreutils';
 
-export const layoutExtension: JupyterFrontEndPlugin<ILayoutRestorer> = {
+export const layoutExtension: JupyterFrontEndPlugin<void> = {
   id: '@ulab/application-extension:layout',
   //id: '@jupyterlab/application:ILayoutRestorer',
 
@@ -23,8 +23,7 @@ export const layoutExtension: JupyterFrontEndPlugin<ILayoutRestorer> = {
   // Doesn't work
   requires: [IStateDB, ILabShell, IDocumentManager],
 
-  activate: (app: JupyterFrontEnd, state: IStateDB, shell: ILabShell, docmanager: IDocumentManager) => {//, docmanager: IDocumentManager) => {
-
+  activate: (app: JupyterFrontEnd, state: IStateDB, shell: ILabShell, docmanager: IDocumentManager) => {
     const first = app.started;
     const registry = app.commands;
     const restorer = new LayoutRestorer({ first, registry, state });
@@ -40,10 +39,9 @@ export const layoutExtension: JupyterFrontEndPlugin<ILayoutRestorer> = {
     }
     // Do things
     initLayoutRestorer(null);
-    return restorer;
+    return;
     },
-    autoStart: true,
-    provides: ILayoutRestorer
+    autoStart: true
   };

@blink1073
Copy link
Member

@afshin do you think this is something we could catch? More than one plugin trying to provide the same token?

@vidartf
Copy link
Member

vidartf commented Jun 13, 2019

@blink1073 The original submission says:

In our extension we provide our own layout restorer by disabling the default restorer and activating ours.

So I don't think asking them to stop providing the token will work?

@vidartf
Copy link
Member

vidartf commented Jun 13, 2019

I do think that they should probably ensure that they supply a unique plugin ID though!

I see that they already do, I missed the deletion of the leading characters.

@vidartf
Copy link
Member

vidartf commented Jun 13, 2019

One dependency loop (there might be more):

  • User code
  • IDocumentManager
  • IMainMenu
  • IInspector (optional)
  • ILayoutRestorer (user code)

@vidartf
Copy link
Member

vidartf commented Jun 13, 2019

Regarding that loop, the relevant question would be if it would be better for the IInspector provider to add its own commands to the file menu. This seems like a reasonable request, as there is no guarantee that all providers of IInspector will add the command inspector:open. This highlights some deeper questions though:

  • Should the mainmenu extension make assumptions about which commands the other providers will supply?
  • Should we make (a subset of) the supplied commands a part of the token interface guarantee?

@vidartf
Copy link
Member

vidartf commented Jun 13, 2019

PS: Phosphor does have code for detecting cyclical token dependencies: https://github.com/phosphorjs/phosphor/blob/0d842588195ec7b056db946ae0c55de2ef06886f/packages/application/src/index.ts#L212-L213

@vidartf
Copy link
Member

vidartf commented Jun 13, 2019

I opened an issue on the phosphor repo about the cycle check (phosphorjs/phosphor#389), but that should not supersede the discussion in this issue.

@afshin
Copy link
Member

afshin commented Jun 13, 2019

@afshin do you think this is something we could catch? More than one plugin trying to provide the same token?

I don't know if we can catch it in JupyterLab, but I think one level up, we could probably check if the extension's return value is identical to another on at the Phosphor level ... but is that sound behavior? We would probably have to special-case it to only identical Token instances, because imagine if we return 42 in two separate extensions ... that is probably supposed to be okay.

@vidartf
Copy link
Member

vidartf commented Jun 13, 2019

cc @ian-r-rose which previously suggested that duplicate tokens should always overwrite core ones.

@ian-r-rose
Copy link
Member

My thinking was, if an extension wants to override a core token, we should respect that and allow it to take precedence. We already do some logic around deferredExtensions, this might add another level of granularity.

@afshin
Copy link
Member

afshin commented Jun 13, 2019

What if there are multiple extensions that want to override the core token?

@ian-r-rose
Copy link
Member

Unsure. At that point, I suppose it could be last-write-wins, as documented in the phosphor docs.

@afshin
Copy link
Member

afshin commented Jun 13, 2019

Hm, I wonder if we should somehow let the developer know, maybe in the console by giving a hint that an extension was overridden and should probably be disabled.

@tugceakin
Copy link
Author

tugceakin commented Jun 13, 2019

Found it! Tokens are meant to be singletons, and your extension was returning ILayoutRestorer when that was already provided by the application extension here. The DI system was getting stuck trying to resolve.

Here is a working diff:

diff --git a/src/index.ts b/src/index.ts
index 5949fbc..b9479c6 100644
--- a/src/index.ts
+++ b/src/index.ts
@@ -14,7 +14,7 @@ import {
   IStateDB
 } from '@jupyterlab/coreutils';
 
-export const layoutExtension: JupyterFrontEndPlugin<ILayoutRestorer> = {
+export const layoutExtension: JupyterFrontEndPlugin<void> = {
   id: '@ulab/application-extension:layout',
   //id: '@jupyterlab/application:ILayoutRestorer',
 
@@ -23,8 +23,7 @@ export const layoutExtension: JupyterFrontEndPlugin<ILayoutRestorer> = {
   // Doesn't work
   requires: [IStateDB, ILabShell, IDocumentManager],
 
-  activate: (app: JupyterFrontEnd, state: IStateDB, shell: ILabShell, docmanager: IDocumentManager) => {//, docmanager: IDocumentManager) => {
-
+  activate: (app: JupyterFrontEnd, state: IStateDB, shell: ILabShell, docmanager: IDocumentManager) => {
     const first = app.started;
     const registry = app.commands;
     const restorer = new LayoutRestorer({ first, registry, state });
@@ -40,10 +39,9 @@ export const layoutExtension: JupyterFrontEndPlugin<ILayoutRestorer> = {
     }
     // Do things
     initLayoutRestorer(null);
-    return restorer;
+    return;
     },
-    autoStart: true,
-    provides: ILayoutRestorer
+    autoStart: true
   };

@blink1073 Thank you for looking into the issue! Yes, that's why I'm disabling application-extension:layout extension and providing my own layout restorer.
jupyter labextension disable @jupyterlab/application-extension:layout

If I don't disable this extension layout restoration logic could be racy I think.

I tried your code and now I'm getting another error:

Screen Shot 2019-06-13 at 8 32 38 AM

(That's why I have provides: ILayoutRestorer in my own custom extension)

@saulshanabrook saulshanabrook added this to the 1.0 milestone Jun 17, 2019
@michaelaye
Copy link

is this multiple declaration maybe the reason 1.0.0rc0 won't start in Safari@macOS for me? Just shows blank screen, and console gives this:

Screenshot 2019-06-24 22 08 45

In Chrome it launches, but complains about all the outdated extensions.

@vidartf
Copy link
Member

vidartf commented Jun 25, 2019

@tugceakin Two updates here:

  • With move contextual help to the help menu #6678, the dependency loop you had was broken (IMainMenu now no longer depends on IInspector), so your code should work with 1.0.0-rc.0.
  • With the latest patch release of @phosphor/application (1.6.4), the dependency loop detection now includes optional dependencies. This should give a much clearer error message in the future.

If you can confirm that your code works with the RC, we can close this issue I think, but there was quite a few topics discussed here, so maybe some other TODOs should be moved to separate issues?

@jasongrout
Copy link
Contributor

Given the above comment by @vidartf, moving this issue to 1.1. If there is indeed something crucial to 1.0, let's open a new specific issue to fix before 1.0.

@jasongrout jasongrout modified the milestones: 1.0, 1.1 Jun 25, 2019
@tugceakin
Copy link
Author

@vidartf @jasongrout Thank you! I'm not seeing call stack error after upgrading to 1.0.0-rc.0 — I haven't tested my extension very thoroughly but everything looks good.

@vidartf vidartf closed this as completed Jun 26, 2019
@lock
Copy link

lock bot commented Aug 6, 2019

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related discussion.

@lock lock bot locked as resolved and limited conversation to collaborators Aug 6, 2019
@jasongrout jasongrout added the status:resolved-locked Closed issues are locked after 30 days inactivity. Please open a new issue for related discussion. label Aug 9, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
status:resolved-locked Closed issues are locked after 30 days inactivity. Please open a new issue for related discussion.
Projects
None yet
Development

No branches or pull requests

8 participants