Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Issue Debuging pulling from a Private repo #4997

Open
ajulienla opened this issue May 14, 2024 · 0 comments
Open

Issue Debuging pulling from a Private repo #4997

ajulienla opened this issue May 14, 2024 · 0 comments

Comments

@ajulienla
Copy link

ajulienla commented May 14, 2024 •
edited

I have issue pulling from a private gitlab repository, and the error is unclear for me. Maybe clarifying it would be helpful for other users.

Expected behavior and actual behavior

As described on slack (initial problem description) , I am trying to pull an internal nextflow pipeline from within my company network. Our code is stored on gitlab.

The error obtained is not clear, It would be nice to be able to see the html page that seems to be returned . I am assuming it is a 404.

Following nextflow documentation, I updated my ~/.nextflow/scm file:

providers {
company_gitlab {
        platform = 'gitlab'
        user = 'julieal1'
        password = 'a_token_with_readright'
        token = 'a_token_with_readright'
        server = 'my_company_gitlab_server'

}
}

_ I tried with a personal token or a project token, both returned the same issue _

then using either nextflow pull or nextflow run I always retrieve the same error:

nextflow run subproject/project_name  -hub company_gitlab 
 Checking https://my_company_gitlab_server/some_user/nfcoreranseqfork ...
WARN: Cannot read project manifest -- Cause: Unable to determine the current character, it is not a string, number, array, or object

The current character read is '<' with an int value of 60
Unable to determine the current character, it is not a string, number, array, or object
line number 1
index number 0
<!DOCTYPE html>
^
Unable to determine the current character, it is not a string, number, array, or object

The current character read is '<' with an int value of 60
Unable to determine the current character, it is not a string, number, array, or object
line number 1
index number 0
<!DOCTYPE html>
^

It looks like a 404 but the log is not informative enough, nor do I see the html page returned to help me debug this.

I would like to be able to see something more informative.

Program output

May-07 11:47:30.242 [main] DEBUG nextflow.cli.Launcher - $> nextflow run subproject/project_name -hub company_gitlab
May-07 11:47:30.304 [main] INFO  nextflow.cli.CmdRun - N E X T F L O W  ~  version 23.10.1
May-07 11:47:30.329 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; embedded=false; plugins-dir=/home/myusername/.nextflow/plugins; core-plugins: nf-amazon@2.1.4,nf-azure@1.3.3,nf-cloudcache@0.3.0,nf-codecommit@0.1.5,nf-console@1.0.6,nf-ga4gh@1.1.0,nf-google@1.8.3,nf-tower@1.6.3,nf-wave@1.0.1
May-07 11:47:30.338 [main] INFO  o.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
May-07 11:47:30.339 [main] INFO  o.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
May-07 11:47:30.342 [main] INFO  org.pf4j.DefaultPluginManager - PF4J version 3.4.1 in 'deployment' mode
May-07 11:47:30.350 [main] INFO  org.pf4j.AbstractPluginManager - No plugins
May-07 11:47:30.362 [main] DEBUG nextflow.scm.ProviderConfig - Using SCM config path: /home/myusername/.nextflow/scm
May-07 11:47:30.799 [main] DEBUG nextflow.scm.ProviderConfig - Detected SCM config: [providers:[company_gitlab:[server:https://gitlab.company.com/team/directory, user:myusername, platform:gitlab, token:******]]]
May-07 11:47:30.812 [main] DEBUG nextflow.scm.RepositoryFactory - Found Git repository result: [RepositoryFactory]
May-07 11:47:30.826 [main] INFO  nextflow.cli.CmdRun - Pulling subproject/project_name ...
May-07 11:47:30.828 [main] DEBUG nextflow.scm.RepositoryProvider - Request [credentials ****:-] -> https://gitlab.company.com/team/directory/api/v4/projects/subproject%2Fproject_name
May-07 11:47:31.329 [main] WARN  nextflow.scm.AssetManager - Cannot read project manifest -- Cause: Unable to determine the current character, it is not a string, number, array, or object

The current character read is '<' with an int value of 60
Unable to determine the current character, it is not a string, number, array, or object
line number 1
index number 0
<!DOCTYPE html>
^
groovy.json.JsonException: Unable to determine the current character, it is not a string, number, array, or object

The current character read is '<' with an int value of 60
Unable to determine the current character, it is not a string, number, array, or object
line number 1
index number 0
<!DOCTYPE html>
^
	at org.apache.groovy.json.internal.JsonParserCharArray.decodeValueInternal(JsonParserCharArray.java:202)
	at org.apache.groovy.json.internal.JsonParserCharArray.decodeValue(JsonParserCharArray.java:153)
	at org.apache.groovy.json.internal.JsonParserCharArray.decodeFromChars(JsonParserCharArray.java:43)
	at org.apache.groovy.json.internal.JsonParserCharArray.parse(JsonParserCharArray.java:380)
	[... log too long]
May-07 11:47:31.337 [main] DEBUG nextflow.scm.RepositoryProvider - Request [credentials ****:-] -> https://gitlab.company.com/team/project/api/v4/projects/subproject%2Fproject_name
May-07 11:47:31.512 [main] ERROR nextflow.cli.Launcher - @unknown
groovy.json.JsonException: Unable to determine the current character, it is not a string, number, array, or object

The current character read is '<' with an int value of 60
Unable to determine the current character, it is not a string, number, array, or object
line number 1
index number 0
<!DOCTYPE html>
^
	at org.apache.groovy.json.internal.JsonParserCharArray.decodeValueInternal(JsonParserCharArray.java:202)
	at org.apache.groovy.json.internal.JsonParserCharArray.decodeValue(JsonParserCharArray.java:153)
	at org.apache.groovy.json.internal.JsonParserCharArray.decodeFromChars(JsonParserCharArray.java:43)
	at org.apache.groovy.json.internal.JsonParserCharArray.parse(JsonParserCharArray.java:380)
	at org.apache.groovy.json.internal.BaseJsonParser.parse(BaseJsonParser.java:113)
	at groovy.json.JsonSlurper.parseText(JsonSlurper.java:204)
	at nextflow.scm.RepositoryProvider.memoizedMethodPriv$invokeAndParseResponseString(RepositoryProvider.groovy:283)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107)
	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1254)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1030)
	at org.codehaus.groovy.runtime.InvokerHelper.invokePogoMethod(InvokerHelper.java:1036)
	at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:1019)
	at org.codehaus.groovy.runtime.InvokerHelper.invokeMethodSafe(InvokerHelper.java:97)
	at nextflow.scm.RepositoryProvider$_closure3.doCall(RepositoryProvider.groovy)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107)
	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
	at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:274)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1030)
	at groovy.lang.Closure.call(Closure.java:427)
	at org.codehaus.groovy.runtime.memoize.Memoize$MemoizeFunction.lambda$call$0(Memoize.java:137)
	at org.codehaus.groovy.runtime.memoize.ConcurrentCommonCache.getAndPut(ConcurrentCommonCache.java:137)
	at org.codehaus.groovy.runtime.memoize.ConcurrentCommonCache.getAndPut(ConcurrentCommonCache.java:113)
	at org.codehaus.groovy.runtime.memoize.Memoize$MemoizeFunction.call(Memoize.java:136)
	at groovy.lang.Closure.call(Closure.java:416)
	at nextflow.scm.RepositoryProvider.invokeAndParseResponse(RepositoryProvider.groovy)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.codehaus.groovy.runtime.callsite.PlainObjectMetaMethodSite.doInvoke(PlainObjectMetaMethodSite.java:48)
	at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:189)
	at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:57)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:185)
	at nextflow.scm.GitlabRepositoryProvider.getDefaultBranch(GitlabRepositoryProvider.groovy:60)
	at nextflow.scm.GitlabRepositoryProvider.getContentUrl(GitlabRepositoryProvider.groovy:88)
	at nextflow.scm.GitlabRepositoryProvider.readBytes(GitlabRepositoryProvider.groovy:115)
	at nextflow.scm.RepositoryProvider.validateFor(RepositoryProvider.groovy:310)
	at nextflow.scm.AssetManager.checkValidRemoteRepo(AssetManager.groovy:376)
	at nextflow.scm.AssetManager.download(AssetManager.groovy:584)
	at nextflow.cli.CmdRun.getScriptFile0(CmdRun.groovy:529)
	at nextflow.cli.CmdRun.getScriptFile(CmdRun.groovy:462)
	at nextflow.cli.CmdRun.run(CmdRun.groovy:317)
	at nextflow.cli.Launcher.run(Launcher.groovy:500)
	at nextflow.cli.Launcher.main(Launcher.groovy:672)

Environment

  • Nextflow version: 23.10.1
  • Java version: openjdk 11.0.22 2024-01-16
  • Operating system: Ubuntu 22.04
  • Bash version: bash -> 5.0.17(1)

Workaround

As a workaround we just pull from the repo directly and execture main.nf ourselves
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ajulienla