Add support for Bitbucket Server as a ReaderProcessor

[goober]

Add Provider Configuration for Bitbucket Server Processor

Currently, the BitbucketReaderProcessor.ts does not support additional parameters such as target or apiBaseUrl.

Possible Implementation

Copy the providerconfig and implementation from GithubReaderProcessor in a way that makes sense for BitbucketReaderProcessor. Make sure that api differences between bitbucket.org and a self-hosted Bitbucket server are taken into acount.

Take inspiration of changes made in #2582

Context

Access a self-hosted Bitbucket server instance with a custom url.

[freben]

Related: #2665

[goober]

I can take a shot at this issue if it is accepted by the project

[stefanalund]

Thanks @goober -- I've assigned you!

[goober]

I have opened an issue against the git-url-parse repository IonicaBizau/git-url-parse#113 for adding support for bitbucket server url parsing.

I will hold on further implementation of this feature until the above pull request has been sorted out

[amitkhurana92]

Hi Team, any update on this? I need to integrate repo from bitbucket server.

[goober]

@amitkhurana92 Thanks for the reminder. IonicaBizau/git-url-parse#113 is now ready for review, and work can be continued when it has been accepted.

[amitkhurana92]

@goober the code you have suggested is not working. I think you also need to change in the type mapping

 const typeMapping = [
    { url: /^https:\/\/gitlab\.com\/.*/, type: 'gitlab/api' },
    { url: /^https:\/\/bitbucket\.org\/.*/, type: 'bitbucket/api' },
    { url: /^https:\/\/dev\.azure\.com\/.*/, type: 'azure/api' },
    { url: /.*/, type: 'github' },
  ];

Getting this also when I change the host in yaml file

Backend failed to start up Error: Bitbucket integration currently only supports 'bitbucket.org', tried to use host 'https://bitbucket.companyname.africa/bitbucket'
at new BitbucketUrlReader (webpack-internal:///../backend-common/src/reading/BitbucketUrlReader.ts:75:13)
at eval (webpack-internal:///../backend-common/src/reading/BitbucketUrlReader.ts:67:22)
at Array.map ()
at factory (webpack-internal:///../backend-common/src/reading/BitbucketUrlReader.ts:66:31)
at Function.create (webpack-internal:///../backend-common/src/reading/UrlReaders.ts:62:22)
at Function.default (webpack-internal:///../backend-common/src/reading/UrlReaders.ts:80:23)
at makeCreateEnv (webpack-internal:///./src/index.ts:59:94)
at main (webpack-internal:///./src/index.ts:76:21)

[goober]

The open pull request that is referred in the comment above is to add support in the underlying library that backstage is using for fetching content from bitbucket server. No work has started yet in the backstage project

[goober]

@amitkhurana92 Please feel free to test the open pull request #3030 to verify that my implementation solves your issues

[amitkhurana92]

@amitkhurana92 Please feel free to test the open pull request #3030 to verify that my implementation solves your issues

@goober good to see that. I will check this over the weekend.

[JdataNich]

Is there a template with an example of the Integration for the custom bitbucket urls?

[regicsolutions]

I would also benefit from any documentation on a sample config for Bitbucket Server.

I am getting Error: Unable to read url, Error: Unable to read https://bitbucket..... failed, reason: unable to get local issuer certificate

@goober is there a way to point to a cert or maybe disable it as a workaround for now?

[amitkhurana92]

Hi @goober, can you share an example where should I write the credentials?

[goober]

Hi @regicsolutions.
Sorry for the late reply. It looks like you are trying to access a bitbucket server that has a self-signed certificate.
We have the same situation and has solved it by telling node to trust our self-signed cert using NODE_EXTRA_CA_CERTS environment variable. See https://docs.axway.com/bundle/API_Builder_4x_allOS_en/page/enable_additional_ca_certificates.html for some guidance

[goober]

@JdataNich and @amitkhurana92 if you are using Bitbucket Server the following setup should work for you:

integrations:
   bitbucket:
      - host: bitbucket.mycompany.com
        apiBaseUrl: https://bitbucket.mycompany.com/rest/api/1.0
        token: <your personal access token> # https://confluence.atlassian.com/bitbucketserver/personal-access-tokens-939515499.html

[regicsolutions]

Thanks @goober ill give that a try. Have you had any luck integrating Bitbucket Server with the scaffolder backend for software templates?

[amitkhurana92]

@JdataNich and @amitkhurana92 if you are using Bitbucket Server the following setup should work for you:

integrations:
   bitbucket:
      - host: bitbucket.mycompany.com
        apiBaseUrl: https://bitbucket.mycompany.com/rest/api/1.0
        token: <your personal access token> # https://confluence.atlassian.com/bitbucketserver/personal-access-tokens-939515499.html

Hi Goober,

It is not working

This is my app-cofig.yaml
bitbucket: - host: https://bitbucket.mycompany.com/bitbucket apiBaseUrl: https://bitbucket.mycompany.com/bitbucket/rest/api/1.0 token: abcd

Below is the exception I am getting
Error: Expected object at root, got string at Module.generalError (webpack-internal:///../../node_modules/@backstage/plugin-catalog-backend/src/ingestion/processors/results.ts:59:56) at parseEntityYaml (webpack-internal:///../../node_modules/@backstage/plugin-catalog-backend/src/ingestion/processors/util/parse.ts:52:68) at parseEntityYaml.next (<anonymous>) at UrlReaderProcessor.readLocation (webpack-internal:///../../node_modules/@backstage/plugin-catalog-backend/src/ingestion/processors/UrlReaderProcessor.ts:64:18) at processTicksAndRejections (internal/process/task_queues.js:97:5) at async LocationReaders.handleLocation (webpack-internal:///../../node_modules/@backstage/plugin-catalog-backend/src/ingestion/LocationReaders.ts:142:13) at async LocationReaders.read (webpack-internal:///../../node_modules/@backstage/plugin-catalog-backend/src/ingestion/LocationReaders.ts:74:11) at async HigherOrderOperations.addLocation (webpack-internal:///../../node_modules/@backstage/plugin-catalog-backend/src/ingestion/HigherOrderOperations.ts:80:26) at async eval (webpack-internal:///../../node_modules/@backstage/plugin-catalog-backend/src/service/router.ts:115:22)

This is catalog-info.yaml
apiVersion: backstage.io/v1alpha1 kind: Component metadata: name: Backstage Test description: Backstage Test Desc spec: type: service lifecycle: production owner: amit-khurana

This is the URL I am entering https://bitbucket.mycompany.com/bitbucket/projects/project-name/repos/repo-name/browse/component.yaml?at=backstage-testing

[Rugvip]

For reference the issue ^ was https://bitbucket.mycompany.com/bitbucket not being a proper host. Added a check for that in #3692