Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error on mass fetching az repos ref list #1355

Open
jeroensmink98 opened this issue May 12, 2023 · 1 comment
Open

Error on mass fetching az repos ref list #1355

jeroensmink98 opened this issue May 12, 2023 · 1 comment

Comments

@jeroensmink98
Copy link

This is autogenerated. Please review and update as needed.

Describe the bug

(See the additional content for my script) which uses the Azure CLI

I have a custom script which I use to fetch all branches for all my repositories under my project. The end goal would be to see what branches haven't been commited to in the last 120 days so we can mark them for deletion as part of our internal policy.

For that I wrote a custom script which fetches all repositories under a project and then fetches each branch for a repository but the Azure CLI doesn't really seem to like this. approach

Command Name
az devops invoke Extension Name: azure-devops. Version: 0.26.0.

Errors:

The command failed with an unexpected error. Here is the traceback:
'defaultBranch=refs/heads/main; id=6b9be419-daf8-480a-aee6-01d369a64384; isDisabled=False; isFork=; isInMaintenance=False; name=tres-internet-terraform; parentRepository=; project=; remoteUrl=https'
Traceback (most recent call last):
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\knack/cli.py", line 233, in invoke
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 663, in execute
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 726, in _run_jobs_serially
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 718, in _run_job
  File "C:\Users\JeroenSmink\.azure\cliextensions\azure-devops\azext_devops\dev\common\exception_handler.py", line 31, in azure_devops_exception_handler
    reraise(*sys.exc_info())
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\six.py", line 703, in reraise
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 697, in _run_job
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 333, in __call__
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/command_operation.py", line 121, in handler
  File "C:\Users\JeroenSmink\.azure\cliextensions\azure-devops\azext_devops\dev\team\invoke.py", line 101, in invoke
    response = client._send(http_method=http_method,
  File "C:\Users\JeroenSmink\.azure\cliextensions\azure-devops\azext_devops\devops_sdk\client.py", line 60, in _send
    request = self._create_request_message(http_method=http_method,
  File "C:\Users\JeroenSmink\.azure\cliextensions\azure-devops\azext_devops\devops_sdk\client.py", line 117, in _create_request_message
    request = ClientRequest(method=http_method, url=self._client.format_url(url))
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\msrest/service_client.py", line 151, in format_url
KeyError: 'defaultBranch=refs/heads/main; id=6b9be419-daf8-480a-aee6-01d369a64384; isDisabled=False; isFork=; isInMaintenance=False; name=tres-internet-terraform; parentRepository=; project=; remoteUrl=https'

To Reproduce:

Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

  • Put any pre-requisite steps here...
  • az devops invoke --area {} --resource {} --route-parameters {} {} {}

Expected Behavior

Environment Summary

Windows-10-10.0.22621-SP0
Python 3.10.5
Installer: MSI

azure-cli 2.40.0 *

Extensions:
azure-devops 0.26.0

Dependencies:
msal 1.18.0b1
azure-mgmt-resource 21.1.0b1

Additional Context

This is my powershell script

$project    = "SOME_PROJECT_NAME"
try {
    $repos = az repos list -p $project  | ConvertFrom-Json
} catch {
    Write-Host "Failed to fetch repositories for project $project"
    exit
}

$results = @()

Write-Host "Project: $project"
foreach ($repo in $repos) {
    Write-Host "Repository: $($repo.name)"
    try {
        $refs = az repos ref list -p $project -r $repo.name --filter heads | ConvertFrom-Json
    } catch {
        Write-Host "Failed to fetch refs for repository $($repo.name)"
        continue
    }

    foreach($ref in $refs) {
        $objectId = $ref.objectId

        # fetch individual commit details
        try {
            $commit = az devops invoke `
                         --area git `
                         --resource commits `
                         --route-parameters `
                             project=$project `
                             repositoryId=$repo.name `
                             commitId=$objectId |
                      ConvertFrom-Json
        } catch {
            Write-Host "Failed to fetch commit details for ref $objectId"
            continue
        }

        $result = [PSCustomObject]@{
            repository = $repo.name
            name = $ref.name
            creator = $ref.creator.uniqueName
            lastAuthor = $commit.committer.email
            lastModified = $commit.push.date
        }
        
        $results += ,$result
    }
    # add a 500ms wait
    Start-Sleep -Milliseconds 500
}

$results | Sort-Object -Property lastModified -Descending | Format-Table -AutoSize

# Write to CSV
$results | Sort-Object -Property lastModified -Descending | Export-Csv -Path "output.csv" -NoTypeInformation
@jeroensmink98
Copy link
Author

Here is the output when I set --verbose for the az devops invoke

INFO: GitDetect: Could not detect current remotes based on current working directory.
INFO: Detect: Url discovery took 0:00:00.037583
ERROR: The command failed with an unexpected error. Here is the traceback:
ERROR: 'defaultBranch=refs/heads/master; id=547913c5-ea8f-4d51-8221-005ce0246615; isDisabled=False; isFork=; isInMaintenance=False; name=efo-umbraco; parentRepository=; project=; remoteUrl=https'
Traceback (most recent call last):
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\knack/cli.py", line 233, in invoke
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 663, in execute
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 726, in _run_jobs_serially
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 718, in _run_job
  File "C:\Users\JeroenSmink\.azure\cliextensions\azure-devops\azext_devops\dev\common\exception_handler.py", line 31, in azure_devops_exception_handler
    reraise(*sys.exc_info())
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\six.py", line 703, in reraise
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 697, in _run_job
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 333, in __call__
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/command_operation.py", line 121, in handler
  File "C:\Users\JeroenSmink\.azure\cliextensions\azure-devops\azext_devops\dev\team\invoke.py", line 101, in invoke
    response = client._send(http_method=http_method,
  File "C:\Users\JeroenSmink\.azure\cliextensions\azure-devops\azext_devops\devops_sdk\client.py", line 60, in _send
    request = self._create_request_message(http_method=http_method,
  File "C:\Users\JeroenSmink\.azure\cliextensions\azure-devops\azext_devops\devops_sdk\client.py", line 117, in _create_request_message
    request = ClientRequest(method=http_method, url=self._client.format_url(url))
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\msrest/service_client.py", line 151, in format_url
KeyError: 'defaultBranch=refs/heads/master; id=547913c5-ea8f-4d51-8221-005ce0246615; isDisabled=False; isFork=; isInMaintenance=False; name=efo-umbraco; parentRepository=; project=; remoteUrl=https'
To open an issue, please run: 'az feedback'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant