Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move to go 1.17 #12868

Merged
merged 15 commits into from Oct 21, 2021
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
494 changes: 247 additions & 247 deletions .circleci/config.yml

Large diffs are not rendered by default.

474 changes: 237 additions & 237 deletions .circleci/config/@build-release.yml

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion .circleci/config/commands/go_test.yml
Expand Up @@ -14,7 +14,7 @@ parameters:
default: false
go_image:
type: string
default: "docker.mirror.hashicorp.services/circleci/golang:1.16.7-buster"
default: "docker.mirror.hashicorp.services/circleci/golang:1.17.2-buster"
use_docker:
type: boolean
default: false
Expand Down
8 changes: 4 additions & 4 deletions .circleci/config/executors/@executors.yml
Expand Up @@ -3,7 +3,7 @@ go-machine:
shell: /usr/bin/env bash -euo pipefail -c
environment:
CIRCLECI_CLI_VERSION: 0.1.5546 # Pin CircleCI CLI to patch version (ex: 1.2.3)
GO_VERSION: 1.16.7 # Pin Go to patch version (ex: 1.2.3)
GO_VERSION: 1.17.2 # Pin Go to patch version (ex: 1.2.3)
GOTESTSUM_VERSION: 0.5.2 # Pin gotestsum to patch version (ex: 1.2.3)
GO_TAGS: ""
working_directory: /go/src/github.com/hashicorp/vault
Expand All @@ -25,23 +25,23 @@ alpine:
docker-env-go-test-remote-docker:
resource_class: medium
docker:
- image: "docker.mirror.hashicorp.services/circleci/golang:1.16.7-buster"
- image: "docker.mirror.hashicorp.services/circleci/golang:1.17.2-buster"
environment:
CIRCLECI_CLI_VERSION: 0.1.5546 # Pin CircleCI CLI to patch version (ex: 1.2.3)
GO_TAGS: ""
working_directory: /go/src/github.com/hashicorp/vault
docker-env-go-test:
resource_class: large
docker:
- image: "docker.mirror.hashicorp.services/circleci/golang:1.16.7-buster"
- image: "docker.mirror.hashicorp.services/circleci/golang:1.17.2-buster"
environment:
CIRCLECI_CLI_VERSION: 0.1.5546 # Pin CircleCI CLI to patch version (ex: 1.2.3)
GO_TAGS: ""
working_directory: /go/src/github.com/hashicorp/vault
docker-env-go-test-race:
resource_class: xlarge
docker:
- image: "docker.mirror.hashicorp.services/circleci/golang:1.16.7-buster"
- image: "docker.mirror.hashicorp.services/circleci/golang:1.17.2-buster"
environment:
CIRCLECI_CLI_VERSION: 0.1.5546 # Pin CircleCI CLI to patch version (ex: 1.2.3)
GO_TAGS: ""
Expand Down
2 changes: 1 addition & 1 deletion Makefile
Expand Up @@ -15,7 +15,7 @@ EXTERNAL_TOOLS=\
GOFMT_FILES?=$$(find . -name '*.go' | grep -v pb.go | grep -v vendor)


GO_VERSION_MIN=1.16.7
GO_VERSION_MIN=1.17.2
GO_CMD?=go
CGO_ENABLED?=0
ifneq ($(FDB_ENABLED), )
Expand Down
2 changes: 1 addition & 1 deletion README.md
Expand Up @@ -72,7 +72,7 @@ Developing Vault

If you wish to work on Vault itself or any of its built-in systems, you'll
first need [Go](https://www.golang.org) installed on your machine. Go version
1.16.7+ is *required*.
1.17.2+ is *required*.

For local dev first make sure Go is properly installed, including setting up a
[GOPATH](https://golang.org/doc/code.html#GOPATH). Ensure that `$GOPATH/bin` is in
Expand Down
9 changes: 9 additions & 0 deletions builtin/credential/approle/path_role.go
Expand Up @@ -18,6 +18,7 @@ import (
"github.com/hashicorp/vault/sdk/helper/policyutil"
"github.com/hashicorp/vault/sdk/helper/tokenutil"
"github.com/hashicorp/vault/sdk/logical"
k8snet "k8s.io/utils/net"
)

// roleStorageEntry stores all the options that are set on an role
Expand Down Expand Up @@ -818,6 +819,14 @@ func (b *backend) roleEntry(ctx context.Context, s logical.Storage, roleName str
needsUpgrade = true
}

for i, cidr := range role.SecretIDBoundCIDRs {
_, ipn, err := k8snet.ParseCIDRSloppy(cidr)
if err != nil {
return nil, fmt.Errorf("error decoding SecretIDBoundCIDRs field of role storage entry: %w", err)
}
role.SecretIDBoundCIDRs[i] = ipn.String()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is the case but still asking this for confirmation. I assume that you intentionally left out setting needsUpgrade, with the rationale that we don't necessarily need to fix this in storage, as we fix things with every read? (same case with ssh engine)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, though I'm open to discussing it. This is me being a bit cautious, since atm even if we migrated all the bad data, we'd still have to keep this code around indefinitely unless we started mandating stepwise upgrades. Given that, I didn't see any reason to take the extra risk of re-writing the records; if there's a bug somewhere, at least we won't damage the data, and once we release a fixed version they can move forward without any manual remediation.

}

if role.TokenPeriod == 0 && role.Period > 0 {
role.TokenPeriod = role.Period
}
Expand Down
42 changes: 38 additions & 4 deletions builtin/credential/approle/validation.go
Expand Up @@ -12,6 +12,7 @@ import (
"github.com/hashicorp/vault/sdk/helper/cidrutil"
"github.com/hashicorp/vault/sdk/helper/locksutil"
"github.com/hashicorp/vault/sdk/logical"
k8snet "k8s.io/utils/net"
)

// secretIDStorageEntry represents the information stored in storage
Expand Down Expand Up @@ -110,6 +111,39 @@ func (b *backend) secretIDAccessorLock(secretIDAccessor string) *locksutil.LockE
return locksutil.LockForKey(b.secretIDAccessorLocks, secretIDAccessor)
}

func decodeSecretIDStorageEntry(entry *logical.StorageEntry) (*secretIDStorageEntry, error) {
result := secretIDStorageEntry{}
if err := entry.DecodeJSON(&result); err != nil {
return nil, err
}

cleanup := func(in []string) ([]string, error) {
var out []string
swayne275 marked this conversation as resolved.
Show resolved Hide resolved
for _, s := range in {
_, ipn, err := k8snet.ParseCIDRSloppy(s)
if err != nil {
return nil, err
}
out = append(out, ipn.String())
}
return out, nil
}

cidrList, err := cleanup(result.CIDRList)
if err != nil {
return nil, fmt.Errorf("error decoding CIDRList field of secretid storage entry: %w", err)
}
result.CIDRList = cidrList

tokenCidrList, err := cleanup(result.TokenBoundCIDRs)
if err != nil {
return nil, fmt.Errorf("error decoding TokenBoundCIDRs field of secretid storage entry: %w", err)
}
result.TokenBoundCIDRs = tokenCidrList

return &result, nil
}

// nonLockedSecretIDStorageEntry fetches the secret ID properties from physical
// storage. The entry will be indexed based on the given HMACs of both role
// name and the secret ID. This method will not acquire secret ID lock to fetch
Expand All @@ -134,8 +168,8 @@ func (b *backend) nonLockedSecretIDStorageEntry(ctx context.Context, s logical.S
return nil, nil
}

result := secretIDStorageEntry{}
if err := entry.DecodeJSON(&result); err != nil {
result, err := decodeSecretIDStorageEntry(entry)
if err != nil {
return nil, err
}

Expand All @@ -154,12 +188,12 @@ func (b *backend) nonLockedSecretIDStorageEntry(ctx context.Context, s logical.S
}

if persistNeeded {
if err := b.nonLockedSetSecretIDStorageEntry(ctx, s, roleSecretIDPrefix, roleNameHMAC, secretIDHMAC, &result); err != nil {
if err := b.nonLockedSetSecretIDStorageEntry(ctx, s, roleSecretIDPrefix, roleNameHMAC, secretIDHMAC, result); err != nil {
return nil, fmt.Errorf("failed to upgrade role storage entry %w", err)
}
}

return &result, nil
return result, nil
}

// nonLockedSetSecretIDStorageEntry creates or updates a secret ID entry at the
Expand Down
8 changes: 4 additions & 4 deletions builtin/logical/ssh/util.go
Expand Up @@ -13,11 +13,11 @@ import (
"strings"
"time"

log "github.com/hashicorp/go-hclog"
"github.com/hashicorp/go-secure-stdlib/parseutil"
"github.com/hashicorp/vault/sdk/logical"

log "github.com/hashicorp/go-hclog"
"golang.org/x/crypto/ssh"
k8snet "k8s.io/utils/net"
)

// Creates a new RSA key pair with the given key length. The private key will be
Expand Down Expand Up @@ -142,11 +142,11 @@ func cidrListContainsIP(ip, cidrList string) (bool, error) {
return false, fmt.Errorf("IP does not belong to role")
}
for _, item := range strings.Split(cidrList, ",") {
_, cidrIPNet, err := net.ParseCIDR(item)
_, cidrIPNet, err := k8snet.ParseCIDRSloppy(item)
if err != nil {
return false, fmt.Errorf("invalid CIDR entry %q", item)
}
if cidrIPNet.Contains(net.ParseIP(ip)) {
if cidrIPNet.Contains(k8snet.ParseIPSloppy(ip)) {
return true, nil
}
}
Expand Down
3 changes: 3 additions & 0 deletions changelog/12868.txt
@@ -0,0 +1,3 @@
```release-note:improvement
core: build with Go 1.17, and mitigate a breaking change they made that could impact how approle and ssh interpret IPs/CIDRs
```