Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: run tests on windows workers #2449

Open
wants to merge 57 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 37 commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
b6785c6
chore: run core on windows
mdelapenya Mar 18, 2024
a46f5db
revert: simplify pipeline
mdelapenya Mar 18, 2024
4045675
chore: do not use bridge driver on Windows
mdelapenya Mar 18, 2024
ddb0d94
docs: document driver
mdelapenya Mar 18, 2024
b3bd89f
Merge branch 'main' into windows-workers
mdelapenya Mar 25, 2024
27becea
chore: convert to constant
mdelapenya Mar 25, 2024
c07b9ee
chore: remove duplicated variable
mdelapenya Mar 25, 2024
a989e10
chore: do not expose windows constant
mdelapenya Mar 25, 2024
d874b4b
revert: run only on windows workers
mdelapenya Mar 26, 2024
1c36f14
fix: always detect OS and Docker Server version
mdelapenya Mar 26, 2024
977c28a
feat: run windows tests in Testcontainers Cloud
mdelapenya Mar 26, 2024
0f263ef
chore: inherit secrets in the reusable workflow
mdelapenya Mar 26, 2024
2da2575
chore: properly close docker client in tests
mdelapenya Mar 26, 2024
e5652c3
chore: do not apply nat driver on tcc
mdelapenya Mar 26, 2024
10ea4dd
chore: skip test for windows
mdelapenya Mar 26, 2024
82309e1
fix: use docker socket from testcontainers in subtests
mdelapenya Mar 26, 2024
2a497c2
Revert "chore: skip test for windows"
mdelapenya Mar 26, 2024
96a8ee7
chore: skip provider test on windows
mdelapenya Mar 26, 2024
17414f6
chore: better config tests for single initialisation
mdelapenya Mar 26, 2024
0b6ef18
fix: proper cleanup of docker client in tests
mdelapenya Mar 26, 2024
8b76f65
fix: do not set bridge as default network driver
mdelapenya Mar 26, 2024
50aa2b6
chore: check for remote host properly
mdelapenya Mar 26, 2024
2f303a1
fix: use extracted socket
mdelapenya Mar 26, 2024
d56bb5e
chore: simplify conditions
mdelapenya Mar 26, 2024
75705b0
fix: typo
mdelapenya Mar 26, 2024
5a7e0e9
chore: extract docker socket path verification to a function
mdelapenya Mar 26, 2024
7748feb
chore: add test for windows remote
mdelapenya Mar 26, 2024
4bf55ac
chore: skip host network tests on windows
mdelapenya Mar 26, 2024
b50dbd9
revert: log for errors
mdelapenya Mar 26, 2024
22c76de
chore: skip socket check tests
mdelapenya Mar 26, 2024
efef173
fix: do not timeout in tests
mdelapenya Mar 27, 2024
9c0d768
chore: skip test on Windows
mdelapenya Mar 27, 2024
dcd2737
chore: use pinned bash image
mdelapenya Mar 27, 2024
11905eb
revert: print container logs
mdelapenya Mar 27, 2024
db46e94
chore: leverage container lifecycle to create paths
mdelapenya Mar 27, 2024
19612a4
revert: use string
mdelapenya Mar 27, 2024
39c107d
chore: force LF line endings in Git
mdelapenya Mar 27, 2024
07e2781
Revert "revert: use string"
mdelapenya Mar 27, 2024
b8b7800
Revert "revert: print container logs"
mdelapenya Mar 27, 2024
0c9736e
Revert "revert: log for errors"
mdelapenya Mar 27, 2024
fe32906
Revert "revert: run only on windows workers"
mdelapenya Mar 27, 2024
02ac372
Revert "revert: simplify pipeline"
mdelapenya Mar 27, 2024
8f93deb
fix: update tests
mdelapenya Mar 27, 2024
e8ac173
chore: run core tests for PRs+Ubuntu or merge-commits+all platforms
mdelapenya Mar 27, 2024
b142495
revert: force always running tests
mdelapenya Mar 27, 2024
37463ff
fix: bring back removed modules
mdelapenya Mar 27, 2024
7acaea8
fix: pass secrets to reusable workflows
mdelapenya Mar 27, 2024
8a55ec3
chore: proper calculation of rootless socket
mdelapenya Mar 27, 2024
4756134
fix: create a default docker config.json file if it does not exists
mdelapenya Mar 27, 2024
05e52b6
fix: update comment on rootless
mdelapenya Mar 27, 2024
70bd817
chore: honour rootless socket path
mdelapenya Mar 27, 2024
51c2ceb
revert: log env for subprocess
mdelapenya Mar 27, 2024
2c91259
fix: use proper docker host in subprocess test
mdelapenya Mar 27, 2024
20caafd
chore: simplify test
mdelapenya Mar 27, 2024
853b194
chore: initialise DockerSocketPath on init
mdelapenya Mar 27, 2024
0ed8418
Revert "chore: initialise DockerSocketPath on init"
mdelapenya Mar 27, 2024
6e468bc
fix: testcontainers socket override checks
mdelapenya Mar 27, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitattributes
@@ -0,0 +1 @@
*.sh text eol=lf
14 changes: 10 additions & 4 deletions .github/workflows/ci-test-go.yml
Expand Up @@ -37,6 +37,9 @@ on:
type: boolean
default: false
description: "Disable the ryuk container for the test."
secrets:
TC_CLOUD_TOKEN:
required: true

permissions:
contents: read
Expand Down Expand Up @@ -92,15 +95,18 @@ jobs:
working-directory: ./${{ inputs.project-directory }}
run: make tools-tidy

- name: Setup Testcontainers Cloud Client
# Use Testcontainers Cloud for Windows and MacOS, but only for merge commits on main
if: ${{ inputs.run-tests && (inputs.platform == 'windows-latest' || inputs.platform == 'macos-latest') }}
uses: atomicjar/testcontainers-cloud-setup-action@v1
with:
token: ${{ secrets.TC_CLOUD_TOKEN }}

- name: ensure compilation
working-directory: ./${{ inputs.project-directory }}
run: go build

- name: go test
# only run tests on linux, there are a number of things that won't allow the tests to run on anything else
# many (maybe, all?) images used can only be build on Linux, they don't have Windows in their manifest, and
# we can't put Windows Server in "Linux Mode" in Github actions
# another, host mode is only available on Linux, and we have tests around that, do we skip them?
if: ${{ inputs.run-tests }}
working-directory: ./${{ inputs.project-directory }}
timeout-minutes: 30
Expand Down
111 changes: 3 additions & 108 deletions .github/workflows/ci.yml
Expand Up @@ -27,119 +27,14 @@ jobs:
strategy:
matrix:
go-version: [1.21.x, 1.x]
platform: [ubuntu-latest, macos-latest]
platform: [windows-latest]
uses: ./.github/workflows/ci-test-go.yml
secrets: inherit
with:
go-version: ${{ matrix.go-version }}
fail-fast: true
platform: ${{ matrix.platform }}
project-directory: "."
rootless-docker: false
run-tests: ${{ matrix.platform == 'ubuntu-latest' }}
run-tests: ${{ matrix.platform == 'windows-latest' }}
ryuk-disabled: false

# The job below is a copy of the job above, but with ryuk disabled.
# It's executed in the first stage to avoid concurrency issues.
test-reaper-off:
name: "Test with reaper off"
strategy:
matrix:
go-version: [1.21.x, 1.x]
uses: ./.github/workflows/ci-test-go.yml
with:
go-version: ${{ matrix.go-version }}
fail-fast: false
platform: "ubuntu-latest"
project-directory: "."
rootless-docker: false
run-tests: true
ryuk-disabled: true

# The job below is a copy of the job above, but with Docker rootless.
# It's executed in the first stage to avoid concurrency issues.
test-rootless-docker:
name: "Test with Rootless Docker"
strategy:
matrix:
go-version: [1.21.x, 1.x]
platform: [ubuntu-latest]
uses: ./.github/workflows/ci-test-go.yml
with:
go-version: ${{ matrix.go-version }}
fail-fast: false
platform: "ubuntu-latest"
project-directory: "."
rootless-docker: true
run-tests: true
ryuk-disabled: false

test-module-generator:
strategy:
matrix:
go-version: [1.21.x, 1.x]
platform: [ubuntu-latest, macos-latest, windows-latest]
uses: ./.github/workflows/ci-test-go.yml
with:
go-version: ${{ matrix.go-version }}
fail-fast: true
platform: ${{ matrix.platform }}
project-directory: "modulegen"
rootless-docker: false
run-tests: true
ryuk-disabled: false

test-modules:
needs: test
strategy:
matrix:
go-version: [1.21.x, 1.x]
platform: [ubuntu-latest]
module: [artemis, cassandra, chroma, clickhouse, cockroachdb, compose, consul, couchbase, elasticsearch, gcloud, inbucket, influxdb, k3s, k6, kafka, localstack, mariadb, milvus, minio, mockserver, mongodb, mssql, mysql, nats, neo4j, ollama, openfga, openldap, opensearch, postgres, pulsar, qdrant, rabbitmq, redis, redpanda, registry, surrealdb, vault, weaviate]
uses: ./.github/workflows/ci-test-go.yml
with:
go-version: ${{ matrix.go-version }}
fail-fast: false
platform: ${{ matrix.platform }}
project-directory: modules/${{ matrix.module }}
rootless-docker: false
run-tests: ${{ matrix.platform == 'ubuntu-latest' }}
ryuk-disabled: false

test-examples:
needs: test-modules
strategy:
matrix:
module: [nginx, toxiproxy]
uses: ./.github/workflows/ci-test-go.yml
with:
go-version: "1.21.x"
fail-fast: true
platform: 'ubuntu-latest'
project-directory: examples/${{ matrix.module }}
rootless-docker: false
run-tests: true
ryuk-disabled: false

sonarcloud:
permissions:
contents: read # for actions/checkout to fetch code
pull-requests: read # for sonarsource/sonarcloud-github-action to determine which PR to decorate
if: ${{ github.ref_name == 'main' && github.repository_owner == 'testcontainers' }}
needs: test-examples
runs-on: ubuntu-latest
steps:
- name: Check out code into the Go module directory
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
with:
# Disabling shallow clone is recommended for improving relevancy of reporting
fetch-depth: 0

- uses: actions/download-artifact@9bc31d5ccc31df68ecc42ccf4149144866c47d8a # v3.0.2
with:
name: sonarcloud

- name: Analyze with SonarCloud
uses: sonarsource/sonarcloud-github-action@49e6cd3b187936a73b8280d59ffd9da69df63ec9 # v2.1.1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
18 changes: 5 additions & 13 deletions config_test.go
Expand Up @@ -25,21 +25,13 @@ func TestReadConfig(t *testing.T) {
t.Run("Config is read just once", func(t *testing.T) {
t.Setenv("HOME", "")
t.Setenv("USERPROFILE", "") // Windows support
t.Setenv("TESTCONTAINERS_RYUK_DISABLED", "true")

cfg := testcontainers.ReadConfig()

expected := testcontainers.TestcontainersConfig{
RyukDisabled: true,
Config: config.Config{
RyukDisabled: true,
},
}

assert.Equal(t, expected, cfg)
t.Setenv("TESTCONTAINERS_RYUK_DISABLED", "true")
cfg1 := testcontainers.ReadConfig()

t.Setenv("TESTCONTAINERS_RYUK_DISABLED", "false")
cfg = testcontainers.ReadConfig()
assert.Equal(t, expected, cfg)
cfg2 := testcontainers.ReadConfig()

assert.Equal(t, cfg1, cfg2)
})
}
8 changes: 8 additions & 0 deletions docker_client_test.go
Expand Up @@ -13,6 +13,10 @@ func TestGetDockerInfo(t *testing.T) {
ctx := context.Background()
c, err := NewDockerClientWithOpts(ctx)
require.NoError(t, err)
t.Cleanup(func() {
err := c.Close()
require.NoError(t, err)
})

info, err := c.Info(ctx)
require.NoError(t, err)
Expand All @@ -23,6 +27,10 @@ func TestGetDockerInfo(t *testing.T) {
ctx := context.Background()
c, err := NewDockerClientWithOpts(ctx)
require.NoError(t, err)
t.Cleanup(func() {
err := c.Close()
require.NoError(t, err)
})

count := 1024
wg := sync.WaitGroup{}
Expand Down
65 changes: 43 additions & 22 deletions docker_files_test.go
Expand Up @@ -2,6 +2,7 @@ package testcontainers_test

import (
"context"
"io"
"os"
"path/filepath"
"testing"
Expand Down Expand Up @@ -30,7 +31,7 @@ func TestCopyFileToContainer(t *testing.T) {

container, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
ContainerRequest: testcontainers.ContainerRequest{
Image: "docker.io/bash",
Image: "docker.io/bash:5.2.26",
Files: []testcontainers.ContainerFile{
{
Reader: r,
Expand All @@ -51,8 +52,7 @@ func TestCopyFileToContainer(t *testing.T) {
}

func TestCopyFileToRunningContainer(t *testing.T) {
ctx, cnl := context.WithTimeout(context.Background(), 30*time.Second)
defer cnl()
ctx := context.Background()

// Not using the assertations here to avoid leaking the library into the example
// copyFileAfterCreate {
Expand Down Expand Up @@ -83,6 +83,13 @@ func TestCopyFileToRunningContainer(t *testing.T) {
t.Fatal(err)
}

r, err := container.Logs(ctx)
require.NoError(t, err)

bs, err := io.ReadAll(r)
require.NoError(t, err)
t.Log(">> container logs:", string(bs))

err = container.CopyFileToContainer(ctx, helloPath, "/scripts/hello.sh", 0o700)
// }

Expand All @@ -108,7 +115,7 @@ func TestCopyDirectoryToContainer(t *testing.T) {

container, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
ContainerRequest: testcontainers.ContainerRequest{
Image: "docker.io/bash",
Image: "docker.io/bash:5.2.26",
Files: []testcontainers.ContainerFile{
{
HostFilePath: dataDirectory,
Expand All @@ -131,8 +138,7 @@ func TestCopyDirectoryToContainer(t *testing.T) {
}

func TestCopyDirectoryToRunningContainerAsFile(t *testing.T) {
ctx, cnl := context.WithTimeout(context.Background(), 30*time.Second)
defer cnl()
ctx := context.Background()

// copyDirectoryToRunningContainerAsFile {
dataDirectory, err := filepath.Abs(filepath.Join(".", "testdata"))
Expand All @@ -146,7 +152,7 @@ func TestCopyDirectoryToRunningContainerAsFile(t *testing.T) {

container, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
ContainerRequest: testcontainers.ContainerRequest{
Image: "docker.io/bash",
Image: "docker.io/bash:5.2.26",
Files: []testcontainers.ContainerFile{
{
HostFilePath: waitForPath,
Expand All @@ -155,19 +161,27 @@ func TestCopyDirectoryToRunningContainerAsFile(t *testing.T) {
},
},
Cmd: []string{"bash", "/waitForHello.sh"},
LifecycleHooks: []testcontainers.ContainerLifecycleHooks{
{
PostReadies: []testcontainers.ContainerHook{
func(ctx context.Context, container testcontainers.Container) error {
// as the container is started, we can create the directory right after the container is ready
_, _, err = container.Exec(ctx, []string{"mkdir", "-p", "/scripts"})
if err != nil {
return err
}
return nil
},
},
},
},
},
Started: true,
})
if err != nil {
t.Fatal(err)
}

// as the container is started, we can create the directory first
_, _, err = container.Exec(ctx, []string{"mkdir", "-p", "/scripts"})
if err != nil {
t.Fatal(err)
}

// because the container path is a directory, it will use the copy dir method as fallback
err = container.CopyFileToContainer(ctx, dataDirectory, "/scripts", 0o700)
if err != nil {
Expand All @@ -180,8 +194,7 @@ func TestCopyDirectoryToRunningContainerAsFile(t *testing.T) {
}

func TestCopyDirectoryToRunningContainerAsDir(t *testing.T) {
ctx, cnl := context.WithTimeout(context.Background(), 30*time.Second)
defer cnl()
ctx := context.Background()

// Not using the assertations here to avoid leaking the library into the example
// copyDirectoryToRunningContainerAsDir {
Expand All @@ -196,7 +209,7 @@ func TestCopyDirectoryToRunningContainerAsDir(t *testing.T) {

container, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
ContainerRequest: testcontainers.ContainerRequest{
Image: "docker.io/bash",
Image: "docker.io/bash:5.2.26",
Files: []testcontainers.ContainerFile{
{
HostFilePath: waitForPath,
Expand All @@ -205,19 +218,27 @@ func TestCopyDirectoryToRunningContainerAsDir(t *testing.T) {
},
},
Cmd: []string{"bash", "/waitForHello.sh"},
LifecycleHooks: []testcontainers.ContainerLifecycleHooks{
{
PostReadies: []testcontainers.ContainerHook{
func(ctx context.Context, container testcontainers.Container) error {
// as the container is started, we can create the directory right after the container is ready
_, _, err = container.Exec(ctx, []string{"mkdir", "-p", "/scripts"})
if err != nil {
return err
}
return nil
},
},
},
},
},
Started: true,
})
if err != nil {
t.Fatal(err)
}

// as the container is started, we can create the directory first
_, _, err = container.Exec(ctx, []string{"mkdir", "-p", "/scripts"})
if err != nil {
t.Fatal(err)
}

err = container.CopyDirToContainer(ctx, dataDirectory, "/scripts", 0o700)
if err != nil {
t.Fatal(err)
Expand Down