Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: fixes path issues on windows #829

Merged
merged 49 commits into from Feb 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
b67aef4
return corrent path in windows with usiing path lib
Monirzadeh Jan 26, 2024
ac8433c
ci: run tests in windows and macos too
fmartingr Jan 26, 2024
2a94c26
avoid testing mysql/psql in windows and macos
fmartingr Jan 26, 2024
95e0319
windows & macos matrix
fmartingr Jan 26, 2024
1737a89
disable gotestfmt
fmartingr Jan 26, 2024
0f65bfc
replace which in shell script
fmartingr Jan 26, 2024
f12ff1e
handle error in test db removal
fmartingr Jan 26, 2024
1e3a5ac
fix expected path baseed on platform
Monirzadeh Jan 28, 2024
aa10509
add leading seprator
Monirzadeh Jan 28, 2024
89d9599
Merge branch 'fix-file-not-serve-in-windows' into fix-test-in-windows
Monirzadeh Jan 28, 2024
83f0c94
proper temporary storage dir and db cleanup
fmartingr Jan 29, 2024
35fcad8
fix failed to create destination dir file does not exist in windows
Monirzadeh Jan 30, 2024
f71d4d2
move temp to /tmp
Monirzadeh Jan 30, 2024
22538da
update temp folder
Monirzadeh Jan 30, 2024
79ff230
fix config tests in windows
Monirzadeh Jan 31, 2024
dcf6ea4
apply patch for db
Monirzadeh Feb 2, 2024
8fc314e
revert temp dir creation.
Monirzadeh Feb 2, 2024
ac5df12
unify account db tests pach
Monirzadeh Feb 3, 2024
58c6aa3
remove TmpDir for sqlite tests
Monirzadeh Feb 3, 2024
e514d0e
try to force CGO disable with enviroment variable
Monirzadeh Feb 3, 2024
6c5788e
Remove unneeded log
Monirzadeh Feb 4, 2024
2832fd1
remove unneeded comment
Monirzadeh Feb 4, 2024
266685c
fix file path for download images
Monirzadeh Feb 4, 2024
cd200f6
change way to create temp directory
Monirzadeh Feb 4, 2024
134a615
use diffrent file name for each test
Monirzadeh Feb 4, 2024
b403da4
fix typo
Monirzadeh Feb 4, 2024
dd1a3be
fix absolute path in successful download image
Monirzadeh Feb 4, 2024
034e190
correct filename with png
Monirzadeh Feb 4, 2024
51c5168
change test to download image from internet instead of local machine
Monirzadeh Feb 4, 2024
0068406
remvoe unneeded import
Monirzadeh Feb 4, 2024
ebe397f
remove os.RemoveAll(.env)
Monirzadeh Feb 4, 2024
43c3245
unify variable names in unit test
Monirzadeh Feb 4, 2024
71ebb9a
return CGO_ENABLED=0
Monirzadeh Feb 4, 2024
60ccc16
test other way to set enviroment variable
Monirzadeh Feb 4, 2024
4cd7893
try to set enviroment variable sepratly in macos and windows
Monirzadeh Feb 4, 2024
91c909c
set enviroment variable before run commands in windows
Monirzadeh Feb 4, 2024
fe24a9c
fix windows test name
Monirzadeh Feb 4, 2024
466d75e
Merge branch 'master' into fix-test-in-windows
Monirzadeh Feb 4, 2024
75eb57e
combine two workflow for windows and macos again
Monirzadeh Feb 4, 2024
d9c5107
Merge branch 'fix-test-in-windows' of https://github.com/Monirzadeh/s…
Monirzadeh Feb 4, 2024
9ec2823
fix typo
Monirzadeh Feb 4, 2024
7c9aa5f
remove env
Monirzadeh Feb 4, 2024
54e32f2
change env path
Monirzadeh Feb 4, 2024
4d493b8
cleanup unneeded env
Monirzadeh Feb 4, 2024
a4f5f8c
general CGO_ENABLED environ
fmartingr Feb 4, 2024
dece920
use absolute path to run fileserver instead of relative
Monirzadeh Feb 4, 2024
c46cd1e
serve file test from internet shiori repository
Monirzadeh Feb 4, 2024
ec23aaf
check file existance after download and unify varibale name from temp…
Monirzadeh Feb 4, 2024
ffe6f26
remove unneeded log
Monirzadeh Feb 5, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
40 changes: 37 additions & 3 deletions .github/workflows/_test.yml
Expand Up @@ -2,8 +2,11 @@ name: "Unit Tests"

on: workflow_call

env:
CGO_ENABLED: 0

jobs:
test:
test-linux:
runs-on: ubuntu-latest
services:
postgres:
Expand All @@ -27,7 +30,7 @@ jobs:
ports:
- 3306:3306

name: Go unit tests
name: Go unit tests (ubuntu-latest)
steps:
- uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1

Expand All @@ -53,10 +56,41 @@ jobs:
env:
SHIORI_TEST_PG_URL: "postgres://shiori:shiori@localhost:5432/shiori?sslmode=disable"
SHIORI_TEST_MYSQL_URL: "shiori:shiori@(localhost:3306)/shiori"
CGO_ENABLED: 1 # go test -race requires cgo

- run: CGO_ENABLED=0 go build -tags osusergo,netgo -ldflags="-s -w -X main.version=$(git describe --tags) -X main.date=$(date --iso-8601=seconds)"
- run: go build -tags osusergo,netgo -ldflags="-s -w -X main.version=$(git describe --tags) -X main.date=$(date --iso-8601=seconds)"

- name: Upload coverage reports to Codecov
uses: codecov/codecov-action@4fe8c5f003fae66aa5ebb77cfd3e7bfbbda0b6b0 # 3.1.5
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

test-windows-macos:
strategy:
matrix:
os: [windows-latest, macos-latest]
runs-on: ${{ matrix.os }}
name: Go unit tests (${{ matrix.os }})
steps:
- uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1

- name: Setup go
uses: actions/setup-go@0c52d547c9bc32b1aa3301fd7a9cb496313a4491 # v5.0.0
with:
go-version-file: ./go.mod

- uses: actions/cache@e12d46a63a90f2fae62d114769bbf2a179198b5c # 3.3.3
with:
path: |
~/.cache/go-build
~/go/pkg
key: golangci-lint.cache-{platform-arch}-{interval_number}-{go.mod_hash}
restore-keys: |
golangci-lint.cache-{interval_number}-
golangci-lint.cache-

- run: make unittest GO_TEST_FLAGS="-tags test_sqlite_only"
Monirzadeh marked this conversation as resolved.
Show resolved Hide resolved
env:
CGO_ENABLED: 1 # go test -race requires cgo

- run: go build -tags osusergo,netgo -ldflags="-s -w -X main.version=$(git describe --tags) -X main.date=$(date --iso-8601=seconds)"
2 changes: 1 addition & 1 deletion Makefile
Expand Up @@ -5,7 +5,7 @@ BASH ?= $(shell command -v bash 2> /dev/null)
SHIORI_DIR ?= dev-data

# Testing
GO_TEST_FLAGS ?= -v -race -count=1 -covermode=atomic -coverprofile=coverage.out
override GO_TEST_FLAGS += -v -race -count=1 -covermode=atomic -coverprofile=coverage.out
GOTESTFMT_FLAGS ?=

# Build
Expand Down
14 changes: 2 additions & 12 deletions internal/config/config_test.go
Expand Up @@ -67,12 +67,7 @@ func TestReadDotEnv(t *testing.T) {
t.Run(testCase.name, func(t *testing.T) {
tmpDir, err := os.MkdirTemp("", "")
require.NoError(t, err)

os.Chdir(tmpDir)

t.Cleanup(func() {
require.NoError(t, os.RemoveAll(tmpDir))
})
require.NoError(t, os.Chdir(tmpDir))

// Write the .env file in the temporary directory
handler, err := os.OpenFile(".env", os.O_CREATE|os.O_WRONLY, 0655)
Expand All @@ -89,12 +84,7 @@ func TestReadDotEnv(t *testing.T) {
t.Run("no file", func(t *testing.T) {
tmpDir, err := os.MkdirTemp("", "")
require.NoError(t, err)

os.Chdir(tmpDir)

t.Cleanup(func() {
require.NoError(t, os.RemoveAll(tmpDir))
})
require.NoError(t, os.Chdir(tmpDir))

e := readDotEnv(log)

Expand Down
73 changes: 39 additions & 34 deletions internal/core/ebook_test.go
Expand Up @@ -21,11 +21,10 @@ func TestGenerateEbook(t *testing.T) {

t.Run("Successful ebook generate", func(t *testing.T) {
t.Run("valid bookmarkId that return HasEbook true", func(t *testing.T) {
// test cae
tempDir := t.TempDir()
dstDir := t.TempDir()
dstFile := "/ebook/1.epub"
tmpDir := t.TempDir()

deps.Domains.Storage = domains.NewStorageDomain(deps, afero.NewBasePathFs(afero.NewOsFs(), dstDir))
deps.Domains.Storage = domains.NewStorageDomain(deps, afero.NewBasePathFs(afero.NewOsFs(), tmpDir))

mockRequest := core.ProcessRequest{
Bookmark: model.BookmarkDTO{
Expand All @@ -34,138 +33,144 @@ func TestGenerateEbook(t *testing.T) {
HTML: "<html><body>Example HTML</body></html>",
HasEbook: false,
},
DataDir: dstDir,
DataDir: tmpDir,
ContentType: "text/html",
}

bookmark, err := core.GenerateEbook(deps, mockRequest, fp.Join(tempDir, "1"))
bookmark, err := core.GenerateEbook(deps, mockRequest, dstFile)

assert.True(t, bookmark.HasEbook)
assert.NoError(t, err)
})
t.Run("ebook generate with valid BookmarkID EbookExist ImagePathExist ReturnWithHasEbookTrue", func(t *testing.T) {
dstDir := t.TempDir()
dstFile := "/ebook/2.epub"
tmpDir := t.TempDir()

deps.Domains.Storage = domains.NewStorageDomain(deps, afero.NewBasePathFs(afero.NewOsFs(), dstDir))
deps.Domains.Storage = domains.NewStorageDomain(deps, afero.NewBasePathFs(afero.NewOsFs(), tmpDir))

bookmark := model.BookmarkDTO{
ID: 2,
HasEbook: false,
}
mockRequest := core.ProcessRequest{
Bookmark: bookmark,
DataDir: dstDir,
DataDir: tmpDir,
ContentType: "text/html",
}
// Create the thumbnail file
imagePath := model.GetThumbnailPath(&bookmark)
deps.Domains.Storage.FS().MkdirAll(fp.Dir(imagePath), os.ModePerm)
imagedirPath := fp.Dir(imagePath)
deps.Domains.Storage.FS().MkdirAll(imagedirPath, os.ModePerm)
file, err := deps.Domains.Storage.FS().Create(imagePath)
if err != nil {
t.Fatal(err)
}
defer file.Close()

bookmark, err = core.GenerateEbook(deps, mockRequest, model.GetEbookPath(&bookmark))
expectedImagePath := "/bookmark/2/thumb"
bookmark, err = core.GenerateEbook(deps, mockRequest, dstFile)
expectedImagePath := string(fp.Separator) + fp.Join("bookmark", "2", "thumb")
assert.NoError(t, err)
assert.True(t, bookmark.HasEbook)
assert.Equalf(t, expectedImagePath, bookmark.ImageURL, "Expected imageURL %s, but got %s", expectedImagePath, bookmark.ImageURL)
})
t.Run("generate ebook valid BookmarkID EbookExist ReturnHasArchiveTrue", func(t *testing.T) {
tempDir := t.TempDir()
dstDir := t.TempDir()
dstFile := "/ebook/3.epub"
tmpDir := t.TempDir()

deps.Domains.Storage = domains.NewStorageDomain(deps, afero.NewBasePathFs(afero.NewOsFs(), dstDir))
deps.Domains.Storage = domains.NewStorageDomain(deps, afero.NewBasePathFs(afero.NewOsFs(), tmpDir))

bookmark := model.BookmarkDTO{
ID: 3,
HasEbook: false,
}
mockRequest := core.ProcessRequest{
Bookmark: bookmark,
DataDir: dstDir,
DataDir: tmpDir,
ContentType: "text/html",
}
// Create the archive file
archivePath := model.GetArchivePath(&bookmark)
deps.Domains.Storage.FS().MkdirAll(fp.Dir(archivePath), os.ModePerm)
archiveDirPath := fp.Dir(archivePath)
deps.Domains.Storage.FS().MkdirAll(archiveDirPath, os.ModePerm)
file, err := deps.Domains.Storage.FS().Create(archivePath)
if err != nil {
t.Fatal(err)
}
defer file.Close()

bookmark, err = core.GenerateEbook(deps, mockRequest, fp.Join(tempDir, "1"))
bookmark, err = core.GenerateEbook(deps, mockRequest, fp.Join(dstFile, "1"))
assert.True(t, bookmark.HasArchive)
assert.NoError(t, err)
})
})
t.Run("specific ebook generate case", func(t *testing.T) {
t.Run("invalid bookmarkId that return Error", func(t *testing.T) {
tempDir := t.TempDir()
dstFile := "/ebook/0.epub"
tmpDir := t.TempDir()
mockRequest := core.ProcessRequest{
Bookmark: model.BookmarkDTO{
ID: 0,
HasEbook: false,
},
DataDir: tempDir,
DataDir: tmpDir,
ContentType: "text/html",
}

bookmark, err := core.GenerateEbook(deps, mockRequest, tempDir)
bookmark, err := core.GenerateEbook(deps, mockRequest, dstFile)

assert.Equal(t, model.BookmarkDTO{
ID: 0,
HasEbook: false,
}, bookmark)
assert.Error(t, err)
assert.EqualError(t, err, "bookmark ID is not valid")
})
t.Run("ebook exist return HasEbook true", func(t *testing.T) {
dstDir := t.TempDir()
dstFile := "/ebook/1.epub"
tmpDir := t.TempDir()

deps.Domains.Storage = domains.NewStorageDomain(deps, afero.NewBasePathFs(afero.NewOsFs(), dstDir))
deps.Domains.Storage = domains.NewStorageDomain(deps, afero.NewBasePathFs(afero.NewOsFs(), tmpDir))

bookmark := model.BookmarkDTO{
ID: 1,
HasEbook: false,
}
mockRequest := core.ProcessRequest{
Bookmark: bookmark,
DataDir: dstDir,
DataDir: tmpDir,
ContentType: "text/html",
}
// Create the ebook file
ebookFilePath := model.GetEbookPath(&bookmark)
deps.Domains.Storage.FS().MkdirAll(fp.Dir(ebookFilePath), os.ModePerm)
file, err := deps.Domains.Storage.FS().Create(ebookFilePath)
ebookPath := model.GetEbookPath(&bookmark)
ebookDirPath := fp.Dir(ebookPath)
deps.Domains.Storage.FS().MkdirAll(ebookDirPath, os.ModePerm)
file, err := deps.Domains.Storage.FS().Create(ebookPath)
if err != nil {
t.Fatal(err)
}
defer file.Close()

bookmark, err = core.GenerateEbook(deps, mockRequest, ebookFilePath)
bookmark, err = core.GenerateEbook(deps, mockRequest, dstFile)

assert.True(t, bookmark.HasEbook)
assert.NoError(t, err)
})
t.Run("generate ebook valid BookmarkID RetuenError for PDF file", func(t *testing.T) {
tempDir := t.TempDir()
dstFile := "/ebook/1.epub"
tmpDir := t.TempDir()

mockRequest := core.ProcessRequest{
Bookmark: model.BookmarkDTO{
ID: 1,
HasEbook: false,
},
DataDir: tempDir,
DataDir: tmpDir,
ContentType: "application/pdf",
}

bookmark, err := core.GenerateEbook(deps, mockRequest, tempDir)
bookmark, err := core.GenerateEbook(deps, mockRequest, dstFile)

assert.False(t, bookmark.HasEbook)
assert.Error(t, err)
assert.Contains(t, err.Error(), "can't create ebook for pdf")
assert.EqualError(t, err, "can't create ebook for pdf")
})
})
}