Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(git): Use fs instead of git for file listing #7277

Closed
wants to merge 6 commits into from
Closed

feat(git): Use fs instead of git for file listing #7277

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
ad5cce9
feat(git): Use "readdir" instead of "git-ls-tree" for file listing
Sep 14, 2020
5ff46e7
Fix types
Sep 14, 2020
838733e
Add recursion
Sep 15, 2020
25ad15c
Apply suggestions from code review
rarkins Sep 17, 2020
b84baf0
refactor
rarkins Sep 17, 2020
1e76e7f
Merge branch 'master' into feat/git-fs-file-list
rarkins Sep 21, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions lib/util/git/__snapshots__/index.spec.ts.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ Array [
exports[`platform/git getFileList() should exclude submodules 1`] = `
Array [
".gitmodules",
"deeply/nested/file",
"file_to_delete",
"master_file",
"past_file",
Expand Down
3 changes: 3 additions & 0 deletions lib/util/git/index.spec.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,9 @@ describe('platform/git', () => {
});
it('should exclude submodules', async () => {
const repo = Git(base.path).silent(true);
await fs.mkdirs(base.path + '/deeply/nested');
await fs.writeFile(base.path + '/deeply/nested/file', 'foobar');
await repo.add('deeply/nested/file');
await repo.submoduleAdd(base.path, 'submodule');
await repo.commit('Add submodule');
await git.initRepo({
Expand Down
40 changes: 26 additions & 14 deletions lib/util/git/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import { Dirent } from 'fs';
import { join } from 'path';
import URL from 'url';
import fs from 'fs-extra';
Expand Down Expand Up @@ -361,23 +362,34 @@ export async function checkoutBranch(branchName: string): Promise<CommitSha> {
}
}

async function fileList(
directory: string,
excludes: string[] = []
): Promise<string[]> {
let list: string[] = [];

const files = (await fs.readdir(directory)).filter((file) =>
excludes.every(
(exclude) => file !== exclude && !file.startsWith(`${exclude}/`)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a doubt in my code now. This may not filter correctly is the submodule is nested deeper than the top level directory

)
);
for (const file of files) {
const p = join(directory, file);
if ((await fs.stat(p)).isDirectory()) {
list = [...list, ...(await fileList(p))];
} else {
list.push(p);
}
}

return list;
}

export async function getFileList(): Promise<string[]> {
await syncGit();
const branch = config.currentBranch;
const submodules = await getSubmodules();
const files: string = await git.raw(['ls-tree', '-r', branch]);
// istanbul ignore if
if (!files) {
return [];
}
return files
.split('\n')
.filter(Boolean)
.filter((line) => line.startsWith('100'))
.map((line) => line.split(/\t/).pop())
.filter((file: string) =>
submodules.every((submodule: string) => !file.startsWith(submodule))
);
const files = await fileList(config.localDir, ['.git', ...submodules]);
return files.map((file) => file.replace(`${config.localDir}/`, ''));
}

export function getBranchList(): string[] {
Expand Down