Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue 1498 #1118

Merged
merged 105 commits into from
Mar 29, 2021
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
105 commits
Select commit Hold shift + click to select a range
f7e3efd
Add getLastObservedReleaseId
PiotrKozlowski Mar 5, 2021
ec416a2
Pull out writing file logic from createRedirects.ts
PiotrKozlowski Mar 5, 2021
92a3fb3
Redirect on locationChange if url was found in the redirects.json file
PiotrKozlowski Mar 5, 2021
8f78cd4
Add support for developments version fo redirects json
PiotrKozlowski Mar 5, 2021
d1daf32
Create prepareRedirects function from createRedirects
PiotrKozlowski Mar 5, 2021
26483c0
It is not working yet, but I am commiting so I can go back to this
PiotrKozlowski Mar 5, 2021
2edd83b
revert changes
PiotrKozlowski Mar 9, 2021
dcdd608
new way of handling redirects
PiotrKozlowski Mar 9, 2021
086e9af
Add tests
PiotrKozlowski Mar 9, 2021
4f93e12
Merge branch 'master' into issue-1498
staxly[bot] Mar 9, 2021
f23a6bd
Merge branch 'master' into issue-1498
staxly[bot] Mar 10, 2021
8164541
Merge branch 'master' into issue-1498
staxly[bot] Mar 11, 2021
f995b1c
Merge branch 'master' into issue-1498
staxly[bot] Mar 11, 2021
d474031
Merge branch 'master' into issue-1498
staxly[bot] Mar 11, 2021
7c9343b
Pull out writing file logic from createRedirects.ts
PiotrKozlowski Mar 5, 2021
7b285a7
Create prepareRedirects function from createRedirects
PiotrKozlowski Mar 5, 2021
dd6c9f3
Merge branch 'master' into issue-1498
staxly[bot] Mar 12, 2021
ac5691d
Create redirects on the fly
PiotrKozlowski Mar 12, 2021
0773fc8
move prepareRedirects to scripts folder
PiotrKozlowski Mar 12, 2021
58433ee
Merge branch 'issue-1498' of github.com:openstax/rex-web into issue-1498
PiotrKozlowski Mar 12, 2021
18fcdf0
dont forget about required presets and plugins
PiotrKozlowski Mar 12, 2021
3067341
Try increasing file watchers
PiotrKozlowski Mar 12, 2021
2573771
share babel config
PiotrKozlowski Mar 12, 2021
c929e90
Revert "Try increasing file watchers"
PiotrKozlowski Mar 12, 2021
ea1d61a
Merge branch 'master' into issue-1498
staxly[bot] Mar 12, 2021
fdf33ca
Merge branch 'master' into issue-1498
staxly[bot] Mar 12, 2021
2bf14b8
Merge branch 'master' into issue-1498
staxly[bot] Mar 12, 2021
896e141
Move prepareRedirects outside of the middleware
PiotrKozlowski Mar 15, 2021
662a2d2
mock implementation for history replace
PiotrKozlowski Mar 15, 2021
a4b070d
increase number of watches
PiotrKozlowski Mar 15, 2021
c33c9f7
try without sudo
PiotrKozlowski Mar 15, 2021
cfdd282
remove command for increasing file watches
PiotrKozlowski Mar 15, 2021
6207ee0
Check if this helps
PiotrKozlowski Mar 15, 2021
90d9a24
Check if requiring babel config twice breaks something
PiotrKozlowski Mar 15, 2021
dda6d81
Does requiring more stuff cause error
PiotrKozlowski Mar 15, 2021
6987549
What if i import only prepareRedirects?
PiotrKozlowski Mar 15, 2021
08812ab
Uncomment everything in setupProxy - error is caused by prepareRedire…
PiotrKozlowski Mar 15, 2021
242ed7e
Instead of reading all files in data/redirects/, read list off books …
PiotrKozlowski Mar 15, 2021
04b6102
Revert "Instead of reading all files in data/redirects/, read list of…
PiotrKozlowski Mar 15, 2021
d638bdd
mock prepareRedirects to check if it is really realted to this function
PiotrKozlowski Mar 15, 2021
145e960
Revert "mock prepareRedirects to check if it is really realted to thi…
PiotrKozlowski Mar 15, 2021
431821c
What if we require only createArchiveLoader
PiotrKozlowski Mar 15, 2021
c7641a1
And what about importing only createOSWebLoader
PiotrKozlowski Mar 15, 2021
6f9f55c
Uncomment import of prepareRedirects
PiotrKozlowski Mar 15, 2021
c2d4905
change prepareRedirects to always return []
PiotrKozlowski Mar 15, 2021
67a5a2c
Revert "change prepareRedirects to always return []"
PiotrKozlowski Mar 15, 2021
253212a
Instead of reading all files in data/redirects/, read list off books …
PiotrKozlowski Mar 15, 2021
c04d565
Revert "Instead of reading all files in data/redirects/, read list of…
PiotrKozlowski Mar 15, 2021
aaec10a
Check if this is about dynamic imports
PiotrKozlowski Mar 15, 2021
569cbf9
Check if importing content route is breaking something
PiotrKozlowski Mar 15, 2021
97c258a
Run only Test CI job
PiotrKozlowski Mar 15, 2021
f95969a
remove whole for loop from preprareRedirects
PiotrKozlowski Mar 15, 2021
8dc8e96
setup job is still required
PiotrKozlowski Mar 15, 2021
7f7939b
Check if removing makeUnifiedBookLoader from prepareRedirects will help
PiotrKozlowski Mar 15, 2021
bf6fcc0
Reset to commit 662a2d2a4bc20f6517dd807639423c0259dcfc1f
PiotrKozlowski Mar 15, 2021
9c6aaa2
narrow ci to run only test and setup jobs
PiotrKozlowski Mar 16, 2021
76fb01f
Remove stripIdVersion from createArchiveLoader
PiotrKozlowski Mar 16, 2021
157ea27
Import only createArchiveLoader
PiotrKozlowski Mar 16, 2021
94aa04c
Import only createArchiveLoader
PiotrKozlowski Mar 16, 2021
0eacff5
Import also prepareRedirects
PiotrKozlowski Mar 16, 2021
1a8167d
remove makeUnifiedBookLoader from prepareRedirects
PiotrKozlowski Mar 16, 2021
6dbd19b
remove findArchiveTreeNodeById from prepareRedirects
PiotrKozlowski Mar 16, 2021
a9f32f5
use stripIdVersion in createArchiveLoader again
PiotrKozlowski Mar 16, 2021
2bcdc41
Import stripIdVersion from idUtils
PiotrKozlowski Mar 16, 2021
31a5265
i didnt want to import findArchiveTreeNodeById in prepareRedirects
PiotrKozlowski Mar 16, 2021
1eadfc8
import only prepareRedirects
PiotrKozlowski Mar 16, 2021
5a6c0b2
Display max_user_watches limit on gh
PiotrKozlowski Mar 16, 2021
8b9d863
Add modulePathIgnorePatterns to jest config
PiotrKozlowski Mar 16, 2021
6b6f95f
Revert "Add modulePathIgnorePatterns to jest config"
PiotrKozlowski Mar 16, 2021
cdec20c
Try adding watchmanconfig - jest uses it by default
PiotrKozlowski Mar 16, 2021
a8e044a
Try adding hint_num_dirs to watchmanconfig
PiotrKozlowski Mar 16, 2021
0928d1e
Reset to 662a2d2a4bc20f6517dd807639423c0259dcfc1f
PiotrKozlowski Mar 16, 2021
ae3e384
remove watchmanconfig
PiotrKozlowski Mar 17, 2021
71b2964
Dont remove util module even if it is not used
PiotrKozlowski Mar 17, 2021
1837aa3
Merge branch 'master' into issue-1498
staxly[bot] Mar 19, 2021
40b16e3
Add sysctls settings to test job container
PiotrKozlowski Mar 22, 2021
a227145
rearange sysctls option
PiotrKozlowski Mar 22, 2021
093afae
nest sysctls in options
PiotrKozlowski Mar 22, 2021
1918fd1
try other syntax
PiotrKozlowski Mar 22, 2021
ad02fa6
one more try
PiotrKozlowski Mar 22, 2021
9bc0cf3
try adding privileged to the options
PiotrKozlowski Mar 22, 2021
2467324
add alowed-unsafe-sysctls
PiotrKozlowski Mar 22, 2021
fb8b57f
revert ci.yml
PiotrKozlowski Mar 22, 2021
fdc7bd8
Add --watchAll=false flag to test:unit
PiotrKozlowski Mar 23, 2021
79e527f
Revert "Add --watchAll=false flag to test:unit"
PiotrKozlowski Mar 23, 2021
aea0b2c
remove -i flag from test:browser
PiotrKozlowski Mar 23, 2021
d744b00
Revert "remove -i flag from test:browser"
PiotrKozlowski Mar 23, 2021
411fadd
add logs to check what is triggering inotify
PiotrKozlowski Mar 23, 2021
8d6b5ea
Revert "add logs to check what is triggering inotify"
PiotrKozlowski Mar 23, 2021
5abc7e9
jest watchman: false
PiotrKozlowski Mar 23, 2021
f143acf
Revert "jest watchman: false"
PiotrKozlowski Mar 23, 2021
083ea95
Add SKIP_PREFLIGHT_CHECK=true SERVER_MODE=built to test:unit and set …
PiotrKozlowski Mar 24, 2021
dfffea3
Leave only SERVER_MODE=built and add it to test:browser
PiotrKozlowski Mar 24, 2021
6eb465c
just use yarn server instead of yarn start in jest-puppeteer.config.js
PiotrKozlowski Mar 24, 2021
7d8fa6f
Add and configure cracto to fix inotify limit issue
PiotrKozlowski Mar 25, 2021
a4d32b0
Use another config for test:unit
PiotrKozlowski Mar 25, 2021
a28c1b3
Run test:browser after test:unit/build is finished
PiotrKozlowski Mar 25, 2021
9682270
Remove default jest preset and use it in commands
PiotrKozlowski Mar 25, 2021
639489f
move browser job to matrix again
PiotrKozlowski Mar 25, 2021
8a1058d
make sure that craco options that wasnt working locally does not work…
PiotrKozlowski Mar 25, 2021
9e6e92a
Ignore all files
PiotrKozlowski Mar 25, 2021
de18be1
Ignore only node_modules files
PiotrKozlowski Mar 25, 2021
9b6fe4a
Remove craco and do not test:browser in ci
PiotrKozlowski Mar 25, 2021
15c2fee
test:screenshots should use jest-puppeteer preset
PiotrKozlowski Mar 25, 2021
32dd6a2
Merge branch 'master' into issue-1498
staxly[bot] Mar 29, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
3 changes: 2 additions & 1 deletion data/redirects/types.ts
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
export type Redirects = Array<{ bookId: string, pageId: string, pathname: string }>;
export type RedirectsData = Array<{ bookId: string, pageId: string, pathname: string }>;
export type Redirects = Array<{ from: string, to: string }>;
4 changes: 2 additions & 2 deletions script/prerender/createRedirects.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import fs from 'fs';
import path from 'path';
import { Redirects } from '../../data/redirects/types';
import { RedirectsData } from '../../data/redirects/types';
import { content } from '../../src/app/content/routes';
import { makeUnifiedBookLoader } from '../../src/app/content/utils';
import { findArchiveTreeNodeById } from '../../src/app/content/utils/archiveTreeUtils';
Expand All @@ -18,7 +18,7 @@ const createRedirects = async(archiveLoader: AppServices['archiveLoader'], osWeb
const redirects: Array<{ from: string, to: string }> = [];

for (const fileName of books) {
const bookRedirects: Redirects = await import(`${redirectsPath}/${fileName}`);
const bookRedirects: RedirectsData = await import(`${redirectsPath}/${fileName}`);

for (const { bookId, pageId, pathname } of bookRedirects) {
const configForBook: { defaultVersion: string } | undefined = config[bookId];
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ import fs from 'fs';
import isEqual from 'lodash/fp/isEqual';
import path from 'path';
import { argv } from 'yargs';
import { Redirects } from '../data/redirects/types';
import { RedirectsData } from '../data/redirects/types';
import { content } from '../src/app/content/routes';
import { LinkedArchiveTreeNode } from '../src/app/content/types';
import { flattenArchiveTree } from '../src/app/content/utils';
Expand Down Expand Up @@ -46,7 +46,7 @@ async function updateRedirections(bookId: string, currentVersion: string, newVer
});

const redirectsBookPath = path.resolve(redirectsPath, bookId + '.json');
const redirects: Redirects = fs.existsSync(redirectsBookPath) ? await import(redirectsBookPath) : [];
const redirects: RedirectsData = fs.existsSync(redirectsBookPath) ? await import(redirectsBookPath) : [];

const flatCurrentTree = flattenArchiveTree(currentTree);

Expand Down
2 changes: 2 additions & 0 deletions src/app/content/hooks/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,14 @@ import searchHooks from '../search/hooks';
import studyGuidesHooks from '../studyGuides/hooks';
import locationChangeBody from './locationChange';
import receiveContentBody from './receiveContent';
import receivePageNotFoundId from './receivePageNotFoundId';

export default [
...searchHooks,
...highlightHooks,
...studyGuidesHooks,
...practiceQuestionsHooks,
receivePageNotFoundId,
routeHook(routes.content, locationChangeBody),
actionHook(actions.receivePage, receiveContentBody),
];
67 changes: 67 additions & 0 deletions src/app/content/hooks/receivePageNotFoundId.spec.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
import createTestServices from '../../../test/createTestServices';
import createTestStore from '../../../test/createTestStore';
import { MiddlewareAPI, Store } from '../../types';
import { receivePageNotFoundId } from '../actions';

const mockFetch = (valueToReturn: any, error?: any) => () => new Promise((resolve, reject) => {
if (error) {
reject(error);
}
resolve({ json: () => new Promise((res) => res(valueToReturn)) });
});

describe('receivePageNotFoundId hook', () => {
let hook: ReturnType<typeof import ('./receivePageNotFoundId').receivePageNotFoundIdHookBody>;
let store: Store;
let helpers: MiddlewareAPI & ReturnType<typeof createTestServices>;
let historyReplaceSpy: jest.SpyInstance;
let fetchBackup: any;

beforeEach(() => {
store = createTestStore();

helpers = {
...createTestServices(),
dispatch: store.dispatch,
getState: store.getState,
};

helpers.history.location = {
pathname: '/books/physics/pages/1-introduction301',
} as any;

historyReplaceSpy = jest.spyOn(helpers.history, 'replace');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

try throwing a mockImplementation on this, i wonder if the history implementation is causing issues for you

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it didn't help, i'll keep trying


fetchBackup = (globalThis as any).fetch;

hook = require('./receivePageNotFoundId').receivePageNotFoundIdHookBody(helpers);
});

afterEach(() => {
(globalThis as any).fetch = fetchBackup;
});

it('checks for redirects when receivePageNotFoundId is dispatched and noops if path wasn\'t found', async() => {
(globalThis as any).fetch = mockFetch([{ from: 'asd', to: 'asd' }]);

await hook(receivePageNotFoundId('asdf'));

expect(historyReplaceSpy).not.toHaveBeenCalled();
});

it('noops if fetch fails', async() => {
(globalThis as any).fetch = mockFetch(undefined, 'error');

await hook(receivePageNotFoundId('asdf'));

expect(historyReplaceSpy).not.toHaveBeenCalled();
});

it('calls history.replace if redirect is found', async() => {
(globalThis as any).fetch = mockFetch([{ from: helpers.history.location.pathname, to: 'redirected' }]);

await hook(receivePageNotFoundId('asdf'));

expect(historyReplaceSpy).toHaveBeenCalledWith('redirected');
});
});
21 changes: 21 additions & 0 deletions src/app/content/hooks/receivePageNotFoundId.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
import { Redirects } from '../../../../data/redirects/types';
import { ActionHookBody } from '../../types';
import { actionHook } from '../../utils';
import { receivePageNotFoundId } from '../actions';

export const receivePageNotFoundIdHookBody: ActionHookBody<typeof receivePageNotFoundId> = (
services
) => async() => {
const redirects: Redirects = await fetch('/rex/redirects.json')
.then((res) => res.json())
.catch(() => []);

for (const {from, to} of redirects) {
if (from === services.history.location.pathname) {
services.history.replace(to);
return;
}
}
};

export default actionHook(receivePageNotFoundId, receivePageNotFoundIdHookBody);
3 changes: 3 additions & 0 deletions src/redirects.development.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[
{ "from": "/books/physics/pages/1-introduction301", "to": "/books/physics/pages/1-introduction" }
]
15 changes: 14 additions & 1 deletion src/setupProxy.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
* this file is shared between webpack-dev-server and the pre-renderer
*/
const url = require('url');
const util = require('util');
const fs = require('fs');
const path = require('path');
const proxy = require('http-proxy-middleware');
Expand Down Expand Up @@ -166,6 +165,19 @@ function stubEnvironment(app) {
});
}

function stubRedirects(app) {
app.use((req, res, next) => {
const {pathname} = url.parse(req.url);

if (pathname === '/rex/redirects.json') {
const redirectsFile = path.join(__dirname, 'redirects.development.json');
sendFile(res, redirectsFile);
} else {
next();
}
});
}

function setupProxy(app) {
if (!ARCHIVE_URL) { throw new Error('ARCHIVE_URL configuration must be defined'); }
if (!OS_WEB_URL) { throw new Error('OS_WEB_URL configuration must be defined'); }
Expand All @@ -176,6 +188,7 @@ function setupProxy(app) {
highlightsProxy(app);
osWebApiProxy(app);
stubEnvironment(app);
stubRedirects(app);

if (!SKIP_OS_WEB_PROXY) {
osWebProxy(app);
Expand Down