{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":309258998,"defaultBranch":"main","name":"browsertrix-crawler","ownerLogin":"webrecorder","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2020-11-02T04:37:14.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/13686290?v=4","public":true,"private":false,"isOrgOwned":true},"refInfo":{"name":"","listCacheKey":"v0:1717781662.0","currentOid":""},"activityList":{"items":[{"before":"0d66ec2afcbf9fabbd71d4ca67980caa3566d375","after":"bf006ce54fc8d1db04ed43f484410a28152cc1d4","ref":"refs/heads/gh-pages","pushedAt":"2024-06-07T17:36:26.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"Deployed b83d1c5 with MkDocs version: 1.6.0","shortMessageHtmlLink":"Deployed <a class=\"commit-link\" data-hovercard-type=\"commit\" data-hovercard-url=\"https://github.com/webrecorder/browsertrix-crawler/commit/b83d1c58da66f5a8bba0fc92401234bbde60d94d/hovercard\" href=\"https://github.com/webrecorder/browsertrix-crawler/commit/b83d1c58da66f5a8bba0fc92401234bbde60d94d\"><tt>b83d1c5</tt></a> with MkDocs version: 1.6.0"}},{"before":"8267899bde5f04356d7ff9db97cb881c3c6e980b","after":null,"ref":"refs/heads/dry-run-mode","pushedAt":"2024-06-07T17:34:22.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"ikreymer","name":"Ilya Kreymer","path":"/ikreymer","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1015759?s=80&v=4"}},{"before":"32435bfac785dcc834d69ada1d3f0d47291d3ff9","after":"b83d1c58da66f5a8bba0fc92401234bbde60d94d","ref":"refs/heads/main","pushedAt":"2024-06-07T17:34:19.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"ikreymer","name":"Ilya Kreymer","path":"/ikreymer","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1015759?s=80&v=4"},"commit":{"message":"add --dryRun flag and mode (#594)\n\n- if set, runs the crawl but doesn't store any archive data (WARCS,\r\nWACZ, CDXJ) while logs and pages are still written, and saved state can be\r\ngenerated (per the --saveState options).\r\n- adds test to ensure only 'logs' and 'pages' dirs are generated with --dryRun\r\n- screenshot, text extraction are skipped altogether in dryRun mode,\r\nwarning is printed that storage and archiving-related options may be\r\nignored\r\n- fixes #593","shortMessageHtmlLink":"add --dryRun flag and mode (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2338904536\" data-permission-text=\"Title is private\" data-url=\"https://github.com/webrecorder/browsertrix-crawler/issues/594\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/webrecorder/browsertrix-crawler/pull/594/hovercard\" href=\"https://github.com/webrecorder/browsertrix-crawler/pull/594\">#594</a>)"}},{"before":"e8009474264067340fb2b9409925f24f3efcc584","after":"8267899bde5f04356d7ff9db97cb881c3c6e980b","ref":"refs/heads/dry-run-mode","pushedAt":"2024-06-07T17:17:44.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ikreymer","name":"Ilya Kreymer","path":"/ikreymer","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1015759?s=80&v=4"},"commit":{"message":"post merge fix","shortMessageHtmlLink":"post merge fix"}},{"before":"1bd94d93a1efe8c39ec8b394078d246846c18457","after":"32435bfac785dcc834d69ada1d3f0d47291d3ff9","ref":"refs/heads/main","pushedAt":"2024-06-07T17:13:15.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"ikreymer","name":"Ilya Kreymer","path":"/ikreymer","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1015759?s=80&v=4"},"commit":{"message":"Consider disk usage of collDir instead of default /crawls (#586)\n\nFix #585 \r\n\r\nChanges:\r\n- compute disk usage based on crawler `collDir` property instead of\r\nalways computing it on `/crawls` directory","shortMessageHtmlLink":"Consider disk usage of collDir instead of default /crawls (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2320484246\" data-permission-text=\"Title is private\" data-url=\"https://github.com/webrecorder/browsertrix-crawler/issues/586\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/webrecorder/browsertrix-crawler/pull/586/hovercard\" href=\"https://github.com/webrecorder/browsertrix-crawler/pull/586\">#586</a>)"}},{"before":"3ffdb6965fa84b39dc11f82faa9fd897471ef145","after":"e8009474264067340fb2b9409925f24f3efcc584","ref":"refs/heads/dry-run-mode","pushedAt":"2024-06-07T17:08:48.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"ikreymer","name":"Ilya Kreymer","path":"/ikreymer","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1015759?s=80&v=4"},"commit":{"message":"include pages in dryrun","shortMessageHtmlLink":"include pages in dryrun"}},{"before":"5ff89c2324279d79d4fcb6b1fd130905062e0463","after":"3ffdb6965fa84b39dc11f82faa9fd897471ef145","ref":"refs/heads/dry-run-mode","pushedAt":"2024-06-07T17:03:52.000Z","pushType":"push","commitsCount":3,"pusher":{"login":"ikreymer","name":"Ilya Kreymer","path":"/ikreymer","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1015759?s=80&v=4"},"commit":{"message":"Merge branch 'dry-run-mode' of github.com:webrecorder/browsertrix-crawler into dry-run-mode","shortMessageHtmlLink":"Merge branch 'dry-run-mode' of github.com:webrecorder/browsertrix-cra…"}},{"before":"48db6e310ac9ba1e0805182dc29e2369174562c0","after":"5ff89c2324279d79d4fcb6b1fd130905062e0463","ref":"refs/heads/dry-run-mode","pushedAt":"2024-06-07T00:08:03.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"tw4l","name":"Tessa Walsh","path":"/tw4l","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6758804?s=80&v=4"},"commit":{"message":"add --dryRun flag, which if set, runs the crawl but doesn't store any archive data (logs + saved state still written)\ntests: add --dryRun test to ensure only ./logs dir is created","shortMessageHtmlLink":"add --dryRun flag, which if set, runs the crawl but doesn't store any…"}},{"before":"40a22fd37ed9a8ec6614bca31b3dec4862ad0333","after":"2168dbdd633a1f4b2d5bf4093ae9004352b383b2","ref":"refs/heads/use-js-wacz","pushedAt":"2024-06-06T19:15:27.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"ikreymer","name":"Ilya Kreymer","path":"/ikreymer","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1015759?s=80&v=4"},"commit":{"message":"Merge branch 'main' into use-js-wacz","shortMessageHtmlLink":"Merge branch 'main' into use-js-wacz"}},{"before":"22da4b13234d5a85e389a29833b9f098c5f0fe23","after":null,"ref":"refs/heads/cleanup-fix-pdf-test","pushedAt":"2024-06-06T19:14:47.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"ikreymer","name":"Ilya Kreymer","path":"/ikreymer","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1015759?s=80&v=4"}},{"before":"068ee79288f8bf1c5ee13b2a1d7e973afe50b829","after":"1bd94d93a1efe8c39ec8b394078d246846c18457","ref":"refs/heads/main","pushedAt":"2024-06-06T19:14:44.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"ikreymer","name":"Ilya Kreymer","path":"/ikreymer","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1015759?s=80&v=4"},"commit":{"message":"cleanup dockerfile + fix test (#595)\n\n- remove obsolete line from Dockerfile\r\n- fix pdf test to webrecorder-hosted pdf","shortMessageHtmlLink":"cleanup dockerfile + fix test (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2338989407\" data-permission-text=\"Title is private\" data-url=\"https://github.com/webrecorder/browsertrix-crawler/issues/595\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/webrecorder/browsertrix-crawler/pull/595/hovercard\" href=\"https://github.com/webrecorder/browsertrix-crawler/pull/595\">#595</a>)"}},{"before":"c29b1b231b843fefa572c551a53623ed8c2b4ae7","after":"40a22fd37ed9a8ec6614bca31b3dec4862ad0333","ref":"refs/heads/use-js-wacz","pushedAt":"2024-06-06T18:39:16.000Z","pushType":"push","commitsCount":3,"pusher":{"login":"ikreymer","name":"Ilya Kreymer","path":"/ikreymer","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1015759?s=80&v=4"},"commit":{"message":"Merge branch 'main' into use-js-wacz","shortMessageHtmlLink":"Merge branch 'main' into use-js-wacz"}},{"before":null,"after":"22da4b13234d5a85e389a29833b9f098c5f0fe23","ref":"refs/heads/cleanup-fix-pdf-test","pushedAt":"2024-06-06T18:38:20.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"ikreymer","name":"Ilya Kreymer","path":"/ikreymer","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1015759?s=80&v=4"},"commit":{"message":"cleanup:\n- remove obsolete line from Dockerfile\n- fix pdf test to webrecorder-hosted pdf","shortMessageHtmlLink":"cleanup:"}},{"before":"1b35b1a61dc19e2e0eec5035c26bd29f5084961f","after":"48db6e310ac9ba1e0805182dc29e2369174562c0","ref":"refs/heads/dry-run-mode","pushedAt":"2024-06-06T18:26:41.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"ikreymer","name":"Ilya Kreymer","path":"/ikreymer","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1015759?s=80&v=4"},"commit":{"message":"add --dryRun flag, which if set, runs the crawl but doesn't store any archive data (logs + saved state still written)\ntests: add --dryRun test to ensure only ./logs dir is created","shortMessageHtmlLink":"add --dryRun flag, which if set, runs the crawl but doesn't store any…"}},{"before":null,"after":"1b35b1a61dc19e2e0eec5035c26bd29f5084961f","ref":"refs/heads/dry-run-mode","pushedAt":"2024-06-06T18:17:30.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"ikreymer","name":"Ilya Kreymer","path":"/ikreymer","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1015759?s=80&v=4"},"commit":{"message":"add --dryRun flag, which if set, runs the crawl but doesn't store any archive data (logs + saved state still written)\ntests: add --dryRun test to ensure only ./logs dir is created","shortMessageHtmlLink":"add --dryRun flag, which if set, runs the crawl but doesn't store any…"}},{"before":"eb47cb7a5030164253c810f3ac6c71a98caa0845","after":"3764df33b63c2ecf90c90b4cdc7ebcfaa3baad08","ref":"refs/heads/fix-proxy","pushedAt":"2024-06-06T16:29:32.000Z","pushType":"push","commitsCount":9,"pusher":{"login":"ikreymer","name":"Ilya Kreymer","path":"/ikreymer","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1015759?s=80&v=4"},"commit":{"message":"remove logging test","shortMessageHtmlLink":"remove logging test"}},{"before":"b6942786b1de7e8be728394dc0196fb9c23c3109","after":"4ee0ce620f3a3c7b8589bb65822f3a9aebfe70cc","ref":"refs/heads/fix-socks-proxy","pushedAt":"2024-06-06T05:26:47.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"ikreymer","name":"Ilya Kreymer","path":"/ikreymer","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1015759?s=80&v=4"},"commit":{"message":"ensure --proxyServer cli flag also supported by both direct fetch and browser, simplify\nproxy init","shortMessageHtmlLink":"ensure --proxyServer cli flag also supported by both direct fetch and…"}},{"before":"b28a417cffcde6932da5b064c7eecd7874508d19","after":"b6942786b1de7e8be728394dc0196fb9c23c3109","ref":"refs/heads/fix-socks-proxy","pushedAt":"2024-06-06T01:43:54.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ikreymer","name":"Ilya Kreymer","path":"/ikreymer","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1015759?s=80&v=4"},"commit":{"message":"add missing file","shortMessageHtmlLink":"add missing file"}},{"before":"1202b52853c9137fbebb1fe77bb04057c93e492b","after":"b28a417cffcde6932da5b064c7eecd7874508d19","ref":"refs/heads/fix-socks-proxy","pushedAt":"2024-06-06T00:40:19.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ikreymer","name":"Ilya Kreymer","path":"/ikreymer","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1015759?s=80&v=4"},"commit":{"message":"use global dispatcher\ntests: add proxy tests for socks5 and http!","shortMessageHtmlLink":"use global dispatcher"}},{"before":"ec07cc99f10f26a20a5a1b45977bea2fc3398f63","after":"1202b52853c9137fbebb1fe77bb04057c93e492b","ref":"refs/heads/fix-socks-proxy","pushedAt":"2024-06-05T22:24:13.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"ikreymer","name":"Ilya Kreymer","path":"/ikreymer","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1015759?s=80&v=4"},"commit":{"message":"tweak logging, hostname use","shortMessageHtmlLink":"tweak logging, hostname use"}},{"before":"1f9fa7a77f1467165a7d38593e91bc6d973b4cdf","after":"19b6bca6611f7f08bd96b9d8a2e53ae11c55ca74","ref":"refs/heads/dom-snapshot","pushedAt":"2024-06-05T19:47:32.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ikreymer","name":"Ilya Kreymer","path":"/ikreymer","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1015759?s=80&v=4"},"commit":{"message":"use stylex to avoid parsing style","shortMessageHtmlLink":"use stylex to avoid parsing style"}},{"before":"7f114776543ed534ebddfc27c8f25efc82d5c8d4","after":null,"ref":"refs/heads/add-group-policies","pushedAt":"2024-06-05T19:46:52.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"ikreymer","name":"Ilya Kreymer","path":"/ikreymer","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1015759?s=80&v=4"}},{"before":"757e838832f27214456731e1494a105e002594b0","after":"068ee79288f8bf1c5ee13b2a1d7e973afe50b829","ref":"refs/heads/main","pushedAt":"2024-06-05T19:46:49.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"ikreymer","name":"Ilya Kreymer","path":"/ikreymer","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1015759?s=80&v=4"},"commit":{"message":"Add group policies, limit browser access to container filesystem (#579)\n\nAdd some default policy settings to disable unneeded Brave features.\r\nHelps a bit with #463, but Brave unfortunately doesn't provide all\r\nmentioned settings as policy options.\r\n\r\nMost important changes are in\r\n`config/policies/lockdown-profilebrowser.json` it limits access to the\r\ncontainer filesystem especially during interactive profile browser\r\ncreation.","shortMessageHtmlLink":"Add group policies, limit browser access to container filesystem (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2308368523\" data-permission-text=\"Title is private\" data-url=\"https://github.com/webrecorder/browsertrix-crawler/issues/579\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/webrecorder/browsertrix-crawler/pull/579/hovercard\" href=\"https://github.com/webrecorder/browsertrix-crawler/pull/579\">#579</a>)"}},{"before":"9e19f6f0844d883dedddd68e797156a067f5af45","after":null,"ref":"refs/heads/version-bump","pushedAt":"2024-06-04T20:35:16.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"ikreymer","name":"Ilya Kreymer","path":"/ikreymer","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1015759?s=80&v=4"}},{"before":"a7d279cfbd3f6ca8375e564d1e0ef751af7e345f","after":"757e838832f27214456731e1494a105e002594b0","ref":"refs/heads/main","pushedAt":"2024-06-04T20:35:13.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"ikreymer","name":"Ilya Kreymer","path":"/ikreymer","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1015759?s=80&v=4"},"commit":{"message":"base image version bump to brave 1.66.115 (#592)","shortMessageHtmlLink":"base image version bump to brave 1.66.115 (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2326706473\" data-permission-text=\"Title is private\" data-url=\"https://github.com/webrecorder/browsertrix-crawler/issues/592\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/webrecorder/browsertrix-crawler/pull/592/hovercard\" href=\"https://github.com/webrecorder/browsertrix-crawler/pull/592\">#592</a>)"}},{"before":null,"after":"ec07cc99f10f26a20a5a1b45977bea2fc3398f63","ref":"refs/heads/fix-socks-proxy","pushedAt":"2024-05-31T07:32:32.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"vnznznz","name":"Vinzenz Sinapius","path":"/vnznznz","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1849859?s=80&v=4"},"commit":{"message":"WIP 1","shortMessageHtmlLink":"WIP 1"}},{"before":"459343fe5feb700bdfaea1930d8b75fe7a8b5b98","after":"9e19f6f0844d883dedddd68e797156a067f5af45","ref":"refs/heads/version-bump","pushedAt":"2024-05-30T21:12:56.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ikreymer","name":"Ilya Kreymer","path":"/ikreymer","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1015759?s=80&v=4"},"commit":{"message":"bump to 1.66.115","shortMessageHtmlLink":"bump to 1.66.115"}},{"before":null,"after":"459343fe5feb700bdfaea1930d8b75fe7a8b5b98","ref":"refs/heads/version-bump","pushedAt":"2024-05-30T20:24:33.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"ikreymer","name":"Ilya Kreymer","path":"/ikreymer","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1015759?s=80&v=4"},"commit":{"message":"version bump: bump base image to brave 1.65.133","shortMessageHtmlLink":"version bump: bump base image to brave 1.65.133"}},{"before":"22ddc92784c66b27ff449b554d73c129804aac41","after":"c29b1b231b843fefa572c551a53623ed8c2b4ae7","ref":"refs/heads/use-js-wacz","pushedAt":"2024-05-30T16:53:06.000Z","pushType":"push","commitsCount":18,"pusher":{"login":"ikreymer","name":"Ilya Kreymer","path":"/ikreymer","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1015759?s=80&v=4"},"commit":{"message":"Merge branch 'main' into use-js-wacz","shortMessageHtmlLink":"Merge branch 'main' into use-js-wacz"}},{"before":"a548163bff0aea95000cbc0a574b13a7e80c1b07","after":"1f9fa7a77f1467165a7d38593e91bc6d973b4cdf","ref":"refs/heads/dom-snapshot","pushedAt":"2024-05-30T12:41:50.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ikreymer","name":"Ilya Kreymer","path":"/ikreymer","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1015759?s=80&v=4"},"commit":{"message":"move domsnapshot to recorder with 'useDomSnapshot':\n- take snapshot at the very end after behaviors, instead of with initial text extraction\n- store in place of response record to avoid duplicates in replay (experimental)","shortMessageHtmlLink":"move domsnapshot to recorder with 'useDomSnapshot':"}}],"hasNextPage":true,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAAEX3aZkQA","startCursor":null,"endCursor":null}},"title":"Activity · webrecorder/browsertrix-crawler"}