feat(blog): add options.createFeedItems to filter/limit/transform feed items #8378

johnnyreilly · 2022-11-26T08:51:17Z

Pre-flight checklist

I have read the Contributing Guidelines on pull requests.
If this is a code change: I have written unit tests and/or added dogfooding pages to fully verify the new behavior.
If this is a new API or substantial change: the PR has an accompanying issue (closes #0000) and the maintainers have approved on my working plan.

THIS IS AN EXAMPLE IMPLEMENTATION TO ILLUSTRATE #8376

Motivation

My blog turns out to be quite large. It was recently brought to my attention that the RSS feed wasn't working well for readers as a consequence.

At present, Docusaurus includes all blog entries in the Atom / RSS feeds: https://docusaurus.io/docs/blog#feed

The proposal is to add another option to support filtering the responses in the feed. This would allow people to reduce the size of their blogs when they become large.

If this existed, then this would not be necessary: johnnyreilly/blog.johnnyreilly.com#352

Test Plan

See automated tests and observe the atom.xml below - 2 entries only due to filtering:

or go here: https://deploy-preview-8378--docusaurus-2.netlify.app/blog/rss.xml

Test links

Deploy preview: https://deploy-preview-8378--docusaurus-2.netlify.app/blog/rss.xml

Docs:

Related issues/PRs

fix #8376

netlify · 2022-11-26T18:30:43Z

✅ [V2]

Name	Link
🔨 Latest commit	`f5bb167`
🔍 Latest deploy log	https://app.netlify.com/sites/docusaurus-2/deploys/63ad7c49ee1147000849fd83
😎 Deploy Preview	https://deploy-preview-8378--docusaurus-2.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

github-actions · 2022-11-26T18:31:24Z

⚡️ Lighthouse report for the deploy preview of this PR

URL	Performance	Accessibility	Best Practices	SEO	PWA	Report
/	🟠 61	🟢 97	🟢 100	🟢 100	🟢 90	Report
/docs/installation	🟠 81	🟢 100	🟢 100	🟢 100	🟢 90	Report

slorber

See comment

packages/docusaurus-plugin-content-blog/src/feed.ts

johnnyreilly · 2022-12-27T10:51:09Z

Reworked this based on feedback to replace the filter API with a createFeedItems API based on @slorber's suggestion:

    createFeedItems?: (options: {
      blogPosts: BlogPost[];
      siteConfig: DocusaurusConfig;
      outDir: string;
      defaultCreateFeedItems: (params: {
        blogPosts: BlogPost[];
        siteConfig: DocusaurusConfig;
        outDir: string;
      }) => Promise<BlogFeedItem[]>;
    }) => Promise<BlogFeedItem[]>;

Usage looks like this:

            createFeedItems: async (options) => {
              const { blogPosts, defaultCreateFeedItems, ...others } = options;
              const blogPostsFiltered = blogPosts.filter((item, index) => index < 2);
              return defaultCreateFeedItems({ blogPosts: blogPostsFiltered, ...others });
            }

The above implements the a filter; merely one of many things that could be done.

slorber

Thanks, looks almost good 👍

packages/docusaurus-plugin-content-blog/src/plugin-content-blog.d.ts

website/docs/blog.mdx

slorber · 2022-12-27T18:42:14Z

packages/docusaurus-plugin-content-blog/src/feed.ts

@@ -54,11 +55,35 @@ async function generateBlogFeed({
    copyright: feedOptions.copyright,
  });

+  const feedItems = await (feedOptions.createFeedItems


maybe simplify with:

const createFeedItems = feedOptions.createFeedItems ?? defaultCreateFeedItems

?

Would love to but the signatures are subtly different. Did ponder other simplifications but didn't come up with anything that seemed worthwhile. Happy to implement any other ideas?

Would it be a big problem if the 2 functions had the exact same signature? It shouldn't harm anyone if defaultCreateFeedItems would "receive itself" as a param, although it wouldn't be used in practice.

gave it a try and apparently TS is fine with that thanks to duck typing 👍 updated the PR

slorber · 2022-12-28T11:06:32Z

LGTM

Need a bit of docs in https://deploy-preview-8378--docusaurus-2.netlify.app/docs/api/plugins/@docusaurus/plugin-content-blog/#configuration

Also remove current test before merge ;)

johnnyreilly · 2022-12-28T14:31:24Z

Awesome! I'll add some docs shortly

johnnyreilly · 2022-12-28T19:24:12Z

Docs added https://deploy-preview-8378--docusaurus-2.netlify.app/docs/api/plugins/@docusaurus/plugin-content-blog/#configuration

johnnyreilly · 2022-12-29T11:49:42Z

Nice - params is more accurate than options

…d items (#8378) Co-authored-by: sebastienlorber <lorber.sebastien@gmail.com>

slorber · 2023-07-26T08:59:51Z

Glad you submitted that PR @johnnyreilly 😄

That will allow me to stay under Vercel fair use free plan for a little bit more. My 4MO feed is exploding the plan otherwise.

Wonder if we shouldn't limit by default. Considering we output blog content in the feed and it's not rare to see blogs with 100+ blog posts. Wonder what would be a good default limit (20?) and what would be the API to change it or disable it in case you really want all blog posts in your feed.

johnnyreilly · 2023-07-26T10:29:03Z

Haha - glad it's of use to you! Yeah I did wonder about this. I think the other thing to factor in is that it's the size of the feed that's the issue for consumers generally, and that's not just a factor of the number of blog entries. But also the length of the underlying posts. It'd be nice if it could limit by size of feed, including the max number of entries that keeps the feed under a reasonable size. Though maybe this is too complicated to expose to users.

The idea of a default of a max number of entries of 20 seems reasonable. Again, being able to opt out is probably wise.

slorber · 2023-07-27T09:24:42Z

It'd be nice if it could limit by size of feed, including the max number of entries that keeps the feed under a reasonable size.

That seems hard to achieve with our current system and a bit overkill. We write the XML at once from in-memory data and don't stream to it with the ability to stop at a given time.

Limiting the number of entries is much simpler and probably good enough.

I guess we can have a feed.limit: number | false | null = 20

Let me know if you want to submit a PR

johnnyreilly · 2023-07-27T12:12:40Z

Yeah I'd be happy to submit a PR; I'm on holiday right now but maybe sometime in August. Feel free to ping me if I forget

slorber · 2023-07-27T12:33:36Z

👍 Here's an issue: #9179

For some reason I can't assign it to you

johnnyreilly · 2023-07-27T12:48:59Z

That's funny - I can't assign it to myself either. Maybe I have to be added as a member to the Docusaurus project to be assigned something? Not sure

slorber · 2023-07-27T13:06:57Z

I can assign the issue to many john... persons but not you, you don't appear in the autocomplete 😅 maybe it's a profile setting? 🤔

johnnyreilly · 2023-07-27T13:54:03Z

Maybe - mysterious!

Josh-Cena · 2023-07-28T15:11:08Z

I think you can only assign to people who have participated and people affiliated with the organization?

johnnyreilly added 3 commits November 10, 2022 18:03

fix: update historic g-tags

921a57c

Merge branch 'facebook:main' into main

add4de2

feat: example implementation of blog filter

0bbe8fe

johnnyreilly requested a review from slorber as a code owner November 26, 2022 08:51

facebook-github-bot added the CLA Signed Signed Facebook CLA label Nov 26, 2022

johnnyreilly mentioned this pull request Nov 26, 2022

Provide ability to filter RSS / Atom feed #8376

Closed

2 tasks

johnnyreilly added 3 commits November 26, 2022 09:27

feat: demo on website

915dc45

feat: function

fb9eb21

docs: add to docs

2d88ab4

fix: type

aeeee38

slorber requested changes Dec 9, 2022

View reviewed changes

packages/docusaurus-plugin-content-blog/src/feed.ts Outdated Show resolved Hide resolved

slorber changed the title ~~example blog filter implementation~~ feat(blog): add more control over blog feed items (filter/limit/transform) Dec 9, 2022

slorber added the pr: new feature This PR adds a new API or behavior. label Dec 9, 2022

slorber marked this pull request as draft December 9, 2022 10:03

johnnyreilly added 2 commits December 27, 2022 10:08

feat: replace filter with createFeedItems

d3317d9

Merge branch 'main' into feat/blog-filter

a0aa0fe

johnnyreilly marked this pull request as ready for review December 27, 2022 10:50

johnnyreilly requested review from lex111 and Josh-Cena as code owners December 27, 2022 10:50

johnnyreilly added 2 commits December 27, 2022 17:08

fix: update snapshots

941edcc

fix: import

3b984ce

slorber reviewed Dec 27, 2022

View reviewed changes

johnnyreilly added 2 commits December 27, 2022 19:57

fix: address feedback

745b0ab

fix: import

64bc442

minor simplifications

5980c82

johnnyreilly added 2 commits December 28, 2022 19:19

docs: add docs

3b900e9

feat: remove test

0ad7575

johnnyreilly and others added 3 commits December 28, 2022 19:32

fix: error

69485cd

rename createFeedItems options => params

1665b3d

rename createFeedItems options => params + better docs

f5bb167

slorber added the to backport This PR is planned to be backported to a stable version of Docusaurus label Dec 29, 2022

slorber changed the title ~~feat(blog): add more control over blog feed items (filter/limit/transform)~~ feat(blog): add options.createFeedItems to filter/limit/transform feed items Dec 29, 2022

slorber merged commit 022e005 into facebook:main Dec 29, 2022

johnnyreilly deleted the feat/blog-filter branch December 29, 2022 14:29

slorber added a commit that referenced this pull request Jan 26, 2023

feat(blog): add options.createFeedItems to filter/limit/transform fee…

25a4ec3

…d items (#8378) Co-authored-by: sebastienlorber <lorber.sebastien@gmail.com>

slorber added backported This PR has been backported to a stable version of Docusaurus and removed to backport This PR is planned to be backported to a stable version of Docusaurus labels Jan 27, 2023

johnnyreilly mentioned this pull request Feb 11, 2023

feat: in defence of pull requests johnnyreilly/blog.johnnyreilly.com#430

Merged

slorber mentioned this pull request Jul 27, 2023

Blog feed: add a first-class API to limit size of the RSS / Atom feeds #9179

Closed

2 tasks

johnnyreilly mentioned this pull request Jul 31, 2023

feat(blog-plugin): limit option for blog feedOptions #9189

Merged

3 tasks

slorber mentioned this pull request Oct 19, 2023

chore: v3.0.0-rc.0 release #9418

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(blog): add options.createFeedItems to filter/limit/transform feed items #8378

feat(blog): add options.createFeedItems to filter/limit/transform feed items #8378

johnnyreilly commented Nov 26, 2022 •

edited by slorber

netlify bot commented Nov 26, 2022 •

edited

github-actions bot commented Nov 26, 2022 •

edited

slorber left a comment

johnnyreilly commented Dec 27, 2022 •

edited

slorber left a comment

slorber Dec 27, 2022

johnnyreilly Dec 27, 2022

slorber Dec 28, 2022

slorber Dec 28, 2022

slorber commented Dec 28, 2022

johnnyreilly commented Dec 28, 2022

johnnyreilly commented Dec 28, 2022

johnnyreilly commented Dec 29, 2022

slorber commented Jul 26, 2023

johnnyreilly commented Jul 26, 2023

slorber commented Jul 27, 2023

johnnyreilly commented Jul 27, 2023

slorber commented Jul 27, 2023

johnnyreilly commented Jul 27, 2023

slorber commented Jul 27, 2023

johnnyreilly commented Jul 27, 2023

Josh-Cena commented Jul 28, 2023

feat(blog): add options.createFeedItems to filter/limit/transform feed items #8378

feat(blog): add options.createFeedItems to filter/limit/transform feed items #8378

Conversation

johnnyreilly commented Nov 26, 2022 • edited by slorber

Pre-flight checklist

Motivation

Test Plan

Test links

Related issues/PRs

netlify bot commented Nov 26, 2022 • edited

✅ [V2]

github-actions bot commented Nov 26, 2022 • edited

⚡️ Lighthouse report for the deploy preview of this PR

slorber left a comment

Choose a reason for hiding this comment

johnnyreilly commented Dec 27, 2022 • edited

slorber left a comment

Choose a reason for hiding this comment

slorber Dec 27, 2022

Choose a reason for hiding this comment

johnnyreilly Dec 27, 2022

Choose a reason for hiding this comment

slorber Dec 28, 2022

Choose a reason for hiding this comment

slorber Dec 28, 2022

Choose a reason for hiding this comment

slorber commented Dec 28, 2022

johnnyreilly commented Dec 28, 2022

johnnyreilly commented Dec 28, 2022

johnnyreilly commented Dec 29, 2022

slorber commented Jul 26, 2023

johnnyreilly commented Jul 26, 2023

slorber commented Jul 27, 2023

johnnyreilly commented Jul 27, 2023

slorber commented Jul 27, 2023

johnnyreilly commented Jul 27, 2023

slorber commented Jul 27, 2023

johnnyreilly commented Jul 27, 2023

Josh-Cena commented Jul 28, 2023

johnnyreilly commented Nov 26, 2022 •

edited by slorber

netlify bot commented Nov 26, 2022 •

edited

github-actions bot commented Nov 26, 2022 •

edited

johnnyreilly commented Dec 27, 2022 •

edited