Gatsby library search page #3456

jastack · 2018-01-09T23:47:04Z

Added a draft of the plugin search with Algolia. Check it out by going to "/plugins/". The left side search bar functions in a very similar way to the regular sidebar, so I made some changes to the template index.js as well. Let me know if this is a good approach / going in the right direction! Doesn't have filter right now, but should be easy to add. @calcsam

…sby plugins

gatsbybot · 2018-01-09T23:52:58Z

Deploy preview for gatsbygram ready!

Built with commit cb316ca

https://deploy-preview-3456--gatsbygram.netlify.com

KyleAMathews · 2018-01-10T00:17:45Z

Looking good!

https://deploy-preview-3456--gatsbyjs.netlify.com/plugins/

One quick bit of feedback — we use Yarn to manage dependencies not NPM so please remove the package-lock.json files that were added.

calcsam · 2018-01-10T00:45:44Z

Awesome to see this on #3003! Thanks @jastack!

Some next steps:

URL stuff:
(1) plugin links should be /plugins/name not /packages/name
(2) search queries should be /plugins?text=URL_ENCODED_SEARCH_STRING so that if a user refreshes they get the same page. As you type things in the search bar the URL should update.
Right sidebar default is funky. Whenever the search bar is empty this should display the default panel as shown in @rdela's mock (but omit the learn link for now). If there are no results it should be a blank panel

Don't worry about the landing page in the mock -- let's just stick with the search for now.
We'll want to pull in all plugins, not just official plugins
Mobile support

Random design stuff:

The selected plugin should be highlighted
The left panel should have the light purple background
Border or box shadow on top, left and bottom.
whole package result box should be a click target, not just the name
Margin between the title and the body is too small

calcsam · 2018-01-09T23:54:16Z

www/src/components/searchbar-body.js

+
+import typography from "../utils/typography"
+
+function Search(){


const Search = () =>

calcsam · 2018-01-10T00:06:05Z

www/src/components/searchbar-body.js

+          style={{
+            color: `white`,
+            border: `0`,
+            boxShadow: `0 0 0 0`,


boxShadow: none should work

calcsam · 2018-01-10T00:07:10Z

www/src/components/searchbar-body.js

+        </Link>
+        <div css={{
+          fontFamily: typography.options.bodyFontFamily.join(`,`),
+          fontSize: `${13 / 16 * 100}%`,


calcsam · 2018-01-10T00:07:21Z

www/src/components/searchbar-body.js

+        <div css={{
+          fontFamily: typography.options.bodyFontFamily.join(`,`),
+          fontSize: `${13 / 16 * 100}%`,
+          lineHeight: `1.1`,


(theoretically a whole # or simple fraction like 3/4)

calcsam · 2018-01-10T00:07:56Z

www/src/components/searchbar-body.js

+          fontFamily: typography.options.bodyFontFamily.join(`,`),
+          fontSize: `${13 / 16 * 100}%`,
+          lineHeight: `1.1`,
+          paddingTop: `7px`,


rhythm(n)

(theoretically a whole # or simple fraction like 3/4)

calcsam · 2018-01-10T00:13:52Z

www/src/html.js

+          <link
+            rel="stylesheet"
+            href="https://unpkg.com/react-instantsearch-theme-algolia@4.0.0/style.min.css"
+          />


do we need / use this?

I do use their styling for the search box, but mostly included it so I could have something that looked decent fairly quickly. Can definitely take this out and do our own.

Got it. Didn't realize we were using it. Let's talk about styling in one fell swoop, but for now just add rel="preload"

calcsam · 2018-01-10T00:14:28Z

www/src/components/searchbar-body.js

+      borderBottom: `1px solid grey`,
+    }}>
+      <div css={{
+        padding: `10px`,


calcsam · 2018-01-10T00:27:09Z

www/src/components/searchbar-body.js

+  )
+}
+
+function Result({ hit }){


const Result = ({ hit }) => {

KyleAMathews · 2018-01-10T00:48:04Z

plugin links should be /plugins/name not /packages/name

Is there a compelling reason to change the existing URL structure?

KyleAMathews · 2018-01-10T00:49:10Z

On pulling in all plugins — we should include in the search any package on NPM with the gatsby-plugin tag. We'll probably have to create a blacklist at some point if the tag gets spammed.

calcsam · 2018-01-10T00:50:37Z

Is there a compelling reason to change the existing URL structure?

@KyleAMathews could be a bit weird if the landing page is /plugins but links go to /packages/x

calcsam · 2018-01-10T01:16:28Z

another thought on routing -- since we're not searching strictly plugins, we're searching all packages, let's have this live at /packages (we could make /plugins redirect to /packages too if we want).

calcsam · 2018-01-10T01:26:09Z

re docsearch & algolia -- two things:

(1) The medium-term plan is to, at build time, have a source plugin that uses Algolia npm-search (https://blog.algolia.com/yarn-search-javascript-packages/) to pull all package details including the readme into Gatsby data, build a page for each package.

(2) In the short run, this is a great start and probably no need to fiddle further with algolia stuffbefore merging

jastack · 2018-01-10T21:43:39Z

Thanks guys! I'm going to work on some fixes based on your feedback then get back to you

shannonbux · 2018-01-10T23:30:06Z

@jastack This is exciting! I helped with the first prototypes for this and interviewed a bunch of Gatsby users to figure out what features are essential, so it's cool to see it progressing!! A couple notes:

Another thing we can use Algiola for (from my understanding) is pulling more Github data beyond just the READme file. We've gotten feedback that popularity (maybe stars?) and number of downloads could be useful data points. I'm hoping those could appear on the left hand side somehow, or if nothing else, in some sort of header above the READme file on the right. I can provide screenshots of these ideas. This might not be essential in first version since we don't have lots of duplicate plugins yet.
Also, we will need to use some sort of truncation for the plugin names and descriptions on the left hand side, because some names are really long haha.
@KyleAMathews and @calcsam This mock shows the title "Gatsby Library" in the search bar. What are we going to call this? I like that because it's shorter than Gatsby Plugin Search Tool...

KyleAMathews · 2018-01-10T23:51:55Z

Another thing we can use Algiola for (from my understanding) is pulling more Github data beyond just the READme file. We've gotten feedback that popularity (maybe stars?) and number of downloads could be useful data points.

💯 to this

This mock shows the title "Gatsby Library" in the search bar

yeah, let's use Gatsby Library

rdela · 2018-01-11T00:02:42Z

Also please do not forget "search by Algolia" @jastack #3003 (comment)

Probably want to link "Algolia" to either
https://github.com/algolia/npm-search
or
https://www.algolia.com/

like:
Search by Algolia

The main website
https://www.algolia.com/

…is probably safest bet since I think we plan to include both doc search and npm-search. @KyleAMathews? @calcsam? thoughts? Should we loop Haroen in to help decide?

…to text in packages

jastack · 2018-01-12T22:59:15Z

Alright, finally pushed another draft! Check it out at "/packages" now.

@calcsam and @KyleAMathews - the biggest change is that now the search pulls from Algolia's npm-search, instead of from only source packages, which means any package with the "gatsby-plugin" tag shows up. I realize this means more work because as Sam mentioned, we will need to have a source plugin in that pulls in all the other package details to build a page for each. So if it makes sense to stick with source package search in the short-term we can do that.

@shannonbux let me know what you think about the styling. I included more info on each search result, like tags, last updated, and downloads and can add icons too.

KyleAMathews · 2018-01-13T01:04:23Z

Let's just do the source plugin for npm-search so we can build out pages for all the packages.

Two tweaks to the search

Remove deprecated packages (if that data is exposed)
also search for gatsby-component keyword (e.g. Create new npm keyword for gatsby components #3507)

Really starting to come together! Excited to see this happen!

KyleAMathews · 2018-01-13T01:09:50Z

www/src/pages/packages.js

+              textAlign: `center`,
+            }}
+
+            >Welcome to the Gatsby Plugin and Starter Library!</h1>


Let's change "plugin and starter" to just "package" since we're not going to search for starters here

KyleAMathews · 2018-01-13T01:15:38Z

www/src/components/searchbar-body.js

+        justifyContent: `center`,
+      }}>
+      <SearchBox
+        translations={{placeholder: 'Search Gatsby Library by keyword'}}


Should be Search Gatsby Library

calcsam · 2018-01-13T00:16:10Z

www/gatsby-browser.js

@@ -0,0 +1 @@
+


prob want to remove this file :)

calcsam · 2018-01-13T00:26:01Z

www/src/components/withUrlSync.js

+
+const updateAfter = 700;
+const searchStateToQueryString = searchState => ({
+  q: searchState.query,


no using one-letter variables outside of math class :) query and page are more understandable

ignore if not using file

calcsam · 2018-01-13T00:26:53Z

www/src/components/withUrlSync.js

+    ...(searchState.refinementList.keywords && {
+      keywords: searchState.refinementList.keywords,
+    }),
+  }),


can you break this out into a couple of lines with descriptive variable names? it would be more understandable that way.

ignore if not using file

calcsam · 2018-01-13T00:31:45Z

www/src/components/withUrlSync.js

+    page: p || 1,
+    refinementList: {
+      ...(keywords && { keywords }),
+      ...(owner && { 'owner.name': owner }),


what's the reason for 'owner.name' instead of 'owner'?

is there a reason why this function isn't just:

const parsedQuery = qs.parse(queryString) return { ...parsedQuery, page: parsedQuery.page || 1 }

ignore if not using file

calcsam · 2018-01-13T00:34:45Z

www/src/components/withUrlSync.js

+      this.state = {
+        searchState: queryStringToSearchState(location.search.slice(1)),
+      };
+      window.addEventListener('popstate', ({ state: searchState }) => {


event listeners should be in componentDidMount and cleaned up in componentWillUnmount

ignore if not using file

calcsam · 2018-01-13T01:09:09Z

www/src/components/searchbar-body.js

+  onSearchStateChange = searchState => {
+    clearTimeout(this.debouncedSetState);
+    this.debouncedSetState = setTimeout(() => {
+      this.props.history.replace(`/packages?=${searchState.query}`)


keep this for now, but...

...we shouldn't be manually manipulating props // the global , we should be calling navigateTo which is how gatsby handles navigation.....

....however, this may not work at the moment with search queries due to a separate bug: #2074. (We need to check for differences in location.pathname and location.search) So hold off on changing this for a bit, I'll let you know when I address that and then you can use navigateTo.

calcsam · 2018-01-13T01:15:23Z

www/src/html.js

+          <link
+            rel="stylesheet"
+            href="https://unpkg.com/react-instantsearch-theme-algolia@4.0.0/style.min.css"
+          />


Got it. Didn't realize we were using it. Let's talk about styling in one fell swoop, but for now just add rel="preload"

calcsam · 2018-01-13T01:19:03Z

www/src/components/withUrlSync.js

+  };
+};
+
+const originalPathName = location.pathname;


this is going to be evaluated outside of the component lifecycle, which is bad. try evaluating it in a constructor

ignore if not using file

calcsam · 2018-01-13T01:20:36Z

www/src/components/withUrlSync.js

@@ -0,0 +1,91 @@
+import React, { Component } from 'react';


File should be capitalized 'WithUrlSync' since it's a component

.......wait, are we using this file at all? I don't see it referenced anywhere

calcsam · 2018-01-13T01:24:28Z

www/src/components/withUrlSync.js

+    }
+
+    onSearchStateChange = searchState => {
+      clearTimeout(this.debouncedSetState);


see above stuff re debouncing

ignore if not using file

…and package detail page

gillkyle · 2018-02-07T02:30:39Z

I made some progress but I'm trying to resolve a couple of these testing/build errors on the most current commit (cb316ca) to the branch without much luck, the tests that I see failing (4 in total) running yarn test give errors akin to this:

FAIL  packages/gatsby-transformer-sharp/src/__tests__/gatsby-node.js
  ● Test suite failed to run

    dlopen(/Users/kyle/Kyle/GitHub/plugin-gatsby-library/node_modules/sharp/build/Release/sharp.node, 1): Library not loaded: @rpath/libvips-cpp.42.dylib
      Referenced from: /Users/kyle/Kyle/GitHub/plugin-gatsby-library/node_modules/sharp/build/Release/sharp.node
      Reason: image not found

This looked similar to errors on #1754, and seem like they could be issues introduced by changes in the yarn.lock with updated dependencies. Does that seem possible?

And the snapshots with errors (4) are little changes like this. I'm not familiar with when you'd need to update a jest snapshot but the changes I see to the code for the plugin library don't look like they'd be causing an issue like this, and it could be the result of something that was merged oddly.

- Snapshot
+ Received
    
-  <pre class="custom-js"><code><span class="token comment" spellcheck="true">// Fake</span>
+  <pre class="custom-js"><code><span class="token comment">// Fake</span>

I think I'll reset back to find a commit where these tests weren't failing to identify what really is causing them. Any other ideas @KyleAMathews? Maybe it's even simpler than I think.

calcsam · 2018-02-07T17:13:27Z

@gillkyle the changes being caused by an odd merge could definitely be an issue. You may want to try a cherry-picking or interactive rebase strategy to isolate and squash the commits that are part of this branch (the ones by jastack) and rebase on top of a fresh master.

KyleAMathews · 2018-02-07T19:14:25Z

@gillkyle tests can sometimes fail locally due to no fault of your own — Gatsby has a ton of packages and if a few of them fail to install correctly (seems like a near guarantee...) then some of tests will fail. Pay more attention to TravisCI failing then what's happening locally.

None of those tests are affected by this PR so there's probably nothing that needs fixed.

Resolve the conflicting files and then see if TravisCI passes tests.

KyleAMathews · 2018-02-07T23:36:03Z

@gillkyle turned out that a minor release of one our dependencies broke some of the snapshots 🤷‍♂️ Fixed upstream with #3902. Merge in the lastest master and you'll be good.

gillkyle · 2018-02-07T23:37:58Z

I was just looking into it, I'll merge it in and try building it on my branch with Travis CI, thanks for the help!

fk · 2018-02-22T19:06:55Z

Just chatted about this with @shannonbux a little and remembered that I had drawn a bunch of icons for the mobile navigation in case "Plugins" became its own section:

The two above the "magic wand" icon a) pick up the quite common "puzzle pieces" metaphor for "Plugins", b) show data flowing through those plugins (and/or "Transformers"), all within the Gatsby monogram/"G" shape. Not super clear, so probably not a way to go.

Anyways…questions:

I see we decided to go with "Gatsby Library" (which I like a lot)—I suppose non of these icons really fit here anymore?
Will this section be called "Library" in the main navigation?
Where do we place it in the main navigation—in between "Tutorial" and "Community"?
As said I like "Library" a lot, but I also find it a little misleading as I also could imagine the "Gatsby Guides" to be living there. "Plugins" doesn't have this issue IMO. Does that make any sense or am I overthinking?

Adding another item to the main navigation is another nudge to re-think the latter in terms of the home page layout… ;-) For now I think the additional "Library" should fit in with that landing page from ~900px browser width.

Since I'd hate to lose the other navigation items when going below that width, maybe https://css-tricks.com/the-priority-navigation-pattern/ is one way to proceed long-term? Or we could revamp the homepage layout… :-(

fk · 2018-02-22T21:28:00Z

Will starters be indexed/part of the library at some point too?

shannonbux · 2018-02-22T22:30:04Z

@fk I have no attachment to any particular name at this point; plugins is probably clearer! And about starters--we considered combining them with plugins, then decided against it because we wanted a more visual showcase that in my imagination, could also be a way to share templates. Open to discussion on where to put an index of starters.

…

On Thu, Feb 22, 2018 at 2:28 PM, Florian Kissling ***@***.***> wrote: Will starters be indexed/part of the library at some point too? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3456 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/Ae9o2q1-aYBJ5S1TiljFcstnT4duZgabks5tXdvpgaJpZM4RYm-i> .

KyleAMathews · 2018-02-22T23:01:36Z

Will starters be indexed/part of the library at some point too?

yeah, like Shannon said. The Library is for plugins and Gatsby-specific React components.

KyleAMathews · 2018-02-22T23:02:08Z

Though let's close this PR since work is continuing over on @gillkyle's PR

jastack added 3 commits January 9, 2018 14:54

Add plugins.js and searchbar-body.js for searching and displaying gat…

4c608f7

…sby plugins

Merge branch 'master' of https://github.com/gatsbyjs/gatsby into plugins

8b1be4e

Remove unneccesary css

2af2990

calcsam reviewed Jan 10, 2018

View reviewed changes

KyleAMathews changed the title ~~Plugins~~ Plugin search page Jan 10, 2018

Removed package-lock.json and renamed plugins.js to packages.js

d893895

KyleAMathews changed the title ~~Plugin search page~~ Gatsby library search page Jan 11, 2018

jastack added 6 commits January 11, 2018 16:39

Add algolia-npm search and url syncing

9d33f44

Add background color when plugin selected

5c42e5a

Fix doubling-up layout bug, add border to hits component, add margin …

39ce0cb

…to text in packages

Merge branch 'master' into plugins

c5509d6

Remove unnecessary dependencies from package.json

77fa8c6

Merge branch 'plugins' of https://github.com/jastack/gatsby into plugins

0ede9c9

KyleAMathews reviewed Jan 13, 2018

View reviewed changes

calcsam reviewed Jan 13, 2018

View reviewed changes

jastack added 15 commits January 26, 2018 14:18

Remove withUrlSync.js

de22f28

Change Algolia link from Link to a tag

45a0d2f

Add email-validator to package.json to merge with updated site

9e147ca

Add source plugin to pull in npm package info, add metadata to cards …

c6d8e7b

…and package detail page

Remove results display when no results

2bb8d13

Remove multiple div

ce4878f

Remove startes and filters from packages description

96c4839

Update searchbar styles

a6ce75b

Remove searchbar scroll

9224b8e

Update packages.js

a0987d0

Empty commit

dd20c4c

Update template-docs to account for both local and remote packages

9976291

Remove duplicate code

8d32524

Fix merge conflicts

99da0b6

Remove commented out code

cb316ca

gillkyle mentioned this pull request Feb 8, 2018

Plugin Library build fixes #3906

Merged

KyleAMathews closed this Feb 22, 2018

shannonbux added this to Done in UX Project Feb 28, 2018

shannonbux added the UX Design label Feb 28, 2018

fk mentioned this pull request Mar 13, 2018

[WIP][www] Replace main navigation item "Community" with "Plugins" #4472

Merged


		import typography from "../utils/typography"

		function Search(){

Gatsby library search page #3456

Gatsby library search page #3456

Conversation

jastack commented Jan 9, 2018

gatsbybot commented Jan 9, 2018 • edited

KyleAMathews commented Jan 10, 2018

calcsam commented Jan 10, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

KyleAMathews commented Jan 10, 2018

KyleAMathews commented Jan 10, 2018

calcsam commented Jan 10, 2018

calcsam commented Jan 10, 2018

calcsam commented Jan 10, 2018

jastack commented Jan 10, 2018

shannonbux commented Jan 10, 2018 • edited

KyleAMathews commented Jan 10, 2018

rdela commented Jan 11, 2018

jastack commented Jan 12, 2018

KyleAMathews commented Jan 13, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gillkyle commented Feb 7, 2018

calcsam commented Feb 7, 2018

KyleAMathews commented Feb 7, 2018

KyleAMathews commented Feb 7, 2018

gillkyle commented Feb 7, 2018

fk commented Feb 22, 2018

fk commented Feb 22, 2018

shannonbux commented Feb 22, 2018 via email

KyleAMathews commented Feb 22, 2018

KyleAMathews commented Feb 22, 2018

gatsbybot commented Jan 9, 2018 •

edited

shannonbux commented Jan 10, 2018 •

edited