☔ Redesign and rework on Gatsby

[veksen]

Me and aerori have spent quite a bit of time working on an iteration of my last work, which can be seen in #140.

Aside from purely design, I would start an effort to build this on top of Gatsby. I have a small army of people that have voiced their interest to help. This could solve #143, by making it possible to index pages (avoid using anchors, and use an actual route/url).

Also, this could make it possible to incorporate the unofficial FP docs: https://github.com/jfmengels/lodash-fp-docs see: #192.

Original work started with a light theme, and then forked to a dark theme, to then completely abandon the light. I'm still not 100% happy with the way the information of the docs is presented. Somewhere from argument/return type feels a little weird to read.

I was not involved with the generation of the docs, so if I could get a little insight, it would help me a lot. How is it generated? In the best of worlds, I would use markdown data, for Gatsby, split by method, instead of the actual HTML we have now.

I can setup a temporary Netlify site while this is being implemented. I feel like this will move a lot quicker with actual development, leading to fruitful conversations, rather than continuing in Figma.

deployed here: https://lodash-gatsby.netlify.com/

Website:

Docs:

Current progress:
repo: https://github.com/veksen/lodash.com/tree/gatsby
deployed on netlify: https://lodash-gatsby.netlify.com/

Rough draft of TODOs.

TODO general:

• Extract colors/globals
• Performance benchmarks
• UX benchmarks
• Accessibility benchmarks (aria, and colors)
• General code review
• Possibly go in beta before a full launch
• Go over TODOs left in the code

TODO home:

• Mobile
• Rework hierarchy of the emphasized content:

Lodash v5 will stress ECMAScript modules (ESM) and Node use so browser examples will be secondary. The core build, full build, and custom builds will be deemphasized as well, leaving just the CDN link which can be reworded.

TODO docs:

• Mobile
• Improve types (mainly around static frontmatter groups/methods
• Debug performance issue of rendering 295 elements (React-virtualized/React-window?)
• Support aliases
• Link methods to npm/source
• Attach to the correct versions of Lodash (version selector)
• Support clicking outside of version selector
• Support generating docs other versions of Lodash
• Double-check support for prototype methods, like _.prototype[Symbol.iterator], or _.chain (versus chain _.prototype.chain)
• Support arrow navigation from input
• Support focusing input (/)

[zackhall]

I was not involved with the generation of the docs, so if I could get a little insight, it would help me a lot. How is it generated? In the best of worlds, I would use markdown data, for Gatsby, split by method, instead of the actual HTML we have now.

If I recall correctly, the docs are autogenerated from the comments in the lodash code. So those were either used as the source for the jekyll content or there is some transform that happens on those to place them as the jekyll content. I'll try to dig a little deeper.

For reference, here's the original commit for the last lodash site redesign: 0694e74

I think gatsby for the site re-launch would be perfect. I'd be happy to help contribute some hours over the next couple of weeks to build this out. I know you said you'd had a small army of developers to help launch this. Perhaps if y'all wanted to get a branch going we could collaborate on this.

[zackhall]

I was trying to track this down through the commits, but the doc generation steps were added in the README. 🤦

Anyhow, it's done through an npm script on the lodash package.

...
"doc:site":` "node lib/main/build-doc site",
...

Having said that, I don't know if it's been updated since. I'm having trouble tracking down the lib/main/build-doc script or any of the scripts in lib/.

[bnjmnt4n]

For reference: previously docdown was used to generate a markdown file from the comments in the monolithic lodash.js file. Now that lodash is switching to a modular system, that opens up the possibility of using a different documentation generation system or modifying docdown to accept multiple files when generating the docs.

Edit: link to relevant code: https://github.com/lodash/lodash/tree/4.17.12-pre/lib/main.

[veksen]

Markdown is still possible, even prefered by Gatsby. Multiple or single file doesn't make much of a difference.

[veksen]

RE: added the current progress in the original post:

repo: https://github.com/veksen/lodash.com/tree/gatsby
deployed on netlify: https://lodash-gatsby.netlify.com/

Performance is really good as of right now - considering almost a barebone Gatsby install from the default starter. I'll be curious to explore what a starter like https://www.gatsbyjs.org/starters/fabe/gatsby-universal/ uses to further optimize.

But not exactly as good as the previous site:

(top is new, bottom is old)

[veksen]

I've got more progress on the site (see https://lodash-gatsby.netlify.com/docs), but I'm at the point where I need to generate markdown docs from JSDoc. Most of the magic happens in https://github.com/jdalton/docdown (hello @jdalton 👋), which I wasn't aware was a thing, haha. I would need to fork docdown to output some semantic, object-like data and no html.

[veksen]

Does @jdalton have a stand on merging the unofficial FP docs from @mathiasbynens (https://github.com/jfmengels/lodash-fp-docs)?

[jdalton]

@veksen

I've got more progress on the site (see https://lodash-gatsby.netlify.com/docs), but I'm at the point where I need to generate markdown docs from JSDoc.

This is so nice!

Does @jdalton have a stand on merging the unofficial FP docs from @mathiasbynens (https://github.com/jfmengels/lodash-fp-docs)?

We've gone this long without it. I'm fine continuing to punt on it.

[zackhall]

(First, sorry for the "pin." I thought that would pin it to my dashboard, not pin it globally.)

But with the npm run doc:site script deprecated, how're you currently building the documentation file today @jdalton?

I was able to generate the docs by building lodash from the CLI and reviving the old script (from @bnjmnt4n's link above) with mini-modifications. But I'm curious what the path you're currently following is because I can't find that in the documentation.

And @veksen – for the gatsby site, what is the desired format of the doc output? Do you want one giant markdown file (with no HTML)? Or do you want one markdown file for each method?

[zackhall]

Yo, @veksen – this animation right here is 👌! Love that the fade shows its scrollable but it disappears on hover.

---

Aside from that, was looking at docdown today to see what we would want to do to export this in a format that's helpful for gatsby.

So we'll want to extend it to accept multiple files, since lodash is modular now. And if it's exporting multiple files, then the new site can have individual pages for each method (e.g. at /docs/chunk for the chunk method) in addition to the full documentation (at /docs).

And then as far as the actual output of each file:

• We wouldn't need the Table of Contents (ToC) since it's one method.
• We can add important metadata to the YAML frontmatter so that it's queryable in the GraphQL. E.g. something like:

graphql`
  query {
    ...
    node {
      frontmatter {
        methodName
        description
        arguments
        example
        ...etc
      }
    }
  }
`

---

⚠One question, @veksen : do we want the body each file to still include the markdown (like below)? Or do we want each file simply to include all of this data in the frontmatter so that you can build the <Method /> component from the queryable data?

### _.chunk(array, [size=1])

[source](https://github.com/lodash/lodash/blob/5.0.0/lodash.js#L6831) [npm package](https://www.npmjs.com/package/lodash.chunk)

Creates an array of elements split into groups the length of `size`.
If `array` can't be split evenly, the final chunk will be the remaining
elements.

#### Since
3.0.0

#### Arguments
1. `array` *(Array)*: The array to process.
2. `[size=1]` *(number)*: The length of each chunk

#### Returns
*(Array)*: Returns the new array of chunks.

#### Example
```js
_.chunk(['a', 'b', 'c', 'd'], 2);
// => [['a', 'b'], ['c', 'd']]

_.chunk(['a', 'b', 'c', 'd'], 3);
// => [['a', 'b', 'c'], ['d']]

---

But I'll get started on outputting a) to multiple files, and b) in the markdown with YAML frontmatter format. I'll try to send out a pull request in a couple days.

[veksen]

@zackhall I would prefer frontmatter data rather than markdown preformatted. It will be so much easier to navigate, and the data can easily be used for fuzzy-searching, as well.

[jdalton]

@zackhall

But with the npm run doc:site script deprecated, how're you currently building the documentation file today?

I use the 4.17.12-pre branch.

[zackhall]

Ohhhhhhh! That's the missing puzzle piece we'd been looking for, @jdalton.

[zackhall]

Alright, team. I've got it working so that all of the methods' metadata is queryable in gatsby using graphql. As we had no need for the markdown, I built a new gatsby source plugin instead which re-uses the lib of docdown to grab the methods from the lodash source.

Now all of the methods can be grabbed inside Gatsby from a GraphQL query like the following:

{
  allLodashMethod(limit: 20) {
    group(field:category) {
      field
      fieldValue
      totalCount
      edges {
        node {
          name
          category
          aliases
          params
          call
        }
      } 
    }
  }
}

And this will return:

{
  "data": {
    "allLodashMethod": {
      "group": [
        {
          "field": "category",
          "fieldValue": "Array",
          "totalCount": 59,
          "edges": [
            {
              "node": {
                "name": "compact",
                "category": "Array",
                "aliases": [],
                "params": [
                  [
                    "Array",
                    "array",
                    "The array to compact."
                  ]
                ],
                "call": "compact(array)"
              }
            },
            {
              "node": {
                "name": "concat",
                "category": "Array",
                "aliases": [],
                "params": [
                  [
                    "Array",
                    "array",
                    "The array to concatenate."
                  ],
                  [
                    "...*",
                    "[values]",
                    "The values to concatenate."
                  ]
                ],
                "call": "concat(array, [values])"
              }
            },
...

@veksen – I'll do a little bit of cleanup, but should be ready to submit a pull request later tonight. Do you want to move your gatsby branch upstream into the lodash.com repo so that we can work off of it there?

[veksen]

Awesome! I've pushed the branch on the Lodash.com repo for easier contributions. I've had a Netlify deployment set up, would it be possible to get this too? Deploy command is npm run build, and dir is /public

[veksen]

@zackhall should the params be an array of objects? We can discuss this in the PR if you wish

[zackhall]

Good callout on the params, @veksen. Added some handling on-top of the docdown lib to make those objects rather than arrays.

Can you help me understand what you're asking in this question?

I've had a Netlify deployment set up, would it be possible to get this too? Deploy command is npm run build, and dir is /public

[veksen]

I'm asking if we can attach the current branch (Gatsby) to deploy with Netlify

[zackhall]

I believe so. I think if you go into the deploy settings you can see a "branch" option for the deploy. For now, the branch could be gatsby, no?

[MathRobin]

Hi ! Thanks for your work, it's enormous. Just a point about the dark theme, I know for ordinary people, it's the more confortable way to display things on screen. Can we imagine to have the possibility to switch to a basic light theme (even if it's not as complete as the dark one). For color blind people like me, dark themes are painful, providing daily headaches :s

Again : it's just a small point comparing to big amount of work you provided, thanks to you again !

[veksen]

@MathRobin It was definitely in the back on our head, but it further makes sense now that you bring up color blindness. Thank you!

[aerori]

@MathRobin - We originally had a light theme, but it was thrown out instead of being improved upon. Shoot me an email or a ping on discord and tell me what you can and can't see, and I'll do something about it. I'm going to try this on my dad as well- he's colorblind.

@veksen I have secretly been working on revamping the light theme. Let's bring it back! :^)

[MathRobin]

Probably will PR a switch button to change theme

[veksen]

Sure! We're using Context and it would make sense to use Context for this as well. Useful read: https://markoskon.com/dark-mode-in-react/

[MathRobin]

What is Context ? Is it something from react ecosystem ? I'm not really a big fan of react, mainly because of JSX, so if it's react, will be more complicated to make a PR, because don't know it.