Minimize lodash dependency

[exoego]

Use case description

#7680 (comment)

lodash offers lots of utility, but nowadays we only need the small parts of lodash, since recent Node.js supports native convenient counterpart (e.g. Array#forEach(f) over _.forEach(array, f)).

Minimizing lodash dependency will achieve

• Faster runtime speed by native implementation
• Smaller bundle size of serverless framework
• Friendly for developers who are more familiar with standard JS, not lodash.

Proposed solution

• Use convenient native counterpart as much as possible
• Depend only some part of lodash, e.g. lodash-memoize and so on.

Note: Ideally if we handle all below cases, and each is addressed with separate PR

• Replace _.assign with Object.assign (Addressed with Replace _.assign and _.extend with Object.assign #7766)
• Refactor _.chain to flow constructed with native functions (Addressed with Replace lodash chain with native array chain #7862)
• Replace _.compact with array.filter(Boolean) (Addressed with replaced lodash compact with array.filter(Boolean) #7858)
• Replace _.concat with array.concat (Addressed with Replace _.concat with array.concat #7851)
• Replace _.each(Array) with array.forEach for array cases (Addressed with Replace lodash forEach(array) with native Array.forEach #7748)
• Replace _.each(object) with Object.entries(object).forEach (Renamed to _.forEach with Replace lodash forEach(array) with native Array.forEach #7748 and addressed with chore: replace _.forEach with Object.entries().forEach #8280)
• Replace _.entries with Object.entries (Addressed with chore: replace _.entries, _.entriesIn and _.toPairs with Object.entries #8275)
• Replace _.entriesIn with Object.entries, note both are not equivalent, we need to confirm that conversion is safe (Addressed with chore: replace _.entries, _.entriesIn and _.toPairs with Object.entries #8275)
• Replace _.every with array.every (Addressed with Replace lodash _.every(array) with native Array.every #7764)
• Replace _.extend with Object.assign Note they're not equivalent, but most likely Object.assign can be used safely in most of the cases (Addressed with Replace _.assign and _.extend with Object.assign #7766)
• Replace _.filter with array.filter (Addressed with refactor(lodash): Replace _.filter with array.filter #7775)
• Replace _.find with array.find (Addressed with Replace lodash _.find with native Array.find #7782)
• Replace _.findKey with Object.keys(object).find (Addressed with Replace _.findKey with Object.keys(object).find #7881)
• Replace _.first with array[0] (Addressed with Replace lodash _.first with native array[0] #7816)
• Replace _.flat with array.flat (Addressed with refactor: Replace _.flatten with Array.prototype.flat #11271)
• Replace _.flatMap with array.flatMap (Addressed with refactor: Replace _.flatMap with Array.prototype.flatMap #11272)
• Replace _.forEach(Array) with array.forEach for array cases (Addressed with Replace lodash forEach(array) with native Array.forEach #7748)
• Replace _.forEach(object) with Object.entries(object).forEach (Addressed with chore: replace _.forEach with Object.entries().forEach #8280)
• Replace _.forOwn with Object.entries().forEach (Addressed with chore: replace _.forOwn with Object.entries().forEach #8284)
• Replace _.get with normal obj.prop checks, but only in top level property checks or where given property path is ensured to exist
• Replace _.has with normal obj.prop checks, in top level property checks, or where we're safe from Undefined is not an object errors (Addressed with Replaced _.has(object, property) with object.property #7915)
• Replace _.head with array[0] (Addressed with Replace lodash _.head with array[0] #7817)
• Replace _.includes with either string.includes or array.includes (Addressed with Replace lodash _.includes with native includes #7818)
• Replace _.indexOf with either string.includes, array.includes or string.indexOf, array.indexOf (Addressed with Replace _.indexOf with native includes #7825)
• Remove _.isBoolean, we should just check whether value is truthy or falsy, no need to confirm on boolean type (Addressed with Remove is boolean #7880)
• Replace _.isEmpty with array.length (Addressed with replace _.isEmpty with array.length and Object.keys() #7873)
• Remove _.isInteger, in most cases it's used validation is superfluous (so this check can be removed entirely) (Addressed with Remove _.isInteger #7878)
• Replace _.isFunction with typeof value === 'function' (Addressed with refactor: Replace _.isFunction(value) with typeof value === 'function' #7810)
• Replace _.isNil with value == null (Addressed with refactor: Replace _.isNil(value) with value == null #7809)
• Replace _.isString with typeof value === 'string' (Addressed with refactor: Replace _.isString(value) with typeof value === 'string' #7812)
• Replace _.isUndefined with value == null (Addressed with Replace _.isUndefined with native undefined check #7826)
• Replace _.join with array.join (Addressed with Replace lodash _.join with native array.join #7805)
• Replace _.keyBy(collection) with simple set collection (Addressed with Replace _.keyBy with array.reduce #7882)
• Replace _.keys with Object.keys (Addressed with Replace lodash _.keys with native Object.keys #7784)
• Replace _.map with array.map (Addressed with Replace _.map with array.map #7827)
• Replace _merge with Object.assign. Note they're not equivalent (_.merge does deep merge), so it should be done with care in all places where it's safe to rely on Object.assign instead (majority of cases)
• Replace _.min with Math.min(...array) (Addressed with Replace _.min with Math.min #7840)
• Replace _.nth with arr[index] (Addressed with Replace _.nth with array[index] #7841)
• Replace _.parseInt with Number (Addressed with Replace _.parseInt with Number #7877)
• Replace _.pullAllWith with array.filter (Addressed with Replaced lodash pullAllWith with Array.filter() #7861)
• Replace _.reduce with array.reduce (in this case I'd also investigate if reduce is right pattern used) (Addressed with Replace _.reduce with array.reduce #7883)
• Replace _.repeat with string.repeat (Addressed with Replace _.repeat with string.repeat #7842)
• Replace _.replace with string.replace (Addressed with Replace _.replace with string.replace #7843)
• Replace _.set with normal obj.prop = value assignments, but only in top level property checks or where given property path is ensured to exist
• Replace _.size with array.length check on arrays and `Object.keys(object).length on objects (Addressed with Replace lodash _.size with native array length & object.keys length #7798)
• Replace _.some with array.some (Addressed with chore: reduce _.some usage #7901)
• Replace _.sortBy with array.sort (Addressed with Replace _.sortBy with array.sort #7823)
• Replace _.split with string.split (Addressed with Replace _.split with string.split #7820)
• Replace _.takeRight with array.slice (Addressed with refactor: Replace _.takeRight with array.slice #7831)
• Replace _.toPairs with Object.entries (Addressed with chore: replace _.entries, _.entriesIn and _.toPairs with Object.entries #8275)
• Replace _.toString with String(value) Note: behavior is not equivalent for null and undefined. We need to be sure to handle cases properly (Addressed with Replace toString #7893)
• Replace _.toUpper with string.toUpper (Addressed with refactor: Replace _.toUpper(string) with string.toUpperCase #7808)
• Replace _.unset with delete obj.prop (Addressed with Replace lodash unset with native delete objectProp #7813)
• Replace _.values with Object.values (Addressed with chore: replace usage of _.values with Object.values #8274)

[medikoo]

@exoego great thanks for putting that into issue.

I've updated description with list of all functions which I believe we can replace. I think best if each is handled with separate PR's, so we process it in small steps (change touches sensitive parts, so we need to be careful)

I've also added good first issue and help wanted labels as it appears there's a lot to cover

[chrisVillanueva]

Some news about lodash. There's a security issue noted in this issue:

https://github.com/lodash/lodash/issues/4738

The current version fails 3rd party security scans.

[chrisVillanueva]

@medikoo
Is there a current file or sub-directory we can place utility functions? Where can we add a function to execute deep and/or strict equality Object checks? I looked at /utils and /lib/classes/Utils.js. I didn't see a place to introduce a general, shareable utility function. Utils.js defines a class, but I only need a simple function to use across files. Any suggestions?

[medikoo]

Is there a current file or sub-directory we can place utility functions?

@chrisVillanueva if you mean a language level utilities, then I think we should not introduce any here, but instead rely of utility library of choice. Currently we rely on lodash, and problem is that we overuse it for cases which should rather be handled with native API, and this issue is purely about that.

We may at some point consider replacing lodash completely, but I think then it should replaced with other utility library. We definitely should not attempt to build and maintain lang utilities in scope of this project.

[chrisVillanueva]

Thanks for clarification. This simplifies the issue.

[G-Rath]

Looks like _.reduce and _.forOwn are done, but not checked off :)

Also if you've got a build tool like TS or Babel, you could use optional chaining to replace most of the _.get usages (or something like ts-optchain to replace all the usages).

[medikoo]

Looks like _.reduce and _.forOwn are done, but not checked off :)

@G-Rath thanks for pointing. Updated description

Also if you've got a build tool like TS or Babel, you could use optional chaining to replace most of the _.get usages (or something like ts-optchain to replace all the usages).

Once support for Node.js v12 is dropped we will simply move to native syntax. We do not plan to introduce any transpilation step into this package, as it comes with significant maintenance cost

[pjmattingly]

I'd like to work on:

Replace _.merge with Object.assign. Note they're not equivalent (_.merge does deep merge), so it should be done with care in all places where it's safe to rely on Object.assign instead (majority of cases)

If that would be okay. It looks like it's still up for grabs.

[pgrzesik]

Sure thing, that would be awesome @pjmattingly 👍

[medikoo]

I've added also _.flat and _.flatMap to the list, as those can be replaced now since we dropped support for Node.js v10

[pjmattingly]

I'd like to take Replace _.flat with array.flat; I'm guessing this should be _.flatten?

Submitted PR: #11271

[pjmattingly]

Next, on to: Replace _.flatMap with array.flatMap

PR: #11272

[pjmattingly]

Next, Replace _.set with normal obj.prop = value assignments, but only in top level property checks or where given property path is ensured to exist

Edit: did my best! #11273

[pjmattingly]

Then, the last one: Replace _.get with normal obj.prop checks, but only in top level property checks or where given property path is ensured to exist

[sleepwithcoffee]

hi @medikoo what about the case with _.isObject(obj)? I see there is a mix between lodash and using const isObject = require('type/object/is')

[medikoo]

hi @medikoo what about the case with _.isObject(obj)? I see there is a mix between lodash and using const isObject = require('type/object/is')

@duytrandev I'd welcome PR that switches to type/object/is, still it's that's not really a scope here. Point of this issue is get rid of lodash in cases where native counterpart can be used, and we don't have plans to get rid of it completely (at least not at this point)