Use Peter Novig spell corrector for similar_names #300
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
gyin94
reviewed
Feb 1, 2024
| """Use Peter Novig's spell correcter at https://norvig.com/spell-correct.html""" | ||
| word_count = Counter([_ for _ in candidates]) | ||
|
|
||
| def P(word, N=sum(word_count.values())): |
There was a problem hiding this comment.
consider a type hint and return type?
There was a problem hiding this comment.
I agree that we should add the type hints if we merge this pull request; however, after more thoughts, I am no longer sure that we should merge Peter's algorithm.
This algorithm expands the misspelled word to a small set of similar words, and filter out those not in the vocabulary. This is because the vocabulary is too large to compute the editing distance between each word in it and the misspelled word.
However, a Python class wouldn't have this many candidate symbols as in a vocabulary. So it may be affordable to compute the pair-wise editing distance. What do you think?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Guoli Yin pointed me to this code snippet for field name correction. I instantly thought of Peter Novig's spell checker, which is available at https://norvig.com/spell-check.html. This pull request makes advantage of Peter Novig's technique for name suggestions.
I'm preserving the original function name,'similar_names'. However, the new algorithm only suggests one name. I think this makes more sense than suggesting multiple choices because a class's collection of properties is typically far smaller than the English vocabulary, and users may prefer one accurate correction when they misspell.
Please let me know if I mis-interpret the problem here. Thanks!