Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Handle words with common prefix #6

Open
wengxt opened this issue May 26, 2020 · 1 comment
Open

Handle words with common prefix #6

wengxt opened this issue May 26, 2020 · 1 comment

Comments

@wengxt
Copy link

wengxt commented May 26, 2020 •
edited

Certain words has common prefix, this is due to the natural of Wikipedia.

图片

I suggest at least use it as a hint to split the word into multiple words, and reduce the number of "non-word" phrase.
@felixonmars
Copy link
Owner

I think adding a blacklist of suffix might be a way for this, like we already filtered out "列表" in #3. we can add "登场人物" too.

felixonmars added a commit that referenced this issue May 26, 2020
@felixonmars
Attempt to filter out words with long common suffix that's already in…
83c721e 
… the dict (#6)
anaer pushed a commit to anaer/rime-pinyin-zhwiki that referenced this issue May 18, 2023
@felixonmars
Attempt to filter out words with long common suffix that's already in…
b28978a 
… the dict (felixonmars#6)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@wengxt @felixonmars