Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid regenerating embeddings at dump import #4606

Open
8 tasks
curquiza opened this issue Apr 30, 2024 · 0 comments · May be fixed by #4633
Open
8 tasks

Avoid regenerating embeddings at dump import #4606

curquiza opened this issue Apr 30, 2024 · 0 comments · May be fixed by #4633
Labels
experimental feature Related to an experimental feature impacts docs This issue involves changes in the Meilisearch's documentation impacts integrations This issue involves changes in the Meilisearch's integrations missing usage in PRD Description of the feature usage is missing in the PRD performance Related to the performance in term of search/indexation speed or RAM/CPU/Disk consumption
Milestone

Comments

@curquiza
Copy link
Member

curquiza commented Apr 30, 2024

Related product team resources: PRD (internal only)
Related product discussion: https://github.com/orgs/meilisearch/discussions/677

Motivation

Improve AI search performance

Usage

Avoid regenerating embeddings at dump import

https://meilisearch.notion.site/v1-9-AI-search-changes-e90d6803eca8417aa70a1ac5d0225697?pvs=74

TODO

Reminders when modifying the Setting API

  • Ensure the new setting route is at least tested by the test_setting_routes macro
  • Ensure Analytics are fully implemented
  • Ensure the dump serializing is consistent with the /settings route serializing, e.g., enums case can be different (camelCase in route and PascalCase in the dump)

Special cases when adding a setting for an experimental feature

  • ⚠️ API stability: The setting does not appear on the main settings route when the feature has never been enabled (e.g. mark it Unset when returned from the index in this situation. See an example)
  • The setting cannot be set when the feature is disabled, either by the main settings route or the subroute (see validate_settings function)
  • If possible, the setting is reset when the feature is disabled (hard if it requires reindexing)

Impacted teams

@meilisearch/docs-team @meilisearch/integration-team

@curquiza curquiza added impacts docs This issue involves changes in the Meilisearch's documentation impacts integrations This issue involves changes in the Meilisearch's integrations missing usage in PRD Description of the feature usage is missing in the PRD experimental feature Related to an experimental feature labels Apr 30, 2024
@curquiza curquiza added this to the v1.9.0 milestone Apr 30, 2024
@curquiza curquiza changed the title AI search: Avoid regenerating embeddings at dump import Avoid regenerating embeddings at dump import Apr 30, 2024
@curquiza curquiza added the performance Related to the performance in term of search/indexation speed or RAM/CPU/Disk consumption label Apr 30, 2024
@dureuill dureuill linked a pull request May 14, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
experimental feature Related to an experimental feature impacts docs This issue involves changes in the Meilisearch's documentation impacts integrations This issue involves changes in the Meilisearch's integrations missing usage in PRD Description of the feature usage is missing in the PRD performance Related to the performance in term of search/indexation speed or RAM/CPU/Disk consumption
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant