Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Project as database for create/retrain/finetune #9135

Merged
merged 7 commits into from May 17, 2024
Merged

Project as database for create/retrain/finetune #9135

merged 7 commits into from May 17, 2024

Conversation

ea-rus
Copy link
Contributor

@ea-rus ea-rus commented Apr 25, 2024

Description

Changes:

  • Model can query sql in project to get data to model. It means that user can use project name as database
    • CREATE MODEL model FROM mindsdb (SELECT * FROM example_db.home_rentals) PREDICT rental_price;
  • integration name is optional for create/retrain/finetune. It is possible to create model without database specified. Current project will be used
    • CREATE MODEL model FROM (SELECT * FROM example_db.home_rentals) PREDICT rental_price;
  • It is also possible to use LAST in these commands:
retrain home_rentals_model1 from (
  select * from my_pg.home_rentals where id>last
) 

Side feature: second context for conditional job
It will allow to run job like this, when we have LAST in both conditional and execution blocks

create job j1 (
    FINETUNE MODEL home_rentals_model
    FROM (     
        select * from my_pg.home_rentals where id>last
    )
)
every 1 min
if (
    select * from my_pg.home_rentals where id>last
);

It creates second 'job-if' context to not change 'job' context and not hide new data from job.

Dependent on mindsdb/mindsdb_sql#373

Please include a summary of the change and the issue it solves.

Fixes #9051

Type of change

(Please delete options that are not relevant)

  • 🐛 Bug fix (non-breaking change which fixes an issue)
  • ⚡ New feature (non-breaking change which adds functionality)
  • 📢 Breaking change (fix or feature that would cause existing functionality not to work as expected)
  • 📄 This change requires a documentation update

Verification Process

To ensure the changes are working as expected:

  • Test Location: Specify the URL or path for testing.
  • Verification Steps: Outline the steps or queries needed to validate the change. Include any data, configurations, or actions required to reproduce or see the new functionality.

Additional Media:

  • I have attached a brief loom video or screenshots showcasing the new functionality or change.

Checklist:

  • My code follows the style guidelines(PEP 8) of MindsDB.
  • I have appropriately commented on my code, especially in complex areas.
  • Necessary documentation updates are either made or tracked in issues.
  • Relevant unit and integration tests are updated or added.

@ea-rus ea-rus requested a review from StpMax April 25, 2024 18:01
# Conflicts:
#	mindsdb/api/executor/command_executor.py
@@ -61,11 +61,9 @@ def learn_process(data_integration_ref: dict, problem_definition: dict, fetch_da
)
)
sqlquery = SQLQuery(query, session=sql_session)
elif data_integration_ref['type'] == 'view':
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe keep this for back compatibility?

Copy link
Contributor Author

@ea-rus ea-rus May 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It won't be used because it is removed here

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will break existing 'predictors' with data_integration_ref->>'type' = 'view'

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, returned 'type=view' block

@ea-rus ea-rus merged commit e62634a into main May 17, 2024
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug]: Cannot use query with LAST when fine-tuning a model
2 participants