Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Introspection] Cross Schema Foreign Keys #1175

Closed
janpio opened this issue Dec 17, 2019 · 59 comments
Closed

[Introspection] Cross Schema Foreign Keys #1175

janpio opened this issue Dec 17, 2019 · 59 comments

Comments

@janpio
Copy link
Member

janpio commented Dec 17, 2019

relational.fit.cvut.cz DB server has 2 schemas ctu_crossSchema1 and ctu_crossSchema2 that reference each other in foreign keys:

Error messages during introspection:

λ prisma2 introspect --url="mysql://guest:relational@relational.fit.cvut.cz:3306/ctu_crossSchema1"
Introspecting …
setting lastError
Error: Error in introspection engine: {"message":"[src\\libcore\\result.rs:1165:5] called `Result::unwrap()` on an `Err` value: \"Table district not found\"","backtrace":"stack backtrace:\n   0: backtrace::backtrace::trace\n   1: backtrace::capture::Backtrace::new\n   2: user_facing_errors::UnknownError::new_in_panic_hook\n   3: user_facing_errors::panic_hook::set_panic_hook::{{closure}}\n   4: std::panicking::rust_panic_with_hook\n             at src\\libstd/panicking.rs:477\n   5: std::panicking::continue_panic_fmt\n             at src\\libstd/panicking.rs:380\n   6: rust_begin_unwind\n             at src\\libstd/panicking.rs:307\n   7: core::panicking::panic_fmt\n             at src\\libcore/panicking.rs:85\n   8: core::result::unwrap_failed\n             at src\\libcore/result.rs:1165\n   9: sql_schema_describer::SqlSchema::table_bang\n  10: sql_introspection_connector::calculate_datamodel::calculate_field_type\n  11: sql_introspection_connector::calculate_datamodel::calculate_model\n  12: <std::future::GenFuture<T> as core::future::future::Future>::poll\n  13: <std::future::GenFuture<T> as core::future::future::Future>::poll\n  14: tokio::runtime::enter::Enter::block_on\n  15: std::thread::local::LocalKey<T>::with\n  16: std::thread::local::LocalKey<T>::with\n  17: tokio::runtime::spawner::Spawner::enter\n  18: tokio::runtime::time::variant::with_default\n  19: std::thread::local::LocalKey<T>::with\n  20: <introspection_engine::rpc::RpcImpl as introspection_engine::rpc::rpc_impl_Rpc::gen_server::Rpc>::introspect\n  21: <jsonrpc_core::delegates::DelegateAsyncMethod<T,F> as jsonrpc_core::calls::RpcMethod<M>>::call\n  22: <futures::future::lazy::Lazy<F,R> as futures::future::Future>::poll\n  23: futures::future::chain::Chain<A,B,C>::poll\n  24: <futures::future::then::Then<A,B,F> as futures::future::Future>::poll\n  25: <futures::future::either::Either<A,B> as futures::future::Future>::poll\n  26: <futures::future::map::Map<A,F> as futures::future::Future>::poll\n  27: <futures::future::either::Either<A,B> as futures::future::Future>::poll\n  28: <futures::future::map::Map<A,F> as futures::future::Future>::poll\n  29: <futures::future::map_err::MapErr<A,F> as futures::future::Future>::poll\n  30: <futures::stream::and_then::AndThen<S,F,U> as futures::stream::Stream>::poll\n  31: <futures::stream::forward::Forward<T,U> as futures::future::Future>::poll\n  32: <futures::future::map::Map<A,F> as futures::future::Future>::poll\n  33: <futures::future::map_err::MapErr<A,F> as futures::future::Future>::poll\n  34: futures::task_impl::std::set\n  35: <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once\n  36: std::panicking::try::do_call\n  37: __rust_maybe_catch_panic\n             at src\\libpanic_unwind/lib.rs:80\n  38: tokio_threadpool::task::Task::run\n  39: tokio_threadpool::worker::Worker::run_task\n  40: tokio_threadpool::worker::Worker::run\n  41: tokio_timer::clock::clock::with_default\n  42: tokio::runtime::threadpool::builder::Builder::build::{{closure}}\n  43: std::thread::local::LocalKey<T>::with\n  44: std::thread::local::LocalKey<T>::with\n  45: std::sys_common::backtrace::__rust_begin_short_backtrace\n  46: std::panicking::try::do_call\n  47: __rust_maybe_catch_panic\n             at src\\libpanic_unwind/lib.rs:80\n  48: core::ops::function::FnOnce::call_once{{vtable.shim}}\n  49: <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once\n             at /rustc/4560ea788cb760f0a34127156c78e2552949f734\\src\\liballoc/boxed.rs:922\n  50: <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once\n             at /rustc/4560ea788cb760f0a34127156c78e2552949f734\\src\\liballoc/boxed.rs:922\n      std::sys_common::thread::start_thread\n             at src\\libstd\\sys_common/thread.rs:13\n      std::sys::windows::thread::Thread::new::thread_start\n
             at src\\libstd\\sys\\windows/thread.rs:47\n  51: sqlite3GenerateConstraintChecks\n  52: sqlite3GenerateConstraintChecks\n"}
λ prisma2 introspect --url="mysql://guest:relational@relational.fit.cvut.cz:3306/ctu_crossSchema2"
Introspecting …
setting lastError
Error: Error in introspection engine: {"message":"[src\\libcore\\result.rs:1165:5] called `Result::unwrap()` on an `Err` value: \"Table account not found\"","backtrace":"stack backtrace:\n   0: backtrace::backtrace::trace\n   1: backtrace::capture::Backtrace::new\n   2: user_facing_errors::UnknownError::new_in_panic_hook\n   3: user_facing_errors::panic_hook::set_panic_hook::{{closure}}\n   4: std::panicking::rust_panic_with_hook\n             at src\\libstd/panicking.rs:477\n   5: std::panicking::continue_panic_fmt\n             at src\\libstd/panicking.rs:380\n   6: rust_begin_unwind\n             at src\\libstd/panicking.rs:307\n   7: core::panicking::panic_fmt\n             at src\\libcore/panicking.rs:85\n   8: core::result::unwrap_failed\n             at src\\libcore/result.rs:1165\n   9: sql_schema_describer::SqlSchema::table_bang\n  10: sql_introspection_connector::calculate_datamodel::calculate_field_type\n  11: sql_introspection_connector::calculate_datamodel::calculate_model\n  12: <std::future::GenFuture<T> as core::future::future::Future>::poll\n
  13: <std::future::GenFuture<T> as core::future::future::Future>::poll\n  14: tokio::runtime::enter::Enter::block_on\n  15: std::thread::local::LocalKey<T>::with\n  16: std::thread::local::LocalKey<T>::with\n  17: tokio::runtime::spawner::Spawner::enter\n  18: tokio::runtime::time::variant::with_default\n  19: std::thread::local::LocalKey<T>::with\n  20: <introspection_engine::rpc::RpcImpl as introspection_engine::rpc::rpc_impl_Rpc::gen_server::Rpc>::introspect\n  21: <jsonrpc_core::delegates::DelegateAsyncMethod<T,F> as jsonrpc_core::calls::RpcMethod<M>>::call\n  22: <futures::future::lazy::Lazy<F,R> as futures::future::Future>::poll\n
  23: futures::future::chain::Chain<A,B,C>::poll\n  24: <futures::future::then::Then<A,B,F> as futures::future::Future>::poll\n  25: <futures::future::either::Either<A,B> as futures::future::Future>::poll\n  26: <futures::future::map::Map<A,F> as futures::future::Future>::poll\n  27: <futures::future::either::Either<A,B> as futures::future::Future>::poll\n  28: <futures::future::map::Map<A,F> as futures::future::Future>::poll\n  29: <futures::future::map_err::MapErr<A,F> as futures::future::Future>::poll\n  30: <futures::stream::and_then::AndThen<S,F,U> as futures::stream::Stream>::poll\n  31: <futures::stream::forward::Forward<T,U> as futures::future::Future>::poll\n  32: <futures::future::map::Map<A,F> as futures::future::Future>::poll\n  33: <futures::future::map_err::MapErr<A,F> as futures::future::Future>::poll\n  34: futures::task_impl::std::set\n  35: <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once\n  36: std::panicking::try::do_call\n  37: __rust_maybe_catch_panic\n             at src\\libpanic_unwind/lib.rs:80\n  38: tokio_threadpool::task::Task::run\n  39: tokio_threadpool::worker::Worker::run_task\n  40: tokio_threadpool::worker::Worker::run\n  41: tokio_timer::clock::clock::with_default\n  42: tokio::runtime::threadpool::builder::Builder::build::{{closure}}\n  43: std::thread::local::LocalKey<T>::with\n  44: std::thread::local::LocalKey<T>::with\n  45: std::sys_common::backtrace::__rust_begin_short_backtrace\n  46: std::panicking::try::do_call\n  47: __rust_maybe_catch_panic\n             at src\\libpanic_unwind/lib.rs:80\n  48: core::ops::function::FnOnce::call_once{{vtable.shim}}\n  49: <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once\n             at /rustc/4560ea788cb760f0a34127156c78e2552949f734\\src\\liballoc/boxed.rs:922\n  50: <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once\n             at /rustc/4560ea788cb760f0a34127156c78e2552949f734\\src\\liballoc/boxed.rs:922\n      std::sys_common::thread::start_thread\n             at src\\libstd\\sys_common/thread.rs:13\n      std::sys::windows::thread::Thread::new::thread_start\n
            at src\\libstd\\sys\\windows/thread.rs:47\n  51: sqlite3GenerateConstraintChecks\n  52: sqlite3GenerateConstraintChecks\n"}

SQL:

/*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */;
/*!40101 SET NAMES utf8 */;
/*!50503 SET NAMES utf8mb4 */;
/*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 */;
/*!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO' */;

CREATE DATABASE IF NOT EXISTS `ctu_crossSchema1` /*!40100 DEFAULT CHARACTER SET utf8 */;
USE `ctu_crossSchema1`;

CREATE TABLE IF NOT EXISTS `account` (
  `account_id` int(11) NOT NULL,
  `district_id` int(11) DEFAULT NULL,
  `frequency` varchar(18) DEFAULT NULL,
  `date` date DEFAULT NULL,
  PRIMARY KEY (`account_id`),
  UNIQUE KEY `account_account_id_key` (`account_id`),
  KEY `account_ibfk_1` (`district_id`),
  CONSTRAINT `account_ibfk_1` FOREIGN KEY (`district_id`) REFERENCES `ctu_crossSchema2`.`district` (`district_id`) ON DELETE NO ACTION ON UPDATE NO ACTION
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE IF NOT EXISTS `loan` (
  `loan_id` int(11) NOT NULL,
  `account_id` int(11) DEFAULT NULL,
  `date` date DEFAULT NULL,
  `amount` bigint(20) DEFAULT NULL,
  `duration` int(11) DEFAULT NULL,
  `payments` decimal(6,2) DEFAULT NULL,
  `status` varchar(1) DEFAULT NULL,
  PRIMARY KEY (`loan_id`),
  KEY `loan_ibfk_1` (`account_id`),
  CONSTRAINT `loan_ibfk_1` FOREIGN KEY (`account_id`) REFERENCES `account` (`account_id`) ON DELETE NO ACTION ON UPDATE NO ACTION
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE IF NOT EXISTS `order` (
  `order_id` int(11) NOT NULL,
  `account_id` int(11) DEFAULT NULL,
  `bank_to` varchar(2) DEFAULT NULL,
  `account_to` bigint(20) DEFAULT NULL,
  `amount` decimal(6,1) DEFAULT NULL,
  `k_symbol` varchar(8) DEFAULT NULL,
  PRIMARY KEY (`order_id`),
  KEY `order_ibfk_1` (`account_id`),
  CONSTRAINT `order_ibfk_1` FOREIGN KEY (`account_id`) REFERENCES `account` (`account_id`) ON DELETE NO ACTION ON UPDATE NO ACTION
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE IF NOT EXISTS `trans` (
  `trans_id` int(11) NOT NULL,
  `account_id` int(11) DEFAULT NULL,
  `date` date DEFAULT NULL,
  `type` varchar(255) DEFAULT NULL,
  `operation` varchar(255) DEFAULT NULL,
  `amount` decimal(30,0) DEFAULT NULL,
  `balance` decimal(30,0) DEFAULT NULL,
  `k_symbol` varchar(255) DEFAULT NULL,
  `bank` varchar(255) DEFAULT NULL,
  `account` bigint(20) DEFAULT NULL,
  PRIMARY KEY (`trans_id`),
  KEY `trans_ibfk_1` (`account_id`),
  CONSTRAINT `trans_ibfk_1` FOREIGN KEY (`account_id`) REFERENCES `account` (`account_id`) ON DELETE NO ACTION ON UPDATE NO ACTION
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE DATABASE IF NOT EXISTS `ctu_crossSchema2` /*!40100 DEFAULT CHARACTER SET utf8 */;
USE `ctu_crossSchema2`;

CREATE TABLE IF NOT EXISTS `card` (
  `card_id` int(11) NOT NULL,
  `disp_id` int(11) DEFAULT NULL,
  `type` varchar(7) DEFAULT NULL,
  `issued` date DEFAULT NULL,
  PRIMARY KEY (`card_id`),
  KEY `card_disp_id_fkey` (`disp_id`),
  CONSTRAINT `card_disp_id_fkey` FOREIGN KEY (`disp_id`) REFERENCES `disp` (`disp_id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE IF NOT EXISTS `client` (
  `client_id` int(11) NOT NULL,
  `gender` varchar(1) DEFAULT NULL,
  `birth_date` date DEFAULT NULL,
  `district_id` int(11) DEFAULT NULL,
  PRIMARY KEY (`client_id`),
  UNIQUE KEY `client_client_id_key` (`client_id`),
  KEY `client_district_id_fkey` (`district_id`),
  CONSTRAINT `client_district_id_fkey` FOREIGN KEY (`district_id`) REFERENCES `district` (`district_id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE IF NOT EXISTS `disp` (
  `disp_id` int(11) NOT NULL,
  `client_id` int(11) DEFAULT NULL,
  `account_id` int(11) DEFAULT NULL,
  `type` varchar(9) DEFAULT NULL,
  PRIMARY KEY (`disp_id`),
  KEY `disp_ibfk_1` (`account_id`),
  KEY `disp_ibfk_2` (`client_id`),
  CONSTRAINT `disp_ibfk_1` FOREIGN KEY (`account_id`) REFERENCES `ctu_crossSchema1`.`account` (`account_id`) ON DELETE NO ACTION ON UPDATE NO ACTION,
  CONSTRAINT `disp_ibfk_2` FOREIGN KEY (`client_id`) REFERENCES `client` (`client_id`) ON DELETE NO ACTION ON UPDATE NO ACTION
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE IF NOT EXISTS `district` (
  `district_id` int(11) NOT NULL,
  `A2` varchar(19) NOT NULL,
  `A3` varchar(15) NOT NULL,
  `A4` int(11) NOT NULL,
  `A5` int(11) NOT NULL,
  `A6` int(11) NOT NULL,
  `A7` int(11) NOT NULL,
  `A8` int(11) NOT NULL,
  `A9` int(11) NOT NULL,
  `A10` decimal(4,1) NOT NULL,
  `A11` int(11) NOT NULL,
  `A12` decimal(4,1) DEFAULT NULL,
  `A13` decimal(3,2) NOT NULL,
  `A14` int(11) NOT NULL,
  `A15` int(11) DEFAULT NULL,
  `A16` int(11) NOT NULL,
  PRIMARY KEY (`district_id`),
  UNIQUE KEY `district_district_id_key` (`district_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

/*!40101 SET SQL_MODE=IFNULL(@OLD_SQL_MODE, '') */;
/*!40014 SET FOREIGN_KEY_CHECKS=IF(@OLD_FOREIGN_KEY_CHECKS IS NULL, 1, @OLD_FOREIGN_KEY_CHECKS) */;
/*!40101 SET CHARACTER_SET_CLIENT=@OLD_CHARACTER_SET_CLIENT */;

@pantharshit00 pantharshit00 added bug/2-confirmed Bug has been reproduced and confirmed. kind/bug A reported bug. labels Dec 18, 2019
@pantharshit00
Copy link
Contributor

We don't support cross schema foreign keys. So in that case I think this should be a general feature request.

@janpio janpio added kind/feature A request for a new feature. and removed bug/2-confirmed Bug has been reproduced and confirmed. kind/bug A reported bug. labels Feb 4, 2020
@mlibk
Copy link

mlibk commented May 1, 2020

I've run into this issue when trying to run npx prisma introspect against an existing database. There are several hundred tables across a dozen Postgres schemas mostly for organisational reasons and most of them have cross-schema foreign keys.

It seems that Prisma considers each schema as an independent data source and I suspect that's often not the case - especially for bigger databases. Unfortunately, until this is addressed, it won't be possible to use Prisma in these situations.

@albertoperdomo albertoperdomo added the team/schema Issue for team Schema. label Nov 25, 2020
@janpio janpio changed the title [Introspection] Postgres Cross Schema Foreign Keys [Introspection] Cross Schema Foreign Keys Jan 5, 2021
@janpio
Copy link
Member Author

janpio commented Jan 5, 2021

This functionality also exists for SQL Server, not just Postgres: #4762

This also includes a current snapshot of the error message you get on cross schema foreign keys:

Environment variables loaded from .env
Prisma schema loaded from prisma\schema.prisma
Introspecting based on datasource defined in prisma\schema.prisma …
Error: P4002
The schema of the introspected database was inconsistent: Table RhionsUsers not found.

@edgarasben
Copy link

edgarasben commented May 9, 2021

Ran into the same issue trying to use Prisma with Supabase db.
Illegal cross schema reference from 'public.profiles' to 'auth.users' in constraint 'profiles_id_fkey'

This prevents using Supabase Auth together with Supabase Database.

@janpio
Copy link
Member Author

janpio commented May 10, 2021

Note: This is actually related to #1122 which we are currently looking into, so will hopefully be covered by that as well. Feel free to subscribe to that issue as well for updates.

@lukeramsden
Copy link

lukeramsden commented May 12, 2021

This shut down pretty quick any hopes or interest in trying Prisma against our MSSQL database at work. We have a schema for each service in our SOA architecture, which is fairly standard. Would love to see this implemented.
I suspect this will not be covered by #1122 because we still won't be able to introspect across schemas because they have foreign key relations between them. Being able to query cross-schema is a great first step though!

@janpio
Copy link
Member Author

janpio commented May 12, 2021

To query across we need to represent it in the schema, which means we would also need to test these which we usually do using Introspection - so the chance that we do both (and migrating such tables) at the same time is pretty high.

@MonsterDeveloper
Copy link

Same here, unable to use Supabase PostgreSQL DB with Prisma. @edgarasben did you find any workarounds?

@edgarasben
Copy link

@MonsterDeveloper I ended up using only the public schema and skipping the auth schema. I use Next Auth now for the Auth. And it works nicely with Prisma and Supabase PostgreSQL DB. So all my DB tables now live in public schema.

@MonsterDeveloper
Copy link

@edgarasben thanks! Will dig into it.

@edgarasben
Copy link

@MonsterDeveloper np! If you want to use Supabase Auth, you can use trigger to copy changes from auth.users to public.users (supabase/supabase#1502 (comment))

@janpio
Copy link
Member Author

janpio commented Aug 30, 2022

Introspection is not part of it, but we shipped a first version of multi schema support in 4.3.0: #1122 (comment) Everyone who is interested in Introspection of multi schema can probably also help us test this first implementation step, so plese go and have a look.

@ashrielbrian
Copy link

Any updates on this? We're approaching the end of 2022 :(

@Karbust
Copy link

Karbust commented Nov 8, 2022

Any updates on this? We're approaching the end of 2022 :(

According to the roadmap, it's already in early availability: https://prismaio.notion.site/Prisma-Roadmap-50766227b779464ab98899accb98295f?p=f7107b3ed0194763835c42f0fc54fc8e&pm=s

@janpio
Copy link
Member Author

janpio commented Nov 8, 2022

The query part is available via multiSchema, Introspection will follow soon. Here is the current state: #1122 (comment)

@philefstat
Copy link

This would be super useful / necessary to work with https://nhost.io/ (similar to supabase). Hope it's supported soon!

@janpio
Copy link
Member Author

janpio commented Nov 22, 2022

Can you share exactly why this is specifically important for Nhost? Do they have some default setups across multiple schemas @philefstat? Thanks.

@chrskerr
Copy link

chrskerr commented Nov 29, 2022

+1 on this request. We have foreign keys between many of our schemas so cannot swap over until we can begin to use them.

I just tried the introspection in 4.7 though, and it's awesome! Great work there :D

@floelhoeffel
Copy link

Hey Everyone - we just shipped full support for multiple schemas (and foreign keys between them) for PostgreSQL in 4.7.0

@chrskerr
Copy link

chrskerr commented Nov 30, 2022

Including for migrations? When I run yarn prisma migrate dev I get the following (:

Error: P4002

The schema of the introspected database was inconsistent: Illegal cross schema reference from `public.xyz` to `abc.def` in constraint `xxxxxx`. Foreign keys between database schemas are not supported in Prisma. Please follow the GitHub ticket: https://github.com/prisma/prisma/issues/1175
prisma                  : 4.7.0
@prisma/client          : 4.7.0
Current platform        : darwin-arm64
Query Engine (Node-API) : libquery-engine 39190b250ebc338586e25e6da45e5e783bc8a635 (at ../../node_modules/@prisma/engines/libquery_engine-darwin-arm64.dylib.node)
Migration Engine        : migration-engine-cli 39190b250ebc338586e25e6da45e5e783bc8a635 (at ../../node_modules/@prisma/engines/migration-engine-darwin-arm64)
Introspection Engine    : introspection-core 39190b250ebc338586e25e6da45e5e783bc8a635 (at ../../node_modules/@prisma/engines/introspection-engine-darwin-arm64)
Format Binary           : prisma-fmt 39190b250ebc338586e25e6da45e5e783bc8a635 (at ../../node_modules/@prisma/engines/prisma-fmt-darwin-arm64)
Format Wasm             : @prisma/prisma-fmt-wasm 4.7.0-74.39190b250ebc338586e25e6da45e5e783bc8a635
Default Engines Hash    : 39190b250ebc338586e25e6da45e5e783bc8a635
Studio                  : 0.477.0
Preview Features        : multiSchema, postgresqlExtensions, clientExtension

@janpio
Copy link
Member Author

janpio commented Nov 30, 2022

Can you open a new issue @chrskerr and provide more information? Having the schema for example would help a lot so we can reproduce this. (If you can not share publicly, still create the issue but send the schema to schemas@prisma.io [If needed we can also do an NDA].)

@oalexdoda
Copy link

This is absolutely critical towards wider Prisma adoption. There's third party platforms like Hasura that can do cross-schema queries with zero effort. Why has this been stalling for over a year when it's probably one of the most significant obstacles in having developers use it with popular platforms like Supabase (and others)?

@janpio
Copy link
Member Author

janpio commented Dec 4, 2022

This has been implemented and shipped with 4.7.0 for PostgreSQL behind the multiSchema feature flag. Read more about it in our release notes: https://github.com/prisma/prisma/releases/tag/4.7.0

@chanmathew
Copy link

@janpio On 4.7.1 prisma-client and it is still failing when using it with Supabase trying to reference auth.users, any ideas?

@janpio
Copy link
Member Author

janpio commented Dec 16, 2022

@chanmathew
How did you set up your project?
Did you add the multiSchema preview feature flag?
Did you add @@schema to your models representing your tables or use prisma db pull to use Introspection to automate that?

@chanmathew
Copy link

chanmathew commented Dec 16, 2022

hi @janpio, thanks for the prompt response, I have the flag on in the schema file:

generator client {
  provider = "prisma-client-js"
  previewFeatures = ["multiSchema"]
}

In my Supabase I have auth.users, and I'm simply referencing the user_id as a foreign key in a trips table.

However when I run prisma db pull to grab my existing tables in Supabase I get this error:

✖ Introspecting based on datasource defined in prisma/schema.prisma

Error: P4002

The schema of the introspected database was inconsistent: Illegal cross schema reference from `public.trips` to `auth.users` in constraint `trips_user_id_fkey`. Foreign keys between database schemas are not supported in Prisma.

UPDATE
Seems like it was able to get prisma db pull working after I added schemas = ["auth", "public"] to my schema file, and it generated the schemas.

Also side note for others who may run into this issue, Prisma will seem to fail to run Prisma Studio if you happen to have another users table in the public DB schemas, as there will be a collision between the private auth.users table Supabase generates.

@janpio
Copy link
Member Author

janpio commented Dec 17, 2022

There should be no clashes via db pull optimally. If there are, please open a new issue for that @chanmathew. Thanks!

@kirbby
Copy link

kirbby commented Dec 17, 2022

I can confirm what @chanmathew said, db pull doesn't work without defining "schemas" like this:

datasource supabase {
  provider = "postgresql"
  url      = env("DATABASE_URL")
  schemas  = ["auth", "public"]
}

With the "schemas" it works as intended.

@aleksey-mukho
Copy link

I call prisma db pull and get a lot private tables and enums


model audit_log_entries {
  instance_id String?   @db.Uuid
  id          String    @id @db.Uuid
  payload     Json?     @db.Json
  created_at  DateTime? @db.Timestamptz(6)
  ip_address  String    @default("") @db.VarChar(64)

  @@index([instance_id], map: "audit_logs_instance_id_idx")
  @@schema("auth")
}

model identities {
  id              String
  user_id         String    @db.Uuid
  identity_data   Json
  provider        String
  last_sign_in_at DateTime? @db.Timestamptz(6)
  created_at      DateTime? @db.Timestamptz(6)
  updated_at      DateTime? @db.Timestamptz(6)
  users           users     @relation(fields: [user_id], references: [id], onDelete: Cascade, onUpdate: NoAction)

  @@id([provider, id])
  @@index([user_id])
  @@schema("auth")
}

model instances {
  id              String    @id @db.Uuid
  uuid            String?   @db.Uuid
  raw_base_config String?
  created_at      DateTime? @db.Timestamptz(6)
  updated_at      DateTime? @db.Timestamptz(6)

  @@schema("auth")
}

model mfa_amr_claims {
  session_id            String   @db.Uuid
  created_at            DateTime @db.Timestamptz(6)
  updated_at            DateTime @db.Timestamptz(6)
  authentication_method String
  id                    String   @id(map: "amr_id_pk") @db.Uuid
  sessions              sessions @relation(fields: [session_id], references: [id], onDelete: Cascade, onUpdate: NoAction)

  @@unique([session_id, authentication_method], map: "mfa_amr_claims_session_id_authentication_method_pkey")
  @@schema("auth")
}

model mfa_challenges {
  id          String      @id @db.Uuid
  factor_id   String      @db.Uuid
  created_at  DateTime    @db.Timestamptz(6)
  verified_at DateTime?   @db.Timestamptz(6)
  ip_address  String      @db.Inet
  mfa_factors mfa_factors @relation(fields: [factor_id], references: [id], onDelete: Cascade, onUpdate: NoAction, map: "mfa_challenges_auth_factor_id_fkey")

  @@schema("auth")
}

model mfa_factors {
  id             String           @id @db.Uuid
  user_id        String           @db.Uuid
  friendly_name  String?
  factor_type    factor_type
  status         factor_status
  created_at     DateTime         @db.Timestamptz(6)
  updated_at     DateTime         @db.Timestamptz(6)
  secret         String?
  mfa_challenges mfa_challenges[]
  users          users            @relation(fields: [user_id], references: [id], onDelete: Cascade, onUpdate: NoAction)

  @@index([user_id, created_at], map: "factor_id_created_at_idx")
  @@schema("auth")
}

model refresh_tokens {
  instance_id String?   @db.Uuid
  id          BigInt    @id @default(autoincrement())
  token       String?   @unique(map: "refresh_tokens_token_unique") @db.VarChar(255)
  user_id     String?   @db.VarChar(255)
  revoked     Boolean?
  created_at  DateTime? @db.Timestamptz(6)
  updated_at  DateTime? @db.Timestamptz(6)
  parent      String?   @db.VarChar(255)
  session_id  String?   @db.Uuid
  sessions    sessions? @relation(fields: [session_id], references: [id], onDelete: Cascade, onUpdate: NoAction)

  @@index([session_id], map: "refresh_token_session_id")
  @@index([instance_id])
  @@index([instance_id, user_id])
  @@index([parent])
  @@index([session_id, revoked])
  @@index([token])
  @@schema("auth")
}

model saml_providers {
  id                String        @id @db.Uuid
  sso_provider_id   String        @db.Uuid
  entity_id         String        @unique
  metadata_xml      String
  metadata_url      String?
  attribute_mapping Json?
  created_at        DateTime?     @db.Timestamptz(6)
  updated_at        DateTime?     @db.Timestamptz(6)
  sso_providers     sso_providers @relation(fields: [sso_provider_id], references: [id], onDelete: Cascade, onUpdate: NoAction)

  @@index([sso_provider_id])
  @@schema("auth")
}

model saml_relay_states {
  id              String        @id @db.Uuid
  sso_provider_id String        @db.Uuid
  request_id      String
  for_email       String?
  redirect_to     String?
  from_ip_address String?       @db.Inet
  created_at      DateTime?     @db.Timestamptz(6)
  updated_at      DateTime?     @db.Timestamptz(6)
  sso_providers   sso_providers @relation(fields: [sso_provider_id], references: [id], onDelete: Cascade, onUpdate: NoAction)

  @@index([for_email])
  @@index([sso_provider_id])
  @@schema("auth")
}

model schema_migrations {
  version String @id @db.VarChar(255)

  @@schema("auth")
}

model sessions {
  id             String           @id @db.Uuid
  user_id        String           @db.Uuid
  created_at     DateTime?        @db.Timestamptz(6)
  updated_at     DateTime?        @db.Timestamptz(6)
  factor_id      String?          @db.Uuid
  aal            aal_level?
  not_after      DateTime?        @db.Timestamptz(6)
  mfa_amr_claims mfa_amr_claims[]
  refresh_tokens refresh_tokens[]
  users          users            @relation(fields: [user_id], references: [id], onDelete: Cascade, onUpdate: NoAction)
  sso_sessions   sso_sessions[]

  @@index([user_id])
  @@index([user_id, created_at], map: "user_id_created_at_idx")
  @@schema("auth")
}

model sso_domains {
  id              String        @id @db.Uuid
  sso_provider_id String        @db.Uuid
  domain          String
  created_at      DateTime?     @db.Timestamptz(6)
  updated_at      DateTime?     @db.Timestamptz(6)
  sso_providers   sso_providers @relation(fields: [sso_provider_id], references: [id], onDelete: Cascade, onUpdate: NoAction)

  @@index([sso_provider_id])
  @@schema("auth")
}

model sso_providers {
  id                String              @id @db.Uuid
  resource_id       String?
  created_at        DateTime?           @db.Timestamptz(6)
  updated_at        DateTime?           @db.Timestamptz(6)
  saml_providers    saml_providers[]
  saml_relay_states saml_relay_states[]
  sso_domains       sso_domains[]
  sso_sessions      sso_sessions[]

  @@schema("auth")
}

model sso_sessions {
  id              String         @id @db.Uuid
  session_id      String         @db.Uuid
  sso_provider_id String?        @db.Uuid
  not_before      DateTime?      @db.Timestamptz(6)
  not_after       DateTime?      @db.Timestamptz(6)
  idp_initiated   Boolean?       @default(false)
  created_at      DateTime?      @db.Timestamptz(6)
  updated_at      DateTime?      @db.Timestamptz(6)
  sessions        sessions       @relation(fields: [session_id], references: [id], onDelete: Cascade, onUpdate: NoAction)
  sso_providers   sso_providers? @relation(fields: [sso_provider_id], references: [id], onDelete: Cascade, onUpdate: NoAction)

  @@index([session_id])
  @@index([sso_provider_id])
  @@schema("auth")
}

model users {
  instance_id                 String?       @db.Uuid
  id                          String        @id @db.Uuid
  aud                         String?       @db.VarChar(255)
  role                        String?       @db.VarChar(255)
  email                       String?       @unique @db.VarChar(255)
  encrypted_password          String?       @db.VarChar(255)
  email_confirmed_at          DateTime?     @db.Timestamptz(6)
  invited_at                  DateTime?     @db.Timestamptz(6)
  confirmation_token          String?       @db.VarChar(255)
  confirmation_sent_at        DateTime?     @db.Timestamptz(6)
  recovery_token              String?       @db.VarChar(255)
  recovery_sent_at            DateTime?     @db.Timestamptz(6)
  email_change_token_new      String?       @db.VarChar(255)
  email_change                String?       @db.VarChar(255)
  email_change_sent_at        DateTime?     @db.Timestamptz(6)
  last_sign_in_at             DateTime?     @db.Timestamptz(6)
  raw_app_meta_data           Json?
  raw_user_meta_data          Json?
  is_super_admin              Boolean?
  created_at                  DateTime?     @db.Timestamptz(6)
  updated_at                  DateTime?     @db.Timestamptz(6)
  phone                       String?       @unique @db.VarChar(15)
  phone_confirmed_at          DateTime?     @db.Timestamptz(6)
  phone_change                String?       @default("") @db.VarChar(15)
  phone_change_token          String?       @default("") @db.VarChar(255)
  phone_change_sent_at        DateTime?     @db.Timestamptz(6)
  confirmed_at                DateTime?     @default(dbgenerated("LEAST(email_confirmed_at, phone_confirmed_at)")) @db.Timestamptz(6)
  email_change_token_current  String?       @default("") @db.VarChar(255)
  email_change_confirm_status Int?          @default(0) @db.SmallInt
  banned_until                DateTime?     @db.Timestamptz(6)
  reauthentication_token      String?       @default("") @db.VarChar(255)
  reauthentication_sent_at    DateTime?     @db.Timestamptz(6)
  identities                  identities[]
  mfa_factors                 mfa_factors[]
  sessions                    sessions[]
  profiles                    profiles?

  @@index([instance_id])
  @@schema("auth")
}

model profiles {
  id       String @id @db.Uuid
  provider String
  email    String
  users    users  @relation(fields: [id], references: [id], onDelete: NoAction, onUpdate: NoAction)

  @@schema("public")
}

enum aal_level {
  aal1
  aal2
  aal3

  @@schema("auth")
}

enum factor_status {
  unverified
  verified

  @@schema("auth")
}

enum factor_type {
  totp
  webauthn

  @@schema("auth")
}

But when I'm trying to call npx prisma generate, I'm getting error

DbError { severity: "ERROR", parsed_severity: Some(Error), code: SqlState(E42710), message: "type \"aal_level\" already exists", detail: None, hint: None, position: None, where_: None, schema: None, table: None, column: None, datatype: None, constraint: None, file: Some("typecmds.c"), line: Some(1170), routine: Some("DefineEnum") }

@floelhoeffel floelhoeffel added the kind/bug A reported bug. label Dec 19, 2022
@pimeys
Copy link
Contributor

pimeys commented Dec 19, 2022

I call prisma db pull and get a lot private tables and enums


model audit_log_entries {
  instance_id String?   @db.Uuid
  id          String    @id @db.Uuid
  payload     Json?     @db.Json
  created_at  DateTime? @db.Timestamptz(6)
  ip_address  String    @default("") @db.VarChar(64)

  @@index([instance_id], map: "audit_logs_instance_id_idx")
  @@schema("auth")
}

model identities {
  id              String
  user_id         String    @db.Uuid
  identity_data   Json
  provider        String
  last_sign_in_at DateTime? @db.Timestamptz(6)
  created_at      DateTime? @db.Timestamptz(6)
  updated_at      DateTime? @db.Timestamptz(6)
  users           users     @relation(fields: [user_id], references: [id], onDelete: Cascade, onUpdate: NoAction)

  @@id([provider, id])
  @@index([user_id])
  @@schema("auth")
}

model instances {
  id              String    @id @db.Uuid
  uuid            String?   @db.Uuid
  raw_base_config String?
  created_at      DateTime? @db.Timestamptz(6)
  updated_at      DateTime? @db.Timestamptz(6)

  @@schema("auth")
}

model mfa_amr_claims {
  session_id            String   @db.Uuid
  created_at            DateTime @db.Timestamptz(6)
  updated_at            DateTime @db.Timestamptz(6)
  authentication_method String
  id                    String   @id(map: "amr_id_pk") @db.Uuid
  sessions              sessions @relation(fields: [session_id], references: [id], onDelete: Cascade, onUpdate: NoAction)

  @@unique([session_id, authentication_method], map: "mfa_amr_claims_session_id_authentication_method_pkey")
  @@schema("auth")
}

model mfa_challenges {
  id          String      @id @db.Uuid
  factor_id   String      @db.Uuid
  created_at  DateTime    @db.Timestamptz(6)
  verified_at DateTime?   @db.Timestamptz(6)
  ip_address  String      @db.Inet
  mfa_factors mfa_factors @relation(fields: [factor_id], references: [id], onDelete: Cascade, onUpdate: NoAction, map: "mfa_challenges_auth_factor_id_fkey")

  @@schema("auth")
}

model mfa_factors {
  id             String           @id @db.Uuid
  user_id        String           @db.Uuid
  friendly_name  String?
  factor_type    factor_type
  status         factor_status
  created_at     DateTime         @db.Timestamptz(6)
  updated_at     DateTime         @db.Timestamptz(6)
  secret         String?
  mfa_challenges mfa_challenges[]
  users          users            @relation(fields: [user_id], references: [id], onDelete: Cascade, onUpdate: NoAction)

  @@index([user_id, created_at], map: "factor_id_created_at_idx")
  @@schema("auth")
}

model refresh_tokens {
  instance_id String?   @db.Uuid
  id          BigInt    @id @default(autoincrement())
  token       String?   @unique(map: "refresh_tokens_token_unique") @db.VarChar(255)
  user_id     String?   @db.VarChar(255)
  revoked     Boolean?
  created_at  DateTime? @db.Timestamptz(6)
  updated_at  DateTime? @db.Timestamptz(6)
  parent      String?   @db.VarChar(255)
  session_id  String?   @db.Uuid
  sessions    sessions? @relation(fields: [session_id], references: [id], onDelete: Cascade, onUpdate: NoAction)

  @@index([session_id], map: "refresh_token_session_id")
  @@index([instance_id])
  @@index([instance_id, user_id])
  @@index([parent])
  @@index([session_id, revoked])
  @@index([token])
  @@schema("auth")
}

model saml_providers {
  id                String        @id @db.Uuid
  sso_provider_id   String        @db.Uuid
  entity_id         String        @unique
  metadata_xml      String
  metadata_url      String?
  attribute_mapping Json?
  created_at        DateTime?     @db.Timestamptz(6)
  updated_at        DateTime?     @db.Timestamptz(6)
  sso_providers     sso_providers @relation(fields: [sso_provider_id], references: [id], onDelete: Cascade, onUpdate: NoAction)

  @@index([sso_provider_id])
  @@schema("auth")
}

model saml_relay_states {
  id              String        @id @db.Uuid
  sso_provider_id String        @db.Uuid
  request_id      String
  for_email       String?
  redirect_to     String?
  from_ip_address String?       @db.Inet
  created_at      DateTime?     @db.Timestamptz(6)
  updated_at      DateTime?     @db.Timestamptz(6)
  sso_providers   sso_providers @relation(fields: [sso_provider_id], references: [id], onDelete: Cascade, onUpdate: NoAction)

  @@index([for_email])
  @@index([sso_provider_id])
  @@schema("auth")
}

model schema_migrations {
  version String @id @db.VarChar(255)

  @@schema("auth")
}

model sessions {
  id             String           @id @db.Uuid
  user_id        String           @db.Uuid
  created_at     DateTime?        @db.Timestamptz(6)
  updated_at     DateTime?        @db.Timestamptz(6)
  factor_id      String?          @db.Uuid
  aal            aal_level?
  not_after      DateTime?        @db.Timestamptz(6)
  mfa_amr_claims mfa_amr_claims[]
  refresh_tokens refresh_tokens[]
  users          users            @relation(fields: [user_id], references: [id], onDelete: Cascade, onUpdate: NoAction)
  sso_sessions   sso_sessions[]

  @@index([user_id])
  @@index([user_id, created_at], map: "user_id_created_at_idx")
  @@schema("auth")
}

model sso_domains {
  id              String        @id @db.Uuid
  sso_provider_id String        @db.Uuid
  domain          String
  created_at      DateTime?     @db.Timestamptz(6)
  updated_at      DateTime?     @db.Timestamptz(6)
  sso_providers   sso_providers @relation(fields: [sso_provider_id], references: [id], onDelete: Cascade, onUpdate: NoAction)

  @@index([sso_provider_id])
  @@schema("auth")
}

model sso_providers {
  id                String              @id @db.Uuid
  resource_id       String?
  created_at        DateTime?           @db.Timestamptz(6)
  updated_at        DateTime?           @db.Timestamptz(6)
  saml_providers    saml_providers[]
  saml_relay_states saml_relay_states[]
  sso_domains       sso_domains[]
  sso_sessions      sso_sessions[]

  @@schema("auth")
}

model sso_sessions {
  id              String         @id @db.Uuid
  session_id      String         @db.Uuid
  sso_provider_id String?        @db.Uuid
  not_before      DateTime?      @db.Timestamptz(6)
  not_after       DateTime?      @db.Timestamptz(6)
  idp_initiated   Boolean?       @default(false)
  created_at      DateTime?      @db.Timestamptz(6)
  updated_at      DateTime?      @db.Timestamptz(6)
  sessions        sessions       @relation(fields: [session_id], references: [id], onDelete: Cascade, onUpdate: NoAction)
  sso_providers   sso_providers? @relation(fields: [sso_provider_id], references: [id], onDelete: Cascade, onUpdate: NoAction)

  @@index([session_id])
  @@index([sso_provider_id])
  @@schema("auth")
}

model users {
  instance_id                 String?       @db.Uuid
  id                          String        @id @db.Uuid
  aud                         String?       @db.VarChar(255)
  role                        String?       @db.VarChar(255)
  email                       String?       @unique @db.VarChar(255)
  encrypted_password          String?       @db.VarChar(255)
  email_confirmed_at          DateTime?     @db.Timestamptz(6)
  invited_at                  DateTime?     @db.Timestamptz(6)
  confirmation_token          String?       @db.VarChar(255)
  confirmation_sent_at        DateTime?     @db.Timestamptz(6)
  recovery_token              String?       @db.VarChar(255)
  recovery_sent_at            DateTime?     @db.Timestamptz(6)
  email_change_token_new      String?       @db.VarChar(255)
  email_change                String?       @db.VarChar(255)
  email_change_sent_at        DateTime?     @db.Timestamptz(6)
  last_sign_in_at             DateTime?     @db.Timestamptz(6)
  raw_app_meta_data           Json?
  raw_user_meta_data          Json?
  is_super_admin              Boolean?
  created_at                  DateTime?     @db.Timestamptz(6)
  updated_at                  DateTime?     @db.Timestamptz(6)
  phone                       String?       @unique @db.VarChar(15)
  phone_confirmed_at          DateTime?     @db.Timestamptz(6)
  phone_change                String?       @default("") @db.VarChar(15)
  phone_change_token          String?       @default("") @db.VarChar(255)
  phone_change_sent_at        DateTime?     @db.Timestamptz(6)
  confirmed_at                DateTime?     @default(dbgenerated("LEAST(email_confirmed_at, phone_confirmed_at)")) @db.Timestamptz(6)
  email_change_token_current  String?       @default("") @db.VarChar(255)
  email_change_confirm_status Int?          @default(0) @db.SmallInt
  banned_until                DateTime?     @db.Timestamptz(6)
  reauthentication_token      String?       @default("") @db.VarChar(255)
  reauthentication_sent_at    DateTime?     @db.Timestamptz(6)
  identities                  identities[]
  mfa_factors                 mfa_factors[]
  sessions                    sessions[]
  profiles                    profiles?

  @@index([instance_id])
  @@schema("auth")
}

model profiles {
  id       String @id @db.Uuid
  provider String
  email    String
  users    users  @relation(fields: [id], references: [id], onDelete: NoAction, onUpdate: NoAction)

  @@schema("public")
}

enum aal_level {
  aal1
  aal2
  aal3

  @@schema("auth")
}

enum factor_status {
  unverified
  verified

  @@schema("auth")
}

enum factor_type {
  totp
  webauthn

  @@schema("auth")
}

But when I'm trying to call npx prisma generate, I'm getting error

DbError { severity: "ERROR", parsed_severity: Some(Error), code: SqlState(E42710), message: "type \"aal_level\" already exists", detail: None, hint: None, position: None, where_: None, schema: None, table: None, column: None, datatype: None, constraint: None, file: Some("typecmds.c"), line: Some(1170), routine: Some("DefineEnum") }

I tried to reproduce this issue. Copied your data model and tried to run npx prisma generate with versions 4.7.1 and the latest main. Could not reproduce, generate and db push both work just as they should.

@pimeys
Copy link
Contributor

pimeys commented Dec 19, 2022

@aleksey-mukho Your error message does not look like it's coming from generate. Could you do a proper reproduction and maybe open an issue for this?

@floelhoeffel
Copy link

floelhoeffel commented Jan 17, 2023

We consider this work done as of 4.9.0 except for MySQL. To track that work, please check: #16943

@nahtnam
Copy link

nahtnam commented Jan 28, 2023

Anyone having issues with prisma db push after doing a pull?

  1. I pulled the schema from production
  2. Switched the connection to a local development instance of supabase
  3. Tried to do a prisma db push and got the following error:
Prisma schema loaded from prisma/schema.prisma
Datasource "db": PostgreSQL database "postgres", schema "public" at "localhost:54322"
Error: db error: ERROR: cannot drop index auth.users_email_key because constraint users_email_key on table auth.users requires it
HINT: You can drop constraint users_email_key on table auth.users instead.
   0: sql_migration_connector::apply_migration::migration_step
           with step=DropIndex { index_id: IndexId(13) }
             at migration-engine/connectors/sql-migration-connector/src/apply_migration.rs:21
   1: sql_migration_connector::apply_migration::apply_migration
             at migration-engine/connectors/sql-migration-connector/src/apply_migration.rs:10
   2: migration_core::state::SchemaPush
             at migration-engine/core/src/state.rs:402

@janpio
Copy link
Member Author

janpio commented Jan 30, 2023

Please open a new issue @nahtnam - that should definitely not happen and we would love to debug that with you. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests