Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FE][Dead Loop]starrocks-mysql-nio-pool thread dead loop caught in JoinNode.pushDownRuntimeFilters #45578

Open
ever4Kenny opened this issue May 14, 2024 · 3 comments
Open

[FE][Dead Loop]starrocks-mysql-nio-pool thread dead loop caught in JoinNode.pushDownRuntimeFilters #45578

ever4Kenny opened this issue May 14, 2024 · 3 comments
Labels
type/bug Something isn't working

Comments

@ever4Kenny
Copy link

ever4Kenny commented May 14, 2024 •
edited

Steps to reproduce the behavior (Required)

Query hive catalog with the following two tables:

  1. temp.fxh_credit_member_mars301421, a unpartitioned table with 657323 rows:
CREATE TABLE `temp.fxh_credit_member_mars301421`(
    `member_id` string 
    `product_code` string
    `apply_day` string, 
    `credit_day` string,  
    `ffr` string, 
    `page` string, 
    `description` string) 
    ROW FORMAT SERDE
    'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
    STORED AS INPUTFORMAT
     'org.apache.hadoop.mapred.TextInputFormat'
    OUTPUTFORMAT
     'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
    LOCATION 
    'hdfs://lan/user/hive/warehouse/temp.db/fxh_credit_member_mars301421';
  1. ods.ods_fin_loan_augur_app_loan_order_d a partitioned table with 90 partitions and there are 6,000,000+ rows in each partition.
CREATE TABLE `ods.ods_fin_loan_augur_app_loan_order_d`( 
    `id` bigint COMMENT '',    
    `apply_order_no` string,  
     `merchant_loan_order_no` string, 
     `loan_order_no` string,  
    `investor_loan_order_no` string, 
    `loan_bill_no` string,   
    `channel_loan_bill_no` string, 
    `member_id` string,     
     `account_id` string,   
    `merchant_user_id` string,  
    `merchant_code` string,
    `product_type` string,
     `product_code` string,
    `assistor` string,
     `investor` string,
    `insurance` string,
    `apply_term` int,
    `apply_term_unit` string,
    `apply_amount` string,
     `loan_amount` string,
    `loan_term` int,
    `loan_term_unit` string,
    `loan_purpose` string,
    `apply_time` string,
    `risk_audit_time` string,
     `loan_submit_time` string,
    `loan_time` string ,
    `loan_bankcard_no` string,
    `loan_type` int,
    `loan_rate` string,
   `investor_rate` string,
   `refund_amount` string, 
   `remark` string,
   `extension` string,
   `create_by` string,
   `update_by` string, 
   `create_time` string,
    `update_time` string,
    `enter_account_status` int,
    `status` int, 
    `withdraw_status` int, 
    `fail_reason` string,
    `loan_bankcard_id` string, 
    `lend_amount` string,
    `coupon_id` string, 
    `gds_field_name` string)  
PARTITIONED BY ( `pt` string)  
ROW FORMAT SERDE  
    'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe' 
STORED AS INPUTFORMAT  
    'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat' 
OUTPUTFORMAT   
    'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat'
LOCATION  
    'hdfs://lan/user/hive/warehouse/ods.db/ods_fin_loan_augur_app_loan_order_d';
  1. Select query:
select first_credit_month as "月份" ,
       count(distinct member_id) as "人数" ,
       count(distinct if(months=0, member_id, null))*1.0000/count(distinct member_id) as mob0 ,
       count(distinct if(months=1, member_id, null))*1.0000/count(distinct member_id) as mob1 ,
       count(distinct if(months=2, member_id, null))*1.0000/count(distinct member_id) as mob2 ,
       count(distinct if(months=3, member_id, null))*1.0000/count(distinct member_id) as mob3 ,
       count(distinct if(months=4, member_id, null))*1.0000/count(distinct member_id) as mob4 ,
       count(distinct if(months=5, member_id, null))*1.0000/count(distinct member_id) as mob5 ,
       count(distinct if(months=6, member_id, null))*1.0000/count(distinct member_id) as mob6 ,
       count(distinct if(months=7, member_id, null))*1.0000/count(distinct member_id) as mob7 ,
       count(distinct if(months=8, member_id, null))*1.0000/count(distinct member_id) as mob8 ,
       count(distinct if(months=9, member_id, null))*1.0000/count(distinct member_id) as mob9 ,
       count(distinct if(months=10, member_id, null))*1.0000/count(distinct member_id) as mob10 ,
       count(distinct if(months=11, member_id, null))*1.0000/count(distinct member_id) as mob11 ,
       count(distinct if(months=12, member_id, null))*1.0000/count(distinct member_id) as mob12 ,
       count(distinct if(months=13, member_id, null))*1.0000/count(distinct member_id) as mob13 ,
       count(distinct if(months=14, member_id, null))*1.0000/count(distinct member_id) as mob14 ,
       count(distinct if(months=15, member_id, null))*1.0000/count(distinct member_id) as mob15 ,
       count(distinct if(months=16, member_id, null))*1.0000/count(distinct member_id) as mob16 ,
       count(distinct if(months=17, member_id, null))*1.0000/count(distinct member_id) as mob17 ,
       count(distinct if(months=18, member_id, null))*1.0000/count(distinct member_id) as mob18 ,
       count(distinct if(months=19, member_id, null))*1.0000/count(distinct member_id) as mob19 ,
       count(distinct if(months=20, member_id, null))*1.0000/count(distinct member_id) as mob20 ,
       count(distinct if(months=21, member_id, null))*1.0000/count(distinct member_id) as mob21 ,
       count(distinct if(months=22, member_id, null))*1.0000/count(distinct member_id) as mob22 ,
       count(distinct if(months=23, member_id, null))*1.0000/count(distinct member_id) as mob23 ,
       count(distinct if(months=24, member_id, null))*1.0000/count(distinct member_id) as mob24 ,
       count(distinct if(months=25, member_id, null))*1.0000/count(distinct member_id) as mob25 ,
       count(distinct if(months=26, member_id, null))*1.0000/count(distinct member_id) as mob26 ,
       count(distinct if(months=27, member_id, null))*1.0000/count(distinct member_id) as mob27 ,
       count(distinct if(months=28, member_id, null))*1.0000/count(distinct member_id) as mob28 ,
       count(distinct if(months=29, member_id, null))*1.0000/count(distinct member_id) as mob29 ,
       count(distinct if(months=30, member_id, null))*1.0000/count(distinct member_id) as mob30 ,
       count(distinct if(months=31, member_id, null))*1.0000/count(distinct member_id) as mob31 ,
       count(distinct if(months=32, member_id, null))*1.0000/count(distinct member_id) as mob32 ,
       count(distinct if(months=33, member_id, null))*1.0000/count(distinct member_id) as mob33 ,
       count(distinct if(months=34, member_id, null))*1.0000/count(distinct member_id) as mob34 ,
       count(distinct if(months=35, member_id, null))*1.0000/count(distinct member_id) as mob35 ,
       count(distinct if(months=36, member_id, null))*1.0000/count(distinct member_id) as mob36 ,
       count(distinct if(months=37, member_id, null))*1.0000/count(distinct member_id) as mob37 ,
       count(distinct if(months=38, member_id, null))*1.0000/count(distinct member_id) as mob38 ,
       count(distinct if(months=39, member_id, null))*1.0000/count(distinct member_id) as mob39
from (
         select t1.member_id ,
                product_code ,
                first_credit_month ,
                (cast(substring(apply_month, 1, 4) as int) - cast(substring(first_credit_month, 1, 4) as int))*12 + cast(substring(apply_month, 6, 7) as int) - cast(substring(first_credit_month, 6, 7) as int) as months ,
                amt
         from (
                  select member_id,
                         product_code,
                         credit_month as first_credit_month
                  from (
                           select row_number() over(partition by member_id order by credit_day) as rank,
                           member_id,product_code,
                           substring(credit_day, 1, 7) as credit_month
                           from temp.fxh_credit_member_mars301421 where page = '商详页面'
                       ) t1 where rank = 1
              ) t1 left join (
             select member_id,
                    substring(apply_time, 1, 7) as apply_month ,
                    sum(cast(loan_amount as double) - cast(refund_amount as double)) as amt
             from ods.ods_fin_loan_augur_app_loan_order_d
             where cast(pt as date) = current_date - interval '1' day
               and status = 50
               and enter_account_status = 80
             group by member_id, substring(apply_time, 1, 7) ) t2 on t1.member_id = t2.member_id
     ) a1
where first_credit_month >= '2022-01'
group by first_credit_month
order by first_credit_month
        desc limit 2000

Expected behavior (Required)

Get the correct result as trinno(which will return the result within one miniute).

Real behavior (Required)

The FE starrocks mysql nio pool thread stuck forever, and we can't get the result forever. See the cpu time and elapsed:
"

starrocks-mysql-nio-pool-230" #4749 daemon prio=5 os_prio=0 cpu=234995552.26ms elapsed=239447.22s tid=0x00007f62e8363800 nid=0x4d9a8 runnable [0x00007f5faf1de000]
java.lang.Thread.State: RUNNABLE
at java.util.stream.Sink$ChainedReference.begin(java.base@11.0.18/Sink.java:253)
at java.util.stream.AbstractPipeline.copyInto(java.base@11.0.18/AbstractPipeline.java:483)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(java.base@11.0.18/AbstractPipeline.java:474)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(java.base@11.0.18/ReduceOps.java:913)
at java.util.stream.AbstractPipeline.evaluate(java.base@11.0.18/AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(java.base@11.0.18/ReferencePipeline.java:578)
at com.starrocks.planner.JoinNode.candidatesOfSlotExprsForChild(JoinNode.java:282)
at com.starrocks.planner.JoinNode.pushDownRuntimeFiltersForChild(JoinNode.java:291)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(JoinNode.java:309)
at com.starrocks.planner.PlanNode.tryPushdownRuntimeFilterToChild(PlanNode.java:907)
at com.starrocks.planner.PlanNode.pushdownRuntimeFilterForChildOrAccept(PlanNode.java:936)
at com.starrocks.planner.JoinNode.pushDownRuntimeFiltersForChild(JoinNode.java:289)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(JoinNode.java:308)
at com.starrocks.planner.PlanNode.tryPushdownRuntimeFilterToChild(PlanNode.java:907)
at com.starrocks.planner.PlanNode.pushdownRuntimeFilterForChildOrAccept(PlanNode.java:936)
at com.starrocks.planner.JoinNode.pushDownRuntimeFiltersForChild(JoinNode.java:289)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(JoinNode.java:308)
at com.starrocks.planner.PlanNode.pushDownRuntimeFilters(PlanNode.java:809)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(JoinNode.java:313)
at com.starrocks.planner.PlanNode.tryPushdownRuntimeFilterToChild(PlanNode.java:907)
at com.starrocks.planner.PlanNode.pushdownRuntimeFilterForChildOrAccept(PlanNode.java:936)
at com.starrocks.planner.JoinNode.pushDownRuntimeFiltersForChild(JoinNode.java:289)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(JoinNode.java:308)
at com.starrocks.planner.PlanNode.tryPushdownRuntimeFilterToChild(PlanNode.java:907)
at com.starrocks.planner.PlanNode.pushdownRuntimeFilterForChildOrAccept(PlanNode.java:936)
at com.starrocks.planner.JoinNode.pushDownRuntimeFiltersForChild(JoinNode.java:289)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(JoinNode.java:308)
at com.starrocks.planner.PlanNode.tryPushdownRuntimeFilterToChild(PlanNode.java:907)
at com.starrocks.planner.PlanNode.pushdownRuntimeFilterForChildOrAccept(PlanNode.java:936)
at com.starrocks.planner.JoinNode.pushDownRuntimeFiltersForChild(JoinNode.java:289)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(JoinNode.java:308)
at com.starrocks.planner.PlanNode.pushDownRuntimeFilters(PlanNode.java:809)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(JoinNode.java:313)
at com.starrocks.planner.PlanNode.pushDownRuntimeFilters(PlanNode.java:809)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(JoinNode.java:313)
at com.starrocks.planner.PlanNode.tryPushdownRuntimeFilterToChild(PlanNode.java:907)
at com.starrocks.planner.PlanNode.pushdownRuntimeFilterForChildOrAccept(PlanNode.java:936)
at com.starrocks.planner.JoinNode.pushDownRuntimeFiltersForChild(JoinNode.java:289)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(JoinNode.java:308)
at com.starrocks.planner.PlanNode.tryPushdownRuntimeFilterToChild(PlanNode.java:907)
at com.starrocks.planner.PlanNode.pushdownRuntimeFilterForChildOrAccept(PlanNode.java:936)
at com.starrocks.planner.JoinNode.pushDownRuntimeFiltersForChild(JoinNode.java:289)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(JoinNode.java:308)
at com.starrocks.planner.PlanNode.pushDownRuntimeFilters(PlanNode.java:809)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(JoinNode.java:313)
at com.starrocks.planner.PlanNode.tryPushdownRuntimeFilterToChild(PlanNode.java:907)
at com.starrocks.planner.PlanNode.pushdownRuntimeFilterForChildOrAccept(PlanNode.java:936)
at com.starrocks.planner.JoinNode.pushDownRuntimeFiltersForChild(JoinNode.java:289)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(JoinNode.java:308)
at com.starrocks.planner.PlanNode.pushDownRuntimeFilters(PlanNode.java:809)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(JoinNode.java:313)
at com.starrocks.planner.PlanNode.tryPushdownRuntimeFilterToChild(PlanNode.java:907)
at com.starrocks.planner.PlanNode.pushdownRuntimeFilterForChildOrAccept(PlanNode.java:936)
at com.starrocks.planner.JoinNode.pushDownRuntimeFiltersForChild(JoinNode.java:289)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(JoinNode.java:308)
at com.starrocks.planner.PlanNode.tryPushdownRuntimeFilterToChild(PlanNode.java:907)
at com.starrocks.planner.PlanNode.pushdownRuntimeFilterForChildOrAccept(PlanNode.java:936)
at com.starrocks.planner.JoinNode.pushDownRuntimeFiltersForChild(JoinNode.java:289)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(JoinNode.java:308)
at com.starrocks.planner.PlanNode.pushDownRuntimeFilters(PlanNode.java:809)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(JoinNode.java:313)
at com.starrocks.planner.PlanNode.tryPushdownRuntimeFilterToChild(PlanNode.java:907)
at com.starrocks.planner.PlanNode.pushdownRuntimeFilterForChildOrAccept(PlanNode.java:936)
at com.starrocks.planner.JoinNode.pushDownRuntimeFiltersForChild(JoinNode.java:289)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(JoinNode.java:308)
at com.starrocks.planner.PlanNode.tryPushdownRuntimeFilterToChild(PlanNode.java:907)
at com.starrocks.planner.PlanNode.pushdownRuntimeFilterForChildOrAccept(PlanNode.java:936)
at com.starrocks.planner.JoinNode.pushDownRuntimeFiltersForChild(JoinNode.java:289)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(JoinNode.java:308)
at com.starrocks.planner.PlanNode.pushDownRuntimeFilters(PlanNode.java:809)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(JoinNode.java:313)
at com.starrocks.planner.PlanNode.pushDownRuntimeFilters(PlanNode.java:809)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(JoinNode.java:313)
at com.starrocks.planner.PlanNode.pushDownRuntimeFilters(PlanNode.java:809)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(JoinNode.java:313)
at com.starrocks.planner.PlanNode.pushDownRuntimeFilters(PlanNode.java:809)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(JoinNode.java:313)
at com.starrocks.planner.PlanNode.tryPushdownRuntimeFilterToChild(PlanNode.java:907)
at com.starrocks.planner.PlanNode.pushdownRuntimeFilterForChildOrAccept(PlanNode.java:936)
at com.starrocks.planner.JoinNode.pushDownRuntimeFiltersForChild(JoinNode.java:289)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(JoinNode.java:308)
at com.starrocks.planner.PlanNode.pushDownRuntimeFilters(PlanNode.java:809)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(JoinNode.java:313)
at com.starrocks.planner.PlanNode.pushDownRuntimeFilters(PlanNode.java:809)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(JoinNode.java:313)
at com.starrocks.planner.PlanNode.pushDownRuntimeFilters(PlanNode.java:809)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(JoinNode.java:313)
at com.starrocks.planner.PlanNode.tryPushdownRuntimeFilterToChild(PlanNode.java:907)
at com.starrocks.planner.PlanNode.pushdownRuntimeFilterForChildOrAccept(PlanNode.java:936)
at com.starrocks.planner.JoinNode.pushDownRuntimeFiltersForChild(JoinNode.java:289)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(JoinNode.java:308)
at com.starrocks.planner.PlanNode.tryPushdownRuntimeFilterToChild(PlanNode.java:907)
at com.starrocks.planner.PlanNode.pushdownRuntimeFilterForChildOrAccept(PlanNode.java:936)
at com.starrocks.planner.JoinNode.pushDownRuntimeFiltersForChild(JoinNode.java:289)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(JoinNode.java:308)
at com.starrocks.planner.PlanNode.pushDownRuntimeFilters(PlanNode.java:809)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(JoinNode.java:313)
at com.starrocks.planner.PlanNode.tryPushdownRuntimeFilterToChild(PlanNode.java:907)
at com.starrocks.planner.PlanNode.pushdownRuntimeFilterForChildOrAccept(PlanNode.java:936)
at com.starrocks.planner.JoinNode.pushDownRuntimeFiltersForChild(JoinNode.java:289)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(JoinNode.java:308)
at com.starrocks.planner.PlanNode.pushDownRuntimeFilters(PlanNode.java:809)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(Unknown Source)
at com.starrocks.planner.PlanNode.pushDownRuntimeFilters(Unknown Source)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(Unknown Source)
at com.starrocks.planner.PlanNode.pushDownRuntimeFilters(Unknown Source)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(Unknown Source)
at com.starrocks.planner.PlanNode.tryPushdownRuntimeFilterToChild(Unknown Source)
at com.starrocks.planner.PlanNode.pushdownRuntimeFilterForChildOrAccept(Unknown Source)
at com.starrocks.planner.JoinNode.pushDownRuntimeFiltersForChild(Unknown Source)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(Unknown Source)
at com.starrocks.planner.PlanNode.tryPushdownRuntimeFilterToChild(Unknown Source)
at com.starrocks.planner.PlanNode.pushdownRuntimeFilterForChildOrAccept(Unknown Source)
at com.starrocks.planner.JoinNode.pushDownRuntimeFiltersForChild(Unknown Source)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(Unknown Source)
at com.starrocks.planner.PlanNode.tryPushdownRuntimeFilterToChild(Unknown Source)
at com.starrocks.planner.PlanNode.pushdownRuntimeFilterForChildOrAccept(Unknown Source)
at com.starrocks.planner.JoinNode.pushDownRuntimeFiltersForChild(Unknown Source)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(Unknown Source)
at com.starrocks.planner.PlanNode.tryPushdownRuntimeFilterToChild(Unknown Source)
at com.starrocks.planner.PlanNode.pushdownRuntimeFilterForChildOrAccept(Unknown Source)
at com.starrocks.planner.JoinNode.pushDownRuntimeFiltersForChild(Unknown Source)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(Unknown Source)
at com.starrocks.planner.PlanNode.tryPushdownRuntimeFilterToChild(Unknown Source)
at com.starrocks.planner.PlanNode.pushdownRuntimeFilterForChildOrAccept(Unknown Source)
at com.starrocks.planner.JoinNode.pushDownRuntimeFiltersForChild(Unknown Source)
at com.starrocks.planner.JoinNode.pushDownRuntimeFilters(Unknown Source)
at com.starrocks.planner.PlanNode.tryPushdownRuntimeFilterToChild(Unknown Source)
at com.starrocks.planner.PlanNode.pushdownRuntimeFilterForChildOrAccept(Unknown Source)
at com.starrocks.planner.ProjectNode.pushDownRuntimeFilters(Unknown Source)
at com.starrocks.planner.SortNode.buildRuntimeFilters(Unknown Source)
at com.starrocks.sql.plan.PlanFragmentBuilder$PhysicalPlanTranslator.buildPartialTopNFragment(PlanFragmentBuilder.java:2161)
at com.starrocks.sql.plan.PlanFragmentBuilder$PhysicalPlanTranslator.visitPhysicalTopN(PlanFragmentBuilder.java:2041)
at com.starrocks.sql.plan.PlanFragmentBuilder$PhysicalPlanTranslator.visitPhysicalTopN(PlanFragmentBuilder.java:380)
at com.starrocks.sql.optimizer.operator.physical.PhysicalTopNOperator.accept(PhysicalTopNOperator.java:141)
at com.starrocks.sql.plan.PlanFragmentBuilder$PhysicalPlanTranslator.visit(PlanFragmentBuilder.java:397)
at com.starrocks.sql.plan.PlanFragmentBuilder$PhysicalPlanTranslator.visitPhysicalTopN(PlanFragmentBuilder.java:2037)
at com.starrocks.sql.plan.PlanFragmentBuilder$PhysicalPlanTranslator.visitPhysicalTopN(PlanFragmentBuilder.java:380)
at com.starrocks.sql.optimizer.operator.physical.PhysicalTopNOperator.accept(PhysicalTopNOperator.java:141)
at com.starrocks.sql.plan.PlanFragmentBuilder$PhysicalPlanTranslator.visit(PlanFragmentBuilder.java:397)
at com.starrocks.sql.plan.PlanFragmentBuilder$PhysicalPlanTranslator.translate(PlanFragmentBuilder.java:391)
at com.starrocks.sql.plan.PlanFragmentBuilder.createPhysicalPlan(PlanFragmentBuilder.java:227)
at com.starrocks.sql.StatementPlanner.createQueryPlan(StatementPlanner.java:201)
at com.starrocks.sql.StatementPlanner.plan(StatementPlanner.java:130)
at com.starrocks.sql.StatementPlanner.plan(StatementPlanner.java:87)
at com.starrocks.qe.StmtExecutor.execute(StmtExecutor.java:486)
at com.starrocks.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:394)
at com.starrocks.qe.ConnectProcessor.dispatch(ConnectProcessor.java:588)
at com.starrocks.qe.ConnectProcessor.processOnce(ConnectProcessor.java:872)
at com.starrocks.mysql.nio.ReadListener.lambda$handleEvent$0(ReadListener.java:69)
at com.starrocks.mysql.nio.ReadListener$$Lambda$434/0x000000080081a040.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.18/ThreadPoolExecutor.java:1128)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.18/ThreadPoolExecutor.java:628)
at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)

StarRocks version (Required)

  • 3.2.3
@ever4Kenny ever4Kenny added the type/bug Something isn't working label May 14, 2024
@ever4Kenny
Copy link
Author

And it is related to pusing down the count distinct if 'month' column filter in the outer select to the inner 'substring' on 'apply_month'.
If we reduce the 'count distinct' conditions count to half, starrocks might return the result.

@ever4Kenny ever4Kenny changed the title FE starrocks-mysql-nio-pool thread dead loop caught in JoinNode.pushDownRuntimeFilters [FE][Dead Loop]starrocks-mysql-nio-pool thread dead loop caught in JoinNode.pushDownRuntimeFilters May 14, 2024
@ever4Kenny
Copy link
Author

Even the explain query on the select will cause the fe stuck forever.

@ever4Kenny
Copy link
Author

ever4Kenny commented May 14, 2024 •
edited

Plus, killing the query will not work: the thread is still hanging there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type/bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ever4Kenny