`test_wrap_index_cupy`, `test_from_dlpack_cupy` and 361 cuda tests fail with 'PerformanceWarning' #3111

ianna · 2024-05-13T12:37:12Z

Version of Awkward Array

master branch (2.6.4)

Description and code to reproduce

The test_wrap_index_cupy test fails with a warning: cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up to populate the persistent cache, this may take a few seconds and will be improved in a future release...

awkward array master branch on 13.05.2024 (v 2.6.4)
Python 3.11.9
cupy 13.1.0

_____________________________________________ test_wrap_index_cupy _____________________________________________

    def test_wrap_index_cupy():
        cp = pytest.importorskip("cupy")
        data = cp.arange(10, dtype=cp.int64)
        index = ak.index.Index64(data)
        other_data = cp.asarray(index)
>       assert cp.shares_memory(data, other_data)

cp         = <module 'cupy' from '/home/ianna/anaconda3/envs/awkward-py3.11/lib/python3.11/site-packages/cupy/__init__.py'>
data       = array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
index      = <Index dtype='int64' len='10'>[0 1 2 3 4 5 6 7 8 9]</Index>
other_data = array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

tests/test_2327_array_interface.py:16: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../anaconda3/envs/awkward-py3.11/lib/python3.11/site-packages/cupy/_misc/memory_ranges.py:42: in shares_memory
    return bool((x != y).any())
        a          = array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
        a_ptrs     = array([128746273112064, 128746273112072, 128746273112080, 128746273112088,
       128746273112096, 128746273112104, 128746273112112, 128746273112120,
       128746273112128, 128746273112136], dtype=uint64)
        b          = array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
        b_ptrs     = array([[128746273112064],
       [128746273112072],
       [128746273112080],
       [128746273112088],
       [128746...
       [128746273112112],
       [128746273112120],
       [128746273112128],
       [128746273112136]], dtype=uint64)
        max_work   = None
        x          = array([[0],
       [1],
       [2],
       [3],
       [4],
       [5],
       [6],
       [7],
       [8],
       [9]])
        y          = array([[ 1],
       [ 2],
       [ 3],
       [ 4],
       [ 5],
       [ 6],
       [ 7],
       [ 8],
       [ 9],
       [10]])
cupy/_core/core.pyx:1173: in cupy._core.core._ndarray_base.any
    ???
cupy/_core/core.pyx:1175: in cupy._core.core._ndarray_base.any
    ???
cupy/_core/_routines_logic.pyx:12: in cupy._core._routines_logic._ndarray_any
    ???
cupy/_core/_reduction.pyx:618: in cupy._core._reduction._SimpleReductionKernel.__call__
    ???
cupy/_core/_reduction.pyx:370: in cupy._core._reduction._AbstractReductionKernel._call
    ???
cupy/_core/_cub_reduction.pyx:689: in cupy._core._cub_reduction._try_to_call_cub_reduction
    ???
cupy/_core/_cub_reduction.pyx:526: in cupy._core._cub_reduction._launch_cub
    ???
cupy/_core/_cub_reduction.pyx:461: in cupy._core._cub_reduction._cub_two_pass_launch
    ???
cupy/_util.pyx:64: in cupy._util.memoize.decorator.ret
    ???
cupy/_core/_cub_reduction.pyx:240: in cupy._core._cub_reduction._SimpleCubReductionKernel_get_cached_function
    ???
cupy/_core/_cub_reduction.pyx:223: in cupy._core._cub_reduction._create_cub_reduction_function
    ???
cupy/_core/core.pyx:2254: in cupy._core.core.compile_with_cache
    ???
../../../anaconda3/envs/awkward-py3.11/lib/python3.11/site-packages/cupy/cuda/compiler.py:484: in _compile_module_with_cache
    return _compile_with_cache_cuda(
        arch       = None
        backend    = 'nvrtc'
        cache_dir  = None
        cache_in_memory = False
        enable_cooperative_groups = False
        extra_source = '/*  Copyright 2008-2013 NVIDIA Corporation\n *  Copyright 2013 Filipe RNC Maia\n *\n *  Licensed under the Apache Lic...)\n#define CUDART_NAN              __longlong_as_double(0xfff8000000000000ULL)\n\n\n#endif  // CUPY_MATH_CONSTANTS_H\n'
        jitify     = True
        log_stream = None
        name_expressions = None
        options    = ('-I/home/ianna/anaconda3/envs/awkward-py3.11/lib/python3.11/site-packages/cupy/_core/include/cupy/_cccl/cub', '-I/hom...packages/cupy/_core/include/cupy/_cccl/libcudacxx', '-DFIRST_PASS=1', '--std=c++11', '-DCUB_DISABLE_BF16_SUPPORT', ...)
        source     = '#include <cupy/cuda_workaround.h>\n#include <cupy/complex.cuh>\n#include <cupy/carray.cuh>\n#include <cupy/atomics.cu...ge\n  }\n\n  if (_tid == 0) {\n      type_mid_out& out0 = *(_out0 + blockIdx.x);\n      POST_MAP(aggregate);\n  }\n}\n'
../../../anaconda3/envs/awkward-py3.11/lib/python3.11/site-packages/cupy/cuda/compiler.py:562: in _compile_with_cache_cuda
    ptx, mapping = compile_using_nvrtc(
        arch       = '86'
        backend    = 'nvrtc'
        base       = '//\n// Generated by NVIDIA NVVM Compiler\n//\n// Compiler Build ID: CL-34043715\n// Cuda compilation tools, release 12.4, V12.4.127\n// Based on NVVM 7.0.1\n//'
        cache_dir  = '/home/ianna/.cupy/kernel_cache'
        cache_in_memory = False
        cu_name    = 'c7e41b4c836f497da46ef4899af409792569f558.cubin.cu'
        enable_cooperative_groups = False
        env        = ('86', ('-I/home/ianna/anaconda3/envs/awkward-py3.11/lib/python3.11/site-packages/cupy/_core/include/cupy/_cccl/cub', ...udacxx', '-DFIRST_PASS=1', '--std=c++11', '-DCUB_DISABLE_BF16_SUPPORT', ...), (12, 4), 'nvrtc', '-arch=sm_86', 'cubin')
        extra_source = '/*  Copyright 2008-2013 NVIDIA Corporation\n *  Copyright 2013 Filipe RNC Maia\n *\n *  Licensed under the Apache Lic...)\n#define CUDART_NAN              __longlong_as_double(0xfff8000000000000ULL)\n\n\n#endif  // CUPY_MATH_CONSTANTS_H\n'
        is_jitify_requested = False
        jitify     = True
        key_src    = b'(\'86\', (\'-I/home/ianna/anaconda3/envs/awkward-py3.11/lib/python3.11/site-packages/cupy/_core/include/cupy/_cccl/c...)\n#define CUDART_NAN              __longlong_as_double(0xfff8000000000000ULL)\n\n\n#endif  // CUPY_MATH_CONSTANTS_H\n'
        log_stream = None
        mod        = <cupy.cuda.function.Module object at 0x7518289ba340>
        name       = 'c7e41b4c836f497da46ef4899af409792569f558.cubin'
        name_expressions = None
        options    = ('-I/home/ianna/anaconda3/envs/awkward-py3.11/lib/python3.11/site-packages/cupy/_core/include/cupy/_cccl/cub', '-I/hom...packages/cupy/_core/include/cupy/_cccl/libcudacxx', '-DFIRST_PASS=1', '--std=c++11', '-DCUB_DISABLE_BF16_SUPPORT', ...)
        path       = '/home/ianna/.cupy/kernel_cache/c7e41b4c836f497da46ef4899af409792569f558.cubin'
        source     = '#include <cupy/cuda_workaround.h>\n#include <cupy/complex.cuh>\n#include <cupy/carray.cuh>\n#include <cupy/atomics.cu...ge\n  }\n\n  if (_tid == 0) {\n      type_mid_out& out0 = *(_out0 + blockIdx.x);\n      POST_MAP(aggregate);\n  }\n}\n'
../../../anaconda3/envs/awkward-py3.11/lib/python3.11/site-packages/cupy/cuda/compiler.py:319: in compile_using_nvrtc
    return _compile(source, options, cu_path,
        _compile   = <function compile_using_nvrtc.<locals>._compile at 0x7518289bcfe0>
        arch       = '86'
        cache_in_memory = False
        cu_file    = <_io.TextIOWrapper name='/tmp/tmpll4ozgn9/c7e41b4c836f497da46ef4899af409792569f558.cubin.cu' mode='w' encoding='ANSI_X3.4-1968'>
        cu_path    = '/tmp/tmpll4ozgn9/c7e41b4c836f497da46ef4899af409792569f558.cubin.cu'
        filename   = 'c7e41b4c836f497da46ef4899af409792569f558.cubin.cu'
        jitify     = True
        log_stream = None
        name_expressions = None
        options    = ('-I/home/ianna/anaconda3/envs/awkward-py3.11/lib/python3.11/site-packages/cupy/_core/include/cupy/_cccl/cub', '-I/hom...packages/cupy/_core/include/cupy/_cccl/libcudacxx', '-DFIRST_PASS=1', '--std=c++11', '-DCUB_DISABLE_BF16_SUPPORT', ...)
        root_dir   = '/tmp/tmpll4ozgn9'
        source     = '#include <cupy/cuda_workaround.h>\n#include <cupy/complex.cuh>\n#include <cupy/carray.cuh>\n#include <cupy/atomics.cu...ge\n  }\n\n  if (_tid == 0) {\n      type_mid_out& out0 = *(_out0 + blockIdx.x);\n      POST_MAP(aggregate);\n  }\n}\n'
../../../anaconda3/envs/awkward-py3.11/lib/python3.11/site-packages/cupy/cuda/compiler.py:290: in _compile
    options, headers, include_names = _jitify_prep(
        arch       = '86'
        arch_opt   = '-arch=sm_86'
        cu_path    = '/tmp/tmpll4ozgn9/c7e41b4c836f497da46ef4899af409792569f558.cubin.cu'
        jitify     = True
        log_stream = None
        method     = 'cubin'
        name_expressions = None
        options    = ('-I/home/ianna/anaconda3/envs/awkward-py3.11/lib/python3.11/site-packages/cupy/_core/include/cupy/_cccl/cub', '-I/hom...packages/cupy/_core/include/cupy/_cccl/libcudacxx', '-DFIRST_PASS=1', '--std=c++11', '-DCUB_DISABLE_BF16_SUPPORT', ...)
        source     = '#include <cupy/cuda_workaround.h>\n#include <cupy/complex.cuh>\n#include <cupy/carray.cuh>\n#include <cupy/atomics.cu...ge\n  }\n\n  if (_tid == 0) {\n      type_mid_out& out0 = *(_out0 + blockIdx.x);\n      POST_MAP(aggregate);\n  }\n}\n'
../../../anaconda3/envs/awkward-py3.11/lib/python3.11/site-packages/cupy/cuda/compiler.py:233: in _jitify_prep
    jitify._init_module()
        core       = <module 'cupy._core.core' from '/home/ianna/anaconda3/envs/awkward-py3.11/lib/python3.11/site-packages/cupy/_core/core.cpython-311-x86_64-linux-gnu.so'>
        cu_path    = '/tmp/tmpll4ozgn9/c7e41b4c836f497da46ef4899af409792569f558.cubin.cu'
        jitify     = <module 'cupy.cuda.jitify' from '/home/ianna/anaconda3/envs/awkward-py3.11/lib/python3.11/site-packages/cupy/cuda/jitify.cpython-311-x86_64-linux-gnu.so'>
        options    = ('-I/home/ianna/anaconda3/envs/awkward-py3.11/lib/python3.11/site-packages/cupy/_core/include/cupy/_cccl/cub', '-I/hom...packages/cupy/_core/include/cupy/_cccl/libcudacxx', '-DFIRST_PASS=1', '--std=c++11', '-DCUB_DISABLE_BF16_SUPPORT', ...)
        source     = '#include <cupy/cuda_workaround.h>\n#include <cupy/complex.cuh>\n#include <cupy/carray.cuh>\n#include <cupy/atomics.cu...ge\n  }\n\n  if (_tid == 0) {\n      type_mid_out& out0 = *(_out0 + blockIdx.x);\n      POST_MAP(aggregate);\n  }\n}\n'
cupy/cuda/jitify.pyx:220: in cupy.cuda.jitify._init_module
    ???
cupy/cuda/jitify.pyx:244: in cupy.cuda.jitify._init_module
    ???
cupy/cuda/jitify.pyx:217: in cupy.cuda.jitify._init_cupy_headers
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

>   ???
E   cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up to populate the persistent cache, this may take a few seconds and will be improved in a future release...


cupy/cuda/jitify.pyx:189: PerformanceWarning
=========================================== short test summary info ============================================
FAILED tests/test_2327_array_interface.py::test_wrap_index_cupy - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up to populate the persistent cach...

The same for other 361 cuda tests:

=========================== short test summary info ============================
SKIPPED [1] tests-cuda/test_2922a_new_cuda_kernels.py:376: could not import 'pyarrow': No module named 'pyarrow'
SKIPPED [1] tests-cuda/test_2922a_new_cuda_kernels.py:394: could not import 'pyarrow': No module named 'pyarrow'
SKIPPED [1] tests-cuda/test_2922a_new_cuda_kernels.py:414: could not import 'pyarrow': No module named 'pyarrow'
SKIPPED [1] tests-cuda/test_3065c_cuda_kernels.py:203: could not import 'pyarrow': No module named 'pyarrow'
FAILED tests-cuda/test_1276_cuda_transfers.py::test_tocuda_unimplementedkernels6 - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_1276_cuda_transfers.py::test_tocuda_unimplementedkernels8 - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_1276_cuda_transfers.py::test_tocuda_unimplementedkernels11 - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_1276_cuda_transfers.py::test_tocuda_unimplementedkernels13 - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_1276_cuda_transfers.py::test_tocuda_unimplementedkernels14 - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_1276_cuda_transfers.py::test_tocuda_unimplementedkernels15 - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_1276_from_cupy.py::test_add - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_1276_from_cupy.py::test_add_2 - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_0184_concatenate_operation_records - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_1904_drop_none_BitMaskedArray_RecordArray_NumpyArray - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_1904_drop_none_ByteMaskedArray_RecordArray_NumpyArray - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_1904_drop_none_IndexedOptionArray_NumpyArray_outoforder - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_1904_drop_none_ListArray_IndexedOptionArray_RecordArray_NumpyArray - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_1904_drop_none_all_axes - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_1914_improved_axis_to_posaxis_is_none - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_1914_improved_axis_to_posaxis_singletons - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_2623_optiontype_outside_record_strings_ByteMaskedArray - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_2623_optiontype_outside_record_strings_IndexedOptionArray - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_2564_string_broadcast_regular_string_mixed_invalid - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_2564_string_broadcast_regular_string_mixed_valid - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_2564_string_broadcast_regular_string_mixed_below - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_2564_string_broadcast_regular_string_string_valid - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_2549_list_nominal_type_string_class - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_2425_forms_from_type_categorical - ImportError: to use ak.str.to_categorical, you must install pyarrow:
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_2425_forms_from_type_categorical_option - ImportError: to use ak.str.to_categorical, you must install pyarrow:
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_2411_cartesian_axis_validation_simple - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_2410_string_broadcast_deep_string_string - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_2410_string_broadcast_deep_numbers_string - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_2410_string_broadcast_same_depth - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_2385_with_field_empty_record_union_record - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_2240_simplify_merge_as_union_many - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_2185_merge_option_of_records - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_2185_merge_option_of_records_2 - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_2185_merge_option_of_records_3 - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_2108_fill_none_indexed - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_2082_broadcast_zero_size[0] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_2082_broadcast_zero_size[1] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_2082_broadcast_zero_size[2] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_2078_array_function_wrap - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_2071_unflatten_non_packed_indexed_counts - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_2071_unflatten_non_packed_counts_indexed_layout - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_2064_fill_none_record_axis_none - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_2064_fill_none_record_axis_last - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_2064_fill_none_record_option_outside_record - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_2058_merge_numpy_array - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_2058_merge_numpy_array_regular - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_2058_merge_numpy_array_regular_mergebool_true - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_2021_check_TypeTracerArray_in_ak_where - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_1943_regular_indexing_non_list_index - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_1826_ravel_preserve_none - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_1823_fill_none_axis_none - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_1671_categorical_type - ImportError: to use ak.str.to_categorical, you must install pyarrow:
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_1671_categorical_type_to_categorical - ImportError: to use ak.str.to_categorical, you must install pyarrow:
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_1604_preserve_form_in_concatenate_ListOffsetArray - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_1604_preserve_form_in_concatenate_ListOffsetArray_ListOffsetArray - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_1604_preserve_form_in_concatenate_BitMaskedArray - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922a_new_cuda_kernels.py::test_0111_jagged_and_masked_getitem_numpyarray - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922b_new_cuda_kernels.py::test_2651_parameter_union - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922b_new_cuda_kernels.py::test_1928_replace_simplify_method_with_classmethod_constructor_indexed_of_union - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922b_new_cuda_kernels.py::test_1928_replace_simplify_method_with_classmethod_constructor_indexedoption_of_union - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922b_new_cuda_kernels.py::test_1928_replace_simplify_method_with_classmethod_constructor_indexedoption_of_union_of_option_1 - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922b_new_cuda_kernels.py::test_1928_replace_simplify_method_with_classmethod_constructor_indexedoption_of_union_of_option_2 - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922b_new_cuda_kernels.py::test_1928_replace_simplify_method_with_classmethod_constructor_indexedoption_of_union_of_option_1_2 - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922b_new_cuda_kernels.py::test_0078_argcross_and_cross_axis0 - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922b_new_cuda_kernels.py::test_0078_argcross_and_cross_axis1 - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922b_new_cuda_kernels.py::test_0078_argcross_and_cross_axis2 - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922b_new_cuda_kernels.py::test_0078_argcross_and_cross_argcartesian - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922b_new_cuda_kernels.py::test_0078_argcross_and_cross_argcartesian_negative_axis - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922b_new_cuda_kernels.py::test_0093_simplify_uniontypes_and_optiontypes_concatenate - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922b_new_cuda_kernels.py::test_0093_simplify_uniontypes_and_optiontypes_indexedarray_merge - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922b_new_cuda_kernels.py::test_0093_simplify_uniontypes_and_optiontypes_indexedarray_simplify - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922b_new_cuda_kernels.py::test_0093_simplify_uniontypes_and_optiontypes_where - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922b_new_cuda_kernels.py::test_0093_simplify_uniontypes_and_optiontypes_unionarray_simplify_one - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922b_new_cuda_kernels.py::test_0093_simplify_uniontypes_and_optiontypes_unionarray_simplify - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922b_new_cuda_kernels.py::test_0093_simplify_uniontypes_and_optiontypes_merge_parameters - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922b_new_cuda_kernels.py::test_0093_simplify_uniontypes_and_optiontypes_unionarray_merge - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922b_new_cuda_kernels.py::test_0093_simplify_uniontypes_and_optiontypes_numpyarray_merge - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922b_new_cuda_kernels.py::test_0093_simplify_uniontypes_and_optiontypes_listarray_merge - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_2922b_new_cuda_kernels.py::test_0193_is_none_axis_parameter - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_0449_merge_many_arrays_in_one_pass_concatenate - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_0449_merge_many_arrays_in_one_pass_lists - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_0449_merge_many_arrays_in_one_pass_records - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_0449_merge_many_arrays_in_one_pass_tuples - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_0449_merge_many_arrays_in_one_pass_indexed - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_0449_merge_many_arrays_in_one_pass_reverse_indexed - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_0449_merge_many_arrays_in_one_pass_bytemasked - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_0449_merge_many_arrays_in_one_pass_union - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_0449_merge_many_arrays_in_one_pass_union_option - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_0449_merge_many_arrays_in_one_pass_strings - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right0-left0] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right0-left1] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right0-left2] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right0-left3] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right0-left4] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right0-left5] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right0-left6] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right0-left7] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right0-left8] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right0-left9] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right0-left10] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right0-left11] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right0-left12] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right0-left13] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right0-left14] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right1-left0] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right1-left1] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right1-left2] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right1-left3] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right1-left4] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right1-left5] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right1-left6] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right1-left7] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right1-left8] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right1-left9] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right1-left10] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right1-left11] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right1-left12] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right1-left13] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right1-left14] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right2-left0] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right2-left1] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right2-left2] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right2-left3] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right2-left4] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right2-left5] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right2-left6] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right2-left7] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right2-left8] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right2-left9] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right2-left10] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right2-left11] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right2-left12] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right2-left13] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right2-left14] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right3-left0] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right3-left1] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right3-left2] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right3-left3] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right3-left4] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right3-left5] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right3-left6] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right3-left7] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right3-left8] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right3-left9] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right3-left10] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right3-left11] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right3-left12] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right3-left13] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right3-left14] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right4-left0] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right4-left1] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right4-left2] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right4-left3] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right4-left4] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right4-left5] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right4-left6] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right4-left7] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right4-left8] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right4-left9] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right4-left10] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right4-left11] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right4-left12] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right4-left13] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right4-left14] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right5-left0] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right5-left1] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right5-left2] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right5-left3] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right5-left4] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right5-left5] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right5-left6] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right5-left7] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right5-left8] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right5-left9] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right5-left10] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right5-left11] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right5-left12] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right5-left13] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right5-left14] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right6-left0] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right6-left1] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right6-left2] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right6-left3] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right6-left4] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right6-left5] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right6-left6] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right6-left7] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right6-left8] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right6-left9] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right6-left10] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right6-left11] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right6-left12] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right6-left13] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right6-left14] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right7-left0] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right7-left1] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right7-left2] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right7-left3] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right7-left4] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right7-left5] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right7-left6] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right7-left7] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right7-left8] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right7-left9] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right7-left10] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right7-left11] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right7-left12] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right7-left13] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right7-left14] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right8-left0] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right8-left1] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right8-left2] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right8-left3] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right8-left4] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right8-left5] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right8-left6] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right8-left7] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right8-left8] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right8-left9] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right8-left10] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right8-left11] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right8-left12] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right8-left13] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right8-left14] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right9-left0] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right9-left1] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right9-left2] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right9-left3] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right9-left4] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right9-left5] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right9-left6] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right9-left7] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right9-left8] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right9-left9] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right9-left10] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right9-left11] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right9-left12] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right9-left13] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right9-left14] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right10-left0] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right10-left1] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right10-left2] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right10-left3] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right10-left4] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right10-left5] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right10-left6] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right10-left7] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right10-left8] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right10-left9] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right10-left10] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right10-left11] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right10-left12] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right10-left13] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right10-left14] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right11-left0] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right11-left1] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right11-left2] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right11-left3] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right11-left4] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right11-left5] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right11-left6] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right11-left7] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right11-left8] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right11-left9] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right11-left10] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right11-left11] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right11-left12] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right11-left13] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right11-left14] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right12-left0] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right12-left1] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right12-left2] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right12-left3] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right12-left4] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right12-left5] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right12-left6] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right12-left7] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right12-left8] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right12-left9] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right12-left10] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right12-left11] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right12-left12] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right12-left13] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right12-left14] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right13-left0] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right13-left1] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right13-left2] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right13-left3] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right13-left4] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right13-left5] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right13-left6] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right13-left7] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right13-left8] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right13-left9] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right13-left10] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right13-left11] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right13-left12] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right13-left13] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right13-left14] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right14-left0] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right14-left1] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right14-left2] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right14-left3] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right14-left4] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right14-left5] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right14-left6] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right14-left7] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right14-left8] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right14-left9] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right14-left10] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right14-left11] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right14-left12] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right14-left13] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065a_cuda_kernels.py::test_2860_enforce_concatenated_form_symmetric[right14-left14] - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065b_cuda_kernels.py::test_0582_propagate_context_in_broadcast_and_apply_firsts - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065b_cuda_kernels.py::test_0582_propagate_context_in_broadcast_and_apply_cartesian - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065b_cuda_kernels.py::test_0193_is_none_axis_parameter - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065b_cuda_kernels.py::test_0493_zeros_ones_full_like - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065b_cuda_kernels.py::test_0496_provide_local_index - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065b_cuda_kernels.py::test_0527_fix_unionarray_ufuncs_and_parameters_in_merging_0459 - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065b_cuda_kernels.py::test_0527_fix_unionarray_ufuncs_and_parameters_in_merging_0522 - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065b_cuda_kernels.py::test_0959_getitem_array_implementation_UnionArray_NumpyArray - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065b_cuda_kernels.py::test_0959_getitem_array_implementation_IndexedOptionArray_RecordArray_NumpyArray - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065b_cuda_kernels.py::test_0959_getitem_array_implementation_IndexedOptionArray_NumpyArray - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065b_cuda_kernels.py::test_0959_getitem_array_implementation_RecordArray_NumpyArray - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065b_cuda_kernels.py::test_0959_getitem_array_implementation_RegularArray_NumpyArray - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065b_cuda_kernels.py::test_0959_getitem_array_implementation_ListArray_NumpyArray - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065b_cuda_kernels.py::test_0959_getitem_array_implementation_UnionArray_RecordArray_NumpyArray - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065b_cuda_kernels.py::test_0959_getitem_array_implementation_RecordArray_NumpyArray_lazy - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065b_cuda_kernels.py::test_0959_getitem_array_implementation_RegularArray_RecordArray_NumpyArray - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065b_cuda_kernels.py::test_0959_getitem_array_implementation_ListArray_RecordArray_NumpyArray - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065c_cuda_kernels.py::test_0546_fill_none_replacement_value_type - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065c_cuda_kernels.py::test_0912_packed_indexed_option_array - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065c_cuda_kernels.py::test_0912_packed_union_array - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065c_cuda_kernels.py::test_0627_behavior_from_dict_of_arrays - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065c_cuda_kernels.py::test_0713_getitem_field_should_simplify_optiontype - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065c_cuda_kernels.py::test_0766_prevent_combinations_of_characters_cartesian - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065c_cuda_kernels.py::test_0866_getitem_field_and_flatten_unions_getitem_field - AttributeError: while trying to get field 'a', an exception occurred:
FAILED tests-cuda/test_3065c_cuda_kernels.py::test_2426_is_equal_to_equal_union - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065c_cuda_kernels.py::test_2426_is_equal_to_unequal_union - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065c_cuda_kernels.py::test_0198_tutorial_documentation_1_firsts - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065c_cuda_kernels.py::test_0198_tutorial_documentation_1_singletons - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065c_cuda_kernels.py::test_0198_tutorial_documentation_1_flatten0 - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065c_cuda_kernels.py::test_0334_fully_broadcastable_where - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065c_cuda_kernels.py::test_0150_flatten_axis_none_0866 - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3065c_cuda_kernels.py::test_0150_flatten_0198 - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3086_cuda_concatenate.py::test_0184_concatenate_number - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3086_cuda_concatenate.py::test_0184_negative_axis_concatenate - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3086_cuda_concatenate.py::test_0184_broadcast_and_apply_levels_concatenate - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3086_cuda_concatenate.py::test_0184_list_array_concatenate - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3086_cuda_concatenate.py::test_0184_indexed_array_concatenate - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3086_cuda_concatenate.py::test_0184_listoffsetarray_concatenate - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3086_cuda_concatenate.py::test_0184_even_more - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3086_cuda_concatenate.py::test_1586_regular_option - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3086_cuda_concatenate.py::test_1586_option_option - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3086_cuda_concatenate.py::test_1586_option_option_axis1 - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3086_cuda_concatenate.py::test_1586_regular_option_axis1 - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3086_cuda_concatenate.py::test_1586_option_regular - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3086_cuda_concatenate.py::test_1586_option_regular_axis1 - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3086_cuda_concatenate.py::test_2663_broadcast_tuples_record_record - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
FAILED tests-cuda/test_3086_cuda_flatten.py::test_flatten_UnionArray - cupy._util.PerformanceWarning: Jitify is performing a one-time only warm-up...
================= 361 failed, 122 passed, 4 skipped in 38.07s ==================

The text was updated successfully, but these errors were encountered:

ianna · 2024-05-13T15:00:00Z

# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
argcomplete               3.3.0                    pypi_0    pypi
awkward                   2.6.4                    pypi_0    pypi
awkward-cpp               33                       pypi_0    pypi
bzip2                     1.0.8                h5eee18b_6  
ca-certificates           2024.3.11            h06a4308_0  
colorlog                  6.8.2                    pypi_0    pypi
cuda-nvrtc                12.4.127             hd3aeb46_1    conda-forge
cuda-version              12.4                 h3060b56_3    conda-forge
cupy                      13.1.0          py311hf829483_4    conda-forge
cupy-core                 13.1.0          py311he1e6e68_4    conda-forge
distlib                   0.3.8                    pypi_0    pypi
fastrlock                 0.8.2           py311hb755f60_2    conda-forge
filelock                  3.14.0                   pypi_0    pypi
fsspec                    2024.3.1                 pypi_0    pypi
importlib-metadata        7.1.0                    pypi_0    pypi
iniconfig                 2.0.0                    pypi_0    pypi
ld_impl_linux-64          2.38                 h1181459_1  
libblas                   3.9.0           22_linux64_openblas    conda-forge
libcblas                  3.9.0           22_linux64_openblas    conda-forge
libcublas                 12.4.5.8             hd3aeb46_1    conda-forge
libcufft                  11.2.1.3             hd3aeb46_1    conda-forge
libcurand                 10.3.5.147           hd3aeb46_1    conda-forge
libcusolver               11.6.1.9             hd3aeb46_1    conda-forge
libcusparse               12.3.1.170           hd3aeb46_1    conda-forge
libffi                    3.4.4                h6a678d5_1  
libgcc-ng                 13.2.0               h77fa898_7    conda-forge
libgfortran-ng            13.2.0               h69a702a_7    conda-forge
libgfortran5              13.2.0               hca663fb_7    conda-forge
libgomp                   13.2.0               h77fa898_7    conda-forge
liblapack                 3.9.0           22_linux64_openblas    conda-forge
libnvjitlink              12.4.127             hd3aeb46_1    conda-forge
libopenblas               0.3.27          pthreads_h413a1c8_0    conda-forge
libstdcxx-ng              13.2.0               hc0a3c3a_7    conda-forge
libuuid                   1.41.5               h5eee18b_0  
llvmlite                  0.42.0                   pypi_0    pypi
ncurses                   6.4                  h6a678d5_0  
nox                       2024.4.15                pypi_0    pypi
numba                     0.59.1                   pypi_0    pypi
numpy                     1.26.4          py311h64a7726_0    conda-forge
openssl                   3.3.0                hd590300_0    conda-forge
packaging                 24.0                     pypi_0    pypi
pip                       24.0            py311h06a4308_0  
platformdirs              4.2.1                    pypi_0    pypi
pluggy                    1.5.0                    pypi_0    pypi
pytest                    8.2.0                    pypi_0    pypi
python                    3.11.9               h955ad1f_0  
python_abi                3.11                    2_cp311    conda-forge
readline                  8.2                  h5eee18b_0  
setuptools                69.5.1          py311h06a4308_0  
sqlite                    3.45.3               h5eee18b_0  
tk                        8.6.14               h39e8969_0  
tzdata                    2024a                h04d1e81_0  
virtualenv                20.26.1                  pypi_0    pypi
wheel                     0.43.0          py311h06a4308_0  
xz                        5.4.6                h5eee18b_1  
zipp                      3.18.1                   pypi_0    pypi
zlib                      1.2.13               h5eee18b_1

jpivarski · 2024-05-13T15:28:52Z

It would be fine to add a filter to ignore this warning. In general, we make warnings be errors in our test suite, mostly to catch deprecations.

This is a performance warning about JIT-compilation. We know that JIT-compilation takes time, and JIT-compilation will take an unusually large fraction of the time in a test suite, which has unrealistically small datasets. Moreover, they say that they'll be changing it in the future, so it's a temporary performance thing. This is not something we need to take action on, except for disabling it to allow our tests to pass.

It is a little odd that we're hearing about something in CuPy. CuPy can't even be installed on GitHub Actions test-runners (no GPU), so how did this come up? Is it happening when you run the tests locally? It doesn't happen in pytest tests, just pytest tests-cuda, right?

ianna · 2024-05-13T20:15:43Z

It would be fine to add a filter to ignore this warning. In general, we make warnings be errors in our test suite, mostly to catch deprecations.

This is a performance warning about JIT-compilation. We know that JIT-compilation takes time, and JIT-compilation will take an unusually large fraction of the time in a test suite, which has unrealistically small datasets. Moreover, they say that they'll be changing it in the future, so it's a temporary performance thing. This is not something we need to take action on, except for disabling it to allow our tests to pass.

It is a little odd that we're hearing about something in CuPy. CuPy can't even be installed on GitHub Actions test-runners (no GPU), so how did this come up? Is it happening when you run the tests locally? It doesn't happen in pytest tests, just pytest tests-cuda, right?

Yes, it's happening locally on my Ubuntu-GPU box. There are two failing cupy tests in tests that should be moved to tests-cuda as you've suggested.

ianna added the bug (unverified) The problem described would be a bug, but needs to be triaged label May 13, 2024

ianna changed the title ~~test_wrap_index_cupy fails with 'PerformanceWarning'~~ test_wrap_index_cupy and test_from_dlpack_cupy fail with 'PerformanceWarning' May 13, 2024

ianna changed the title ~~test_wrap_index_cupy and test_from_dlpack_cupy fail with 'PerformanceWarning'~~ test_wrap_index_cupy, test_from_dlpack_cupy and 361 cuda tests fail with 'PerformanceWarning' May 13, 2024

jpivarski added tests More or better tests are needed and removed bug (unverified) The problem described would be a bug, but needs to be triaged labels May 13, 2024

ianna mentioned this issue May 13, 2024

fix: wait for Jitify performing a one-time only warm-up #3113

Merged

jpivarski closed this as completed in #3113 May 28, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`test_wrap_index_cupy`, `test_from_dlpack_cupy` and 361 cuda tests fail with 'PerformanceWarning' #3111

`test_wrap_index_cupy`, `test_from_dlpack_cupy` and 361 cuda tests fail with 'PerformanceWarning' #3111

ianna commented May 13, 2024 •

edited

ianna commented May 13, 2024

jpivarski commented May 13, 2024

ianna commented May 13, 2024

test_wrap_index_cupy, test_from_dlpack_cupy and 361 cuda tests fail with 'PerformanceWarning' #3111

test_wrap_index_cupy, test_from_dlpack_cupy and 361 cuda tests fail with 'PerformanceWarning' #3111

Comments

ianna commented May 13, 2024 • edited

Version of Awkward Array

Description and code to reproduce

ianna commented May 13, 2024

jpivarski commented May 13, 2024

ianna commented May 13, 2024

`test_wrap_index_cupy`, `test_from_dlpack_cupy` and 361 cuda tests fail with 'PerformanceWarning' #3111

`test_wrap_index_cupy`, `test_from_dlpack_cupy` and 361 cuda tests fail with 'PerformanceWarning' #3111

ianna commented May 13, 2024 •

edited