Cannot convert keras separable convolution layers with fallback backend

[SlipknotTN]

I have encountered an issue similar to 717 and 727.

At the bottom of the post, I put the exact error that I get and some logs.

Analyzing in debug, I think that the problem is during the graph creation (traverse.py listup_nodes) that adds Im2Col layer not supported by fallback backend (accordingly to answer to 717), but I can't be more specific, because I don't have deep understanding of webdnn graph representation.

These are two models very similar: one with some separable convolutions and the other only with standard convolutions.
sep_vs_std_conv.zip
Only the second one is correctly converted on all backends, while the first is converted to webgpu, webgl and wasm (although it takes a lot of time). Both of them use keras fallback converter, because in TF converter "DepthwiseConv2dNative is not supported yet".

Error:
[convert_keras.py] Failed generating descriptor for fallback backend Traceback (most recent call last): File "./bin/convert_keras.py", line 102, in main graph_exec_data = generate_descriptor(backend, graph, constant_encoder_name=args.encoding) File "/Users/michele/Anaconda2/anaconda/envs/webdnn_converter/lib/python3.6/site-packages/webdnn-1.2.2-py3.6.egg/webdnn/backend/interface/generator.py", line 107, in generate_descriptor return generator(graph, **kwargs) File "/Users/michele/Anaconda2/anaconda/envs/webdnn_converter/lib/python3.6/site-packages/webdnn-1.2.2-py3.6.egg/webdnn/backend/fallback/generator.py", line 75, in generate return FallbackDescriptorGenerator.generate(graph, **kwargs) File "/Users/michele/Anaconda2/anaconda/envs/webdnn_converter/lib/python3.6/site-packages/webdnn-1.2.2-py3.6.egg/webdnn/backend/fallback/generator.py", line 64, in generate kernels=cls.generate_kernels(graph, memory_layout), File "/Users/michele/Anaconda2/anaconda/envs/webdnn_converter/lib/python3.6/site-packages/webdnn-1.2.2-py3.6.egg/webdnn/backend/interface/generator.py", line 57, in generate_kernels raise NotImplementedError(f"[{cls.__name__}] Operator {op} is not handled by any generator handler") NotImplementedError: [FallbackDescriptorGenerator] Operator <Im2Col inputs={'im': <Variable Variable10 shape=(1, 44, 44, 8), order=[N, H, W, C]>}, outputs={'col': <Variable Variable12 shape=(1, 44, 44, 3, 3, 8), order=[N, H, W, KH, KW, C]>}> is not handled by any generator handler

Subset (related to the first separable convolution) of debug outputs in traverse.py dump function:

[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Im2Col : Im2Col0
[DEBUG] In : {'im': <Variable Variable10 shape=(1, 44, 44, 8), order=[N, H, W, C]>}
[DEBUG] Out : {'col': <Variable Variable12 shape=(1, 44, 44, 3, 3, 8), order=[N, H, W, KH, KW, C]>}
[DEBUG] Attr: Tensorwise[C], Tensorwise[N]
[DEBUG] Parameters: {'dilation_rate': (1, 1), 'ksize': (3, 3), 'padding': (1, 1), 'stride': (1, 1)}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Slice : Slice9
[DEBUG] In : {'x': <Variable Variable12 shape=(1, 44, 44, 3, 3, 8), order=[N, H, W, KH, KW, C]>}
[DEBUG] Out : {'y': <Variable Variable28 shape=(1, 44, 44, 3, 3), order=[N, H, W, KH, KW]>}
[DEBUG] Attr: Tensorwise[H], Tensorwise[KH], Tensorwise[KW], Tensorwise[N], Tensorwise[W]
[DEBUG] Parameters: {'indices': {N:slice(None, None, None), H:slice(None, None, None), W:slice(None, None, None), KH:slice(None, None, None), KW:slice(None, None, None), C:3}}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Slice : Slice7
[DEBUG] In : {'x': <ConstantVariable ConstantVariable15 shape=(3, 3, 8, 1), order=[KH, KW, C, ?63]>}
[DEBUG] Out : {'y': <Variable Variable24 shape=(3, 3, 1), order=[KH, KW, ?63]>}
[DEBUG] Attr: Tensorwise[?63], Tensorwise[KH], Tensorwise[KW]
[DEBUG] Parameters: {'indices': {KH:slice(None, None, None), KW:slice(None, None, None), C:2, ?63:slice(None, None, None)}}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Slice : Slice6
[DEBUG] In : {'x': <Variable Variable12 shape=(1, 44, 44, 3, 3, 8), order=[N, H, W, KH, KW, C]>}
[DEBUG] Out : {'y': <Variable Variable23 shape=(1, 44, 44, 3, 3), order=[N, H, W, KH, KW]>}
[DEBUG] Attr: Tensorwise[H], Tensorwise[KH], Tensorwise[KW], Tensorwise[N], Tensorwise[W]
[DEBUG] Parameters: {'indices': {N:slice(None, None, None), H:slice(None, None, None), W:slice(None, None, None), KH:slice(None, None, None), KW:slice(None, None, None), C:2}}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Tensordot : Tensordot4
[DEBUG] In : {'A': <Variable Variable23 shape=(1, 44, 44, 3, 3), order=[N, H, W, KH, KW]>, 'B': <Variable Variable24 shape=(3, 3, 1), order=[KH, KW, ?63]>}
[DEBUG] Out : {'C': <Variable Variable25 shape=(1, 44, 44, 1), order=[N, H, W, ?63]>}
[DEBUG] Attr: Tensorwise[?63], Tensorwise[H], Tensorwise[N], Tensorwise[W]
[DEBUG] Parameters: {'axes': ((, ), (, ))}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Reshape : Reshape0
[DEBUG] In : {'x': <ConstantVariable ConstantVariable16 shape=(1, 1, 8, 16), order=[KH, KW, C, ?62]>}
[DEBUG] Out : {'y': <Variable Variable11 shape=(8, 1, 16), order=[C, ?63, ?62]>}
[DEBUG] Attr: InplaceOperator
[DEBUG] Parameters: {'in_order': [KH, KW, C, ?62], 'out_order': [C, ?63, ?62], 'out_shape': [8, 1, 16]}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Slice : Slice5
[DEBUG] In : {'x': <Variable Variable11 shape=(8, 1, 16), order=[C, ?63, ?62]>}
[DEBUG] Out : {'y': <Variable Variable21 shape=(1, 16), order=[?63, ?62]>}
[DEBUG] Attr: Tensorwise[?62], Tensorwise[?63]
[DEBUG] Parameters: {'indices': {C:1, ?63:slice(None, None, None), ?62:slice(None, None, None)}}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Slice : Slice4
[DEBUG] In : {'x': <ConstantVariable ConstantVariable15 shape=(3, 3, 8, 1), order=[KH, KW, C, ?63]>}
[DEBUG] Out : {'y': <Variable Variable19 shape=(3, 3, 1), order=[KH, KW, ?63]>}
[DEBUG] Attr: Tensorwise[?63], Tensorwise[KH], Tensorwise[KW]
[DEBUG] Parameters: {'indices': {KH:slice(None, None, None), KW:slice(None, None, None), C:1, ?63:slice(None, None, None)}}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Slice : Slice3
[DEBUG] In : {'x': <Variable Variable12 shape=(1, 44, 44, 3, 3, 8), order=[N, H, W, KH, KW, C]>}
[DEBUG] Out : {'y': <Variable Variable18 shape=(1, 44, 44, 3, 3), order=[N, H, W, KH, KW]>}
[DEBUG] Attr: Tensorwise[H], Tensorwise[KH], Tensorwise[KW], Tensorwise[N], Tensorwise[W]
[DEBUG] Parameters: {'indices': {N:slice(None, None, None), H:slice(None, None, None), W:slice(None, None, None), KH:slice(None, None, None), KW:slice(None, None, None), C:1}}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Tensordot : Tensordot2
[DEBUG] In : {'A': <Variable Variable18 shape=(1, 44, 44, 3, 3), order=[N, H, W, KH, KW]>, 'B': <Variable Variable19 shape=(3, 3, 1), order=[KH, KW, ?63]>}
[DEBUG] Out : {'C': <Variable Variable20 shape=(1, 44, 44, 1), order=[N, H, W, ?63]>}
[DEBUG] Attr: Tensorwise[?63], Tensorwise[H], Tensorwise[N], Tensorwise[W]
[DEBUG] Parameters: {'axes': ((, ), (, ))}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Tensordot : Tensordot3
[DEBUG] In : {'A': <Variable Variable20 shape=(1, 44, 44, 1), order=[N, H, W, ?63]>, 'B': <Variable Variable21 shape=(1, 16), order=[?63, ?62]>}
[DEBUG] Out : {'C': <Variable Variable22 shape=(1, 44, 44, 16), order=[N, H, W, ?62]>}
[DEBUG] Attr: Tensorwise[?62], Tensorwise[H], Tensorwise[N], Tensorwise[W]
[DEBUG] Parameters: {'axes': ((<Axis ?63>,), (<Axis ?63>,))}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Slice : Slice2
[DEBUG] In : {'x': <Variable Variable11 shape=(8, 1, 16), order=[C, ?63, ?62]>}
[DEBUG] Out : {'y': <Variable Variable16 shape=(1, 16), order=[?63, ?62]>}
[DEBUG] Attr: Tensorwise[?62], Tensorwise[?63]
[DEBUG] Parameters: {'indices': {C:0, ?63:slice(None, None, None), ?62:slice(None, None, None)}}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Slice : Slice1
[DEBUG] In : {'x': <ConstantVariable ConstantVariable15 shape=(3, 3, 8, 1), order=[KH, KW, C, ?63]>}
[DEBUG] Out : {'y': <Variable Variable14 shape=(3, 3, 1), order=[KH, KW, ?63]>}
[DEBUG] Attr: Tensorwise[?63], Tensorwise[KH], Tensorwise[KW]
[DEBUG] Parameters: {'indices': {KH:slice(None, None, None), KW:slice(None, None, None), C:0, ?63:slice(None, None, None)}}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Slice : Slice0
[DEBUG] In : {'x': <Variable Variable12 shape=(1, 44, 44, 3, 3, 8), order=[N, H, W, KH, KW, C]>}
[DEBUG] Out : {'y': <Variable Variable13 shape=(1, 44, 44, 3, 3), order=[N, H, W, KH, KW]>}
[DEBUG] Attr: Tensorwise[H], Tensorwise[KH], Tensorwise[KW], Tensorwise[N], Tensorwise[W]
[DEBUG] Parameters: {'indices': {N:slice(None, None, None), H:slice(None, None, None), W:slice(None, None, None), KH:slice(None, None, None), KW:slice(None, None, None), C:0}}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Tensordot : Tensordot0
[DEBUG] In : {'A': <Variable Variable13 shape=(1, 44, 44, 3, 3), order=[N, H, W, KH, KW]>, 'B': <Variable Variable14 shape=(3, 3, 1), order=[KH, KW, ?63]>}
[DEBUG] Out : {'C': <Variable Variable15 shape=(1, 44, 44, 1), order=[N, H, W, ?63]>}
[DEBUG] Attr: Tensorwise[?63], Tensorwise[H], Tensorwise[N], Tensorwise[W]
[DEBUG] Parameters: {'axes': ((, ), (, ))}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Tensordot : Tensordot1
[DEBUG] In : {'A': <Variable Variable15 shape=(1, 44, 44, 1), order=[N, H, W, ?63]>, 'B': <Variable Variable16 shape=(1, 16), order=[?63, ?62]>}
[DEBUG] Out : {'C': <Variable Variable17 shape=(1, 44, 44, 16), order=[N, H, W, ?62]>}
[DEBUG] Attr: Tensorwise[?62], Tensorwise[H], Tensorwise[N], Tensorwise[W]
[DEBUG] Parameters: {'axes': ((<Axis ?63>,), (<Axis ?63>,))}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] ElementwiseAdd : ElementwiseAdd0
[DEBUG] In : {'x0': <Variable Variable17 shape=(1, 44, 44, 16), order=[N, H, W, ?62]>, 'x1': <Variable Variable22 shape=(1, 44, 44, 16), order=[N, H, W, ?62]>}
[DEBUG] Out : {'y': <Variable Variable53 shape=(1, 44, 44, 16), order=[N, H, W, ?62]>}
[DEBUG] Attr: Associative, Commutative, InplaceOperator, Tensorwise[?62], Tensorwise[H], Tensorwise[N], Tensorwise[W]
[DEBUG] Parameters: {}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Slice : Slice11
[DEBUG] In : {'x': <Variable Variable11 shape=(8, 1, 16), order=[C, ?63, ?62]>}
[DEBUG] Out : {'y': <Variable Variable31 shape=(1, 16), order=[?63, ?62]>}
[DEBUG] Attr: Tensorwise[?62], Tensorwise[?63]
[DEBUG] Parameters: {'indices': {C:3, ?63:slice(None, None, None), ?62:slice(None, None, None)}}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Slice : Slice10
[DEBUG] In : {'x': <ConstantVariable ConstantVariable15 shape=(3, 3, 8, 1), order=[KH, KW, C, ?63]>}
[DEBUG] Out : {'y': <Variable Variable29 shape=(3, 3, 1), order=[KH, KW, ?63]>}
[DEBUG] Attr: Tensorwise[?63], Tensorwise[KH], Tensorwise[KW]
[DEBUG] Parameters: {'indices': {KH:slice(None, None, None), KW:slice(None, None, None), C:3, ?63:slice(None, None, None)}}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Tensordot : Tensordot6
[DEBUG] In : {'A': <Variable Variable28 shape=(1, 44, 44, 3, 3), order=[N, H, W, KH, KW]>, 'B': <Variable Variable29 shape=(3, 3, 1), order=[KH, KW, ?63]>}
[DEBUG] Out : {'C': <Variable Variable30 shape=(1, 44, 44, 1), order=[N, H, W, ?63]>}
[DEBUG] Attr: Tensorwise[?63], Tensorwise[H], Tensorwise[N], Tensorwise[W]
[DEBUG] Parameters: {'axes': ((, ), (, ))}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Tensordot : Tensordot7
[DEBUG] In : {'A': <Variable Variable30 shape=(1, 44, 44, 1), order=[N, H, W, ?63]>, 'B': <Variable Variable31 shape=(1, 16), order=[?63, ?62]>}
[DEBUG] Out : {'C': <Variable Variable32 shape=(1, 44, 44, 16), order=[N, H, W, ?62]>}
[DEBUG] Attr: Tensorwise[?62], Tensorwise[H], Tensorwise[N], Tensorwise[W]
[DEBUG] Parameters: {'axes': ((<Axis ?63>,), (<Axis ?63>,))}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Slice : Slice8
[DEBUG] In : {'x': <Variable Variable11 shape=(8, 1, 16), order=[C, ?63, ?62]>}
[DEBUG] Out : {'y': <Variable Variable26 shape=(1, 16), order=[?63, ?62]>}
[DEBUG] Attr: Tensorwise[?62], Tensorwise[?63]
[DEBUG] Parameters: {'indices': {C:2, ?63:slice(None, None, None), ?62:slice(None, None, None)}}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Tensordot : Tensordot5
[DEBUG] In : {'A': <Variable Variable25 shape=(1, 44, 44, 1), order=[N, H, W, ?63]>, 'B': <Variable Variable26 shape=(1, 16), order=[?63, ?62]>}
[DEBUG] Out : {'C': <Variable Variable27 shape=(1, 44, 44, 16), order=[N, H, W, ?62]>}
[DEBUG] Attr: Tensorwise[?62], Tensorwise[H], Tensorwise[N], Tensorwise[W]
[DEBUG] Parameters: {'axes': ((<Axis ?63>,), (<Axis ?63>,))}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] ElementwiseAdd : ElementwiseAdd1
[DEBUG] In : {'x0': <Variable Variable27 shape=(1, 44, 44, 16), order=[N, H, W, ?62]>, 'x1': <Variable Variable32 shape=(1, 44, 44, 16), order=[N, H, W, ?62]>}
[DEBUG] Out : {'y': <Variable Variable54 shape=(1, 44, 44, 16), order=[N, H, W, ?62]>}
[DEBUG] Attr: Associative, Commutative, InplaceOperator, Tensorwise[?62], Tensorwise[H], Tensorwise[N], Tensorwise[W]
[DEBUG] Parameters: {}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] ElementwiseAdd : ElementwiseAdd4
[DEBUG] In : {'x0': <Variable Variable53 shape=(1, 44, 44, 16), order=[N, H, W, ?62]>, 'x1': <Variable Variable54 shape=(1, 44, 44, 16), order=[N, H, W, ?62]>}
[DEBUG] Out : {'y': <Variable Variable57 shape=(1, 44, 44, 16), order=[N, H, W, ?62]>}
[DEBUG] Attr: Associative, Commutative, InplaceOperator, Tensorwise[?62], Tensorwise[H], Tensorwise[N], Tensorwise[W]
[DEBUG] Parameters: {}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Slice : Slice23
[DEBUG] In : {'x': <Variable Variable11 shape=(8, 1, 16), order=[C, ?63, ?62]>}
[DEBUG] Out : {'y': <Variable Variable51 shape=(1, 16), order=[?63, ?62]>}
[DEBUG] Attr: Tensorwise[?62], Tensorwise[?63]
[DEBUG] Parameters: {'indices': {C:7, ?63:slice(None, None, None), ?62:slice(None, None, None)}}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Slice : Slice22
[DEBUG] In : {'x': <ConstantVariable ConstantVariable15 shape=(3, 3, 8, 1), order=[KH, KW, C, ?63]>}
[DEBUG] Out : {'y': <Variable Variable49 shape=(3, 3, 1), order=[KH, KW, ?63]>}
[DEBUG] Attr: Tensorwise[?63], Tensorwise[KH], Tensorwise[KW]
[DEBUG] Parameters: {'indices': {KH:slice(None, None, None), KW:slice(None, None, None), C:7, ?63:slice(None, None, None)}}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Slice : Slice21
[DEBUG] In : {'x': <Variable Variable12 shape=(1, 44, 44, 3, 3, 8), order=[N, H, W, KH, KW, C]>}
[DEBUG] Out : {'y': <Variable Variable48 shape=(1, 44, 44, 3, 3), order=[N, H, W, KH, KW]>}
[DEBUG] Attr: Tensorwise[H], Tensorwise[KH], Tensorwise[KW], Tensorwise[N], Tensorwise[W]
[DEBUG] Parameters: {'indices': {N:slice(None, None, None), H:slice(None, None, None), W:slice(None, None, None), KH:slice(None, None, None), KW:slice(None, None, None), C:7}}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Tensordot : Tensordot14
[DEBUG] In : {'A': <Variable Variable48 shape=(1, 44, 44, 3, 3), order=[N, H, W, KH, KW]>, 'B': <Variable Variable49 shape=(3, 3, 1), order=[KH, KW, ?63]>}
[DEBUG] Out : {'C': <Variable Variable50 shape=(1, 44, 44, 1), order=[N, H, W, ?63]>}
[DEBUG] Attr: Tensorwise[?63], Tensorwise[H], Tensorwise[N], Tensorwise[W]
[DEBUG] Parameters: {'axes': ((, ), (, ))}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Tensordot : Tensordot15
[DEBUG] In : {'A': <Variable Variable50 shape=(1, 44, 44, 1), order=[N, H, W, ?63]>, 'B': <Variable Variable51 shape=(1, 16), order=[?63, ?62]>}
[DEBUG] Out : {'C': <Variable Variable52 shape=(1, 44, 44, 16), order=[N, H, W, ?62]>}
[DEBUG] Attr: Tensorwise[?62], Tensorwise[H], Tensorwise[N], Tensorwise[W]
[DEBUG] Parameters: {'axes': ((<Axis ?63>,), (<Axis ?63>,))}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Slice : Slice20
[DEBUG] In : {'x': <Variable Variable11 shape=(8, 1, 16), order=[C, ?63, ?62]>}
[DEBUG] Out : {'y': <Variable Variable46 shape=(1, 16), order=[?63, ?62]>}
[DEBUG] Attr: Tensorwise[?62], Tensorwise[?63]
[DEBUG] Parameters: {'indices': {C:6, ?63:slice(None, None, None), ?62:slice(None, None, None)}}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Slice : Slice19
[DEBUG] In : {'x': <ConstantVariable ConstantVariable15 shape=(3, 3, 8, 1), order=[KH, KW, C, ?63]>}
[DEBUG] Out : {'y': <Variable Variable44 shape=(3, 3, 1), order=[KH, KW, ?63]>}
[DEBUG] Attr: Tensorwise[?63], Tensorwise[KH], Tensorwise[KW]
[DEBUG] Parameters: {'indices': {KH:slice(None, None, None), KW:slice(None, None, None), C:6, ?63:slice(None, None, None)}}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Slice : Slice18
[DEBUG] In : {'x': <Variable Variable12 shape=(1, 44, 44, 3, 3, 8), order=[N, H, W, KH, KW, C]>}
[DEBUG] Out : {'y': <Variable Variable43 shape=(1, 44, 44, 3, 3), order=[N, H, W, KH, KW]>}
[DEBUG] Attr: Tensorwise[H], Tensorwise[KH], Tensorwise[KW], Tensorwise[N], Tensorwise[W]
[DEBUG] Parameters: {'indices': {N:slice(None, None, None), H:slice(None, None, None), W:slice(None, None, None), KH:slice(None, None, None), KW:slice(None, None, None), C:6}}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Tensordot : Tensordot12
[DEBUG] In : {'A': <Variable Variable43 shape=(1, 44, 44, 3, 3), order=[N, H, W, KH, KW]>, 'B': <Variable Variable44 shape=(3, 3, 1), order=[KH, KW, ?63]>}
[DEBUG] Out : {'C': <Variable Variable45 shape=(1, 44, 44, 1), order=[N, H, W, ?63]>}
[DEBUG] Attr: Tensorwise[?63], Tensorwise[H], Tensorwise[N], Tensorwise[W]
[DEBUG] Parameters: {'axes': ((, ), (, ))}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Tensordot : Tensordot13
[DEBUG] In : {'A': <Variable Variable45 shape=(1, 44, 44, 1), order=[N, H, W, ?63]>, 'B': <Variable Variable46 shape=(1, 16), order=[?63, ?62]>}
[DEBUG] Out : {'C': <Variable Variable47 shape=(1, 44, 44, 16), order=[N, H, W, ?62]>}
[DEBUG] Attr: Tensorwise[?62], Tensorwise[H], Tensorwise[N], Tensorwise[W]
[DEBUG] Parameters: {'axes': ((<Axis ?63>,), (<Axis ?63>,))}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] ElementwiseAdd : ElementwiseAdd3
[DEBUG] In : {'x0': <Variable Variable47 shape=(1, 44, 44, 16), order=[N, H, W, ?62]>, 'x1': <Variable Variable52 shape=(1, 44, 44, 16), order=[N, H, W, ?62]>}
[DEBUG] Out : {'y': <Variable Variable56 shape=(1, 44, 44, 16), order=[N, H, W, ?62]>}
[DEBUG] Attr: Associative, Commutative, InplaceOperator, Tensorwise[?62], Tensorwise[H], Tensorwise[N], Tensorwise[W]
[DEBUG] Parameters: {}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Slice : Slice17
[DEBUG] In : {'x': <Variable Variable11 shape=(8, 1, 16), order=[C, ?63, ?62]>}
[DEBUG] Out : {'y': <Variable Variable41 shape=(1, 16), order=[?63, ?62]>}
[DEBUG] Attr: Tensorwise[?62], Tensorwise[?63]
[DEBUG] Parameters: {'indices': {C:5, ?63:slice(None, None, None), ?62:slice(None, None, None)}}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Slice : Slice16
[DEBUG] In : {'x': <ConstantVariable ConstantVariable15 shape=(3, 3, 8, 1), order=[KH, KW, C, ?63]>}
[DEBUG] Out : {'y': <Variable Variable39 shape=(3, 3, 1), order=[KH, KW, ?63]>}
[DEBUG] Attr: Tensorwise[?63], Tensorwise[KH], Tensorwise[KW]
[DEBUG] Parameters: {'indices': {KH:slice(None, None, None), KW:slice(None, None, None), C:5, ?63:slice(None, None, None)}}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Slice : Slice15
[DEBUG] In : {'x': <Variable Variable12 shape=(1, 44, 44, 3, 3, 8), order=[N, H, W, KH, KW, C]>}
[DEBUG] Out : {'y': <Variable Variable38 shape=(1, 44, 44, 3, 3), order=[N, H, W, KH, KW]>}
[DEBUG] Attr: Tensorwise[H], Tensorwise[KH], Tensorwise[KW], Tensorwise[N], Tensorwise[W]
[DEBUG] Parameters: {'indices': {N:slice(None, None, None), H:slice(None, None, None), W:slice(None, None, None), KH:slice(None, None, None), KW:slice(None, None, None), C:5}}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Tensordot : Tensordot10
[DEBUG] In : {'A': <Variable Variable38 shape=(1, 44, 44, 3, 3), order=[N, H, W, KH, KW]>, 'B': <Variable Variable39 shape=(3, 3, 1), order=[KH, KW, ?63]>}
[DEBUG] Out : {'C': <Variable Variable40 shape=(1, 44, 44, 1), order=[N, H, W, ?63]>}
[DEBUG] Attr: Tensorwise[?63], Tensorwise[H], Tensorwise[N], Tensorwise[W]
[DEBUG] Parameters: {'axes': ((, ), (, ))}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Tensordot : Tensordot11
[DEBUG] In : {'A': <Variable Variable40 shape=(1, 44, 44, 1), order=[N, H, W, ?63]>, 'B': <Variable Variable41 shape=(1, 16), order=[?63, ?62]>}
[DEBUG] Out : {'C': <Variable Variable42 shape=(1, 44, 44, 16), order=[N, H, W, ?62]>}
[DEBUG] Attr: Tensorwise[?62], Tensorwise[H], Tensorwise[N], Tensorwise[W]
[DEBUG] Parameters: {'axes': ((<Axis ?63>,), (<Axis ?63>,))}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Slice : Slice14
[DEBUG] In : {'x': <Variable Variable11 shape=(8, 1, 16), order=[C, ?63, ?62]>}
[DEBUG] Out : {'y': <Variable Variable36 shape=(1, 16), order=[?63, ?62]>}
[DEBUG] Attr: Tensorwise[?62], Tensorwise[?63]
[DEBUG] Parameters: {'indices': {C:4, ?63:slice(None, None, None), ?62:slice(None, None, None)}}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Slice : Slice13
[DEBUG] In : {'x': <ConstantVariable ConstantVariable15 shape=(3, 3, 8, 1), order=[KH, KW, C, ?63]>}
[DEBUG] Out : {'y': <Variable Variable34 shape=(3, 3, 1), order=[KH, KW, ?63]>}
[DEBUG] Attr: Tensorwise[?63], Tensorwise[KH], Tensorwise[KW]
[DEBUG] Parameters: {'indices': {KH:slice(None, None, None), KW:slice(None, None, None), C:4, ?63:slice(None, None, None)}}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Slice : Slice12
[DEBUG] In : {'x': <Variable Variable12 shape=(1, 44, 44, 3, 3, 8), order=[N, H, W, KH, KW, C]>}
[DEBUG] Out : {'y': <Variable Variable33 shape=(1, 44, 44, 3, 3), order=[N, H, W, KH, KW]>}
[DEBUG] Attr: Tensorwise[H], Tensorwise[KH], Tensorwise[KW], Tensorwise[N], Tensorwise[W]
[DEBUG] Parameters: {'indices': {N:slice(None, None, None), H:slice(None, None, None), W:slice(None, None, None), KH:slice(None, None, None), KW:slice(None, None, None), C:4}}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Tensordot : Tensordot8
[DEBUG] In : {'A': <Variable Variable33 shape=(1, 44, 44, 3, 3), order=[N, H, W, KH, KW]>, 'B': <Variable Variable34 shape=(3, 3, 1), order=[KH, KW, ?63]>}
[DEBUG] Out : {'C': <Variable Variable35 shape=(1, 44, 44, 1), order=[N, H, W, ?63]>}
[DEBUG] Attr: Tensorwise[?63], Tensorwise[H], Tensorwise[N], Tensorwise[W]
[DEBUG] Parameters: {'axes': ((, ), (, ))}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Tensordot : Tensordot9
[DEBUG] In : {'A': <Variable Variable35 shape=(1, 44, 44, 1), order=[N, H, W, ?63]>, 'B': <Variable Variable36 shape=(1, 16), order=[?63, ?62]>}
[DEBUG] Out : {'C': <Variable Variable37 shape=(1, 44, 44, 16), order=[N, H, W, ?62]>}
[DEBUG] Attr: Tensorwise[?62], Tensorwise[H], Tensorwise[N], Tensorwise[W]
[DEBUG] Parameters: {'axes': ((<Axis ?63>,), (<Axis ?63>,))}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] ElementwiseAdd : ElementwiseAdd2
[DEBUG] In : {'x0': <Variable Variable37 shape=(1, 44, 44, 16), order=[N, H, W, ?62]>, 'x1': <Variable Variable42 shape=(1, 44, 44, 16), order=[N, H, W, ?62]>}
[DEBUG] Out : {'y': <Variable Variable55 shape=(1, 44, 44, 16), order=[N, H, W, ?62]>}
[DEBUG] Attr: Associative, Commutative, InplaceOperator, Tensorwise[?62], Tensorwise[H], Tensorwise[N], Tensorwise[W]
[DEBUG] Parameters: {}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] ElementwiseAdd : ElementwiseAdd5
[DEBUG] In : {'x0': <Variable Variable55 shape=(1, 44, 44, 16), order=[N, H, W, ?62]>, 'x1': <Variable Variable56 shape=(1, 44, 44, 16), order=[N, H, W, ?62]>}
[DEBUG] Out : {'y': <Variable Variable58 shape=(1, 44, 44, 16), order=[N, H, W, ?62]>}
[DEBUG] Attr: Associative, Commutative, InplaceOperator, Tensorwise[?62], Tensorwise[H], Tensorwise[N], Tensorwise[W]
[DEBUG] Parameters: {}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] ElementwiseAdd : ElementwiseAdd6
[DEBUG] In : {'x0': <Variable Variable57 shape=(1, 44, 44, 16), order=[N, H, W, ?62]>, 'x1': <Variable Variable58 shape=(1, 44, 44, 16), order=[N, H, W, ?62]>}
[DEBUG] Out : {'y': <Variable Variable59 shape=(1, 44, 44, 16), order=[N, H, W, ?62]>}
[DEBUG] Attr: Associative, Commutative, InplaceOperator, Tensorwise[?62], Tensorwise[H], Tensorwise[N], Tensorwise[W]
[DEBUG] Parameters: {}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] ReinterpretAxis : ReinterpretAxis0
[DEBUG] In : {'x': <Variable Variable59 shape=(1, 44, 44, 16), order=[N, H, W, ?62]>}
[DEBUG] Out : {'y': <Variable Variable60 shape=(1, 44, 44, 16), order=[N, H, W, C]>}
[DEBUG] Attr: InplaceOperator, Tensorwise[?62], Tensorwise[H], Tensorwise[N], Tensorwise[W]
[DEBUG] Parameters: {'in_order': [N, H, W, ?62], 'out_order': [N, H, W, C]}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Transpose : Transpose0
[DEBUG] In : {'x0': <Variable Variable60 shape=(1, 44, 44, 16), order=[N, H, W, C]>}
[DEBUG] Out : {'y': <Variable Variable61 shape=(1, 44, 44, 16), order=[N, H, W, C]>}
[DEBUG] Attr: InplaceOperator, Tensorwise[C], Tensorwise[H], Tensorwise[N], Tensorwise[W]
[DEBUG] Parameters: {}
[DEBUG] ---------------------------------------------------------------------------
[DEBUG] Relu : Relu4
[DEBUG] In : {'x0': <Variable Variable61 shape=(1, 44, 44, 16), order=[N, H, W, C]>}
[DEBUG] Out : {'y': <Variable Variable62 shape=(1, 44, 44, 16), order=[N, H, W, C]>}
[DEBUG] Attr: InplaceOperator, Tensorwise[C], Tensorwise[H], Tensorwise[N], Tensorwise[W]
[DEBUG] Parameters: {}
[DEBUG] ---------------------------------------------------------------------------

Keras model architecture:

---

Layer (type) Output Shape Param Connected to
input (InputLayer) (None, 48, 48, 1) 0
conv2d_1 (Conv2D) (None, 46, 46, 8) 72 input[0][0]
activation_1 (Activation) (None, 46, 46, 8) 0 conv2d_1[0][0]
conv2d_2 (Conv2D) (None, 44, 44, 8) 576 activation_1[0][0]
activation_2 (Activation) (None, 44, 44, 8) 0 conv2d_2[0][0]
separable_conv2d_1 (SeparableCo (None, 44, 44, 16) 200 activation_2[0][0]
activation_3 (Activation) (None, 44, 44, 16) 0 separable_conv2d_1[0][0]
separable_conv2d_2 (SeparableCo (None, 44, 44, 16) 400 activation_3[0][0]
max_pooling2d_1 (MaxPooling2D) (None, 22, 22, 16) 0 separable_conv2d_2[0][0]
conv2d_3 (Conv2D) (None, 22, 22, 16) 128 activation_2[0][0]
add_1 (Add) (None, 22, 22, 16) 0 max_pooling2d_1[0][0]
conv2d_3[0][0]
separable_conv2d_3 (SeparableCo (None, 22, 22, 32) 656 add_1[0][0]
activation_4 (Activation) (None, 22, 22, 32) 0 separable_conv2d_3[0][0]
separable_conv2d_4 (SeparableCo (None, 22, 22, 32) 1312 activation_4[0][0]
max_pooling2d_2 (MaxPooling2D) (None, 11, 11, 32) 0 separable_conv2d_4[0][0]
conv2d_4 (Conv2D) (None, 11, 11, 32) 512 add_1[0][0]
add_2 (Add) (None, 11, 11, 32) 0 max_pooling2d_2[0][0]
conv2d_4[0][0]
separable_conv2d_5 (SeparableCo (None, 11, 11, 64) 2336 add_2[0][0]
activation_5 (Activation) (None, 11, 11, 64) 0 separable_conv2d_5[0][0]
separable_conv2d_6 (SeparableCo (None, 11, 11, 64) 4672 activation_5[0][0]
max_pooling2d_3 (MaxPooling2D) (None, 6, 6, 64) 0 separable_conv2d_6[0][0]
conv2d_5 (Conv2D) (None, 6, 6, 64) 2048 add_2[0][0]
add_3 (Add) (None, 6, 6, 64) 0 max_pooling2d_3[0][0]
conv2d_5[0][0]
separable_conv2d_7 (SeparableCo (None, 6, 6, 128) 8768 add_3[0][0]
activation_6 (Activation) (None, 6, 6, 128) 0 separable_conv2d_7[0][0]
separable_conv2d_8 (SeparableCo (None, 6, 6, 128) 17536 activation_6[0][0]
max_pooling2d_4 (MaxPooling2D) (None, 3, 3, 128) 0 separable_conv2d_8[0][0]
conv2d_6 (Conv2D) (None, 3, 3, 128) 8192 add_3[0][0]
add_4 (Add) (None, 3, 3, 128) 0 max_pooling2d_4[0][0]
conv2d_6[0][0]
base_network_output (Activation (None, 3, 3, 128) 0 add_4[0][0]
conv_ft (Conv2D) (None, 3, 3, 7) 8071 base_network_output[0][0]
global_avg_pooling_ft (GlobalAv (None, 7) 0 conv_ft[0][0]
softmax (Activation) (None, 7) 0 global_avg_pooling_ft[0][0]