Layer Information

Benanza affords you a few ways to examine the topological structure (or architecture) of models. This section describes the base case, where given a model it prints out all the layers. Subsequent sections describe how to get the layer shape information and layer statistics across models from the tool.

NOTE: For illustration purposes we use AlexNet/ResNet50 as example for this document, since otherwise the output will be big.

Theoretical Flops

Users can get the theoretical flops of a model by using the flopsinfo command. The command allows one to compute the theoretical flops for each layer within the network.

benanza flopsinfo -h
Get flops information about the model

Usage:
  benanza flopsinfo [flags]

Aliases:
  flopsinfo, flops

Flags:
  -h, --help   help for flopsinfo

Global Flags:
  -b, --batch_size int       batch size (default 1)
  -f, --format string        print format to use (default "automatic")
      --full                 print all information about the layers
      --human                print flops in human form
  -d, --model_dir string     model directory
  -p, --model_path string    path to the model prototxt file
      --no_header            show header labels for output
  -o, --output_file string   output file name

Usage

You can also get the flop information as a table (default). For example, for AlexNet, the theoretical flops are:

benanza flopsinfo --model_path //AlexNet/model.onnx

FLOP TYPE	#
MultipleAdds	655622016
Additions	463184
Divisions	906984
Exponentiations	453992
Comparisons	1623808
General	0
Total	1314692000

You can also get the theoretical flops of each layer AlexNet. This is done by passing the --full option and here we pass the --human option to make the output more readable.

benanza flopsinfo --model_path //AlexNet/model.onnx --full --human

LAYERNAME	LAYERTYPE	MULTIPLYADDS	ADDITIONS	DIVISIONS	EXPONENTIATIONS	COMPARISONS	TOTAL
conv_1	Conv	102 MFlop	0 Flops	0 Flops	0 Flops	0 Flops	203 MFlop
relu_1	Relu	280 kFlop	0 Flops	0 Flops	0 Flops	280 kFlop	840 kFlop
lrn_1	LRN	280 kFlop	280 kFlop	560 kFlop	280 kFlop	0 Flops	1.7 MFlop
maxpool_1	Pooling	0 Flops	0 Flops	0 Flops	0 Flops	584 kFlop	584 kFlop
conv2_2	Conv	208 MFlop	0 Flops	0 Flops	0 Flops	0 Flops	415 MFlop
relu_2	Relu	173 kFlop	0 Flops	0 Flops	0 Flops	173 kFlop	519 kFlop
lrn_2	LRN	173 kFlop	173 kFlop	346 kFlop	173 kFlop	0 Flops	1.0 MFlop
maxpool_2	Pooling	0 Flops	0 Flops	0 Flops	0 Flops	332 kFlop	332 kFlop
conv_3	Conv	127 MFlop	0 Flops	0 Flops	0 Flops	0 Flops	255 MFlop
relu_3	Relu	55 kFlop	0 Flops	0 Flops	0 Flops	55 kFlop	166 kFlop
conv4_2	Conv	96 MFlop	0 Flops	0 Flops	0 Flops	0 Flops	191 MFlop
relu_4	Relu	55 kFlop	0 Flops	0 Flops	0 Flops	55 kFlop	166 kFlop
conv5_2	Conv	64 MFlop	0 Flops	0 Flops	0 Flops	0 Flops	127 MFlop
relu_5	Relu	37 kFlop	0 Flops	0 Flops	0 Flops	37 kFlop	111 kFlop
maxpool_3	Pooling	0 Flops	0 Flops	0 Flops	0 Flops	83 kFlop	83 kFlop
reshape_1	Reshape	0 Flops	0 Flops	0 Flops	0 Flops	0 Flops	0 Flops
gemm_1	Gemm	38 MFlop	4.1 kFlop	0 Flops	0 Flops	0 Flops	76 MFlop
relu_6	Relu	4.1 kFlop	0 Flops	0 Flops	0 Flops	4.1 kFlop	12 kFlop
dropout_1	Dropout	0 Flops	0 Flops	0 Flops	0 Flops	4.1 kFlop	4.1 kFlop
dropout_1	Dropout	0 Flops	0 Flops	0 Flops	0 Flops	4.1 kFlop	4.1 kFlop
gemm_2	Gemm	17 MFlop	4.1 kFlop	0 Flops	0 Flops	0 Flops	34 MFlop
relu_7	Relu	4.1 kFlop	0 Flops	0 Flops	0 Flops	4.1 kFlop	12 kFlop
dropout_2	Dropout	0 Flops	0 Flops	0 Flops	0 Flops	4.1 kFlop	4.1 kFlop
dropout_2	Dropout	0 Flops	0 Flops	0 Flops	0 Flops	4.1 kFlop	4.1 kFlop
gemm_3	Gemm	4.1 MFlop	1.0 kFlop	0 Flops	0 Flops	0 Flops	8.2 MFlop
softmax_1	Softmax	0 Flops	1.0 kFlop	1.0 kFlop	1.0 kFlop	0 Flops	3.0 kFlop

Click to See Theoretical Flops for ResNet50 ↕

Theoretical Flops for ResNet50

benanza flopsinfo -p //ResNet50-v1 -f csv

This will output a csv file ResNet050-v1.csv which contains the flop information for the model

Flop Type	#
MultipleAdds	3867055104
Additions	16107496
Divisions	10587136
Exponentiations	0
Comparisons	10888192
General	0
Total	3904637928

To get the flops for each layer within the network, you need to pass in the --full flag

benanza flopsinfo -p //ResNet50-v1 --full -f csv

This will output a csv file ResNet050-v1.csv which contains the flop information for each layer withing the model

LayerName	LayerType	MultiplyAdds	Additions	Divisions	Comparisons	Total
resnetv17_conv0_fwd	Conv	118013952	0	0	0	118013952
resnetv17_batchnorm0_fwd	BatchNorm	0	802816	802816	0	1605632
resnetv17_relu0_fwd	Relu	802816	0	0	802816	1605632
resnetv17_pool0_fwd	Pooling	0	0	0	1806336	1806336
resnetv17_stage1_conv0_fwd	Conv	12845056	0	0	0	12845056
resnetv17_stage1_batchnorm0_fwd	BatchNorm	0	200704	200704	0	401408
resnetv17_stage1_relu0_fwd	Relu	200704	0	0	200704	401408
resnetv17_stage1_conv1_fwd	Conv	115605504	0	0	0	115605504
resnetv17_stage1_batchnorm1_fwd	BatchNorm	0	200704	200704	0	401408
resnetv17_stage1_relu1_fwd	Relu	200704	0	0	200704	401408
resnetv17_stage1_conv2_fwd	Conv	51380224	0	0	0	51380224
resnetv17_stage1_batchnorm2_fwd	BatchNorm	0	802816	802816	0	1605632
resnetv17_stage1_conv3_fwd	Conv	51380224	0	0	0	51380224
resnetv17_stage1_batchnorm3_fwd	BatchNorm	0	802816	802816	0	1605632
resnetv17_stage1__plus0	ElementWise	0	802816	0	0	802816
resnetv17_stage1_activation0	Relu	802816	0	0	802816	1605632
resnetv17_stage1_conv4_fwd	Conv	51380224	0	0	0	51380224
resnetv17_stage1_batchnorm4_fwd	BatchNorm	0	200704	200704	0	401408
resnetv17_stage1_relu2_fwd	Relu	200704	0	0	200704	401408
resnetv17_stage1_conv5_fwd	Conv	115605504	0	0	0	115605504
resnetv17_stage1_batchnorm5_fwd	BatchNorm	0	200704	200704	0	401408
resnetv17_stage1_relu3_fwd	Relu	200704	0	0	200704	401408
resnetv17_stage1_conv6_fwd	Conv	51380224	0	0	0	51380224
resnetv17_stage1_batchnorm6_fwd	BatchNorm	0	802816	802816	0	1605632
resnetv17_stage1__plus1	ElementWise	0	802816	0	0	802816
resnetv17_stage1_activation1	Relu	802816	0	0	802816	1605632
resnetv17_stage1_conv7_fwd	Conv	51380224	0	0	0	51380224
resnetv17_stage1_batchnorm7_fwd	BatchNorm	0	200704	200704	0	401408
resnetv17_stage1_relu4_fwd	Relu	200704	0	0	200704	401408
resnetv17_stage1_conv8_fwd	Conv	115605504	0	0	0	115605504
resnetv17_stage1_batchnorm8_fwd	BatchNorm	0	200704	200704	0	401408
resnetv17_stage1_relu5_fwd	Relu	200704	0	0	200704	401408
resnetv17_stage1_conv9_fwd	Conv	51380224	0	0	0	51380224
resnetv17_stage1_batchnorm9_fwd	BatchNorm	0	802816	802816	0	1605632
resnetv17_stage1__plus2	ElementWise	0	802816	0	0	802816
resnetv17_stage1_activation2	Relu	802816	0	0	802816	1605632
resnetv17_stage2_conv0_fwd	Conv	25690112	0	0	0	25690112
resnetv17_stage2_batchnorm0_fwd	BatchNorm	0	100352	100352	0	200704
resnetv17_stage2_relu0_fwd	Relu	100352	0	0	100352	200704
resnetv17_stage2_conv1_fwd	Conv	115605504	0	0	0	115605504
resnetv17_stage2_batchnorm1_fwd	BatchNorm	0	100352	100352	0	200704
resnetv17_stage2_relu1_fwd	Relu	100352	0	0	100352	200704
resnetv17_stage2_conv2_fwd	Conv	51380224	0	0	0	51380224
resnetv17_stage2_batchnorm2_fwd	BatchNorm	0	401408	401408	0	802816
resnetv17_stage2_conv3_fwd	Conv	102760448	0	0	0	102760448
resnetv17_stage2_batchnorm3_fwd	BatchNorm	0	401408	401408	0	802816
resnetv17_stage2__plus0	ElementWise	0	401408	0	0	401408
resnetv17_stage2_activation0	Relu	401408	0	0	401408	802816
resnetv17_stage2_conv4_fwd	Conv	51380224	0	0	0	51380224
resnetv17_stage2_batchnorm4_fwd	BatchNorm	0	100352	100352	0	200704
resnetv17_stage2_relu2_fwd	Relu	100352	0	0	100352	200704
resnetv17_stage2_conv5_fwd	Conv	115605504	0	0	0	115605504
resnetv17_stage2_batchnorm5_fwd	BatchNorm	0	100352	100352	0	200704
resnetv17_stage2_relu3_fwd	Relu	100352	0	0	100352	200704
resnetv17_stage2_conv6_fwd	Conv	51380224	0	0	0	51380224
resnetv17_stage2_batchnorm6_fwd	BatchNorm	0	401408	401408	0	802816
resnetv17_stage2__plus1	ElementWise	0	401408	0	0	401408
resnetv17_stage2_activation1	Relu	401408	0	0	401408	802816
resnetv17_stage2_conv7_fwd	Conv	51380224	0	0	0	51380224
resnetv17_stage2_batchnorm7_fwd	BatchNorm	0	100352	100352	0	200704
resnetv17_stage2_relu4_fwd	Relu	100352	0	0	100352	200704
resnetv17_stage2_conv8_fwd	Conv	115605504	0	0	0	115605504
resnetv17_stage2_batchnorm8_fwd	BatchNorm	0	100352	100352	0	200704
resnetv17_stage2_relu5_fwd	Relu	100352	0	0	100352	200704
resnetv17_stage2_conv9_fwd	Conv	51380224	0	0	0	51380224
resnetv17_stage2_batchnorm9_fwd	BatchNorm	0	401408	401408	0	802816
resnetv17_stage2__plus2	ElementWise	0	401408	0	0	401408
resnetv17_stage2_activation2	Relu	401408	0	0	401408	802816
resnetv17_stage2_conv10_fwd	Conv	51380224	0	0	0	51380224
resnetv17_stage2_batchnorm10_fwd	BatchNorm	0	100352	100352	0	200704
resnetv17_stage2_relu6_fwd	Relu	100352	0	0	100352	200704
resnetv17_stage2_conv11_fwd	Conv	115605504	0	0	0	115605504
resnetv17_stage2_batchnorm11_fwd	BatchNorm	0	100352	100352	0	200704
resnetv17_stage2_relu7_fwd	Relu	100352	0	0	100352	200704
resnetv17_stage2_conv12_fwd	Conv	51380224	0	0	0	51380224
resnetv17_stage2_batchnorm12_fwd	BatchNorm	0	401408	401408	0	802816
resnetv17_stage2__plus3	ElementWise	0	401408	0	0	401408
resnetv17_stage2_activation3	Relu	401408	0	0	401408	802816
resnetv17_stage3_conv0_fwd	Conv	25690112	0	0	0	25690112
resnetv17_stage3_batchnorm0_fwd	BatchNorm	0	50176	50176	0	100352
resnetv17_stage3_relu0_fwd	Relu	50176	0	0	50176	100352
resnetv17_stage3_conv1_fwd	Conv	115605504	0	0	0	115605504
resnetv17_stage3_batchnorm1_fwd	BatchNorm	0	50176	50176	0	100352
resnetv17_stage3_relu1_fwd	Relu	50176	0	0	50176	100352
resnetv17_stage3_conv2_fwd	Conv	51380224	0	0	0	51380224
resnetv17_stage3_batchnorm2_fwd	BatchNorm	0	200704	200704	0	401408
resnetv17_stage3_conv3_fwd	Conv	102760448	0	0	0	102760448
resnetv17_stage3_batchnorm3_fwd	BatchNorm	0	200704	200704	0	401408
resnetv17_stage3__plus0	ElementWise	0	200704	0	0	200704
resnetv17_stage3_activation0	Relu	200704	0	0	200704	401408
resnetv17_stage3_conv4_fwd	Conv	51380224	0	0	0	51380224
resnetv17_stage3_batchnorm4_fwd	BatchNorm	0	50176	50176	0	100352
resnetv17_stage3_relu2_fwd	Relu	50176	0	0	50176	100352
resnetv17_stage3_conv5_fwd	Conv	115605504	0	0	0	115605504
resnetv17_stage3_batchnorm5_fwd	BatchNorm	0	50176	50176	0	100352
resnetv17_stage3_relu3_fwd	Relu	50176	0	0	50176	100352
resnetv17_stage3_conv6_fwd	Conv	51380224	0	0	0	51380224
resnetv17_stage3_batchnorm6_fwd	BatchNorm	0	200704	200704	0	401408
resnetv17_stage3__plus1	ElementWise	0	200704	0	0	200704
resnetv17_stage3_activation1	Relu	200704	0	0	200704	401408
resnetv17_stage3_conv7_fwd	Conv	51380224	0	0	0	51380224
resnetv17_stage3_batchnorm7_fwd	BatchNorm	0	50176	50176	0	100352
resnetv17_stage3_relu4_fwd	Relu	50176	0	0	50176	100352
resnetv17_stage3_conv8_fwd	Conv	115605504	0	0	0	115605504
resnetv17_stage3_batchnorm8_fwd	BatchNorm	0	50176	50176	0	100352
resnetv17_stage3_relu5_fwd	Relu	50176	0	0	50176	100352
resnetv17_stage3_conv9_fwd	Conv	51380224	0	0	0	51380224
resnetv17_stage3_batchnorm9_fwd	BatchNorm	0	200704	200704	0	401408
resnetv17_stage3__plus2	ElementWise	0	200704	0	0	200704
resnetv17_stage3_activation2	Relu	200704	0	0	200704	401408
resnetv17_stage3_conv10_fwd	Conv	51380224	0	0	0	51380224
resnetv17_stage3_batchnorm10_fwd	BatchNorm	0	50176	50176	0	100352
resnetv17_stage3_relu6_fwd	Relu	50176	0	0	50176	100352
resnetv17_stage3_conv11_fwd	Conv	115605504	0	0	0	115605504
resnetv17_stage3_batchnorm11_fwd	BatchNorm	0	50176	50176	0	100352
resnetv17_stage3_relu7_fwd	Relu	50176	0	0	50176	100352
resnetv17_stage3_conv12_fwd	Conv	51380224	0	0	0	51380224
resnetv17_stage3_batchnorm12_fwd	BatchNorm	0	200704	200704	0	401408
resnetv17_stage3__plus3	ElementWise	0	200704	0	0	200704
resnetv17_stage3_activation3	Relu	200704	0	0	200704	401408
resnetv17_stage3_conv13_fwd	Conv	51380224	0	0	0	51380224
resnetv17_stage3_batchnorm13_fwd	BatchNorm	0	50176	50176	0	100352
resnetv17_stage3_relu8_fwd	Relu	50176	0	0	50176	100352
resnetv17_stage3_conv14_fwd	Conv	115605504	0	0	0	115605504
resnetv17_stage3_batchnorm14_fwd	BatchNorm	0	50176	50176	0	100352
resnetv17_stage3_relu9_fwd	Relu	50176	0	0	50176	100352
resnetv17_stage3_conv15_fwd	Conv	51380224	0	0	0	51380224
resnetv17_stage3_batchnorm15_fwd	BatchNorm	0	200704	200704	0	401408
resnetv17_stage3__plus4	ElementWise	0	200704	0	0	200704
resnetv17_stage3_activation4	Relu	200704	0	0	200704	401408
resnetv17_stage3_conv16_fwd	Conv	51380224	0	0	0	51380224
resnetv17_stage3_batchnorm16_fwd	BatchNorm	0	50176	50176	0	100352
resnetv17_stage3_relu10_fwd	Relu	50176	0	0	50176	100352
resnetv17_stage3_conv17_fwd	Conv	115605504	0	0	0	115605504
resnetv17_stage3_batchnorm17_fwd	BatchNorm	0	50176	50176	0	100352
resnetv17_stage3_relu11_fwd	Relu	50176	0	0	50176	100352
resnetv17_stage3_conv18_fwd	Conv	51380224	0	0	0	51380224
resnetv17_stage3_batchnorm18_fwd	BatchNorm	0	200704	200704	0	401408
resnetv17_stage3__plus5	ElementWise	0	200704	0	0	200704
resnetv17_stage3_activation5	Relu	200704	0	0	200704	401408
resnetv17_stage4_conv0_fwd	Conv	25690112	0	0	0	25690112
resnetv17_stage4_batchnorm0_fwd	BatchNorm	0	25088	25088	0	50176
resnetv17_stage4_relu0_fwd	Relu	25088	0	0	25088	50176
resnetv17_stage4_conv1_fwd	Conv	115605504	0	0	0	115605504
resnetv17_stage4_batchnorm1_fwd	BatchNorm	0	25088	25088	0	50176
resnetv17_stage4_relu1_fwd	Relu	25088	0	0	25088	50176
resnetv17_stage4_conv2_fwd	Conv	51380224	0	0	0	51380224
resnetv17_stage4_batchnorm2_fwd	BatchNorm	0	100352	100352	0	200704
resnetv17_stage4_conv3_fwd	Conv	102760448	0	0	0	102760448
resnetv17_stage4_batchnorm3_fwd	BatchNorm	0	100352	100352	0	200704
resnetv17_stage4__plus0	ElementWise	0	100352	0	0	100352
resnetv17_stage4_activation0	Relu	100352	0	0	100352	200704
resnetv17_stage4_conv4_fwd	Conv	51380224	0	0	0	51380224
resnetv17_stage4_batchnorm4_fwd	BatchNorm	0	25088	25088	0	50176
resnetv17_stage4_relu2_fwd	Relu	25088	0	0	25088	50176
resnetv17_stage4_conv5_fwd	Conv	115605504	0	0	0	115605504
resnetv17_stage4_batchnorm5_fwd	BatchNorm	0	25088	25088	0	50176
resnetv17_stage4_relu3_fwd	Relu	25088	0	0	25088	50176
resnetv17_stage4_conv6_fwd	Conv	51380224	0	0	0	51380224
resnetv17_stage4_batchnorm6_fwd	BatchNorm	0	100352	100352	0	200704
resnetv17_stage4__plus1	ElementWise	0	100352	0	0	100352
resnetv17_stage4_activation1	Relu	100352	0	0	100352	200704
resnetv17_stage4_conv7_fwd	Conv	51380224	0	0	0	51380224
resnetv17_stage4_batchnorm7_fwd	BatchNorm	0	25088	25088	0	50176
resnetv17_stage4_relu4_fwd	Relu	25088	0	0	25088	50176
resnetv17_stage4_conv8_fwd	Conv	115605504	0	0	0	115605504
resnetv17_stage4_batchnorm8_fwd	BatchNorm	0	25088	25088	0	50176
resnetv17_stage4_relu5_fwd	Relu	25088	0	0	25088	50176
resnetv17_stage4_conv9_fwd	Conv	51380224	0	0	0	51380224
resnetv17_stage4_batchnorm9_fwd	BatchNorm	0	100352	100352	0	200704
resnetv17_stage4__plus2	ElementWise	0	100352	0	0	100352
resnetv17_stage4_activation2	Relu	100352	0	0	100352	200704
resnetv17_pool1_fwd	GlobalPooling	0	0	0	0	0
flatten_473	Flatten	0	0	0	0	0
resnetv17_dense0_fwd	Gemm	2048000	1000	0	0	2049000

Weight Information

Benanza can examine the weights within the model to compute statistics such as distribution and histogram information. While this is outside the scope of the the paper, it shows that the tool is capable of being extended and generalized to other domains. Here we show how to get weights information for AlexNet

benanza weightsinfo --model_path ///BVLC_AlexNet/model.onnx

The command outputs a table (shown bellow)

LAYERNAME	LAYERTYPE	LENGTH	LAYERWEIGHTSMAX	LAYERWEIGHTSMIN	LAYERWEIGHTSSDEV
conv_1	Conv	34944	0.44255533814430237	-0.800920307636261	0.06429448279744572
conv2_2	Conv	307456	0.4831761121749878	-0.2586861848831177	0.02538710473291357
conv_3	Conv	885120	0.46737396717071533	-0.21888871490955353	0.01679956665487417
conv4_2	Conv	663936	0.4463724195957184	-0.15847662091255188	0.018438743467314958
conv5_2	Conv	442624	1.3109842538833618	-0.48335349559783936	0.021065162399881654
gemm_1	Gemm	37752832	0.24130532145500183	-0.03843037411570549	0.004962945202842958
gemm_2	Gemm	16781312	0.6115601062774658	-0.03297363966703415	0.00794123731339782
gemm_3	Gemm	4097000	0.40080997347831726	-0.43072983622550964	0.010868346074723081

The command also generates figures that show the distribution of weights within the layers