pythonでpycaffeを実行すると、実際にcaffe内でconvやfc層の値やパラメータがどうなっているのかが見えにくいときがある。そんなときは、以下のように、self.solver.step(1)前後でpdbで止めて、self.solverを参照して中身を見ることができる。
import pdb pdb.set_trace() self.solver.step(1)
例えば、値を見たい時はself.solver.net.blobsを用いる。例えば、pdbで「self.solver.net.blobs」を打つとblobsの中身が見れるので、そのなかでさらに詳細に見たいblobsを選択する。
(Pdb) self.solver.net.blobs
OrderedDict([('data', <caffe._caffe.Blob object at 0x7fc96e564b90>), ('im_info', <caffe._caffe.Blob object at 0x7fc96e564c08>), ('gt_boxes', <caffe._caffe.Blob object at 0x7fc96e564c80>), ('data_input-data_0_split_0', <caffe._caffe.Blob object at 0x7fc96e564cf8>), ('data_input-data_0_split_1', <caffe._caffe.Blob object at 0x7fc96e564d70>), ('im_info_input-data_1_split_0', <caffe._caffe.Blob object at 0x7fc96e564de8>), ('im_info_input-data_1_split_1', <caffe._caffe.Blob object at 0x7fc96e564e60>), ('gt_boxes_input-data_2_split_0', <caffe._caffe.Blob object at 0x7fc96e564ed8>), ('gt_boxes_input-data_2_split_1', <caffe._caffe.Blob object at 0x7fc96e564f50>), ('conv1', <caffe._caffe.Blob object at 0x7fc96e577050>), ('norm1', <caffe._caffe.Blob object at 0x7fc96e5770c8>), ('pool1', <caffe._caffe.Blob object at 0x7fc96e577140>), ('conv2', <caffe._caffe.Blob object at 0x7fc96e5771b8>), ('norm2', <caffe._caffe.Blob object at 0x7fc96e577230>), ('pool2', <caffe._caffe.Blob object at 0x7fc96e5772a8>), ('conv3', <caffe._caffe.Blob object at 0x7fc96e577320>), ('conv4', <caffe._caffe.Blob object at 0x7fc96e577398>), ('conv5', <caffe._caffe.Blob object at 0x7fc96e577410>), ('conv5_relu5_0_split_0', <caffe._caffe.Blob object at 0x7fc96e577488>), ('conv5_relu5_0_split_1', <caffe._caffe.Blob object at 0x7fc96e577500>), ('rpn/output', <caffe._caffe.Blob object at 0x7fc96e577578>), ('rpn/output_rpn_relu/3x3_0_split_0', <caffe._caffe.Blob object at 0x7fc96e5775f0>), ('rpn/output_rpn_relu/3x3_0_split_1', <caffe._caffe.Blob object at 0x7fc96e577668>), ('rpn_cls_score', <caffe._caffe.Blob object at 0x7fc96e5776e0>), ('rpn_cls_score_rpn_cls_score_0_split_0', <caffe._caffe.Blob object at 0x7fc96e577758>), ('rpn_cls_score_rpn_cls_score_0_split_1', <caffe._caffe.Blob object at 0x7fc96e5777d0>), ('rpn_bbox_pred', <caffe._caffe.Blob object at 0x7fc96e577848>), ('rpn_bbox_pred_rpn_bbox_pred_0_split_0', <caffe._caffe.Blob object at 0x7fc96e5778c0>), ('rpn_bbox_pred_rpn_bbox_pred_0_split_1', <caffe._caffe.Blob object at 0x7fc96e577938>), ('rpn_cls_score_reshape', <caffe._caffe.Blob object at 0x7fc96e5779b0>), ('rpn_cls_score_reshape_rpn_cls_score_reshape_0_split_0', <caffe._caffe.Blob object at 0x7fc96e577a28>), ('rpn_cls_score_reshape_rpn_cls_score_reshape_0_split_1', <caffe._caffe.Blob object at 0x7fc96e577aa0>), ('rpn_labels', <caffe._caffe.Blob object at 0x7fc96e577b18>), ('rpn_bbox_targets', <caffe._caffe.Blob object at 0x7fc96e577b90>), ('rpn_bbox_inside_weights', <caffe._caffe.Blob object at 0x7fc96e577c08>), ('rpn_bbox_outside_weights', <caffe._caffe.Blob object at 0x7fc96e577c80>), ('rpn_cls_loss', <caffe._caffe.Blob object at 0x7fc96e577cf8>), ('rpn_loss_bbox', <caffe._caffe.Blob object at 0x7fc96e577d70>), ('rpn_cls_prob', <caffe._caffe.Blob object at 0x7fc96e577de8>), ('rpn_cls_prob_reshape', <caffe._caffe.Blob object at 0x7fc96e577e60>), ('rpn_rois', <caffe._caffe.Blob object at 0x7fc96e577ed8>), ('rois', <caffe._caffe.Blob object at 0x7fc96e577f50>), ('labels', <caffe._caffe.Blob object at 0x7fc96e578050>), ('bbox_targets', <caffe._caffe.Blob object at 0x7fc96e5780c8>), ('bbox_inside_weights', <caffe._caffe.Blob object at 0x7fc96e578140>), ('bbox_outside_weights', <caffe._caffe.Blob object at 0x7fc96e5781b8>), ('pool5', <caffe._caffe.Blob object at 0x7fc96e578230>), ('fc6', <caffe._caffe.Blob object at 0x7fc96e5782a8>), ('fc7', <caffe._caffe.Blob object at 0x7fc96e578320>), ('fc7_drop7_0_split_0', <caffe._caffe.Blob object at 0x7fc96e578398>), ('fc7_drop7_0_split_1', <caffe._caffe.Blob object at 0x7fc96e578410>), ('cls_score', <caffe._caffe.Blob object at 0x7fc96e578488>), ('bbox_pred', <caffe._caffe.Blob object at 0x7fc96e578500>), ('loss_cls', <caffe._caffe.Blob object at 0x7fc96e578578>), ('loss_bbox', <caffe._caffe.Blob object at 0x7fc96e5785f0>)])blobsの中身を詳細にみるためには、以下のようにdataを参照する。
> (Pdb) self.solver.net.blobs['conv1'].data
array([[[[ 0., 0., 0., ..., 0., 0., 0.],
[ 0., 0., 0., ..., 0., 0., 0.],
[ 0., 0., 0., ..., 0., 0., 0.],
...,
[ 0., 0., 0., ..., 0., 0., 0.],
[ 0., 0., 0., ..., 0., 0., 0.],
[ 0., 0., 0., ..., 0., 0., 0.]],
...
> (Pdb) self.solver.net.blobs['conv1'].data.shape
(1, 96, 247, 497)forwardとbackwardを実行するためには、以下のようにする。
> self.solver.net.forward()
{'loss_bbox': array(0.399600088596344, dtype=float32), 'rpn_cls_loss': array(0.7076734304428101, dtype=float32), 'loss_cls': array(0.7948347926139832, dtype=float32), 'rpn_loss_bbox': array(0.010186992585659027, dtype=float32)}
> self.solver.net.backward()
{}各レイヤーのパラメータを見るためには、self.solver.net.paramsを用いる。
(Pdb) self.solver.net.params
OrderedDict([('conv1', <caffe._caffe.BlobVec object at 0x7fc53753dc90>), ('conv2', <caffe._caffe.BlobVec object at 0x7fc53753dd00>), ('conv3', <caffe._caffe.BlobVec object at 0x7fc53753dc20>), ('conv4', <caffe._caffe.BlobVec object at 0x7fc53753dd70>), ('conv5', <caffe._caffe.BlobVec object at 0x7fc53753dde0>), ('rpn_conv/3x3', <caffe._caffe.BlobVec object at 0x7fc53753de50>), ('rpn_cls_score', <caffe._caffe.BlobVec object at 0x7fc53753dec0>), ('rpn_bbox_pred', <caffe._caffe.BlobVec object at 0x7fc53753df30>), ('fc6', <caffe._caffe.BlobVec object at 0x7fc53753dfa0>), ('fc7', <caffe._caffe.BlobVec object at 0x7fc5374e9050>), ('cls_score', <caffe._caffe.BlobVec object at 0x7fc5374e90c0>), ('bbox_pred', <caffe._caffe.BlobVec object at 0x7fc5374e9130>)])さらいに詳細をみるためには、dataを用いる。例えば、conv1のwとbを見るためには以下のようにする。
(Pdb) self.solver.net.params['conv1'][0].data.shape
(96, 3, 7, 7)
(Pdb) self.solver.net.params['conv1'][0].data
array([[[[ 1.32567152e-01, 6.84457272e-02, 1.83226704e-03, ...,
-1.22019038e-01, -1.79185256e-01, -2.30399281e-01],
[ 1.62634373e-01, 9.79008675e-02, 2.15274505e-02, ...,
-1.09140977e-01, -1.71696663e-01, -2.26968318e-01],
[ 1.87837422e-01, 1.21260315e-01, 4.76242229e-02, ...,
-9.10851136e-02, -1.60597667e-01, -2.20204458e-01],
(Pdb) self.solver.net.params['conv1'][1].data.shape
(96,)
(Pdb) self.solver.net.params['conv1'][1].data
array([-1.57771218, -0.69981641, -1.58167875, -1.70307279, -1.75285602,
-0.88742614, -1.14932168, -1.43839383, -2.02191281, -1.45102441,
-1.62427282, -1.99904907, -1.64861023, -1.50930917, -1.19354439,
-1.07889366, -1.61664736, 0.19126397, -1.62222862, -1.73484313,
-1.96931612, -1.98989189, -1.61901534, -1.3672961 , 0.02061985,
-0.67420292, -1.39617205, -2.08821225, -2.02410698, -1.24440253,
-1.26593232, -1.83775723, -1.85998356, -1.47129214, -1.47418547,
-0.52472562, -1.10081184, -0.15183373, -1.02605069, -1.08456528,
-0.32465318, -1.25606322, -1.47240341, -1.78586793, -1.77220678,
-1.56166732, -1.26420057, -0.93922412, -0.33436561, -1.56884706,
-1.57687819, -2.37079573, -2.00665712, -1.93187177, -1.92664075,
-1.09698212, -1.45953357, -1.32682753, -1.97277617, -1.26517308,
-0.76611769, -1.25607574, -1.15794134, -1.09959459, -0.38230255,
-0.93944383, -0.42870682, -1.84481764, -1.6430434 , -1.58583581,
-2.35817051, -0.57982498, -0.76150614, 0.39267823, -0.39711541,
-0.39922512, -2.31891799, -2.05666471, -1.92897189, -2.00091481,
-1.37036109, -0.62744516, -0.83082926, 0.54076028, -0.66058046,
-0.62742412, -1.25799012, -0.55127829, -0.0374692 , 0.34946525,
-1.13100505, -1.16287112, -1.85470808, -1.52229035, -1.31701291,
-1.92791569], dtype=float32)また、各レイヤー内で設定してある独自のインスタンス変数やメソッドを実行するためには、self.solver.net.layersを用いる。
まずは、それぞれのレイヤーの名前を調べる。
(Pdb) for ind in range(len(self.solver.net.layers)): print ind,":",self.solver.net._layer_names[ind] 0 : input-data 1 : data_input-data_0_split 2 : im_info_input-data_1_split 3 : gt_boxes_input-data_2_split 4 : conv1 5 : relu1 6 : norm1 7 : pool1 8 : conv2 9 : relu2 10 : norm2 11 : pool2 12 : conv3 13 : relu3 14 : conv4 15 : relu4 16 : conv5 17 : relu5 18 : conv5_relu5_0_split 19 : rpn_conv/3x3 20 : rpn_relu/3x3 21 : rpn/output_rpn_relu/3x3_0_split 22 : rpn_cls_score 23 : rpn_cls_score_rpn_cls_score_0_split 24 : rpn_bbox_pred 25 : rpn_bbox_pred_rpn_bbox_pred_0_split 26 : rpn_cls_score_reshape 27 : rpn_cls_score_reshape_rpn_cls_score_reshape_0_split 28 : rpn-data 29 : rpn_loss_cls 30 : rpn_loss_bbox 31 : rpn_cls_prob 32 : rpn_cls_prob_reshape 33 : proposal 34 : roi-data 35 : roi_pool5 36 : fc6 37 : relu6 38 : drop6 39 : fc7 40 : relu7 41 : drop7 42 : fc7_drop7_0_split 43 : cls_score 44 : bbox_pred 45 : loss_cls 46 : loss_bbox
そして、詳細を見たいレイヤーのインデックスを指定して、以下のようにインスタンス変数やメソッドを参照する。
(Pdb) self.solver.net.layers[28]._anchors
array([[ -84. , -40. , 99. , 55. ,
16.24107358],
[-176. , -88. , 191. , 103. ,
8.08820901],
[-360. , -184. , 375. , 199. ,
4.03607311],
[ -56. , -56. , 71. , 71. ,
16.86158361],
[-120. , -120. , 135. , 135. ,
8.39772988],
[-248. , -248. , 263. , 263. ,
4.19064798],
[ -36. , -80. , 51. , 95. ,
17.35495256],
[ -80. , -168. , 95. , 183. ,
8.64031347],
[-168. , -344. , 183. , 359. ,
4.31092891]])
(Pdb) self.solver.net.layers[0]._num_classes
2
(Pdb) self.solver.net.layers[0]._get_next_minibatch()
{'gt_boxes': array([[ 321. , 185. , 375. , 235. ,
34.72399902, 1. ]], dtype=float32), 'data': array([[[[ 32.01990128, 34.01990128, 38.01990128, ..., -29.98010063,
-36.98009872, -44.98009872],
[ 30.01989937, 31.01989937, 32.01990128, ..., -29.98010063,
-37.98009872, -43.98009872],
[ 30.01989937, 32.01990128, 32.01990128, ..., -35.98009872,
-42.98009872, -48.98009872],
...
(Pdb) self.solver.net.layers[4].blobs[0].data.shape
(96, 3, 7, 7)
(Pdb) self.solver.net.params['conv1'][0].data.shape
(96, 3, 7, 7)
(Pdb) self.solver.net.layers[4].blobs[1].data.shape
(96,)
(Pdb) self.solver.net.params['conv1'][1].data.shape
(96,)これを利用すると、各レイヤーのforwardを個々に実行することもできる。
例えば、レイヤー0の定義をtrain.prototxtで確認し、必要なbottomとtopを用意する。
layer {
name: 'input-data'
type: 'Python'
top: 'data'
top: 'im_info'
top: 'gt_boxes'
python_param {
module: 'roi_data_layer.layer'
layer: 'RoIDataLayer'
param_str: "'num_classes': 2" # classes
}
}上記の定義から、bottomは無くて、topが３つあるので以下のようにtopを用意（ただし、topはあくまでも出力なのでarrayの形がそろっていれば空でも大丈夫なはず？）。topの３つのblobは、以下のようにself.solver.net.blobsで確認できる。
(Pdb) self.solver.net.blobs
OrderedDict([('data', <caffe._caffe.Blob object at 0x7f4c99acaf50>), ('im_info', <caffe._caffe.Blob object at 0x7f4c99acad70>), ('gt_boxes', <caffe._caffe.Blob object at 0x7f4c99acab90>)
...上記のdata, im_infoおよびgt_boxesのblobをtopリストに入れて、self.solver.net.layers[0].forwardを実行してみる。
(Pdb) top = [self.solver.net.blobs['data'],self.solver.net.blobs['im_info'],self.solver.net.blobs['gt_boxes']]
(Pdb) top[0].data[0][0]
array([[ -30.98010063, -30.98010063, -18.98010063, ..., -41.98009872,
-47.98009872, -51.98009872],
[ -30.98010063, -23.98010063, -7.98010015, ..., -37.98009872,
-43.98009872, -48.98009872],
[ -23.98010063, -21.98010063, -10.98009968, ..., -39.98009872,
-45.98009872, -48.98009872],
...,
[ -89.98010254, -90.98010254, -87.98010254, ..., -101.98010254,
-99.98010254, -98.98010254],
[ -92.98010254, -90.98010254, -87.98010254, ..., -102.98010254,
-101.98010254, -100.98010254],
[ -92.98010254, -91.98010254, -89.98010254, ..., -101.98010254,
-99.98010254, -98.98010254]], dtype=float32)
(Pdb) top[1].data
array([[ 500., 375., 1., 548.]], dtype=float32)
(Pdb) top[2].data
array([[ 82. , 210. , 336. , 428. ,
9.08930016, 1. ]], dtype=float32)
(Pdb) self.solver.net.layers[0].forward([],top)
(Pdb) top[0].data[0][0]
array([[-99.98010254, -99.98010254, -98.98010254, ..., -88.98010254,
-86.98010254, -84.98010254],
[-99.98010254, -99.98010254, -98.98010254, ..., -85.98010254,
-87.98010254, -89.98010254],
[-99.98010254, -99.98010254, -98.98010254, ..., -86.98010254,
-87.98010254, -88.98010254],
...,
[-80.98010254, -81.98010254, -80.98010254, ..., -92.98010254,
-93.98010254, -93.98010254],
[-81.98010254, -81.98010254, -80.98010254, ..., -93.98010254,
-93.98010254, -93.98010254],
[-83.98010254, -82.98010254, -82.98010254, ..., -93.98010254,
-93.98010254, -92.98010254]], dtype=float32)
(Pdb) top[1].data
array([[ 375., 500., 1., 635.]], dtype=float32)
(Pdb) top[2].data
array([[ 225. , 185. , 297. , 231. ,
34.89910126, 1. ]], dtype=float32)forwardの前後で、topの値が変わっていることが確認できるので、ちゃんと実行できている！
その他の例
(Pdb) bottom = [self.solver.net.blobs['rpn_cls_score'],self.solver.net.blobs['gt_boxes'],self.solver.net.blobs['im_info'],self.solver.net.blobs['data']] (Pdb) top = [self.solver.net.blobs['rpn_labels'],self.solver.net.blobs['rpn_bbox_targets'],self.solver.net.blobs['rpn_bbox_inside_weights'],self.solver.net.blobs['rpn_bbox_outside_weights']] (Pdb) self.solver.net.layers[28].forward(bottom,top)
その他の方法として、gdbを使ったデバッグもできる。
Makefile.configにて、「DEBUG:=1」に設定してビルドし、以下のようにgdbを頭につけて実行する。
time gdb --args python ./tools/train_net.py --gpu ${GPU_ID} \
--solver models/${PT_DIR}/${NET}/faster_rcnn_end2end/solver.prototxt \
--weights data/imagenet_models/${NET}.v2.caffemodel \
--imdb ${TRAIN_IMDB} \
--iters ${ITERS} \
--cfg experiments/cfgs/faster_rcnn_end2end.yml \
${EXTRA_ARGS}「r」で実行、「b」で現在の場所、「backtrace」でエラーがでたところからさかのぼることができる。
