Faster-RCNN

Breakthrough: RPN and Fast RCNN share convolutions at test-time, so that marginal cost for computing proposals is small.

Region Proposal Network

Objective: to generate detection proposals, serves as the "attention"

What is it: a fully convolutional network (FCN) that can be trained end-to-end

                Input: feature map
                Output: objects bounds, objectness scores

Fast RCNN

Objective: uses proposed regions to classify objects into categories and background

Training Scheme

Alternating fashion: RPN -> Fast RCNN -> RPN -> Fast RCNN

Implementation Details

• Non-Maximum Suppression (NMS)

Since RPN proposals highly overlap with each other, NMS is implemented on the proposal regions based on cls scores, i.e. objectness scores. By fixing IoU threshold at 0.7, NMS leaves use about 2,000 proposal regions per image.