In this paper, we propose a hierarchical three step search algorithm using pyramid hierarchy. Hierarchical Minkowski inequality is adopted to reduce the cost of the sum of absolute difference (SAD) computations. A top-town procedure is developed to search motion vectors hierarchically and the search pattern used in the well-known Three Step Search (TSS) is adopted. We also propose a hybrid motion estimation scheme by combining the proposed Hierarchical Block Sum Pyramid Three Step Search (HBSPTSS) with the famous Block-Based Gradient Search (BBGDS) algorithms. Experimental results show that, compared to other existing search algorithms, the proposed hybrid search algorithm can achieve high computation efficiency for both slow-motion and fast-motion video contents while maintaining excellent PSNR performance. It is thus suitable for wide application fields with real-time requirement.