Benchmarking is a common practice employed to quantify the performance of various approaches toward the same task. By maintaining a consistent test environment, the inherent behaviours between methods may be distinguished—which is key to progressing a research field. In robotic manipulation research there is a current lack of standardisation, making it challenging to fairly assess and compare various approaches throughout literature. This paper proposes new criteria in conjunction with a benchmarking platform to measure the effectiveness of a grasping pipeline. The proposed benchmarking template offers a testing platform for 2-fingered, vision-based grasp synthesis methodologies. A prototype system was constructed. The prototype was shown to serve as a suitable benchmarking platform for the deployment of various grasp synthesis methodologies. 4000 trials were conducted to evaluate the differing approaches. Results showed that the proposed metrics provide useful insights into the quality of grasp poses produced by a grasp synthesis methodology. Moreover, such metrics provide more comprehensive insights into grasp outcome than traditional methods used to quantify performance of a methodology and present a fair baseline for comparison between different approaches.