Abstract Background: This study was designed to examine the validity of a surgery clerkship’s performance evaluation system. The study also assessed faculty members’ confidence in how clerks are evaluated and promotion decisions made. Methods: Student files from five classes (n = 339) were analyzed. A 25-item published survey designed to study faculty perceptions of a student evaluation system was distributed. Chi-square tests of independence and descriptive statistics were used. Results: Faculty survey results showed faculty perceptions of strengths and weaknesses in the evaluation system. Significant relationships were found with prior performance indicators and clerkship performance yielding evidence of concurrent and predictive validity. Conclusions: Findings provide guidance for enhancing how clerks are evaluated as well as specific profiles of students who may need special attention or additional challenges during the surgery clerkship. This study provides a model for other clerkships to assess their student evaluations systems.