Security has always been the main agenda in ensuring the safety and welfare for government and agencies especially in public areas where possible threat that could cause a massive damage is intolerable. For this reason, immediate steps such as putting on CCTV cameras and employing sophisticated surveillance have been placed in some public and high security areas. More often than not, the system still involves manual monitoring by the security officer. As a solution, this project proposed an improved system with an automatic object tracking capability. The project will be focusing on tracking operation system such that a single person can be tracked in a semi-dense crowd. A person will be determined by a user clicking on that person and a box will be drawn encapsulating the person. As this person moves within the scene, the box will follow him/her until the person leaves the camera view. The research undertaken in this report is mainly concentrated on developing detection and tracking system which incorporates with some operation on images such as thresholding, blob labelling, blob matching, filtering and blob analysis. All the process will be done in a grayscale image in order to make the detection process becomes easier. Background subtraction model is being used for extracting the moving object from the background. Method used for object tracking is featurebased model method which used area, center point of each moving people and the Euclidean distance between object to recognized tracking object. Based on the experimental results, the percentage of accuracy decreases when the number of person increases in the scene.