Each level of the coronary artery has different sizes and properties. The primary coronary arteries usually have high contrast to the background, while the secondary coronary arteries have low contrast to the background and thin structures. Furthermore, several small vessels are disconnected or broken up vascular segments. It is a challenging task to use a single model to segment all coronary artery sizes. To overcome this problem, we propose a novel segmenting method for coronary artery extraction from angiograms based on the primary and secondary coronary artery. Our method is a coarse-to-fine strategic approach for extracting coronary arteries in many different sizes. We construct the first U-net model to segment the main coronary artery extraction and build a new algorithm to determine the junctions of the main coronary artery with the secondary coronary artery. Using these junctions, we determine regions of the secondary coronary arteries (rectangular regions) for a secondary coronary artery-extracted segment with the second U-net model. The experiment result is 76.40% in terms of Dice coefficient on coronary X-ray datasets. The proposed approach presents its potential in coronary vessel segmentation.