This thesis is about the stability analysis of a coupled finite dimensional system and an infinite dimensional one. This kind of systems emerges in the physics since it is related to the modeling of structures for instance. The generic analysis of such systems is complex, mainly because of their different nature. Here, the analysis is conducted using different methodologies. First, the recent Quadratic Separation framework is used to deal with the frequency aspect of such systems. Then, a second result is derived using a Lyapunov-based argument. All the results are obtained considering the projections of the infinite dimensional state on a basis of polynomials. It is then possible to take into account the coupling between the two systems. That results in tractable and reliable numerical tests with a moderate conservatism. Moreover, a hierarchy on the stability conditions is shown in the Lyapunov case.The real application to a drilling mechanism is proposed to illustrate the efficiency of the method and it opens new perspectives. For instance, using the notion of practical stability, we show that a PI-controlled drillstring is subject to a limit cycle and that it is possible to estimate its amplitude.