The use of quantum dots (QDots) as bright and photostable probes for long-term fluorescence imaging is gaining more interest. Thus far, (pre)clinical use of QDots remains limited, which is primarily caused by the potential toxicity of QDots. Most QDots consist of Cd2+ ions, which are known to cause high levels of toxicity. In order to overcome this problem, several strategies have been tested, such as the generation of cadmium-free QDots. In the present study, two types of cadmium-free QDots, composed of ZnSe/ZnS (QDot(ZnSe)) and InP/ZnS (QDot(InP)), were studied with respect to their cytotoxicity and cellular uptake in a variety of cell types. A multiparametric cytotoxicity approach is used, where the QDots are studied with respect to cell viability, oxidative stress, cell morphology, stem cell differentiation, and neurite outgrowth. The data reveal slight differences in uptake levels for both types of QDots (maximal for QDot(ZnSe)), but clear differences in cytotoxicity and cell functionality effects exist, with highest toxicity for QDot(ZnSe). Differences between cell types and between both types of QDots can be explained by the intrinsic sensitivity of certain cell types and chemical composition of the QDots. At concentrations at which no toxic effects can be observed, the functionality of the QDots for fluorescence cell visualization is evaluated, revealing that the higher brightness of QDot(ZnSe) overcomes most of the toxicity issues compared to that of QDot(InP). Comparing the results obtained with common Cd2+-containing QDots tested under identical conditions, the importance of particle functionality is demonstrated, revealing that cadmium-free QDots tested in this study are not significantly better than Cd2+-containing QDots for long-term cell imaging and that more work needs to be performed in optimizing the brightness and surface chemistry of cadmium-free QDots for them to replace currently used Cd2+-containing QDots.