Abstract The fascinating architecture of hyperbranched polymer imparts a truck load of novel properties to the material. Epoxy resin modified Mesua ferrea L. seed oil based hyperbranched polyurethane (MHBPU) nanocomposites were prepared by in situ technique using s-triazine based highly branched poly(amido amine) (HBPAA) modified organo-nanoclay. The HBPAA was synthesized by A 2 + B 3 technique with good yield (>75%) using urea and s-triazine. The formation of the polymer was confirmed with the help of 1H NMR, FTIR, UV spectroscopic, and measurements of solution viscosity with other physical properties. This HBPAA was successfully utilized to swell the montmorillonite organo-nanoclay as the interlayer gallery distance increases up to 8.2 Å, obtained by XRD study. The FTIR further confirmed the presence of interactions of the HBPAA moiety with the organo-clay layers. The formation of nanocomposites was confirmed by FTIR, XRD, SEM, TEM and rheological studies. The improvements of tensile strength (1.7 times) and scratch hardness (2.3 times) along with the dramatic enhancement of thermostability and flame retardancy without compromising impact resistance, bending, and elongation at break of the nanocomposites compared to pristine MHBPU thermoset are the noticeable credits of the present investigation. The results signify the great potential of the studied materials for various advanced applications.