Monitoring public psychological and behavioural responses during the early phase of the coronavirus disease 2019 (COVID-19) outbreak is important for the management and control of infection. This study aims to investigate the temporal trend in (1) avoidance and protective behaviors, (2) fear, (3) socio-economic impact, and (4) anxiety levels during the early phase of the COVID-19 pandemic. As a high level of anxiety may have a detrimental impact during an infectious disease outbreak, factors associated with anxiety were also explored. The survey was carried out for 10 weeks and the responses were divided into three periods of around 3 weeks: 25 January–21 February, 22 February–17 March and 18 March–3 April (the period the Malaysian Government issued Movement Control Order). Findings revealed that most of the pyschobehavioural variables showed small increases during first (25 January–21 February) and second (22 February–17 March) periods, and high psychobehavioral responses were reported during the third period. A total of 72.1% (95%CI = 69.2–75.0) reported moderate to severe anxiety as measured by the State-Trait Anxiety Inventory. Factor influencing moderate to severe anxiety is a high perception of severity (OR = 2.09; 95%CI = 1.48–2.94), high perceived susceptibility (OR = 1.71; 95%CI = 1.17–2.50), high impact score (OR = 1.63; 95%CI = 1.17–2.26) and high fear score (OR = 1.47; 95%CI = 1.01–2.14). In conclusion, the psychological and behavioural responses were found to increase with the progression of the outbreak. High anxiety levels found in this study warrant provision of mental health intervention during the early phase of COVID-19 outbreak. Electronic supplementary material The online version of this article (10.1007/s10865-020-00172-z) contains supplementary material, which is available to authorized users.