Anosmia was identified as a hallmark of COVID-19 early in the pandemic, however, with the emergence of variants of concern, the clinical profile induced by SARS-CoV-2 infection has changed, with anosmia being less frequent. Here, we assessed the clinical, olfactory and neuroinflammatory conditions of golden hamsters infected with the original Wuhan SARS-CoV-2 strain, its isogenic ORF7-deletion mutant and three variants: Gamma, Delta, and Omicron/BA.1. We show that infected animals develop a variant-dependent clinical disease including anosmia, and that the ORF7 of SARS-CoV-2 contributes to the induction of olfactory dysfunction. Conversely, all SARS-CoV-2 variants are neuroinvasive, regardless of the clinical presentation they induce. Taken together, this confirms that neuroinvasion and anosmia are independent phenomena upon SARS-CoV-2 infection. Using newly generated nanoluciferase-expressing SARS-CoV-2, we validate the olfactory pathway as a major entry point into the brain in vivo and demonstrate in vitro that SARS-CoV-2 travels retrogradely and anterogradely along axons in microfluidic neuron-epithelial networks.