Knee Osteoarthritis (KOA) is a progressive joint disease characterised by stiffness and pain, among others. It is generally diagnosed by evaluating physical symptoms, medical history, and screening techniques. However, conventional methods are often subjective, posing a significant challenge to the early grading of disease progression. To address this issue and support clinical decision-making, we propose an ordinal deep learning framework to study the optimal combination of loss functions, and output methodologies with soft labelling approaches, for automatic KOA severity grading based on Kellgren and Lawrence scores from X-ray images. A total of 20 combinations (2 loss functions x 2 output methodologies x 5 soft labelling approaches) are compared in this study, using a public dataset. The optimal configuration uses the categorical cross entropy loss, a cumulative link model as output, and a beta distribution for soft labelling. The results achieved demonstrate the efficacy of these ordinal classification approaches.