Diagnosis of prostate cancer currently involves visual examination of samples for the assignment of Gleason grades using a microscope, a time-consuming and subjective process. Computer-aided diagnosis (CAD) of histopathology images has become an important research area in diagnostic pathology. This paper presents a scheme to improve the accuracy of existing CAD systems for Gleason grading on digital biopsy slides by combining color and multi-scale information using quaternion algebra. The distinguishing features of presented algorithm are: 1) use of the quaternion wavelet transform and modified local binary patterns for the analysis of image texture in regions of interest; 2) A two-stage classification method: (a) a quaternion neural network with a new high-speed learning algorithm used for multiclass classification, and (b) several binary Support Vector Machine (SVM) classifiers used for classification refinement. In order to evaluate performance, hold-one-out cross validation is applied to a data set of 71 images of prostatic carcinomas belonging to Gleason grades 3, 4 and 5. The developed system assigns the correct Gleason grade in 98.87% of test cases and outperforms other published automatic Gleason grading systems. Moreover, averaged over all the classes, testing of the proposed method shows a specificity rate of 0.990 and a sensitivity rate of 0.967. Experimental results demonstrate the proposed scheme can help pathologists and radiologists diagnose prostate cancer more efficiently and with better reproducability.