Magnetic resonance imaging and spectroscopy (MRI/MRS) plays a unique role in multiple sclerosis (MS) evaluation, because of its ability to provide both high image contrast and significant chemical change among brain tissues. The image contrast renders the possibility of quantifying the tissue volumetric and texture variations, e.g., cerebral atrophy and progressing speed, reflecting the ongoing destructive pathologic processes. Any chemical change reflects an early sign of pathological alteration, e.g., decreased N-acetyl aspartate (NAA) in lesions and normal appearing white matter, related to axonal damage or dysfunction. Both MRI and MRS encounter partial volume (PV) effect, which compromises the quantitative capability, especially for MRS. This work aims to develop a statistical framework to segment the tissue mixtures inside each image element, eliminating theoretically the PV effect, and apply the framework to the evaluation of MS with cognitive impairment. The quantitative measures from MRI/MRS neuroimaging are strongly correlated with the qualitative neuropsychological scores of Brief Repeatable Battery (BRB) test on cognitive impairment, demonstrating the usefulness of the PV image segmentation framework in this clinically significant problem.