Interobserver agreement on the radiological criteria of the International Panel on the diagnosis of multiple sclerosis

We assessed the interobserver agreement on the radiological part of the International Panel (IP) criteria for the diagnosis of multiple sclerosis (MS), comprising the assessment of dissemination in space (DIS) and time (DIT) based exclusively on MRI. Four radiologists trained and four radiologists naive in the application of the IP criteria scored the fulfillment for DIS (i.e., > or =3 periventricular, > or =1 infratentorial, > or =1 juxtacortical, > or = 1 enhancing lesion or a total of > or =9 T2-weighted lesions) and DIT (presence of new or enhancing lesions at follow-up) in baseline and two follow-up scans from 20 patients suspected for having MS. The IP-trained radiologists agreed at least moderately on all assessments (kappa>0.40), whereas the IP-naive radiologists showed fair agreement (kappa<0.40) on five of 16 assessments. In the final conclusion on DIS and DIT, the IP-trained radiologists agreed substantially on both items (kappa=0.62 and kappa=0.60, respectively) compared with a fair agreement on DIS (kappa=0.29) and moderate agreement on DIT (kappa=0.52) among the IP-naive radiologists. Given the poor interobserver agreement among IP-naive observers, the new IP criteria for MS require additional training and should perhaps be simplified.