Objective(s): Accurate detection and competent management of thyroid nodules, as a common disease, basically depends on the reliability of the ultrasonography (US) report. In this research, we evaluated inter and intra-observer variation among ultrasonography reporters, based on ACR-TIRADS.
Methods: In this retrospective study, 345 thyroid US images of 150 patients were reviewed. Three clinicians with at least 6-year experience in thyroid US reviewed the images twice at 6-8 weeks’ intervals. Composition, echogenicity, shape, margin, and echogenic foci based on ACR-TIRADS were reported, independently. Inter and intra-observer variations were calculated based on Cohen’s Kappa statistics.
Results: 345 ultrasonography images of 150 patients with thyroid nodules (83 women and 67 men) with a mean age of 65 years were reviewed. Moderate to the substantial intra-observer agreement was achieved with the highest Kapa value in the category of shape (k=0.61-0.77). For TIRADS level, the moderate intra-observer agreement was observed (k=0.42-0.46). Inter-observer agreement for the US category of thyroid nodules was obtained slightly to moderate. Composition 
(k=0.42 and 0.51) and echogenicity (k=0.45 and 0.46) showed the highest overall agreement and margin showed the lowest overall agreement (k=0.18 and 0.19). In assessing TIRADS level of nodules, a fair agreement was obtained (k=0.23 and 0.29) .
Conclusion: Moderate to substantial intra-observer agreement and slight to moderate inter-observer variation for evaluation of thyroid nodules; shows the need for a computer-aided diagnosis system based on artificial intelligence to 
assist our physicians in differentiating thyroid nodule characteristics based on explicit image features. An additional training course based on ACR-TIRADS for physicians can be another useful recommendation.