Nowadays, there is a trend toward early diagnosis and treatment of rheumatoid arthritis (RA) especially in patients with early signs of bone erosion which can be detected by magnetic resonance imaging (MRI). The aim of following study is to compare the sensitivity and specificity of ultrasonography (US) and conventional radiography (CR) compared to MRI for early detection of bone erosion in RA patients. In 12 patients with RA diagnosis, 120 first to fifth metacarpophalangeal joints and 96 second to fifth proximal interphalangeal joints were examined. Non-contrast MRI, US and CR were performed for bone erosion evaluation. For further analysis, the patients were divided in two equal groups according to disease activity score (DAS28). The overall sensitivity and specificity of US compared to MRI in detecting bone erosion were 0.63 and 0.98, respectively with a considerable agreement (kappa = 0.68, p < 0.001). Sensitivity and specificity of CR compared to MRI in detecting bone erosion were 0.13 and 1.00, respectively (kappa = 0.20, p < 0.001). In patients with more active disease, the sensitivity and specificity were 0.67 and 0.99 (kappa = 0.74, p < 0.001) compared to 0.59 and 0.97 (kappa = 0.61, p < 0.001) for the rest of patients according to DAS28. Conclusively, these findings reveal an acceptable agreement between US and MRI for detection of bone erosion in patients with early RA but not CR. US might be considered as a valuable tool for early detection of bone erosion especially when MRI is not available or affordable. Besides, it seems the US could be more reliable when the disease is more active.