• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of annrheumdAnnals of the Rheumatic DiseasesVisit this articleSubmit a manuscriptReceive email alertsContact usBMJ
Ann Rheum Dis. Jul 2005; 64(7): 1043–1049.
Published online Jan 7, 2005. doi:  10.1136/ard.2004.030387
PMCID: PMC1755572

Interobserver reliability of rheumatologists performing musculoskeletal ultrasonography: results from a EULAR "Train the trainers" course


Objective: To evaluate the interobserver reliability among 14 experts in musculoskeletal ultrasonography (US) and to determine the overall agreement about the US results compared with magnetic resonance imaging (MRI), which served as the imaging "gold standard".

Methods: The clinically dominant joint regions (shoulder, knee, ankle/toe, wrist/finger) of four patients with inflammatory rheumatic diseases were ultrasonographically examined by 14 experts. US results were compared with MRI. Overall agreements, sensitivities, specificities, and interobserver reliabilities were assessed.

Results: Taking an agreement in US examination of 10 out of 14 experts into account, the overall κ for all examined joints was 0.76. Calculations for each joint region showed high κ values for the knee (1), moderate values for the shoulder (0.76) and hand/finger (0.59), and low agreement for ankle/toe joints (0.28). κ Values for bone lesions, bursitis, and tendon tears were high (κ = 1). Relatively good agreement for most US findings, compared with MRI, was found for the shoulder (overall agreement 81%, sensitivity 76%, specificity 89%) and knee joint (overall agreement 88%, sensitivity 91%, specificity 88%). Sensitivities were lower for wrist/finger (overall agreement 73%, sensitivity 66%, specificity 88%) and ankle/toe joints (overall agreement 82%, sensitivity 61%, specificity 92%).

Conclusion: Interobserver reliabilities, sensitivities, and specificities in comparison with MRI were moderate to good. Further standardisation of US scanning techniques and definitions of different pathological US lesions are necessary to increase the interobserver agreement in musculoskeletal US.

Full Text

The Full Text of this article is available as a PDF (286K).

Selected References

These references are in PubMed. This may not be the complete list of references from this article.
  • Koski JM. Ultrasonography of the subtalar and midtarsal joints. J Rheumatol. 1993 Oct;20(10):1753–1755. [PubMed]
  • Grassi W, Filippucci E, Farina A, Salaffi F, Cervini C. Ultrasonography in the evaluation of bone erosions. Ann Rheum Dis. 2001 Feb;60(2):98–103. [PMC free article] [PubMed]
  • Backhaus M, Kamradt T, Sandrock D, Loreck D, Fritz J, Wolf KJ, Raber H, Hamm B, Burmester GR, Bollow M. Arthritis of the finger joints: a comprehensive approach comparing conventional radiography, scintigraphy, ultrasound, and contrast-enhanced magnetic resonance imaging. Arthritis Rheum. 1999 Jun;42(6):1232–1245. [PubMed]
  • Szkudlarek M, Court-Payen M, Strandberg C, Klarlund M, Klausen T, Ostergaard M. Power Doppler ultrasonography for assessment of synovitis in the metacarpophalangeal joints of patients with rheumatoid arthritis: a comparison with dynamic magnetic resonance imaging. Arthritis Rheum. 2001 Sep;44(9):2018–2023. [PubMed]
  • Wakefield RJ, Gibbon WW, Conaghan PG, O'Connor P, McGonagle D, Pease C, Green MJ, Veale DJ, Isaacs JD, Emery P. The value of sonography in the detection of bone erosions in patients with rheumatoid arthritis: a comparison with conventional radiography. Arthritis Rheum. 2000 Dec;43(12):2762–2770. [PubMed]
  • Naredo E, Iagnocco A, Valesini G, Uson J, Beneyto P, Crespo M. Ultrasonographic study of painful shoulder. Ann Rheum Dis. 2003 Oct;62(10):1026–1027. [PMC free article] [PubMed]
  • Iagnocco A, Coari G, Palombi G, Valesini G. Sonography in the study of metatarsalgia. J Rheumatol. 2001 Jun;28(6):1338–1340. [PubMed]
  • Koski JM. Detection of plantar tenosynovitis of the forefoot by ultrasound in patients with early arthritis. Scand J Rheumatol. 1995;24(5):312–313. [PubMed]
  • Grassi W, Filippucci E, Farina A, Cervini C. Sonographic imaging of tendons. Arthritis Rheum. 2000 May;43(5):969–976. [PubMed]
  • D'Agostino Maria-Antonietta, Said-Nahal Roula, Hacquard-Bouder Cécile, Brasseur Jean-Louis, Dougados Maxime, Breban Maxime. Assessment of peripheral enthesitis in the spondylarthropathies by ultrasonography combined with power Doppler: a cross-sectional study. Arthritis Rheum. 2003 Feb;48(2):523–533. [PubMed]
  • Terslev L, Torp-Pedersen S, Savnik A, von der Recke P, Qvistgaard E, Danneskiold-Samsøe B, Bliddal H. Doppler ultrasound and magnetic resonance imaging of synovial inflammation of the hand in rheumatoid arthritis: a comparative study. Arthritis Rheum. 2003 Sep;48(9):2434–2441. [PubMed]
  • Schmidt WA, Völker L, Zacher J, Schläfke M, Ruhnke M, Gromnica-Ihle E. Colour Doppler ultrasonography to detect pannus in knee joint synovitis. Clin Exp Rheumatol. 2000 Jul-Aug;18(4):439–444. [PubMed]
  • Carotti M, Salaffi F, Manganelli P, Salera D, Simonetti B, Grassi W. Power Doppler sonography in the assessment of synovial tissue of the knee joint in rheumatoid arthritis: a preliminary experience. Ann Rheum Dis. 2002 Oct;61(10):877–882. [PMC free article] [PubMed]
  • Wakefield Richard J, Brown Andrew K, O'Connor Philip J, Emery Paul. Power Doppler sonography: improving disease activity assessment in inflammatory musculoskeletal disease. Arthritis Rheum. 2003 Feb;48(2):285–288. [PubMed]
  • Manger B, Kalden JR. Joint and connective tissue ultrasonography--a rheumatologic bedside procedure? A German experience. Arthritis Rheum. 1995 Jun;38(6):736–742. [PubMed]
  • Szkudlarek Marcin, Court-Payen Michel, Jacobsen Søren, Klarlund Mette, Thomsen Henrik S, Østergaard Mikkel. Interobserver agreement in ultrasonography of the finger and toe joints in rheumatoid arthritis. Arthritis Rheum. 2003 Apr;48(4):955–962. [PubMed]
  • Østergaard Mikkel, Wiell Charlotte. Ultrasonography in rheumatoid arthritis: a very promising method still needing more validation. Curr Opin Rheumatol. 2004 May;16(3):223–230. [PubMed]
  • Ribbens Clio, André Béatrice, Marcelis Stefaan, Kaye Olivier, Mathy Luc, Bonnet Valérie, Beckers Catherine, Malaise Michel G. Rheumatoid hand joint synovitis: gray-scale and power Doppler US quantifications following anti-tumor necrosis factor-alpha treatment: pilot study. Radiology. 2003 Nov;229(2):562–569. [PubMed]
  • Backhaus M, Burmester GR, Gerber T, Grassi W, Machold KP, Swen WA, Wakefield RJ, Manger B. Guidelines for musculoskeletal ultrasound in rheumatology. Ann Rheum Dis. 2001 Jul;60(7):641–649. [PMC free article] [PubMed]
  • Balint PV, Sturrock RD. Intraobserver repeatability and interobserver reproducibility in musculoskeletal ultrasound imaging measurements. Clin Exp Rheumatol. 2001 Jan-Feb;19(1):89–92. [PubMed]
  • Scheel Alexander K, Backhaus Marina. Ultrasonographic assessment of finger and toe joint inflammation in rheumatoid arthritis: comment on the article by Szkudlarek et al. Arthritis Rheum. 2004 Mar;50(3):1008–1009. [PubMed]
  • Ostergaard M, Klarlund M, Lassere M, Conaghan P, Peterfy C, McQueen F, O'Connor P, Shnier R, Stewart N, McGonagle D, et al. Interreader agreement in the assessment of magnetic resonance images of rheumatoid arthritis wrist and finger joints--an international multicenter study. J Rheumatol. 2001 May;28(5):1143–1150. [PubMed]
  • Lassere Marissa, McQueen Fiona, Østergaard Mikkel, Conaghan Philip, Shnier Ron, Peterfy Charles, Klarlund Mette, Bird Paul, O'Connor Philip, Stewart Neal, et al. OMERACT Rheumatoid Arthritis Magnetic Resonance Imaging Studies. Exercise 3: an international multicenter reliability study using the RA-MRI Score. J Rheumatol. 2003 Jun;30(6):1366–1375. [PubMed]
  • Østergaard Mikkel, Peterfy Charles, Conaghan Philip, McQueen Fiona, Bird Paul, Ejbjerg Bo, Shnier Ron, O'Connor Philip, Klarlund Mette, Emery Paul, et al. OMERACT Rheumatoid Arthritis Magnetic Resonance Imaging Studies. Core set of MRI acquisitions, joint pathology definitions, and the OMERACT RA-MRI scoring system. J Rheumatol. 2003 Jun;30(6):1385–1386. [PubMed]
  • Hermann Kay-Geert A, Backhaus Marina, Schneider Udo, Labs Karsten, Loreck Dieter, Zühlsdorf Svenda, Schink Tania, Fischer Thomas, Hamm Bernd, Bollow Matthias. Rheumatoid arthritis of the shoulder joint: comparison of conventional radiography, ultrasound, and dynamic contrast-enhanced magnetic resonance imaging. Arthritis Rheum. 2003 Dec;48(12):3338–3349. [PubMed]

Figures and Tables

Figure 1
 Shoulder joint. (A and B) Humeral head erosions. (A) In MRI, multiple erosions can be seen from the anterior and posterior sides of the humeral head as bone defects with sharp margins (arrows). (B) Distinct bone defects below the bone surface ...
Figure 2
 Finger joint (MCP II). (A) The MR image shows the MCP joints II–V in transverse section. Focusing on MCP joint II shows slight contrast enhancement from the dorsal and palmar aspects, representing synovitis (arrowheads). Also, tenosynovitis ...
Figure 3
 Knee joint. (A) MRI shows some contrast agent enhancement in the suprapatellar recess, reflecting inflammatory effusion (two arrows). (B) US also clearly depicts the effusion in the suprapatellar recess (arrows). (C) In MRI, a popliteal cyst ...
Figure 4
 Ankle/toe joints. (A) MRI of the ankle shows contrast enhancement in the tibiotalar joint from anterior and posterior aspects (arrows). (B) The longitudinal US image is an example of the anterior side of the tibiotalar joint. The anechoic area ...

Articles from Annals of the Rheumatic Diseases are provided here courtesy of BMJ Group


Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...


  • PubMed
    PubMed citations for these articles

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...