CIN and clinical research:
How should we interpret the data?

Donal Reddan^l and Arif Mutwali^2
lConsultant Nephrologist, Galway University Hospitals, Republic of Ireland
²SpR in Nephrology, Galway University Hospitals, Republic of Ireland

Address for correspondence:
Donal Reddan, Unit 1, Merlin Park,
Merlin Park University Hospital,
Dublin Road,
Galway,
Republic of Ireland
Tel: +353-(091)-751131
Fax: +353-(091)-775565
Email: donal@westernnephrology.com

Abstract
When interpreting clinical papers, it is important that a number of key principles are considered. These include internal validity (the robustness of the conclusions drawn from the study), external validity (the relevance of the study’s results to the broader population), power, and clinical relevance.

This paper is an attempt to illustrate these principles as they relate to research in the area of contrast-induced nephropathy (CIN), a common cause of hospital-acquired renal insufficiency. Clinical research in the area of CIN is complicated by varying definitions, the clinical relevance of chosen endpoints, and the timing and frequency of post-dose SCr measurements. These issues and others should be considered when interpreting the results from clinical studies of CIN.

Introduction
Clinical research encompasses a wide range of scientific endeavour, including retrospective database analyses, clinical trials, and meta-analyses. When considering such clinical papers, it is important that a number of key principles are considered. Of particular relevance to clinical trials are the issues of internal validity, external validity, power, and clinical relevance.

In recent years much evidence has accumulated relating to CIN. Various papers report incidence, consequences, comparisons of prevention strategies, and comparisons of various contrast agents. The published papers in this area vary considerably with regard to the nature and robustness of their conclusions. This paper is an attempt to outline some of the limitations of CIN-associated literature and to provide some guidance as to how the various limitations of the studies can be incorporated into interpretation of results and conclusions. Issues of interpretation of epidemiological studies, clinical trials, and meta-analyses involving the study of contrast nephropathy will all be considered. This paper is not a comprehensive review of the CIN literature but merely an exercise in pointing out some common limitations in study design.

Contrast-induced nephropathy is a common cause of hospital-acquired renal insufficiency. In the hospital setting, nearly half the cases of CIN occur after coronary angiography/angioplasty while one third appear to be associated with computed tomography (CT) procedures[1]. The number of patients at risk is increasing. Estimated population-based prevalence of chronic kidney disease (CKD) has increased from 10.0% in 1988–1994 to 13.1% in 1999–2004[2]. More and more elderly patients who are at increased risk of CIN due to renal and cardiovascular comorbidities commonly found in this age group are undergoing procedures in the CT and/or cardiology/angiography setting[3-5]. Recent concerns about nephrogenic systemic fibrosis associated with the use of gadolinium-based magnetic resonance imaging agents in patients with advanced CKD (stages 4 and 5: eGFR <30 mL/min) may have also created a demand for alternative procedures, resulting in increased use of iodinated contrast media[6]. Because of the increased use of iodinated contrast media arising from these circumstances, the overall burden of CIN is likely to increase and evidence-based direction from well-designed studies is needed to optimise outcomes.

Observational studies
The majority of the evidence relating to the epidemiology of CIN arises from observational studies. Indeed, the incidence and predictors of CIN have been extensively explored in many large institutional databases[7-10 ].

These analyses are quite robust and can be helpful in identifying populations at risk in whom appropriate precautions should be implemented. Nevertheless, it is important to recognise that the studies were carried out in different populations and use different definitions of CIN. In the general population, the overall incidence of CIN has been reported at 0.6–2.3%[11]. This wide range in incidence is representative of considerable heterogeneity between studies, both in terms of populations studied and in methodology.

Incidences vary considerably depending on the population studied. In an interventional cardiology registry from the Mayo clinic, the incidence of CIN using a definition of an increment in serum creatinine (SCr) ≥0.5 mg/dL (44.2 µmol/L) was reported at 3.3%.8 In another study of percutaneous coronary intervention (PCI) patients, the incidence was as high as 14.5%[12]. However, the latter study used the considerably more sensitive CIN definition of an increment in SCr of ≥25%. Percentage increases are generally more sensitive in this context, as small increments in creatinine that do not meet absolute change criteria often do meet percentage criteria. A study of patients undergoing non-emergent, contrast-enhanced computed tomography (CT) in the inpatient and outpatient setting reported an incidence of 6.5% using a definition of an increment in SCr ≥25%.10 The variations in incidences seen can, to some extent, be attributed to differing levels of baseline risk in studied populations. For example, CT scan study patient populations would in general be expected to be more stable haemodynamically than patients undergoing PCI and would also be more stable when exposed to contrast volumes above threshold limits for safety. Differences in definition are also an important consideration. Because the definition has lacked standardisation over the years, there are often inconsistencies between studies in terms of definitions used which make direct comparisons between populations difficult. It is also important that the CIN definition incorporates the element of timing of creatinine measurements.

These studies and others that report varying incidences of CIN also differ in terms of the timing of SCr measurements. Studies with multiple post-procedure measurements at standardised time-points may have higher reported incidences than studies involving patients whose follow-up assessment of renal function is somewhat less rigorous, such as a single post-procedure SCr measurement at a random time-point. Differences in timing or frequency of measurement are also often not clearly evident from study reports. The timing is of particular relevance when one considers that the peak and time-course of CIN can vary from patient to patient and also between different contrast agents[13].

Other studies determine associations of CIN with various risk factors and the association of the development of CIN with adverse outcomes[7, 8] The interpretation of multivariable analyses such as these is often fraught with the issue of causation. An association between two variables or clinical events does not confirm that one event occurs as a direct result of the other. Contrast-induced nephropathy is clearly associated with morbidity, yet it is important to remember that this association does not necessarily imply that the morbidity occurs as a result of CIN. The precise role of CIN in some of the outcomes with which it has been associated still needs to be determined.

Nevertheless, many studies have described a clear relationship between CIN and various adverse outcomes such as in-hospital mortality and morbidity[8, 12]. The association of CIN with outcomes such as these has led to the hypothesis that preventing CIN might lead to reduced mortality and reduced morbid events. However, this hypothesis is difficult to test rigorously, as clinical trials tend to use CIN itself as an outcome, as it can be defined in clear biochemical terms, rather than more clinically relevant outcomes, such as mortality or morbidity, which can be harder to define or have much lower event rates and therefore require more patients to achieve adequate power.

Clinical trials
When considering clinical trials, it is important to systematically evaluate the evidence to determine validity. There are many clinical trials exploring various aspects of CIN. The majority of studies can be divided into two groups: first, those studies involving comparisons of peri-procedural CIN prevention strategies/measures, and second, comparisons between contrast agents to determine their relative safety in patients at higher risk.

The external validity[14] of a study refers to the applicability of its results to a population of interest. For example, one of the CIN trials was performed in a South Korean population[15]. As SCr measurements are generally lower in that population than in European or North American populations, it is difficult to relate absolute SCr measurements to a Western population. Nevertheless, it is still possible to draw information from that study, in terms of the relative risk of CIN for the two contrast agents, as patients in both treatment arms had similar baseline SCr measurements. The issue that arises is the external validity of the CIN definition used in that trial when applied to other studies.

Contrast-induced nephropathy studies are in general strikingly lacking in adequate power. This raises the possibility that many of the studies that failed to reach statistical significance may have resulted in more definitive conclusions, had numbers been adequate; conversely, some statistically significant results may have occurred secondary to chance. The study by Merten and colleagues[16] that demonstrated superiority of sodium bicarbonate over normal saline in the prevention of contrast nephropathy is important in that context, as the significance of the study is dependent on a one-patient differential. If one less patient in the saline group had developed CIN then the results would have not have been significant. This study is only one example of a small study in a high-impact journal that may have influenced care; other examples include the first study to demonstrate efficacy for N-acetyl cysteine (NAC)[17] and a study demonstrating efficacy for haemofiltration in preventing CIN[18] In addition, studies with small numbers and those with inconclusive results rarely make it into high-impact journals.

Power is also a factor when considering the conclusions drawn from studies. Studies designed to demonstrate equivalence between two agents generally require more subjects and greater power than studies designed to demonstrate superiority of one agent over another. Where studies designed to demonstrate superiority do not reveal one agent to be superior over the other, it is often not appropriate to claim that such studies demonstrate equivalence.

In addition, it is important to consider the clinical relevance of all of these studies. Caution may be necessary when interpreting conclusions based on various outcomes (endpoints), which have not been linked clearly with more robust clinical outcomes, such as death or requirement for renal replacement therapy.

Many pharmacologic premedications have been tested as potential strategies to prevent CIN, and unfortunately the majority have not proven to be efficacious. There are many examples of small studies that suggest the efficacy of certain strategies but the area is severely limited by positive publication bias and small numbers. The two areas that have been studied most extensively in terms of CIN prevention include NAC and comparisons between various forms of volume repletion strategies.

N-acetyl cysteine is perhaps the most studied preventative intervention. There have been at least 22 studies and at least 7 published meta-analyses of NAC in the prevention of CIN to date[19-25]. Whether NAC is truly of benefit remains unclear. However, recent guidelines and review articles appear to suggest that an equivocal evaluation is appropriate[26, 27] An interesting recent observation has been the fact that studies reaching a positive conclusion relating to NAC were published in journals with higher impact factors than studies reaching negative conclusions[28]. It is also evident that a significant number of negative studies have remained unpublished or published only in abstract form[28]. Thus, positive publication bias has had an impact on and perhaps led to an over-estimation of the perceived benefit of NAC.

Aside from the usual limitations of small numbers and endpoints of questionable clinical relevance, it is also important to consider that NAC may actually cause a small decrease in SCr levels that leads to an apparent increment in eGFR. This apparent ‘effect’ of NAC on renal function is not however evident when other surrogates for renal function, such as cystatin C, are used[29]. Such an apparent improvement could be inappropriately interpreted as a reduction in the incidence of CIN unless adjusted for and needs to be considered in the interpretation of any study purporting to show benefit.

Many of the studies comparing various fluid regimens such as bicarbonate-based solutions and saline-based solutions are also worth considering. Timing of fluid administration and rates also vary. These studies are often small and may potentially be underpowered[30]. Power is an important consideration as small positive studies tend to get published more frequently than small negative ones.

Timing of measurements
Among the trials comparing different contrast agents, a major issue to consider is the different timing of SCr measurements. It has become clear from recent trials that the timing of creatinine elevations after contrast procedures varies considerably between different agents, and this can lead to completely different conclusions as to the safety of one agent versus another. Few studies have rigorously evaluated the time course of SCr changes pre-/post-contrast however it is apparent that SCr peak occurs at different times in different patients. It may be that SCr levels peak later in patients with underlying renal insufficiency, raising further issues as to the importance of the timing and frequency of testing required[31].

The definition of CIN is another important factor. It is unfortunate that it is not clear how clinically relevant the currently accepted definition is. Over the past decade, the definition of CIN used in studies has varied considerably. The definitions generally rely on changes in SCr and the changes can either be reflected as absolute changes, percentage changes, or a combination of both. There is also significant variability in terms of the magnitude of the changes required to meet the criteria. For the range of absolute change in creatinine required, the accepted criteria range from 0.2–1.0 mg/dL (17.6–88.4 µmol/L) while for percentage change, varying estimates from 25–50% have been incorporated. The clinical relevance of these thresholds has never been clearly tested. Currently, the most widely accepted definition of CIN incorporates either an absolute increment in creatinine of 0.5 mg/dL (44.2 µmol/L) or a percentage increment of 25%. Aside from the increments involved, standardised timing is critical, as are the number of measurements taken. Too many measurements (for example, morning and night evaluations for 5 days) could result in unusual one-off changes being deemed clinically relevant whereas too few (for example, one evaluation every 5 days) could lead to non-detection of actual cases.

A number of studies have compared contrast agents in CT, which involves the intravenous administration of a contrast medium. The four most recent studies – IMPACT[32], ACTIVE[33], PREDICT[34] and Nguyen et al [35] – are compared in Tables 1 and 2. In Table 1, it is apparent that there are many differences in study design between the various studies that make interpretation and comparison difficult. For example, PREDICT[34] included only diabetic patients with renal impairment, whilst the majority of patients in both IMPACT[32] and ACTIVE[33] did not have diabetes. The use of volume expansion and/or NAC also varied considerably within and between studies and could have had an impact on the results. The numbers involved in all of the studies were relatively small and this precluded the use of hard and hypothetically more clinically relevant outcomes than CIN.

Table 2 points out the considerable variation in CIN definitions used and how this might impact the interpretation of results. For example, in the ACTIVE study[33], the results were significant for one definition of CIN (SCr increase ≥0.5 mg/dL; 44.2 μmol/L) and not for another (SCr increase ≥25%). Furthermore, with only five cases of CIN in this study, issues relating to population size and causality also need to be considered, especially as four of the five patients who experienced a CIN event had a medical history that included chronic renal failure (whilst no information regarding the medical history of patients without CIN was given). It is also clear from Table 1 that the frequency of SCr assessments differed considerably and this could account for some of the differing results observed. Only the Nguyen study[35] used multiple standardised measurements with internal consistency; the other three studies relied on a single non-standardised assessment of post-procedure SCr, taken at a random point within a broad time interval from 48–72 hours after the administration of the contrast medium (IMPACT trial[32], 48–72+/-6 h; ACTIVE[33] and PREDICT[34] trials 48–72 h).

Studies investigating CIN that involved intra-arterial administration of a contrast medium—such as cardiac catheterisation—also have differences in design that make interpretation of the results a challenge. A key difference is in the timing of post-dose SCr measurement, with many studies employing a standardised timing[15, 36-42] The CARE study[43] is the only intra-arterial trial published to date that did not follow a standardised post-dose measurement for all patients, and used a single post-dose measurement per patient, with timing ranging from 2–5 days. It is notable that results from the CARE study[43] differed considerably depending on whether early or late SCr values were used to define outcomes.

In considering study design and reporting, the issue of pharmaceutical industry funding cannot be ignored. Subtle alterations in study design with regard to endpoint definition and population selection can have profound implications for study results and potential conflicts of interest cannot be ignored in that context. Efforts of the medical community to address important issues such as these are reflected in reported Good Publication Practice guidelines and required public registration of clinical trial details[44, 45].

Conclusion
Clinical research in the area of CIN is complicated by varying definitions, the clinical relevance of chosen endpoints, the number and timing of post-dose SCr measurements, and the lack of adequate power in many studies. In evaluating the evidence from both observational studies and clinical trials, the implications of heterogeneic populations, varied and non-standardised CIN definitions and the timing of SCr measurements in reaching conclusions should be considered together with any potential conflicts of interest among those authoring and designing the various studies.


References

1. Nash K, Hafeez A, Hou S. Hospital-acquired renal insufficiency. Am J Kidney Dis 2002;39:930-936.
2. Coresh J, Selvin E, Stevens LA, et al. Prevalence of chronic kidney disease in the United States. JAMA 2007;298:2038-47.
3. Toms AP, Cash CJ, Linton SJ, Dixon AK. Requests for body computed tomography: increasing workload, increasing indications and increasing age. Eur Radiol 2001;11:2633-7.
4. Gerber Y, Rihal CS, Sundt TM, 3rd, et al. Coronary revascularization in the community a population-based study, 1990 to 2004. J Am Coll Cardiol 2007;50:1223-9.
5. Detrenis S, Meschi M, Bertolini L, Savazzi G. Contrast medium administration in the elderly patient: is advancing age an independent risk factor for contrast nephropathy after angiographic procedures? J Vasc Interv Radiol 2007;18:177-85; quiz 185.
6. Kanal E, Broome DR, Martin DR, Thomsen HS. Response to the FDA's May 23, 2007, nephrogenic systemic fibrosis update. Radiology 2008;246:11-4.
7. Mehran R, Aymong ED, Nikolsky E, et al. A simple risk score for prediction of contrast-induced nephropathy after percutaneous coronary intervention: development and initial validation. J Am Coll Cardiol 2004;44:1393-9.
8. Rihal CS, Textor SC, Grill DE, et al. Incidence and prognostic importance of acute renal failure after percutaneous coronary intervention. Circulation 2002;105:2259-2264.
9. Weisbord SD, Chen H, Stone RA, et al. Associations of increases in serum creatinine with mortality and length of hospital stay after coronary angiography. J Am Soc Nephrol 2006;17:2871-7.
10. Weisbord SD, Mor MK, Resnick AL, Hartwig KC, Palevsky PM, Fine MJ. Incidence and outcomes of contrast-induced AKI following computed tomography. Clin J Am Soc Nephrol 2008;3:1274-81.
11. Lasser EC, Lyon SG, Berry CC. Reports on contrast media reactions: analysis of data from reports to the U.S. Food and Drug Administration. Radiology 1997;203:605-10.
12. McCullough PA, Wolyn R, Rocher LL, Levin RN, O'Neill WW. Acute renal failure after coronary intervention: incidence, risk factors, and relationship to mortality. Am J Med 1997;103:368-75.
13. Rudnick MR, Goldfarb S, Wexler L, et al. Nephrotoxicity of ionic and nonionic contrast media in 1196 patients: a randomized trial. The Iohexol Cooperative Study. Kidney Int 1995;47:254-61.
14. Persaud N, Mamdani MM. External validity: the neglected dimension in evidence ranking. J Eval Clin Pract 2006;12:450-3.
15. Jo SH, Youn TJ, Koo BK, et al. Renal toxicity evaluation and comparison between Visipaque (iodixanol) and Hexabrix (ioxaglate) in patients with renal insufficiency undergoing coronary angiography: the RECOVER study: a randomized controlled trial. J Am Coll Cardiol 2006;48:924-30.
16. Merten GJ, Burgess WP, Gray LV, et al. Prevention of contrast-induced nephropathy with sodium bicarbonate: a randomized controlled trial. JAMA 2004;291:2328-34.
17. Tepel M, van der Giet M, Schwarzfeld C, Laufer U, Liermann D, Zidek W. Prevention of radiographic-contrast-agent-induced reductions in renal function by acetylcysteine. N Engl J Med 2000;343:180-4.
18. Marenzi G, Marana I, Lauri G, et al. The prevention of radiocontrast-agent-induced nephropathy by hemofiltration. N Engl J Med 2003;349:1333-40.
19. Alonso A, Lau J, Jaber BL, Weintraub A, Sarnak MJ. Prevention of radiocontrast nephropathy with N-acetylcysteine in patients with chronic kidney disease: a meta-analysis of randomized, controlled trials. Am J Kidney Dis 2004;43:1-9.
20. Duong MH, MacKenzie TA, Malenka DJ. N-acetylcysteine prophylaxis significantly reduces the risk of radiocontrast-induced nephropathy: comprehensive meta-analysis. Catheter Cardiovasc Interv 2005;64:471-9.
21. Kshirsagar AV, Poole C, Mottl A, et al. N-acetylcysteine for the prevention of radiocontrast induced nephropathy: a meta-analysis of prospective controlled trials. J Am Soc Nephrol 2004;15:761-9.
22. Liu R, Nair D, Ix J, Moore DH, Bent S. N-acetylcysteine for the prevention of contrast-induced nephropathy. A systematic review and meta-analysis. J Gen Intern Med 2005;20:193-200.
23. Misra D, Leibowitz K, Gowda RM, Shapiro M, Khan IA. Role of N-acetylcysteine in prevention of contrast-induced nephropathy after cardiovascular procedures: a meta-analysis. Clin Cardiol 2004;27:607-10.
24. Zagler A, Azadpour M, Mercado C, Hennekens CH. N-acetylcysteine and contrast-induced nephropathy: a meta-analysis of 13 randomized trials. Am Heart J 2006;151:140-5.
25. Gonzales DA, Norsworthy KJ, Kern SJ, et al. A meta-analysis of N-acetylcysteine in contrast-induced nephrotoxicity: unsupervised clustering to resolve heterogeneity. BMC Med 2007;5:32.
26. Barrett BJ, Parfrey PS. Clinical practice. Preventing nephropathy induced by contrast medium. N Engl J Med 2006;354:379-86.
27. Thomsen HS. European Society of Urogenital Radiology (ESUR) guidelines on the safe use of iodinated contrast media. Eur J Radiol 2006;60:307-13.
28. Vaitkus PT, Brar C. N-acetylcysteine in the prevention of contrast-induced nephropathy: publication bias perpetuated by meta-analyses. Am Heart J 2007;153:275-80.
29. Hoffmann U, Fischereder M, Kruger B, Drobnik W, Kramer BK. The value of N-acetylcysteine in the prevention of radiocontrast agent-induced nephropathy seems questionable. J Am Soc Nephrol 2004;15:407-10.
30. Navaneethan SD, Singh S, Appasamy S, Wing RE, Sehgal AR. Sodium bicarbonate therapy for prevention of contrast-induced nephropathy: a systematic review and meta-analysis. Am J Kidney Dis 2009;53:617-627.
31. Chen SL, Zhang J, Yei F, et al. Clinical outcomes of contrast-induced nephropathy in patients undergoing percutaneous coronary intervention: A prospective, multicenter, randomized study to analyze the effect of hydration and acetylcysteine. Int J Cardiol 2008;126:407-413.
32. Barrett BJ, Katzberg RW, Thomsen HS, et al. Contrast-induced nephropathy in patients with chronic kidney disease undergoing computed tomography: a double-blind comparison of iodixanol and iopamidol. Invest Radiol 2006;41:815-21.
33. Thomsen HS, Morcos SK, Erley CM, Grazioloi L, Bonomo L, Ni Z. The ACTIVE Trial: Comparison of the effects on renal function of iomeprol-400 and idoixanol-320 in patients with chronic kidney disease undergoing abdominal computed tomography. Invest Radiol 2008;43:170-178.
34. Kuhn MJ, Chen N, Sahani DV, et al. The PREDICT study: a randomized double-blind comparison of contrast-induced nephropathy after low- or isoosmolar contrast agent exposure. AJR Am J Roentgenol 2008;191:151-7.
35. Nguyen SA, Suranyi P, Ravenel JG, et al. Iso-osmolality versus low-osmolality iodinated contrast medium at intravenous contrast-enhanced CT: effect on kidney function. Radiology 2008;248:97-105.
36. Aspelin P, Aubry P, Fransson SG, Strasser R, Willenbrock R, Berg KJ. Nephrotoxic effects in high-risk patients undergoing angiography. N Engl J Med 2003;348:491-499.
37. Nie B, Cheng WJ, Li YF, et al. A prospective, double-blind, randomized, controlled trial on the efficacy and cardiorenal safety of iodixanol vs. iopromide in patients with chronic kidney disease undergoing coronary angiography with or without percutaneous coronary intervention. Catheter Cardiovasc Interv 2008;72:958-65.
38. Hardiek KJ, Katholi RE, Robbs RS, Katholi CE. Renal effects of contrast media in diabetic patients undergoing diagnostic or interventional coronary angiography. J Diabetes Complications 2008;22:171-7.
39. Rudnick MR, Davidson C, Laskey W, Stafford JL, Sherwin PF. Nephrotoxicity of iodixanol versus ioversol in patients with chronic kidney disease: the Visipaque Angiography/Interventions with Laboratory Outcomes in Renal Insufficiency (VALOR) Trial. Am Heart J 2008;156:776-82.
40. Hernandez F, Mora L, Suberviola V, et al. Comparison of iodixanol versus ioversol for prevention of contrast induced nephropathy in diabetic patients undergoing coronary angiography or intervention. Eur Heart J 2007;28:454.
41. Wessely R, Koppara T, Kastrati A, et al. Randomized clinical trial to compare the nephrotoxic effects of iso-osmolar versus low-osmolar contrast medium in patients with impaired renal function undergoing percutaneous coronary intervention: The COntrast media and NephroToxicity following coronary Revascularization by AngioplaSTy (CONTRAST) Study. Presentation numer 2411-13. Paper presented at: American College of Cardiology Annual Scientific Session; March 29-April 1, 2008; Chicago, IL, USA.
42. Mehran R, Nikolsky E, Kirtane AJ, et al. Ionic low-osmolar versus nonionic iso-osmolar contrast media to obviate worsening nephropathy after angioplasty in chronic renal failure patients. The ICON (Ionic versus non-ionic Contrast to Obviate worsening Nephropathy after angioplasty in chronic renal failure patients) Study. J Am Coll Cardiol Intv 2009;2:415-421.
43. Solomon RJ, Natarajan MK, Doucet S, et al. Cardiac Angiography in Renally Impaired Patients (CARE) study: a randomized double-blind trial of contrast-induced nephropathy in patients with chronic kidney disease. Circulation 2007;115:3189-96.
44. Wager E, Field EA, Grossman L. Good publication practice for pharmaceutical companies. Curr Med Res Opin 2003;19:149-54.
45. De Angelis C, Drazen JM, Frizelle FA, et al. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. Lancet 2004;364:911-2.


JB3733

 

Source: C2I2, web 09-01

Read also - related articles