Power analysis (English) - Revision history

Rebecca Holman at 15:30, 29 January 2020

2020-01-29T15:30:51Z

← Older revision		Revision as of 17:30, 29 January 2020
Line 1:		Line 1:
	This text is an edited version of the AMC sample size calculation manual <cite>[geloven2009]</cite>~~. It~~ provides a practical guide into sample size calculations used in clinical research. After reading the manual, a researcher will know:		This text is an edited version of the AMC sample size calculation manual <cite>[geloven2009]</cite> and provides a practical guide into sample size calculations used in clinical research. After reading the manual, a researcher will know:
	* why power analysis is used to plan and evaluate medical research		* why power analysis is used to plan and evaluate medical research
	* what power and statistical significance mean		* what power and statistical significance mean

Rebecca Holman: /* Why perform a sample size calculation? */

2020-01-29T15:26:59Z

Why perform a sample size calculation?

← Older revision		Revision as of 17:26, 29 January 2020
Line 8:		Line 8:
	In addition, the manual contains two practical examples of sample size calculations.		In addition, the manual contains two practical examples of sample size calculations.

	== ~~Why perform a~~ sample size ~~calculation?~~ ==		== Reasons for performing sample size calculations ==
	The main reasons for perfomring a sample size calculation are ethical. If the number of subjects tested in a study is too small to detect the effect being investigated, the study has an unacceptably large probability of reaching a false negative conclusion. This means that the subjects will be subjected to the risks of participating in the study in vain. On the other hand, including too many subjects may also lead to undesirable situations. If an intervention turns out to be effective, too many subjects have missed out on this intervention. If the intervention is not effective, too many have been exposed to this ineffective intervention. For these reasons a trial should always consider what number of subjects would be appropriate to answer the study question. Sample size calculations prior to a study can help focus on the number of subjects that is needed and sufficient for a study. Moreover, a sample size calculation helps one to focus on a clinically relevant effect, instead of the erroneous strategy of testing as many subjects as needed to reach statistical significance of an irrelevant effect.		The main reasons for perfomring a sample size calculation are ethical. If the number of subjects tested in a study is too small to detect the effect being investigated, the study has an unacceptably large probability of reaching a false negative conclusion. This means that the subjects will be subjected to the risks of participating in the study in vain. On the other hand, including too many subjects may also lead to undesirable situations. If an intervention turns out to be effective, too many subjects have missed out on this intervention. If the intervention is not effective, too many have been exposed to this ineffective intervention. For these reasons a trial should always consider what number of subjects would be appropriate to answer the study question. Sample size calculations prior to a study can help focus on the number of subjects that is needed and sufficient for a study. Moreover, a sample size calculation helps one to focus on a clinically relevant effect, instead of the erroneous strategy of testing as many subjects as needed to reach statistical significance of an irrelevant effect.

Rebecca Holman: /* Why perform a sample size calculation? */

2020-01-29T15:26:34Z

Why perform a sample size calculation?

@@ Line 9: / Line 9: @@
 == Why perform a sample size calculation? ==
-The main reasons for perfomring a sample size calculation are ethical. If the number of subjects tested in a study is too small to detect the effect being investigated, the subjects will be subjected to the risks of participating in the study in vain.
+The main reasons for perfomring a sample size calculation are ethical. If the number of subjects tested in a study is too small to detect the effect being investigated, the study has an unacceptably large probability of reaching a false negative conclusion. This means that the subjects will be subjected to the risks of participating in the study in vain. On the other hand, including too many subjects may also lead to undesirable situations. If an intervention turns out to be effective, too many subjects have missed out on this intervention. If the intervention is not effective, too many have been exposed to this ineffective intervention. For these reasons a trial should always consider what number of subjects would be appropriate to answer the study question. Sample size calculations prior to a study can help focus on the number of subjects that is needed and sufficient for a study. Moreover, a sample size calculation helps one to focus on a clinically relevant effect, instead of the erroneous strategy of testing as many subjects as needed to reach statistical significance of an irrelevant effect.
 == Power and statistical significance ==

Rebecca Holman: /* Comparing two proportions - methods to help stop smoking */

2020-01-29T15:20:11Z

Comparing two proportions - methods to help stop smoking

← Older revision		Revision as of 17:20, 29 January 2020
Line 176:		Line 176:
	9		9

	== Comparing two proportions - methods to help stop smoking ==		=== Comparing two proportions - methods to help stop smoking ===
	Suppose we want set up a randomized controlled trial to compare two methods of helping		Suppose we want set up a randomized controlled trial to compare two methods of helping
	smokers to give up smoking. One group is to be given a new kind of nicotine chewing		smokers to give up smoking. One group is to be given a new kind of nicotine chewing

Rebecca Holman: /* Example of calculations */

2020-01-29T15:19:39Z

Example of calculations

← Older revision		Revision as of 17:19, 29 January 2020
Line 175:		Line 175:
	Figure 2: The assumptions for the CRS study (nQuery screenshot)		Figure 2: The assumptions for the CRS study (nQuery screenshot)
	9		9

			== Comparing two proportions - methods to help stop smoking ==
			Suppose we want set up a randomized controlled trial to compare two methods of helping
			smokers to give up smoking. One group is to be given a new kind of nicotine chewing
			gum and the other group will receive advice from their doctor and a booklet. We want to
			know how many patients to include in our study. Based on published evidence we expect
			that in the advice group 15% of smokers will remain non-smokers at 6 months. We would
			be interested in an improvement to 30% in the group given gum. In nQuery we take the
			following steps:
			1. We go to File → New.
			2. We tick that we are comparing two proportions and that we want to test the difference
			between these two proportions using a Chi-square test (Figure 3).
			3. In the next screen we enter our study assumptions: significance level 0.05, 2 sided
			test, group 1 proportion 0.15, group 2 proportion 0.30, power 80% and see that
			nQuery calculates 121 persons per group to be tested (Figure 4).
			4. To test the effect of using slightly different assumptions and we repeat the calculation
			and see that we need 250 persons per group to detect a difference between proportions
			of 0.10 (0.15 versus 0.25) and only 73 per group to detect a difference between
			proportions of 0.20 (0.15 versus 0.35) (Figure 4). Note: If numbers per group were
			low, we could also check how using a continuity corrected chi-squared test would
			alter our calculations. Continuity adjustment to chi-square is usually applied to
			tables with one or more cells with frequencies less than five. Some authors also apply
			it to all 2 by 2 tables since the correction gives a better approximation to the binomial
			distribution. (Garson, G. David, 2009).
			We here accept that our study is not powered on finding a difference smaller than 15%
			between groups. Anticipating on a 10% attrition rate and keeping our initial assumptions
			for the difference in proportions, we decide to include 135 (121/0.9 = 134,44) persons per
			group, or 270 in total.

	== Software ==		== Software ==

Rebecca Holman: /* Sources of information for sample size calculations */

2020-01-29T15:19:15Z

Sources of information for sample size calculations

← Older revision		Revision as of 17:19, 29 January 2020
Line 141:		Line 141:
	may consider doing a pilot study first.		may consider doing a pilot study first.

			== Example of calculations ==
			=== Comparing two means - fever and chronic rhinosinusitus ===
			Suppose we want to investigate the symptoms of patients undergoing surgery for chronic
			rhinosinusitus (CRS). In particular we are interested whether these patients show signs
			of fever. We plan to measure pre-operative temperature of CRS patients and of healthy
			control patients undergoing a cosmetic nose surgery. We want to know how many patients
			to include in our study to be able to detect a clinically relevant difference between both
			groups.
			From previous studies in which temperature of healthy persons was measured, we estimate that the mean temperature in the control group will be 37o C, with a standard
			deviation of 0.4. In our study we are interested in detecting the clinically relevant difference of 0.5
			o C, which means we want to be able to detect if the CRS group has mean
			temperature 37.5
			o C or higher. We assume that the CRS and the nose surgery group have
			comparable spread in temperature (common standard deviation of 0.4).
			In nQuery we take the following steps:
			1. We go to File → New.
			2. We tick that we are comparing two means and that we want to test the difference
			between these two means using a Student’s t-test assuming equal variances (Figure 1).
			3. In the next screen we enter our study assumptions: significance level 0.05, 2 sided
			test, group 1 mean 37, group 2 mean 37.5, common standard deviation 0.4, power
			80% and see that nQuery calculates 12 persons per group to be tested (Figure 2).
			4. To test the effect of slightly different assumptions on this calculation, we can alter
			some settings and see that we need 17 persons per group to detect a difference of 0.4
			o
			C or to detect the difference 0.5
			o C when standard deviation is higher (Figure 2).
			Anticipating on a 10% attrition rate due to invalid measurements, we decide to include 19
			(
			17
			0.9 = 18.9) persons per group, or 38 in total.
			8
			Figure 1: Standard settings for comparing two means (nQuery screenshot)
			Figure 2: The assumptions for the CRS study (nQuery screenshot)
			9

	== Software ==		== Software ==

Rebecca Holman: /* Software */

2020-01-29T15:18:13Z

Software

← Older revision		Revision as of 17:18, 29 January 2020
Line 143:		Line 143:

	== Software ==		== Software ==
			Several high quality computer programs exist for performing sample size calculations. The AMC has a license for the program nQuery Adviso and for more advanced study designs, the Helpdesk Statistics access to the program NCSS PASS 15. On the internet several free power programs exist. Reliability of these free programs is not always guaranteed.
	Several computer programs exist for performing sample size calculations. The AMC has a
	license for the program nQuery ~~Advisor, a user friendly application that supports several~~
	~~study designs~~ and ~~types of data. For~~ more advanced study designs, ~~KEBB and CRU have~~ access to the program NCSS PASS. On the
	internet several free power programs exist. Reliability of these free programs is not always
	~~ensured. We advise you to use nQuery for your power calculations and contact us when~~
	~~the study design and/or planned analysis is not supported by nQuery~~.

	== Advanced topics ==		== Advanced topics ==

Rebecca Holman: /* Advanced topics */

2020-01-29T15:11:33Z

Advanced topics

← Older revision		Revision as of 17:11, 29 January 2020
Line 152:		Line 152:

	== Advanced topics ==		== Advanced topics ==
	~~Several situations exist which call for more advanced~~ sample size calculations. ~~Here we~~		Some study designs require other types of sample size calculations. Reseachers planning these types of studies should contact a statistician to discuss the calculations required.
	~~point out some~~ of ~~these situations~~ to ~~make you aware of~~ the ~~need for extra effort when the~~
	~~situation occurs~~.

	; Equivalence design		; Equivalence design
	: In an equivalence design you do not want to test for differences, ~~instead you want~~ to show equivalence. In such a design you will need to specify what your interpretation of similar is. Perfect equivalence can never be demonstrated. A limit has to be determined of which small difference between groups will be considered not meaningful and lead to the conclusion of equivalence, this is called the equivalence limit difference. Also the expected difference between groups has to be given. A special type of equivalence designs is a non-inferiority design. In this design one is interested in equivalence in only one test direction. For instance when a new, less invasive diagnostic procedure is compared to the current invasive one, the new procedure does not have to prove better than the current one. If it has at least similar diagnostic strength as the invasive one it would be preferred.		: In an equivalence design you do not want to test for differences, but to show equivalence. In such a design you will need to specify what your interpretation of similar is. Perfect equivalence can never be demonstrated. A limit has to be determined of which small difference between groups will be considered not meaningful and lead to the conclusion of equivalence, this is called the equivalence limit difference. Also the expected difference between groups has to be given. A special type of equivalence designs is a non-inferiority design. In this design one is interested in equivalence in only one test direction. For instance when a new, less invasive diagnostic procedure is compared to the current invasive one, the new procedure does not have to prove better than the current one. If it has at least similar diagnostic strength as the invasive one it would be preferred.

	;Clustered design		;Clustered design
	:~~When randomization~~ of ~~patients in not done~~ individually ~~but in~~ clusters (e.~~g., per treating physician or per department)~~, ~~it is expected that~~ the ~~outcome~~ of patients within a cluster are not independent of each other~~. In the sample size calculation~~ the correlation between patients needs to be ~~accounted for~~. ~~Also when~~ multiple observations per patient are obtained, the power calculation has to be suited to take along the correlation between measurements in the same patient.		:In some studies, patients are randomized inclusters instead of individually. Examples of clusters are physicians' practices or hospital wards. In this type of study, the outcomes of patients within a cluster are not statistically independent of each other and the correlation between patients needs to be included in the sample size calculation. If multiple observations per patient are obtained, the power calculation has to be suited to take along the correlation between measurements in the same patient.

	;Advanced analyses		;Advanced analyses
	:Planned statistical analyses such as survival analysis, regression analysis, and reliability analysis call for their own specific sample size calculation		:Planned statistical analyses such as survival analysis, regression analysis, and reliability analysis call for their own specific sample size calculation



	== Referenties ==		== Referenties ==

Rebecca Holman at 15:03, 29 January 2020

2020-01-29T15:03:51Z

← Older revision		Revision as of 17:03, 29 January 2020
Line 151:		Line 151:
	the study design and/or planned analysis is not supported by nQuery.		the study design and/or planned analysis is not supported by nQuery.

			== Advanced topics ==
			Several situations exist which call for more advanced sample size calculations. Here we
			point out some of these situations to make you aware of the need for extra effort when the
			situation occurs.

			; Equivalence design
			: In an equivalence design you do not want to test for differences, instead you want to show equivalence. In such a design you will need to specify what your interpretation of similar is. Perfect equivalence can never be demonstrated. A limit has to be determined of which small difference between groups will be considered not meaningful and lead to the conclusion of equivalence, this is called the equivalence limit difference. Also the expected difference between groups has to be given. A special type of equivalence designs is a non-inferiority design. In this design one is interested in equivalence in only one test direction. For instance when a new, less invasive diagnostic procedure is compared to the current invasive one, the new procedure does not have to prove better than the current one. If it has at least similar diagnostic strength as the invasive one it would be preferred.

			;Clustered design
			:When randomization of patients in not done individually but in clusters (e.g., per treating physician or per department), it is expected that the outcome of patients within a cluster are not independent of each other. In the sample size calculation the correlation between patients needs to be accounted for. Also when multiple observations per patient are obtained, the power calculation has to be suited to take along the correlation between measurements in the same patient.

			;Advanced analyses
			:Planned statistical analyses such as survival analysis, regression analysis, and reliability analysis call for their own specific sample size calculation

Rebecca Holman at 14:59, 29 January 2020

2020-01-29T14:59:06Z

← Older revision		Revision as of 16:59, 29 January 2020
Line 140:		Line 140:
	absolutely no information is available for the estimation or the necessary input values, one		absolutely no information is available for the estimation or the necessary input values, one
	may consider doing a pilot study first.		may consider doing a pilot study first.


			== Software ==

			Several computer programs exist for performing sample size calculations. The AMC has a
			license for the program nQuery Advisor, a user friendly application that supports several
			study designs and types of data. For more advanced study designs, KEBB and CRU have access to the program NCSS PASS. On the
			internet several free power programs exist. Reliability of these free programs is not always
			ensured. We advise you to use nQuery for your power calculations and contact us when
			the study design and/or planned analysis is not supported by nQuery.