Assignment 1 – Simulating Between-Subject and Within-subject t-tests (Due Jan21/23) – TO BE COMPLETED INDIVIDUALLY

Connect to the simulation page (http://www.ruf.rice.edu/~lane/stat_sim/index.html). Select Repeated Measures. Follow the instructions for how to change the sample size, the mean values, the standard deviation and the value of rho (the correlation between the two groups). Then hit Begin and change the values accordingly. For each simulation, place the percent significant values along with the values used for each simulation in a summary table (you can use the table provided in ACME). Please answer ALL questions and type your answers on another sheet. For Parts A-F use the ‘Simulate 5000’ button. For Part G use the ‘Simulate’ button.

Please hand in both your written answers and the summary tables.

Part A

1) Enter the following numbers into the appropriate spaces in the simulation table.

N	Population Mean A	Population Mean B	Rho (r_AB)	Population s.d.
16	45	55	0	15

Now select between-subjects and select the 5000 Simulations button. Then select within-subjects and select the button for 5000 Simulations (make sure that rho = 0). Report the percent significant for both types of t-tests. What does this tell you about the relative power of the two tests when there is no correlation between the two groups?

2) Now do the same thing (i.e., conduct a between-subjects simulation and a within-subjects simulation), but for the within-subjects simulation, set rho at 0.6. What happens to the percent significant? What do these two simulations tell you about the relative power of the two tests when there is a relationship between the two groups?

3) Calculate the effect size of these data. What is your Type II error rate?

Part B

4) Now change the N to 24 and repeat what you did in the question 2 and conduct both tests. Make sure that rho = 0.6 for the within-subject test. Compare your between-subject result to the between-subject result of question 2. Compare the within-subject result to the within-subject result of question 2. What happens to the percent significant in each case?

5) Change the N to 8 and do the tests. Compare your between-subject and within-subject results to the comparable results of question 2. What happens to the percent significant?

Part C – return N back to 16

6) Now change the population s.d. from 15 to 10 and conduct both tests (make sure that rho = 0.6 for the within-subject test). Compare your between-subject and within-subject results to the comparable results of question 2. What happens to the percent significant?

7) Now change the population s.d to 20 and conduct the simulation. Compare your between-subject and within-subject results to the comparable results of question 2. What happens to the percent significant?

Part D – return SD back to 15

8) Change the Population Mean A to 50 and conduct both a between-subject and a within-subject simulation. Make sure that rho = 0.6 for the within-subject test. Compare these between-subject and within-subject results to the comparable results you obtained in question 2. What happens to the percent significant?

9) Change Population Mean A to 40 and re-do the simulations. What happens to the percent significant for the between-subject and within-subject simulations relative to the results for question 2?

Part E– Set both Population Mean A and Population Mean B at 45

10) Conduct both a between-subject and a within-subject simulation (Make sure that rho = 0.6 for the within subject test). What is the percent significant?

11) Change the SD to 10 and conduct both a between-subject and a within-subject simulation (Make sure that rho = 0.6 for the within subject test). What is the percent significant?

12) Change the SD to 20 and conduct both a between-subject and a within-subject simulation (Make sure that rho = 0.6 for the within subject test). What is the percent significant?

13) Return the SD to 15. Now change the N to 24 and conduct both a between-subject and a within-subject simulation (Make sure that rho = 0.6 for the within subject test). What is the percent significant?

14) Now change the N to 8 and conduct both a between-subject and a within-subject simulation (Make sure that rho = 0.6 for the within subject test). What is the percent significant?

15) What do you notice about the results of questions 10 through 14? Does this make sense? Explain your reasoning.

Part F

16) Assume that you have an effect size of 0.5. How many participants would you need, to have Power of .95 in a between-subjects experiment?

17) Assume that you have an effect size of 0.5. How many participants would you need, to have Power of .95 in a within-subjects experiment with a rho = 0?

18) Assume that you have an effect size of 0.5. How many participants would you need, to have Power of .95 in a within-subjects experiment with a rho = 0.6?

Part G - use the ‘Simulate’ button not the ‘Simulate 5000’ button for the remaining simulations

19) Enter the following numbers into the simulation table.

N	Population Mean A	Population Mean B	Rho (r_AB)	Population s.d.
16	45	55	0	15

Now select within-subjects and then hit the ‘Simulate’ button 10 times. You will see a series of lines appear on the grid below, as well as the t value appear. Observe the lines and record the t-value for each of the 10 simulations.

20) Now change the value of rho to 0.9 and hit the ‘Simulate’ button 10 times. Observe what happens to the lines and record the t-value for each of the 10 simulations.

21) In general, what happens to the variability of the lines as rho increases?

22) In general, what happens to the t-value as rho increases?

23) What does this tell you about the relationship between rho (r in the equation) and the value of t?

24) Enter the following information into the simulation table.

N	Population Mean A	Population Mean B	Rho (r_AB)	Population s.d.
16	45	55	0	15

Select the between-subjects button. Hit the ‘Simulate’ button 16 times, recording the t-value, numerator and denominator values, the df and the critical t value each time. Place all of this information in the table corresponding to this question.

Now enter the following information into the simulation table.

N	Population Mean A	Population Mean B	Rho (r_AB)	Population s.d.
16	45	55	0.6	15

Select the within-subjects button. Hit the ‘Simulate’ button 16 times, recording the t-value, numerator and denominator, the df and the critical t-value each time. Place all of this information in the table corresponding to this question.

(i) Calculate the mean t-value for each type of t-test and provide the appropriate df and critical value for each type of test. Please report these values.

(ii) Calculate the mean numerator and mean denominator for each type of test. Please report these values.

(iii) What do you notice about the mean t-values for the two types of tests?

(iv) What do you notice about the mean numerator and denominator values across the two types of tests?

(v) Take a look at the df and the critical values for the between-subjects and within-subjects t-tests. What do you notice? What does this tell you about the relationship between df and critical values?