Assignment 1: Simulating Between-Subject and Within-subject t-tests [Due Feb.1(A&B)/Feb.3(C)]

TO BE COMPLETED INDIVIDUALLY

Connect to the simulation page (http://www.ruf.rice.edu/~lane/stat_sim/index.html). Select Repeated Measures. Follow the instructions for how to change the sample size, the mean values, the standard deviation and the value of rho (the correlation between the two groups). Then hit Begin and change the values accordingly. For each simulation, place the percent significant values along with the values used for each simulation in a summary table (you can use the table provided). For Parts A-F use the ‘Simulate 5000’ button. For Part H use the ‘Simulate’ button.

Please also answer ALL questions that require written answers and type your answers on another sheet. Please hand in both your written answers and the summary tables sheet.

Part A

1) Enter the following numbers into the appropriate spaces in the simulation table.

N	Population Mean A	Population Mean B	Rho (r_AB)	Population s.d.
16	148	156	0	16

Now select between-subjects and select the 5000 Simulations button. Record the percent significant. Then select within-subjects and select the button for 5000 Simulations. Make sure that the value of rho = 0.7. Record the percent significant. What happened to the percent significant as you moved from a between-subjects test to a within-subjects test? What do these two simulations tell you about the relative power of the two tests when there is a relationship between the two groups?

Part B

2) Now change the population s.d. from 16 to 8 and conduct both tests (make sure that rho = 0.7 for the within-subject test). Compare your between-subject result to the between-subject result of question 1. Compare your between-subject and within-subject results to the comparable results of question 1. What happens to the percent significant?

3) Now change the population s.d to 24 and conduct the simulation. Make sure that rho = 0.7 for the within-subject test. Compare your between-subject and within-subject results to the comparable results of question 1. What happens to the percent significant?

Part C – return sd back to 16

4) Now change the N to 6 and repeat what you did in the question 1 and conduct both tests. Make sure that rho = 0.7 for the within-subject test. Compare the within-subject result to the within-subject result of question 1. What happens to the percent significant in each case?

5) Change the N to 26 and do the tests. Make sure that rho = 0.7 for the within-subject test. Compare your between-subject and within-subject results to the comparable results of question 1. What happens to the percent significant?

Part D – return N back to 16

6) Change the Population Mean A to 142 and conduct both a between-subject and a within-subject simulation. Make sure that rho = 0.7 for the within-subject test. Compare these between-subject and within-subject results to the comparable results you obtained in question 1. What happens to the percent significant?

7) Change Population Mean A to 154 and re-do the simulations. Make sure that rho = 0.7 for the within-subject test. What happens to the percent significant for the between-subject and within-subject simulations relative to the results for question 1?

Part E – return Population Mean A to 148

8) Conduct a between-subjects simulation. Conduct a within-subject simulation but make sure that rho = 0. Compare the percent significant of the between-subject and within-subject simulations. Which has a greater percent significant? Why do you think this is the case (Hint: take a look at the equation for the t-test to help you)? How to the results compare to the comparable results in question 1?

Part F– Set both Population Mean A and Population Mean B at 148

9) Conduct both a between-subject and a within-subject simulation (Make sure that rho = 0.7 for the within subject test). What is the percent significant?

10) Change the SD to 8 and conduct both a between-subject and a within-subject simulation (Make sure that rho = 0.7 for the within subject test). What is the percent significant?

11) Change the SD to 24 and conduct both a between-subject and a within-subject simulation (Make sure that rho = 0.7 for the within subject test). What is the percent significant?

12) Return the SD to 16. Now change the N to 6 and conduct both a between-subject and a within-subject simulation (Make sure that rho = 0.7 for the within subject test). What is the percent significant?

13) Now change the N to 26 and conduct both a between-subject and a within-subject simulation (Make sure that rho = 0.7 for the within subject test). What is the percent significant?

14) If you decided to change the value of rho for the within-subjects test, what value would you expect to emerge for the percent significant under these circumstances? Explain your reasoning.

Part G – Look at your findings for Parts A through F to help you answer the following questions

15) If you had a large SD what could you do to increase the likelihood of finding a significant difference if it really exists?

16) If you only had access to a limited number of participants, what could you do to increase the likelihood of finding a significant difference if it really exists?

17) If you know that the difference between means is very small, what could you do to increase the likelihood of finding a significant effect?

Part H - use the ‘Simulate’ button not the ‘Simulate 5000’ button for the remaining simulations

18) Enter the following numbers into the simulation table. Set the test for Within-subjects

N	Population Mean A	Population Mean B	Rho (r_AB)	Population s.d.
20	22	12	0.1	18

Now hit the ‘Simulate’ button 10 times. You will see a series of lines appear on the grid below, as well as the t value appear. Record the t-value for each of the 10 simulations in the second table on the Summary Tables sheet.

19) Now change the value of rho to 0.9 and hit the ‘Simulate’ button 10 times. Record the t-value for each of the 10 simulations in the second table on the Summary Tables sheet.

20) In general, what happens to the t-value as rho increases?

21) What does this tell you about the nature of the relationship (i.e., what type of relationship exists) between rho (r in the equation) and the value of t?

22) Enter the following information into the simulation table.

N	Population Mean A	Population Mean B	Rho (r_AB)	Population s.d.
8	92	85	0	12

Select the between-subjects button. Hit the ‘Simulate’ button 10 times, recording the t-value, numerator and denominator values, the df and the critical t value each time. Place all of this information in the table corresponding to this question (the third table on the Summary Tables sheet).

Now enter the following information into the simulation table. Select Within-Subjects

N	Population Mean A	Population Mean B	Rho (r_AB)	Population s.d.
8	92	85	0.5	12

Hit the ‘Simulate’ button 10 times, recording the t-value, numerator and denominator, the df and the critical t-value each time. Place all of this information in the table corresponding to this question (the third table on the Summary tables sheet).

Now enter the following information into the simulation table. Select Within-subjects

N	Population Mean A	Population Mean B	Rho (r_AB)	Population s.d.
8	92	85	0.8	12

(i) Calculate the mean t-value for each of the 3 sets of simulations and provide the appropriate df and critical value for each type of test.

(ii) Calculate the mean denominator for each of the 3 sets of simulations. Please report these values.

(iii) What do you notice about the mean t-values for the three simulations?

(iv) What do you notice about the mean denominator values across the three simulations? Does this make sense given what you know about the denominator of the t-test? Explain your reasoning.

(v) Take a look at the df and the critical values for the between-subjects and within-subjects t-tests. What do you notice? What does this tell you about the relationship between df and critical values?