Assignment 1:
Simulating Between-Subject and Within-subject t-tests [Due Jan.31(A&B)/Feb.2(C)]
TO BE COMPLETED INDIVIDUALLY
Connect to the simulation page (http://www.ruf.rice.edu/~lane/stat_sim/index.html). Select Repeated Measures. Follow the instructions for how to change the
sample size, the mean values, the standard deviation and the value of rho (the
correlation between the two groups).
Then hit Begin and change the values accordingly. For each simulation, place the percent significant values along
with the values used for each simulation in a summary table (you can use the
table provided). For Parts A-F use
the ‘Simulate 5000’ button. For Part H
use the ‘Simulate’ button.
Please also answer
ALL questions that require written
answers and type your answers on another sheet. Please hand in both
your written answers and the summary tables sheet.
1) Enter the following
numbers into the appropriate spaces in the simulation table.
|
N |
Population Mean A |
Population Mean B |
|
Population s.d. |
|
20 |
50 |
76 |
0 |
32 |
Now select between-subjects and select the 5000
Simulations button. Record the percent
significant. Then select within-subjects
and select the button for 5000 Simulations.
Make sure that the value of rho = 0.6. Record the percent significant. What happened to the percent significant as
you moved from a between-subjects test to a within-subjects test? What do these two simulations tell you about
the relative power of the two tests when there is a relationship between the
two groups?
2) Now change the N
to 30 and repeat what you did in question 1 (make sure that rho = 0.6 for the within-subject test). Compare your between-subject result to the
between-subject result of question 1.
Compare your within-subject results to the comparable results of
question 1. What happens to the percent significant in each case?
3) Change the N to
10 and do the tests. Make sure that rho = 0.6
for the within-subject test.
Compare your between-subject and within-subject results to the
comparable results of question 1.
What happens to the percent significant?
Part C – return N back to 20
4) Now change the
population s.d. from 32 to 38 and conduct both
tests. Make sure that rho = 0.6 for the
within-subject test. Compare your
between-subject and within-subject results to the comparable results of question
1. What happens to the percent
significant?
5) Now change the
population s.d to 26 and conduct the
simulation. Make sure that rho = 0.6 for the within-subject test. Compare your between-subject and
within-subject results to the comparable results of question 1. What happens to the percent significant?
6) Change the Population
Mean A to 55 and conduct both a between-subject and a within-subject
simulation. Make sure that rho = 0.6
for the within-subject test. Compare
these between-subject and within-subject results to the comparable results you
obtained in question 1. What
happens to the percent significant?
7) Change Population
Mean A to 45 and re-do the simulations.
Make sure that rho = 0.6 for the
within-subject test. What happens
to the percent significant for the between-subject and within-subject
simulations relative to the results for question 1?
8)
Conduct a between-subjects simulation. Conduct a within-subject simulation but make
sure that rho = 0.
Compare the percent significant of the between-subject and
within-subject simulations. Which has a
greater percent significant? Why do you
think this is the case (Hint: take a look at the equation for the t-test to
help you)? How do the results compare to
the comparable results in question 1?
9) Conduct both a
between-subject and a within-subject simulation (Make sure that rho = 0.6 for the within
subject test). What is the percent
significant?
10) Change the N to 30 and conduct both a
between-subject and a within-subject simulation (Make sure that rho = 0.6 for the within subject test). What is the percent significant?
11) Change the N to 10 and conduct both a
between-subject and a within-subject simulation (Make sure that rho = 0.6 for the within subject test). What is the percent significant?
12) Return the N to 20. Now change the SD to 38 and conduct both a between-subject and a within-subject
simulation (Make sure that rho = 0.6 for
the within subject test). What is
the percent significant?
13) Now change the SD to 26 and conduct both a between-subject
and a within-subject simulation (Make
sure that rho = 0.6 for the within subject test). What is the percent significant?
14) If you decided to
change the value of rho for the within-subjects test,
what value would you expect to emerge for the percent significant under these
circumstances? Explain your reasoning.
Part G – Look at your findings
for Parts A through F to help you answer the following questions
15) If you only had
access to a limited number of participants, what could you do to increase the
likelihood of finding a significant difference if it really exists?
16) If you had a large
SD what could you do to increase the likelihood of finding a significant
difference if it really exists?
17) If you know that
the difference between means is very small, what could you do to increase the
likelihood of finding a significant effect?
Part H - use the ‘Simulate’ button not the
‘Simulate 5000’ button for the remaining simulations
18) Enter the following
numbers into the simulation table. Set the test for Within-subjects
|
N |
Population Mean A |
Population Mean B |
|
Population s.d. |
|
15 |
35 |
42 |
0.2 |
14 |
Now hit the ‘Simulate’ button
10 times. You will see a series of lines
appear on the grid below, as well as the t value appear. Record the t-value for each of the 10 simulations
in the second table on the Summary Tables sheet.
19) Now change the
value of rho to 0.95 and hit the ‘Simulate’ button 10 times. Record
the t-value for each of the 10 simulations in the second table on the Summary
Tables sheet.
20) In general, what
happens to the t-value as rho increases?
21) What does this tell
you about the nature of the relationship (i.e., what type of relationship
exists) between rho (r in the equation) and the value
of t?
22) Enter the following information into the
simulation table.
|
N |
Population Mean A |
Population Mean B |
|
Population s.d. |
|
25 |
30 |
35 |
0 |
20 |
Select the between-subjects
button. Hit the ‘Simulate’ button 8
times, recording the t-value, numerator and denominator values, the df and the critical t value each time. Place all of this information in the table
corresponding to this question (the
third table on the Summary Tables sheet).
Now enter the
following information into the simulation table. Select Within-Subjects
|
N |
Population Mean A |
Population Mean B |
|
Population s.d. |
|
25 |
30 |
35 |
0.7 |
20 |
Hit the ‘Simulate’ button 8 times,
recording the t-value, numerator and denominator, the df
and the critical t-value each time. Place all of this information in the
table corresponding to this question (the third table on the Summary tables sheet).
Now enter the
following information into the simulation table. Select Within-subjects
|
N |
Population Mean A |
Population Mean B |
|
Population s.d. |
|
25 |
30 |
35 |
0.9 |
20 |
Hit the ‘Simulate’ button 8 times,
recording the t-value, numerator and denominator, the df and the critical t-value each time. Place all of this information in the table corresponding to this
question (the third
table on the Summary tables sheet)
(i) Calculate the mean t-value for each of the 3 sets of simulations and provide the appropriate df and critical value for each type of test.
(ii) Calculate the mean denominator for each of the 3 sets of simulations. Please report these values.
(iii) What do you notice about the mean t-values for the three simulations?
(iv) What do you notice about the mean denominator values across the three simulations? Does this make sense given what you know about the denominator of the t-test? Explain your reasoning.
(v) Take a look at the df and the critical values for the between-subjects and within-subjects t-tests. What do you notice? What does this tell you about the relationship between df and critical values?