Week 6 Forum

In this week’s forum, we will study two statistical procedures: Measures of Association (MOA) and Test of Significance.

Please recall in Week 4, we explored crosstab and used epsilons and 10-percent-point rule to determine if a potential relationship between two variables is worth examining further. If we decide to move forward, MOA is the procedure that tell us the exact strength such relationship and direction of such relationship if both variables are at ordinal or interval/ratio level (both ordinal, both interval/ratio, or one ordinal and one ratio). If one variable is at nominal level of measurement, we don’t need to report direction of association. The strength of the relationship is interpreted using the logic of proportional reduction of error (PRE). Simply put: how does knowledge of IV help us reduce error in predicting the DV. More error reduction means stronger the relationship. Please refer to more details in the class handout available for download. Look for it toward the bottom of the page. 

The next step is test of significance, which tells us if the relationship (including strength) observed in the sample is generalizable to the population this sample represents. We’ll apply both procedures to our project variables.

Measures of association (MOA) (class handout available)

Tests for MOA are specific to the level of measurement of your variables. Here are the general guidelines:

1. Both DV and IV are nominal variables: Lambda (when it is not a 2X2 table)
   1. If it is a 2X2 table: Phi
2. Both DV and IV are ordinal variables: Gamma
3. One variable ordinal AND the other variable dichotomous nominal (like Yes/No, male/female, etc.): Gamma
   1. One variable ordinal AND the other variable nominal (not dichotomous, has more than 2 categories): Cramer’s V.
4. Both DV and IV are I/R variables: Pearson’s r

To interpret the output, see attached handout. Keep in mind measures of association is a statistical procedure based on the logic of Proportional Reduction of Error (PRE). Thus the format of interpretation will be: Knowing the IV will reduce error in predicting the DV by *%. 

Please note: Don’t just say “IV” and “DV” in your explanation. You need to enter your variables names for IV and DV, and replace * for the exact test value from the output. If the value of Lambda is .34, then it will be interpreted as 34%.

Test of Significance 

Again, the levels of measurement of our variables determine which test of significance works for the research project. Here is the guideline: 

1. Before-and-after design and the DV is at I/R level: Dependent Sample T-test

2. DV and IV are BOTH categorical variables (nominal/ordinal): Chi-square

*Special note for Chi-square: you should have less than 20% of the cells with an expected count of 5 or less. This information is reported automatically, right below the chi-square output table. If your chi-square test fails to meet this requirement, it is necessary to use “recoding” to combine certain answer categories together so the issues would be resolved.  

3. DV and IV are both continuous (interval/ratio) variables: regression 

*Special note: Continuous variable are always at interval/ratio level, but not vice versa. Double check if a interval/ratio variable is a continuous variable before regression. if IV is not continuous, we can create dummy variables based on the IV and include the dummy variable in the regression. DV, however, HAS to be a continuous variable no matter what.  

4. Comparison of group means (DV has to be continuous; grouping variable is categorical. For example, researchers want to study if exam score means are different among several school districts. In this research project, variable “exam score” is the DV and record exam scores in points; variable “School district” is the grouping variable, with each school district marked as 1, 2, 3, or 4, etc. in the data set).   

           a. Between 2 groups: Independent Sample T-test

           b. Among 3 or more groups: ANOVA

Why do we need to run tests of significance?

1. They allow us to see if our relationship is “statistically significant.” To be more specific, these tests tell us if a relationship observed in a sample, like your research project based on GSS 2016 data set, is generalizable to the population from which this sample was drawn.
2. Test results reported under “p” in the SPSS output tells us the chances that a relationship observed in the sample is not real, but rather due to factors like a sampling error. We compare this “chance” with level of significance, commonly set as .05 or .01. If this chance is smaller than level of significance, we can reject the null hypothesis, and keep the research hypothesis.

****Ok, now it is time for you to try! For this week’s forum, be sure to perform the test for measure of association (choose one) and test of significance (choose one) on your variables for the final project. 

This week in the forum discussion:

I. You will decide which measure of association you will use for your project. Use the guideline above to make your choice. Include the SPSS output in your post. Based on the output, explain the strength of association follow the logic of PRE and direction of the relationship (if your variables at ordinal or interval/ratio level).

II. You will decide which test of significance you will use for your project. Use the guideline above to make your choice.

II. You will use the process called “Five Steps of Hypothesis Testing” (see below) to complete test of significance. 

1. Write your research hypothesis (H1) and your null hypothesis (H0). Special note: Hypotheses should be written as assertive statements, not as questions end with question marks. H1 and H0 should be written as a pair, exactly the opposite of each other. Simply put: H1 always assumes there is a relationship; H0 always assumes there is no relationship. See templates below. Plug in your variable names for IV and DV. 
   1. template for H1: DV and IV are correlated; IV influences DV, DV is dependent on IV; etc. 
   2. template for H0: DV and IV are not correlated; IV has no influence/does not influence DV, DV is independent on IV; etc. 
2. Identify your level of significance (alpha): either .05 or .01.
3. Complete the significance test using SPSS. (Include the spss output in your post.)
4. Identify the number under Sig. (2-tail).  This will be represented by “p.” Compare this p-value with level of significance (.05 or .01) identified in step 2 and apply the following rule:
   1. If p < or = level of significance (alpha), reject the null hypothesis and the research hypothesis is true
   2. If p > alpha, than you fail to reject the null hypothesis
5. Conclusion: Be sure to go beyond the phrase “reject or fail to reject the null” and explain what that means to your research.