Assignment Prompt & Instructions
[>Operations Management homework help](https://www.sweetstudy.com/fields/business-finance-operations-management)
See attached
– a year ago
**files (2)**
– ## UnitVIPowerPointPresentation.docx
– ## UnitVIResearchDesignMethodsStudyGuide.pdf
**## UnitVIPowerPointPresentation.docx**
Unit VI PowerPoint Presentation
This PowerPoint Presentation measures your mastery of ULOs 1.1, 2.1, 2.3, 3.1, 4.1, and 4.3.
Imagine you, the student, are the professor for this course. Drawing from the information presented in Units I-VI, create an 8-slide PowerPoint presentation with speaker notes communicating the most important aspects of research methods. Be sure to cover the following points:
· Schematize philosophical assumptions, research worldviews, research methodologies, research traditions, and research designs.
· Diagram differences between a quantitative methodological research strategy and qualitative research strategy.
· Differentiate descriptive, explanatory, and exploratory research designs based on their unique characteristics.
· Compare and contrast descriptive statistics and inferential statistics. Explain the relationship of population, sampling frame, sampling design, and generalization.
· Illustrate the use of hypothesis testing in determining statistical significance.
Each slide must have speaker notes that further explains the information on the slide. Your presentation must be at least eight slides in length, not including cover slide and references, and you must use at least two academic resources, one of which must come from the CSU Online Library. Any resources used must be cited and referenced in APA format.
**## UnitVIResearchDesignMethodsStudyGuide.pdf**
RCH 5301, Research Design and Methods 1
Course Learning Outcomes for Unit VI
At the end of this unit, you should be able to:
1. Integrate philosophical assumptions and worldviews held by researchers.
1.1 Appraise philosophical assumptions, research worldviews, research methodologies,
research traditions, and research designs.
2. Discriminate among the characteristics of qualitative and quantitative research methods.
2.1 Deduce differences between a quantitative methodological research strategy and
qualitative research strategy.
2.3 Compare and contrast descriptive statistics and inferential statistics.
3. Evaluate research designs.
3.1 Differentiate between descriptive, causal, and explanatory research designs based on their
unique characteristics.
4. Apply research methods within a research design.
4.1 the relationship of population, sampling frame, sampling design, and generalization.
4.3 Illustrate the use of hypothesis testing for determining statistical significance.
Required Unit Resources
Chapter 7: Research Questions and Hypotheses (ULO 4.3)
Chapter 8: Quantitative Methods (ULO 4.3)
Read the following sections from this chapter:
• Interpreting Results and Writing a Discussion Section
• Components of an Experimental Study Method Plan
• Participants
• Variables
• Instrumentation and Materials
• Experimental Procedures
• Threats to Validity
• The Procedure
• Data Analysis
• Preregistering the Study Plan
• Interpreting Results and Writing a Discussion Section
Unit Lesson
Lesson: Using Statistical Significance to Evaluate Data Analysis Results (ULO 4.3)
Statistical Significance and Hypothesis Testing
Hypotheses are a best guess about the researcher’s phenomenon of interest, and they serve as a starting
point for testing the guess. The null hypothesis is the assumption that no significant relationship or difference
exists between the variables being tested. The null hypothesis is designated by Ho. The alternative hypothesis
is the assumption that a significant relationship or difference exists between variables being tested. The
UNIT VI STUDY GUIDE
Quantitative Data Analysis and
Inferential Statistics
RCH 5301, Research Design and Methods 2
UNIT x STUDY GUIDE
Title
alternative hypothesis is designated by Ha. The null and alternative hypotheses are mutually exclusive,
meaning that after statistical results are produced, one will be accepted and the other rejected.
It is useful to think of hypothesis testing as serving one of two purposes: a) testing for statistically significant
relationships between variables, or b) testing for statistically significant differences between group means.
Correlational research methods, such as correlation and regression, look for statistically significant
relationships between variables. Causal-comparative methods, such as t-test and ANOVA, look for
statistically significant differences between group means.
To prevent confusion when conducting hypothesis testing, it is advisable to state hypotheses in a specific and
standard format. Specificity relates to precisely defining the variables to be measured and tested.
Standardization pertains to using the same verbiage when stating the null and alternative hypotheses. In the
examples below, the highlighted verbiage is standard language that can be used for most hypotheses.
Relationship Between Variables
Example One – Statistical test looking for a relationship between variables.
Ho1 There is no statistically significant relationship between x and y.
Ha1 There is a statistically significant relationship between x and y.
Differences Between Groups
Example Two – Statistical test looking for a difference between group means.
Ho2: There is no statistically significant difference in means between Group A and Group B.
Ha2: There is a statistically significant difference in means between Group A and Group B.
Hypothesis Testing Steps
Much of the content in this course was created to help students understand the practical use of research
methods. Nowhere is this more obvious than the use of statistical analysis. It is more important to understand
how to interpret the statistics to aid decision-making than it is to be able to explain all the math behind the
results. That is the domain of academics and statisticians. The following steps will highlight the relative
simplicity of hypothesis testing.
Hypothesis testing follows a logical progression regardless of whether the researcher is testing for
relationships (correlation) or testing for differences (causal-comparative). According to Cooper and Schindler
(2014) that progression includes the following steps:
1. State the null and alternative hypotheses.
2. Choose a statistical procedure that will be used to test the hypotheses (e.g., correlation, regression, t-
test, ANOVA, etc.)
3. Select the desired alpha (α), or significance level, that represents the risk of a Type I error. A .05 α is
customarily used in research.
4. Use the chosen statistical procedure to test the data, which produces a p value. This is accomplished
using a statistical application.
5. Interpret the test results. If the p value < .05 α, reject the null hypothesis and accept the alternative
hypothesis. If the p value ≥ .05 α, accept the null hypothesis and reject the alternative hypothesis.
Statistical Significance and Importance
Are significant results important? Are non-significant results useless? Many results can be statistically
significant but offer little practical importance. Bell et al. (2022) provide a clear example of how statistically
significant results can have no practical significance. As they explained, a significant r of .10 equating to R
squared of 1% is essentially useless for decision-making.
RCH 5301, Research Design and Methods 3
UNIT x STUDY GUIDE
Title
Conversely, non-significant results can offer the practitioner valuable insights about the domain of interest.
Assume a global company’s safety director wants to assess the effectiveness of a new safety training
program. Two different groups of employees; those having taken the training and those who have not, are
being tested for significant differences in recordable work-related injuries. The hypotheses are:
• Ho: There is no statistically significant difference in recordable work-related injuries between Group A
and Group B.
• Ha: There is a statistically significant difference in recordable work-related injuries between Group A
and Group B.
After an independent samples t-test is conducted, the results indicate a p value 17 > .05 α, therefore, the
result is non-significant, and the null hypothesis is accepted (and the alternative hypothesis is rejected) that
there is no statistically significant difference in recordable work-related injuries between Group A and Group
B. Although the conclusion to be drawn from the results of the hypothesis test is that the training is not
effective, this finding has immense practical value. The company may save thousands, hundreds of
thousands, or millions of dollars by not implementing the new, but ineffective, safety training program. So,
even though the results are not statistically significant, they offer great practical significance by providing
invaluable insights for decision-making.
Statistical Significance and Hypothesis Testing Summary
• Hypotheses either test for relationships between variables or differences between group means.
• Hypotheses should be specific and use standardized verbiage.
• Hypothesis testing follows a logical progression of steps.
• Significant results do not necessarily mean they offer practical importance.
• Non-significant results do not necessarily mean they lack practical importance.
• Significant results do not provide definitive proof the alternative hypothesis is true since a Type I error
is always possible.
• Non-significant results do not provide definitive proof the null hypothesis is true since a Type II error is
always possible.
The Lady Tasting Tea Experiment and the Origin of .05 α
As mentioned previously, an alpha of .05 is customarily used in research as the cut-off for statistical
significance, and many other applications of hypothesis testing. There is an interesting story of the origin of
the .05 α. It goes like this.
Once upon a time in Cambridge, England, on a lovely summer day during the 1920s, a group of academics
and their friends were enjoying afternoon tea. One of the women present made a claim to the astonished
group that tea tasted differently depending on whether the tea was poured first, or the milk was poured first,
and that she could blindly differentiate cups by taste alone. To say the academics were skeptical would be an
understatement. One of the academics in attendance, Dr. Ronald Fisher, decided to give her the benefit of
the doubt by constructing an ad-hoc, blind taste test, an experiment to test the hypothesis that the lady could
differentiate cups by taste (Salsburg, 2001).
The hypotheses were:
• Ho: There is no statistically significant difference in the lady’s ability to distinguish cups by taste vs.
random selection.
• Ha: There is a statistically significant difference in the lady’s ability to distinguish cups by taste vs.
random selection.
For the purposes of revealing the nature of statistical significance and its relation to probability for hypothesis
testing, assume the following three hypothesis tests were administered to the lady.
Test 1
In the first test, two choices were available: a) cup 1, tea poured first, and b) cup 2, milk poured first. If the
lady was able to identify either cup 1 or cup 2 by taste alone, nobody would be impressed since she had a 1
RCH 5301, Research Design and Methods 4
UNIT x STUDY GUIDE
Title
out of 2 chance of success, or 50% probability (see graphic), of making a correct selection by simply
guessing. Based on the results of this test, Ha would have been accepted that the lady could distinguish cups;
however, this test would have had a very low threshold for statistical significance and a very high chance for a
Type I error with what is effectively a .50 α.
Test 2
A second test involved four cups: a) cup 1, tea poured first, b) cup 2, tea poured first, c) cup 3, milk poured
first, and d) cup 4 milk poured first. A combination calculator can be used to determine that these four cups
result in 6 2-cup combinations from which the lady could have chosen.
Combination 1 = Cup 1, Cup 2
Combination 2 = Cup 3, Cup 4
Combination 3 = Cup 1, Cup 3
Combination 4 = Cup 1, Cup 4
Combination 5 = Cup 2, Cup 3
Combination 6 = Cup 2, Cup 4
If the lady had successfully identified the 2 cups with tea added first (Combination 1) or the 2 cups with milk
added first (Combination 2), it might have generated some applause, but it would be unlikely most observers
would have been overly impressed since she had a 1 out of 6 chance, or 16.7% probability (see graphic), of
selecting the correct combination by simply guessing. Once again, based on the results of this test, Ha would
have been accepted that the lady could distinguish cups based on taste alone. This test would have had a
moderate threshold for statistical significance and moderate chance of a Type I error with what is effectively
a .167α.
Test 3
This final test scenario uses 6 cups, with 3 cups having tea added first, and 3 cups having milk added first.
Again, using a combination calculator, these 6 cups result in 20 3-cup combinations from which the lady could
have chosen. If the lady had successfully identified the 3 cups with tea added first, or the 3 cups with milk
added first, that would have certainly elicited hoots and hollers. Observers would truly be impressed since she
had only a 1 out of 20 chance, or 5% probability, of selecting the correct permutation by simply guessing.
Based on the results of this test, Ha would have been accepted that the lady could distinguish cups based on
taste alone. This is a high threshold for statistical significance and a very low chance of a Type I error with
what is effectively a .05 α.
(Sustainability Methods, 2021)
Several years after the tea-tasting experiment, in 1929, Fisher wrote The Statistical Method in Psychical
Research in which he recommended p value < .05 α as an arbitrary, but convenient, cut-off for evaluating
statistical significance. He described a statistically significant event as one that would rarely be observed in
the absence of an independent explanatory variable(s). Rarity, from Fisher’s perspective, would be an event
RCH 5301, Research Design and Methods 5
UNIT x STUDY GUIDE
Title
that occurs randomly not more often than 1 in 20 tries, which translates into rejecting the null hypothesis and
accepting the alternative hypothesis when a statistical test returns a p value < .05 α. Legend holds that the
lady tasting tea was indeed able to successfully select three like cups. The question remains as to whether
this was one of those rare statistically significant events that was observed in the absence of an explanatory
variable, or if the test measured a true effect (Fisher, 1929; Lewis & Sauro, 2021).
Fisher was a mathematical genius who dedicated his life to science and math and was famous for his
contribution to the field of statistics and the many statistical procedures he created. Although Fisher’s writings
have been pored over, there is no indication if the Lady Tasting Tea experiment was the creative spark for
codifying .05 α as the arbitrary and convenient threshold for statistical significance, or if the Lady Tasting Tea
experiment was simply an opportunity to promote Fisher’s preconceived desire to see the scientific
community begin to adopt .05 α as the threshold for statistical significance. Regardless, .05 α has become the
standard, and the Lady Tasting Tea experiment has become legendary in statistical history.
Additional Inferential Tests of Differences
Sometimes, a researcher will find that One-Way ANOVA is limited in the number and types of variables that
can be included in the analysis. There are, fortunately, additional causal-comparative procedures that provide
flexibility to meet the needs of any combination or quantity of variables. The following will compare these
additional procedures with One-Way ANOVA. They include Two-Way ANOVA, ANCOVA, One-Way
MANOVA, Two-Way MANOVA, and MANCOVA.
Terminology
Terminology is once again important to understand the content in this section. The following will describe the
three main variable types used in the additional causal-comparative tests to be discussed below. These
variables are named response variable, factor, and covariate.
Response Variable
When using ANOVA, ANCOVA, One-Way MANOVA, Two-Way MANOVA, and MANCOVA, the dependent
variable (DV) is referred to as a response variable. A response variable is a continuous variable, which means
it is measured on at least an interval or ratio scale level. Examples of response variables include income,
time, height, length, depth, weight, growth, temperature, moisture, humidity, speed, distance, frequency,
antibodies, grades, etc.
Factor
A factor is an independent variable (IV) that is sometimes referred to as the grouping variable. The factor is a
categorical variable, which means it is measured on a nominal scale and specifies group membership.
Examples of factors include socio-economic status, race, ethnicity, religion, education level, industry,
business type, department, work shift, work team, batch, lot, production line, athletic team, gate, turnstile,
country, state, county, city, neighborhood, society, culture, treatment, eye color, hair color, etc.
Covariate
A covariate is a continuous independent variable, which means it is measured on at least an interval or ratio
scale level.
One-Way ANOVA (Analysis of Variance)
Like the t-test, ANOVA is a powerful causal-comparative inferential statistical procedure for analyzing
differences between means; however, it has an advantage over the bivariate t-test in that it can test for
differences in more than two means. For example, a researcher may be interested in determining if
statistically significant differences exist between average income (response variable) depending on degree
held (IV factor). More specifically, average income could be compared among the holders of the following
degrees: high school diploma, associate’s degree, bachelor’s degree, master’s degree, and terminal degree.
In this example, five income means would be compared for differences.
RCH 5301, Research Design and Methods 6
UNIT x STUDY GUIDE
Title
This example used one response variable and one IV factor, which is the reason for the name One-Way
ANOVA. Now, suppose the researcher is interested in analyzing the mean differences for a single response
variable, but for two IV factors. Adding two or more factors requires the use of Two-Way ANOVA.
Two-Way ANOVA (Analysis of Variance)
Two-Way ANOVA is used when a researcher is interested in determining if statistically significant differences
exist in means for a single response variable, but for two factors (Field, 2005). Expanding on the above
example, the researcher could compare average income (response variable) depending on degree held
(Factor1) and region (Factor2). At a granular level, the Two-Way ANOVA would compare average income for
a) the holders of the following degrees: high school diploma, associate’s degree, bachelor’s degree, master’s
degree, and terminal degree, and b) region, including Northeast, Southeast, Mid-Atlantic, Midwest,
Southwest, Northwest, and West.
It should be noted that there can be more than two factors used in this procedure. Whenever more than two
factors are used in ANOVA, with one response variable, it is referred to as factorial ANOVA.
ANCOVA (Analysis of Covariance)
ANCOVA is similar to ANOVA in that there is only a single response variable, which is continuous. However,
ANCOVA uses two IVs: a) one that is a categorical factor, and b) one that is a continuous variable. Therefore,
the C in ANCOVA represents the test of covariance between the response variable and the continuous
variable. The continuous IV is called a covariate (Field, 2005).
In the original example, a researcher was interested in determining if statistically significant differences exist
between average income (response variable) depending on degree held (Factor1), such as high school
diploma, associate degree, bachelor’s degree, master’s degree, and terminal degree. Using ANCOVA, the
researcher now includes the covariate GPA (Covariate1).
RCH 5301, Research Design and Methods 7
UNIT x STUDY GUIDE
Title
One-Way MANOVA (Multivariate Analysis of Variance)
Until this point, only one DV continuous response variable was used in the ANOVA and ANCOVA examples.
This changes with MANOVA and the addition of the M, which represents multivariate. One-way MANOVA
involves two continuous response variables and one categorical factor (Field, 2005). For example, a
researcher is interested in determining if statistically significant differences exist between: a) average income
(Response1), and b) average hours worked each week (Response2) depending on degree held (Factor1),
such as high school diploma, associate’s degree, bachelor’s degree, master’s degree, and terminal degree.
Two-Way MANOVA (Multivariate Analysis of Variance)
Like ANOVA, MANOVA can take a two-way approach, which includes two continuous response variables and
two IV categorical factors (Field, 2005). For example, the researcher in this example is interested in
determining if statistically significant differences exist between: a) average income (Response1), and b)
average hours worked each week (Response2) depending on: c) degree (Factor1), such as high school
diploma, and AA Degree, Bachelor’s Degree, Master’s Degree, and Terminal Degree, and d) region (Factor2),
including Northeast, Southeast, Mid-Atlantic, Midwest, Southwest, Northwest, and West.
It should be noted that there can be more than two categorical factors used in this procedure, and more than
two continuous response variables. Whenever more than two factors or more than two response variables are
used in MANOVA, it is referred to as factorial MANOVA.
RCH 5301, Research Design and Methods 8
UNIT x STUDY GUIDE
Title
MANCOVA (Multivariate Analysis of Covariance)
MANCOVA is similar to MANOVA with the addition of the C, representing covariance. MANCOVA and
MANOVA both include two or more continuous response variables, but MANCOVA includes at least one
categorical factor and at least one continuous variable, which is the covariate (Field, 2005). In this example of
MANCOVA, the researcher is interested in determining if statistically significant differences exist between: a)
average income (Response1), and b) average hours worked each week (Response2) depending on: c) degree
(Factor1 categorical variable) such as high school diploma, associate’s degree, bachelor’s degree, master’s
degree, and terminal degree, and d) hours of sleep each week (Covariate1 continuous variable).
Conclusion
Many students will never apply the statistical tests discussed in this course, but most will be exposed to
information that includes the results of these types of procedures, like regression and ANOVA, throughout
their careers. That is why it is important to have a good foundation for interpreting basic statistical results.
Statistical results are also often published in daily news reports on a variety of topics, so knowledge of
interpretation is helpful even outside of one’s interests. While it is not necessary for most students to
memorize the purpose and particulars of procedures like ANCOVA, MANOVA, and MANCOVA, it is important
that students are aware of the robustness of causal-comparative tests. If a need arises in this area, a good
research website or textbook can help anyone determine the appropriate test for their needs.
RCH 5301, Research Design and Methods 9
UNIT x STUDY GUIDE
Title
References
Bell, E., Bryman, A., & Harley, B. (2022). Business research methods (6th ed.). Oxford University Press.
https://online.vitalsource.com/#/books/9780192640505
Cooper, D. R., & Schindler, P. S. (2014). Business research methods (12th ed.). McGraw-Hill.
Creswell, J. W., & Creswell, J. D. (2022). Research design: Qualitative, quantitative, and mixed methods
approaches (6th ed.). SAGE. https://online.vitalsource.com/#/books/9781071817964
Field, A. (2005). Discovering stats using SPSS (2nd ed.). SAGE.
Fisher, R. A. (1929). The statistical method in psychical research. Proceedings of the Society for Psychical
Research, 39, 189–192 (1929).
Lewis, J., & Sauro, J. (2021). For statistical significance, must p be < .05? Measuring U.
https://measuringu.com/setting-alpha/
Salsburg, D. (2001). The lady tasting tea: How statistics revolutionized science in the twentieth century. Henry
Holt and Company.
Sustainability Methods. (2021, September 20). Designing studies [Image].
https://sustainabilitymethods.org/index.php/Designing_studies
– Course Learning Outcomes for Unit VI
– Required Unit Resources
– Chapter 7: Research Questions and Hypotheses (ULO 4.3)
– Chapter 8: Quantitative Methods (ULO 4.3)
– Unit Lesson
– Lesson: Using Statistical Significance to Evaluate Data Analysis Results (ULO 4.3)
– Statistical Significance and Hypothesis Testing
– Hypothesis Testing Steps
– Statistical Significance and Importance
– Statistical Significance and Hypothesis Testing Summary
– The Lady Tasting Tea Experiment and the Origin of .05 α
– Additional Inferential Tests of Differences
– Terminology
– Response Variable
– Factor
– Covariate
– One-Way ANOVA (Analysis of Variance)
– Two-Way ANOVA (Analysis of Variance)
– ANCOVA (Analysis of Covariance)
– One-Way MANOVA (Multivariate Analysis of Variance)
– Two-Way MANOVA (Multivariate Analysis of Variance)
– MANCOVA (Multivariate Analysis of Covariance)
– Conclusion
– References
**## UnitVIResearchDesignMethodsStudyGuide.pdf**
RCH 5301, Research Design and Methods 1
Course Learning Outcomes for Unit VI
At the end of this unit, you should be able to:
1. Integrate philosophical assumptions and worldviews held by researchers.
1.1 Appraise philosophical assumptions, research worldviews, research methodologies,
research traditions, and research designs.
2. Discriminate among the characteristics of qualitative and quantitative research methods.
2.1 Deduce differences between a quantitative methodological research strategy and
qualitative research strategy.
2.3 Compare and contrast descriptive statistics and inferential statistics.
3. Evaluate research designs.
3.1 Differentiate between descriptive, causal, and explanatory research designs based on their
unique characteristics.
4. Apply research methods within a research design.
4.1 the relationship of population, sampling frame, sampling design, and generalization.
4.3 Illustrate the use of hypothesis testing for determining statistical significance.
Required Unit Resources
Chapter 7: Research Questions and Hypotheses (ULO 4.3)
Chapter 8: Quantitative Methods (ULO 4.3)
Read the following sections from this chapter:
• Interpreting Results and Writing a Discussion Section
• Components of an Experimental Study Method Plan
• Participants
• Variables
• Instrumentation and Materials
• Experimental Procedures
• Threats to Validity
• The Procedure
• Data Analysis
• Preregistering the Study Plan
• Interpreting Results and Writing a Discussion Section
Unit Lesson
Lesson: Using Statistical Significance to Evaluate Data Analysis Results (ULO 4.3)
Statistical Significance and Hypothesis Testing
Hypotheses are a best guess about the researcher’s phenomenon of interest, and they serve as a starting
point for testing the guess. The null hypothesis is the assumption that no significant relationship or difference
exists between the variables being tested. The null hypothesis is designated by Ho. The alternative hypothesis
is the assumption that a significant relationship or difference exists between variables being tested. The
UNIT VI STUDY GUIDE
Quantitative Data Analysis and
Inferential Statistics
RCH 5301, Research Design and Methods 2
UNIT x STUDY GUIDE
Title
alternative hypothesis is designated by Ha. The null and alternative hypotheses are mutually exclusive,
meaning that after statistical results are produced, one will be accepted and the other rejected.
It is useful to think of hypothesis testing as serving one of two purposes: a) testing for statistically significant
relationships between variables, or b) testing for statistically significant differences between group means.
Correlational research methods, such as correlation and regression, look for statistically significant
relationships between variables. Causal-comparative methods, such as t-test and ANOVA, look for
statistically significant differences between group means.
To prevent confusion when conducting hypothesis testing, it is advisable to state hypotheses in a specific and
standard format. Specificity relates to precisely defining the variables to be measured and tested.
Standardization pertains to using the same verbiage when stating the null and alternative hypotheses. In the
examples below, the highlighted verbiage is standard language that can be used for most hypotheses.
Relationship Between Variables
Example One – Statistical test looking for a relationship between variables.
Ho1 There is no statistically significant relationship between x and y.
Ha1 There is a statistically significant relationship between x and y.
Differences Between Groups
Example Two – Statistical test looking for a difference between group means.
Ho2: There is no statistically significant difference in means between Group A and Group B.
Ha2: There is a statistically significant difference in means between Group A and Group B.
Hypothesis Testing Steps
Much of the content in this course was created to help students understand the practical use of research
methods. Nowhere is this more obvious than the use of statistical analysis. It is more important to understand
how to interpret the statistics to aid decision-making than it is to be able to explain all the math behind the
results. That is the domain of academics and statisticians. The following steps will highlight the relative
simplicity of hypothesis testing.
Hypothesis testing follows a logical progression regardless of whether the researcher is testing for
relationships (correlation) or testing for differences (causal-comparative). According to Cooper and Schindler
(2014) that progression includes the following steps:
1. State the null and alternative hypotheses.
2. Choose a statistical procedure that will be used to test the hypotheses (e.g., correlation, regression, t-
test, ANOVA, etc.)
3. Select the desired alpha (α), or significance level, that represents the risk of a Type I error. A .05 α is
customarily used in research.
4. Use the chosen statistical procedure to test the data, which produces a p value. This is accomplished
using a statistical application.
5. Interpret the test results. If the p value < .05 α, reject the null hypothesis and accept the alternative
hypothesis. If the p value ≥ .05 α, accept the null hypothesis and reject the alternative hypothesis.
Statistical Significance and Importance
Are significant results important? Are non-significant results useless? Many results can be statistically
significant but offer little practical importance. Bell et al. (2022) provide a clear example of how statistically
significant results can have no practical significance. As they explained, a significant r of .10 equating to R
squared of 1% is essentially useless for decision-making.
RCH 5301, Research Design and Methods 3
UNIT x STUDY GUIDE
Title
Conversely, non-significant results can offer the practitioner valuable insights about the domain of interest.
Assume a global company’s safety director wants to assess the effectiveness of a new safety training
program. Two different groups of employees; those having taken the training and those who have not, are
being tested for significant differences in recordable work-related injuries. The hypotheses are:
• Ho: There is no statistically significant difference in recordable work-related injuries between Group A
and Group B.
• Ha: There is a statistically significant difference in recordable work-related injuries between Group A
and Group B.
After an independent samples t-test is conducted, the results indicate a p value 17 > .05 α, therefore, the
result is non-significant, and the null hypothesis is accepted (and the alternative hypothesis is rejected) that
there is no statistically significant difference in recordable work-related injuries between Group A and Group
B. Although the conclusion to be drawn from the results of the hypothesis test is that the training is not
effective, this finding has immense practical value. The company may save thousands, hundreds of
thousands, or millions of dollars by not implementing the new, but ineffective, safety training program. So,
even though the results are not statistically significant, they offer great practical significance by providing
invaluable insights for decision-making.
Statistical Significance and Hypothesis Testing Summary
• Hypotheses either test for relationships between variables or differences between group means.
• Hypotheses should be specific and use standardized verbiage.
• Hypothesis testing follows a logical progression of steps.
• Significant results do not necessarily mean they offer practical importance.
• Non-significant results do not necessarily mean they lack practical importance.
• Significant results do not provide definitive proof the alternative hypothesis is true since a Type I error
is always possible.
• Non-significant results do not provide definitive proof the null hypothesis is true since a Type II error is
always possible.
The Lady Tasting Tea Experiment and the Origin of .05 α
As mentioned previously, an alpha of .05 is customarily used in research as the cut-off for statistical
significance, and many other applications of hypothesis testing. There is an interesting story of the origin of
the .05 α. It goes like this.
Once upon a time in Cambridge, England, on a lovely summer day during the 1920s, a group of academics
and their friends were enjoying afternoon tea. One of the women present made a claim to the astonished
group that tea tasted differently depending on whether the tea was poured first, or the milk was poured first,
and that she could blindly differentiate cups by taste alone. To say the academics were skeptical would be an
understatement. One of the academics in attendance, Dr. Ronald Fisher, decided to give her the benefit of
the doubt by constructing an ad-hoc, blind taste test, an experiment to test the hypothesis that the lady could
differentiate cups by taste (Salsburg, 2001).
The hypotheses were:
• Ho: There is no statistically significant difference in the lady’s ability to distinguish cups by taste vs.
random selection.
• Ha: There is a statistically significant difference in the lady’s ability to distinguish cups by taste vs.
random selection.
For the purposes of revealing the nature of statistical significance and its relation to probability for hypothesis
testing, assume the following three hypothesis tests were administered to the lady.
Test 1
In the first test, two choices were available: a) cup 1, tea poured first, and b) cup 2, milk poured first. If the
lady was able to identify either cup 1 or cup 2 by taste alone, nobody would be impressed since she had a 1
RCH 5301, Research Design and Methods 4
UNIT x STUDY GUIDE
Title
out of 2 chance of success, or 50% probability (see graphic), of making a correct selection by simply
guessing. Based on the results of this test, Ha would have been accepted that the lady could distinguish cups;
however, this test would have had a very low threshold for statistical significance and a very high chance for a
Type I error with what is effectively a .50 α.
Test 2
A second test involved four cups: a) cup 1, tea poured first, b) cup 2, tea poured first, c) cup 3, milk poured
first, and d) cup 4 milk poured first. A combination calculator can be used to determine that these four cups
result in 6 2-cup combinations from which the lady could have chosen.
Combination 1 = Cup 1, Cup 2
Combination 2 = Cup 3, Cup 4
Combination 3 = Cup 1, Cup 3
Combination 4 = Cup 1, Cup 4
Combination 5 = Cup 2, Cup 3
Combination 6 = Cup 2, Cup 4
If the lady had successfully identified the 2 cups with tea added first (Combination 1) or the 2 cups with milk
added first (Combination 2), it might have generated some applause, but it would be unlikely most observers
would have been overly impressed since she had a 1 out of 6 chance, or 16.7% probability (see graphic), of
selecting the correct combination by simply guessing. Once again, based on the results of this test, Ha would
have been accepted that the lady could distinguish cups based on taste alone. This test would have had a
moderate threshold for statistical significance and moderate chance of a Type I error with what is effectively
a .167α.
Test 3
This final test scenario uses 6 cups, with 3 cups having tea added first, and 3 cups having milk added first.
Again, using a combination calculator, these 6 cups result in 20 3-cup combinations from which the lady could
have chosen. If the lady had successfully identified the 3 cups with tea added first, or the 3 cups with milk
added first, that would have certainly elicited hoots and hollers. Observers would truly be impressed since she
had only a 1 out of 20 chance, or 5% probability, of selecting the correct permutation by simply guessing.
Based on the results of this test, Ha would have been accepted that the lady could distinguish cups based on
taste alone. This is a high threshold for statistical significance and a very low chance of a Type I error with
what is effectively a .05 α.
(Sustainability Methods, 2021)
Several years after the tea-tasting experiment, in 1929, Fisher wrote The Statistical Method in Psychical
Research in which he recommended p value < .05 α as an arbitrary, but convenient, cut-off for evaluating
statistical significance. He described a statistically significant event as one that would rarely be observed in
the absence of an independent explanatory variable(s). Rarity, from Fisher’s perspective, would be an event
RCH 5301, Research Design and Methods 5
UNIT x STUDY GUIDE
Title
that occurs randomly not more often than 1 in 20 tries, which translates into rejecting the null hypothesis and
accepting the alternative hypothesis when a statistical test returns a p value < .05 α. Legend holds that the
lady tasting tea was indeed able to successfully select three like cups. The question remains as to whether
this was one of those rare statistically significant events that was observed in the absence of an explanatory
variable, or if the test measured a true effect (Fisher, 1929; Lewis & Sauro, 2021).
Fisher was a mathematical genius who dedicated his life to science and math and was famous for his
contribution to the field of statistics and the many statistical procedures he created. Although Fisher’s writings
have been pored over, there is no indication if the Lady Tasting Tea experiment was the creative spark for
codifying .05 α as the arbitrary and convenient threshold for statistical significance, or if the Lady Tasting Tea
experiment was simply an opportunity to promote Fisher’s preconceived desire to see the scientific
community begin to adopt .05 α as the threshold for statistical significance. Regardless, .05 α has become the
standard, and the Lady Tasting Tea experiment has become legendary in statistical history.
Additional Inferential Tests of Differences
Sometimes, a researcher will find that One-Way ANOVA is limited in the number and types of variables that
can be included in the analysis. There are, fortunately, additional causal-comparative procedures that provide
flexibility to meet the needs of any combination or quantity of variables. The following will compare these
additional procedures with One-Way ANOVA. They include Two-Way ANOVA, ANCOVA, One-Way
MANOVA, Two-Way MANOVA, and MANCOVA.
Terminology
Terminology is once again important to understand the content in this section. The following will describe the
three main variable types used in the additional causal-comparative tests to be discussed below. These
variables are named response variable, factor, and covariate.
Response Variable
When using ANOVA, ANCOVA, One-Way MANOVA, Two-Way MANOVA, and MANCOVA, the dependent
variable (DV) is referred to as a response variable. A response variable is a continuous variable, which means
it is measured on at least an interval or ratio scale level. Examples of response variables include income,
time, height, length, depth, weight, growth, temperature, moisture, humidity, speed, distance, frequency,
antibodies, grades, etc.
Factor
A factor is an independent variable (IV) that is sometimes referred to as the grouping variable. The factor is a
categorical variable, which means it is measured on a nominal scale and specifies group membership.
Examples of factors include socio-economic status, race, ethnicity, religion, education level, industry,
business type, department, work shift, work team, batch, lot, production line, athletic team, gate, turnstile,
country, state, county, city, neighborhood, society, culture, treatment, eye color, hair color, etc.
Covariate
A covariate is a continuous independent variable, which means it is measured on at least an interval or ratio
scale level.
One-Way ANOVA (Analysis of Variance)
Like the t-test, ANOVA is a powerful causal-comparative inferential statistical procedure for analyzing
differences between means; however, it has an advantage over the bivariate t-test in that it can test for
differences in more than two means. For example, a researcher may be interested in determining if
statistically significant differences exist between average income (response variable) depending on degree
held (IV factor). More specifically, average income could be compared among the holders of the following
degrees: high school diploma, associate’s degree, bachelor’s degree, master’s degree, and terminal degree.
In this example, five income means would be compared for differences.
RCH 5301, Research Design and Methods 6
UNIT x STUDY GUIDE
Title
This example used one response variable and one IV factor, which is the reason for the name One-Way
ANOVA. Now, suppose the researcher is interested in analyzing the mean differences for a single response
variable, but for two IV factors. Adding two or more factors requires the use of Two-Way ANOVA.
Two-Way ANOVA (Analysis of Variance)
Two-Way ANOVA is used when a researcher is interested in determining if statistically significant differences
exist in means for a single response variable, but for two factors (Field, 2005). Expanding on the above
example, the researcher could compare average income (response variable) depending on degree held
(Factor1) and region (Factor2). At a granular level, the Two-Way ANOVA would compare average income for
a) the holders of the following degrees: high school diploma, associate’s degree, bachelor’s degree, master’s
degree, and terminal degree, and b) region, including Northeast, Southeast, Mid-Atlantic, Midwest,
Southwest, Northwest, and West.
It should be noted that there can be more than two factors used in this procedure. Whenever more than two
factors are used in ANOVA, with one response variable, it is referred to as factorial ANOVA.
ANCOVA (Analysis of Covariance)
ANCOVA is similar to ANOVA in that there is only a single response variable, which is continuous. However,
ANCOVA uses two IVs: a) one that is a categorical factor, and b) one that is a continuous variable. Therefore,
the C in ANCOVA represents the test of covariance between the response variable and the continuous
variable. The continuous IV is called a covariate (Field, 2005).
In the original example, a researcher was interested in determining if statistically significant differences exist
between average income (response variable) depending on degree held (Factor1), such as high school
diploma, associate degree, bachelor’s degree, master’s degree, and terminal degree. Using ANCOVA, the
researcher now includes the covariate GPA (Covariate1).
RCH 5301, Research Design and Methods 7
UNIT x STUDY GUIDE
Title
One-Way MANOVA (Multivariate Analysis of Variance)
Until this point, only one DV continuous response variable was used in the ANOVA and ANCOVA examples.
This changes with MANOVA and the addition of the M, which represents multivariate. One-way MANOVA
involves two continuous response variables and one categorical factor (Field, 2005). For example, a
researcher is interested in determining if statistically significant differences exist between: a) average income
(Response1), and b) average hours worked each week (Response2) depending on degree held (Factor1),
such as high school diploma, associate’s degree, bachelor’s degree, master’s degree, and terminal degree.
Two-Way MANOVA (Multivariate Analysis of Variance)
Like ANOVA, MANOVA can take a two-way approach, which includes two continuous response variables and
two IV categorical factors (Field, 2005). For example, the researcher in this example is interested in
determining if statistically significant differences exist between: a) average income (Response1), and b)
average hours worked each week (Response2) depending on: c) degree (Factor1), such as high school
diploma, and AA Degree, Bachelor’s Degree, Master’s Degree, and Terminal Degree, and d) region (Factor2),
including Northeast, Southeast, Mid-Atlantic, Midwest, Southwest, Northwest, and West.
It should be noted that there can be more than two categorical factors used in this procedure, and more than
two continuous response variables. Whenever more than two factors or more than two response variables are
used in MANOVA, it is referred to as factorial MANOVA.
RCH 5301, Research Design and Methods 8
UNIT x STUDY GUIDE
Title
MANCOVA (Multivariate Analysis of Covariance)
MANCOVA is similar to MANOVA with the addition of the C, representing covariance. MANCOVA and
MANOVA both include two or more continuous response variables, but MANCOVA includes at least one
categorical factor and at least one continuous variable, which is the covariate (Field, 2005). In this example of
MANCOVA, the researcher is interested in determining if statistically significant differences exist between: a)
average income (Response1), and b) average hours worked each week (Response2) depending on: c) degree
(Factor1 categorical variable) such as high school diploma, associate’s degree, bachelor’s degree, master’s
degree, and terminal degree, and d) hours of sleep each week (Covariate1 continuous variable).
Conclusion
Many students will never apply the statistical tests discussed in this course, but most will be exposed to
information that includes the results of these types of procedures, like regression and ANOVA, throughout
their careers. That is why it is important to have a good foundation for interpreting basic statistical results.
Statistical results are also often published in daily news reports on a variety of topics, so knowledge of
interpretation is helpful even outside of one’s interests. While it is not necessary for most students to
memorize the purpose and particulars of procedures like ANCOVA, MANOVA, and MANCOVA, it is important
that students are aware of the robustness of causal-comparative tests. If a need arises in this area, a good
research website or textbook can help anyone determine the appropriate test for their needs.
RCH 5301, Research Design and Methods 9
UNIT x STUDY GUIDE
Title
References
Bell, E., Bryman, A., & Harley, B. (2022). Business research methods (6th ed.). Oxford University Press.
https://online.vitalsource.com/#/books/9780192640505
Cooper, D. R., & Schindler, P. S. (2014). Business research methods (12th ed.). McGraw-Hill.
Creswell, J. W., & Creswell, J. D. (2022). Research design: Qualitative, quantitative, and mixed methods
approaches (6th ed.). SAGE. https://online.vitalsource.com/#/books/9781071817964
Field, A. (2005). Discovering stats using SPSS (2nd ed.). SAGE.
Fisher, R. A. (1929). The statistical method in psychical research. Proceedings of the Society for Psychical
Research, 39, 189–192 (1929).
Lewis, J., & Sauro, J. (2021). For statistical significance, must p be < .05? Measuring U.
https://measuringu.com/setting-alpha/
Salsburg, D. (2001). The lady tasting tea: How statistics revolutionized science in the twentieth century. Henry
Holt and Company.
Sustainability Methods. (2021, September 20). Designing studies [Image].
https://sustainabilitymethods.org/index.php/Designing_studies
– Course Learning Outcomes for Unit VI
– Required Unit Resources
– Chapter 7: Research Questions and Hypotheses (ULO 4.3)
– Chapter 8: Quantitative Methods (ULO 4.3)
– Unit Lesson
– Lesson: Using Statistical Significance to Evaluate Data Analysis Results (ULO 4.3)
– Statistical Significance and Hypothesis Testing
– Hypothesis Testing Steps
– Statistical Significance and Importance
– Statistical Significance and Hypothesis Testing Summary
– The Lady Tasting Tea Experiment and the Origin of .05 α
– Additional Inferential Tests of Differences
– Terminology
– Response Variable
– Factor
– Covariate
– One-Way ANOVA (Analysis of Variance)
– Two-Way ANOVA (Analysis of Variance)
– ANCOVA (Analysis of Covariance)
– One-Way MANOVA (Multivariate Analysis of Variance)
– Two-Way MANOVA (Multivariate Analysis of Variance)
– MANCOVA (Multivariate Analysis of Covariance)
– Conclusion
– References
**## UnitVIPowerPointPresentation.docx**
Unit VI PowerPoint Presentation
This PowerPoint Presentation measures your mastery of ULOs 1.1, 2.1, 2.3, 3.1, 4.1, and 4.3.
Imagine you, the student, are the professor for this course. Drawing from the information presented in Units I-VI, create an 8-slide PowerPoint presentation with speaker notes communicating the most important aspects of research methods. Be sure to cover the following points:
· Schematize philosophical assumptions, research worldviews, research methodologies, research traditions, and research designs.
· Diagram differences between a quantitative methodological research strategy and qualitative research strategy.
· Differentiate descriptive, explanatory, and exploratory research designs based on their unique characteristics.
· Compare and contrast descriptive statistics and inferential statistics. Explain the relationship of population, sampling frame, sampling design, and generalization.
· Illustrate the use of hypothesis testing in determining statistical significance.
Each slide must have speaker notes that further explains the information on the slide. Your presentation must be at least eight slides in length, not including cover slide and references, and you must use at least two academic resources, one of which must come from the CSU Online Library. Any resources used must be cited and referenced in APA format.
**## UnitVIResearchDesignMethodsStudyGuide.pdf**
RCH 5301, Research Design and Methods 1
Course Learning Outcomes for Unit VI
At the end of this unit, you should be able to:
1. Integrate philosophical assumptions and worldviews held by researchers.
1.1 Appraise philosophical assumptions, research worldviews, research methodologies,
research traditions, and research designs.
2. Discriminate among the characteristics of qualitative and quantitative research methods.
2.1 Deduce differences between a quantitative methodological research strategy and
qualitative research strategy.
2.3 Compare and contrast descriptive statistics and inferential statistics.
3. Evaluate research designs.
3.1 Differentiate between descriptive, causal, and explanatory research designs based on their
unique characteristics.
4. Apply research methods within a research design.
4.1 the relationship of population, sampling frame, sampling design, and generalization.
4.3 Illustrate the use of hypothesis testing for determining statistical significance.
Required Unit Resources
Chapter 7: Research Questions and Hypotheses (ULO 4.3)
Chapter 8: Quantitative Methods (ULO 4.3)
Read the following sections from this chapter:
• Interpreting Results and Writing a Discussion Section
• Components of an Experimental Study Method Plan
• Participants
• Variables
• Instrumentation and Materials
• Experimental Procedures
• Threats to Validity
• The Procedure
• Data Analysis
• Preregistering the Study Plan
• Interpreting Results and Writing a Discussion Section
Unit Lesson
Lesson: Using Statistical Significance to Evaluate Data Analysis Results (ULO 4.3)
Statistical Significance and Hypothesis Testing
Hypotheses are a best guess about the researcher’s phenomenon of interest, and they serve as a starting
point for testing the guess. The null hypothesis is the assumption that no significant relationship or difference
exists between the variables being tested. The null hypothesis is designated by Ho. The alternative hypothesis
is the assumption that a significant relationship or difference exists between variables being tested. The
UNIT VI STUDY GUIDE
Quantitative Data Analysis and
Inferential Statistics
RCH 5301, Research Design and Methods 2
UNIT x STUDY GUIDE
Title
alternative hypothesis is designated by Ha. The null and alternative hypotheses are mutually exclusive,
meaning that after statistical results are produced, one will be accepted and the other rejected.
It is useful to think of hypothesis testing as serving one of two purposes: a) testing for statistically significant
relationships between variables, or b) testing for statistically significant differences between group means.
Correlational research methods, such as correlation and regression, look for statistically significant
relationships between variables. Causal-comparative methods, such as t-test and ANOVA, look for
statistically significant differences between group means.
To prevent confusion when conducting hypothesis testing, it is advisable to state hypotheses in a specific and
standard format. Specificity relates to precisely defining the variables to be measured and tested.
Standardization pertains to using the same verbiage when stating the null and alternative hypotheses. In the
examples below, the highlighted verbiage is standard language that can be used for most hypotheses.
Relationship Between Variables
Example One – Statistical test looking for a relationship between variables.
Ho1 There is no statistically significant relationship between x and y.
Ha1 There is a statistically significant relationship between x and y.
Differences Between Groups
Example Two – Statistical test looking for a difference between group means.
Ho2: There is no statistically significant difference in means between Group A and Group B.
Ha2: There is a statistically significant difference in means between Group A and Group B.
Hypothesis Testing Steps
Much of the content in this course was created to help students understand the practical use of research
methods. Nowhere is this more obvious than the use of statistical analysis. It is more important to understand
how to interpret the statistics to aid decision-making than it is to be able to explain all the math behind the
results. That is the domain of academics and statisticians. The following steps will highlight the relative
simplicity of hypothesis testing.
Hypothesis testing follows a logical progression regardless of whether the researcher is testing for
relationships (correlation) or testing for differences (causal-comparative). According to Cooper and Schindler
(2014) that progression includes the following steps:
1. State the null and alternative hypotheses.
2. Choose a statistical procedure that will be used to test the hypotheses (e.g., correlation, regression, t-
test, ANOVA, etc.)
3. Select the desired alpha (α), or significance level, that represents the risk of a Type I error. A .05 α is
customarily used in research.
4. Use the chosen statistical procedure to test the data, which produces a p value. This is accomplished
using a statistical application.
5. Interpret the test results. If the p value < .05 α, reject the null hypothesis and accept the alternative
hypothesis. If the p value ≥ .05 α, accept the null hypothesis and reject the alternative hypothesis.
Statistical Significance and Importance
Are significant results important? Are non-significant results useless? Many results can be statistically
significant but offer little practical importance. Bell et al. (2022) provide a clear example of how statistically
significant results can have no practical significance. As they explained, a significant r of .10 equating to R
squared of 1% is essentially useless for decision-making.
RCH 5301, Research Design and Methods 3
UNIT x STUDY GUIDE
Title
Conversely, non-significant results can offer the practitioner valuable insights about the domain of interest.
Assume a global company’s safety director wants to assess the effectiveness of a new safety training
program. Two different groups of employees; those having taken the training and those who have not, are
being tested for significant differences in recordable work-related injuries. The hypotheses are:
• Ho: There is no statistically significant difference in recordable work-related injuries between Group A
and Group B.
• Ha: There is a statistically significant difference in recordable work-related injuries between Group A
and Group B.
After an independent samples t-test is conducted, the results indicate a p value 17 > .05 α, therefore, the
result is non-significant, and the null hypothesis is accepted (and the alternative hypothesis is rejected) that
there is no statistically significant difference in recordable work-related injuries between Group A and Group
B. Although the conclusion to be drawn from the results of the hypothesis test is that the training is not
effective, this finding has immense practical value. The company may save thousands, hundreds of
thousands, or millions of dollars by not implementing the new, but ineffective, safety training program. So,
even though the results are not statistically significant, they offer great practical significance by providing
invaluable insights for decision-making.
Statistical Significance and Hypothesis Testing Summary
• Hypotheses either test for relationships between variables or differences between group means.
• Hypotheses should be specific and use standardized verbiage.
• Hypothesis testing follows a logical progression of steps.
• Significant results do not necessarily mean they offer practical importance.
• Non-significant results do not necessarily mean they lack practical importance.
• Significant results do not provide definitive proof the alternative hypothesis is true since a Type I error
is always possible.
• Non-significant results do not provide definitive proof the null hypothesis is true since a Type II error is
always possible.
The Lady Tasting Tea Experiment and the Origin of .05 α
As mentioned previously, an alpha of .05 is customarily used in research as the cut-off for statistical
significance, and many other applications of hypothesis testing. There is an interesting story of the origin of
the .05 α. It goes like this.
Once upon a time in Cambridge, England, on a lovely summer day during the 1920s, a group of academics
and their friends were enjoying afternoon tea. One of the women present made a claim to the astonished
group that tea tasted differently depending on whether the tea was poured first, or the milk was poured first,
and that she could blindly differentiate cups by taste alone. To say the academics were skeptical would be an
understatement. One of the academics in attendance, Dr. Ronald Fisher, decided to give her the benefit of
the doubt by constructing an ad-hoc, blind taste test, an experiment to test the hypothesis that the lady could
differentiate cups by taste (Salsburg, 2001).
The hypotheses were:
• Ho: There is no statistically significant difference in the lady’s ability to distinguish cups by taste vs.
random selection.
• Ha: There is a statistically significant difference in the lady’s ability to distinguish cups by taste vs.
random selection.
For the purposes of revealing the nature of statistical significance and its relation to probability for hypothesis
testing, assume the following three hypothesis tests were administered to the lady.
Test 1
In the first test, two choices were available: a) cup 1, tea poured first, and b) cup 2, milk poured first. If the
lady was able to identify either cup 1 or cup 2 by taste alone, nobody would be impressed since she had a 1
RCH 5301, Research Design and Methods 4
UNIT x STUDY GUIDE
Title
out of 2 chance of success, or 50% probability (see graphic), of making a correct selection by simply
guessing. Based on the results of this test, Ha would have been accepted that the lady could distinguish cups;
however, this test would have had a very low threshold for statistical significance and a very high chance for a
Type I error with what is effectively a .50 α.
Test 2
A second test involved four cups: a) cup 1, tea poured first, b) cup 2, tea poured first, c) cup 3, milk poured
first, and d) cup 4 milk poured first. A combination calculator can be used to determine that these four cups
result in 6 2-cup combinations from which the lady could have chosen.
Combination 1 = Cup 1, Cup 2
Combination 2 = Cup 3, Cup 4
Combination 3 = Cup 1, Cup 3
Combination 4 = Cup 1, Cup 4
Combination 5 = Cup 2, Cup 3
Combination 6 = Cup 2, Cup 4
If the lady had successfully identified the 2 cups with tea added first (Combination 1) or the 2 cups with milk
added first (Combination 2), it might have generated some applause, but it would be unlikely most observers
would have been overly impressed since she had a 1 out of 6 chance, or 16.7% probability (see graphic), of
selecting the correct combination by simply guessing. Once again, based on the results of this test, Ha would
have been accepted that the lady could distinguish cups based on taste alone. This test would have had a
moderate threshold for statistical significance and moderate chance of a Type I error with what is effectively
a .167α.
Test 3
This final test scenario uses 6 cups, with 3 cups having tea added first, and 3 cups having milk added first.
Again, using a combination calculator, these 6 cups result in 20 3-cup combinations from which the lady could
have chosen. If the lady had successfully identified the 3 cups with tea added first, or the 3 cups with milk
added first, that would have certainly elicited hoots and hollers. Observers would truly be impressed since she
had only a 1 out of 20 chance, or 5% probability, of selecting the correct permutation by simply guessing.
Based on the results of this test, Ha would have been accepted that the lady could distinguish cups based on
taste alone. This is a high threshold for statistical significance and a very low chance of a Type I error with
what is effectively a .05 α.
(Sustainability Methods, 2021)
Several years after the tea-tasting experiment, in 1929, Fisher wrote The Statistical Method in Psychical
Research in which he recommended p value < .05 α as an arbitrary, but convenient, cut-off for evaluating
statistical significance. He described a statistically significant event as one that would rarely be observed in
the absence of an independent explanatory variable(s). Rarity, from Fisher’s perspective, would be an event
RCH 5301, Research Design and Methods 5
UNIT x STUDY GUIDE
Title
that occurs randomly not more often than 1 in 20 tries, which translates into rejecting the null hypothesis and
accepting the alternative hypothesis when a statistical test returns a p value < .05 α. Legend holds that the
lady tasting tea was indeed able to successfully select three like cups. The question remains as to whether
this was one of those rare statistically significant events that was observed in the absence of an explanatory
variable, or if the test measured a true effect (Fisher, 1929; Lewis & Sauro, 2021).
Fisher was a mathematical genius who dedicated his life to science and math and was famous for his
contribution to the field of statistics and the many statistical procedures he created. Although Fisher’s writings
have been pored over, there is no indication if the Lady Tasting Tea experiment was the creative spark for
codifying .05 α as the arbitrary and convenient threshold for statistical significance, or if the Lady Tasting Tea
experiment was simply an opportunity to promote Fisher’s preconceived desire to see the scientific
community begin to adopt .05 α as the threshold for statistical significance. Regardless, .05 α has become the
standard, and the Lady Tasting Tea experiment has become legendary in statistical history.
Additional Inferential Tests of Differences
Sometimes, a researcher will find that One-Way ANOVA is limited in the number and types of variables that
can be included in the analysis. There are, fortunately, additional causal-comparative procedures that provide
flexibility to meet the needs of any combination or quantity of variables. The following will compare these
additional procedures with One-Way ANOVA. They include Two-Way ANOVA, ANCOVA, One-Way
MANOVA, Two-Way MANOVA, and MANCOVA.
Terminology
Terminology is once again important to understand the content in this section. The following will describe the
three main variable types used in the additional causal-comparative tests to be discussed below. These
variables are named response variable, factor, and covariate.
Response Variable
When using ANOVA, ANCOVA, One-Way MANOVA, Two-Way MANOVA, and MANCOVA, the dependent
variable (DV) is referred to as a response variable. A response variable is a continuous variable, which means
it is measured on at least an interval or ratio scale level. Examples of response variables include income,
time, height, length, depth, weight, growth, temperature, moisture, humidity, speed, distance, frequency,
antibodies, grades, etc.
Factor
A factor is an independent variable (IV) that is sometimes referred to as the grouping variable. The factor is a
categorical variable, which means it is measured on a nominal scale and specifies group membership.
Examples of factors include socio-economic status, race, ethnicity, religion, education level, industry,
business type, department, work shift, work team, batch, lot, production line, athletic team, gate, turnstile,
country, state, county, city, neighborhood, society, culture, treatment, eye color, hair color, etc.
Covariate
A covariate is a continuous independent variable, which means it is measured on at least an interval or ratio
scale level.
One-Way ANOVA (Analysis of Variance)
Like the t-test, ANOVA is a powerful causal-comparative inferential statistical procedure for analyzing
differences between means; however, it has an advantage over the bivariate t-test in that it can test for
differences in more than two means. For example, a researcher may be interested in determining if
statistically significant differences exist between average income (response variable) depending on degree
held (IV factor). More specifically, average income could be compared among the holders of the following
degrees: high school diploma, associate’s degree, bachelor’s degree, master’s degree, and terminal degree.
In this example, five income means would be compared for differences.
RCH 5301, Research Design and Methods 6
UNIT x STUDY GUIDE
Title
This example used one response variable and one IV factor, which is the reason for the name One-Way
ANOVA. Now, suppose the researcher is interested in analyzing the mean differences for a single response
variable, but for two IV factors. Adding two or more factors requires the use of Two-Way ANOVA.
Two-Way ANOVA (Analysis of Variance)
Two-Way ANOVA is used when a researcher is interested in determining if statistically significant differences
exist in means for a single response variable, but for two factors (Field, 2005). Expanding on the above
example, the researcher could compare average income (response variable) depending on degree held
(Factor1) and region (Factor2). At a granular level, the Two-Way ANOVA would compare average income for
a) the holders of the following degrees: high school diploma, associate’s degree, bachelor’s degree, master’s
degree, and terminal degree, and b) region, including Northeast, Southeast, Mid-Atlantic, Midwest,
Southwest, Northwest, and West.
It should be noted that there can be more than two factors used in this procedure. Whenever more than two
factors are used in ANOVA, with one response variable, it is referred to as factorial ANOVA.
ANCOVA (Analysis of Covariance)
ANCOVA is similar to ANOVA in that there is only a single response variable, which is continuous. However,
ANCOVA uses two IVs: a) one that is a categorical factor, and b) one that is a continuous variable. Therefore,
the C in ANCOVA represents the test of covariance between the response variable and the continuous
variable. The continuous IV is called a covariate (Field, 2005).
In the original example, a researcher was interested in determining if statistically significant differences exist
between average income (response variable) depending on degree held (Factor1), such as high school
diploma, associate degree, bachelor’s degree, master’s degree, and terminal degree. Using ANCOVA, the
researcher now includes the covariate GPA (Covariate1).
RCH 5301, Research Design and Methods 7
UNIT x STUDY GUIDE
Title
One-Way MANOVA (Multivariate Analysis of Variance)
Until this point, only one DV continuous response variable was used in the ANOVA and ANCOVA examples.
This changes with MANOVA and the addition of the M, which represents multivariate. One-way MANOVA
involves two continuous response variables and one categorical factor (Field, 2005). For example, a
researcher is interested in determining if statistically significant differences exist between: a) average income
(Response1), and b) average hours worked each week (Response2) depending on degree held (Factor1),
such as high school diploma, associate’s degree, bachelor’s degree, master’s degree, and terminal degree.
Two-Way MANOVA (Multivariate Analysis of Variance)
Like ANOVA, MANOVA can take a two-way approach, which includes two continuous response variables and
two IV categorical factors (Field, 2005). For example, the researcher in this example is interested in
determining if statistically significant differences exist between: a) average income (Response1), and b)
average hours worked each week (Response2) depending on: c) degree (Factor1), such as high school
diploma, and AA Degree, Bachelor’s Degree, Master’s Degree, and Terminal Degree, and d) region (Factor2),
including Northeast, Southeast, Mid-Atlantic, Midwest, Southwest, Northwest, and West.
It should be noted that there can be more than two categorical factors used in this procedure, and more than
two continuous response variables. Whenever more than two factors or more than two response variables are
used in MANOVA, it is referred to as factorial MANOVA.
RCH 5301, Research Design and Methods 8
UNIT x STUDY GUIDE
Title
MANCOVA (Multivariate Analysis of Covariance)
MANCOVA is similar to MANOVA with the addition of the C, representing covariance. MANCOVA and
MANOVA both include two or more continuous response variables, but MANCOVA includes at least one
categorical factor and at least one continuous variable, which is the covariate (Field, 2005). In this example of
MANCOVA, the researcher is interested in determining if statistically significant differences exist between: a)
average income (Response1), and b) average hours worked each week (Response2) depending on: c) degree
(Factor1 categorical variable) such as high school diploma, associate’s degree, bachelor’s degree, master’s
degree, and terminal degree, and d) hours of sleep each week (Covariate1 continuous variable).
Conclusion
Many students will never apply the statistical tests discussed in this course, but most will be exposed to
information that includes the results of these types of procedures, like regression and ANOVA, throughout
their careers. That is why it is important to have a good foundation for interpreting basic statistical results.
Statistical results are also often published in daily news reports on a variety of topics, so knowledge of
interpretation is helpful even outside of one’s interests. While it is not necessary for most students to
memorize the purpose and particulars of procedures like ANCOVA, MANOVA, and MANCOVA, it is important
that students are aware of the robustness of causal-comparative tests. If a need arises in this area, a good
research website or textbook can help anyone determine the appropriate test for their needs.
RCH 5301, Research Design and Methods 9
UNIT x STUDY GUIDE
Title
References
Bell, E., Bryman, A., & Harley, B. (2022). Business research methods (6th ed.). Oxford University Press.
https://online.vitalsource.com/#/books/9780192640505
Cooper, D. R., & Schindler, P. S. (2014). Business research methods (12th ed.). McGraw-Hill.
Creswell, J. W., & Creswell, J. D. (2022). Research design: Qualitative, quantitative, and mixed methods
approaches (6th ed.). SAGE. https://online.vitalsource.com/#/books/9781071817964
Field, A. (2005). Discovering stats using SPSS (2nd ed.). SAGE.
Fisher, R. A. (1929). The statistical method in psychical research. Proceedings of the Society for Psychical
Research, 39, 189–192 (1929).
Lewis, J., & Sauro, J. (2021). For statistical significance, must p be < .05? Measuring U.
https://measuringu.com/setting-alpha/
Salsburg, D. (2001). The lady tasting tea: How statistics revolutionized science in the twentieth century. Henry
Holt and Company.
Sustainability Methods. (2021, September 20). Designing studies [Image].
https://sustainabilitymethods.org/index.php/Designing_studies
– Course Learning Outcomes for Unit VI
– Required Unit Resources
– Chapter 7: Research Questions and Hypotheses (ULO 4.3)
– Chapter 8: Quantitative Methods (ULO 4.3)
– Unit Lesson
– Lesson: Using Statistical Significance to Evaluate Data Analysis Results (ULO 4.3)
– Statistical Significance and Hypothesis Testing
– Hypothesis Testing Steps
– Statistical Significance and Importance
– Statistical Significance and Hypothesis Testing Summary
– The Lady Tasting Tea Experiment and the Origin of .05 α
– Additional Inferential Tests of Differences
– Terminology
– Response Variable
– Factor
– Covariate
– One-Way ANOVA (Analysis of Variance)
– Two-Way ANOVA (Analysis of Variance)
– ANCOVA (Analysis of Covariance)
– One-Way MANOVA (Multivariate Analysis of Variance)
– Two-Way MANOVA (Multivariate Analysis of Variance)
– MANCOVA (Multivariate Analysis of Covariance)
– Conclusion
– References
**## UnitVIPowerPointPresentation.docx**
Unit VI PowerPoint Presentation
This PowerPoint Presentation measures your mastery of ULOs 1.1, 2.1, 2.3, 3.1, 4.1, and 4.3.
Imagine you, the student, are the professor for this course. Drawing from the information presented in Units I-VI, create an 8-slide PowerPoint presentation with speaker notes communicating the most important aspects of research methods. Be sure to cover the following points:
· Schematize philosophical assumptions, research worldviews, research methodologies, research traditions, and research designs.
· Diagram differences between a quantitative methodological research strategy and qualitative research strategy.
· Differentiate descriptive, explanatory, and exploratory research designs based on their unique characteristics.
· Compare and contrast descriptive statistics and inferential statistics. Explain the relationship of population, sampling frame, sampling design, and generalization.
· Illustrate the use of hypothesis testing in determining statistical significance.
Each slide must have speaker notes that further explains the information on the slide. Your presentation must be at least eight slides in length, not including cover slide and references, and you must use at least two academic resources, one of which must come from the CSU Online Library. Any resources used must be cited and referenced in APA format.
**## UnitVIResearchDesignMethodsStudyGuide.pdf**
RCH 5301, Research Design and Methods 1
Course Learning Outcomes for Unit VI
At the end of this unit, you should be able to:
1. Integrate philosophical assumptions and worldviews held by researchers.
1.1 Appraise philosophical assumptions, research worldviews, research methodologies,
research traditions, and research designs.
2. Discriminate among the characteristics of qualitative and quantitative research methods.
2.1 Deduce differences between a quantitative methodological research strategy and
qualitative research strategy.
2.3 Compare and contrast descriptive statistics and inferential statistics.
3. Evaluate research designs.
3.1 Differentiate between descriptive, causal, and explanatory research designs based on their
unique characteristics.
4. Apply research methods within a research design.
4.1 the relationship of population, sampling frame, sampling design, and generalization.
4.3 Illustrate the use of hypothesis testing for determining statistical significance.
Required Unit Resources
Chapter 7: Research Questions and Hypotheses (ULO 4.3)
Chapter 8: Quantitative Methods (ULO 4.3)
Read the following sections from this chapter:
• Interpreting Results and Writing a Discussion Section
• Components of an Experimental Study Method Plan
• Participants
• Variables
• Instrumentation and Materials
• Experimental Procedures
• Threats to Validity
• The Procedure
• Data Analysis
• Preregistering the Study Plan
• Interpreting Results and Writing a Discussion Section
Unit Lesson
Lesson: Using Statistical Significance to Evaluate Data Analysis Results (ULO 4.3)
Statistical Significance and Hypothesis Testing
Hypotheses are a best guess about the researcher’s phenomenon of interest, and they serve as a starting
point for testing the guess. The null hypothesis is the assumption that no significant relationship or difference
exists between the variables being tested. The null hypothesis is designated by Ho. The alternative hypothesis
is the assumption that a significant relationship or difference exists between variables being tested. The
UNIT VI STUDY GUIDE
Quantitative Data Analysis and
Inferential Statistics
RCH 5301, Research Design and Methods 2
UNIT x STUDY GUIDE
Title
alternative hypothesis is designated by Ha. The null and alternative hypotheses are mutually exclusive,
meaning that after statistical results are produced, one will be accepted and the other rejected.
It is useful to think of hypothesis testing as serving one of two purposes: a) testing for statistically significant
relationships between variables, or b) testing for statistically significant differences between group means.
Correlational research methods, such as correlation and regression, look for statistically significant
relationships between variables. Causal-comparative methods, such as t-test and ANOVA, look for
statistically significant differences between group means.
To prevent confusion when conducting hypothesis testing, it is advisable to state hypotheses in a specific and
standard format. Specificity relates to precisely defining the variables to be measured and tested.
Standardization pertains to using the same verbiage when stating the null and alternative hypotheses. In the
examples below, the highlighted verbiage is standard language that can be used for most hypotheses.
Relationship Between Variables
Example One – Statistical test looking for a relationship between variables.
Ho1 There is no statistically significant relationship between x and y.
Ha1 There is a statistically significant relationship between x and y.
Differences Between Groups
Example Two – Statistical test looking for a difference between group means.
Ho2: There is no statistically significant difference in means between Group A and Group B.
Ha2: There is a statistically significant difference in means between Group A and Group B.
Hypothesis Testing Steps
Much of the content in this course was created to help students understand the practical use of research
methods. Nowhere is this more obvious than the use of statistical analysis. It is more important to understand
how to interpret the statistics to aid decision-making than it is to be able to explain all the math behind the
results. That is the domain of academics and statisticians. The following steps will highlight the relative
simplicity of hypothesis testing.
Hypothesis testing follows a logical progression regardless of whether the researcher is testing for
relationships (correlation) or testing for differences (causal-comparative). According to Cooper and Schindler
(2014) that progression includes the following steps:
1. State the null and alternative hypotheses.
2. Choose a statistical procedure that will be used to test the hypotheses (e.g., correlation, regression, t-
test, ANOVA, etc.)
3. Select the desired alpha (α), or significance level, that represents the risk of a Type I error. A .05 α is
customarily used in research.
4. Use the chosen statistical procedure to test the data, which produces a p value. This is accomplished
using a statistical application.
5. Interpret the test results. If the p value < .05 α, reject the null hypothesis and accept the alternative
hypothesis. If the p value ≥ .05 α, accept the null hypothesis and reject the alternative hypothesis.
Statistical Significance and Importance
Are significant results important? Are non-significant results useless? Many results can be statistically
significant but offer little practical importance. Bell et al. (2022) provide a clear example of how statistically
significant results can have no practical significance. As they explained, a significant r of .10 equating to R
squared of 1% is essentially useless for decision-making.
RCH 5301, Research Design and Methods 3
UNIT x STUDY GUIDE
Title
Conversely, non-significant results can offer the practitioner valuable insights about the domain of interest.
Assume a global company’s safety director wants to assess the effectiveness of a new safety training
program. Two different groups of employees; those having taken the training and those who have not, are
being tested for significant differences in recordable work-related injuries. The hypotheses are:
• Ho: There is no statistically significant difference in recordable work-related injuries between Group A
and Group B.
• Ha: There is a statistically significant difference in recordable work-related injuries between Group A
and Group B.
After an independent samples t-test is conducted, the results indicate a p value 17 > .05 α, therefore, the
result is non-significant, and the null hypothesis is accepted (and the alternative hypothesis is rejected) that
there is no statistically significant difference in recordable work-related injuries between Group A and Group
B. Although the conclusion to be drawn from the results of the hypothesis test is that the training is not
effective, this finding has immense practical value. The company may save thousands, hundreds of
thousands, or millions of dollars by not implementing the new, but ineffective, safety training program. So,
even though the results are not statistically significant, they offer great practical significance by providing
invaluable insights for decision-making.
Statistical Significance and Hypothesis Testing Summary
• Hypotheses either test for relationships between variables or differences between group means.
• Hypotheses should be specific and use standardized verbiage.
• Hypothesis testing follows a logical progression of steps.
• Significant results do not necessarily mean they offer practical importance.
• Non-significant results do not necessarily mean they lack practical importance.
• Significant results do not provide definitive proof the alternative hypothesis is true since a Type I error
is always possible.
• Non-significant results do not provide definitive proof the null hypothesis is true since a Type II error is
always possible.
The Lady Tasting Tea Experiment and the Origin of .05 α
As mentioned previously, an alpha of .05 is customarily used in research as the cut-off for statistical
significance, and many other applications of hypothesis testing. There is an interesting story of the origin of
the .05 α. It goes like this.
Once upon a time in Cambridge, England, on a lovely summer day during the 1920s, a group of academics
and their friends were enjoying afternoon tea. One of the women present made a claim to the astonished
group that tea tasted differently depending on whether the tea was poured first, or the milk was poured first,
and that she could blindly differentiate cups by taste alone. To say the academics were skeptical would be an
understatement. One of the academics in attendance, Dr. Ronald Fisher, decided to give her the benefit of
the doubt by constructing an ad-hoc, blind taste test, an experiment to test the hypothesis that the lady could
differentiate cups by taste (Salsburg, 2001).
The hypotheses were:
• Ho: There is no statistically significant difference in the lady’s ability to distinguish cups by taste vs.
random selection.
• Ha: There is a statistically significant difference in the lady’s ability to distinguish cups by taste vs.
random selection.
For the purposes of revealing the nature of statistical significance and its relation to probability for hypothesis
testing, assume the following three hypothesis tests were administered to the lady.
Test 1
In the first test, two choices were available: a) cup 1, tea poured first, and b) cup 2, milk poured first. If the
lady was able to identify either cup 1 or cup 2 by taste alone, nobody would be impressed since she had a 1
RCH 5301, Research Design and Methods 4
UNIT x STUDY GUIDE
Title
out of 2 chance of success, or 50% probability (see graphic), of making a correct selection by simply
guessing. Based on the results of this test, Ha would have been accepted that the lady could distinguish cups;
however, this test would have had a very low threshold for statistical significance and a very high chance for a
Type I error with what is effectively a .50 α.
Test 2
A second test involved four cups: a) cup 1, tea poured first, b) cup 2, tea poured first, c) cup 3, milk poured
first, and d) cup 4 milk poured first. A combination calculator can be used to determine that these four cups
result in 6 2-cup combinations from which the lady could have chosen.
Combination 1 = Cup 1, Cup 2
Combination 2 = Cup 3, Cup 4
Combination 3 = Cup 1, Cup 3
Combination 4 = Cup 1, Cup 4
Combination 5 = Cup 2, Cup 3
Combination 6 = Cup 2, Cup 4
If the lady had successfully identified the 2 cups with tea added first (Combination 1) or the 2 cups with milk
added first (Combination 2), it might have generated some applause, but it would be unlikely most observers
would have been overly impressed since she had a 1 out of 6 chance, or 16.7% probability (see graphic), of
selecting the correct combination by simply guessing. Once again, based on the results of this test, Ha would
have been accepted that the lady could distinguish cups based on taste alone. This test would have had a
moderate threshold for statistical significance and moderate chance of a Type I error with what is effectively
a .167α.
Test 3
This final test scenario uses 6 cups, with 3 cups having tea added first, and 3 cups having milk added first.
Again, using a combination calculator, these 6 cups result in 20 3-cup combinations from which the lady could
have chosen. If the lady had successfully identified the 3 cups with tea added first, or the 3 cups with milk
added first, that would have certainly elicited hoots and hollers. Observers would truly be impressed since she
had only a 1 out of 20 chance, or 5% probability, of selecting the correct permutation by simply guessing.
Based on the results of this test, Ha would have been accepted that the lady could distinguish cups based on
taste alone. This is a high threshold for statistical significance and a very low chance of a Type I error with
what is effectively a .05 α.
(Sustainability Methods, 2021)
Several years after the tea-tasting experiment, in 1929, Fisher wrote The Statistical Method in Psychical
Research in which he recommended p value < .05 α as an arbitrary, but convenient, cut-off for evaluating
statistical significance. He described a statistically significant event as one that would rarely be observed in
the absence of an independent explanatory variable(s). Rarity, from Fisher’s perspective, would be an event
RCH 5301, Research Design and Methods 5
UNIT x STUDY GUIDE
Title
that occurs randomly not more often than 1 in 20 tries, which translates into rejecting the null hypothesis and
accepting the alternative hypothesis when a statistical test returns a p value < .05 α. Legend holds that the
lady tasting tea was indeed able to successfully select three like cups. The question remains as to whether
this was one of those rare statistically significant events that was observed in the absence of an explanatory
variable, or if the test measured a true effect (Fisher, 1929; Lewis & Sauro, 2021).
Fisher was a mathematical genius who dedicated his life to science and math and was famous for his
contribution to the field of statistics and the many statistical procedures he created. Although Fisher’s writings
have been pored over, there is no indication if the Lady Tasting Tea experiment was the creative spark for
codifying .05 α as the arbitrary and convenient threshold for statistical significance, or if the Lady Tasting Tea
experiment was simply an opportunity to promote Fisher’s preconceived desire to see the scientific
community begin to adopt .05 α as the threshold for statistical significance. Regardless, .05 α has become the
standard, and the Lady Tasting Tea experiment has become legendary in statistical history.
Additional Inferential Tests of Differences
Sometimes, a researcher will find that One-Way ANOVA is limited in the number and types of variables that
can be included in the analysis. There are, fortunately, additional causal-comparative procedures that provide
flexibility to meet the needs of any combination or quantity of variables. The following will compare these
additional procedures with One-Way ANOVA. They include Two-Way ANOVA, ANCOVA, One-Way
MANOVA, Two-Way MANOVA, and MANCOVA.
Terminology
Terminology is once again important to understand the content in this section. The following will describe the
three main variable types used in the additional causal-comparative tests to be discussed below. These
variables are named response variable, factor, and covariate.
Response Variable
When using ANOVA, ANCOVA, One-Way MANOVA, Two-Way MANOVA, and MANCOVA, the dependent
variable (DV) is referred to as a response variable. A response variable is a continuous variable, which means
it is measured on at least an interval or ratio scale level. Examples of response variables include income,
time, height, length, depth, weight, growth, temperature, moisture, humidity, speed, distance, frequency,
antibodies, grades, etc.
Factor
A factor is an independent variable (IV) that is sometimes referred to as the grouping variable. The factor is a
categorical variable, which means it is measured on a nominal scale and specifies group membership.
Examples of factors include socio-economic status, race, ethnicity, religion, education level, industry,
business type, department, work shift, work team, batch, lot, production line, athletic team, gate, turnstile,
country, state, county, city, neighborhood, society, culture, treatment, eye color, hair color, etc.
Covariate
A covariate is a continuous independent variable, which means it is measured on at least an interval or ratio
scale level.
One-Way ANOVA (Analysis of Variance)
Like the t-test, ANOVA is a powerful causal-comparative inferential statistical procedure for analyzing
differences between means; however, it has an advantage over the bivariate t-test in that it can test for
differences in more than two means. For example, a researcher may be interested in determining if
statistically significant differences exist between average income (response variable) depending on degree
held (IV factor). More specifically, average income could be compared among the holders of the following
degrees: high school diploma, associate’s degree, bachelor’s degree, master’s degree, and terminal degree.
In this example, five income means would be compared for differences.
RCH 5301, Research Design and Methods 6
UNIT x STUDY GUIDE
Title
This example used one response variable and one IV factor, which is the reason for the name One-Way
ANOVA. Now, suppose the researcher is interested in analyzing the mean differences for a single response
variable, but for two IV factors. Adding two or more factors requires the use of Two-Way ANOVA.
Two-Way ANOVA (Analysis of Variance)
Two-Way ANOVA is used when a researcher is interested in determining if statistically significant differences
exist in means for a single response variable, but for two factors (Field, 2005). Expanding on the above
example, the researcher could compare average income (response variable) depending on degree held
(Factor1) and region (Factor2). At a granular level, the Two-Way ANOVA would compare average income for
a) the holders of the following degrees: high school diploma, associate’s degree, bachelor’s degree, master’s
degree, and terminal degree, and b) region, including Northeast, Southeast, Mid-Atlantic, Midwest,
Southwest, Northwest, and West.
It should be noted that there can be more than two factors used in this procedure. Whenever more than two
factors are used in ANOVA, with one response variable, it is referred to as factorial ANOVA.
ANCOVA (Analysis of Covariance)
ANCOVA is similar to ANOVA in that there is only a single response variable, which is continuous. However,
ANCOVA uses two IVs: a) one that is a categorical factor, and b) one that is a continuous variable. Therefore,
the C in ANCOVA represents the test of covariance between the response variable and the continuous
variable. The continuous IV is called a covariate (Field, 2005).
In the original example, a researcher was interested in determining if statistically significant differences exist
between average income (response variable) depending on degree held (Factor1), such as high school
diploma, associate degree, bachelor’s degree, master’s degree, and terminal degree. Using ANCOVA, the
researcher now includes the covariate GPA (Covariate1).
RCH 5301, Research Design and Methods 7
UNIT x STUDY GUIDE
Title
One-Way MANOVA (Multivariate Analysis of Variance)
Until this point, only one DV continuous response variable was used in the ANOVA and ANCOVA examples.
This changes with MANOVA and the addition of the M, which represents multivariate. One-way MANOVA
involves two continuous response variables and one categorical factor (Field, 2005). For example, a
researcher is interested in determining if statistically significant differences exist between: a) average income
(Response1), and b) average hours worked each week (Response2) depending on degree held (Factor1),
such as high school diploma, associate’s degree, bachelor’s degree, master’s degree, and terminal degree.
Two-Way MANOVA (Multivariate Analysis of Variance)
Like ANOVA, MANOVA can take a two-way approach, which includes two continuous response variables and
two IV categorical factors (Field, 2005). For example, the researcher in this example is interested in
determining if statistically significant differences exist between: a) average income (Response1), and b)
average hours worked each week (Response2) depending on: c) degree (Factor1), such as high school
diploma, and AA Degree, Bachelor’s Degree, Master’s Degree, and Terminal Degree, and d) region (Factor2),
including Northeast, Southeast, Mid-Atlantic, Midwest, Southwest, Northwest, and West.
It should be noted that there can be more than two categorical factors used in this procedure, and more than
two continuous response variables. Whenever more than two factors or more than two response variables are
used in MANOVA, it is referred to as factorial MANOVA.
RCH 5301, Research Design and Methods 8
UNIT x STUDY GUIDE
Title
MANCOVA (Multivariate Analysis of Covariance)
MANCOVA is similar to MANOVA with the addition of the C, representing covariance. MANCOVA and
MANOVA both include two or more continuous response variables, but MANCOVA includes at least one
categorical factor and at least one continuous variable, which is the covariate (Field, 2005). In this example of
MANCOVA, the researcher is interested in determining if statistically significant differences exist between: a)
average income (Response1), and b) average hours worked each week (Response2) depending on: c) degree
(Factor1 categorical variable) such as high school diploma, associate’s degree, bachelor’s degree, master’s
degree, and terminal degree, and d) hours of sleep each week (Covariate1 continuous variable).
Conclusion
Many students will never apply the statistical tests discussed in this course, but most will be exposed to
information that includes the results of these types of procedures, like regression and ANOVA, throughout
their careers. That is why it is important to have a good foundation for interpreting basic statistical results.
Statistical results are also often published in daily news reports on a variety of topics, so knowledge of
interpretation is helpful even outside of one’s interests. While it is not necessary for most students to
memorize the purpose and particulars of procedures like ANCOVA, MANOVA, and MANCOVA, it is important
that students are aware of the robustness of causal-comparative tests. If a need arises in this area, a good
research website or textbook can help anyone determine the appropriate test for their needs.
RCH 5301, Research Design and Methods 9
UNIT x STUDY GUIDE
Title
References
Bell, E., Bryman, A., & Harley, B. (2022). Business research methods (6th ed.). Oxford University Press.
https://online.vitalsource.com/#/books/9780192640505
Cooper, D. R., & Schindler, P. S. (2014). Business research methods (12th ed.). McGraw-Hill.
Creswell, J. W., & Creswell, J. D. (2022). Research design: Qualitative, quantitative, and mixed methods
approaches (6th ed.). SAGE. https://online.vitalsource.com/#/books/9781071817964
Field, A. (2005). Discovering stats using SPSS (2nd ed.). SAGE.
Fisher, R. A. (1929). The statistical method in psychical research. Proceedings of the Society for Psychical
Research, 39, 189–192 (1929).
Lewis, J., & Sauro, J. (2021). For statistical significance, must p be < .05? Measuring U.
https://measuringu.com/setting-alpha/
Salsburg, D. (2001). The lady tasting tea: How statistics revolutionized science in the twentieth century. Henry
Holt and Company.
Sustainability Methods. (2021, September 20). Designing studies [Image].
https://sustainabilitymethods.org/index.php/Designing_studies
– Course Learning Outcomes for Unit VI
– Required Unit Resources
– Chapter 7: Research Questions and Hypotheses (ULO 4.3)
– Chapter 8: Quantitative Methods (ULO 4.3)
– Unit Lesson
– Lesson: Using Statistical Significance to Evaluate Data Analysis Results (ULO 4.3)
– Statistical Significance and Hypothesis Testing
– Hypothesis Testing Steps
– Statistical Significance and Importance
– Statistical Significance and Hypothesis Testing Summary
– The Lady Tasting Tea Experiment and the Origin of .05 α
– Additional Inferential Tests of Differences
– Terminology
– Response Variable
– Factor
– Covariate
– One-Way ANOVA (Analysis of Variance)
– Two-Way ANOVA (Analysis of Variance)
– ANCOVA (Analysis of Covariance)
– One-Way MANOVA (Multivariate Analysis of Variance)
– Two-Way MANOVA (Multivariate Analysis of Variance)
– MANCOVA (Multivariate Analysis of Covariance)
– Conclusion
– References