Question 1

Which one of the following scenarios illustrates a stratified random sampling procedure?

Accepted Answer

A)  Putting all participants' names into a hat, and blindly drawing names from the hat. 
B)  Putting all female participants' names into one hat, male participants' names - into another, and blindly drawing names from each hat separately. 
C)  Seating all participants around square tables, and choosing participants who sit facing North (one side of each table). 
D)  Making a list of all potential participants, and choosing every third person on the list. 
A)  Putting all participants' names into a hat, and blindly drawing names from the hat. 
B)  Putting all female participants' names into one hat, male participants' names - into another, and blindly drawing names from each hat separately. 
C)  Seating all participants around square tables, and choosing participants who sit facing North (one side of each table). 
D)  Making a list of all potential participants, and choosing every third person on the list. B

Question 2

What is the reason that geography is especially useful for sample stratification purposes?

Accepted Answer

A)  People live in every type of geographic area. 
B)  People would not be offended if selected for a study based on their geographic location. 
C)  Many important variables in social sciences are clustered based on a geographic area. 
D)  Different geographic areas would allow researchers to enjoy the travel aspect of their study. 
A)  People live in every type of geographic area. 
B)  People would not be offended if selected for a study based on their geographic location. 
C)  Many important variables in social sciences are clustered based on a geographic area. 
D)  Different geographic areas would allow researchers to enjoy the travel aspect of their study. C

Question 3

What is the biggest threat from self-selection bias, according to the text?

Accepted Answer

A)  Those who have refused to participate would have a negative view of the study. 
B)  Those who have agreed to participate enjoyed their study experience. 
C)  The results of the study may not be generalizable to the population. 
D)  The results of the study may not reach statistical significance. 
A)  Those who have refused to participate would have a negative view of the study. 
B)  Those who have agreed to participate enjoyed their study experience. 
C)  The results of the study may not be generalizable to the population. 
D)  The results of the study may not reach statistical significance. C

Question 4

All of the following examples correctly reflect concerns with selection bias, EXCEPT:

Accepted Answer

A)  Those who refused to participate in the study may be different in some important respects from those who agreed to participate. 
B)  In a longitudinal study, some participants who started participating, later withdraw from the study. 
C)  Additional participants have been selected for the study when it was already under way. 
D)  Researchers have found out that those from more affluent neighborhoods have a higher response rate to the study. 
A)  Those who refused to participate in the study may be different in some important respects from those who agreed to participate. 
B)  In a longitudinal study, some participants who started participating, later withdraw from the study. 
C)  Additional participants have been selected for the study when it was already under way. 
D)  Researchers have found out that those from more affluent neighborhoods have a higher response rate to the study.

Question 5

When several subgroups are compared in a study, what is the rule of thumb about the sample or subsample size?

Accepted Answer

A)  At least 100 participants in the study is needed. 
B)  At least 50 participants in the study is needed. 
C)  At least 30 participants in each subgroup is needed. 
D)  At least 15 participants in each subgroup is needed. 
A)  At least 100 participants in the study is needed. 
B)  At least 50 participants in the study is needed. 
C)  At least 30 participants in each subgroup is needed. 
D)  At least 15 participants in each subgroup is needed.

Question 6

Which one of the following statements about sampling issues is INCORRECT?

Accepted Answer

A)  One of the main goals of research is to generalize from samples to populations. 
B)  Generalizations cannot be drawn from studies with small or nonrandom samples. 
C)  A researcher may have serious flaws in the sampling and yet he has done a reasonably good job in the study. 
D)  When evaluating sampling procedures in a study, it is important to take into account real-world constraints. 
A)  One of the main goals of research is to generalize from samples to populations. 
B)  Generalizations cannot be drawn from studies with small or nonrandom samples. 
C)  A researcher may have serious flaws in the sampling and yet he has done a reasonably good job in the study. 
D)  When evaluating sampling procedures in a study, it is important to take into account real-world constraints.

Question 7

There are several considerations important to evaluating if the sample size is adequate. Which one of the following considerations is IRRELEVANT?

Accepted Answer

A)  How accessible is the population that the researchers are interested in? 
B)  Could the study have been published if a smaller sample were used? 
C)  Is the number of participants so low that it would be hard to make sound generalizations? 
D)  Is the study supposed to provide important answers for informing policies and practice? 
A)  How accessible is the population that the researchers are interested in? 
B)  Could the study have been published if a smaller sample were used? 
C)  Is the number of participants so low that it would be hard to make sound generalizations? 
D)  Is the study supposed to provide important answers for informing policies and practice?

Question 8

Which one of the following statements about random sampling is INCORRECT?

Accepted Answer

A)  Most studies in social sciences are conducted on random samples. 
B)  Nonrandom samples can be used for generalizing. 
C)  If some participants randomly selected for a study have refused to participate, the resulting sample is not random. 
D)  When a professor invites all the students attending her classes to participate in her study, the sample is not random. 
A)  Most studies in social sciences are conducted on random samples. 
B)  Nonrandom samples can be used for generalizing. 
C)  If some participants randomly selected for a study have refused to participate, the resulting sample is not random. 
D)  When a professor invites all the students attending her classes to participate in her study, the sample is not random.

Question 9

There are several important rules about research ethics. Which one of the following rules is NOT among them?

Accepted Answer

A)  Any study involving human subjects needs prior approval by an ethics review agency. 
B)  The goal of ethics review is to make sure that the study does not present undue harms to the participants. 
C)  Informed consent is required for any research study involving human subjects. 
D)  Informed consent notifies potential participants that they can withdraw from the study at any time without penalty. 
A)  Any study involving human subjects needs prior approval by an ethics review agency. 
B)  The goal of ethics review is to make sure that the study does not present undue harms to the participants. 
C)  Informed consent is required for any research study involving human subjects. 
D)  Informed consent notifies potential participants that they can withdraw from the study at any time without penalty.

Question 10

There are several important reasons why researchers are using nonrandom samples in their studies. Which one of the following is NOT a valid reason?

Accepted Answer

A)  Some people randomly selected for participation in a study refuse to participate. 
B)  Random sampling is often cumbersome, time-consuming, and expensive. 
C)  Nonrandom sampling produces a sample that is more representative of the population. 
D)  There are target populations for which there is no available list of all members. 
A)  Some people randomly selected for participation in a study refuse to participate. 
B)  Random sampling is often cumbersome, time-consuming, and expensive. 
C)  Nonrandom sampling produces a sample that is more representative of the population. 
D)  There are target populations for which there is no available list of all members.

Question 11

All of the following conditions help researchers to achieve an unbiased sample, EXCEPT:

Accepted Answer

A)  Using random sampling. 
B)  Using a reasonably large sample. 
C)  Using a stratified random sample. 
D)  Using a convenience sample. 
A)  Using random sampling. 
B)  Using a reasonably large sample. 
C)  Using a stratified random sample. 
D)  Using a convenience sample.

Question 12

When some participants refuse to participate in a study, what participation rate is considered to be reasonably high?

Accepted Answer

A)  Above 50%. 
B)  Above 30%. 
C)  Higher than in the previous studies on a similar topic. 
D)  There is no clear rule for deciding which participation rate is reasonably high. 
A)  Above 50%. 
B)  Above 30%. 
C)  Higher than in the previous studies on a similar topic. 
D)  There is no clear rule for deciding which participation rate is reasonably high.

Question 13

When a sample is nonrandom, there are some important ways to improve the researchers' ability to generalize their study results. Which one of the following sampling strategies does NOT improve the nonrandom sampling process?

Accepted Answer

A)  A sample for a study of alcohol use among high school students was solicited from multiple high schools. 
B)  A sample for a study of mental health needs of homeless people was solicited from several homeless shelters. 
C)  A sample for a study of local-area parents was solicited at various school and community events for parents. 
D)  A sample for a study of young adults was solicited from several classes at a local college where the researcher works. 
A)  A sample for a study of alcohol use among high school students was solicited from multiple high schools. 
B)  A sample for a study of mental health needs of homeless people was solicited from several homeless shelters. 
C)  A sample for a study of local-area parents was solicited at various school and community events for parents. 
D)  A sample for a study of young adults was solicited from several classes at a local college where the researcher works.

Question 14

When comparing survey response rates for mail and online surveys, which one of the following statements is INCORRECT?

Accepted Answer

A)  Multiple attempts to contact potential survey participants can be made using both methods. 
B)  It is generally preferable to contact potential study participants by mail first. 
C)  Any study comparing the mail vs web-based survey response rates would be limited to a specific target population and specific survey topic. 
D)  There is no clear answer whether the response rate is higher for mailed or online surveys. 
A)  Multiple attempts to contact potential survey participants can be made using both methods. 
B)  It is generally preferable to contact potential study participants by mail first. 
C)  Any study comparing the mail vs web-based survey response rates would be limited to a specific target population and specific survey topic. 
D)  There is no clear answer whether the response rate is higher for mailed or online surveys.

Question 15

Which one of the following statements about sample size is correct?

Accepted Answer

A)  The size of the sample depends on how much error a researcher is willing to tolerate. 
B)  Studies using large nonrandom samples produce more generalizable results than small studies using random samples. 
C)  Random samples are generally larger than nonrandom ones. 
D)  Whether a sample is large enough does not have any bearing on the study results. 
A)  The size of the sample depends on how much error a researcher is willing to tolerate. 
B)  Studies using large nonrandom samples produce more generalizable results than small studies using random samples. 
C)  Random samples are generally larger than nonrandom ones. 
D)  Whether a sample is large enough does not have any bearing on the study results.

Question 16

Researchers are supposed to describe the demographics of the sample in their article. Which one of the following variables is unlikely to be included among the demographics?

Accepted Answer

A)  Age. 
B)  Gender. 
C)  Race. 
D)  Eye color. 
A)  Age. 
B)  Gender. 
C)  Race. 
D)  Eye color.

Question 17

All of the following considerations would be relevant for deciding whether the participation rate in a specific study is reasonably high, EXCEPT:

Accepted Answer

A)  Were the study participants asked if they enjoyed their study experience? 
B)  Were potential participants contacted multiple times before the study? 
C)  Was the study invasive or uncomfortable? 
D)  Was the typical study participation session long? 
A)  Were the study participants asked if they enjoyed their study experience? 
B)  Were potential participants contacted multiple times before the study? 
C)  Was the study invasive or uncomfortable? 
D)  Was the typical study participation session long?

Question 18

All of the following statements about the relationship between a sample and population are true, EXCEPT:

Accepted Answer

A)  A sample is a subset of a population. 
B)  Researchers study samples but not populations. 
C)  Researchers often study samples for the sake of efficiency. 
D)  When study results are obtained on a sample, they are then generalized to the relevant population. 
A)  A sample is a subset of a population. 
B)  Researchers study samples but not populations. 
C)  Researchers often study samples for the sake of efficiency. 
D)  When study results are obtained on a sample, they are then generalized to the relevant population.

Question 19

There are several good reasons why it is important for researchers to discuss the limitations of their sampling in a research report. Which one of the following is the LEAST important reason?

Accepted Answer

A)  The Discussion section needs to be filled with enough information. 
B)  Discussing study limitations helps the reader understand the flaws in the study. 
C)  Sampling limitations directly affect the generalizability of study results. 
D)  The readers of the journal will be expecting to see a discussion of study limitations. 
A)  The Discussion section needs to be filled with enough information. 
B)  Discussing study limitations helps the reader understand the flaws in the study. 
C)  Sampling limitations directly affect the generalizability of study results. 
D)  The readers of the journal will be expecting to see a discussion of study limitations.

Which one of the following scenarios illustrates a stratified random sampling procedure?

What is the reason that geography is especially useful for sample stratification purposes?

What is the biggest threat from self-selection bias, according to the text?

All of the following examples correctly reflect concerns with selection bias, EXCEPT:

When several subgroups are compared in a study, what is the rule of thumb about the sample or subsample size?

Which one of the following statements about sampling issues is INCORRECT?

There are several considerations important to evaluating if the sample size is adequate. Which one of the following considerations is IRRELEVANT?

Which one of the following statements about random sampling is INCORRECT?

There are several important rules about research ethics. Which one of the following rules is NOT among them?

There are several important reasons why researchers are using nonrandom samples in their studies. Which one of the following is NOT a valid reason?

All of the following conditions help researchers to achieve an unbiased sample, EXCEPT:

When some participants refuse to participate in a study, what participation rate is considered to be reasonably high?

When a sample is nonrandom, there are some important ways to improve the researchers' ability to generalize their study results. Which one of the following sampling strategies does NOT improve the nonrandom sampling process?

When comparing survey response rates for mail and online surveys, which one of the following statements is INCORRECT?

Which one of the following statements about sample size is correct?

Researchers are supposed to describe the demographics of the sample in their article. Which one of the following variables is unlikely to be included among the demographics?

All of the following considerations would be relevant for deciding whether the participation rate in a specific study is reasonably high, EXCEPT:

All of the following statements about the relationship between a sample and population are true, EXCEPT:

There are several good reasons why it is important for researchers to discuss the limitations of their sampling in a research report. Which one of the following is the LEAST important reason?

Background for Evaluating Research Reports

Evaluating Titles

Evaluating Abstracts

Evaluating Introductions and Literature Reviews

A Closer Look at Evaluating Literature Reviews

Evaluating Samples When Researchers Do Not Generalize

Evaluating Measures

Evaluating Experimental Procedures

Evaluating Analysis and Results Sections: Quantitative Research

Evaluating Analysis and Results Sections: Qualitative Research

Evaluating Analysis and Results Sections: Mixed Methods Research

Evaluating Discussion Sections

Evaluating Systematic Reviews and Meta-Analyses: Towards Evidence-Based Practice

Putting It All Together

Filters

Exam 6: Evaluating Samples When Researchers Generalize

Which one of the following scenarios illustrates a stratified random sampling procedure?

What is the reason that geography is especially useful for sample stratification purposes?

What is the biggest threat from self-selection bias, according to the text?

All of the following examples correctly reflect concerns with selection bias, EXCEPT:

When several subgroups are compared in a study, what is the rule of thumb about the sample or subsample size?

Which one of the following statements about sampling issues is INCORRECT?

There are several considerations important to evaluating if the sample size is adequate. Which one of the following considerations is IRRELEVANT?

Which one of the following statements about random sampling is INCORRECT?

There are several important rules about research ethics. Which one of the following rules is NOT among them?

There are several important reasons why researchers are using nonrandom samples in their studies. Which one of the following is NOT a valid reason?

All of the following conditions help researchers to achieve an unbiased sample, EXCEPT:

When some participants refuse to participate in a study, what participation rate is considered to be reasonably high?

When a sample is nonrandom, there are some important ways to improve the researchers' ability to generalize their study results. Which one of the following sampling strategies does NOT improve the nonrandom sampling process?

When comparing survey response rates for mail and online surveys, which one of the following statements is INCORRECT?

Which one of the following statements about sample size is correct?

Researchers are supposed to describe the demographics of the sample in their article. Which one of the following variables is unlikely to be included among the demographics?

All of the following considerations would be relevant for deciding whether the participation rate in a specific study is reasonably high, EXCEPT:

All of the following statements about the relationship between a sample and population are true, EXCEPT:

There are several good reasons why it is important for researchers to discuss the limitations of their sampling in a research report. Which one of the following is the LEAST important reason?

Background for Evaluating Research Reports

Evaluating Titles

Evaluating Abstracts

Evaluating Introductions and Literature Reviews

A Closer Look at Evaluating Literature Reviews

Evaluating Samples When Researchers Do Not Generalize

Evaluating Measures

Evaluating Experimental Procedures

Evaluating Analysis and Results Sections: Quantitative Research

Evaluating Analysis and Results Sections: Qualitative Research

Evaluating Analysis and Results Sections: Mixed Methods Research

Evaluating Discussion Sections

Evaluating Systematic Reviews and Meta-Analyses: Towards Evidence-Based Practice

Putting It All Together

Filters