Presentation on theme: "Comprehensive Exam Review Click the LEFT mouse key ONCE to continue."— Presentation transcript:
Comprehensive Exam Review Click the LEFT mouse key ONCE to continue
Research and Program Evaluation Part 3 Click the LEFT mouse key ONCE to continue
Ethical Aspects of Research
General ethical principles include: Nonmaleficence - do no harm Beneficence - do as much good as possible Justice - equality for all Fidelity - fulfill all obligations
Ethics Related to Scholarly Work The information derived from a research study should promote the welfare of members of society (Beneficence). The principal researcher has responsibility for all aspects of execution of the study, including the behaviors of all research participants (Beneficence and Fidelity).
The results of research must be reported accurately, honestly, and fairly (Beneficence and Nonmaleficence). Credit for all contributions to the research must be given accurately and adequately (Justice). Acknowledgment must be given to original contributions and/or scholarly insights of others and distinguished clearly from those of the author(s)/researcher(s); plagiarism is always unethical (Justice).
Ethics Related to Subjects Researchers must strive to minimize harm and/or risk to subjects (Nonmaleficence). Researchers must strive to maintain or improve subjects’ welfare and dignity (Beneficence). Researchers must use informed consent procedures that are to the subjects’ benefit (Beneficence and Fidelity).
Researchers must respect subjects’ privacy by maintaining confidentiality and/or anonymity (Fidelity). Researchers must fulfill all treatment obligations and responsibilities offered to subjects (Fidelity). Researchers must be extremely careful to protect subjects’ welfare and dignity in the conduct of experiments involving deception (Nonmaleficence and Fidelity).
Researchers must provide subject debriefing, including dehoaxing or desensitization, for any research involving deception (Beneficence and Fidelity). Researchers must avoid use of any form of “pressure” to coerce subject participation (Nonmaleficence). Researchers are obligated to attempt to countermand any negative consequence(s) of participation in the research (Beneficence and Fidelity).
Research Questions and Hypotheses
The decision of whether to use a research question, null hypothesis, or directional hypothesis is made on the basis of what is known about the topic being studied. Research questions are used when relatively little is known about the topic (i.e., when there is very little basis for making a conjecture about what the results might be).
Research questions typically are in the form: “What [is] / [are] the [differences]/ [relationships] [between] / [among]...?” (The “is” or “are” and “between” or “among” decisions are made simply to achieve correct syntax, i.e., to achieve correct grammatical structure.)
The decision of whether to use “relationships” or “differences” is made based on the types of variables involved. In general, variables studied in research can be classified as either “discrete” or “continuous.” In general, a “discrete” variable may be thought of as having categories, whereas a “continuous” variable may be thought of as having “scores” (i.e., a full range of values).
Typical examples of discrete variables include gender, race/ethnicity, marital status, grade or class level, state of residence, or diagnostic classification. Typical examples of continuous variables include age, years of work experience, annual income, scores on a test, or frequency of a particular behavior.
Use the word “difference” in a research question or an hypothesis if at least one of the variables involved is discrete. Use the word “relationship” in a research question or an hypothesis if all the variables involved are continuous.
Examples of research questions include: What is the relationship between levels of assertiveness and stress among adult, working women? What are the differences among married and divorced males’ and females’ attitudes toward divorce? What is the difference in college seniors’ GRE scores following participation in a test preparation workshop?
The general form of a directional hypothesis is a declarative sentence. An effective directional hypothesis should specify both the nature and direction of the relationship or the difference.
Examples of a directional hypothesis include: Males engage in abusive verbal behavior more frequently than females. The “test breaker” activity is effective in reducing students’ test anxiety. There is a positive relationship between frequencies of use of alcoholic beverages and cigarettes.
In order for a general statement to be “true,” it must hold as specified for all occasions. If any occasion is an exception to the general statement, then the statement is “not true” (i.e., “one exception disproves the rule”). Therefore, hypotheses are often stated in “null form” because researchers study only one occasion at a time and thus cannot prove but can only disprove a “null” statement.
There are two commonly used forms of null hypotheses: the “traditional” form and the “modern” form. An example of a traditional null hypothesis is: There will be no significant relationship between counselor trainees’ personality needs, as measured by the Personality Research Form, and their counseling effectiveness, as measured by the Counselor Evaluation Rating Scale.
The “modern form” is written in the present tense, does not use the word significant, and does not indicate the measurement tools. The “modern” form of the previous hypothesis is: There is no relationship between counselor trainees’ personality needs and their counseling effectiveness.
A study of differences in academic performance on the basis of gender and residence classification would have two major hypotheses: H1:There is no difference in academic performance based on gender. H2:There is no difference in academic performance based on residence classification.
But there are the combinations that need to be considered. Residence Urban Rural Gender Male Female Academic Performance
Therefore, subhypotheses need to be added to the primary hypothesis. H1: There is no difference in academic performance based on gender. H1a:There is no difference in academic performance among males based on residence classification. H1b:There is no difference in academic performance among females based on residence classification.
There is also the possibility that the two variables of gender and residence classification may somehow “interact” with one another to yield a unique result. Therefore, in such situations, an “interaction hypothesis” also should be presented: H1c:There is no gender by residence classification interaction for academic performance.
The evaluation response to a research question is a declarative statement that provides a direct answer to the question posed. There are only two possible evaluation responses for a null hypothesis: “reject” or “fail to reject.” A null hypothesis is never “accepted,” because one study cannot “prove” the truth of the (null) hypothesis.
There are two types of errors that can be made in evaluating hypotheses: A Type I Error occurs when the researcher rejects the null hypothesis when it is in fact true. A Type II Error occurs when the null hypothesis is not rejected when it should have been rejected.
Subjects are the people who participate in the research. Sampling is the procedure used to identify and enlist the subjects. A sample is obtained from a population, which is the group of people to whom the results of the study are to be applied.
Good sampling starts with effective description of the population. Description of the population includes con- sideration of: (a) demographic characteristics and (b) psychosocial characteristics. Demography is the statistical study of pop- ulations (i.e., identifiable groups of people), and demographic information comes typically from statistical summaries (aka statistical abstracts).
Psychosocial characteristics are those attributes, behaviors, and characteristics typically associated with an identifiable group of people. Psychosocial characteristic information comes from the professional literature. Generally, effective sampling works as follows.
First, the population for the study is identified, usually using colloquial descriptors. Next, the primary (or major) demographic characteristics of interest are selected. Third, demographic information about the population is found from some statistical resource. Next, the psychosocial characteristics of the population are identified from the professional literature.
Next, sampling is done based on demographic characteristics. Finally, it is assumed that the psychosocial characteristics of the sample equal those of the population. It should be remembered that representativeness is the key criterion for the effectiveness of sampling, not the sampling procedure used!
There are two categories of approaches to sampling: probability and nonprobability. Probability sampling means that subjects are drawn from the population in such a way that the probability of selecting each member of the population is known. Nonprobability sampling means that subjects are drawn from the population in a logical way such that representativeness can be assumed reasonably.
Probability sampling methods include: Random Sampling, in which each person has an equal likelihood of being selected and the selection of one does not affect the selection of another. Systematic Sampling, in which every n th person from a list of all persons in the population is selected.
Stratified Sampling, in which each person who is a member of the “stratum” (i.e., category) is eligible to be selected and selection is made through a first encounter process. Stratified Random Sampling, in which each member of a stratum has an equal likelihood of being selected, selection of one within the stratum does not affect the selection of another, and the number of subjects drawn within strata may be either proportional or disproportional to the population.
Cluster Sampling, in which naturally occurring groups or units are the population (e.g., classes or neighborhoods), clusters of groups or units are randomly selected from the population of groups or units, and individuals are randomly selected from the clusters selected. In all probability methods, representativeness is assumed to be achieved if the laws of probability are effectively operationalized.
Nonprobability sampling methods include: Convenience Sampling, in which an intact group of people is used as the subjects based simply on ease of access to them. Purposeful Sampling (sometimes called judg- mental sampling), in which specific persons are selected from the population because they are “judged” to be representative and/or informative about the topic being studied.
Quota Sampling, which is used when a probability sampling method cannot be used, but subjects are able to be selected to represent identified characteristics of the population. Quota sampling procedures typically involve use of a “sampling framework,” which is a diagram of the proportionate relationships among the variables upon which the sampling is based.
There are two basic ways to determine the minimum sample size needed. If a quota (or sampling frame) approach is used, effective fulfillment of the various cell entries is the minimum total needed to represent the population effectively. If a probability sampling approach is used, “statistical power” (which is the probability that a given statistical technique will result in rejection of a false null hypothesis) should be computed.
The level of significance of a statistical test is closely related to sample size. For a given level of statistical significance, a statistical power formula provides an estimate of the minimum sample size needed for that significance level. For example, fewer subjects are needed to reject the null hypothesis at the.05 level than at the.01 level. Statistical power is always enhanced by large sample size.
Other factors to consider in sampling: Type of research: correlational research requires larger samples than does experimental research. Research Hypotheses: expectation of smaller differences requires larger samples. Cost: what is the largest possible sample size that can be achieved with available re- sources?
Importance of Results: the greater the implications, the larger the sample should be. Number of Variables: the greater the number of variables, the larger the sample needed. Data Collection Methods: larger samples are needed if there is considerable measurement error.
Accuracy Needed: the greater the accuracy needed, the larger the sample needed. Size of Population: the larger the population, the smaller the percentage needed to represent it.
This concludes Part 3 of the presentation on RESEARCH AND PROGRAM DEVELOPMENT