Samples, Power Analysis, and Design Sensitivity
1. Compare and contrast internal and external validity
Based on the understanding of the research process and the interest for advisory program I would acknowledge an in-depth critical construct of strategies to develop and establish advisory program in middle school to study behavior in a natural environment. Despite the many close connection, the studies would place a different amount of emphasis on the various variables to identify patterns, features and themes through test, observation and surveys. The structure form of quantitative data components from the advisory program provisions will support the characteristics of validity and reliability. The components relevance and reliability of validity would be measured logically and statically per McClure, (2010). The paper details (1) compare and contrast internal and external validity, (2) identify threats to external and internal validity, (3) questions for which internal and external validity are primary concerns, (4) Compare and contrast random selection and random assignment; when to use one or the other to avoid possible consequences for failure to do random selection or random assignment, (5) relationship between sample sizes with all else being equal, as sample size increases the likelihood of finding a statistically significant relationship increases, (6) Compare and contrast probability and non-probability sampling with the advantages and disadvantages, and (7) how validity issues could impact the intended study for advisory program implementation in organization.
The advisory program research study will engage in the effectiveness of school to fulfill the four basic human needs for students; to belong, to gain power, to be free, and to interact with positive behavior in (7-8 grade) middle school. Hence, the pre-experimental design will take on a single one-shot case study. The instruments used to test the competence of the action research will be student NWEA test scores (Reading/Math), observation, surveys, and in addition to researcher administered quizzes and test that relate to standardized questioning similar to the NWEA exams.
External (or ecological validity) reported by Cozby (2012) is concerned with the extent that the effects simplify to the real world to the extent, which the study can present results generalize to other samples, situations, population, setting, treatment, measurement variables and tasks or simply put in more pedestrian terms, external validity is the degree to which the conclusions in your study would hold for other persons in other places and at other times. The threat is valid because to identify the population you want to generalize to and then draw a sample from that population because personal issues may affect the performance. Although generalizable conditions are a valid threat the action research for advisory program could be replicated, the results will vary because students change which will affect the outcome. Selection treatment interaction is also a valid threat to the advisory program research. The students' participants are chosen, rather than selected randomly. Therefore, such a nonrandom selection of the participants may limit the generalizability of the study.
(2) Identify threats to external and internal validity
Factors concerned with external validity are threatened whenever conditions inherent in the research design are such that the generalizability of the results is limited. Spaulding, (1963) a threat to external validity is an explanation of how you might be wrong in making a generalization. For instance, a threat indicates if whether you can generalize the results to a wider population. Four classes of threats to external validity:
• Reactive arrangements or Participants effects
• Selection-treatment interaction
• Interaction effect of selection bias and the intervention
• Generalizable conditions
• Experimenter effects
Ways to overcome external validity are to use a random sample, replicate the study with different people at different time in different places and keep maturation rates low.
Factors concerned with internal validity in agreement with Trochin & Donnelly (2008) are about the causal-effect relationship in a study the differences the dependent variable (measure) due to the operation of the independent variable and not due to an extraneous (confounding) variable. Therefore, history is a valid threat to the action research because there are unexpected factors and events such as weather changes and student misbehavior, as well as classroom distractions that could influence the responses of the students as they work on the surveys and tests. Also, instrumentation due to the consent forms, and student surveys, as well as administered test will be used for the action research study. The threat involves student participants may not feel comfortable revealing their true or honest opinions to the questions found in the survey. The observed changes are the results of an experiment being attributed to the program or the results influenced by alternative causes to the operation of the independent variable. It asks whether the obtained differences between the experimental condition and groups are really the result of the experimental manipulations:
• Reliability of measuring instruments/stimulus delivery- Instruments
• Inconsistent treatment administration
• Single groups
• Selective subject loss- Maturation
• Experimenter bias
• Social interactions
• Nonrandom assignment
• Contamination of the subject pool
• Subject role-related behaviors
• Effects of previous testing (carrying over effects)- Testing
(3) Questions for which internal and external validity are primary concerns
Interpersonal study undertakes (by measuring or observing) each of the variables you are trying to relate. When a study is intended to look at the associations amid two or more variables. A public opinion poll that likens what percentage of males and females say they would vote for a Democratic or a Republican candidate in the next presidential election is basically examining the connection among gender and voting preference. Causal study adopts that you can define both the cause and effect variables and that you can show that they are linked to each other. When a study is intended to regulate whether one or more variables (e.g., a program or treatment variable) causes or affects one or more outcome variables. Example being if there was a public opinion poll to try to determine whether a political marketing operation altered voter partialities, we would fundamentally be studying whether the operation (cause) altered the percentage of supporters who would elect Democratic or Republican (effect).
(4) Compare and contrast random selection and random assignment when to use one or the other to avoid possible consequences for failure to do random selection or random assignment.
According to Fritz & Morgan, (2010) “Random selection is how you draw the sample of people for your study from a population and random assignment is how you assign the sample that you draw to different groups or treatments in your study.” It is possible to have both random selection and assignment in a study. Random selection is associated to sampling. So it is most linked to the external validity of the outcomes. Subsequently, we would randomly sample so that the research participants embody the larger group better from which they're drawn. Random assignment is most related to design. In fact, when we randomly assign participants to treatments we have, by definition, an experimental design. So, random assignment is associated to internal validity. Subsequently, preceding treatment random assignment assists to make sure that the treatment groups are equivalent to each other. Example being, if a researcher draws 100 participants from a group of 1000 applicants with the help of random sample, thenceforth it would be random sampling. But, if the researcher chooses 100 participants for different treatment in its place of the unresolved 900 people from entire population this would stand as random assignment.
(5) Relationship between sample sizes with all else being equal, as sample size increases the likelihood of finding a statistically significant relationship increases.
Acheson, (2010) suggest sample size as the process of selecting units (e.g., people, organizations) from a population of interest so that by studying the sample we may fairly generalize our results back to the population from which they were chosen. The situation could be specified as the sample size provides the radical influences on the results of the study. Given that if there were a small sample size the situation could affect the results of the study substantially. However, an error could be reduced if the research takes a large sample group (StatPac Inc., 2013), It is known that as sample size increases the possibility of finding a statistically important association rises, because the large sample size characterize the population.
(6) Compare and contrast probability and non-probability sampling with the advantages and disadvantages.
A nonprobability and probability sampling technique are different in that a probability sampling technique is any technique used for the sampling that employs a certain procedure for random selection. In order to have random selection procedures have to be set up to assure that the different components in the population are equivalent in the likelihoods of being chosen. Probability sample is used to handpick and make selections with the support of the random process (Hackley, 2003). Sampling method safeguards that the comprised sample is not alike and is completely random. The main benefit of this sampling method is its fairness, due to each chosen participant is given like chances before the gathering of the sample this enhances the validity of the study results. Disadvantages suggest by Hackley, (2003) totally depend over the selected people, and if they could cheat or the research could face the situation related to possibility of flaws that could also affect the fairness of this sampling model (Hackley, 2003).
A nonprobability sample is different, since its participants are systematically selected from small like group population. This sampling method is more effective because this aids the researcher to target the specific group of people (Hackley, 2003). Thus, a disadvantage with this method is its bias character, due to the participants being selected from the small like group populations making its view not representational of the whole populations.
(7) How the aforementioned issues could impact the intended study implementation in organization
In summation, validity issues that could impact the envisioned research for advisory program in middle school would be restricted generalizability across constructs. The concerns aforementioned are threats to external and internal validity, as it encumbers the possibility to replicate the study with different people at different time in different middle schools. With the inconsistent allotted time, days, and treatment the advisory programs could fall under a massive umbrella in how the programs are structured to run. The allotted time frame for which the students receive advisory, as well as to how time is allocated and the number of days advisory is provided in the action research study. The Turning Points, Carnegie Council on Adolescent Development (1989), study noted that schools have been trying to find a consistent method to advisory programs.
...(download the rest of the essay above)