Downloaded 89 times
More Related Content
Similar to Validity of a Research Tool
Validity of a Research Tool
- 1. VALIDITY OF ARESEARCH TOOL Meaning, Methods of establishing Validity, Factors influencing Validity and Research Tool: measures to increase the Validity of a Tool. By Joby Varghese
- 2. 1. Introduction 2. Understanding‘Validity’ 3. Definition 4. Characteristics of Validity 5. Nature of Validity 6. Types of Validity 6.1.Content Validity 6.2.Face Validity 6.3.Construct Validity 6.3.1. Convergent Validity 6.3.2. Discriminant Validity 6.4.Criterion-related Validity 6.4.1. Concurrent Validity 6.4.2. Predictive Validity 6.5.Consequential Validity 6.6.Known-group Validity 7. Factors influencing Validity of a Research Tool 8. Measures to increase Validity of a Tool 9. Conclusion Table of Contents
- 3. 1. Introduction Validity inresearch refers to how accurately a study answers the study question or the strength of the study conclusions. Here validity refers to how well the assessment tool actually measures the underlying outcome of interest. Validity is not a property of the tool itself, but rather of the interpretation or specific purpose of the assessment tool with particular settings and learners. Assessment instruments must be both reliable and valid for study results to be credible.
- 5. 2. Understanding ‘Validity’ “Validityrefers to how well an instrument measures what it is intended to measure. Validity is the extent to which a test measures what it claims to measure. It is vital for a test to be valid in order for the results to be accurately applied and interpreted. Validity isn’t determined by a single statistic, but by a body of research that demonstrates the relationship between the test and the behaviour it is intended to measure.
- 6. Ross defines Validityas follows: One kind of validity concerns the degree to which the test or other measuring instrument, measures what it claims to. In a word validity measures truthfulness. Cureton says, Validity is therefore defined in terms of the correlation between the actual test scores and true criterion scores. Gullikson defines it in a more particular form when he says The validity of a test is the correlation of the test with some criterion. 3. Definition
- 7. According to Freeman(2006) An index of validity shows the degree to which a test measures what it purports to measure when compared with accepted criteria. Guilford explains the meaning of validity in statistical terms as follows: ...What a test measures, in common with the other test and other measures, is in variance, then, is the basis for validity. Cook and Campbell (1979) defines validity Best available approximation to the truth or falsity of a given inference, proposition or conclusion. According to Blumberg et. al., (2005) Validity is often defined as the extent to which an instrument measures what it asserts to measure. Contd…
- 8. 4. Characteristics ofValidity 1. It is an important characteristic of a measuring instrument or a test. 2. It is a measure of constant error while reliability is the measure of variable error. 3. It is an index of external correlates. The test scores are correlated with external criterion scores. 4. The criterion may be a set of operations or success in the job or as a predictor for future course of test scores. 5. It relates the objective of a test score.
- 9. Contd… 6. It connotesthe psychological construct of a variable which is indirectly measured with the help of behaviours. 7. It ensures the reliability of a test. Thus, if a test is valid, it must be reliable. 8. It refers the truthfulness of a test score. 9. It is the function of a length. 10. It determines how an individual performs, in different situations.
- 10. 5. Nature ofValidity There are certain cautions that one has to keep in mind while using the term Validity in evaluation: 1. Validity is a matter of degree and hence does not exist on an all-or-none basis. 2. Validity refers to the results of a test or evaluation tool for a given group of individuals, not the tool itself. 3. Validity being a relative term, a tool would be valid for a particular situation. This means a particular tool is not valid in every situation.
- 11. Types of Validity 6.5. Consequential Validity 6.1.Content Validity 6.3.1. Convergent Validity 6.2. Face Validity 6.3. Construct Validity 6.4. Criterion- related Validity 6.6. Known-group Validity 6.3.2. Discriminant Validity 6.4.1. Concurrent Validity 6.4.2. Predictive Validity
- 12. 6.2.Face Validity • Itis the appearance of validity, ‘apparent validity’. • It refers to whether the tool looks like it measures what it is supposed to measure. • It is an estimate of whether a test ‘appears' to measure a certain criterion; it does not guarantee that the test actually measures phenomena in that domain.
- 13. Contd… For example, ifyou are trying to assess the face validity of a psychological ability measure, it would be more convincing if you sent the test to a carefully selected sample of experts on psychological ability testing and they all reported back with the judgment that your measure appears to be a good measure of psychology ability. advantage disadvantage
- 14. 6.1.Content Validity • Itmeasures the extent to which items on a tool are related in a straight forward way to the characteristics, the tool aims to measure. • It is the evidence or the degree to which the content of the tool matches a content domain associated with the construct. • It ascertains whether the tool contains items from the desired content domain.
- 15. Contd… For example, indeveloping a teacher aptitude test, experts in the field of education would identify the qualities, knowledge, attitude and skills required to be an effective teacher and then choose (or rate) items that represent those areas of qualities, knowledge, attitude and skills. Lawshe developed a formula termed the content validity ratio as follows: Where, CVR= Content validity ratio, Ne = Number of Subject Matter Experts (SME) indicating ‘essential’, N = Total number of SME experts.
- 16. 6.3.Construct Validity It refersto the totality of evidence about whether a particular operationalization of a construct adequately represents what is intended by theoretical account of the construct being measured. In other words, it attempts to ascertain whether the instrument measures the construct that it is intended to measure. They also include relationships between the test and measures of other constructs. 2 types…
- 17. 6.3.1. Convergent Validity Itrefers to the degree to which a measure is correlated with other measures that it is theoretically predicted to correlate with. It is the extent to which the scale correlates with measures of the same or related concepts. For example, a new scale to measure assertiveness should correlate with existing measures of assertiveness and with existing measures of related concepts like independence.
- 18. 6.3.2. Discriminant Validity Discriminantvalidity describes the degree to which the operationalization does not correlate with other operationalization that it theoretically should not be correlated with. It is the extent to which a tool relates to scores on another tool measuring theoretically-related dimension. For example, an assertiveness scale should not correlate with measures of motivation.
- 19. 6.4.Criterion-related Validity Criterion-related validityindicates the degree of the relationship between the predictor (the tool) and a criterion (level of performance the tool is trying to predict). It refers to the degree of consistency between test data and some other measure of the same trait taken either at the same time or (concurrent validity) or at some future time (predictive validity) and the accuracy with which the predictor is able to predict performance on the criterion.
- 20. 6.4.1. Concurrent Validity Itis reflected in the degree to which results from two different instruments employed to measure the same type of learning are in agreement. If the test data and criterion data are collected at the same time, this is referred to as concurrent validity evidence. A tool has concurrent validity when those who do well on one assessment also do well on the other and vice versa.
- 21. 6.4.2. Predictive Validity Itrefers to the degree to which operationalisation can predict or correlate with other measures of the same construct that are measured at some time in future. If the test data are collected first in order to predict criterion data collected at a later point in time, then this is referred to as predictive validity evidence.
- 22. 6.5.Consequential Validity It refersto the impact (positive or negative) and consequences (intended or unintended) that a particular type of assessment has for the related teaching and learning. If an assessment activity, besides gauging learners performance, also encourages further learning or better teaching, it has consequential validity.
- 23. 6.6.Known-group Validity It refersto the extent to which a tool distinguishes between two groups known to differ with respect to the characteristic under study. This is explained in terms of the locus of control. For example, successful candidates in examination or job applicants are likely to have an internal locus of control as compared to unsuccessful candidates who are likely to have an external locus of control.
- 25. 7.0. Factors influencingValidity of a Research Tool 1. Inappropriateness of test items: items that measure knowledge cannot measure skill. 2. Direction: unclear direction reduces validity. Direction that do not clearly indicate how the pupils should answer and record their answers affect validity of test items. 3. Reading vocabulary and sentence structures: too difficult and complicated vocabulary and sentence structure will not measure what it intends to measure. 4. Level of difficulty of Items: too difficult or too easy test items cannot discriminate between bright and slow pupils and will therefore lower its validity.
- 26. Contd… 5. Poorly constructedtest item: test items that provide clues and items that are ambiguous confuse the students and will not reveal a true measure. 6. Length of the test: a test should of sufficient length to measure what it is supposed to measure. A test that is too short cannot adequately sample the performance we want to measure. 7. Arrangement of items: test item should be arranged according to difficulty level, with the easiest items to the difficult ones. Difficult items when encountered earlier may cause mental block and may also cause student to take much time in that number.
- 27. 8. Measures toincrease Validity of a Tool 1. Make sure your goals and objectives are clearly defined and put in operation. Expectations should be written down. 2. Match your assessment measure to your goals and objectives. Additionally, have the test reviewed. 3. Get respondents involved; have the students look over the assessment for troublesome, wording, or other difficulties.
- 28. Contd… 4. If possible,compare your measure with other measures, or data that may be available. 5. If no assessment instruments are available, use content experts to create your own and pilot the instrument prior to using it in your study. Test the reliability and include as many sources of validity evidence as are possible in your paper. Discuss the limitations of this approach openly.
- 29. 9. Conclusion In general,validity is an indication of how sound your research is. More specifically, validity applies to both the design and the methods of your research. Validity in data collection means that your findings truly represent the phenomenon you are claiming to measure. Valid claims are solid claims. Validity is one of the main concerns with research.