Self-report data may be obtained from a test or an interview format of a self-report study. The format of self-report study that will be used to discuss limitations of self-report data will be a test and a personality disorder test will be used as an example. For specific example answers for the test I completed the results all rated “low” for all personality disorders. Limitations arise from decreased reliability and validity and issues with credibility of responses due to response bias. Content validity, construct validity and criterion-related validity as well as test-retest reliability will be presented. The forms of response biases that will be discussed are social desirability, acquiescence, halo effect, extreme responding, midpoint responding, random responding, negative/positive bias, memory recall bias and cultural bias. The discussion will conclude with the role self-report data plays in psychology today.
Limitations of Self-Report Data
Self-report data has the advantage that they come directly from the test taker themselves and not from a third party; who would know someone better than themselves? Self-report studies can examine a large number of variables such as feelings, behaviours, attitudes and beliefs. They are easy to administer, inexpensive and most take a modest amount of time to complete (McDonald, 2008). Self-report tests are able to assess matters that are not observable such as feelings, beliefs and opinions. The scoring of self-report tests is standardized whether it be a scoring template for pen and pencil formats or computer-based interpretations for tests whose answers are entered into a data file (Hood & Johnson, 2007). However, it does have some limitations. The validity of causal conclusions drawn from self-report data may be of concern because it can be influenced by factors such as the psychometric properties (validity and reliability) of the test, the context (environment/situation) of the test, and response biases (Dodd-McCue & Tartaglia, 2010). .
Content validity is the extent to which a test is measuring the domain it is supposed to be measuring (Hood & Johnson). Meaning for example; does the 4degreez.com Personality Disorder Test measure for personality disorders? Computers allow for much more complex self-report test scoring, and are accurate and thorough. A test’s construction can also play a key role. The content validity of some tests relies on the appropriate qualifications and level of expertise of the designers of the test and of the database used for computer-based test interpretations (Hood & Johnson). For the 4degreez.com Personality Disorder Test the designers decide what questions to include in order to identify the different features of personality disorders. The creators of the database decide which answers correlate with a personality disorder. For self-report testing the fundamental way a question is worded can influence a person’s interpretation and thus their response. In the 4degreez.com Personality Disorder Test this may lead to the erroneous detection of the presence or no presence of diagnostic criteria. For example in the: Q1 “Do you believe you have more difficulty with relationships than the average person your age?” (4degreez.com, n.d., p. 1). I didn’t know how to interpret or answer this question; do they mean social relationships or intimate relationships? Had I understood it as both and answered “yes” my answer would have been measured as a diagnostic criteria for Schizoid Personality Disorder. When I completed the test I answered “no” because neither situation applied for myself. There was no information given in regards to the designers of the 4degreez.com Personality Disorder Test nor for the database used for the computer-based test interpretations used to score the test; therefore it is unknown on how it faired for content validity. In...