Ch. 1 and Ch. 2 of Operations Management: •Problem 1-1 •Problem 1-2 •Problem 2-1 1 1. Tried and True Clothing has opened four new stores in college towns across the state. Data on monthly sales volume and labor hours are given below. Which store location has the highest labor productivity? Store Annandale Blacksburg Charlottesville Danville Sales volume $40,000 $12,000 $60,000 $25,000 Labor hours 250 60 500 200 1 2. Tried and True’s accountant (from Problem 1-1) suggests that monthly rent and hourly wage rate also be factored into the productivity calculations. Annandale pays the highest average wage at $6.75 an hour. Blacksburg pays $6.50 an hour, Charlottesville $6, and Danville $5.50. The cost to rent store space is $1800 a month in Annandale, $2000 a month in Blacksburg, $1200 a month in Charlottesville, and $800 a month in Danville. 1. Which store is most productive? 2. Tried and True is not sure it can keep all four stores open. Based on multifactor productivity, which store would you close? What other factors should be considered? 2 1. Backwoods American, Inc., produces expensive water-repellent, down-lined parkas. The company implemented a total quality-management program in 2005. Following are quality-related accounting data that have been accumulated for the five-year period after the program’s start. Year 2006 2007 2008 2009 2010 Quality Costs (000s) Prevention $3.2 10.7 28.3 42.6 50.0 Appraisal 26.3 29.2 30.6 24.1 19.6 Internal failure 39.1 51.3 48.4 35.9 32.1 External failure 118.6 110.5 105.2 91.3 65.2 Accounting Measures (000s) Sales $2,700.6 2,690.1 2,705.3 2,310.2 2,880.7 Manufacturing cost 420.9 423.4 424.7 436.1 435.5 1. Compute the company’s total failure costs as a percentage of total quality costs for each of the five years. Does there appear to be a trend to this result? If so, speculate on what might have caused the trend. 2. Compute prevention costs and appraisal costs, each as a percentage of total costs, during each of the five years. Speculate on what the company’s quality strategy appears to be. 3. Compute quality-sales indices and quality-cost indices for each of the five years. Is it possible to assess the effectiveness of the company’s quality-management program from these index values? 4. List several examples of each quality-related cost—that is, prevention, appraisal, and internal and external failure—that might result from the production of parkas.
Category: Uncategorized
Suppose you are a director of a ballet theater.
Suppose you are a director of a ballet theater.
Suppose you are a director of a ballet theater. After reading a book about the cell cycle, you are intrigued by the similarity between your night show and the cell cycle. Each night the theater fills, the lights go out, the curtain rises, the dancers perform, the curtain falls, the lights go on, and the spectators leave, just like the cell cycle consisting of a defined sequence of events. So you decided to improve theater operation by incorporating checkpoint mechanisms similar to the cell cycle control system. First, you designed a system that raises the curtains automatically when all the seats are filled. In each seat, sensors were installed to detect a seated person. But you have not decided how to connect the sensors to the curtains. a) You can set the sensors to send out a positive signal that adds up to be enough to raise the curtain or set each sensor to send out a negative signal that will stop the curtain from going up until every seat is filled. Which system do you think a cell would use for cell cycle control? b) Compare advantages and disadvantages of the two modes and explain your answer. (Your shows are so popular that all the seats are filled in each show. You don’t have to worry about a block in the system due to empty seats.)
The effect of amylase on enzyme functioning.
The effect of amylase on enzyme functioning.
I have to write a lab report on the effect of amylase on enzyme functioning. I already started by writing the introduction, methods, and results/ figures. However, I am sure there are many errors in my work that need to be fixed and or worked on. If there is any chance you can edit my work that would be great. I also have to add an abstract and discussion section as well. The rubric for all the sections are as follows.... Title: Scientific titles should be very concise descriptions of the work. Generally titles are less than 20 words long but should be informative enough so that a reader could glance at your title and know what the paper will be about. Introduction Text: You can think of your introduction as serving three purposes Get the reader interested in what you studied- What did you study and why is it interesting? Does it have implications for human health, basic scientific understanding, technologies, the environment, etc? You need to explain to your reader why the topic is important/interesting using scientific information to back up your statements. You should also describe scientific knowledge/studies that your project directly builds off of. Be sure to give a clear description of the exact question/hypothesis you studied so the main point of your paper is clear to the reader. Briefly describe how you studied your question- The complete description of your experiments will be included in the results and methods sections of your paper but you should give the reader a vague sense for your approach to the study in the introduction. This gives the reader a general sense for the study that they can use to contextualize the detailed experimental descriptions in results/methods as they continue reading the paper. Touch on your findings and why they’re exciting- The real interpretation of your results will go in the discussion section of your paper but you do want to briefly summarize your most important finding/s in the introduction. This will likely be quite short (perhaps even just a single sentence) but serves to get the reader excited to closely read the rest of your paper. References: You must include at least 4 total references with at least 2 being primary sources (ie a peer-reviewed research article). Lab lectures and the lab manual can not be used as references – find a primary source (from a peer-reviewed scientific article) for that information. Pubmed, Google Scholar and Web Of Science are all good search engines you can use to find primary references. You need to include citations within the text in the format of (author/s, year) as well as a References section at the end of your introduction. Examples of citations within the introduction text could be (Smith & Jones, 2008) or (Smith et. al. 2009) if there are more than 2 authors. The reference section should be ordered alphabetically in the following general format but please refer to ‘A Short Guide To Writing About Biology’ page 69-79 or your instructor for more help. Smith, MC, Jones, KF, Knowles, BG and Carter, SC (2013) How to name a child. J Childhood Dev 131:435-42. Writing Style: Scientific papers are generally written in the past tense – you may use the past tense in the introduction even when describing the experiments you will do next week (this will save you from having to change the tense later when you turn in your final report). Make sure to check your writing for grammar and for a nice, logical flow from one topic to the next. Be as concise as possible while still including all information relevant to understanding your study. Abstract: Abstracts provide an overall summary of the paper so that readers can read a short abstract and decide whether they will read the entire paper or not. Thus, abstracts must catch the reader’s attention by explaining the importance/relevance of the study, the general experimental approach and the most notable finding(s). Your abstract is limited to 250 words or less. Write the methods section for your amylase lab report. Methods sections in scientific papers serve two key functions. First, they allow the reader to have a detailed understanding of how you performed the experiments, which for knowledgeable readers, can affect how they perceive the work (ie. the experiments were performed in an appropriate manner, statistical analyses were used appropriately, and the technical manner of the experiments can support the authors’ conclusions). Second, the methods section allows readers to reproduce the experiments (or modified versions of them) in order to build off of the results to further science. Methods sections are written using full sentences in passive voice and can not be bulleted lists of experimental procedures. The key to writing a good methods section is to provide a clear description of experimental details while being as concise as possible. Thus, methods sections often are broken into sections for various assays so that readers can quickly find the relevant section they wish to read (readers generally do not read the methods section unless they are looking for particular information). If you performed an assay several times with slight variation each time, a methods section would include a single assay description and then describe how the methods varied for each experiment. Similarly, you can assume your reader has a basic understanding of general laboratory equipment and techniques (ie. you do not need to explain how to use a micropipette or include the brand and model of centrifuge used). Results Text: In describing your results you will also necessarily have to describe the experimental set-up. This is different from your experimental descriptions in the methods section. Descriptions of experiments within the results section should give enough information for the reader to understand the general experiment and approach (such as enzyme activity was measured using measure of color change of Lugol’s solution) but should not include all the minute details of the experiment needed to replicate it (readers can refer to the methods section if they are interested in more detail). Results sections often include a statement on the reasoning behind performing an assay (i.e. in order to determine if disulfide bonds are important to amylase activity, we measured amylase-catalyzed starch breakdown in the presence of increasing 2-mercaptoethanol). Note that results sections are generally written in the past tense. Your results section should also refer to your figures (for example - amylase activity was reduced by addition of 2-mercaptoethanol (Figure 1)). This allows you to concisely describe your results without having to list specific numbers in the text (although results often do point out any numbers that are particularly striking or important to note). Figures: You must include at least one figure in your report. Carefully consider the type of figure you choose (bar graph, scatterplot, etc.) relying on the graphing prelab exercise and in-lab discussion on making graphs. Replicate data sets should not graphed separately unless there is a particular need to do so (if one set gave poor data due to an error, for example). Replicate data points should instead be graphed as an average point with error bars indicating standard deviation. Scientific figures include figure legends that should be stand-alone text. Thus, readers should be able to look at the figures, read their legends and understand what the figure is displaying without referring back to the text (in fact, many scientists read papers by reading the figures and only going back to the main text if they are interested). The figure title should describe the main purpose of the figure (ie. Figure 1. Amylase activity is reduced by addition of 2-mercaptoethanol) and is included in the legend and not on the figure itself. Your figures and legends should not be imbedded within the text but can instead be included as additional pages after the text (one figure per page). In your final report you should include the figure page(s) after your references page (this is how scientific articles are submitted and then the publisher embeds the figures within the text). Discussion: Discussion sections are where you finally get to interpret your results for the reader and explain what they might mean in the broad scheme of the topic at hand. You should relate your experimental results to what you know about the biological processes being studied and to the greater importance of the findings. Discussion sections should begin with a short summary of your experiment and question/hypothesis in order to then lay out an argument for how your results speak to that question/hypothesis. The following questions may help guide you in writing your discussion section: Do the results support or refute your hypothesis? Do the results affect how we view the original question? What are some potential problems or limitations with the experiment? Is there primary literature that supports (or contradicts) your results? What is the significance of the results? Are there medical/ecological/biotechnological/etc. implications? Are there lingering questions remaining or new questions that arose? What further experiments would be worthwhile to do to address these?
Deakin Innovative Solutions Pty Ltd
Deakin Innovative Solutions Pty Ltd
Imagine you are employed as a business analyst at Deakin Innovative Solutions Pty Ltd, a business consulting firm. You are assigned to investigate and develop an Internet business solution for your client, Barwon TaiChi. You are required to deliver a proposed business plan to the business owners http://www.betterlifetaichi.com.au/ 1. Who commissioned the project and why? 2. Analyse the company profile and describe their business goal(s). 3. Identify and state the project objective and scope. 4a. Select and evaluate two (2) competitor Web site ?http://www.yulong.com.au/website/home.html http://www.taichiaustralia.com.au/ ? b. Perform a SWOT(strengths, weaknesses, opportunities, and threats) analysis
If you took all of the nuclear DNA in your body and lined it up end to end, would it be long enough to reach the moon?
If you took all of the nuclear DNA in your body and lined it up end to end, would it be long enough to reach the moon? If so, how many times would it reach the moon and back? (Assume you have 50 trillion cells.) Is it the same for males and females? Calculate how many times your DNA would stretch to the moon and back if you also calculated in the polyploidy that occurs in some cells as well as your mitochondrial DNA. Is this different for males and females?
I need help with an essay with Ovid book 5, can…
I need help with an essay with Ovid book 5, can…
I need help with an essay with Ovid book 5, can... I need help with an essay with Ovid book 5, can anyone help? Additional Requirements Min Pages: 4 Other Requirements: In essay form make an argument from Chapter 5 Ovid that presents a specific flashing insight that directly encodes and promotes some specific aspect of social cohesion, the essay must be structured as follows: two distinct major ideas, four secondary ideas on the following ratio: 6 tertiary ideas from book 5 ovid and 2 from lecture (the relation of the sexes, the golden age, silver)
Volcanoes are generally not preserved in the geologic rock record as they are usually eroded away.
Volcanoes are generally not preserved in the geologic rock record as they are usually eroded away.
1 Volcanoes are generally not preserved in the geologic rock record as they are usually eroded away. However, the various materials erupted from volcanoes are often found preserved in the rock record. From what you have learned about the different types of volcanoes, how could you infer what type of volcano erupted in a given area based on the type of volcanic deposits now found as layers of rock? Give specific examples, and briefly discuss how some materials may be linked to different types of volcanoes. need to APA format 200 words. 2. Discuss volcanic hazards. Although commonly thought to be associated with most volcanic eruptions (by the general public), lava is rarely responsible for the loss of life. Which hazards would tend to be more deadly? Finally, indicate and discuss those hazards that are a direct result of a volcanic eruption as well as those hazards that are indirectly linked. (Hint: think of “indirectly associated” hazards as “side effects. need to be 200 words APA formatt
Part III—”Drug Treatment and Drug Resistance: A Double Edged Sword”
Part III—”Drug Treatment and Drug Resistance: A Double Edged Sword”
art III—"Drug Treatment and Drug Resistance: A Double Edged Sword" The good news is that HAART has successfully treated HIV-positive patients, giving them extended lifespans, allowing the immune system to regain control, and decreasing virus loads. The bad news is that HIV is a smart virus; HIV has developed mechanisms to overcome the blocks imposed by drug treatments. Through a microevolutionary process, HIV variants that are resistant to the intended inhibition are able to replicate and produce more virus. Over time, in the presence of a drug such as AZT, random mutations may develop and be selected for that allow these variants to be resistant to the effects of AZT. These drug resistant variants have greater fitness and can replicate in a person even in the presence of a drug. Specifically, mutations in reverse transcriptase (RT) develop in the presence of AZT. These variants allow HIV to continue to replicate in human T cells. HIV variants may only contain a single amino acid change in the protein sequence to render an anti-HIV drug ineffective. For many years, doctors emphasized the necessity for strict compliance with prescribed HAART treatment regimens. Recently, a controversial new therapy called structured treatment interruption (STI) has been suggested to HIV-positive patients by their physicians. STI refers to the practice of alternating time spent on antiretroviral drugs with time spent off drugs, and each cycle may last anywhere from several days to months. However, it is not recommended for all HIV-positive patients. The theory behind STI is that the alternating periods of HAART treatment with regulated withdrawals of drug therapy may serve as a means of inducing immune system control of HIV. Questions: 1. How are natural selection and microevolution illustrated in this scenario? Identify the genetic basis for HIV evolutionary change, the selective pressure, and the resulting adaptation. What happens to mutants that are able to escape detection by the immune system? 2. Predict what will happen to infection rates over time if drug resistant forms of HIV are allowed to spread through the population. What conditions might slow the spread of drug resistant strains?
HCC Eagle Online BMGT 1327 Chapter 4 Exam Spring 2015
HCC Eagle Online BMGT 1327 Chapter 4 Exam Spring 2015
(Spring 2015 BMGT 1327 CRN 40722) Assignment: Chapter 4 Exam 1. award: 3 out of 3.00 points Milton Friedman contended that a business cannot increase its profits while conforming to society's laws and ethical customs. True False 2. award: 3 out of 3.00 points Ecocentric management centers around practices that maximize current economic growth, irrespective of the potential harm to future generations. True False 3. award: 3 out of 3.00 points Which of the following goals of a transcendent education is described as speaking out against unethical actions? Empathy Generativity Mutuality Civil aspiration Intolerance of ineffective humanity 4. award: 3 out of 3.00 points According to Kohlberg's model of cognitive moral development, people in the _____ stage see beyond authority, laws, and norms and follow their self-chosen ethical principles. universal conventional principled preconventional postconventional 5. award: 3 out of 3.00 points In the context of ethical systems, which of the following is true of virtue ethics? It suggests that what is moral must come from what a mature person with good "moral character" would deem right. It bases ethical behavior on the opinions and behaviors of relevant other people. It states that all people should uphold certain values that society needs to function. It states that the greatest good for the greatest number should be the overriding concern of decision makers. It holds that individual self-interest is the actual motive of all conscious action. 6. award: 3 out of 3.00 points In the context of corporate social responsibility, supporting community projects and making charitable contributions are examples of _____. legal responsibility economic responsibility ethical responsibility political responsibility philanthropic responsibility 7. award: 3 out of 3.00 points All About U (AAU) Salon believes in and engages in meeting societal expectations whether or not written as law, and voluntarily uses earth-friendly products. AAU can be described as operating at which level of the pyramid of corporate social responsibility? Legal responsibility Philanthropic responsibility Ethical responsibility Economic responsibility Administrative responsibility 8. award: 3 out of 3.00 points Which of the following may prevent the creation of a business climate conducive to unethical behavior? Consideration of ethics solely as a legal issue or a public relations tool A desire for "quick fix" solutions to ethical problems Failure to establish a written code of ethics Excessive emphasis on short-term revenues Willingness to take an ethical stand that may impose financial costs 9. award: 3 out of 3.00 points _____, an ethical system, defines ethical behavior according to how others behave. Universalism Virtue ethics Utilitarianism Egoism Relativism 10. award: 3 out of 3.00 points _____ has as its goal the creation of sustainable economic development and improvement of quality of life worldwide for all organizational stakeholders. Egoism Compliance building Stonewalling Life-cycle analysis Ecocentric management 11. award: 3 out of 3.00 points The _____ Act established strict accounting and reporting rules to make senior managers more accountable and to improve and maintain investor confidence. Commodity Futures Modernization Glass-Steagall Gramm-Leach-Bliley Sarbanes-Oxley Securities Exchange 12. award: 3 out of 3.00 points _____ refers to the moral principles and standards that guide the behavior of an individual or group. Corporate legitimacy Prioritization Ethics The legal system Justice 13. award: 3 out of 3.00 points Unethical behavior happens only in a business environment. True False 14. award: 3 out of 3.00 points The economic responsibilities of business include satisfying its obligations to investors. True False 15. award: 3 out of 3.00 points Who among the following persons said, "The social responsibility of business is to increase profits"? Adam Smith Robert Giacalone Benjamin Franklin Thomas Jefferson Milton Friedman 16. award: 3 out of 3.00 points Business should look at environmental issues as a no-win situation: either one helps the environment and hurts one's business, or else one helps one's business at a cost to the environment. True False 17. award: 3 out of 3.00 points In the context of making ethical decisions, knowing what actions are morally defensible is referred to as: moral judgment. moral character. moral awareness. moral ability. moral certainty. 18. award: 3 out of 3.00 points A manager of a company is facing an ethical problem. He uses the formal process for ethical decision making to resolve the problem. Which of the following steps will he begin with? Defining the complete moral problem Understanding all the moral standards and recognizing all moral impacts Evaluating the success of the solution Proposing a convincing moral solution Determining the legal requirements 19. award: 3 out of 3.00 points Carol uses certain principles, rules, and values such as honesty, fairness, and cooperation in her daily life to determine what is right or wrong. Which of the following does she employ when making decisions? rev: 03_15_2013_QC_26742 Egoism Moral philosophy Ecocentric management Universalism Carrying capacity 20. award: 3 out of 3.00 points In the context of corporate social responsibility, _____ responsibility involves meeting other social expectations, not written as law. economic legal philanthropic ethical human rights 21. award: 3 out of 3.00 points While working as a receptionist in a lawyer's office, Marcus was instructed to inform callers that the lawyer was in court when, in fact, the lawyer was playing golf. He eventually quit the job because he could not bring himself to lie to people. Marcus was following the ethical system known as: the Taylor Principles. relativism. egoism. social justice. universalism. 22. award: 3 out of 3.00 points A company was required by law to change its physical work environment in such a way so as to enable disabled people to function properly. Since the company followed a(n) _____ ethics program, it did not stop at just making changes that would help it stay out of trouble with the law. It went further and trained its employees to be sensitive toward disabled people. mutuality-based awareness-based compliance-based integrity-based competency-based 23. award: 3 out of 3.00 points In the context of business ethics, the Dodd-Frank Wall Street Reform and Consumer Protection Act of 2010 issued a broader _____ rule in order to address the weaknesses of the Sarbanes-Oxley Act. sexual harassment protection anti-discrimination human rights responsibility whistleblower environmental preservation 24. award: 3 out of 3.00 points Laura, an employee at a company, observed that one of her co-workers accessing confidential company information and carrying it home on a flash drive. She informed her supervisor about it and the errant employee was fired. Unfortunately, Laura found herself ostracized by her co-workers after the incident. Her supervisor did not take any action to protect her. Hence, it can be stated that Laura's company has a poor: rev: 03_15_2013_QC_26742 distribution policy. administrative framework. strategic objective. span of control. ethical climate. 25. award: 3 out of 3.00 points The _____ model is a theory of corporate social responsibility which holds that managers are agents of shareholders whose primary objective is to maximize profits. ethical shareholder stakeholder philanthropic economic 26. award: 3 out of 3.00 points Caring, being honest, showing loyalty, being fair, and acting with integrity are all examples of: citizenship. excellence. ethics. self focus. values. 27. award: 3 out of 3.00 points In the context of business ethics and the law, which of the following statements is true regarding the Sarbanes-Oxley Act? It prohibits senior managers from personally signing off on financial results. It was passed by the Congress in 1992 to improve investor confidence. It requires companies to strictly adhere to a policy of sustainable development. It was passed by the Congress in response to a series of corporate scandals. It requires organizations to have more company insiders as members of the board of directors. 28. award: 3 out of 3.00 points According to the Caux Principles for Business, the ethical ideal of kyosei means: recognizing the intrinsic worth of every human being. accepting the moral constraints of living in a society. deciding what is right by using a set of rules and regulations. living and working together for the common good. being aware of moral situations and then acting morally. 29. award: 3 out of 3.00 points Which of the following is true of unethical corporate behavior? Companies with international activities find it easier to maintain a positive ethical climate. Unethical corporate behavior is the sole responsibility of an unethical individual. Managers must adhere to firm standards in different cultural contexts, rather than decide when relativism is appropriate. Unethical corporate behavior reveals a company culture that is ethically lax. Business ethics are shaped exclusively by laws and by individual development and virtue. 30. award: 3 out of 3.00 points Susan, an employee in a company, is facing a situation in which she has to choose between going along with her colleagues and filling out inaccurate time records, or informing her manager about the malpractice. In this context, Susan is faced with a(n): moral rights issue. ethical issue. universal value. corporate social responsibility. human rights violation.
Psychiatry, Psychology and Predictive Criteria
Psychiatry, Psychology and Predictive Criteria
Write a 1- 2pg response paper discussing the utility of using psychological measures as suggested below in the diagnosis of mental illness: There is a movement in psychology to require the usage of psychological measures to aid in the diagnosis of some mental illnesses. While developing the DSM-V, there was some discussion of including results of specific measures as part of diagnostic criteria. Considering the two articles attached to draft this short paper. Diagnostic utility of the NAB List Learning test in Alzheimer's disease and amnestic mild cognitive impairment Abstract Measures of episodic memory are often used to identify Alzheimer's disease (AD) and mild cognitive impairment (MCI). The Neuropsychological Assessment Battery (NAB) List Learning test is a promising tool for the memory assessment of older adults due to its simplicity of administration, good psychometric properties, equivalent forms, and extensive normative data. This study examined the diagnostic utility of the NAB List Learning test for differentiating cognitively healthy, MCI, and AD groups. One hundred fifty-three participants (age: range, 57-94 years; M = 74 years; SD, 8 years; sex: 61% women) were diagnosed by a multidisciplinary consensus team as cognitively normal, amnestic MCI (aMCI; single and multiple domain), or AD, independent of NAB List Learning performance. In univariate analyses, receiver operating characteristics curve analyses were conducted for four demographically-corrected NAB List Learning variables. Additionally, multivariate ordinal logistic regression and fivefold cross-validation was used to create and validate a predictive model based on demographic variables and NAB List Learning test raw scores. At optimal cutoff scores, univariate sensitivity values ranged from .58 to .92 and univariate specificity values ranged from .52 to .97. Multivariate ordinal regression produced a model that classified individuals with 80% accuracy and good predictive power. (JINS, 2009, 15, 121-129.) [PUBLICATION ABSTRACT] Full Text Translate Full text? (ProQuest: ... denotes non-US-ASCII text omitted.) INTRODUCTION Alzheimer's disease (AD) compromises episodic memory systems, resulting in the earliest symptoms of the disease (Budson & Price, 2005). Measures of anterograde episodic memory are useful in quantifying memory impairment and identifying performance patterns consistent with AD or its prodromal phase, mild cognitive impairment (MCI; Blacker et al., 2007; Salmon et al., 2002). List learning tests are commonly used measures of episodic memory that offer a means of evaluating a multitude of variables relevant to learning and memory. Some of the more common verbal list learning tasks are the California Verbal Learning Test (CVLT; Delis et al., 1987), Auditory Verbal Learning Test (AVLT; Rey, 1941, 1964), Hopkins Verbal Learning Test (HVLT; Brandt & Benedict, 2001), and the Word List Recall test from the Consortium to Establish a Registry for Alzheimer's Disease (CERAD; Morris et al., 1989). List learning tests have been shown to possess adequate sensitivity and specificity in differentiating participants with MCI ( Mdn: sensitivity = .67, specificity = .86) (Ivanoiu et al., 2005; Karrasch et al., 2005; Schrijnemaekers et al., 2006; Woodard et al., 2005) and AD (Mdn: sensitivity = .80, specificity = .89) from controls (Bertolucci et al., 2001; Derrer et al., 2001; Ivanoiu et al., 2005; Karrasch et al., 2005; Kuslansky et al., 2004; Salmon et al., 2002; Schoenberg et al., 2006), as well as AD from MCI (Mdn: sensitivity = .85, specificity = .83) (de Jager et al., 2003). The current study was undertaken to evaluate the diagnostic utility of a new list learning test in a sample of older adults seen as part of a prospective study on aging and dementia. The Neuropsychological Assessment Battery (NAB; Stern & White, 2003a, b) is a recently-developed comprehensive neuropsychological battery that has been standardized for use with individuals ages 18 to 97. It contains several measures of episodic memory, including a List Learning test similar to other commonly used verbal list learning tests. The NAB List Learning test was developed to "create a three trial learning test to avoid the potential difficulties that five trial tasks represent for impaired individuals, include three semantic categories to allow for examination of the use of semantic clustering as a learning strategy, avoid sex, education, and other potential biases, and include both free recall and forced-choice recognition paradigms" (White & Stern, 2003, p. 24). One major benefit of the NAB includes the fact that all of its 33 subtests, together encompassing the major domains of neuropsychological functioning, are co-normed on the same large sample of individuals (n = 1448), with demographic adjustments available for age, sex, and education. This normative group contains a large proportion of individuals ages 60 to 97 (n = 841), making it particularly well suited for use in dementia evaluations. Despite psychometric validation (White & Stern, 2003), its diagnostic utility has yet to be evaluated. For the last several years, several NAB measures have been included in the standard research battery in the Boston University (BU) Alzheimer's Disease Core Center (ADCC) Research Registry. The BU ADCC recruits both healthy and cognitively impaired older adults for comprehensive yearly neurological and neuropsychological assessments. After each individual is assessed, a multidisciplinary consensus diagnostic conference is held to diagnose each individual based on accepted diagnostic criteria. Importantly, the NAB measures have yet to be included for consideration when the consensus team meets to diagnose study participants. Therefore, the current study setting offers optimal clinical conditions (i.e., without neuropathological confirmation) for evaluating the diagnostic utility of the NAB List Learning test. In other words, NAB performance can be judged against current clinical diagnostic criteria without the tautological error that occurs when the reference standard is based on the test under investigation. Samples of participants from the BU ADCC Registry were used to evaluate the utility of the NAB List Learning test in the diagnosis of amnestic (a)MCI and AD. As the diagnostic utility of the NAB List Learning test has yet to be examined empirically, the present study was considered exploratory. METHOD Participants Participant data were drawn from an existing database--the BU ADCC Research Registry--and retrospectively analyzed. Participants were recruited from the greater Boston area through a variety of methods, including newspaper advertisements, physician referrals, community lectures, and referrals from other studies. Participants diagnosed as cognitively healthy controls consisted of community-dwelling older adults, many of whom have neither expressed concern about nor been evaluated clinically for cognitive difficulties. Data collection and diagnostic procedures have been described in detail elsewhere (see Ashendorf et al., 2008 and Jefferson et al., 2006). Briefly, after undergoing a comprehensive participant and informant interview, clinical history taking (i.e., psychosocial, medical), and assessment (i.e., neurological, neuropsychological), participants were diagnosed by a multidisciplinary consensus group that included at least two board certified neurologists, two neuropsychologists, and a nurse practitioner. Of an initial pool of 490 participants, 18 were excluded from the present study because English was not their primary language. An additional 172 were excluded because they were not diagnosed as control, aMCI, or AD. Of the remaining 300 participants, 153 completed all relevant portions of the NAB List Learning test. These 153 participants comprised the current sample, from which three groups were established: controls (i.e., cognitively normal older adults), participants diagnosed with single or multiple domain aMCI (based on Winblad et al., 2004), and participants diagnosed with possible or probable AD (based on NINCDS-ADRDA criteria; McKhann et al., 1984). The sample consisted of 93 women (60.8%) and 60 men (39.2%), ranging in age from 57 to 94 (M = 73.9; SD = 8.1). There were 128 (83.7%) non-Hispanic Caucasian participants and 25 (16.3%) African American participants. The data used in the current study were collected between 2005 and 2007 at each participant's most recent assessment, which ranged from the first to ninth visit ( Mdn = 4.0) of their longitudinal participation. Measures Procedure The BU ADCC Research Registry data collection procedures were approved by the Boston University Medical Center Institutional Review Board. All participants provided written informed consent to participate in the study. Participants were administered a comprehensive neuropsychological test battery designed for the assessment of individuals with known or suspected dementia, including all tests that make up the Uniform Data Set (UDS) of the National Alzheimer's Coordinating Center (Beekly et al., 2007; Morris et al., 2006). Neuropsychological assessment was carried out by a trained psychometrist in a single session. The identification of cognitive impairment in each of the domains assessed (language, memory, attention, visuospatial functioning, and executive functioning) was based on BU ADCC Research Registry procedures, which defined psychometric impairment a priori as a standardized score (e.g., Z-score, T-score) of greater than or equal to 1.5 standard deviation units below appropriate normative means on one or more "primary" variables. Primary variables in the memory domain include Trial 3 and Delayed Recall from the CERAD Word List, and both Immediate and Delayed portions of the Logical Memory and Visual Reproduction subtests from the Wechsler Memory Scales-Revised (WMS-R; Wechsler, 1987). WMS-R subtests were administered according to UDS procedures (e.g., only Story A from Logical Memory is administered) and no other WMS-R subtests were used. In addition to neuropsychological testing, participant information was also obtained via clinical interview with the participant and a close informant, neurological evaluation, review of medical history, and informant questionnaires. Diagnosis The results from the "primary" neuropsychological variables were used by the multidisciplinary consensus team, along with social and medical history, neurological examination results, and self/informant report (i.e., interviews and questionnaires), to arrive at a diagnosis for each participant. Diagnoses were made based only on information obtained during the participant's most recent visit. The NAB List Learning test was not a "primary" neuropsychological variable, and thus, was not considered for diagnostic purposes by the multidisciplinary consensus team. Data Analysis RESULTS A breakdown of the participant demographics among the three diagnostic groups is provided in Table 1. Table 1 also depicts the level of global impairment for each group, based on both Clinical Dementia Rating (CDR; Morris, 1993) Global Score and Mini-Mental State Exam (MMSE; Folstein et al., 1975) scores. Significant group differences were found on age (control < aMCI = AD), education (control > aMCI = AD), CDR Global Score, and average MMSE score (control > aMCI > AD). Table 1. Participant demographics and test results Note. aMCI = amnestic mild cognitive impairment; AD = Alzheimer's disease; AA = African American; CDR = Clinical Dementia Rating; MMSE = Mini-Mental Status Examination; NAB = Neuropsychological Assessment Battery. a Possible AD: n = 6; Probable AD: n = 20. Univariate Analyses Independent samples t tests demonstrated significant group differences on each of the four NAB List Learning tests (Table 1). ROC curve analyses for the NAB List Learning variables are presented in Table 2. The cutoff scores presented in Table 2 were chosen to maximize sensitivity and specificity, with equal emphasis on both (Youden, 1950). The individual NAB List Learning test variables were able to differentiate aMCI from controls ( Mdn: sensitivity = .73; specificity = .71), AD from controls (Mdn: sensitivity = .89; specificity = .94), and AD from aMCI (Mdn: sensitivity = .69; specificity = .78). Additional prevalence-free classification accuracy statistics (i.e., those independent of base rates, such as sensitivity, specificity, PLR, and NLR) for conventional cutoff scores are provided in Table 3. Table 2. Prevalence-free classification accuracy statistics for NAB List Learning variables at optimal cutoff scores Note. PLR = Positive Likelihood Ratio; NLR = Negative Likelihood Ratio; aMCI = Amnestic mild cognitive impairment; AD = Alzheimer's disease. Table 3. Prevalence-free diagnostic accuracy statistics for NAB List Learning variables at conventional cutoff scores Note. aMCI = Mild Cognitive Impairment; AD = Alzheimer's disease; Sn = Sensitivity; Sp = Specificity; PLR = Positive Likelihood Ratio; NLR = Negative Likelihood Ratio. Dashes represent a value of positive infinity due to a specificity of 1.00. a Includes both aMCI and AD. Multivariate Analyses Likelihood ratio and goodness-of-fit tests revealed that the multiple ordinal logistic regression model explained a significant portion of outcome variance and fit the data well, -2 Log Likelihood [chi]2 (4, n = 153) = 127.80; p < .01; Pearson Goodness of Fit [chi]2 (298, n = 153) = 216.86; p = 1.00. Of the four independent variables, List B Immediate Recall (parameter estimate = -0.05; 95% CI = -0.09 to -0.01; Wald (1, n = 153) = 5.30; p = .02) and List A Short Delay Recall (parameter estimate = -0.10; 95% CI = -0.15 to -0.05; Wald (1, n = 153) = 14.5; p < .001) were found to be the two that contributed significantly to the model. List A Immediate Recall (parameter estimate = -0.02; 95% CI = -0.07 to 0.02; Wald (1, n = 153) = 1.33; p = .25) and List A Long Delay Recall (parameter estimate = 0.03; 95% CI = -0.08 to 0.02, Wald (1, n = 153) = 1.11; p = .29) were not significant contributors to the model. Cross-validation The estimated classification accuracy of the model using cross-validation was 80% (95% CI = 72-88%). In identifying aMCI, the model yielded a sensitivity of .47 (95% CI = .17-.77) and a specificity of .91 (95% CI = .83-.99; PLR = 4.96; NLR = .59). In identifying AD, the model yielded a sensitivity of .65 (95% CI = .41-.89) and a specificity of .97 (95% CI = .94-.99; PLR = 21.18; NLR = .36). A frequency table of predicted by actual diagnosis is presented in Table 4. Table 5 presents the positive predictive power (PPP) and negative predictive power (NPP) of the ordinal model across a range of clinically relevant base rates. Table 4. Frequency of predicted diagnosis by actual consensus diagnosis Note. aMCI = Amnestic mild cognitive impairment; AD = Alzheimer's disease. Table 5. Positive and negative predictive power of the ordinal NAB List Learning model at various base rates Note. aMCI = Amnestic mild cognitive impairment; PPP = Positive Predictive Power; NPP = Negative Predictive Power; AD = Alzheimer's disease. DISCUSSION The results of this study show that the NAB List Learning test can differentiate between cognitively normal older adults and those with aMCI and AD. Univariate analyses showed that each of the four variables was able to make dichotomous classifications with sensitivity values ranging from .58 to .92 and specificity values ranging from .52 to .97. For instance, AD was differentiated from controls with over 90% sensitivity and specificity using a cutoff score of T < or = 37 on List A Short Delay Recall or T < or = 40 on List A Long Delay Recall. In addition, AD was differentiated from aMCI with over 70% sensitivity and 80% specificity using a cutoff score of T < or = 30 on List A Short Delay Recall (see Table 2). The multivariate ordinal logistic regression model, which incorporated four NAB List Learning variables, yielded an overall accuracy estimate of 80% based on fivefold cross-validation. In particular, the model was able to identify participants diagnosed with aMCI and AD with high specificity (.91 for aMCI and .97 for AD), but lower sensitivity (.47 for aMCI and .65 for AD). Taking prevalence into account, the ordinal logistic regression model was found to perform best when ruling out aMCI or AD (i.e., higher NPP) at lower base rates and when ruling in aMCI or AD (i.e., higher PPP) at higher base rates (Table 5). More specifically, in settings with clinical base rates of aMCI and AD at 20% or below, good performance on the NAB List Learning test can yield high confidence (i.e., NPP [greater than or equal to] .87) that the patient would not be diagnosed as aMCI or AD by our consensus team. Similarly, in a setting with base rates of aMCI or AD around 50% or greater, as may be seen in a memory disorders clinic, poor performance on the NAB List Learning test can provide a high degree of confidence (i.e., PPP [greater than or equal to] .72) that the patient would be given a diagnosis of aMCI or AD by our consensus team. It should be noted that the current sample excluded individuals who did not complete the entire NAB List Learning test, which, for some participants, was due to excessive cognitive impairment. In addition, the participants with AD in the current study were predominantly in the very mild (CDR = 0.5, n = 5 [19%]) to mild (CDR = 1.0; n = 13 [50%]) stages. Because the current sample is generally free from severe impairment, it may be a valid representation of the types of patients that clinicians are asked to evaluate for early diagnosis. Of the four variables entered into the ordinal logistic regression model, only two were found to contribute significantly: List B Immediate Recall and List A Short Delay Recall. Despite these findings, the results do not necessarily suggest that the nonsignificant variables lack value in differentiating healthy controls from individuals with aMCI from those with AD; in fact, both List A Immediate Recall and List A Long Delay Recall, in isolation, can differentiate control, aMCI, and AD groups with sensitivity values ranging from .58 to .92 and specificity values ranging from .52 to .97 (see Table 2). However, the results do suggest that these nonsignificant variables do not lead to a significant increase in explanatory power beyond what can be attained after considering List B Immediate Recall and List A Short Delay Recall performance. Despite the fact that the MCI and AD groups were older and less educated than the control group, these demographic differences are unlikely to be contributing to the current results. Although age and education differ across groups, the use of demographically-corrected normative data protects against their potential confounding influence. In other words, the use of demographically-corrected norms prevents age and education from being associated with the independent variables. In fact, the NAB Psychometric and Technical Manual (White & Stern, 2003) illustrates that age accounts for 0.0% of the variance and education accounts for 0.0% to 0.2% of the variance in scores on the independent variables that were used in the current study. The classification accuracy of the NAB List Learning test compares favorably to published data on other list learning tests. For instance, the median sensitivity and specificity values of the individual NAB List Learning variables are generally on par with those seen in tests such as the CVLT, AVLT, HVLT, and the CERAD Word List (Table 6). More specifically, for example, a recent study found that long delay free recall on the CVLT differentiated AD from controls with a sensitivity of .98 and a specificity of .88 (Salmon et al., 2002), similar to the reported values of NAB List A Long Delay Recall in the current study (sensitivity = .92, specificity = .97). However, a major strength of the current study is that it validates a single model, developed using multiple ordinal logistic regression, that combines several list learning variables simultaneously to discriminate between three diagnostic groups (i.e., control, aMCI, and AD). One advantage of this ordinal logistic regression model is that it combines the NAB List Learning variables quantitatively, yielding results that can be integrated with applicable base rates to estimate diagnostic likelihood. The use of this model allows for an empirically-validated, quantitative method of combining important variables, as opposed to using clinical judgment for "profile" analysis, which may be susceptible to limitations in human cognitive processing, such as interpreting patterns among multiple neuropsychological test variables (Wedding & Faust, 1989). Table 6. Comparison of sensitivity and specificity between individual variables from the NAB and the AVLT, CERAD, CVLT, and HVLT Note. MCI = Mild cognitive impairment; CERAD = Consortium to Establish a Registry for Alzheimer's Disease; IR = Immediate Recall; DR = Delayed Recall; %Ret = Percent Retention; HVLT = Hopkins Verbal Learning Test; NAB = Neuropsychological Assessment Battery; SDR = Short Delay Recall; LDR = Long Delay Recall; AD = Alzheimer's disease; CVLT = California Verbal Learning Test; AVLT = Auditory Verbal Learning Test. a Karrasch et al. (2005). b Ivanoiu et al. (2005). c Woodard et al. (2005). d Schrijnemaekers et al. (2006). e Current study (see Table 2). f Bertolucci et al. (2001). g Derrer et al. (2001). h Salmon et al. (2002). i Kuslansky et al. (2004). j Schoenberg et al. (2006). k de Jager et al. (2003). Although the current findings support the diagnostic utility of the NAB List Learning test, the generalizability of the current results is limited. For instance, the sample is highly educated; data were collected in a research setting where many individuals volunteered due to self-awareness of memory difficulties; and the specifics of the reference standard, such as the clinicians participating in the consensus team and the assessment protocol used, are unique to our setting. Although the sample contains a fair number of African American participants (16%), representation of other minority groups is lacking. An additional limitation is the fact that the NAB List Learning test was not directly compared with other list learning tests in the same sample, precluding more definitive statements about its diagnostic accuracy in relationship to alternate tests. Finally, the results are limited by the reference standard that was used to establish a diagnosis. Despite the documented advantages of actuarial approaches over subjective approaches to clinical decision making (Dawes et al., 1989; Grove et al., 2000), it is important to emphasize that the reference standard used in the current study is a multidisciplinary consensus diagnosis based on contemporary clinical diagnostic criteria, not neuropathological diagnosis. At the present time, diagnosis of definite AD requires neuropathological confirmation (McKhann et al., 1984). Consequently, the classification accuracy statistics reported herein cannot be interpreted to reflect the likelihood that a patient actually has AD; instead, they indicate the likelihood that this specific consensus diagnostic team would make a particular diagnosis when using the assessment methods described above. It should also be noted that the consensus diagnosis was made, in part, on the basis of other neuropsychological tests, some of which are methodologically and psychometrically similar to the NAB List Learning test. This may have introduced an inherent and unavoidable source of bias. However, the diagnoses was based on consensus after consideration of a wide range of information, thus reducing the likelihood that shared method variance between the NAB List Learning test and other episodic memory measures would have caused significant tautological concerns. From a methodological standpoint, there are other limitations that require future study. The data were analyzed retrospectively and at various points in the longitudinal assessment of participants. An important line of future research would be to longitudinally follow individuals diagnosed with aMCI to prospectively examine whether NAB List Learning test performance is associated with AD progression. Because the current study does not include other dementia subtypes, future studies should also examine non-AD dementias. Finally, to limit the number of predictor variables in the ordinal logistic regression model, the NAB List Learning variables that are considered "secondary" or "descriptive" (White & Stern, 2003) were excluded. However, these additional variables may add additional diagnostic utility to the List Learning test, and future study is warranted. Despite its limitations, the current study has several strengths. For instance, diagnostic accuracy statistics are provided for a large number of cutoff scores, providing users of the test considerable flexibility in interpreting test results. For example, depending on the desired purpose of the examination, users may wish to choose cutoff scores that place a higher value on sensitivity (e.g., clinical settings, where false positive errors may preferable to false negative errors) or specificity (e.g., research settings, where false negative errors may be preferable to false positive errors). Users of the test may choose to interpret results using traditional cutoff scores (e.g., Z-scores < or = 1.5 or 2.0), or to use the empirically-derived cutoff scores presented herein to emphasize sensitivity and specificity equally. In addition, test users may choose to examine each test variable individually, or to interpret the overall pattern of test scores using the multiple ordinal logistic regression model, which accounts for performance on the four primary NAB List Learning variables simultaneously. For the latter approach, positive and negative predictive values are provided for a range of base rates, allowing for a more individually tailored approach to test interpretation. An additional strength of the study was the lack of tautological error, as the NAB List Learning test was not used in diagnostic formulations. Instead, NAB List Learning performance was examined independently against the clinical "gold standard," a multidisciplinary consensus diagnostic conference. The cross-validation of the ordinal logistic regression model allows for examination of the degree of precision in estimates of sensitivity, specificity, and overall accuracy. Based on the reported confidence intervals, there is a good degree of precision in the ordinal model's overall accuracy (accuracy = 80%; 95% CI = 72-88%) and in the model's specificity to the diagnosis of both aMCI (specificity = .91; 95% CI = .83-.99) and AD (specificity = .97; 95% CI = .94-.99). However, in examining the 95% confidence intervals surrounding the sensitivity estimates for both aMCI and AD, it is apparent that the sensitivity of the ordinal model is considerably lower and lacking precision. This may be due in part to the relatively small sizes of the clinical sample and in part due to the negative log-log link function that was used in the multiple ordinal logistic regression model. This link function makes an a priori assumption that the underlying distribution of the data is skewed toward "normality." In other words, the model was chosen based on the assumption that the prevalence of healthy controls is greater than the prevalence of individuals with aMCI and AD. As a result, the ordinal logistic regression model may be more prone to false negative errors (i.e., reduced sensitivity) than to false positive errors (i.e., reduced specificity). This decreased sensitivity to aMCI and AD may also reflect the fact that individuals with aMCI and AD perform similarly on measures of episodic memory, and that functional measures may be necessary to improve diagnostic sensitivity once a certain degree of cognitive decline has occurred in an individual. Although the current results present diagnostic accuracy statistics for the NAB List Learning test, it should be emphasized that a diagnosis of aMCI or AD cannot be made on the basis of a single neuropsychological instrument. The current results demonstrate that the NAB List Learning test was able to classify older adults into cognitively normal, AD, and aMCI groups with accuracy levels similar to other published list learning tests (Bertolucci et al., 2001; de Jager et al., 2003; Derrer et al., 2001; Ivanoiu et al., 2005; Karrasch et al., 2005; Kuslansky et al., 2004; Salmon et al., 2002; Schoenberg et al., 2006; Schrijnemaekers et al., 2006; Woodard et al., 2005). The NAB List Learning test possesses a large and up-to-date set of demographically-corrected normative data ( n = 1441) and it was co-normed as part of a comprehensive neuropsychological test battery. In addition, it was developed to include two equivalent forms; in fact, in the NAB standardization sample (n = 1448), test form accounted for less than 1.5% of the total variance seen in List Learning performance (White & Stern, 2003), making it suitable for clinical re-evaluation and longitudinal research applications. The findings from the current study, along with the overall strengths of the NAB, suggest that the NAB List Learning test is an appropriate and clinically useful tool for the evaluation of older adults with known or suspected Alzheimer's disease. Although the current study did not directly compare the diagnostic utility of the NAB List Learning test to other list learning measures, the classification accuracy data presented herein are similar to those reported in the literature investigating the diagnostic utility of other list learning tests in control, MCI, and AD samples (see Table 6). Future research is warranted to make direct comparisons of diagnostic utility to other list learning instruments. ACKNOWLEDGMENTS The project described was supported by Grant Number M01 RR000533 and the CTSA Grant Number 1UL1RR025771 from the National Center for Research Resources (NCRR), a component of the National Institute of Health (NIH). Its contents are solely the responsibility of the authors and do not necessarily represent the official view of NCRR or NIH. This research was also supported by P30-AG13846 (Boston University Alzheimer's Disease Core Center), R03-AG026610 (ALJ), R03-AG027480 (ALJ), K12-HD043444 (ALJ), K23-AG030962 (ALJ), RO1-HG02213 (RCG), RO1-AG09029 (RCG), R01-MH080295 (RAS), and K24-AG027841 (RCG). Robert A. Stern is one of the developers of the NAB and receives royalties from its publisher, Psychological Assessment Resources Inc. Portions of this manuscript were presented at the International Conference on Alzheimer's Disease, Chicago, July 2008. With this exception, the information in this manuscript and the manuscript itself has never been published either electronically or in print. A scoring program based on the multiple ordinal regression model reported in this manuscript is available as an electronic addendum to this manuscript. Limitations of Diagnostic Precision and Predictive Utility in the Individual Case: A Challenge for Forensic Practice David J. Cooke Æ Christine Michie Received: 24 August 2007 / Accepted: 2 February 2009 / Published online: 11 March 2009 _ American Psychology-Law Society/Division 41 of the American Psychological Association 2009 Abstract Knowledge of group tendencies may not assist accurate predictions in the individual case. This has importance for forensic decision making and for the assessment tools routinely applied in forensic evaluations. In this article, we applied Monte Carlo methods to examine diagnostic agreement with different levels of inter-rater agreement given the distributional characteristics of PCL-R scores. Diagnostic agreement and score agreement were substantially less than expected. In addition, we examined the confidence intervals associated with individual predictions of violent recidivism. On the basis of empirical findings, statistical theory, and logic, we conclude that predictions of future offending cannot be achieved in the individual case with any degree of confidence. We discuss the problems identified in relation to the PCL-R in terms of the broader relevance to all instruments used in forensic decision making. There is an important disjunction between the perspective of science and the perspective of the law; while science seeks universal principles that apply across cases, the law seeks to apply universal principles to the individual case. Bridging these perspectives is a major challenge for psychology (Faigman, 2007 ). It is recognized by statisticians that knowledge of group tendencies—even when precise— may not assist accurate evaluation of the individual case (e.g., Colditz, 2001 ; Henderson & Keiding, 2005 ; Rockhill, 2001 ; Tam & Lopman, 2003 ). It is a statistical truism that the mean of a distribution tells us about everyone, yet no one. This has serious implications for the use of psychological tests in forensic decision making. To illustrate these limitations, we focus on one of the most widely used, and perhaps the most extensively validated, test in the forensic arena—the Psychopathy Checklist Revised (PCL-R1 ; Hare, 2003 ). We emphasize, however, that all psychological tests used in the same way in the forensic arena will suffer from similar limitations (e.g., VRAG, Quinsey, Harris, Rice, & Cormier, 1998 ; Static-99, Hanson & Thornton, 1999 ; COVR, Monahan et al., 2005 ). Mental health professionals are frequently asked to opine whether an individual might be violent in the future; psychopathic personality disorder is an important risk factor to consider (Hart, 1998 ). The PCL-R is the most frequently used measure of psychopathic personality disorder; it has been described as the ‘‘gold standard’’ for that purpose (Edens, Skeem, Cruise, & Cauffman, 2001; as cited in Hare, 2003 ). There can be little doubt that the PCLR has made a major contribution to our understanding of violence (Hart, 1998 ); nonetheless, it is important for the field to consider both its strengths and its limitations. Findings for this instrument will have implications for less well-validated tools. In this introduction, we consider two issues; first, the use of PCL-R scores in forensic practice and second, the general problem of the precision of predictions about an individual case. D. J. Cooke (&) _ C. Michie Department of Psychology, Glasgow Caledonian University, Glasgow G4 0BA, UK e-mail: djcooke@rgardens.vianw.co.uk 1 The PCL-R is a 20-item rating scale of traits and behaviors intended for use in a range of forensic settings. Definitions of each item are provided and evaluators rate the lifetime presence of each item on a 3-point scale (0 = absent, 1 = possibly or partially present, and 2 = definitely present) on the basis of an interview with the participant and a review of case history information. 123 Law Hum Behav (2010) 34:259–274 DOI 10.1007/s10979-009-9176-x PCL-R SCORES AND FORENSIC PRACTICE Much of the interest in the construct psychopathy comes from the relationship between the PCL-R and future criminal behavior (Lyon & Ogloff, 2000 ). Previous research suggests that psychopathy—as assessed using the Psychopathy Checklist-Revised (PCL-R; Hare, 1991 )—is an important risk marker for criminal and violent behavior (Douglas, Vincent, & Edens, 2006 ; Hart, 1998 ; Hart & Hare, 1997 ; Hemphill, Hare, & Wong, 1998 ; Leistico, Salekin, DeCoster, & Rogers, 2008 ; Salekin, Rogers, & Sewell, 1996 ). In fact, the PCL-R has been lauded as an ‘‘unparalleled’’ single predictor of violence (Salekin et al., 1996 ). Hart (1998 ) argued that failure to consider psychopathy in a violence risk assessment may constitute professional negligence. This empirical base has resulted in the PCL-R being used, not merely to measure the trait strength of psychopathy in an individual, but also to make predictions about what he or she will do in the future (Hare, 1993 ). As we demonstrate formally below, this additional step of prediction means that the potential for imprecision in forensic evidence is greatly increased: It expands the gulf between inferences about groups and inferences about individuals. The PCL-R has been incorporated into statutory or legal decision making (Hare, 2003 ). Within England and Wales, a PCL-R score above a cut-off of 25 or 30 can lead to detention in either a Special Hospital or a prison (Maden & Tyrer, 2003 ); in certain Canadian provinces parole boards explicitly consider PCL-R scores (Hare, 2003 ), and in Texas psychopathy assessments are mandated by statute for sexual predator evaluation (Edens & Petrila, 2006 ).2 The PCL-R plays a role in criminal sentencing, including decisions regarding indefinite commitment and capital punishment, institutional placement and treatment, conditional release, juvenile transfer, child custody, witness credibility, civil torts, and indeterminate civil commitment (DeMatteo & Edens, 2006 ; Fitch & Ortega, 2000 ; Hart, 2001 ; Hemphill & Hart, 2002 ; Lyon & Ogloff, 2000 ; Walsh & Walsh, 2006 ; Zinger & Forth, 1998 ). The PCL-R is regarded by many as the best method for operationalizing the construct of psychopathy. For example, Lyon and Ogloff (2000 ) argued that ‘‘… it is critical that the assessment is made using the PCL-R’’ (p. 166) when evidence about violence risk, based on psychopathy, is provided. Because of its central role in forensic decision making it is vital to assess its strengths and limitations and, by comparison, the limitations of less well-validated procedures. PREDICTIONS FOR INDIVIDUALS VERSUS PREDICTIONS FOR GROUPS Prediction is the raison d’eˆtre of many forensic instruments (e.g., VRAG, Quinsey et al., 1998 ; Static-99, Hanson & Thornton, 1999 ; COVR, Monahan et al., 2005 ). While this is not true of the PCL-R its frequent use in forensic practice is underpinned by the assumption—implicit or explicit—that it can predict future offending (Walsh & Walsh, 2006 ). How precise can such predictions be? The precision of any estimate of a parameter (e.g., mean rate of recidivism of a group) can be measured by the width of a confidence interval (CI); a CI gives an estimated range of values, which is likely to include an unknown population parameter. If independent samples are taken repeatedly from the same population, and a CI calculated for each sample, then a certain percentage (confidence level) of the intervals will include the unknown population parameter. Typically, 95% of these intervals should include the unknown population parameter; other intervals may be used (e.g., 68% and 99%). The width of this interval provides a measure of the precision—or certainty— that we can have in the estimate of the population parameter. The width of a CI of a population parameter is linked, in part, to the sample size used to estimate the population parameter (see below for a more technical explanation). The prevailing prediction paradigm has two stages. First, the parameters (mean, slope, and variance) of a regression model linking an independent variable (e.g., PCL-R score) to a dependent variable (e.g., likelihood of reconviction) are estimated. Each of these parameters has uncertainty associated with them, which can be expressed by confidence bands about the regression line. Second, a new case is selected and the PCL-R score is assessed, the model is applied and the likelihood of reconviction is estimated. The best estimate of the likelihood of reconviction for a new case will be identical to the point on the regression line for that PCL-R score. This new estimate has a CI—also known as a prediction interval—that expresses the precision, or certainty, that should be associated with the prediction made about the new case. Often the two steps are conflated, with the unrecognized assumption being made that the prediction interval for the new case is comparable to the CIs for the model. It is not (see below). The problem of making predictions for individuals from statistical models is now recognized in other disciplines. In relation to medical risks, Rose (1992 ) expressed the position clearly: ‘‘Unfortunately the ability to estimate the average risk for a group, which may be good, is not matched by any corresponding ability to predict which individuals are going to fall ill soon’’ (p. 48). In relation to reoffending, Copas and Marshall (1998 ) made a related point ‘‘… the score is not a prediction about an individual [italics added], but an estimate of what rate of conviction 2 The PCL-R is the most commonly used instrument for assessing psychopathy in this setting (Mary Alice Conroy, Personal communication, 10 April 2007). 260 Law Hum Behav (2010) 34:259–274 123 might be expected of a group [italics added] of offenders who match that individual on the set of covariates used by the score’’ (p. 170) (see also Altman & Royston, 2000 ; Bradfield, Huntzickler, & Fruehan, 1970 ; Colditz, 2001 ; Elmore & Fletcher, 2006 ; Henderson, Jones, & Stare, 2001 ; Henderson & Keiding, 2005 ; Rockhill, 2001 ; Rockhill, Kawachi, & Colditz, 2000 ; Tam & Lopman, 2003 ; Wald, Hackshaw, & Frost, 1999 ). It is not generally recognized that a risk factor must have a very strong relative risk (i.e.,[ 50) if it is to have utility as a screening instrument at the individual level (Rockhill et al., 2000 ; see also Kennaway, 1998 ). However, others set the bar higher: A risk factor has to be extremely strongly associated with a disease within a population before it can be considered to be a potentially useful screening test. Even a relative odds of 200 between the highest and lowest fifths will yield a detection rate of no more than about 56% for a 5% false positive rate… (Wald et al., 1999 , p. 1564). To put this in perspective, the relative risk for the association between lung cancer and smoking is between 10 and 15 (Rockhill et al., 2000 ), depending on the definition of exposure. The relative risk for the PCL-R and recidivism is something of the order of 3 for general recidivism and 4 for violent recidivism (Hare, 2003 ). Does the application of current forensic tools provide an adequate basis for testimony concerning the individual case? In this article, we attempt to answer this question by considering three issues pertaining to PCL-R data. How confident can clinicians and legal decision makers be, first, in the use of critical diagnostic cut-offs; second, in the numerical value of PCL-R scores; and third, in individual predictions of violent recidivism? We describe two studies. The first study addresses the accuracy of diagnostic decisions and the potential range of discrepancies between two raters. The second study addresses the accuracy of prediction of future violence in the individual case. The results have relevance beyond the PCL-R to the use of other psychometric instruments in forensic practice: The same limitations may apply to many forensic assessment instruments. STUDY ONE In the first study, we examined diagnostic accuracy, specifically the allocation of individuals around two critical cut-offs, i.e., around 30 and around 25; the first is the standard PCL-R cut-off for the diagnosis of psychopathy and the second, often adopted in the UK, has proven useful in that context including in decisions regarding treatment allocation (Hare, 2003 ). The inter-rater reliability figures presented in the PCL-R manual can be regarded as good (Nunnally & Bernstein, 1994 ); intraclass correlation coefficient for single ratings (ICC1 ) are estimated in some research studies as being above .80 (Male offenders = .86; Male forensic psychiatric patients = .88; Hare, 2003 , Table 5.4).3 Edens and Petrila (2006 ) indicated that these are probably ‘‘best case’’ estimates and ‘‘real world’’ reliabilities may be substantially poorer.4 Murrie, Boccaccini, Johnson, and Janke (2008 ), in one ‘‘real world’’ study, demonstrated poor agreement (ICC1 = .39). These views and findings echo concerns expressed by Hare (1998 ), that while researchers take great pains to ensure reliability in their studies, the level of reliability achieved by individual clinicians remains unknown—and by implication—is likely to be poorer than published studies. Inter-rater reliability is not the only relevant consideration: Diagnostic precision is also influenced by the underlying distribution of test scores. Diagnostic precision is influenced by the location of the cut-off and the shape of the distribution of scores—both skewness and kurtosis. Estimates of the precision of a test score (e.g., standard errors of measurement, SEM) are weighted toward the mean of the distribution whereas cutoffs are generally located substantially above the mean. Item Response Theory (IRT) studies demonstrate that the measurement precision of the PCL-R—in terms of measurement information—falls toward the diagnostic cut-off (Cooke, Michie, & Hart, 2006 ); thus, the SEM estimated on the mean will provide an optimistic estimate of diagnostic precision. The SEM cannot be directly translated into estimates of precision of diagnosis because of the impact of the score distributions. Equally, it is not possible to estimate misclassification rates directly using ICC1 values; therefore, simulation approaches are required. Study one describes a simulation that examines the impact of unreliability on diagnostic accuracy. Method Monte Carlo studies allow the investigation of the properties of distributions and estimates of parameters where results cannot be derived theoretically (Mooney, 1997 ; Robert, 2004 ). Large numbers of simulated datasets can be 3 The estimates of reliability are frequently obtained by re-rating the same interview or with an observer simultaneously rating within an interview. This will tend to inflate reliability, but not validity, as the same information source is being used. 4 The case of THE PEOPLE, Plaintiff and Respondent, v. KURT ADRIAN PARKER, a Sexual Violent Predator ACT case highlights the variability that can emerge in some cases; five accredited experts furnished five PCL-R scores that ranged from 10 to 25. (Edens, John, Personal Communication, 22 May 2006). Law Hum Behav (2010) 34:259–274 261 123 created based on an explicit and replicable data-generation process. The effect of known features designed into the data, such as levels of inter-rater reliability, on outcomes, such as diagnostic precision, can be assessed. Multiple trials of procedures are carried out to allow precise estimation of outcomes. Mooney (1997 ) argued that Monte Carlo simulations could allow social scientists to test classical parametric inference methods and provide more accurate statistical models. In our view, this mainstream statistical technique is underused in forensic research. Materials We used Monte Carlo techniques based on distribution information from two datasets of PCL-R total scores: (1) data for North American Male Offenders (Table 9.1, Hare, 2003 ) and (2) data from UK prisoners (Cooke, Michie, Hart, & Clark, 2005 ).5 The first distribution, being the largest, probably provides the best estimate of the true distribution of scores underlying the PCL-R and is described as ‘‘approximately normal’’ (Hare, 2003 , p. 55). Given the potential impact of a departure from normality we tested whether this distribution was in fact normal. The departure from normality was highly significant (Kolmogorov–Smirnov = .068, df = 5408, p/ .0001, Skewness = - .33, Kurtosis = - .570). Examination of Fig. 1 demonstrates that around the standard cut-off of 30 cases are over-represented while in the right tail of the distribution they are under-represented. In the simulation study,we generated two randomvariables per case using MATHCAD.13 (2005 ). These random variableswere scaled according to one of the two datasets referred to above with mean l and standard deviation r .6 This gives two uncorrelated ratings (x1 and x2 ) from the distribution of scores: x1 is our first rating on the subject, PCL1 . We then calculated a linear combination of the two ratings to provide a second rating on the same subject, which has a correlation of q with the first rating. The linear combination is PCL2 . roundf l . .x1 _ l.q . .x2 _ l. ffiffiffiffiffiffiffiffiffiffiffiffiffi 1 _ q2 p g ; using rounding to ensure an integer score. This process gives two random, correlated scores from the distribution. There is a very small probability of obtaining second ratings less than 0 or greater than 40: These scores have been taken as 0 or 40, respectively. Assuming that the ICC1 represents the best estimate of the correlation between the two scores, we estimated the distributions for four values of reliability, i.e., ICC1 values of .75, .80, .85, and .90. The .80 value is a lower bound estimate for reasonable practice. Hare (1998 ) indicated that at least this level should be achievable with ‘‘… properly conducted assessments’’ (p. 107). The .85 level may be achievable by one rater with good training; the .90 level, perhaps the best case scenario, is the level achievable where two independent sets of ratings are averaged. Values above .90 are rarely if ever achievable—Hare (1998 ) describes .95 and higher as ‘‘unbelievably high’’ (p. 107). The .75 provides a lower-bound estimate of what may be obtained in clinical practice. These values probably represent optimistic estimates for actual clinical practice; we did not assess the ‘‘worst case’’ scenarios implied by Edens and Petrila (2006 ) and by Murrie et al. (2008 ). The estimation procedure was repeated 10,000,000 times for each of the four levels of ICC1 to provide stable estimates of the distribution of the correlated ratings and to ensure at least 10,000 cases within each of the extreme score bands. We examine discrepancies in two ways: First, in terms of diagnostic disagreement and second, in terms of disagreements about total scores. What is the level of diagnostic agreement? Kappa (j ) coefficients measure the proportion of diagnostic agreements corrected for observed base rates (Fleiss, 1981 ). Conventionally, j/ .75 represents excellent agreement, .40/j/ .75 represents fair to good agreement and j/ .40 represents poor agreement (Gail & Benichou, 2000 ). Kappa values for three distributions and the four ICC1 values are given in Table 1 . We calculated Kappa coefficients for agreement in diagnosis between the two ratings using both common cut-offs, i.e., 30 and 25. The vast majority of Kappa values are only in the fair to good range; few values approach the poor range. Kappa is an omnibus statistic, which is useful for summarizing group results; however, it tells us little about agreement in the individual case. The potential for misclassification is clearer when distributions of disagreements 0 10 20 30 40 PCL-R 0 100 200 300 Frequency Fig. 1 Distribution of North American male prisoners and normal curve 5 We carried out a similar analysis of data for Male Forensic Psychiatric Patients (Table 9.2, Hare, 2003); the results, which demonstrate the same pattern, can be obtained from the first author. 6 A full description of the simulation study including the Mathcad code can be obtained from the first author. 262 Law Hum Behav (2010) 34:259–274 123 are considered. The distributions based on the North American Male Offenders are in Table 2 . For ease of interpretation, we tabulated the distributions in 5-point ranges. Examination of the sub-table for ICC1 = .80 indicates that if one rater gives a score between 30 and 34, i.e., just above the diagnostic cut-off then only in 46% of occasions—approximately half the time—will the other rater obtain a score within the same range. In 44% of the occasions, the second rater would place the individual below the critical cut-off. Even in the best case scenario, i.e., ICC1 = .90, if one rater gives a score between 30 and 34 then only in 60% of occasions will the other rater obtain a score within the same range. On 29% of occasions, the second rater would place the participant below the critical cut-off. The distributions based on the UK prisoners are in Table 3 . Examination of the table for ICC1 = .80 indicates that if one rater gives a score between 30 and 34, i.e., just above the diagnostic cut-off then only in 39% of occasions will the second rater obtain a score within the same range. In 54% of the cases, the second rater would place the individual below the critical cut-off.As previously, even in the best case scenario, i.e., ICC1 = .90, if one rater gives a score between 30 and 34 then only in 53% of cases will the other rater obtain a score within the same range. In 39% of cases, the second rater would place the participant below the critical cut-off. In the UK, the cut-off of 25, as well as 30, is often applied (DSPD Programme, 2005 ; Hare, 2003 ). Examination of the table for ICC1 = .80 indicates that if one rater gives a score between 25 and 29, i.e., just above theUKdiagnostic cut-off, then only in 29% of occasions will the other rater obtain a score within the same range. On 49% of occasions, the second rater would place the individual below the critical cutoff. Even in the best case scenario, i.e., ICC1 = .90, if one rater gives a score between 25 and 29 then only in 37% of cases will the other rater obtain a score within the same range. On 37% of occasions, the second rater would place the participant below the critical cut-off. Therefore, in broad terms, all of the findings reported above demonstrate that the allocation of an individual above or below diagnostic cut-offs is much less precise than previously thought. Another way of considering the precision of PCL-R scores is to examine expected discrepancies in scores based on variations in ICC1 while taking into account the distributional characteristics of the PCL-R scores. The PCL-R manual suggests that in 68% of cases the discrepancies between two raters should be up to 3 points, and in 95% of cases it should be up to 6 points (Hare, 2003 ). This assumes normality of the PCL-R score distribution, an assumption that is not met (see above). The cumulative distribution of score discrepancies estimated from the Monte Carlo studies are tabulated in Table 4 . With the North American prisoner sample and an ICC1 of .80, a discrepancy of between 8 and 9 points would be expected in 9% of cases, around 10 points in 5% of cases, and between 12 and 13 points in 1% of cases. With the UK prisoner sample, and an ICC1 of .80, a discrepancy of between 8 and 9 points would be expected in 23% of cases, around 10 points in 5% of cases, and around 12 points in 1% of cases. An alternative approach to summarize the range of possible discrepancies is to estimate the distribution of a 2nd PCL-R rating given the 1st PCL-R rating. This conditional distribution can be summarized by a CI that contains 95% of the 2nd ratings. This interval is thus defined by the lower and upper limits LL and UL given by Prob(LL/ 2nd rating/ ULj 1st rating) . 0: 95: Results for both 68% and 95% CIs for ICC1 = .80, and for both samples, are presented in Table 5 . For example, in the North American prisoner sample, if rater one obtains a total score of 30, then the 95% CI for rater two’s total score will be between 19 and 36 (i.e., between the 35th and 99th percentile). All the estimates in this study are conservative; that is, they assume that the SEM that applies at the mean applies Table 1 Kappa coefficients and levels of agreement for four levels of correlation (q) for two distributions q Both/30 Both C 30 Different j Both/25 Both C 25 Different j North American male offenders 0.75 72.9 10.6 16.4 .46 48.0 29.6 22.4 .54 0.80 73.3 11.5 15.1 .51 48.9 31.0 20.0 .59 0.85 74.4 12.2 13.5 .56 50.5 32.3 17.2 .64 0.90 75.5 13.4 11.1 .64 51.7 34.2 14.1 .71 United Kingdom prisoners 0.75 91.8 2.1 6.1 .38 79.8 7.6 12.6 .47 0.80 92.0 2.4 5.6 .43 83.1 6.8 10.2 .52 0.85 92.4 2.7 5.0 .49 81.2 9.1 9.7 .60 0.90 92.7 3.1 4.2 .57 82.0 10.1 7.9 .67 Law Hum Behav (2010) 34:259–274 263 123 around the cut-off. However, this is an unwarranted assumption. The overall variance of errors of measurement is a weighted average of the errors that pertain across the range of true score values. Precision of measurement of the PCL-R drops as scores approach the diagnostic cut-off (e.g., Cooke & Michie, 1997 ; Cooke et al., 2006 ). Thus, the degree of diagnostic misclassification and score discrepancy is likely to be greater in practice than demonstrated in the simulation above. The conditional SEM (CSEM)7 is the square root of the variance of errors at a particular level of true scores. To Table 2 Distribution of diagnostic disagreements by four levels of correlation between raters based on distribution of North American male offenders PCL-R score 0–4 5–9 10–14 15–19 20–24 25–29 30–34 35–40 q = 0.75 0–4 .209 .086 .033 .006 0 0 0 0 5–9 .505 .305 .200 .079 .008 0 0 0 10–14 .245 .318 .250 .188 .072 .005 0 0 15–19 .041 .235 .266 .270 .239 .078 .002 0 20–24 0 .053 .192 .256 .310 .281 .097 0 25–29 0 .002 .056 .155 .236 .369 .371 .130 30–34 0 0 .003 .044 .121 .230 .445 .551 35–40 0 0 0 .001 .014 .037 .085 .319 q = 0.8 0–4 .245 .096 .032 .002 0 0 0 0 5–9 .523 .372 .211 .065 .003 0 0 0 10–14 .217 .315 .288 .198 .055 .001 0 0 15–19 .014 .198 .280 .306 .240 .055 0 0 20–24 0 .019 .162 .269 .340 .283 .063 0 25–29 0 0 .026 .137 .247 .391 .379 .076 30–34 0 0 0 .023 .106 .237 .465 .585 35–40 0 0 0 0 .010 .031 .093 .339 q = 0.85 0–4 .285 .105 .021 0 0 0 0 0 5–9 .552 .423 .217 .038 0 0 0 0 10–14 .158 .328 .333 .202 .028 0 0 0 15–19 .005 .141 .303 .354 .231 .024 0 0 20–24 0 .003 .121 .287 .386 .265 .029 0 25–29 0 0 .005 .111 .266 .430 .351 .038 30–34 0 0 0 .008 .085 .252 .519 .569 35–40 0 0 0 0 .003 .029 .101 .393 q = 0.9 0–4 .361 .103 .009 0 0 0 0 0 5–9 .578 .486 .215 .012 0 0 0 0 10–14 .061 .348 .390 .190 .010 0 0 0 15–19 0 .063 .319 .422 .216 .005 0 0 20–24 0 0 .067 .304 .449 .238 .004 0 25–29 0 0 0 .071 .267 .500 .289 .006 30–34 0 0 0 0 .056 .239 .609 .494 35–40 0 0 0 0 0 .018 .098 .501 The tables show column percentages, which sum to 1 within rounding error. The rows therefore do not sum to 1 Table 3 Distribution of diagnostic disagreements by four levels of correlation between raters based on distribution of UK prisoners PCL-R score 0–4 5–9 10–14 15–19 20–24 25–29 30–34 35–40 q = 0.75 0–4 .395 .158 .064 .020 .002 0 0 0 5–9 .477 .352 .228 .107 .035 .002 0 0 10–14 .128 .332 .309 .219 .131 .032 0 0 15–19 0 .153 .271 .336 .265 .198 .041 0 20–24 0 .005 .119 .215 .316 .297 .266 .031 25–29 0 0 .009 .086 .162 .261 .291 .254 30–34 0 0 0 .017 .083 .191 .328 .529 35–40 0 0 0 0 .006 .020 .075 .186 q = 0.8 0–4 .438 .163 .059 .011 0 0 0 0 5–9 .485 .387 .232 .097 .014 0 0 0 10–14 .077 .354 .328 .229 .111 .016 0 0 15–19 0 .095 .296 .354 .289 .154 .023 0 20–24 0 0 .083 .237 .331 .321 .214 .007 25–29 0 0 .002 .067 .176 .287 .303 .224 30–34 0 0 0 .005 .077 .200 .387 .545 35–40 0 0 0 0 .002 .022 .073 .224 q = 0.85 0–4 .470 .168 .042 .003 0 0 0 0 5–9 .474 .437 .224 .069 .003 0 0 0 10–14 .056 .342 .375 .226 .075 .003 0 0 15–19 0 .053 .308 .404 .287 .106 .001 0 20–24 0 0 .050 .251 .383 .325 .140 0 25–29 0 0 0 .047 .194 .321 .328 .145 30–34 0 0 0 0 .058 .229 .447 .578 35–40 0 0 0 0 0 .017 .084 .277 q = 0.9 0–4 .530 .169 .020 0 0 0 0 0 5–9 .450 .501 .219 .033 0 0 0 0 10–14 .019 .312 .457 .207 .037 0 0 0 15–19 0 .018 .286 .489 .275 .050 0 0 20–24 0 0 .017 .252 .450 .318 .062 0 25–29 0 0 0 .018 .214 .367 .325 .046 30–34 0 0 0 0 .024 .257 .528 .587 35–40 0 0 0 0 0 .008 .085 .367 7 Professional standards indicate that the CSEM is an important piece of information that should be provided in a test manual. For example, Standard 2.14 ‘‘Conditional standard error of measurements should be reported at several score levels if constancy cannot be assumed. Where cut scores are specified for selection or classification, the standard errors of measurement should be reported in the vicinity of each cut score.’’ (American Educational Research Association/ American Psychological Association, 1999; p. 35 emphasis added). 264 Law Hum Behav (2010) 34:259–274 123 evaluate the true level of agreement of diagnosis likely to apply around a cut-off it is necessary to take the CSEM into account. Item Response Theory indicates that the error of measurement varies with location on the trait (h ). IRT gives SE.h. . 1 ffiffiffiffiffiffiffiffi I.h. p where I (h ) is the information at h . CTT gives SEM . SD ffiffiffiffiffiffiffiffiffiffiffi 1 _ q p Let q1 be the correlation at location 1 (h1 ), q2 be the correlation at location 2 (h2 ). Then q2 . 1 _ . 1 _ q1. I.h1. I.h2. Location 1 is h = 0.0 (PCL-R = 20) and Location 2 is h = 1.0 (PCL-R = 30) (Approximate locations from Hare, 2003 ; Fig. 6.6; see also Cooke & Michie, 1997 ). Overall, the impact of the location of the estimated ICC1 is limited, dropping—at a maximum—from .75 to .69. However, as noted above, even small drops in ICC1 (e.g., from .85 to .80) can substantially affect the misclassification rate and the range of likely score discrepancies (see Table 6 ). It is noteworthy that the magnitude of the drop appears to be proportionately larger the poorer the mean estimated level of inter-rater reliability. This suggests that the effect of the CSEM is larger in cases that start with a relatively poor level of inter-rater reliability. Equally, this would suggest that proportionately greater discrepancies would, in general, be obtained when factor or facet scores are considered because they have lower levels of reliability than the total scores (Hare, 2003 ). STUDY TWO The use of the PCL-R in court is frequently justified based on its predictive utility, the support being garnered from between-subject designs (Edens & Petrila, 2006 ; Hare, 2003 ; Walsh & Walsh, 2006 ). In this study, we are concerned with the individual. We examine the confidence that can be placed in a prediction that an individual with a particular PCL-R score will be reconvicted for a violent offence. All measurements and estimates entail error. As noted above, the degree of error is expressed by CIs. For Table 4 Cumulative distribution of expected discrepancies between two raters for different levels of correlation based on two sample distributions Point discrepancy SEMa North American male offenders United Kingdom prisoners Correlation Correlation 0.75 0.80 0.85 0.90 0.75 0.80 0.85 0.90 0 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1 .741 .934 .927 .919 .901 .936 .928 .918 .900 2 .503 .804 .785 .758 .704 .804 .785 .758 .703 3 .317 .679 .647 .595 .518 .681 .647 .595 .518 4 .303 .560 .516 .453 .352 .561 .518 .454 .357 5 .095 .449 .397 .327 .217 .452 .400 .332 .222 6 .046 .349 .292 .215 .121 .353 .298 .221 .125 7 .020 .262 .205 .136 .058 .269 .212 .141 .063 8 .007 .190 .137 .079 .023 .196 .142 .083 .026 9 .002 .132 .086 .041 .007 .137 .090 .044 .009 10 .001 .088 .050 .018 .002 .091 .054 .021 .003 11 .055 .026 .007 .058 .030 .009 .001 12 .032 .012 .002 .035 .015 .004 13 .017 .005 .019 .007 .001 14 .008 .002 .010 .003 15 .003 .005 .001 16 .001 .002 17 .001 a This column shows the cumulative distribution of discrepancies which was calculated assuming that discrepancies between two raters are normally distributed and that the SEM is 3 (Hare, 2003, pp. 66–67) Law Hum Behav (2010) 34:259–274 265 123 example, while the mean rate of reoffending for a ‘‘High Risk’’ group may be estimated as being 55%; the 95% CI indicates that the true value of the mean rate or reoffending for this group will lie between 44% and 66%, 95% of the time, i.e., 19 times out of 20 (Hart, Michie, & Cooke, 2007 ). However, the clinician and the decision maker are interested in the individual case not the group. Therefore, how much confidence can the clinician and decision maker have in predictions of reoffending in the individual case based on PCL-R scores? We examine CIs for group and individual predictions. Participants Two hundred fifty-five male prisoners between 18 and 40 years of age (M = 26.8, SD = 5.9) were interviewed in Scotland’s largest prison for a study of psychological characteristics and violence (Cooke, Michie, & Ryan, 2001 ; Michie & Cooke, 2006 ). Prisoners were selected by systematic random sampling of the prison. The average sentence length was 39 months (SD = 23 months; range = 3 months to 10 years and life). PCL-R Ratings PCL-R ratings were made according to instructions in the test manual (Hare, 1991 ). All PCL-R evaluations were conducted by trained raters using both interview and file review (ICC1 = .86). Assessment of Recidivism Reconviction data were obtained from two sources: The Scottish Criminal Records Office (SCRO) and the Police National Computer (PNC). The average follow-up period was 29 months. The point-biserial correlation between PCL-R scores and recidivism (r = .31) was above average for the field (Walters, 2003 ). For the purposes of illustration, we consider reconviction for violence that resulted in a prison sentence (i.e., generally a more serious violent offence). Follow-up data were available for 190 cases and PCL-R data for 184 of these. Table 5 The 68% and 95% confidence intervals for 2nd PCL-R total score given 1st PCL-R score and ICC = 0.8 1st PCLR Prisoners UK LL .95 LL .68 UL .68 LL .95 LL .95 LL .68 UL .68 LL .95 0 0 0 9 12 0 0 8 13 1 0 0 10 13