Mark Nelson, Comment: Evaluative Techniques in Consumer Finance--Experimental Results and Policy Implications for Financial Institutions, The Journal of Financial and Quantitative Analysis, Vol. 9, No. 2 (Mar., 1974), pp. 285-286

[The performance of audit tasks has been modeled as a function of the auditor's ability, knowledge and experiences (Libby 1993). Thus, an important aspect of assigning audit tasks is identifying the levels of knowledge and types of experience an auditor must have to achieve a sufficiently high level of performance (Abdolmohammadi and Wright 1987). An objective of the current study is to determine whether auditors' knowledge of basic accounting principles and error frequencies improves over the course of their early careers so as to enhance performance of a common analytical procedure, ratio analysis. Research in psychology suggests that two characteristics of a task (say, analytical procedures) could diminish the accuracy with which auditors learn error frequencies from experience and apply their knowledge to a task. First, auditors' memories of financial statement errors are encoded while they perform other information-processing activities. These competing task demands could use enough of an auditor's information-processing capacity to diminish both the accuracy with which memory traces of errors are encoded and the accuracy of their knowledge of error frequency (Naveh-Benjamin and Jonides 1986). Second, auditors must consider a variety of evidence when performing analytical procedures. In diagnostic tasks like ratio analysis, inordinate attention is given to evidence that is highly diagnostic of low-frequency events, causing an "inverse base rate effect" in which auditors consider such events as more likely (Medin and Edelson 1988). Assessing the extent to which either of these characteristics of the analytical procedures context prevents auditors from learning and applying error frequency knowledge is a second objective of this study. To test hypotheses about these objectives, an experiment is conducted in which experienced auditors, accounting students, and non-accounting students learn the frequencies of financial statement errors through their experience in solving a series of problems using ratio analysis. Subjects are then tested for their accuracy in using frequency information by having them diagnose novel combinations of the same evidence. Other subjects perform similar tasks for an abstract medical diagnosis to provide a benchmark for comparison. The results indicate that differences in accounting knowledge influenced the subjects' performance of ratio analysis, and that neither potential source of inaccurate learning of event frequency knowledge holds in this setting. That is, subjects learned frequencies in the presence of competing task demands, and the inverse-base rate effect was not observed. These results suggest that experienced, but not novice, auditors use both their superior knowledge of accounting and of error frequencies learned through experience. Another implication is that the performance of novice auditors may be improved by increasing their knowledge of basic accounting principles and error frequencies.]

[We report on an experiment in which experienced auditors (1) determine whether to allow a client to adopt an aggressive reporting method when the auditors have an incentive to do so, and (2), justify aggressive reporting by their interpretations of financial accounting standards. In the experiment, the appropriate reporting method depends upon whether an amount can be "reasonably estimated" as that term is used in an applicable accounting standard. The accounting standard relevant to determining the appropriate reporting method was manipulated between subjects (thus varying whether judging that an amount can be reasonably estimated would justify an aggressive or conservative method), as was engagement risk. The results indicate that the auditors responded to moderate engagement risk by permitting the aggressive reporting method and justified their choice with aggressive interpretations of accounting standards. When faced with high engagement risk, the auditors responded by requiring conservative reporting and justified their choice with conservative interpretations of accounting standards.]

[This paper reports the results of an experiment that examines the influence of uncertainty about the probability that a future loss will occur ("ambiguity") on auditors' and financial statement users' judgments about appropriate reference to contingent losses in audit reports. Application of Einhorn and Hogarth's (1985) ambiguity model suggests that, with respect to losses of low (high) probability, both auditors and users will act as if an ambiguous probability of loss is higher (lower) than a precise probability of the same magnitude, thus demonstrating a conservative (unconservative) reaction to ambiguity. In addition, since auditors may jeopardize client relations when they unnecessarily make audit report reference to contingent losses, auditors may react less conservatively to ambiguity than do users. The results of the experiment support these predictions.]

ABSTRACT This commentary considers experimental and archival auditing research published over the past 50 years and suggests directions for future research. Starting with Ashton (1974a) and Simunic (1980), I discuss major research themes occurring in these literatures. I use Brunswik’s lens model as a metaphor that highlights methodological complementarities and opportunities for convergence to address important issues. Possible topics to consider further in future research include the determinants of audit quality, the structure of audit firms and audit regulation, the effect of emerging technologies like AI, teaching and supporting future auditors, potential enhancements of the audit report, and expansion of the services that auditors provide to society.

Abstract The performance of audit tasks has been modeled as a function of the auditor's ability, knowledge and experiences (Libby 1993). Thus, an important aspect of assigning audit tasks is identifying the levels of knowledge and types of experience an auditor must have to achieve a sufficiently high level of performance (Abdolmohammadi and Wright 1987). An objective of the current study is to determine whether auditors' knowledge of basic accounting principles and error frequencies improves over the course of their early careers so as to enhance performance of a common analytical procedure, ratio analysis. Research in psychology suggests that two characteristics of a task (say, analytical procedures) could diminish the accuracy with which auditors learn error frequencies from experience and apply their knowledge to a task. First, auditors' memories of financial statement errors are encoded while they perform other information-processing activities. These competing task demands could use enough of an auditor's information-processing capacity to diminish both the accuracy with which memory traces of errors are encoded and the accuracy of their knowledge of error frequency (Naveh-Benjamin and Jonides 1986). Second, auditors must consider a variety of evidence when performing analytical procedures. In diagnostic tasks like ratio analysis, inordinate attention is given to evidence that is highly diagnostic of low-frequency events, causing an "inverse base rate effect" in which auditors consider such events as more likely (Medin and Edelson 1988). Assessing the extent to which either of these characteristics of the analytical procedures context prevents auditors from learning and applying error frequency knowledge is a second objective of this study. To test hypotheses about these objectives, an experiment is conducted in which experienced auditors, accounting students, and nonaccounting students learn the frequencies of financial statement errors through their experience in solving a series of problems using ratio analysis. Subjects are then tested for their accuracy in using frequency information by having them diagnose novel combinations of the same evidence. Other subjects perform similar tasks for an abstract medical diagnosis to provide a benchmark for comparison. The results indicate that differences in accounting knowledge influenced the subjects' performance of ratio analysis, and that neither potential source of inaccurate learning of event frequency knowledge holds in this setting. That is, subjects learned frequencies in the presence of competing task demands, and the inverse-base rate effect was not observed. These results suggest that experienced, but not novice, auditors use both their superior knowledge of accounting and of error frequencies learned through experience. Another implication is that the performance of novice auditors may be improved by increasing their knowledge of basic accounting principles and error frequencies.

ABSTRACT This paper examines interpretation of accounting standards that provide implementation guidance via affirmative or counter examples. Based on prior psychology research, we predict that practitioners engage in “example‐based reasoning” such that they are more likely to conclude that their case qualifies for the same treatment as the example. We test our predictions in two experiments in which participants judge the appropriateness of income‐statement recognition. Experiment 1 uses Masters of Business Administration (MBA) students and varies example type (affirmative, counter) and case (revenue recognition, expense recognition) in a 2 × 2 design. Experiment 1 supports our predictions. Experiment 2 uses more experienced practitioners, and varies example type (affirmative, counter, both) in a 1 × 3 design. Experiment 2 supports the use of example‐based reasoning, and indicates that practitioners in the “both” condition respond as if they had only received an affirmative example. These results have implications for understanding how guidance that accompanies accounting standards can result in aggressive or conservative application of standards.

Loss contingencies, Expectation gap, Auditor judgment, Preresolution information

[Prior research indicates that auditors encounter difficulty in applying experienced error frequencies to judgments of the probability that an audit objective is violated given a particular transaction cycle. This difficulty may occur because of a mismatch between the organization of this particular judgment task and the organization of auditors' knowledge. We test the effectiveness of two decision aids at counteracting this difficulty: (1) a checklist aid which facilitates knowledge retrieval and (2) a decomposition-and-mechanical-aggregation aid which facilitates both knowledge retrieval and aggregation. The checklist aid slightly improved the degree to which auditors' judgments reflected experienced frequencies and the mechanical-aggregation aid greatly improved auditors' judgments, completely counteracting the effect of the task organization-knowledge organization mismatch.]

[Experienced auditors tend to structure their knowledge of financial statement errors with audit objective as the primary organizing dimension and transaction cycle as secondary. Yet, many audit tasks are structured in the opposite manner, requiring auditors to assess whether objectives are met for each transaction cycle. Our paper reports the results of an experiment which indicates that this mismatch between knowledge structure and task structure may hinder auditors' ability to draw on previous experiences when making conditional probability judgments and when allocating audit hours to various objectives within cycles. These results suggest one instance where knowledge structures that are often functional may have adverse effects when they do not match the task structure to which they are applied.]