Use of process mining as a tool to simplify the financial audit

Contents

Introduction

I. Fundamentals of process mining

II. Relevance of business process audit

III. Existing challenges of process mining in financial audit

IV. Conclusion

Introduction

While the majority of published reviews about the use process mining in general and in the field of financial audit concentrates on the benefits (e.g. [1-7]), the purpose of this paper is to provide an overview of challenges process mining has to face in this area.

The co-founder of Intel predicted in 1965 that the number of components in integrated circuits would double every year. By now, over 50 years later we have to deal with this amount of data.8 A lot of new disciplines emerged, one is process mining, which has its roots in the area of software engineering. The early pioneers of Agrawal et al.9 and Cook & Wolf et al.[10-12] developed algorithms to construct process flow charts by using event logs. Since 2000 the research in this field has rapidly increased due to the amount of data (“Big data”) available. One of the leading representatives of this research area is Wil van der Aalst, who provides a comprehensive overview of process mining in his book “Process Mining: Data Science in Action”13 as well as in his massive open online course (MOOC) at the Eindhoven university of technology.

Process mining visualization (d) Process mining audit (e) Illustration 1: Percental distribution of publication addressing a keywords of process mining in relation to the total amount of publications with this keyword in the timespan between 2004 and 2018 (data gathered from google scholar)¹

Illustration 1 visualised that in recent years the scientific work in the field of process mining is increasingly concerned with expanding the use of application solutions and the way of visualising business processes. A multitude of fields of application have already been developed, this paper focus on the use of process data for financial audit. Although algorithms have developed and techniques have improved, no huge change has occurred in the application of process mining since the 1990s. Information systems generate event data/ event logs, which are transferred to a process model. Table 1 illustrates the typical information presented in an event log. Depending on the algorithm and the technique only parts of these data are used. An event log has three minimal requirements: the event has to be related to a case and an activity and in addition activities have to be ordered.13 Process instances that have the identical path can be grouped in variants.14

Abbildung in dieser Leseprobe nicht enthalten

Table 1: A fragment of some event logs: Each line corresponds to an event13

While the introduction should have given you a short introduction into the context of this seminar paper, the remainder of this paper is organized as follows. The basics of process mining are described in Section I. The relevance of business process audit for the financial audit is shown in Section II. Challenges for process mining of event logs are displayed in Section III. Finally in Section IV a short conclusion regarding the usability of process mining in financial audit is provided.

I. Fundamentals of process mining

Information technology (IT) systems are increasingly linked to the business processes they support. These IT systems (like ERP, CRM, MES, .) automatically generate event data which are made available for business process management by process mining. Therefore, process mining combines process-oriented business process management and business process modelling with non-process-oriented data mining. Methods and algorithms are used to evaluate the company's existing transaction and process data. Process mining gains knowledge from event logs in order to automatically recognize, check and improve processes based on actual processes/transactions that are recorded in IT systems. Process mining enables companies to understand how their business processes work and how to manage them by establishing a strong relationship between the process model and the reality (event log).15 This relationship is reflected by the terms Play-Out, Play-In and Replay.

For Play-In a behaviour (event log) is taken as input with the goal of constructing a process model. Therefore, the captured behaviour seen in the event logs has to be selected wisely, which makes process discovery one of the most challenging process mining tasks.

Play-Out uses an existing Petri-Net to generate a behaviour (recorded in event logs) by following the possible paths.

The most important form of process mining is Replay, in which an event log as well as a process model are used as input. Thereof, it is possible to replay the trace in the Petri-net with the event log. There are three main reasons to do so: conformance checking, enhancement and operational support. With conformance checking it is possible to detect and quantify discrepancies between the model and the log. Enhancement or enrichment can be understood as extending the model with frequencies (of the path/activities) and temporal information (duration/waiting time) to detect e.g. bottlenecks.13 Two aspects of process mining are covered by operational support. The first one is the construction of predictive models which yields in the second one, the realtime assistance during operational processing.15 The positioning of the three main types of process mining (process discovery, conformance checking and enhancement) is shown in Illustration 2.

As the quality of process mining is highly depending on the quality of the input data17, the quality of event logs can easily be described by completeness and noise. The evaluation of the quality of a process model as a result of process mining is much more complex and depends on the use case (e.g. 18). Overall all criteria can be summarized for process discovery in four main criteria: fitness (capability to explain observed behaviour), precision (avoiding underfitting), simplicity and generalization (avoiding overfitting/ spaghetti models). To avoid overfitting, algorithms can use clustering and variant control techniques (e.g. [19-22]).

II. Relevance of business process audit

With an overall objective of the contemporary risk-based approaches of financial audit to identify material risks in the financial reporting and to address those risks through specific audit procedures, business processes and related internal controls play an essential part. One of the major risks is that transactions made are not complete and/or accurate. While ensuring that the control system for these transactions is working faultless, transactions are complete and accurate. To guarantee the consistency and quality of process auditing, international23 and national² standards of accounting are published. These standards specify how to review business processes, as well as internal control systems and relevant information systems. The ISA 315³ states that the “auditor should obtain an understanding of the information system, including the related business processes [and] relevant financial report [...]”. Accordingly one main aspect for auditing business processes is that the auditor requires a sufficient understanding of the relationship between business activities, financial accounts (only those business transactions with a predefined material effect on the financial statement have to be inspected) and the entity's internal controls.24 Illustration 3 displays activities that were executed in a simple purchase process and shows those financial accounts that were involved thereby. From an accounting perspective it is important, that a journal entry item is cleared by one or more items posted on the opposite side of the same account. Without clearing deadlocks would erase, in process models they would appear as end nodes. In reality they are not, which results in an invalid information about the actual structure.25

In current audit practices process models are usually created by conducting time-consuming and error-prone interviews and inspecting available documents on a sample basis. Then Process models are prepared by using simple general modelling tools (e.g. PowerPoint, Word).26 With an increasing degree of automation of business processes handling and a rising number of business transactions the amount of data can hardly be handled with traditional approache s. Manual inspections become inefficient and ineffective.1271 Additionally, if operations are performed with limited or no human interaction the contact persons may not possess the necessary process knowledge, resulting in unreliable information.1261 As the idea of process mining is to discover, monitor and to improve real processes based on reliable facts, it can used as a tool for process auditing to reduce the time consumption and the number of failures. Furthermore according to ISA 240 the auditor has the responsibility to scan for risks of material misstatement due to fraud,28 which could be simplified by using process mining.29

Illustration 4 shows the main phases of a financial audit. Process mining can particularly be used in the first stage of this process to understand the entity and its environment. Thus risks of material misstatement at the financial statement can be identified.1261 Also during the second phase process mining can be useful by performing substance tests of transactions to inspect the process related to the transaction balance.6

Abbildung in dieser Leseprobe nicht enthalten

III. Existing challenges of process mining in financial audit

As shown in the previous chapter, process audit is an elementary part of the financial audit. As process mining can be used for process audit it can accordingly be used as part of the financial audit. The following chapter deals with identified challenges process mining has to face. These challenges can be subsumed in eight clusters: data source, data quality, algorithm, confusion matrix, anomalies, process model, visualisation and mindset. The challenges that are described in the following pages are not an omniscient list of all challenges, others are possible but were mentioned in the literature with a lower frequency.

Challenges in the area of data source result mainly from the difference between the structure and/or platform of Enterprise-Resource-Planning-Systems (ERP-systems) and the data source which is used by process mining.30 While process mining evolves from event logs, ERP- systems are mainly based on rational database structure.31 In addition to the different structure ERP- and Warehouse-systems generate Big Data, that are not linked to financial accounts, which would be needed to conduct business process audits.7 Therefore financial accounts have to be integrated into the enterprise model, unfortunately, this remains only an approach on an abstract level.[24; 32] The second biggest challenge concerning data source emerges from uncertainty about the data access auditors have to deal with.1 Not only due to compliance regulations, but also because of the internationalisation (e.g. prohibition of data exchange over borders by the state government), in some cases auditors have no access to the certain client systems which lead to a lack in process information.33

The problem regarding data security belongs to both data source, as well as data quality. While it has to be ensured that the data are extracted on a secured way, it is also elementary important, that extracted data a secured, so that it is not possible to change them after extraction.31 To ensure a trustful financial audit only those accurate, valid, authorized and complete transactions should be recorded.25 Additionally, in financial audit only those business transactions are inspected, that can have a material effect on the financial statement.34 If the audit trail has been preserved, the accuracy and validity of transactions can be verified by following a transaction from the origin through a process until it is reflected in the company’s financial records, the so called “walkthrough”.[31; 34; 35] In case a complete walkthrough is not possible and only parts of a process can be mined, no assurance can be provided on the outcome. Besides the four criteria mentioned above, a data set should be noise-free, which most of the actual available data sets are not.[30; 33; 36] Noise refers to logs containing either incorrect logging or exceptional/infrequent behaviour that needs human judgement before it can be incorporated in the model.13 A more general problem of process mining refers to the different perspective (e.g. different stakeholder) on the resulting process model and the additional information appended to process events for mining purposes.30

Challenges in the area of algorithms arise mainly at the selection of an adequate algorithm/tool.37 Tiwari et al. provide in their review of business process mining a good overview of the wide range of algorithm approaches that can be chosen from. Since not all industries are working similar, each has its own booking patterns.36 If those “individual cases” would be incorporated into algorithms, and linked with an industry specific, self-learning database, it would improve not only the quality of the process model of one company, but also the knowledge across all clients in the same industry. The existing problem is that this database cannot even be build gapless for one company due to the European regulation of audit rotation33 (see literature [38-40] for more information regarding the audit rotation).

Abbildung in dieser Leseprobe nicht enthalten

Illustration 5: Model of a Confusion/Error matrix4

To show which challenges are subsumed under the term of “confusion matrix”, a short view on Illustration 5, which depicts the model of confusion matrix or the so-called error matrix, is worth it.41 The confusion matrix is widely used to evaluate predictions of algorithms.1421 While true positive and true negative are the targeted categories, it occurs that specific processes are categorized as false positive or false negative. Instances that are categorized as false positive are even worse than false negatives, because a potential risk/an anomaly is not seen from the system and categorised as a correct transaction. As one of the main objectives of business process audit is to detect outliers which represent deviations from standard procedures, false positive incidents have to be eliminated by reducing the level or changing the way of clustering or changing the type of visualisation.1261 False negative instances would mean that more anomalies are discovered than have appeared in reality. Thereof a higher workload has to be managed by the auditor, which could overload the auditors processing capabilities.133’ 371 False negative results can never be eliminated entirely, but the frequency can be minimized via statistical learning, by using a proper algorithm as well as by having a (industry specific) database.[7; 331

Challenges in the area of anomalies primarily bother the auditor's education. At present, freshly graduated auditors have been expected to be proficient in understanding how to apply accounting regulations and how to understand audit risks that are linked to specific accounts. But they are not trained to consider whether a transaction itself makes sense or, more important, how to follow up on a discovered anomaly. Additional trainings on the job have to be implemented by experienced auditors. If not, the lack of requisite skills to properly apply process mining techniques will lead to a shift from auditing to advisory and thus reducing the audit quality.[33; 431

Detected anomalies are the result of well working process models. In the following the hurdles an auditor has to overcome to construct such a process model are described. The major difficulty is the time consumption, which the construction of such a model takes, therefore an effective and efficient process has to be developed. Knowledge from previous projects and experienced auditors is indispensable.25 To evaluate whether the quality of a process model is high or low, van der Aalst postulate four quality criteria (fitness⁴, simplicity⁵, generalization⁶ and precision⁷ ) which have to be fulfilled.13 While van der Aalst provide these quality criteria in a more general perspective on process mining, simplicity is not a key quality criteria in the area of financial audit. The reason is, that at the moment, auditors need weeks to build up a complex process model with information gathered from time-consuming and error-prone interviews.34 Consequently, the resulting process model has only to satisfy three quality criteria fitness, generalization and precision. In addition, the purpose of the created process model should be evaluated. Depending on the purpose and the stakeholder which should be addressed, a process model can focus on logical or temporal dependencies.44

Visualisation can be seen as part of process modelling, but due to the importance of this area, visualisation is considered as as a separate area of challenges. The most challenging problem, which makes the visualisation that difficult, is the fact, that the reality can more or less never been shown in one visualisation. If the reality was shown one-to-one, the result would be a spaghetti model⁸. Different approaches have been conducted to solve this problem. The oldest and most common method to visualise a model is via petri nets, but recent research focuses more on visualising process mining results as maps (e.g. heat map, materiality process map).[26; 45] The already mentioned possibly occurring risk is that the audit quality could decrease due to the shift from audit to advisory.

The following challenges refers to the mindset of people reading a financial statement. At the moment, most people believe that a financial statement guarantees to 100% that all transactions are correct, and the financial audit guarantees 100% correctness of the financial statement and all accounts included. But actually, only a sample of all transactions a company executes is checked on conformance. By using process mining it would be possible to increase the share of check transactions to those that are executed to nearly 100%. However, sampling still would be necessary in some processes to conduct a valid financial audit (like conformation of physical assets).7 Thus, the lack between the mindset and reality would be even bigger, even though the assurance of financial statements would increase.33

IV. Conclusion

As shown, process mining has to face a bunch of challenges to simplify the business process audit as a part of the financial audit. Most difficulties are easy to overcome, but some are more challenging, for those, more research is needed. The benefits of the use of process mining as shown in different papers (e.g. [1-7]) demonstrate that it is worth to intensive the research. The fact, that huge audit firms are still investing in this area strengthens this argument.33 Capriotti⁹ summarizes the current situation very well when he concludes that process mining “has the potential to be the most significant shift in how audit are performed since the adoption of paperless audit tools and technologies”.46

Bibliography

1. Gray, G.L. and R.S. Debreceny, A taxonomy to guide research on the application of data mining to fraud detection in financial statement audits. International Journal of Accounting Information Systems, 2014. 15 (4): p. 357-380.

2. van der Aalst, W.M.P. and A.K.A. de Medeiros, Process Mining and Security: Detecting Anomalous Process Executions and Checking Process Conformance. Electronic Notes in Theoretical Computer Science, 2005. 121: p. 3-21.

3. Aalst, W.M.P., et al., Auditing 2.0: Using Process Mining to Support Tomorrow's Auditor. Computer, 2010. 43: p. 90-93.

4. Jans, M., M. Alles, and M. Vasarhelyi, Process Mining of Event Logs in Auditing: Opportunities and Challenges. SSRN Electronic Journal, 2010.

5. Jans, M., M. G. Alles, and M. Vasarhelyi, A Field Study on the Use of Process Mining of Event Logs as an Analytical Procedure in Auditing. The Accounting Review, 2014. 89: p. 1751-1773.

6. Chiu, T., Performing Tests of Internal Controls Using Process Mining. CPA Journal, 2019. 89 (6): p. 57.

7. Byrnes, P.E. and A.R. Pawlicki. Reimagining Auditing in a Wired World. 2014.

8. Moore, D.G.E., Cramming more components onto integrated circuits. Electronics, 1965. 38 (8).

9. Agrawal, R., D. Gunopulos, and F. Leymann. Mining process models from workflow logs. 1998. Berlin, Heidelberg: Springer Berlin Heidelberg.

10. Cook, J.E. and A.L. Wolf, Discovering Models of Software Processes from Event­Based Data. ACM Trans. Softw. Eng. Methodol., 1996. 7: p. 215-249.

11. Jonathan E. Cook, A.L.W., Software Process Validation: Quantitatively Measuring the Correspondence of a process to a model. ACM Transaction on Software Engineering and Methodology, 1997. 8: p. 147-176.

12. Cook, J.E. and A.L. Wolf, Event-based detection of concurrency. SIGSOFT Softw. Eng. Notes, 1998. 23 (6): p. 35-45.

13. Aalst, W.v.d., Process mining: data science in action. Second edition ed, ed. W.v.d. Aalst. 2016, Berlin: Springer.

14. Garcia, C.d.S., et al., Process mining techniques and applications - A systematic mapping study. Expert Systems with Applications, 2019. 133: p. 260-295.

15. Peters, R., Process-Mining : Geschäftsprozesse: smart, schnell und einfach, ed. R. Peters and M. Nauroth. 2019, Wiesbaden: Springer Fachmedien Wiesbaden.

16. Ghasemi, M., From event logs to goals: a systematic literature review of goal-oriented process mining. Requirements Engineering, 2019.

17. Weerdt, J., et al., Process Mining for the multi-faceted analysis of business processes—A case study in a financial services organization. Computers in Industry, 2013. 64: p. 57-67.

18. Driessen, R., The usability of the Process Mining analysis method to improve processes of the Netherlands Ministry of Defence. 2017.

19. Li, C., M. Reichert, and A. Wombacher, The minadept clustering approach for discovering reference process models out of process variants. International Journal of Cooperative Information Systems, 2010. 19 (03n04): p. 159-203.

20. Prodel, M., et al., Optimal Process Mining for Large and Complex Event Logs. IEEE Transactions on Automation Science and Engineering, 2018. 15 (3): p. 1309-1325.

21. Weerdt, J.D., et al., Active Trace Clustering for Improved Process Discovery. IEEE Transactions on Knowledge and Data Engineering, 2013. 25 (12): p. 2708-2720.

22. Günther, C.W. and W.M.P. van der Aalst. Fuzzy Mining - Adaptive Process Simplification Based on Multi-perspective Metrics. 2007. Berlin, Heidelberg: Springer Berlin Heidelberg.

23. IFAC, Identifying and Assessing the Risks of Material Misstatement through Understanding the Entity and Its Environment. 2012. ISA 315 (Revised.

24. Mueller-Wickop, N., M. Schultz, and M. Peris, Towards Key Concepts for Process Audits - A Multi-Method Research Approach. 2013. 70-92.

25. Werner, M., Financial process mining - Accounting data structure dependent control flow inference. International Journal of Accounting Information Systems, 2017. 25: p. 57-80.

26. Werner, M., Materiality Maps - Process Mining Data Visualization for Financial Audits. 2019.

27. Werner, M., N. Gehrke, and M. Nuttgens, Business Process Mining and Reconstruction for Financial Audits. 2012. 5350-5359.

28. IFAC, ISA 240. 2008.

29. Gehrke, N. Basic Principles of Financial Process Mining A Journey through Financial Data in Accounting Information Systems. 2010.

30. Wil, M.P.v.d.A., Process Mining and Security: Detecting Anomalous Process Executions and Checking Process Conformance, ed. A.K.A.D. Medeiros and X.A. The Pennsylvania State University CiteSeer. 2004.

31. Jans, M., Process Mining in Auditing: From Current Limitations to Future Challenges. Vol. 100. 2011. 394-397.

32. Brocke, J.v., C. Sonnenberg, and U. Baumöl, Linking accounting and process-aware information systems - Towards a generalized information model for process-oriented accounting. 2011.

33. Earley, C.E., Data analytics in auditing: Opportunities and challenges. Business Horizons, 2015. 58 (5): p. 493-500.

34. M. Werner, N.G., Multilevel Process Mining for Financial Audits. IEEE Transactions on Service Computing, 2015. 6 (06): p. 820-832.

35. M. B. Romney, P.J.S., Accounting Information System. Vol. 14 Edition. 2018: Person.

36. Tiwari, A., C. Turner, and B. Majeed, A review of business process mining: State-of- the-art andfuture trends. Business Process Management Journal, 2008. 14: p. 5-22.

37. Brown-Liburd, H., H. Issa, and D. Lombardi, Behavioral Implications of Big Data's Impact on Audit Judgment and Decision Making and Future Research Directions. Accounting Horizons, 2015. 29: p. 150119134654004.

38. (EU), E.U., Verordnung Nr. 537/2014 des europäischen Parlaments und des Rates, in L 158. 27.05.2014: ABl. (Amtsblatt der Europäischen Union). p. 77 - 112.

39. (EU), E.U., Richtlinie 2014/56/EU des europäischen Parlaments und Rates, in L 158. 27.05.2014: ABl. (Amtsblatt der Europäischen Union). p. 196 - 226.

40. EU-Regulierung", A.A.-u.I.d., EU-Regulierung der Abschlussprüfer. 23.05.2018, Institut der Wirtschaftsprüfer (IDW).

41. Stehman, S., Selecting and interpreting measures of thematic classification accuracy. Remote Sensing of Environment, 1997. 62: p. 77-89.

42. D. Puniskis, R.L., R. Dirmeikis, An artificial neural Nets for Spam e-mail recognition. Research Journal Elektronika IR Elektrotechnika, 19.03.2015. 69 (5): p. 73 - 76.

43. Katz, D. Regulators Fear Big Data Threatens Audit Quality. 15.04.2014 [cited 29.08.2019; Available from: https://www.cfo.com/auditing/2014/04/regulators-fear- big-data-threatens-audit-quality/.

44. Werner, M. and M. Nüttgens, Improving Structure: Logical Sequencing of Mined Process Models. 2014. 3888-3897.

45. Aalst, W.M.P.v.d., Process-Aware Information Systems: Lessons to be Learned from Process Mining, ed. X.A. The Pennsylvania State University CiteSeer. 2009.

46. Capriotti, R.J., Big Data Bringing Big Changes to Accounting. Pennsylvania CPA Journal, 2014. 85 (2): p. 3.

[...]

---

¹ ProM is an open source Software solution developed from the university of Eindhoven

² For Germany please refer to IDW Prüfungsstandard 261

³ International Standards of Accounting (ISA)

⁴ Defined as the ability to explain observed behaviour.

⁵ Refers to Occam’s Razor and follows the principle „as simple as possible

⁶ Goal is to avoid overfitting.

⁷ Goal is to avoid underfitting.

⁸ Please refer to 13 for more details about spaghetti models

⁹ Robert J. Capriotti is partner in KPMG LLP’s Philadelphia office and member of the Pennsylvania CPA Journal Editorial Board