Process mining is a knowledge-extraction technique based on the analysis of log files. This method allows us to analyze business processes and discover where there is potential to improve them.
What is process mining?
Process mining comprises a set of techniques belonging to process management and intended to analyze business processes. These methods use information and data recorded in event logs to carry out an after-the-fact analysis of the actual process. Process mining tools build on a series of specific algorithms drawn from data mining which, applied to the log files and the variables obtained, make it possible to identify trends and patterns. The ultimate goal is to improve the understanding of the business processes under study in order to optimize them and make them more efficient.
Types of process mining
In research papers, the term process mining is sometimes referred to as “automated business process discovery” (ABPD). This concept describes a set of techniques used to create, evaluate and extend process models. The manifesto of the IEEE Task Force on Process Mining distinguishes three types of process mining techniques:
Discovery: this type of process mining technique is used to identify processes and create process models.
Conformance: through this type of process mining technique, the data from event logs and existing process models are compared to determine whether they match.
Enhancement: these process mining techniques consist of adapting and improving processes based on real data.
The IEEE Task Force on Process Mining is a research group of the Institute of Electrical and Electronics Engineers (IEEE) at the Eindhoven University of Technology created to promote the development and application of process mining through research and education activities.
IT systems store information about events that happen in the real world in the form of logs. Process mining techniques use this data to build, verify and improve process models
How does process mining work?
Process mining combines techniques belonging to data mining and computational intelligence (CI) with process analysis and modeling. A process is described as a logical sequence of interrelated activities or tasks, which can be recorded as events.
The starting point of any process mining technique lies in the data contained in the logs, which shows the events in chronological order. These, in turn, can be classified into process stages or process instances.
In general, the concept of “process” refers to a business process in planning terms. An instance, on the other hand, is the particular execution of that process. Process instances can be determined individually taking into account parameters such as time, location, or the people and devices involved. For example, an application for a life insurance policy submitted to an insurance company would be considered a process. The handling of this application by Mr. Martínez would be an instance of the previously established standard process.
The IEEE has created a standard scheme for each type of process mining.
Discovery-type process mining techniques take an event log and produce a process model without using any a priori information, solely with the help of process mining algorithms. They use the event logs of the IT systems being used in organizations.
Discovery-type process mining techniques use log files as input and produce a model as output
The result of this type of process mining is generally a process model. A model of this kind could be derived, for example, from a factory in which products are marked with a timestamp to determine at what point they pass through each of the production phases.
The main representation techniques applied to process models are:
BPMN (Business Process Model and Notation)
EPC (event-driven process chain)
Follow-up plans
HIPO diagrams
Communication structure analysis
Petri nets
Semantic object models
UML (Unified Modeling Language)
BPEL (WS-Business Process Execution Language)
Process mining techniques are not necessarily limited to discovering, validating and improving process models; they can also be used to represent corporate structures and create organizational charts, business rules or guidelines.
Conformance-type process mining techniques are used to validate process models. If a process model is already being applied, it is advisable to compare it from time to time with the new data coming from the event log to verify that it corresponds to the actual processes. These process mining techniques are used to compare an existing model with current event data in order to determine the deviations between the model and reality. The result of these conformance checks will be a diagnosis from which we can draw conclusions about the quality of the process model we have analyzed. Conformance tests can be performed on both normative and descriptive process models.
Descriptive models serve to describe processes that are actually being executed. Normative models provide information on how a process should be executed in the best-case scenario. One also speaks of actual models and desired or target models.
Conformance-type process mining techniques use the existing process model and the current log files as input and generate a diagnosis as output, showing the extent to which the model matches the event data.
Enhancement-type process mining techniques aim to develop and improve existing process models by making use of the information discovered. The result is a new, extended process model.
Enhancement-type process mining techniques use the existing process model and the current log files as input and generate a new process model as output.
Analytical perspectives
Process mining encompasses four different levels of observation:
Control-flow perspective: process mining focused on control flow, which seeks to represent a sequence of activities within a process that forms a process model (for example, as occurs in Petri nets, the UML activity diagram, the EPC or the BPMN model).
Organizational perspective: process mining based on the organizational aspect, which highlights the way people and IT systems relate to each other when participating in a business process. Activity profiles and roles are defined and then compared. The result of this type of analysis is a social map in which a network of relationships can be visualized.
Case perspective: process mining focused on specific cases, which serves to analyze individual process instances. These are described and categorized as cases according to their properties. To make the classification, the data values recorded for each relevant process instance are used, taking into account, for example, the parties involved.
Time perspective: process mining focused on time, which studies the absolute and relative time and the frequency of events. For this to be possible, all event records must be given a timestamp. Thanks to this type of analysis, it is possible to run simulations from which conclusions can be drawn about the patterns, trends and obstacles of the process flow. For example, bottlenecks within the process chain can be identified.
In practice, process mining is nowadays mainly used to detect control flows. The most widely used process mining techniques are Discovery-type techniques focused on control flows. Thanks to them, it is possible to identify chronological and logical sequences of the individual stages of a process and then compare them with the desired state.
Phases of process mining
The IEEE has developed the L* life-cycle model as a reference for the application of process mining techniques. This model divides process mining projects into five phases:
Phase
Procedure
Planning and classification
According to the L* life-cycle model, process mining projects begin with a planning phase.In this phase, the following questions are answered:- Which process do we want to study?- What are the important events?- What are the most relevant indicators?- Which actors and IT systems are involved?- How can we obtain the data we need?- What are the goals of the process mining project?
1
Extraction of relevant data
After the planning phase, the relevant data is extracted from the available IT systems:- Log files- Models- Others
2
Creation of a control-flow model
In phase 2, a control-flow model is created from the data obtained and linked to the log files.
3
Creation of an integrated model
If sufficient data is available, the model created in phase 2 can be extended by introducing new perspectives in phase 3.
4
Operational support
In phase 4, the model is used to support operational processes.
In which cases is process mining used?
Process mining can be used whenever there is information on the individual steps of business processes that is detailed, recorded and stored on an ongoing basis by IT systems. It serves companies, for example, in the following cases:
Development of work processes using workflow management systems
Execution of transactions through ERP systems
Management of technical support requests through ticketing systems
Quality assurance of medical treatments through clinical protocols
Therefore, process mining techniques can be used in fields as diverse as retail, original equipment manufacturing, banking, insurance, development or sales departments in order to improve business processes such as orders, production or cash flows.
The main fields of application of process mining techniques are workflow management and knowledge management. In addition, the knowledge extracted through process mining is used in the development of assistance systems.
Countless companies use technologies such as databases, ERP systems or knowledge management systems to discover and make use of information. Normally, processes are not analyzed directly: thanks to the methods used by process mining, it is possible to extract explicit knowledge from implicit processes.
Workflow management systems describe business processes from a normative point of view and automate the coordination and control of the individual steps of each process. The system provides the user with interfaces to carry out communication tasks and access data and programs. Workflow management is based on modeled workflows that allow the system to recognize events (such as receiving a document by email) and react to them automatically. This automated procedure is based on process models that can be created, monitored and improved thanks to process mining methods.
What advantages do process mining techniques provide?
Process mining techniques can be applied whenever business processes and each of their phases are recorded. Thanks to algorithms from the fields of data mining and computational intelligence, it is possible to analyze increasingly complex event data and draw conclusions on how to improve business processes and make them more efficient and secure.
What distinguishes process mining from other more classical techniques used in the creation of process models is its high degree of automation. As the information is obtained from real events occurring in a running business, the methods used in process mining are a faithful reflection of process flows. Compared with manual techniques, process mining is better in terms of speed and accuracy. Moreover, because an ever-larger volume of data is being handled, it is no longer possible to continue managing it manually.
Another advantage of process mining applications in professional environments lies in the wide range of visualization options available. Process models are presented to employees and managers through interactive dashboards that allow process flows to be viewed dynamically and, in many cases, include additional analysis tools.
Challenges when implementing it
Many companies encounter difficulties when implementing process mining techniques when the body of data to be analyzed is inconsistent because it comes from a heterogeneous IT infrastructure. If there is no uniform description of the events, the log files must first be prepared. This entails not only additional effort but can also end up corrupting the data, meaning it no longer serves as real data.
In addition, companies have to deal with technical obstacles during implementation. The use of data mining is only effective if all applications have access to all the relevant IT systems. This implies the need for appropriate interfaces and a complex configuration of the connected systems, which generally requires close collaboration with the provider of the process mining application.
The implementation effort increases even further if companies have to combine standard business process management applications with tools they have developed themselves to meet their individual needs.
