An AI Company’s View on Hyperautomation

From the point of view of the manufacturer of an AI operating system with native support of BPMN processes, hyperautomation is a logical step to expand the portfolio of the system’s application possibilities. In the article, we will take a closer look at the individual building blocks of hyperautomation.

Hyperautomation is a trend that was included by Gartner in its Top Strategic Technology Trends 2020 as well as 2021. The term describes an idea where everything that could be meaningfully automated in an organization should be automated. To do this, we need to create better automation capabilities by combining a variety of technologies. We understand hyperautomation as a combination of the following topics:

  • Business Process Management
  • Robot Process Automation
  • Artificial Intelligence

It is important to note that hyperautomation is not intended to take over all of a human’s work. This is neither technically possible nor desirable – especially with regard to safety-critical or sensitive workflows. Rather, automation frees people from repetitive tasks to focus on those that have higher value to the business.

The combination of automation and human engagement should help companies deliver a better customer experience while reducing operating costs and increasing profitability. If we use a combination of automation technologies, hyperautomation can overcome some of the limitations of individual approaches.

However, automation requires careful planning and implementation. Companies need to understand how digital technologies will fit into their existing workflows and what role they will play in new processes. With the right application of hyperautomation, a company can reap the following benefits:

  • Flexibility
    Due to the use of different technologies, a wide variety of automation tasks can be flexibly implemented
  • Improved employee productivity
    By automating time-consuming tasks, employees are able to do more with fewer resources and take on more valuable tasks in organizations.
  • Integration
    Hyperautomation enables companies to integrate digital technologies into their processes and legacy systems
  • Improved ROI
    Hyperautomation can help increase revenue and reduce costs, and increase productivity in general

To achieve scalability in operations, different automation technologies must work together seamlessly. Careful planning, implementation and improvement of processes is achieved through intelligent business process management (BPM). For these reasons, BPM is a core component of hyperautomation.

Business Process Management

Business Process Management (BPM) represents the foundation used to manage an organization’s hyperautomation strategies and initiatives. In general, BPM deals with improving a business process from start to finish.

A business process is an activity or set of activities designed to achieve a specific organizational goal. In this context, BPM is not a one-time task, but a continuous process that involves constant revision. This methodical approach can be represented in the form of a control loop.

BPM can also be applied if a company does not yet have any process descriptions, since at least “de-facto” processes, defined by the way tasks are processed, can be described. The BPM lifecycle contains several activities, the possible implementation of which is described briefly below:

Design

The design lifecycle step includes both the identification of existing processes and the design of new processes, with the goal of representing these processes in the form of a process flow.

In this context, process mining should be mentioned, which enables business processes to be reconstructed and evaluated based on digital traces in IT systems. Among other things, this makes it possible to model implicit and otherwise hidden process knowledge and thus make it usable.

The Task Force on Process Mining of the Institute of Electrical and Electronic Engineers (IEEE) defines three different types of process mining:

  • Discovery
    Based on existing (text) flow charts, processes are reconstructed and transformed into a process flow. In the following example, the letters A-I represent the process steps and the variables a1-f3 represent the number of process step runs. When creating a process flow, special attention should be paid to loops (See B-C) in the process flow. Either this is an error in the process flow or it is necessary to integrate a logic for the process decision into the process flow.
  • Conformance
    In this type of process mining, a model already exists in the form of a process flow. The existing data is therefore used to check the model for conformance.
  • Enhancement
    Here, too, process protocols and a model of the process are already available. In contrast to the conformance type, however, the process is not only checked for formal conformance, but also adapted and extended as required.

Modelling

The next step is to convert the process flow into a form that can be executed subsequently. For this we rely on the BPMN 2 standard. BPMN 2 is an open notation standard that is used to model business processes. The standard is widely used in business process management because it is easy to understand for business users and at the same time offers technical users the possibility to represent and implement complex processes.

In this context, our hyperautomation solution (AIOS) offers two ways to create a model. In the first option, a business process can be created using a visual designer. It would be just as possible to have the initial process created directly from the process flow. The following model represents the previously depicted process flow as a BPMN 2 model (assumption: f1, f2, and f3 have the same value and are therefore parallel):

As an alternative to creating using a visual designer, our system provides the ability to create BPMN processes using a Modelling DSL so that a developer can directly use a development environment. An example could look like this:

process("process_1") {
   noneStartEvent("startEvent_1")
   sequence("sequence1", "startEvent_1", "scriptTask_1")
   scriptTask("scriptTask_1", "Task 1") {
      script("atreus", "println('test')")
   }
   sequence("scriptTask_1", "endEvent_1")
   noneEndEvent("endEvent_1")
}

BPMN 2 offers a large number of syntactic possibilities. For example, manual steps can also be mapped with the standard using UserTask and ManualTask, so that non-automated or hybrid business processes can also be represented. Furthermore, the standard provides extension possibilities via extensions, which we make use of in the implementation of hyperautomation.

Execution

Due to the semantic expressiveness of BPMN 2, processes can already be modeled in such a way that they can subsequently be executed by a BPMN engine. The BPMN 2 engine of our hyperautomation solution AIOS is a proprietary development that has been optimized for performance, scalability, and reliability. The description of the BPMN engine would have enough material for a separate article. For this reason, I will only deal with it in a brief matter here. In general, the BPMN engine offers decisive technical advantages in contrast to alternative solutions. These are based on the following technical design decisions:

  • Graph database
    As we can see from a process flow, business processes are graphs. For this reason, business processes should also be represented in the form of graphs and not in relational tables as is usually the case with alternative solutions. This approach offers many advantages – including the direct use of graph theory. This is insofar helpful with complex business processes with inclusive or parallel gateways, since simply and elegantly the tokens of the parallel execution can be determined using a graph traversal.
  • Actuator model
    In computer science an actuator model is a model for concurrent computations or programs. These are divided into concurrent units, so-called actuators, which communicate exclusively by message exchange. Applied to the BPMN 2 engine, this means that the individual process steps are represented by actuators. The sequences between the process steps are implemented as asynchronous message exchange between the actuators.
  • Reactive programming
    In computer science, reactive programming is a declarative programming paradigm that deals with data streams and the propagation of changes. With this paradigm, it is possible to express static or dynamic data streams only, thus facilitating the automatic propagation of the data flow.

These technical choices made it possible to create a reactive system (according to the Reactive Manifesto). The Reactive Manifesto defines 4 principles that make up such a system:

Responsive
The system responds on time whenever possible. Responsive in this context means that problems can be identified quickly and dealt with effectively. Responsive systems focus on providing fast and consistent response times and setting reliable upper limits so that they deliver consistent quality of service.

Resilient
The system remains responsive in the event of a failure. Resilience is achieved through replication, containment, isolation, and delegation. Failures are contained within each component, isolating components from each other, ensuring that parts of the system can fail and be recovered without compromising the system as a whole. Recovery of each component is delegated to another component, and high availability is ensured through replication when needed.

Elastic
The system remains responsive as the workload changes. Reactive systems can respond to changes in input rates by increasing or decreasing the resources allocated to service those inputs. This implies designs that have no conflict points or central bottlenecks, leading to the ability to split or replicate components and distribute inputs among them.

Message-driven
Reactive Systems rely on asynchronous message-passing to establish a boundary between components that provides loose coupling, isolation, and location transparency. This boundary also provides the ability to delegate errors as messages. The use of explicit message-passing enables load management, elasticity, and flow control by monitoring message queues in the system and exercising back-pressure when necessary.

Monitoring

The processes to be executed are continuously monitored and visualized through a dashboard. The information required for this can be retrieved from the system at any time in real-time so that third-party systems can also access this information. This data forms the basis for the subsequent optimization of processes and can also be used as a basis for process mining.

In addition to the collection of data, the analysis of this information is also in the monitoring step of the BPM lifecycle. For the evaluation of the business process data, the full potential of the AIOS can be exploited in terms of analysis using AI.

In general, the actual situation of the company is determined during the monitoring of the processes to be able to read out bottlenecks or information about the utilization of resources in a further step.

Furthermore, possible weak points, as well as improvement potentials, are identified. This can be achieved, among other things, by comparing the actual data with the previously modeled target specifications. If a defined threshold is undershot, the system can flexibly perform any action, such as sending the information via e-mail.

To be able to carry out a target/actual comparison, certain parameters must be determined. This can be, for example, the total runtime of a process or the total costs resulting from the use of all resources involved in the process. However, the AI Operating System can also be used to flexibly record other metrics. For this purpose, we have created a BPMN extension in the form of a Notification Intermediate Event. With the help of this intermediate event, it is possible to create and measure any notifications.

This means that measurable and evaluable data or processes can be used to monitor and evaluate business processes on the basis of such target/actual comparisons. In this way, average values and any deviations can be determined. Subsequently, reports are generated from the determined data, which point out the weak points.

Optimization

Based on the findings of the monitoring, improvement measures, as well as specifications for the design of new business processes, can be derived. For optimization of processes, solutions can be designed and tested through a simulation. The possibilities of optimization prove to be versatile. Suggestions for optimization could be

  • Combination of several business processes into one
  • Automation of further subtasks
  • Avoidance of unnecessary process steps
  • Improvement of the data structure

Following optimization, the findings flow back into the design phase in a cycle, thus closing the loop. As we can see from this, business processes run through the BPM lifecycle so that they can be continuously improved and adapted.  This is because progress can only be measured and processes continuously improved if all activities of the BPM lifecycle are run through.

A key aspect here is consistency, which means that the individual activities of the control cycle should be able to be integrated without significant breaks in operation. This is a prerequisite for making flexible adjustments, extensions and optimizations to processes. Since AIOS maps all activities of the BPM lifecycle, this requirement is met.

By applying BPM, you have the foundation for automating any business process. For most business processes, it will be necessary to be able to interact with third-party systems. For this, the hyperautomation theme relies on Robot Process Automation.

Robot Process Automation

Robot Process Automation has become one of the fastest growing enterprise technologies in recent years. Basically, RPA is the ability to control repetitive and rule-based tasks in an automated manner via an API or directly via the presentation layer.

The latter is propagated by many RPA providers, as the simulation of a user on a software interface can be implemented quickly in theory. In practice, however, implementation does not look so easy, as it is often not clearly communicated what an RPA bot can do.

Common marketing campaigns also suggest that the involvement of IT is not necessary due to the simplicity of RPA. These false expectations can then, under certain circumstances, also lead to RPA projects failing. In particular, the widespread claim that RPA can be implemented quickly and easily without coding does not seem ideal in this context.

After all, implementing and sustaining large-scale RPA solutions remains a challenge. First, even RPA solutions for automating complex workflows require a not inconsiderable amount of coding or configuration work. Second, UI-based RPA bots are tightly coupled to the user interface. Therefore, even minor adjustments to the user interface can cause them to fail. Also, creating RPA automation via a screen recording tool does not always prove practical in reality.

For simpler processes, screen recording can be used to good effect in marketing terms. However, it is usually the case that the processes to be automated have a more complex form. With more complex processes, creating an RPA process using screen recording becomes cumbersome.

This can already be demonstrated with a process that seems simple at first glance. The following process executes “Task 1” at the beginning of the workflow. Depending on the output from this task, it is decided whether the upper or the lower path should be continued through the process. If we were to record this process, we would have to make multiple recordings or re-code the decision logic.

We have deliberately introduced a distinction between API and UI automation when automating tasks in the hyperautomation system. Tasks that can be automated via an API are implemented via a standard BPMN ScriptTask or via a Task Extension, the Connector SkillTask. While in the ScriptTask a script is executed in AIOS’ own programming language, in a connector skill different connectors for systems can be created via the skill abstraction (as a kind of microservice).

The call to an external REST interface could look like this:

httpPost("https://domain.com/api/") // <1>
    .header("Content-Type", "application/json")
    .header("api-secret", "top_secret") // <2>
    .body(mapOf("param1": 1)) // <3>
    .execute() // <4>
    .map(response -> $response.abc) // <5>

<1> Initialization of the request as POST request

<2> Setting HTTP header information

<3> Setting HTTP body information

<4> Execution of the request

<5> Transformation of the http response

For the automation of applications via the user interface, we have extended the BPMN standard with an RPATask. Since mostly a developer is needed for the automation of complex workflows, it naturally makes sense from Leftshift One’s point of view to enable him or her to work in his or her usual working environment.

Unlike the ScriptTask, the script is not executed directly on the system, but on an RPA server that runs directly on the machine to be automated. With the RPATask, the AIOS connects to the RPA server and forwards the automation instructions to it. The business process in which the RPATask has executed waits in the WAITING state until a response is received from the RPA server or a timeout occurs.

When creating the automation instructions, we greatly simplified the complexity of Robot Process Automation by creating our own Domain Specific Language (DSL) so that no programming knowledge is required. Nevertheless, developers can create and test these DSL scripts within the familiar development environment.

The DSL is designed to handle both native OS applications and browsers. Here, we use the CSS Selector syntax for the abstraction of the elements to be automated in order to be able to control both web and native OS controls.

For the identification of control IDs in Windows applications, the Accessibility Insights for Windows tool can be used, for example. A simple example of automating a Calculator application could be designed as follows:

launch("calc") { // <1>
    window("Rechner") { // <2>
        click("#num3Button")  // <3>
        click("#plusButton")  // <4>
        click("#num5Button")  // <5>
        click("#equalButton") // <6>
        send(mapOf("result", read("#CalculatorResults"))) // <7>
    }
    close("Rechner") // <8>
}

<1> Launch the Calculator application

<2> Focus the new window

<3> Click on the “3” button

<4> Click on the “+” button

<5> Click on the “5” button

<6> Click on the button “=”

<7> Send the result to the AIOS

<8> Close the window

However, DSL can be used to automate more complex workflows:

launch("App123") {
    window("App") {
        click("#buttonA")
        input("#inputA", "text 123")
        click("#buttonEnter")
        
        if (read("#textField").startsWith("DEV")) {
           // do something dev specific
           send(mapOf("result" to "dev"))
        }
        if (read("#textField").startsWith("BIZ") {
           // do something biz specific
           send(mapOf("result" to "biz"))
        }
    }
    close("App")
}

The DSL can also be used to interact with multiple applications.

launch("App1", "App2", "App3")) {
    window("App1") { // do something with app1
        set("variableName1", "value from app1")
    }
    window("App2") { // do something with app2
        set("variableName2", "value from app2")
    }
    window("App3") { // do something with app3
        set("variableName3", "value from app3")    
    }
    close("App1")
    close("App2")
    close("App3")
}

RPA is a very interesting technology that can quickly lead to added value when used correctly. With the combination of RPA and AI, we can expand the range of possible automation solutions so that even more complex use cases can be implemented.

However, one must also be aware of the effort involved in implementation. This is because, contrary to popular belief, an RPA system is not maintenance-free, especially for more complex workflows. Also, the expectation that an RPA system can be used as a replacement for any human labor is usually disappointed.

Artificial Intelligence

One limitation of RPA is that it is limited to structured data to complete tasks. Therefore, RPA does not have the ability to understand or learn context, nor can it access and make sense of unstructured data sources such as images.

For these reasons, hyperautomation needs the strategic use of artificial intelligence. Using the combination of RPA and AI further increases the scope of automation capabilities by allowing AI to take on cognitive tasks such as interpreting a text, audio, images, etc.

With the help of automated information recognition along with semantic context, one can make more complex business decisions. The expectation that all human work can be automated remains a dangerous fallacy. Often, when describing AI, one is confronted with texts like the following:

“AI has become an umbrella term for technology that helps software think and act like humans, encompassing everything from cognitive computing and optical character recognition to natural language processing.”

It is easy to understand that many do not agree with such a definition, as it tries to create an analogy between human and artificial intelligence.

In fact, a great deal can indeed be implemented by means of AI. In part, the possibilities of AI, in strongly limited areas, also exceed human capabilities. But to equate the cognitive abilities of AI with those of humans does not hold up as a premise by a long shot.

In principle, the fact is that we cannot describe the concept of intelligence precisely. That’s why we should view artificial intelligence as a different form of intelligence and not constantly compare it to human intelligence.

Especially when sensitive and critical workflows should be automated, a human employee must still be integrated into the process at least as a supervisor.

AI is a powerful automation tool, but its implementation requires a significant investment of resources and careful planning to ensure integration with other technologies and processes.

As the operating system for AI, AIOS is predestined to make this point in the hyperautomation space. In this context, the system supports the implementation of AI solutions – starting with the collection of data, through the training of models, to deployment and execution, as well as monitoring and further development.

The executable AI solutions within the system are called “Skills”. A large number of predefined Skills can already be obtained from the AIOS integrated marketplace (“Skill Store”) and seamlessly integrated into a workflow. As mentioned before, we have extended the BPMN standard with a SkillTask for this purpose. The following example demonstrates how AI and RPA can interact in a workflow:

The BPMN process shows a business process where a document (image or PDF) is automatically processed by extracting the text from the document and subsequently persisting it in external systems. For example, the following AI functionalities are included in this concrete process:

Natural Language Processing (NLP)
NLP enables machines to semantically understand unstructured data from e.g. emails or social media posts in order to subsequently solve a task. This involves analyzing the structure and meaning of human language by automatically processing various aspects such as syntax and semantics. For example, the text in the business process shown could be assigned to a class (classification).

Optical Character Recognition (OCR)
OCR is a technology for recognizing the characters of text in images such as printed books, photographs or scanned documents. In the example, OCR can be used to extract the text from images or PDF files that contain images, which can then be processed by the NLP task.

In summary, we can say that hyperautomation is the next step in the automation of business processes. The integration of different technologies makes it possible to automate activities that previously could not be automated through the individual components of hyperautomation, or could only be automated in a cumbersome manner. The integration of AI in particular greatly increases the possibilities of automation, since, among other things, context-sensitive decisions can also be automated. With the right integration and expectations, hyperautomation proves to be an exciting technology that can be used very flexibly in a company.