Enterprise Event

Executive Abstract

Enterprise Event is a concept consisting of Enterprise Customer Event (ECE) and Enterprise System Event (ESE). Enterprise Customer Event enables any business to capture all its customers’ interaction with its business to analyze customers’ needs, usage, business alignment, business trend, and gain a competitive advantage over competitors. Enterprise System Event handles all internal systems’ telemetry, defect, and normal events for production health monitoring, support, and normal business event processing and delegating to other systems.

Introduction

Enterprise Event is a concept that consists of two different ideas, such as the Enterprise Customer Event and the Enterprise System Event. The Enterprise Customer Event consists of two parts. The Customer Event Capture provides a RESTful API entry point for all facing customer systems to connect and delegate events to store in database. The Customer Event Processor reads the events from the database and does various analyses, syntheses, and statistical reporting about customers’ business activity, business segment usage, feature usage, type of users, when and where users use the systems.

The Enterprise System Event is ideal for handling telemetry, defect, and normal events. Telemetry events such as (enterprise) system usage, health monitoring/metrics of issues, and others. Defect events such as error, warning, and tracing. It has an Inference Rule Engine and Rule Engine to support various types and scenarios of events. The System Event Capture provides the RESTful API entry point for other systems to interface. The System Event Processor reads the events from database, processes the data, and renders the results.

The Enterprise System Event can also be used to send normal business events to other systems using the Inference Rule Engine and Rule Engine. The object model should be the same with the addition of payload.

Business Needs

Business exists because there are needs of its customers or clients. Fulfilling exact customer’s needs at the right time, the agility to change along with customer new demands, and the capability and ability to predict the needs and directions of customers, and the power to stay ahead of its competitors are critically essential to sustain and be successful in business.

Enterprise Customer Event provides the capability to analyze, synthesize, and visualize customers’ business activities, usage, needs, and trends through crosscutting all systems that external customers interact to.

The Enterprise System Event is used for crosscutting all internal systems to create a centralized monitoring system to monitor all systems’ health, activity, interaction/telemetry, tracing off an event or related events through correlation ID. Through active health monitoring, production issues and production downtime would be less. The framework can also be used as a catalyst to receive, process, and delegate normal business events to other systems. Below are high-level architecture diagrams of Enterprise Customer Event and Enterprise System Event.

Figure 1.0 Enterprise Customer Event (ECE) showing various external customers using their devices/systems to interface to a company’s internal systems to process their contextual business activities with the addition of interjected ECE to crosscut all customer facing systems.

Figure 2.0 Enterprise System Event (ESE) showing various internal systems within a company processing contextual events with the addition of interjected ESE. ESE can also be used as a normal event receiver, processor, and delegator.

Figure 3.0 High Level Usage of ECE and ESE Event Captures, Processors, and Delegator.

Figure 4.0 ECE Event Flow of stages for ECE Capture and ECE Processor. The Analysis and Synthesis layer may have artificial intelligence capability.

Figure 5.0 ESE Defect and Telemetry Flow of stages for ESE Event Capture. The Health Monitoring part is another RESTful API to read events from database, process, and render to users.

Figure 6.0 Key significant stages of ESE to receive, process, and delegate events.

Figure 7.0 Detail Event Flow of ECE/ESE with Parallel/Multi-thread and Sequential/Single thread. The use of parallel/multi-thread can be scaled vertically in a single microservice. ESE microservice can also be scaled horizontally via multiple instances depending on the demand.

Figure 8.0 ECE Significant key components with input events from external systems.

Figure 9.0 ESE Significant key components with input events from internal systems.

Implementation

The implementation vision is to start small, correct foundation, depth approach, simplification as oppose to complication, and leverage as much as possible existing tools, technologies, frameworks, best practices, and standards. ECE and ESE are ideal to be implemented as event-driven microservices. ECE and ESE each would need to have two microservices.

The ECE RESTful API microservice accepts incoming events, originally from customers, and persists in Cassandra database. Its counterpart, ECE Processor microservice, pulls these events from database and does statistical analyses on similar events and synthesizes these various events to derive knowledge and meta-knowledge for customer business insight, strategic business alignment, opportunity, gap and risk analyses, and gain a competitive advantage over competition.

The ESE RESTful API microservice accepts incoming events generated from various systems within a company and persists in the Cassandra database. An event can be a normal business notification/message between components/systems, or it can be a defect such as an error/warning event. The ESE Processor microservice reads these events to perform analytics and metrics for systems health, event tracing, and other preventions.

Both ECE and ESE can be integrated into, or implemented using, existing framework and features, Consistent Hashing, various network protocol adapters, Spring Integration, and Spring Cloud. They can be implemented as Spring Boot RESTful web services.

Both ECE and ESE are expected to have a high volume of events; therefore, multithread/parallel processing is required. However, it is planned to use Executor Service with properly defined number of threads in a thread pool to limit the number of threads being created to prevent an exponential thread explosion.

Inference Rule Engine

The purpose of Inference Rule Engine is to specify which rule(s) in the Rule Engine to execute based on the content-based contextual event with various criteria such as origination, destination, type/category of event, frequency, threshold, day of the week, date, and others. These criteria could be pulled from database or csv spreadsheet from repository server.

Inference Rule Engine and Rule Engine are agnostic. Both are related to each other by knowing the rule names only.

Rule Engine

Rule Engine is to pull rules from the database or csv in repository server based on the given rule names and run these rules. The rules can be implemented in Groovy and stored in database.

Both Inference Rule Engine and Rule Engine should be loaded into memory cache for best performance.

Business Value

ECE and ESE provide business values as shown below.

ECE Business Value

ECE provides vast business values. Some of them are shown below.

Insight of customer business need and trend
Visibility of customer business segment usage
Gain competitive advantage over competition
Opportunity
Accurate alignment with business needs
Better understanding of customer business activity
Improve customer experience

ESE Business Value

ESE provides many business values as listed below.

Early sign of issue via telemetry
Visibility into potential production issue
Centralized monitoring all systems health
One place to trace an event through various systems
Once place to trace related events through correlation IDs
Less production impacts
Early detection and faster correction of defects

ECE Event Data

The Enterprise Customer Event data contains the follow elements. Additional elements can be added.

<ECE>

<SOURCENAME>Source Name</SOURCENAME>

<SOURCETYPE> Source type</SOURCETYPE>

<EVENTTYPE> Event Type </EVENTTYPE>

<EVENTID> Event ID </EVENTID>

<CORRELATIONID> Correlation ID </CORRELATIONID>

<CATEGORY>Category</CATEGORY>

<SUBCATEGORY>sub-category</SUBCATEGORY>

<EVENTDESC> Event Description </EVENTDESC>

<EVENTDATE> DateTime</EVENTDATE>

</ECE>

ESE Event Data

The Enterprise System Event data contains the follow elements. Additional elements can be added.

Telemetry/defect event

<ESE>

<SYSTEMNAME>System Name</SYSTEMNAME>

<SUBSYSTEMNAME>Sub system Name</SUBSYSTEMNAME>

<SYSTEMTYPE>System Type</SYSTEMTYPE>

<SUBSYSTEMTYPE>Sub System Type</SUBSYSTEMTYPE>

<EVENTTYPE>Event Type</EVENTTYPE>

<EVENTID>Event ID</EVENTID>

<CORRELATIONID>Correlation ID</CORRELATIONID>

<CATEGORY>Category</CATEOGORY>

<SUBCATEGORY>Sub Category</SUBCATEGORY>

<EVENTDESC> Event Description </EVENTDESC>

<EVENTDATE>Event Date</EVENTDATE>

</ESE>

Delegation event – payload

<ESE>

<SYSTEMNAME>System Name</SYSTEMNAME>

<SUBSYSTEMNAME>Sub system Name</SUBSYSTEMNAME>

<SYSTEMTYPE>System Type</SYSTEMTYPE>

<SUBSYSTEMTYPE>Sub System Type</SUBSYSTEMTYPE>

<EVENTTYPE>Event Type</EVENTTYPE>

<EVENTID>Event ID</EVENTID>

<CORRELATIONID>Correlation ID</CORRELATIONID>

<CATEGORY>Category</CATEOGORY>

<SUBCATEGORY>Sub Category</SUBCATEGORY>

<EVENTDESC> Event Description </EVENTDESC>

<EVENTDATE>Event Date</EVENTDATE>

</ESE>

Event Persistent Storage

Cassandra database is ideal for this event processing. Other relational persistent storage such as Oracle database can also be used. At this point, the event data are simple and at the same level; therefore, the object model should virtually match with its entity model except for row ID. For storage of the results of analytics, metrics, and others that have multi-level hierarchy and dependency of relationships to other tables, database normalization should be used up to third normalization and nothing more nothing less to balance performance. There would require two key spaces, one for ECE and the other for ESE.

Database Models

Below are the entity models for ECE and ESE.

ECE Model

Below is entity model which currently matches with its object model.

ESE Model

Below is entity model which currently matches with its object model

A particular event processor, based on the contextual event and business logic, determines which inference rule to invoke.

Inference Rule Model

The RULE1, RULE2, to Rule N are conditions that must be met before it retrieves and runs the rule name(s) to the Rule Engine. Conditions could be origination, destination, event type, event category, frequency, threshold, date, day, and others.

Rule Engine Model

Below is entity model for Rule Engine. Rule is written in Groovy.

ECE and ESE Processors

ECE Processor reads ECE events from Cassandra ECE key space and performs analyses and syntheses of these various events to derive knowledge and meta-knowledge for the usage of business segment, type of customer (traveler), feature, and others.

ECE Processor is implemented in Java. Specific feature can use different programming language such as Python, R, or others. There is a good place that an artificial intelligence component can be implemented to consume this vast amount of data and do various statistical, analytical, and synthetical processing and persist results in database for the UI component to render.

ESE Processor reads ESE events from Cassandra ESE key space and performs analyses and metrics for health and event tracing. ESE Processor and its features should be implemented in Java couple with other programming languages as appropriate.

Application

ECE is applicable and beneficial for any company that has various types of customers interacting with its systems. It is also equally beneficial for a company that has only one type of customer with various products or services. A company still needs to know how customers experience their products or services in order to provide an exceptional quality of service and better products. A company needs to know some information from the customers as following: