Cortex Elements

Cortex Elements form the core components and utilities that users will encounter frequently while navigating through Cortex. These Elements are crucial for understanding how to maximize the platform's capabilities without redundancy in our User Guide.

Applications, Canvases, and Nodes

• At the core of Cortex are Applications, empowering users to construct no-code data pipelines on intuitive Canvases. (cf. Creating Applications using Canvas)

• The essence of these Canvases' no-code feature lies in the ability to drag and drop various Nodes.

• Nodes, the fundamental building blocks of Applications, are organized into three main clusters for streamlined development:

  • Connector Nodes Cluster comprising of Source Nodesand Sink Nodes for diverse data sources and configurations

  • Processing Nodes Cluster comprising of Query, Join, Pattern, Sequence, Processor, and On-demand Query Nodes

  • Buffer Nodes Cluster compsring of Stream, Table, Window, Aggregation, and TriggerNodes

Streams, Events, and Attributes

In Cortex, the concept of Streams are pivotal to its event-driven architecture, designed to meticulously process flows of events.

Not only Streams carry events into and out-of Cortex, but also maintain data processing logic between Nodes within every Application.

• Streams are defined as a logical sequence of events, each chronologically ordered and collectively identified by a unique name.

• Beyond mere identification, each Stream is characterized by a defined schema—comprising a set of Attributes that dictate the structure and nature of the data it carries.

• Integral to this schema is the concept of Mappings, which serve as the bridge between the raw data of events and the structured expectations of the Stream's schema.

Every Streams definition, encompasses the Stream name and its Attributes, establishes a clear and consistent framework for data handling within Cortex via different Mappings. Using Streams

• You can consume and publish events into and out-of Cortex using #streams-entering-and-exiting-cortex via Source Nodes and Sink Nodes.

• You can create #internal-streams-with-stream-nodes within Applications to process event data the way you need.

By adhering to this structured approach, Cortex ensures that all events associated with a Stream share the same schema, facilitating streamlined and coherent event processing across its Nodes.

Creating Applications

Creating Applications using Canvas involves connecting these Nodes following specific connectivity rules, enhancing user experience by guiding permissible connections and simplifying Application development. These Node Connectivites are categorized into:

1. Nodes Connectable as Input and Output: Automating the connection process based on compatibility, eliminating guesswork for users.

2. Minimum and Maximum Connections: Predetermined input and output connection limits for each Node, automatically enforced to streamline configuration.

3. Node Units: A concept introduced to expedite Application development by utilizing a set of three consecutive Nodes (Node Units) to enforce connectivity rules efficiently, ensuring every Application functions as intended.

All the rules above are embedded to Canvas UI, so you need not memorize anything.

Node Preview

Node Preview serves a triple purpose:

• Offering a quick overview of Application flow

• Displaying vital Node-specific details

• Providing buttons for Node configuration and deletion.

Healthchecks

Node and Application Healthchecks ensure the integrity of each Node and the overall Application, respectively, by verifying configurations against the minimum requirements necessary for successful operation.

Expression Builder

Expression Builder aids in custom data processing through Functions or Filters, featuring a built-in 6. Automatic Validation to ensure error-free expression creation. This tool is vital for Nodes requiring custom logic, enriched by Cortex's library of Built-in Functions.

Windows

Multiple Windows can be created either as reusable entities via Window Nodes or on-the-fly within Query Nodes, catering to varied data windowing needs with multiple types and settings.

Store and Cache

Store and Cachefunctionality allows for external storage tables and aggregation tables within Tableand AggregationNodes, with mulitple Store options for data manipulation outside Cortex. Also an optional Cache mechanism can also be configured to minimize latency during database interactions.

By delineating these Elements, we aim to equip users with a clear understanding of Cortex's foundational components, ensuring an efficient and informed experience throughout the platform.