Cortex Quickstart Guide

Welcome to the Quickstart Guide for Cortex, your streamlined pathway to mastering real-time data processing without the need for coding.

This guide is meticulously designed to introduce you to the core functionalities of Cortex, guiding you through creating and deploying your first Application with ease.

Whether you're new to data processing or looking to streamline your workflows, this guide provides the essential stepsa crash course to leverage Cortex's capabilities effectively.

1. What is Cortex?

Hosted on Selfuel Platform; Cortex is a cloud-native, no-code, production-grade Automated Real-time Event Processing Engine designed to revolutionize the way you manage and process data.

2. A Primer on Real-Time Processing

Cortex's real-time processing hinges on Streams, Events, and Attributes, forming the core of its event-driven framework.

• Streams orchestrate the flow of events, defined by chronological order and a unique schema that includes Attributes to ensure data consistency.

• Mappings bridge raw data to the schema, enabling coherent processing.

Within Cortex, Streams facilitate data exchange—ingesting through Source Nodes and outputting via Sink Nodes, while internal Stream Nodes allow for tailored data handling. This setup ensures seamless, real-time data processing across Cortex, optimizing event handling and analysis.

3. Capabilities and Target Audience

Cortex is a versatile platform that combines a suite of powerful capabilities tailored to a wide-ranging target audience, streamlining the creation and deployment of real-time data processing pipelines.

Capabilities Overview: Cortex excels in Streaming Analytics, Rule-based Decision Making, and offers extensive support for Backend Applications. It's equipped with Advanced Analytics for deep insights, supports comprehensive Workflow Automation, and enables Orchestration of complex processes. Additionally, Cortex integrates Streaming Machine Learning and facilitates real-time Data Integration and Synchronization, alongside providing robust Security features and Performance Monitoring tools.

Target Audience Spectrum: Cortex is designed for a diverse audience, from Coders and Non-Coders seeking no-code data pipeline development, to Data Scientists and Engineers needing scalable solutions. Business and Data Analysts, IT Managers, and Decision Makers across various sectors like Healthcare, Retail, and Manufacturing will find Cortex's real-time processing capabilities invaluable for operational efficiency and strategic planning.

With its broad capabilities and inclusive design, Cortex serves as a foundational tool for anyone looking to leverage real-time analytics and data processing to drive informed decisions and operational excellence.

4. More on Cortex

Cortex functions as an event-driven system where data is modeled as events, each detailed with unique Attributes that belong to Streams. Think of an event as a simple timestamped record, like a transaction log entry, capturing a specific change as it happens. Cortex is tailored to process these events effectively.

Cortex enables you to make use of your data with its core enablers such as

• Creating Applications using Canvas leverages the intuitive drag-and-drop interface, you can orchestrate various types of Nodes to manipulate and refine your data processing to perfection. (cf. Connector Nodes Cluster, Processing Nodes Cluster, Buffer Nodes Cluster)

• Highly configurable Runners that enable you to push your Applications from development to production seamlessly. With a few clicks you Create a Runner to Run Applications.

• Security within Cortex is paramount, featuring dual-layer encryption to safeguard your operations. Use any combination of Application-level Vaults, Runner-level Vaults and Security Certificatesthat suit your security requirements.

• Monitoring Performancefor Applications and Runners are user-intuitively separated into Application Details and Runner Details. You can monitor the latency, throughput, and events ingested of your Applications, and CPU, RAM, and Egress consumptions of your Runners in 1 minute intervals.

5. Creating Your First Application

Let's dive deeper into creating your first Cortex application with detailed, step-by-step instructions that guide you through each part of the process:

A. Set Up HTTP Source Node:

1. Access Applications Page: Click on the "Create Application" button.

2. Add Source Node: Drag and drop a Source Node under Connector Nodes Cluster into the Canvas.

3. Configure Source Node: Click on the settings icon on the Node Preview.

   • Step 1: Select "HTTP" as the Source Type; click "Next".

   • Step 2: Name the node "HTTP_source". Add an attribute named "first_attribute" with type "Float"; click "Next".

   • Step 3: Choose the mapping type as JSON; click "Next".

   • Proceed through steps by clicking "Next" until reaching Step 7 (Preview), then click "Complete".

B. Process Input Data:

1. Insert Query Node: From the Processing Nodes Cluster, drag a Query Node onto the Canvas.

2. Link Nodes: Connect it to the Source Node.

3. Setup Query Node: Enter settings via the Node Preview icon.

   • Step 1: Name it "simple_filter"; click "Next".

   • Step 2: Activate the filter toggle; to see the Expression Builder to writea filter. In the Expression Builder, click the "Edit" button to activate it. Input "first_attribute < 15.0"; click "Save". If incorrect, re-enter and save.

   • Complete Setup: Click "Next" to proceed to Step 3 and then click "Complete".

C. Process Output Data:

1. Add Processor Node: Drag a Processor Node from the Processing Nodes Cluster to the Canvas; connect it to the Query Node.

2. Introduce Sink Node: Drag and drop a Sink Node; connect it to the Processor Node.

3. Configure Sink Node:

   • Step 1: Choose "HTTP" as Source Type; click "Next".

   • Step 2: Name it "HTTP_sink". Add "filtered_attribute" and "doubled_attribute", both types "Float"; use the close icon at the top rigt to close the Sink Node and go back to Canvas.

4. Finalize Processor Node: Return to the Processor Node

   • Step 1: Name it "processor_node"; click "Next" twice.

   • Step 3: Use "Edit" buttons at bottom of the Expression Builder to map "first_attribute" to "filtered_attribute" and define "doubled_attribute" as "first_attribute * 2".

   • Complete Setup: Click "Next" until Step 6, then "Complete".

D. Finalize Sink Setup:

1. Re-open Sink Node Settings: Starting from Step 3, choose the mapping type as JSON; click "Next".

2. Step 4: Provide an Event Publishing URL. This URL will receive filtered_attribute and doubled_attribute within output streams.

3. Complete Setup: Click "Next" until Step 8 (Preview), then click "Complete".

E. Save Your Application:

• Save: Click the "Save" button at the top right of the Canvas; name your application.

Congratulations! You've now created and configured your first application in Cortex, ready for deployment using Runners.

6. Deploy Your Application

Deploying your application in Cortex allows you to operationalize it using Runners, leveraging your $150 $150 Free Trial credits for an XS Runner. Here’s how you can bring your application to production:

1. On the Applications Page, locate the "Run" button under the Actionscolumn for your Application and click it.

2. Step 1: You will be presented with two options: "Select an Existing Runner" or "Create Runner". Since you haven't created a Runner yet, opt for the "Create Runner" toggle.

   • The screen will extend to display Cost Management options for your Runner. Choose "Free Trial" and click the "Next" button to continue.

3. Step 2: Select the Resource Limits for your Runner. With the Free Trial, your only option is "XS". Click "Next" to proceed.

4. Step 3 requires you to give your Runner a unique name. You also need to decide if it will be "Fixed Capacity" or "Scalable". For the Free Trial, select "Fixed Capacity" and then click "Next".

5. Step 4: Now decide on the Runner Allocation Type. You can choose between a "Reserved" Runner, which hosts a single Application, or a "Shared" Runner for multiple Applications. Select according to your preference and click "Next" to move forward.

6. Proceed through Steps 5, 6, and 7 by clicking "Next", leading you to the Step 8 Preview. Review your configurations and click "Complete" to finalize the setup.

Upon clicking "Complete", two key actions occur:

• A Runner is created based on your specifications,

• Your Application is deployed onto this Runner, making it operational in a production environment.

This process enables you to transition your application from development to a live state efficiently, using Cortex’s streamlined deployment workflow.

7. Monitor Performance

In Cortex, Application performance is monitored through metrics like Latency, Throughput, and Events Ingested for each node, while Runner performance is evaluated using CPU, RAM, and Egress metrics.

Performance Metrics can be viewed on the Dashboard, Runner Details, or Application Details pages.

• Dashboard: View metrics for all applications and runners in one place.

• Application Details: Access specific application metrics via the "Application Details" button in the Actions column on the Applications Page.

• Runner Details: For runner-specific metrics, use the "Runner Details" button under the Actions column on the Runners Page.

You can view up to a week of historic data for each metric refreshed every minute so that you can precisely track and optimize of your Applications' performance.

8. Security

In this Quickstart Guide, we have not incorporated security measures. However, Cortex provides robust options for safeguarding your data.

• You have the capability to establish both Application-level Vaults and Runner-level Vaults, which encrypt sensitive credentials using decrypt-and-forget

• Additionally, you can also upload and utilize security Certificates to ensure the security of data-in-transit, offering an added layer of protection for your data processing activities.

9. Further Steps

For a deeper dive into the expansive capabilities of Cortex and to enhance your data processing strategies, consider exploring the following resources within our platform:

• Applications: To discover an extensive range of Nodes and delve into the specifics of Application Creation, navigate to the Applications section.

• Runners: Visit the Runners section to gain insights into deploying and managing your Applications efficiently.

• Cortex Elements: Familiarize yourself with the recurring elements that form the backbone of Application Creation in the Cortex Elements section.

• Security: Enhance the security of your Applications and data by implementing Application-level Vaults and Runner-level Vaults, along with utilizing security Certificatesfor data in motion.

• Switching to a Paid Plan: For those ready to elevate their Cortex experience, transitioning to a Paid Plan—be it Pay-as-you-go, Monthly, or Yearly Subscriptions—offers unparalleled flexibility.

Each of these steps offers a pathway to mastering Cortex, enabling you to customize, secure, and optimize your data processing endeavors for maximum efficiency and impact.