Easysearch

Easysearch source connector

Description​

Used to read data from INFINI Easysearch.

Key features​

• batch

• stream

• exactly-once

• column projection

• parallelism

• support user-defined split

Engine Supported

• Supported all versions released by INFINI Easysearch.

Data Type Mapping​

Easysearch Data Type	Nexus Data Type
STRING KEYWORD TEXT	STRING
BOOLEAN	BOOLEAN
BYTE	BYTE
SHORT	SHORT
INTEGER	INT
LONG	LONG
FLOAT HALF_FLOAT	FLOAT
DOUBLE	DOUBLE
Date	LOCAL_DATE_TIME_TYPE

hosts [array]​

Easysearch cluster http address, the format is host:port, allowing multiple hosts to be specified. Such as ["host1:9200", "host2:9200"].

username [string]​

security username.

password [string]​

security password.

index [string]​

Easysearch index name, support * fuzzy matching.

source [array]​

The fields of index. You can get the document id by specifying the field _id.If sink _id to other index,you need specify an alias for _id due to the Easysearch limit. If you don't config source, you must config schema.

query [json]​

Easysearch DSL. You can control the range of data read.

scroll_time [String]​

Amount of time Easysearch will keep the search context alive for scroll requests.

scroll_size [int]​

Maximum number of hits to be returned with each Easysearch scroll request.

schema​

The structure of the data, including field names and field types. If you don't config schema, you must config source.

tls_verify_certificate [boolean]​

Enable certificates validation for HTTPS endpoints

tls_verify_hostname [boolean]​

Enable hostname validation for HTTPS endpoints

tls_keystore_path [string]​

The path to the PEM or JKS key store. This file must be readable by the operating system user running Nexus.

tls_keystore_password [string]​

The key password for the key store specified

tls_truststore_path [string]​

The path to PEM or JKS trust store. This file must be readable by the operating system user running Nexus.

tls_truststore_password [string]​

The key password for the trust store specified

common options​

Source plugin common parameters, please refer to Source Common Options for details

Examples​

simple

Easysearch {
    hosts = ["localhost:9200"]
    index = "nexus-*"
    source = ["_id","name","age"]
    query = {"range":{"firstPacket":{"gte":1700407367588,"lte":1700407367588}}}
}

complex

Easysearch {
    hosts = ["Easysearch:9200"]
    index = "st_index"
    schema = {
        fields {
            c_map = "map<string, tinyint>"
            c_array = "array<tinyint>"
            c_string = string
            c_boolean = boolean
            c_tinyint = tinyint
            c_smallint = smallint
            c_int = int
            c_bigint = bigint
            c_float = float
            c_double = double
            c_decimal = "decimal(2, 1)"
            c_bytes = bytes
            c_date = date
            c_timestamp = timestamp
        }
    }
    query = {"range":{"firstPacket":{"gte":1700407367588,"lte":1700407367588}}}
}

SSL (Disable certificates validation)

source {
    Easysearch {
        hosts = ["https://localhost:9200"]
        username = "admin"
        password = "admin"
        
        tls_verify_certificate = false
    }
}

SSL (Disable hostname validation)

source {
    Easysearch {
        hosts = ["https://localhost:9200"]
        username = "admin"
        password = "admin"
        
        tls_verify_hostname = false
    }
}

SSL (Enable certificates validation)

source {
    Easysearch {
        hosts = ["https://localhost:9200"]
        username = "admin"
        password = "admin"
        
        tls_keystore_path = "${your Easysearch home}/config/certs/http.p12"
        tls_keystore_password = "${your password}"
    }
}