Skip to main content

Integration

The Integration page (/admin/integration/instance) manages connectors that import and index content from external sources — such as AEM, web crawlers, or other content repositories — into the Turing ES search engine. It is accessible from the Enterprise Search section of the sidebar.

Each Integration instance represents a configured connection to a content connector application. The connector runs as a separate process (for example, Viglet Dumont DEP) and communicates with Turing ES via REST. Turing ES acts as a proxy: the API path /api/v2/integration/{integrationId}/** forwards requests to the configured connector endpoint, with built-in SSRF protection.


Instance Listing

The listing page shows all configured Integration instances as a grid of cards, each displaying the integration's title and description. A button at the top of the page opens the creation form.


Creating or Editing an Integration

The creation and edit form is divided into two sections.

General Information

FieldDescription
Integration NameHuman-readable identifier for this integration (required)
Short DescriptionOptional description of the integration's purpose

Connection Details

FieldDescription
Integration TypeConnector type — AEM or Web Crawler
EndpointURL of the connector application
EnabledToggle to activate or deactivate this integration

Integration Detail — Sections

After an integration is created, its detail page provides navigation to several sections. The sections below are common to all connector types. For AEM-specific configuration (Sources, Content Types, Author/Publish environments, Delta Tracking, Locales, Indexing Rules, and the Indexing Manager), see AEM Connector.


Settings

The Settings section contains the same form fields as the creation form (General Information and Connection Details), allowing you to edit the integration name, description, endpoint, type, and enabled state.


Monitoring

The Monitoring section is a real-time dashboard for tracking the indexing pipeline. It displays indexing events as they are received and processed.

Filters:

FilterDescription
Date From / Date ToRestrict results to a time window
Object IDFilter by a specific document ID
StatusFilter by processing status (see status table below)
EnvironmentAuthor or Publish
LanguageLocale code (for example, en_US)
SitesFilter by target SN Site

Auto-refresh: configurable intervals — Off, 1s, 5s, 10s, 30s, 1m, 5m.

Result columns: Date, Object ID, Status, Environment, Language, Sites.

Results are paginated and sortable.

Indexing Status Values

StatusMeaning
PREPARE_INDEXPreparing to index the document
PREPARE_UNCHANGEDNo changes detected since last indexing
PREPARE_REINDEXPreparing a reindexation
PREPARE_FORCED_REINDEXForced reindexation triggered
RECEIVED_AND_SENT_TO_TURINGDocument received by the connector and forwarded to Turing ES
SENT_TO_QUEUEDocument placed in the Artemis processing queue
RECEIVED_FROM_QUEUEDocument consumed from the queue by the indexing pipeline
INDEXEDDocument successfully indexed in Solr
FINISHEDOperation finished
DEINDEXEDDocument removed from the index
NOT_PROCESSED / IGNOREDDocument skipped due to an Indexing Rule or connector decision

Indexing Stats

The Indexing Stats section provides a table of completed bulk indexing operations for this integration.

ColumnDescription
Start TimeTimestamp when the operation started
SourceSource that was operated on
OperationINDEX_ALL or REINDEX_ALL
DocumentsNumber of documents processed
DurationTotal elapsed time
Docs/minThroughput (documents per minute)

Double Check

The Double Check section validates the consistency between the connector's content and the Turing ES search index. It detects drift between what the connector knows about and what is actually indexed.

  • Select a Source to inspect
  • Results are shown in two views:
    • Missing — content that exists in the connector but is not present in the index
    • Extra — content that is present in the index but no longer exists in the connector
  • Results are grouped by Solr core in an accordion, listing the affected document paths

Use Double Check after a partial failure, a forced reindex, or when users report missing or stale search results.


System Information

Displays live diagnostic information from the remote connector application.

Status badge: UP (green) or DOWN (red) — indicates whether the connector endpoint is reachable.

Application:

ItemDescription
ApplicationConnector application name
VersionConnector build version
Java VersionJVM version running the connector
VendorJVM vendor
JVMJVM identifier
OSOperating system name and version

Memory and disk gauges (progress bars with total, used, and free values):

GaugeDescription
Physical Memory (RAM)Host system memory
JVM Heap MemoryJava heap utilisation and limits
Disk SpaceAvailable storage on the connector's host volume

Architecture

Turing ES acts as a transparent proxy to the connector. The API path /api/v2/integration/{integrationId}/** forwards all requests to the configured connector endpoint, including authentication headers. This design keeps the connector application decoupled — it does not need to be publicly accessible, only reachable from the Turing ES server.

Built-in SSRF protection validates the endpoint before forwarding. Requests to private IP ranges, loopback addresses, or disallowed schemes are rejected.


PageDescription
AEM ConnectorAEM-specific configuration — sources, content types, environments, indexing rules, and the indexing manager
Semantic NavigationConfigure the SN Sites that receive indexed content
Architecture OverviewEnd-to-end indexing flow from connector to Solr
REST API ReferenceAPI endpoints for programmatic indexing
Dumont DEP — ConnectorsAvailable connectors (Web Crawler, AEM, Database, FileSystem, WordPress) and how to deploy them
Dumont DEP — REST APIConnector-side API for triggering indexing, monitoring, and source management