Search Framework Status

Tool Search: Search Framework Status

The Search Framework Status tool provides a simplified way to quickly and easily identify the health and status of the search framework within Infinite Campus.

With a shift to a different search framework in Campus.2411, this tool is now obsolete and was removed in said release pack. 

Framework status details are only displayed on quartz enabled Campus app servers. 

Screenshot of search framework status

Users must have the Student Information System (SIS) product security role in order to access and use this tool.

All users with the Student Information System (SIS) product security role will receive Process Alert messages when Search Framework issues/errors occur.

Understanding Status Details

Screenshot of status details

Click here to expand...

Syncing: Search Framework Versus DIS

To be clear, the search framework and DIS (Data Interchange Services) utilize completely different underlying technologies and are not interrelated. These are configured and troubleshot as independent systems. Disabling search framework functionality does not disable DIS functionality.

The only similarities are that both systems keep data in sync between the Campus District Edition and another data source, and both are involved in student and staff locator searches on Cloud-hosted environments. Resyncing search framework data does not resync DIS data.

Use the table below to better understand each field provided: 

FieldDescription
Status Indicator

The color-coded status indicator is a quick and easy way to understand the general health of the search framework.Screenshot of status indicator

  • Green = Everything is working correctly and no issues are present. 
  • Blue = The site is currently in the middle of configuration or indexing. This may indicate errors or issues occurred during startup but the system was able to recover. These issues should be investigated if they reoccur on a regular basis.
  • Yellow = At least one medium to high level priority issue has been found. There are issues that need to be investigated but some systems may still be functioning.
  • Red = The search framework is down. Please contact Campus Support. 
EnabledIndicates if the search framework preference is set to Yes (True) or No (False).
Status
(displayed as a colored coded pill indicating sync status)
SyncingIndicates indexes have successfully been initialized, created, and populated. Everything is running properly and data is actively syncing from Campus to the search framework.
Differential Sync

A differential (partial) sync for specified indices spanning up to 3 days is underway. This was triggered either manually or automatically as part of self-healing.


Previously entered data is still searchable, but new data will not be searchable until the differential sync is complete. This should take considerably less time than a full reindex, but will vary depending on the number and size of records modified during the specified window.

IndexingIndices are being populated with data from the Campus database.
IndicesAt least one index has inconsistent counts.
This status shows as blue if the personnel index is the only index with an inconsistent count.
Note: For all other indices, this status will be yellow.
At least one index should be enabled or initialized but is currently set to disabled or not initialized. This status only applies to the personnel index. For all other search based indices, the status will be red.
InitializingIndexes are being created and populated for the first time. This may take several minutes or hour.
RegistrationsSearch Framework contains a registration for an app instance that does not exist in Camp.


An app server may have been removed from the pool and is no longer active. An automated process removes expired app instance registrations after 2 days

Reindexing

A full reindex for specified indices is underway. This was triggered either manually or automatically as part of self-healing.

Previously entered data is still searchable, but new data will not be searchable until the reindex is complete. This may take several minutes, up to hours, depending on the size of your site's indices.

At least one forced reindex or differential resync was requested within the past day.


This was either triggered manually or automatically as part of self-healing.

The search framework encountered an error during the last reindex attempt.
This message will appear if the error occurred in the past 24 hours. If the error occurs for longer than 24 hours, the status turns to red.

A reindex containing all mapped data for specified indices has been been scheduled, either manually or automatically as part of self-healing. 


This is a temporary status and should only be visual for a brief period of time (few seconds).  Refresh the status page accordingly.

SyncingAt least one sync failure or veto returned within the past day.

A differential (partial) sync, containing data changes from up to 3 days for specified indices, has been been scheduled. This was either triggered manually or automatically as part of self-healing.


This is a temporary status and should only be visual for a brief period of time (few seconds). Refresh the status page accordingly.

One or more indices are enabled when they should be disabled.
IndexingThe search index is taking longer to index than expected. This message appears if a site is taking more than 4 hours to index.
Indices

At least one index has inconsistent counts.


Note: This status shows as blue if the personnel index is the only index with an inconsistent count.

Registrations

The last successful quartz registration was more than 24 hours ago.





At least one app in the pool has not successfully registered within the past 24 hours.
An app instance for at least one of the apps cannot be found in the search framework (missing).
Communication error retrieving registration details from Search Framework
Communication error retrieving app instance details from Search Framework
ReindexingThe search framework is taking longer to reindex than expected.
This message appears if a site is taking more than 4 hours to reindex.
IndexingThe search framework is taking much longer to index than expected.
This message appears if a site is taking more than 24 hours to index.
InitializationThe search framework is encountering errors initializing.
The search framework encountered errors initializing or the initialization failed.
The search framework is not initialized (non-quartz).
The search framework is initialized, but not indexing. The job to create and populate the indices has not yet started.
This should only be present for a brief time. If this state persists after refreshing, then this is a problem.
The search framework is not initialized.
OfflineThe search framework is offline. Either this is intentional or the job to create and populate the indices has failed.
RegistrationsCannot find registration for this site in search framework. This pertain to the Campus quartz app server.
ReindexingThe search framework is taking much longer to reindex than expected.
This message appears if a site is taking more than 24 hours to reindex.
The search framework encountered an error during the last reindex attempt.
This status appears if the error persists longer than 24 hours.
SyncingAt least one index should be enabled or initialized but is currently set to disabled or not initialized.
This status appears if the issue persists longer than 24 hours.
Unique IdentifierThe lowercase value of the Campus database GUID used for unique tracking purposes.
ClusterIndicates the search framework cluster for which the site is configured.
Hosting ModelIndicates your district's hosting model. 
Last Registration TimeIndicates the last time the quartz-enabled Campus app server successfully contacted the centralized search framework, in order to confirm proper registration. This should occur on a daily basis.

Indices

The Indices table allows you to review and monitor the status of each index in the search framework. Indices are essentially data sources that can be used for search or data analysis purposes. If an index is not initialized or enabled, it cannot be used for search purposes.

Screenshot of indices

The table below describes each column in the Indices table:

ColumnDescription
TypeThe name of the index. This name usually describes the type of index.
Enabled / Expect EnabledIndicates whether or not the index is enabled and whether or not it should be enabled. 
InitializedIndicates whether or not the index is initialized.
Campus CountIndicates a count of records in the Campus database mapped to the corresponding index.
Framework CountIndicates the count of records in the centralized search framework index. This number is expected to match the Campus Count within ~7 seconds, based on a scheduled 5 second sync quartz job.
Sync On / Expect OnIndicates whether or not the index is currently configured for syncing and whether or not the index is expected to be configured for syncing.
SharedIndicates whether or not the index is a shared index.

Non-shared indices sync data on 5 second intervals and have two aliases mapped to separate index indices, so that data is searchable during a reindex or selective resync.


Shared indices are common across all Campus sites and do no not sync on 5 second intervals. They are primarily for internal tracking purposes, not Campus search purposes, and are updated on triggered events. 

Tracked Tables

The Tracked Tables section allows you to monitor whether or not change tracking is enabled for each table.

The search framework relies on Microsoft Change Tracking to detect when data has been changed in the Campus datasource. In order for search contexts to provide the most recent and accurate data, change tracking needs to be enabled and working properly for each table shown in this section.

Screenshot of tracked tables

Jobs Table

The Jobs table allows you to view information on existing Quartz jobs relevant to the search framework; including the name of each job, the job’s status, trigger name, and trigger status.

Screenshot of jobs table

The table below describes each job and their function.

Job NameFunctionExpected Trigger Status
ES create and populate indicesResponsible for building the indices and populating them with data from the source Campus database. Indices are the datasets used for search and query purposes.Active at all times
ES data sync with SQLPeriodic sync job that checks to see if there are changes in the source SQL Campus database and syncs them to the search framework accordingly.
The sync job ensures that the search data is constantly populated with the latest data from the Campus database.
Active, unless the search framework is indexing or an individual index is either performing a full reindex or differential sync.
ES ReIndexUsed for forcing a full reindex intended to fix issues where an index may not contain all data from the source Campus database. This job is either initiated automatically via a watcher or manually.Inactive, unless a differential sync was fired to fix an issue.
ESDifferentialSyncJob

Used for forcing a differential sync of all changes spanning up to the last 3 days, intended to fix issues where an index may not contain all data from the source Campus database. This job is either initiated automatically via a watcher or manually.

Inactive, unless a differential sync was fired to fix an issue.
ESRegistrationJob 

A daily phone home job to validate the Campus application can successfully communicate with the search framework and report back metadata about the site's configuration. 

Active at all times.

App Instances

The App Instances table tracks a list of Campus app servers and the last time they successfully registered with the search framework. If Campus is hosted across multiple app servers, then each of those app servers phones home and registers with the search framework on a daily basis. 

It is possible for users to experience search framework issues across all apps or only specific apps.

Screenshot of app instances

Most Recent Reindex Logs

The Most Recent Reindex Logs table details the 20 more recent reindex/differential resync requests, with information on the reason, the action taken, the indices reindexed, the timestamp, the update time, and the duration of the reindex.

Screenshot of most recent reindex logs

Refresh the Search Framework Status

Click the Refresh button near the bottom of the tool to refresh the search framework status.

Screenshot of refresh search framework status

View Diagnostic Information

You can view diagnostic information about your search framework by clicking the View Diagnostic button and selecting either INFO to view core stateful information or DEBUG to view a more comprehensive set of information about the status of the search framework.

Note: The DEBUG option may take a considerable amount of time to load and process data.

Screenshot of Diagnostic Information

Process Alert Messages for Issues and Resolutions

The table below details the Process Alert messages generated when search framework issues are encountered, the impact of each issue, and the success message sent once an issue is resolved. 

Classic CampusNew Look and Feel of Infinite Campus
Screenshot of classic campusScreenshot of new look campus

Process

Name

Details

Impact

Success Message

Search Framework Client Connection

Error [performing action]. Contact support if errors are not resolved for app server: [AppServerName]

Example = turning off CampusTool

Inability to reach the search framework.

No search framework functionality, including new look search contexts.Success [performing action] for app server: [AppServerName]
Search Framework Create/ReindexAn issue has been encountered bulk updating your search data. Please review the linked Search Status page and contact Infinite Campus Support if further assistance is needed.One or more search framework indices are not populated correctly.One of the search indices will have no or inaccurate data.Success bulk updating your search data. Tool Tip: Affected searches [SearchName Array]
Search Framework Index SyncAn issue has been encountered updating your search data. Please review the linked Search Status page and contact Infinite Campus Support if further assistance is needed.One or more search framework indices are not updating.One of the search indices will have inaccurate data.Success updating your search data.
Search Framework Quartz ServerThe Campus SIS has failed to register with the Search Framework. Please review the linked Search Status page and contact Infinite Campus Support if further assistance is needed.The quartz server is not communicating successfully with the search framework.No available search framework functionality such as new look or campus tool search. or search indices may have out of date data.Success registering Campus SIS with the Search Framework.
Search Framework Repeated SearchesFailures have been encountered with the following search: [Search Term Would Show Here]. Please contact Infinite Campus support if search errors have not been resolved on app server: [AppServerName]Users have encountered excessive errors when attempting to perform searches.Temporary or sustained issues with one or more search indices.Search errors resolved on app server: [AppServerName]