Tool Search: Event Queue
The Event Queue lists all data syncing events which are waiting to sync data from Campus to Ed-Fi. This list may also include any change that is made to a table used for a state's Ed-Fi implementation. As events are processed, state-specific logic may remove a change event from the queue or create a JSON event which will appear in the Event Queue list.
Event Queue Tool
Read - View data within the Event Queue
Write - Refresh or process events
Add - Provides no function for this tool
Delete - Provides no function for this tool
For more information about Tool Rights and how they function, see this Tool Rights article.
Prerequisites
- Ed-Fi functionality must be enabled via the Enable Ed-Fi System Preference.
- A valid Ed-Fi Configuration must be established in order to process events in the Event Queue. Please review the Ed-Fi Configuration article for more information. This ensures Campus is communicating with Ed-Fi and successfully sending data to their servers. If there are no valid connections, messages remain in the queue until a valid connection is made. If there are multiple connections to different Operational Data Stores (ODS), messages process for all valid connections. If one connection is valid and one or more other connections are invalid, all messages process in the Event Queue and errors are generated in the Error Log on messages for any invalid connection(s).
- Ed-Fi IDs should be assigned to users via the Demographics tab.
Event Queue Editor
| Field | Description |
|---|---|
| Queue Order | Displays the number the record is in the Event Queue. |
| Action Type | Displays the action of the record: Insert, Update, or Delete. |
| Campus Table / Resource Name | Displays the name of the Campus Table or Resource that triggered the record to send to Ed-Fi. |
| Old Data | Displays the data in the Campus table prior to the changes that triggered the record to be sent to Ed-Fi. |
| New Data | Displays the data in the Campus table that changed to trigger the record to be sent to Ed-Fi. |
| Currently Processing | Indicates if a message is processing or queued to be processed. |
Review Ed-Fi Events
To view what data was and what it was changed to, click the Show audited data link in the Old Data and New Data columns.
Reviewing Event Details
Review Resync Details
When a resync is in process, a temporary table displays in the Event Queue showing what resyncs are in progress and which resource/school combination was selected. This table only shows resyncs kicked off from the resync tool, not resyncs caused by data triggers. This table and all data related will be deleted once the resync is completed. Users can then use the Event Queue History tables to see more detail on how many records were processed.
The Details link can be selected to view an Excel report with additional details about the resync in progress.
Resync Requests Processing Table and Report
Event Queue Processing
The Process Now button forces the processing of events. The next 10,000 records not being processed are packaged to the Event Queue. Every minute, events start processing until the queue is empty. Once an event is processed, it no longer appears in the current Event Queue.
A detailed list of Event Queue statistics for the past five days can be viewed by going to the Ed-Fi Event Queue Statistics tool.
Processing Ed-Fi Events
Ed-Fi v3.x Event Queue Processing
- Events will not hit the event queue for a year that does not have a valid configuration or the resource is toggled off.
- Example: During a calendar roll forward, if there is not a valid configuration for the next school year or there is a valid configuration but all scheduling resources are toggled off, that data will not appear in the Event Queue.
- Ineligible records will hit the queue if there is a valid connection and the resource is turned on but there is business logic that prevents the record from sending.
- Example: A student's enrollment that is marked as "No Show" is modified. This record will hit the event queue but the business logic will then run and prevent it from sending to the ODS.
Ed-Fi v2.x Event Queue Processing
Whenever a change is made to a Campus table either via the UI or directly to the database, a change event is triggered and the event is sent to the Event Queue, with the following exceptions:
- Any calendars or schools marked as Exclude, regardless of scoped year, do not send data to the Event Queue.
Future year data (including scheduling data) does not report or populate events in the Event Queue. The following Campus tables do not send data to the Event Queue:
Campus Table Day EdFiGradingPeriod Course Section SectionPlacement Roster SectionStaffHistory All other changes, other than the ones listed above, are sent to the Event Queue. The Ed-Fi transformers determine if the data should send to Ed-Fi or be removed from the Event Queue.
The following tables do not have a calendarID or trialID to reference for suppression and cannot be excluded from being sent to the Event Queue:
Campus Table All custom tables DayEvent PeriodSchedule Period TermSchedule Term
Multi-Threading
The Event Queue multi-threads items of a single category (see below). Once the Event Queue encounters something of a different category, then it joins the threads. A join is when there is a stop on multi-threading, waiting until all threads have finished processing, before continuing the process anew.
The categories are: Person, Course, Related Pair, Day, Single-Thread.
The Campus tables for these categories are:
- Person - Person, Identity, Contact, Lep, Enrollment, CustomStudent, Employment, EmploymentAssignment, GradingScore, Attendance, AttendanceUnit, TranscriptCourse, TranscriptCredit, POSEligibility, EmploymentCredential, EnrollmentAZ, EnrollmentNE, EnrollmentWI, Graduation, TestScore, Plan, ProgramParticipation, LepService, Evaluation, NEProgramsFact
- Course – Course, Section, SectionPlacement, SectionStaffHistory, Roster, GradingTaskCredit, CustomCourse, CustomSection
- Related Pair – RelatedPair
- Day – Day
- Single-Thread – All others
These categories are sorted by unique data:
- Person – personID
- Course – Course.number, Course.stateCode, term descriptor override, or personID (for SectionStaffHistory and Roster)
- Related Pair – personID1 and personID2
- Day – date and calendarID
- Single-Thread – single threaded, so this is irrelevant
In Ed-Fi v3.x and higher, resynced data for the same resource are multi-threaded when there will not be data collisions. These are grouped into threads as follows:
- sections resource: sectionID
- calendars, calendarDates, classPeriods, and courses resources: object key
- studentSchoolAssociations and staffEducationOrganizationAssignmentAssociations resources: personID
- resources that are person-based and school-based: personID/schoolID combination
- resources that are person-based but not school-based: personID
- resources that are school-based but not person-based: schoolID
- all other resources (very few) will be single threaded
Special considerations:
- If an item has two existing threads, it can be sorted in (for example, the personID changes on a record and each of those personIDs is already assigned a thread), then the item causes a join.
- For categories with multiple unique constraints, all are taken into account. If any of the constraints are assigned into existing threads that are not all the same thread, then the item causes a join.
- If the associated section or course in Campus has been deleted for the Course category objects, they cause a join and then run as a single thread once all threads have finished processing.
- If Course is the table being processed and it does not have an edFiTermType, all edFiTermTypes of any associated sections are used as unique parameters.
Multithreading Simultaneous Core and State Configurations:
If there are configurations for both Core and State types, each item in the event queue will indicate which configuration type it is for. If a trigger applies to both core and state configurations, two events are put in the Event Queue: one for core and one for state. Core and state events can always be multi-threaded regardless of the event data since they are going to separate ODS's.
Refresh Ed-Fi Event Queue
Once events have been processed, click the Refresh button to see any remaining and unprocessed/processing Ed-Fi events.
Refreshing the Event Queue
