Tool Search: Event Queue
The Event Queue lists all data syncing events which are waiting to sync data from Campus to Ed-Fi. This list may also include any change that is made to a table used for a state's Ed-Fi implementation. As events are processed, state-specific logic may remove a change event from the queue or create a JSON event which will appear in the Event Queue list.
Read - View data within the Event Queue
Write - Refresh or process events
Add - Provides no function for this tool
Delete - Provides no function for this tool
For more information about Tool Rights and how they function, see this Tool Rights article.
Event Queue Editor
| Field | Description |
|---|---|
| Resync Timestamp | Displays the time the resync was initiated. |
| Status | Shows the status of the resync. |
| Resync Details | Link that displays more details on how many records were processed. |
| Queue Order | Displays the number the record is in the Event Queue. |
| Action Type |
Displays the action of the record: Insert, Update, or Delete. |
| Campus Table / Resource Name |
Displays the name of the Campus Table or Resource that triggered the record to send to Ed-Fi. |
| Audited Data | Displays what the data was previously and what it was changed to. |
| Currently Processing | Indicates if a message is processing or queued to be processed. |
Event Queue Processing
The Process Now button forces the processing of events. The next 10,000 records not being processed are packaged to the Event Queue. Every minute, events start processing until the queue is empty. Once an event is processed, it no longer appears in the current Event Queue.
A detailed list of Event Queue statistics for the past five days can be viewed by going to the Ed-Fi Event Queue Statistics tool.
- Events will not hit the event queue for a year that does not have a valid configuration or the resource is toggled off.
Example: During a calendar roll forward, if there is not a valid configuration for the next school year or there is a valid configuration but all scheduling resources are toggled off, that data will not appear in the Event Queue. - Ineligible records will hit the queue if there is a valid connection and the resource is turned on but there is business logic that prevents the record from sending.
Example: A student's enrollment that is marked as "No Show" is modified. This record will hit the event queue but the business logic will then run and prevent it from sending to the ODS.
Review Resync Details
When a resync is in process, a table displays in the Event Queue showing what resyncs are in progress and which resource/school combination was selected. The Download Resync Details button can be selected to view an Excel report with the additional details about the resync in progress. This table and all data related will be deleted once the resync is completed.
Download Resync DetailsResync Details Fields
| Field | Description |
|---|---|
| Config Name | The Configuration for which the Resync was performed. The Configuration is selected on the Ed-Fi Resync tool. |
| Resync Timestamp | The date and time the resync was initiated. |
| Selected Schools | The schools that were selected to have data resynced. The schools are selected on the Ed-Fi Resync tool. |
| Selected Resources | The resources selected to have data resynced. The resources are selected on the Ed-Fi Resync tool. |
Review Audited Data
To view what data was and what it was changed to, click the Open Audited Data link in the Audited Data column. A pop-up will display with Old Data and New Data columns. The Old Data column displays what was in the database prior to the change and the New Data column displays the data that was changed in the database.
Open Audited Data
Old Data vs. New Data
Refresh Ed-Fi Event Queue
Once events have been processed, click the Refresh button to see any remaining and unprocessed/processing Ed-Fi events.
A detailed list of event queue statistics for the past five days can be viewed by going to the Ed-Fi Event Queue Statistics tool.
Multi-Threading
The Event Queue multi-threads items of a single category (see below). Once the Event Queue encounters something of a different category, then it joins the threads. A join is when there is a stop on multi-threading, waiting until all threads have finished processing, before continuing the process anew.
The categories are: Person, Course, Related Pair, Day, Single-Thread.
The Campus tables for these categories are:
- Person - Person, Identity, Contact, Lep, Enrollment, CustomStudent, Employment, EmploymentAssignment, GradingScore, Attendance, AttendanceUnit, TranscriptCourse, TranscriptCredit, POSEligibility, EmploymentCredential, EnrollmentAZ, EnrollmentNE, EnrollmentWI, Graduation, TestScore, Plan, ProgramParticipation, LepService, Evaluation, NEProgramsFact
- Course – Course, Section, SectionPlacement, SectionStaffHistory, Roster, GradingTaskCredit, CustomCourse, CustomSection
- Related Pair – RelatedPair
- Day – Day
- Single-Thread – All others
These categories are sorted by unique data:
- Person – personID
- Course – Course.number, Course.stateCode, term descriptor override, or personID (for SectionStaffHistory and Roster)
- Related Pair – personID1 and personID2
- Day – date and calendarID
- Single-Thread – single threaded, so this is irrelevant
Resynced data for the same resource are multi-threaded when there will not be data collisions. These are grouped into threads as follows:
- sections resource: sectionID
- calendars, calendarDates, classPeriods, and courses resources: object key
- studentSchoolAssociations and staffEducationOrganizationAssignmentAssociations resources: personID
- resources that are person-based and school-based: personID/schoolID combination
- resources that are person-based but not school-based: personID
- resources that are school-based but not person-based: schoolID
- all other resources (very few) will be single threaded
Special considerations:
- If an item has two existing threads, it can be sorted in (for example, the personID changes on a record and each of those personIDs is already assigned a thread), then the item causes a join.
- For categories with multiple unique constraints, all are taken into account. If any of the constraints are assigned into existing threads that are not all the same thread, then the item causes a join.
- If the associated section or course in Campus has been deleted for the Course category objects, they cause a join and then run as a single thread once all threads have finished processing.
- If Course is the table being processed and it does not have an edFiTermType, all edFiTermTypes of any associated sections are used as unique parameters.
Multithreading Simultaneous Core and State Configurations:
If there are configurations for both Core and State types, each item in the event queue will indicate which configuration type it is for. If a trigger applies to both core and state configurations, two events are put in the Event Queue: one for core and one for state. Core and state events can always be multi-threaded regardless of the event data since they are going to separate ODS's.
