Data Extract Utility

Path: System Administration > Data Utilities > Data Extract Utility

The Data Extract Utility provides an automated data extract tool for the purpose of transferring data via a flat file from Infinite Campus servers. This tool can be used to upload Campus data into third party software. 

This article describes the following topics:

Screenshot of Data Extract Utility

Setting Up a Data Extract

The following table provides definitions about the data elements available in the Data Extract Utility Editor.

You can remove Social Security Numbers from reporting in the 3 default extracts by setting the parameter @includeSSN bit = 0

Field

Description

Extract Instance Name

The name of the extract.

Delivery Mode

Indicates how the extract will be delivered. 

  • SMB - Server Message Block. This works with an operating system higher than Windows Xp; used in Windows networks to allow resources such as files on one machine to be shared on other machines as if they were local.
  • SFTP Legacy (Being Deprecated) - Secure File Transfer Protocol.

    This is an outdated implementation of SFTP that does not support modern key exchange algorithms or cipher algorithms Users should transition to the other SFTP option as soon as possible to avoid disruptions once this delivery mode is removed in Summer of 2024.

  • SFTP - Secure File Transfer Protocol. An encrypted form of file transfer between a local system and any system on the network. This is an updated library which supports modern key exchange algorithms. 
  • FTPS - File Transfer Protocol Secured. An encrypted form of file transfer between a local system and any system on the network. This option also allows different encryption methods:
    • SSL (Secure Sockets Layer)
    • TLS (Transport Layer Security)
  • PULL - This option can be used in conjunction with the Export Wizard utility. This delivery mode allows for a separate client to be installed on a district server (not to be provided to third party vendors) and setup to pull the desired extract using https via a Windows Scheduled task. See the Pull Delivery Mode section for information on using this option.
SFTP Key Exchange
Only applicable if the SFTP Delivery Mode is selected.
Indicates you want to use the SFTP Key Exchange for this extract.

See the Data Extract Utility SFTP Key Exchange Manager article for more information on this option and process.
SFTP Key Exchange Configuration
Only applicable if the SFTP Delivery Mode is selected and the SFTP Key Exchange checkbox is marked.
If a keypair has been properly configured and tested within the Data Extract Utility SFTP Key Exchange Manager, select the key to be used for authenticating this extract.

See the Data Extract Utility SFTP Key Exchange Manager article for more information on this option and process.

File Name

Name of the the file as it will appear in the destination folder. Often a third party tool needs/requires this file name.

File Extension

Extension of the file.


CSV, HTML, TAB, TXT, and custom file extensions can be added through the attribute dictionary.

Screenshot of file extension

Column Delimiter

Indicates how data in the file will be divided. Options are:

  • TAB - Uses the tab character to separate each of the columns used for storing data.
  • CSV - Comma-separated values file is used for data structured in lists.
  • Tilde (~) - Uses the tilde character (~) to separate each of the columns in the file used for data storage
  • Vertical Bar (|) - Uses the vertical bar character (|) to separate each of the columns in the file used for data storage.
Character Set

Select the character encoding standard used in the file.

  • US-ASCII - The American Standard Code for Information is a common character encoding standard for electronic communication.
  • UTF-8 - The Unicode Transformation Format is a variable width character encoding capable of encoding all valid code points in Unicode using one to four 8-bit bytes.
Append Date to FilenameWhen selected, includes a date in the file, so as not to overwrite an existing file. Choose the format of the date, either the normal date format of yyyymmdd or date and time format of yyyymmddhhmmssSS. 

Include Header Row?

When selected, indicates a header row will be included as part of the file.

Include Double Quotes?

When selected, includes quotes around each piece of data in the column.

Username

Username used to access the destination server.

Password

Password used to access the destination server.

Domain

The domain of the server connecting to and uploading data from the Data Extract Utility.

Server

The IP Address of the server connecting to and uploading data from the Data Extract Utility.

Port

Indicates the Port value required to connect to the server.

  • SFTP default = 22
  • FTPS default = 990

Path

The location on the destination server where the file will be written. This needs to be absolute path and include the /upload indicator.

Error Notification

If marked, any errors which occur will send a notification to the Notifications area of Infinite Campus (see below) to all users within the user group selected in the User Group to Notify dropdown list. 

Screenshot of the notifications icon
User Group to NotifyThis group will be notified via a Notifications message if an error occurs while using the Data Extract Utility Set Up tool. You must have the Error Notification checkbox marked in order to utilize this functionality.
SQL/Prism

Provides a text area to enter a SQL query or Prism call.

Choose the desired SQL or Prism radio button.

  • If choosing SQL, enter the text of the query used to generate the file.
  • If choosing Prism, choose the prism call for a PDF report and add the prism information. This option requires a Firefox browser add-on of Convert Form Methods.
Ad Hoc

If choosing Ad hoc, select any saved filter to which the user has rights. This option generates the same filter as in ad hoc and carries over same restrictions. It must complete in 5 minutes. When a user no longer has rights to a filter, the utility stops running on a schedule.


Ad hoc filters used by the Data Extract Utility are automatically scoped to data within the active year. This is the equivalent of running the ad hoc filter for the year marked as Active in School Year Setup.

Use Data WarehouseIf district has Data Warehouse, use that database instead of the live database. 
Ad hoc Filter When the Ad hoc radio button is selected, this lists every filter available to the user.
Data Extract Schedule

Lists the frequency, date, time and day of the week the extract runs. If this needs to stop for any reason, mark the Stop Automated Extract checkbox.

Create a New Extract

The creation of new extracts should be done by technical administrators only.

  1. Select the New icon. A Detail Information section will appear.
  2. Enter the Extract Instance Name.
  3. Choose the other necessary items for adding the extract by populating and marking the appropriate fields, as defined above.
  4. Click the Save icon when finished. Additional modifications may be made to the extract to determine when the extract should be generated. 

At this time, a user can select the available buttons to test the extract, test the connection to the database or run the extract by selecting one of the appropriate buttons. Extracts will display in a new window in the selected format. 

  • The Test Extract button generates the containing information requested by SQL or Ad hoc. Data is not written to the database or stored in any location.
  • The Test Connection button checks the connection to the entered server by attempting to write a 1-byte file to the location setup. When it is successful, a pop-up displays indicating the connection is good; if it is not successful, it provides the reason the connection failed.
  • The Run Extract button runs and generates the extract, sends and stores the file where needed (in case of an issue when it is an auto-run extract).

Setting Up the Data Extract Client to Utilize the Pull Delivery Mode

For information on setting up the Data Extract Client to utility the pull delivery mode, see this article.

Troubleshooting/Errors

ErrorExplanation
Unable to negotiate key exchange

(client: [list of supported algorithms] / server: diffie-hellman-group14-sha1,diffie-hellman-group-exchange-sha1)"
This error means the SFTP server your Campus site is trying to communicate with does not support new enough algorithms to work with the Data Extract Utility's latest SFTP mode. The issue is the SFTP server you’re connecting to is only offering up these older key exchange algorithms when your Campus site’s Data Extract Utility client tries to communicate with it:
  • diffie-hellman-group1-sha1
  • diffie-hellman-group-exchange-sha1

SHA1 is now considered an insecure key exchange algorithm, so our latest SFTP implementation does not support it.


You will need to reach out to the 3rd parties/individuals who administer this SFTP server and request they update their server/SFTP software to support at least any one of any of the more modern and secure key exchange algorithms mentioned in the "client:" part of the error message.


As soon as the SFTP server is updated to support any of the more secure key exchange algorithms, then you'll be able to use the latest SFTP mode for any connection currently receiving this error. Until this update occurs, you'll need to continue using the legacy SFTP mode.


The legacy SFTP mode is planned for deprecation in Summer 2024, so there's ample time, but if you have any sever still using only these less secure key exchange algorithms come that time, your connections to this server will no longer function.