Application Review

Overview

Application Review tools in Veracross allow the admissions review process to function paperlessly by providing a single place where applications can be reviewed and scored. Application criteria can be created and reviewers assigned: they complete their reviews, assign a recommendation, and can use custom criteria.

Admission review records are either created amnually or during the Create Application Review Records procedure. They are not created during the nightly scripts.

The Process of Application Review

The Application Review system in Veracross is designed to help an admissions office manage the applications that come in by having reviewers review applications and make recommendations directly in Axiom. The resulting recommendations can then be queried, summarized, etc., to help make admission decisions. While the tools in Axiom have built-in flexibility, the following overall outline of the steps in the application review process illustrate one way to use the system. Most steps are managed from the Application Review homepage.

The Application Review process in Veracross is designed to be flexible enough to meet schools’ differing application review processes. For example:

• An application reviewer is able to see as many documents as are available electronically in Axiom, but a school could combine that with a paper-based process as well to arrive at application review recommendations.
• Application review recommendations (Strong Accept, Accept, Reject, etc.) can be customized.
• Reviews can be configured and created in batch according to various filtering criteria, but also manually (and also deleted). This makes the process suitable for a large admissions department managing thousands of application reviews and a smaller department with under a hundred.
• Each application review can be scored on individual criteria, then the criteria scores can automatically calculated a total score or a a reviewer can provide an override. Schools also might also choose to completely ignore the scores and simply rely on the recommendation.

Configuration

1. Set up the main components of application review.

All four of the following components are set up in the Configuration section on the Application Review homepage. They should be considered prior to setting up application review for the first time.

Note: The criteria category is a new “layer” in application review configuration, added 1/17/2018. It allows criteria categories to be independent of reviewer type. Schools can choose the criteria category to match reviewer types 1:1 if desired.

• Reviewer Types: What kinds of reviewers are needed? Examples include Interviewer, Committee, Student Tour Guide Evaluator, Coach. Click the “Application Reviewer Types” query to review existing ones or “Add Application Reviewer Type” to add one (also available on the “Add” menu).
• Reviewers: Who will be reviewing applications? They will fall into one of the types of reviewers above. Click the “Reviewer Configuration” query to review existing ones or “Add Reviewer Configuration” to add one (also available on the “Add” menu).
  • Reviewer Type: Select which type of reviewer they are.
  • Criteria Category: Select which criteria category applies to this reviewer. Note that this is different than Reviewer Type. See next section for more information.
  • Description: Optional text description.
  • Filters: Selecting any of these filters (school level, etc.) will cause review records to be created for the Reviewer that fit the criteria. An example of using this would be to create an upper school principal as a reviewer, and assign them the school level filter of Upper School so that they will review all upper school applications.
• Application Criteria Categories: What types of criteria are needed to evaluate applications? These might match reviewer types 1:1 or be quite different. Examples include Reviewer, Interviewer, Committee, Athletic, Social. Click the “Application Criteria Categories” query to review existing ones or “Add Application Criteria Category” to add one (also available on the “Add” menu).
• Application Criteria: What are the actual criteria to be used? They will fall into a criteria category as defined above. Examples include English Ability, Test Scores, Transcript, Interview, Athletic Potential. Click the “Application Criteria” query to review existing ones or “Add Application Criteria” to add one (also available on the “Add” menu).
  • Scale: Utilizes rubric scales (similar to qualitative grading). Add a new scale from the “Add” menu on the System homepage. Read more about rubric scales.
  • Criteria Category: Select which category this will be a part of.
  • Grade Level From/To: Select which grade levels these apply to when grade review records are created
  • Include in Score: Should this be included in the Total?
  • Weight: Select a number other than “1” if the criteria should have greater weight relative to other criteria.

One final configuration option to review is found on the Application Statuses query on the System homepage:

• Create Review Records field: Review records are created for applications with certain statuses. Check this field for each application status category for which review records should be created.

2. Create the review records.

Once the above elements are configured, the review records can be created. The system is designed to leverage the reviewer types, reviewers, etc. configured above to create review records in batch, but review records can also be created manually.

Option 1: Create in Batch: Run the Create Application Review Records (Admission Candidates Only) Action Item on the Admissions or Application Review homepage. The procedure creates review records for:

• admission candidates
• in the active admissions year
• with application statuses indicated on the Application Statuses query
• with the reviewers and criteria specified in the application review configuration

The procedure can be run more than once if the configuration changes. Review records will never be deleted by the procedure, only added.

After running the procedure, check to ensure that the review records were created as expected. There are several places to look on the Application Review homepage:

• Reviewer Summary button: View a summary by recommendation of all reviewers. After running the procedure for the first time, all statuses will be Pending. Click on a number to drill down and see the actual review records.
• Find Application Reviews query: Run this query to see all reviews regardless of status (though they will all display <None Specified> after the reviews are initially created).
• Application Reviews tab on the Application Reviewer Configuration record. All reviews associated with the reviewer configuration are displayed.
• My Applications for Review: If the user currently logged in was configured as a reviewer, their reviews will be displayed in this query.

Option 2: Create Manually: Instead of or in addition to creating review records in batch, review records can be created manually in several ways:

• Create a review record for an application: Navigate to the Review tab of an application record, click Add Record, enter the information in the fields, and click Update.
• Create a review record for a reviewer: Navigate to the reviewer configuration record and run the Create Application Review Records (only this configuration record) from the Action menu.

Delete Application Review Records If Needed: Review records can be deleted at any time, even if they contain a recommendation. Click the red “X” and Update if in a query results view, or select “Delete” from the “+” Organize menu if on a review record detail screen.

3. Ensure that all reviewers have access to their review records

Once review records exist, reviewers need to be able to access their reviews. Users with the Admissions_1 security role can access the Application Review homepage and use the My Applications for Review button or otherwise search for their own reviews.

Admissions_Reviewer Security Role: Users without that security role can be assigned the supplemental security role Admissions_Reviewer which enables them to see only those applications assigned to them. Users should never have both the Admissions_1 and Admissions_Reviewer security roles. This security role only grants access to their own review records; it does not grant access to the Application Review homepage. In order to access their reviews, therefore, they will need to receive a query link to take them directly to their reviews. The simplest way to achieve this is to send them the My Applications for Review URL. When Admissions_Reviewers click it, they will be shown only their reviews since the assigned reviewer is {me}, meaning whoever is logged in.

Reviewers Complete Reviews

Once review records are created and all reviewers can access their reviews by virtue of their Admissions_1 or Admissions_Reviewer security role, reviewers can begin reviewing applications. Reviewers can access their own reviews via the My Applications for Review query — accessed either on the Application Review homepage — or with a URL that they have been sent.

The basic process to review an application is:

1. Access the review record.
2. Review the record: files are accessible on the left tab, the Candidate Profile Document is available as a link, and basic information about the student is in the header. Schools also may choose to create reports for reviewers with additional information.
3. Provide an assessment (score) for each criterion in the grid.
4. Add notes or concerns for each criterion. Click Update to refresh the header and see the total and average score calculated.
5. Enter a score to override the total score (if desired).
6. Enter notes and concerns if desired. Note: The header-level notes are used in other queries, so if notes and concerns are entered on individual criteria, it is recommended to copy/paste or summarize them into the header notes and concerns.
7. Select a recommendation.
8. Click Update to lock in all changes. A review is considered complete when it has a recommendation.

Content of an Application Review

Basic information about the application is available for the reviewer in the record header.

Record Header

• Score: This is a manually entered field that overrides the Total Score. It allows the reviewer to enter any desired score, similar to the way a teacher may be able to enter any posted grade, regardless of the calculated grade.
• Total Score and Average Score: These are calculated based on the criteria scores below.
• Recommendation: The whole point of the application review process is to arrive at a recommendation. This is where the reviewer makes their final recommendation.
• Notes: The reviewer can indicate any notes for this application. This field — and not the one on the individual criteria — is used in other application review queries, so it is recommended to make sure that any notes on individual criteria are summarized here.
• Concerns: The reviewer can indicate any notes for this application. This field — and not the one on the individual criteria — is used in other application review queries, e.g., the “Reviews with Concerns” query in the Reviews section of the Application Review homepage.

Criteria

Available criteria depend on how the review records were configured. The reviewer should enter individual assessments for each criterion.

Files

The tabs on the left give access to admission files for the reviewer to access. The Files tab allows download access, and the File Preview provides a quick view of relevant admission files such as webform submissions or files uploaded from the Admission Portal. Check the File Security query on the System homepage if there are issues with viewing certain files.

Creating additional reports for reviewers

Schools may find it useful to create overview reports for organizing candidates for a review process. Function fields can be especially helpful in designing them.

Admission Application Review Function Fields:

• Adm Reviewer
• Adm Recommendation Count
• Adm Review Notes
• Adm Review Criteria Score
• Adm Review Criteria Score List
• Add Review Total Score

Admissions Checklist Function Fields:

• Adm Checklist Item Notes
• Adm Checklist Item Status

Standardized Test Score Function Fields:

• Test Summary by Year
• Test Summary by Grade Level
• Test Score by Year (Text)
• Test Score by Year (Numeric)
• Test Score by Grade Level (Text)
• Test Score by Grade Level (Numeric)

Managing the Review Process

Once reviewers have begun their task, reviews will start to “come in,” i.e., be completed. Use the Application Review homepage to manage the reviews. There are a number of queries available to manage the process. Read more about the Application Review homepage, but here are a few highlights:

• Find candidates, reviewer summaries, and “my” applications via quick-access buttons.
• See recently updated and recently completed reviews, as well as reviews with concerns.
• View reviews by recommendation category and summary reports by gender, grade applying for, etc.
• View review exceptions, e.g. applications with mismatched recommendations, “important” rejections (e.g., applications of candidates related to alumni), and recommendation fittingness.