September 30, 2014  Dawn Cullity  Bernadette Ferro  Kent Phillippe VFA Year Two: Show & Tell

Then VFA Overview Now The Voluntary Framework of Accountability is the first national system of accountability SPECIFICALLY for community colleges by community colleges.

User Experience AACC’s Objectives for the VFA  Improve the user experience  Increase utility (so we can meet all the goals listed here) Value-added metrics for accountability (using data to measure performance and make better decisions) National benchmarking for good practices Opportunity to streamline reporting / relieve IR capacity Ability to tell a better, more robust story Better data to inform public policy and advocacy Utility

feedback Objective: Improve Data Input  Feedback on the user set-up and data collection process/data tool  Stability of, and access to, the online collection and display (dashboards)  Tool is clunky, especially the online forms, and needs better validation processes  Workflow is not intuitive and progress bars does not tell us where we are in data collection process improvements

CohortBulk Upload Online Data Entry Raw Data File Upload Not Reporting 2008 Cohort (6 year) OOOO 2012 Cohort (2 year) OOOO CTE OONot AvailableO ABE OONot AvailableO Submit Workflow: Data Configuration

Workflow: Managing Data Collection WORKFLOW ACTIONSSTATUS / NOTES Configure Data “Data last configured on…”– with a list data configured (6 year, 2 year, CTE, and/or ABE) or “Data needs to be configured” Upload Data File(s) If no files to be uploaded show nothing. If files have been uploaded “data last uploaded on.../../..” and lists each file type that has been configured and the date uploaded Edit/View Data Dimmed if data not configured – “Data needs to be configured” Dimmed if data upload files are in the queue – “Data in the queue to be uploaded” If all data is Processed – allow user to select. Data Validation Checks Mark Data for Review If Data configured – allow for the user to click Mark for Review – ‘Display the last notes stored by the users of the system for review’ Publish Data Dashboard and ExportsTakes user to Exports page Public Outcomes ReportsDimmed if data has not been made public yet

VFA Manual Data Entry

VFA Cohorts Disaggregated

feedback Objective: Increase Utility of VFA Exports  Feedback on the use of the VFA and its data  Dashboards need improvement to be useful and understandable  Colleges need to be able to download final values including data and calculated percentages  Benchmarking parameters and policies  Dashboards need a lot of interpreting before passing to senior leadership/board improvements

VFA Export: Outcomes Dashboard

VFA Export: Tabular Data

VFA Benchmarking Dashboard: Benchmarking Selectors

VFA Benchmarking Dashboard: Six-Year Outcomes

VFA Export: Public Dashboard Report

feedback Objective: Improve Data Collection Resources  Feedback on the resources (instructions)  Clarity: make the Metrics Manual user friendly, understandable  Consistency  Authority: be definitive with methodology and how metrics are to be reported improvements

Cohorts Reporting VFA Measures Six Year Cohort Two Year Cohort Main Credential Seeking First Time In College Developmental Education Progress Measures Two-Year Progress Measures Six-Year Outcomes Measures Career & Technical Education Measures Adult Basic Education Measures CTE Cohort ABE Cohort

VFA Metrics Manual

VFA Metrics Manual (cont.)

VFA Overview & Training Webinars

Y2, Y3, Y4… Close-Out  What do you think should be our priorities going forward?  Questions & Feedback  Contact VFA at principle accountability system Reminders  Sign-ups open October 1 st  Data collection opens November 3 rd

Questions During the Webinar Data Dashboards  Have the dashboards been tested with IE, Firefox, and Chrome? The VFA dashboards are supported on the following Internet browsers:  Internet Explorer (IE 8 – IE 10)  Firefox (16.x – 22.x)  Google Chrome (22.x – 28.x)  Please clarify public vs private dashboards. Upon publishing your VFA data, the VFA makes a link available on your College Profile page that takes the public user to your Outcomes data. The Outcomes data are presented to the user in a packaged PDF report, again made available on a link from your College Profile page. All information that is contained in your Outcomes Dashboard is packaged into this public PDF report. The Outcomes data includes the “All Students” (not disaggregated) counts of the following: Developmental Education Progress, Two-Year Progress, and Six-Year Outcomes Measures; CTE Profile & Completer/Leaver Outcomes, and ABE Cohort Outcomes. Raw data, disaggregated data, and the benchmarking tool and data will not be available to the public. We will send the user group a sample in the next few weeks.  Will the new dashboards show data from previous years’ data collections? Yes. The new Outcomes and Benchmarking dashboards will render all data from Year One ( data collection) onward. The Public Dashboard Report – a PDF report showing the graphs or your published Outcomes data –is only available for VFA Year Two ( data collection) ONWARD. To the public, the link to this report will always be for the most recent year.  Can we view the names of benchmarking institutions selected? Currently, the tool does not return a list of the colleges that are in the benchmarking group. AACC is working with the VFA Oversight Board and the dashboard developers on both the policy and technical changes needed to make this happen. We have receive this request and are working toward fulfilling it. Please note that the benchmarking tool will always use selector criterion to establish the colleges in the benchmarking group; colleges will not be able to pick individual colleges against which to benchmark. Questions & Answers

Questions During the Webinar (cont.) Raw File Upload Process  More info about the "raw data" upload The Raw Data upload process requires the college to upload 5 files: 1.Student demographics, 2.Student developmental education placement, 3.Student course data, 4.Completions at two years, and 5.Completions at six years. The data will be extracted, translated and loaded into the VFA database. After the ETL is completed, the data from the five files will be permanently deleted from the system. The VFA does not keep student-unit record data.  If the raw data upload method will become the "method of choice," will the bulk upload method eventually be abandoned? The VFA has no plan to abandon any of the three data input methods at this time. We anticipate that many users will move toward the raw upload method because it is a process where data extraction is streamlined and methodology is preserved. Like all data initiatives, though, we understand that users work differently and have preferences for how to enter data. Questions & Answers

 What data can be downloaded to Excel? All data that you entered for your college can be downloaded to an Excel document, so that the college may manipulate their own data. Downloading benchmarking data requires more development and is not available for Year Two.  How is the VFA handling the "English/Reading" combination topic going forward? For Year One, the VFA allowed colleges who offered English and reading under a combined topic to report the progress and outcomes of these students under an English/Reading combination subject. There was confusion and errors in reporting, so for Year Two we ask all colleges who offer the combined subject to report it under the English subject. For these colleges, the reading subject area will be null.  When will the new versions of the VFA Metrics Manual and User Guide be released? There are aspects of the Metrics Manual and User Guide that require all other data collection development to be completed. Therefore, the VFA Metrics Manual will be released on November 3 rd, when data collection opens. The User Guide will be released soon after.  Is there an easy way to report SAM (Student Achievement Measure) using VFA?Student Achievement Measure The data reported for SAM’s associate/certificate model is a roll-up of the VFA’s Six- Year Outcomes for the full-time and part-time, credential-seeking cohort. If a college reports the VFA, it can easily report SAM. The SAM folks have been working with their developers on a file that helps with getting VFA data into SAM. We will update you on this post-October 20 when SAM launches. See the Resources section of the SAM website for more information: Questions & Answers Questions During the Webinar (cont.) Misc.