Office of the Provost and Vice President for Academic Affairs

Thank you for your assistance in meeting this data demand.  The following questions and responses are designed to help you better understand this process and how Academic Affairs will collect data for academic departments.  

Process Specific Questions (PROC.)

PROC. Q1: Why are we conducting this activity?
A: The UNT Board of Regents (BOR) wants assurance that UNT is maximizing resources to provide our students the best possible education. To that end, BOR has asked that the Vice President for Academic Affairs group academic departments into High, Medium, and Low Performing categories. 

PROC. Q2: From where is the review data derived?   
A: A mixture of data will be provided from the EIS system and from the Department Chairs.  A clear set of definitions necessary to ensure data integrity and to ensure comparable data consistency will be provided from each of the sources listed above.  Item-by-item descriptions for concrete variable parameters are included in this document.

PROC. Q3: What are the years being examined?
A: Generally speaking, we will use fall 2013 official information as reported to the Texas Higher Education Coordinating Board (THECB). Graduation rates rely upon a cohort that started at UNT 6 years ago (undergraduates), 3 years ago (masters students), and 10 years ago (doctoral students).  When one full year of student data is required (i.e. degrees awarded, grant expenditures), the previous academic year (Fall 2013 to Summer 2014) will be utilized. 

PROC. Q4: What is the methodology involved?
A: The same methodology used for Academic Program Review data calculations will be used to establish the programming for queries against the EIS database.  See below for specific details on variables.  Programmatically, IR&E will utilize SAS 9.2 to program the calculations from the official non-transactional data.  A spreadsheet will then being created to merge all of the separate calculations into one worksheet.  Department provided data points (see items below) will be gathered utilizing a Qualtrics survey.  Qualtrics data will be imported and merged with the SAS calculations. Z-Scores are then computed for each variable by looking at how each department compares against the overall average.    

PROC. Q5: Internal unit data may not match the data being used for this project.  Why?
A: Avery specific set of definitions will be used by IR&E in the calculation for the individual data points for each department.  Generally speaking, differences in various datasets are usually attributed to either differences in variable data definitions or the use of transactional vs. official, static data.   As an example, there are five different ways to classify UNT students by minority status.

PROC. Q6: How will my department be reported to the Board of Regents?
A: All academic departments will be categorized as high, medium, or low effectiveness.

PROC. Q7: How will this exercise impact my operations/budget for next year?
A: This process will help to establish where additional Academic Affairs’ funding allocations may be prioritized to advance UNT’s goals.

PROC. Q8: For each of these bullet points [data we are requesting from departments], what benchmarks are being used? UNT system? TEA standards? US Dept. of Education standards? 
A: Academic Affairs is executing Phase 1 in such a manner that the academic departments at UNT will be compared to each other when a sum of Z-scores have be calculated for each metric for each academic department. In Phase 2[NY1]

PROC. Q9: Will I be able to see my data?
A: Departmental data, including data values and Z-Scores for each data point in the evaluation matrix will be made available for your review and information. 

PROC. Q10: Can we see the specific SAS code utilized to compute these variables?
A:  IR&E will make the SAS code available.  Please know that the SAS data tables and code are based on EIS names that may not be meaningful to other who do not use these files and programs on a daily basis. Many of these SAS codes have evolved since before the implementation of Peoplesoft.

PROC. Q11: How will all these data be used by UNT system? 
A: These data will be presented to the BOR to facilitate their understanding of the High, Medium, and Low effectiveness categorization process. The Provost and President will assure the BOR that continued improvements will be made to the process.

PROC. Q12: How will these data be used by the THECB? 
A: There is no plan to send these data to the THECB.

Data Specific Questions – Items Department Collected (Dept.)

DEPT. Q1A: What parameters should I utilize to compute graduate student employment status (after graduation)?   
A: Use the same format that you’ve use to report to your Dean in each of the past several years.  In essence for academic years (2010-2011, 2011-12, and 2012-2013) utilize a simple categorization into “employed in field”, “still seeking employment” or “unknown”?  What we are asking you to provide is the percentage of graduate students in each of the three cohorts who are gainfully employed (which specifically refers to employment in the field). 

DEPT. Q2A: Who do I consider as a member of my Core Faculty?
A: All full-time faculty who are evaluated by the department chair for promotion, tenure or merit.

DEPT. Q2B: What types of Core Faculty Scholarship are you asking for? 
A: Scholarship refers to the totality of academic publications, articles, grants (see clarification below), gallery exhibits, performances, book chapters, editorships, conference presentations or papers, etc.  Please see the specific recommended data sources to provide this response.

DEPT. Q2C: Are these categories [Core Faculty Scholarship] likely to be separated in later reports (Phase 2)? 
A: Working with academic leadership team members, the Provost will determine the continued and appropriate use of evaluation metrics.

DEPT. Q2D: Does a grant count only in the year received or in every year the project is funded?
A: Please only count the grant award once for the year that the award took place.

DEPT. Q2E: Will grants be reported somewhere else as dollar revenues? 
A: One of the metrics to be calculated is the department’s total grant expenditures for the last academic year.  See below for more clarification on this methodology in the next FAQ section.

DEPT. Q2F: Should we be prepared to break these out separately for future reporting?
A: Not at this time.

DEPT. Q2G: What is an acceptable data source for the Core Faculty Scholarship figure?
A: As noted in the survey, these data can be drawn from Academic Analytics, Faculty Profile System, or internal records/other sources which can be verified. Consistent with the deans’ request, the scholarship value will be computed for the past three calendar years. The total number of Core Faculty Scholarship for the past three calendar years will be totaled and then divided by three to calculate a three year total average for each academic department.

DEPT. Q2H: Should we exclude other units of productivity and just count the four categories you mentioned (books, articles, presentations, performances)?
A: No.  See DEPT. Q2B: above.

DEPT. Q2I: An article/book/presentation/performance that is co-authored by two people inside the department counts once or twice?
A: It will be counted twice.

DEPT. Q2J: May we “count” grants for which we have applied, or only those that have been awarded?
A: Only awarded grants may be counted in this metric.

DEPT. Q3A: For the item “Department’s Total Recruiting Scholarships” - does "recruiting" refer only to scholarships for incoming students or does it also refer to scholarships for continuing students we want to retain and graduate?  Should we exclude GATS in the scholarship category?
A: Please account for the total value of the scholarship dollars distributed in the last academic year to all students who had a portion of their tuition and fees paid by the academic department. Please do not include stipends or GATS allocated by the university in this count.

DEPT. Q3B: What is the time frame for this total? 
A: Please use the previous academic year (2013-2014) total scholarship distributed to student within the academic department that were used to pay a portion of that student’s tuition and fees that was not a stipend or GATS. 

DEPT. Q4: What are you looking for when it comes to program goal measurement?  
A: Has the department documented program goals for each of the degrees found in the academic department?

Data Specific Questions – Items Academic Affairs Collected (VPAA)

VPAA Q1: How do you calculate the Number of Degrees Awarded per Year?
A: This is the total number of degrees that was awarded for the 2013/14 academic year by the academic department as recorded in EIS. 

VPAA Q2: How do you calculate the Graduation Rate by Cohort Year?
A: A cohort of entering students in the fall term will be identified for each degree program within an academic department.  The graduation rate calculation will be as follows; a six year graduation rate will be used for full-time, new from High School, undergraduates (cohort Fall 2007); a three year graduation rate will be calculated for new Masters students (cohort Fall 2010); a ten-year rate will be calculated for new doctoral students (cohort Fall 2003). The graduation rate is the percentage of the sum of all students from each of the degree level cohorts who graduate within the specified time frames divided by the total number of students in the cohorts.   

VPAA Q3: How do you calculate the Time to Degree by Cohort Year?
A:  Given the need to utilize a z-score approach for three degree levels, the years to graduation are tracked for each student from the time of matriculation to graduation. This raw score is then compared against the total percentage of time expected for a student to actually graduate (six, three, or ten years) for each degree level and a time to degree percentage is calculated for each students across all degree levels (e.g. a first year student who graduated in 4 years would be awarded a score of 66%, yet a student who graduated in 6 years took 100% of the expected time, a doctoral student who graduates in 5 years would be awarded a score of 50%, etc.).  These percentages are then summed for all students from each of the degree level cohorts and then divided by the number of students who graduated in the department to arrive at an overall time to degree statistic. This process does not penalize a department who may only offer Baccalaureates or conversely a department who only offers Masters and Doctoral degrees. 

VPAA Q4: How do you calculate the Admission Criteria – Average SAT Score?
A:  The decision SAT score used for admissions will be identified for each new from High School student in the department.  All scores will be added together and divided by the total number of student identified.  

VPAA Q5: How do you calculate the Percentage of Full-Time Graduate Students with Financial Support?
A: A data file of graduate students employed in a program will be provided by UNT System HR.  This list of employed students who receive paychecks is compared to the EIS list of all full-time graduate students within the department.  A total percentage will be calculated. Part-time graduate students are not factored into this statistic.

VPAA Q6: How do you calculate the Average Financial Support Provided (Monetary Value)?
A: As defined in the UNT Program Review process, Average Financial Support encompasses graduate students who are RAs, TA, GAs, GRAs, and Fellowship Recipients.  This value will be determined by taking the HR file listed in VPAA Q5 and calculating the total annual payroll for the graduate students within the academic department.  This total payroll value will be divided by the number of graduate students employed in the department to arrive at a final statistic.  These figures do not include undergraduate student employees in the department.  No department awarded scholarships are included.

VPAA Q7: How do you calculate the Core Faculty to Student (FTE) Ratio?
A: A student FTE value is based on the number of SCH taught in the department. It is the sum of undergraduate SCH hours (divided by 15) + masters SCH (divided by 12) + doctoral SCH by 9. This FTE value is then divided by the number of all full-time faculty within the same academic department.  For purposes of this data point Core Faculty include Tenure, Tenure-Track, and Lecturers.  Visiting, adjunct, or temporary faculty are not included.

VPAA Q8: How do you calculate the Core Faculty External Grants?
A: These data are from the Division of Research and Development.  This calculation is total dollars secured by each member of the Core Faculty awarded in the 2013-14 academic year.

VPAA Q9: How do you calculate the Percentage of Full-Time Students (FTE) to Headcount?
A: The FTE is the SCH attempted by departmental majors calculated as follows: for undergraduates, hours attempted/15; for masters, hours attempted/12; for doctoral, hours attempted/ 9. The calculated FTE is then divided by the total headcount majors in the department.

VPAA Q10: How do you get the Number of Core Full-Time Faculty?
A: Provost’s office provides a list of the core faculty.

VPAA Q11: How do you calculate the Student Diversity – % Minority?
A: Based on the EIS enrollment file, this is the percentage of non-white/Caucasian students of the total student population. 

VPAA Q12: How do you calculate the SCH Generated?
A: The SCH taken by students in all of the department’s course offerings.

VPAA Q13: How do you calculate the Student Satisfaction Survey (4 Questions from Graduation Survey)?
A: Six items that are consistent across both the Graduating Senior Survey and the Graduate Student Survey were averaged across each department on the instruments for the 2013-2014 academic year.  These six averages were then summed for a total score.  These items were on a 1-5 scale.  The survey questions included are:

1. How much of the knowledge you have gained while at UNT is applicable in the workplace?
2. My degree from UNT has brought me closer to achieving my goals
3. My experience at UNT was rewarding
4. My coursework at UNT was challenging
5. My degree is marketable
6. How would you rate the quality of your academic program at UNT

VPAA Q14A: How do you calculate the Cost per course by Instructor Rank? 
A: Teaching faculty salaries in the department/ the sum of the number of courses (lecture, labs, and seminars) taught by the department.

VPAA Q14B: Cost per course by Instructor Rank:  Is this credit hour cost? And what are categories of “instructor rank”? Assistant prof? Associate prof, full prof, etc.? Or something else.
A. As mentioned in item VPAA Q14A: We do not break out the costs by rank.

VPAA Q15. How do you calculate the Program Facilities and Equipment- Gross Square Feet?
A: The Office of Space Management provides a total gross square footage for all instructional space, offices, laboratories, etc. (CONFIRM) for 2013-2014 academic year.

VPAA Q16. How do you calculate the Program Finances and Resources – Allocable Budget
A: The total allocable budget will be received from the UNT Budget Office.  This figures will be for the FY2015 allocable budget that was presented to the UNT Board of Regents.