1. — Project Metis – Flexible Academic Careers Acknowledgement to Country
DSC College Academic Forums, March 2018 1

2. — Project Metis – Flexible Academic Careers
Today’s agenda:
Welcome - PVC Business
Project Metis overview and update - DVCE
Proposed performance measures for Academic Promotion DVCE, DPVC R&I
Questions/ feedback channels before VCE decision (April 12)
College Academic Forums, March 13, 2018 2

3. PVC welcome In Business the new Academic Promotion framework will:
Support our targets for gender equity
Support our diverse workforce through a new set of ARtO principles to ensure that personal and professional circumstances are respected and accounted for in career development
Allow our academic staff to focus on their passion, through criteria that recognise and reward excellence in both teaching and research
Recognise the academic leadership role of program managers under new Engagement criteria
Replace Academic Expectations targets with metrics that respond to variation in discipline and other factors
Provide a more robust and criteria-based framework for evaluating individual academic achievement and evidence.
For individual PVC/ College emphasis

4. Today’s Objective
Update the academic community on the new Academic Promotion framework
Seek feedback on the proposed performance measures for the promotion domains
Consult with the academic community before VCE endorsement of the framework for the 2018 round on April 12
DVCE welcome and forum objectives

5. Your feedback - GoSoapBox
During this session you have the opportunity to share your feedback and ask questions
Website (accessible via smartphone):
gosoapbox.rmit.edu.au
Access Code:
________________________________________________________________________

6. The Metis story so far External drivers for change Metis Objectives
Your feedback
Internal drivers for change
Objectives - Addresses the career development requirements for a range of RMIT educational roles,
Articulates the underlying principles for academic progression, role profiles and career pathways at RMIT,
Includes academic promotion processes with clear parameters, descriptors and benchmarks for a range of roles at all academic levels,
Anticipates the future requirements and drivers for academic work at RMIT.
Feedback
A single flexible academic career pathway must encourage and allow academic staff to focus on their passion, with the best way to encourage this focus being a mechanism to recognise and reward excellence in both teaching and in research.
A flexible career pathway should allow researchers to research and teachers to teach - and afford the same opportunities for promotion linked to excellence in either.
The framework should recognise time spent developing new course materials and other demonstration of innovation in teaching.
The framework should recognise the time and effort involved in running programs with large numbers of students and/or many courses.
The current Academic Expectations targets are not fit for purpose and need to be revised for variation in disciplines and delivery (e.g. class size, gender, employment status).
The current promotions standards are too subjective and not criteria/ evidence-based.

7. After 2 years we have a new framework
New Domains with criteria by academic level
New mean GTS and associated categories for closer analysis of data and influences
Seven new discipline clusters for Research metrics
Recognition of the breadth of Engagement practice, rather than just “Leadership”
Recognition of Achievement Relative to Opportunity
The NEW framework

8. CURRENT Academic Promotion Framework
The previous framework uses three domains of academic practice with minimum standards by academic level and no separate criteria for each domain.
Teaching
Leadership
Research
Evidence of strong and sustained performance in student outcomes, student experience and feedback and extensive experience in a range of teaching settings
Motivating, influencing, developing and inspiring others through discipline leadership and by translating the University’s vision and strategy into meaningful actions at the local level
Generation of knowledge and its application to the solution of real world problems (related to research or learning and teaching)
Won’t be going into detail about the current framework as many of you will have used it in the past. Information is available about this on the website
Main difference is that this was not criteria based

9. NEW Academic Promotion Framework
The new framework has three domains of academic practice, each with assessment criteria and practices/outputs by academic level.
Teaching
Engagement
Research
Criterion 1
Research outputs
Teaching, assessment and feedback
Engagement within RMIT
Criterion 2
Student support & development
Criterion 2
Engagement with industry, business, community, arts and cultural organisations and sectors and/or government
Criterion 2
Research income
Criterion 3
Scholarly teaching and the scholarship of teaching
Criterion 3
Higher Degree by Research supervision
Criterion 3
Engagement with the discipline or profession
Criterion 4
Design and development of learning and assessment
Each domain has:
Specific criteria
Specific practices or outputs against each criterion at each of the five academic levels
Indicative qualitative and quantitative evidence to demonstrate effectiveness, impact and quality

10. Current New 2018 is a transition year
Applicants can choose to apply under either the Current or the New framework
Current
New
Previous promotion framework with;
Benchmarks by academic level
Research metrics and GTS from Academic Expectations
Current performance rating (i.e. Not Satisfactory - Outstanding)
New promotion framework with;
Defined criteria for each domain
Practices/outputs by academic level
New research metrics by discipline group
New mGTS and teaching improvement score 10

11. NEW Framework - our approach in 2018
2018 is a transition year and an important one in determining the future of academic progression at RMIT. The new framework introduces a strong foundation for us to build on:
For the first time, we have a criteria-based system of practices/outputs specific to academic level
The proposed Teaching and Research metrics are more granular and provide context for building a case of individual achievement
In 2018 we're starting with some basic performance measures that will become more sophisticated over time. With your continued input we will develop other evaluation methodologies, as well as a guided approach to scholarly reflective practice. We hope to also develop engagement metrics as we become more mature in our application of this domain. This round provides opportunity for action-learning.
The option of applying under the current framework OR the new means that every academic can choose how they would like to present their promotion case this year.
IMPROVEMENT - the OLD framework
does not differentiate criteria between levels – what constitutes outstanding or excellent or good is undifferentiated
makes no reference to expectations or practice at level
is unmediated by disciplinary context

12. Applying under the NEW Framework
Everyone must apply under the Engagement domain
Then choose to respond to the Teaching and/or Research domains to best reflect your practice
Leadership can be demonstrated within any domain
ALL criteria must be addressed under each domain

13. Qualitative and Quantitative performance measures and evidence
Both quantitative and qualitative evidence may be drawn upon to address the criteria in all domains
All evidence should support claims of impact and achievement
Both quantitative and qualitative evidence should build a case for promotion potential
For Research, minimum quantitative metrics (Outputs, Income and HDR completions) are set by academic level and discipline cluster
For Teaching, minimum quantitative metrics (the GTS plus the new change in GTS) are not set by academic level

14. NEW Engagement performance measures
Both qualitative and quantitative measures can be used in responding to the Engagement criteria (e.g. changes made to practices, products or services as a result of consultancies, income sourced through partnerships, industry publications, textbooks, etc.)
There are no standardised quantitative measures set as benchmarks
Income can be reported in this domain or under Research where allowable
Measures used will be appropriate to context and can support claims made about the quality and impact of the engagement activity 14

15. NEW Teaching metric
The mean GTS allows greater exploration of teaching data. It has potential to be modelled with other factors (e.g., course size, gender, etc) to:
Build a case for good teaching practice
Show evidence of:
sustained excellence
improvement over time
impact of new innovative teaching methods

16. NEW Teaching metric - Why change to mGTS?
The % agreement is too coarse a measure to reflect teacher performance accurately (teachers could have the same % agreement and yet vary greatly in feedback)
The mGTS is a finer measure that more clearly illustrates differences in student feedback. It can also be used to drill down into categories to reflect GTS relative to gender and school/ discipline
The mGTS is less susceptible to variation in response rate
The mGTS provides a clearer view of longitudinal teaching performance

17. NEW Teaching metric - the mean GTS
Previously
2018
The % agreement GTS was too coarse a measure of teacher performance
The new mean GTS (mGTS) allows greater exploration of teaching data and illustrates differences in student feedback
eg. 65% agreement

18. New Assessment of GTS performance
The applicant’s mean GTS will be provided. The mGTS can be compared against the RMIT population, the applicant’s School or Program
mean GTS
*Rating bands used for GTS align to current Academic Expectations

19. NEW Research metrics
The new quantitative metrics for Research have been developed in response to feedback about the need for discipline variation
There are benchmarks for 7 new discipline clusters:
Business
Creative Arts/Design
Humanities
Social Sciences
Engineering
Natural and Biomedical Sciences
Mathematical and Health Sciences
Benchmarks are set by academic level and research workload allocation 19

20. NEW Research metrics - Social Sciences
Academic level
Research Workload Allocation
Average Annual ERA eligible Research Outputs
Average Annual Research Income $K
Average Annual HDR Completions A
30%
0.33
50%
0.67
≥85% 1 B 10 2 25 ≥3
≥40
≥0.33 C 3 40 ≥4
≥70
≥0.67 D 5 70 ≥6
≥100
≥1.33 E 6
100
1.33 ≥7
≥150
≥1.67

21. NEW Research metrics - Creative Arts/Design
Academic level
Research Workload Allocation
Average Annual ERA eligible Research Outputs
Average Annual Research Income $K
Average Annual HDR Completions A
30%
50%
0.33
≥85%
0.67 B 1 2
7.5 ≥3 15
≥0.33 C 3 ≥4
≥20
≥1.00 D 5 20 ≥6
≥35
≥1.33 E 6 35
1.33 ≥7
≥50
≥1.67

22. NEW Research metrics - Humanities
Academic level
Research Workload Allocation
Average Annual ERA eligible Research Outputs
Average Annual Research Income $K
Average Annual HDR Completions A
30%
0.33
50%
0.67
≥85% 1 B 2 10 ≥3
≥25
≥0.33 C 3 25 ≥4
≥40
≥1.00 D 5 40 ≥6
≥70
≥1.33 E 6 70
1.33 ≥7
≥100
≥1.67

23. Important notes on proposed Research metrics for DSC23

24. NEW Framework: responding to the criteria and measures
Building your case for promotion
It is important to provide a context for the panel to understand your individual achievement, so the emphasis should be on a holistic narrative built on an evidence base that speaks to the impact and outcomes achieved and draws on qualitative and quantitative evidence as relevant.
Context
Evidence
Impact
NB: you could use a framework to present your evidence like the evidentiary matrix (where evidence is organised against the dimensions of Self, Stakeholders, Achievement and Impact).
We have support for this approach built into the new Readiness/ Self- assessment process

25. Describe the context Joanne is a Level C academic in Urban Planning.
Jo teaches and coordinates a course and is a program manager
She has a research allocation and is part of a small research team working on battery technologies
She has several HDR students and is a secondary supervisor of another
Jo maintains a network of industry/government contacts that interact with her courses by providing visits, guest lectures etc,
She has recently experimented with some interactive augmented reality teaching technologies to engage her students
She has introduced a social media approach to the program she manages
She was the first author on 4 of 12 papers published by her research group
She is lead CI on one ARC Linkage grant
Last year she was invited to present her research at a prestigious European Housing and Planning Conference
She is on the Editorial Board of a major journal
She is a member of Academic Board

26. Align evidence to criteria
Indicative practices
Teaching criterion 1 - Teaching and assessment
Innovation and creativity in teaching
Teaching criterion 2 - Student guidance and support
Serves as a student advisor at course/program level
Teaching criterion 3 - Scholarship of Teaching and Learning
Coordinates work-based learning activities at a course level
Teaching criterion 4 - Learning/ Assessment Design and Development
Demonstrates innovation in the use of learning technologies
Research criterion 1 - outputs
Significant and high impact papers related to industry
Research criterion 2 - grant income
Membership of external grant application team
Research criterion 3 - HDR supervision
Updated supervisor training
Engagement criterion 1 - engagement at RMIT
Course coordination including the management and leadership of staff
Engagement criterion 2 - engagement with external industry bodies
Solutions to significant industry problems (research translation)
Engagement criterion 3 - engagement with the discipline
Contribute to industry conferences
Joanne’s evidence
My GTS score was 75% last year and has moved to 80% this semester after using the augmented reality techniques and social media innovation. The pass rate for my course for last 3 years has been is 70% - average pass rate for comparable institutions is 30%
After presenting my use of augmented reality as a teaching enhancement at a discipline forum, I was invited to run a workshop at our school PD day. The teachers in my course have now all adopted my AR design and the Program Manager has introduced it more widely across other courses. That along with the social media strategy are impacting more than 5,000 students. The outcomes of my research team securing an ARC Linkage grant of $300K have had sector impact and led to further research opportunities. I am first supervisor on three HDR and secondary supervisor for one HDR student and my previous 4 students completed this year. Of these students, two have gone onto academic careers, another has won a prestigious fellowship. 3 of my 4 first authored papers were in Q1 journals over the last three years and I regularly contribute to The Conversation. I have been invited to join an industry round table for the development of national urban planning guidelines which have been implemented and have wide sector impact.

27. Demonstrate the impact
Indicative practices
Teaching criterion 1 - Teaching and assessment
Innovation and creativity in teaching
Teaching criterion 2 - Student guidance and support
Serves as a student advisor at course/program level
Teaching criterion 3 - Scholarship of Teaching and Learning
Engages a team in reflective practice on work based learning
Teaching criterion 4 - Learning/ Assessment Design and Development
Demonstrates innovation in the use of learning technologies
Research criterion 1 - outputs
Significant and high impact papers related to industry
Research criterion 2 - grant income
Membership of external grant application team
Research criterion 3 - HDR supervision
Updated supervisor training
Engagement criterion 1 - engagement at RMIT
Course coordination including the management and leadership of staff
Engagement criterion 2 - engagement with external industry bodies
Solutions to significant industry problems (research translation)
Engagement criterion 3 - engagement with the discipline
Contribute to industry conferences
Joanne’s impact
As program manager the teaching innovation impacted 5,000 students. Students expressed that they were more engaged and attendance in classes improved by 50%. scores and NPS lifted by a stupendous 22%. Team collaboration has been high.
The grades of students across the course also increased. Our low SES students who more likely to struggle reported a 12% improvement in grades. The social media strategy has improved our engagement . My research was been adopted by planning authorities in the development of new guidelines. Planning industry stakeholders have provided further funding to continue research.
Students use my work-based AR projects to successfully seek employment on graduation. I am also reflecting on the effectiveness of my WIL activities through a scholarly framework to inform my learning technology designs going forward. I am considering publishing in the field of Planning Education with an industry partner.

28. What will be considered in your case for promotion
Key Achievements
Qualitative Evidence
Quantitative Evidence
Have the practices/outputs by criteria been demonstrated and appropriate for level ?
What is the context of the individual’s practice? (incl ARtO if applicable)
Engagement: no standardised measure
Is there evidence of effectiveness & impact?
Teaching: mean GTS
(Unsatisfactory/ Satisfactory/ Excellent)
What other evidence supports the case?
Research: meets criteria by discipline cluster at academic level & load?
(No/ Yes)
For D-E only: is there evidence of strategic alignment?

29. Your feedback Specific metrics questions?
How confident are you in the overall proposal?
What do you like?
What concerns remain for you?
Give your feedback via GoSoapbox
Send your feedback by Friday March 16 to:
________________________________________________________________________
Give your feedback NOW!
gosoapbox.rmit.edu.au
Access Code:

30. Key dates March 9-16: Final college consultations
March 16: Deadline for metrics
Late March: Website launch - applicant information and BOOST mentoring registrations open
April 12: VCE meeting to endorse metrics for the new promotion framework
Mid April: BOOST Mentoring for Academic Promotion Workshops
May 2: Townhall launch of the 2018 Academic Promotion round
May 2: BOOST Mentoring for Academic Promotion commences
May: Academic Promotion drop-in sessions
May 22: Head of School/Dean information session
Late May - early July: Applying for Academic Promotion workshops on both frameworks
August / Sept: Panel coaching
Send your feedback to:
Read about Academic Promotions at: and-career/academic-promotion
need more dates here