ANTF Response to the TEF

Response to the BIS Committee about the proposed Teaching Excellence Framework (TEF) on behalf of the UK Association of National Teaching Fellows

By Sally Brown, CANTF Chair

Click to download a PDF of this post –> CANTF TEF response 30 OCTFINAL

This response is submitted on behalf of members of the UK National Teaching Fellows community as represented by the Committee of the Association of National Teaching Fellows: all NTFs have had a chance to view and contribute to this response. We have chosen particularly to respond to questions 3 to 6.

National Teaching Fellowships are awarded to higher education teachers who have been recognised at a national level as excellent university practtioners through a competitive, peer-evaluated process and are therefore uniquely placed to comment on teaching excellence. Each university and all FE colleges that meet certain criteria can submit up to three submissions per year, and the scheme started in 2000. Originally 20, and now a total of 55 university academics and professionals who support HE learning from England, Northern Ireland and Wales are awarded National Teaching Fellowships each year. The National Teaching Fellowship community comprises around 650 members who are represented by the Committee of the Association for National Teaching Fellows.

In July 2015, the Association of National Teaching Fellows organised a TEF consultation event in Oxford, attended by representatives of BIS and HEFCE, and the present response draws on that event as well as on contributions from other National Teaching Fellows.

Sally Brown

Sally Brown focus group 2014 NTF Symposium

Summary

National Teaching Fellows are delighted that the government is aiming to recognise the importance of excellent teaching and acknowledge the value of professionalism, giving teaching equal status with research in universities.

We are keen that the TEF should be multi-dimensional, using data already available to universities as far as possible. We suggest that measuring teaching excellence is not a simple or straightforward task, and that the proposal to use existing metrics in the first year of the scheme is likely to result in skewed outcomes. There are widespread and genuine worries that the wrong metrics for a teaching excellence framework will be chosen. We are keen that any measures used to recognise excellent teaching are ones truly representing improvements which add value to the student experience, brought about by interactions between HEI teachers and students, rather than extraneous factors representing existing differences in the level of advantage of the student body in different institutions.

We are very concerned that if the TEF process is expected to serve too many conflicting purposes (for example, both guiding student choices and providing a justification for the increase of above-inflation fees), it is likely to prove unmanageable. We do not welcome any linkage between the TEF and fee levels. NTFs, as the only community of peer-reviewed and nationally-recognised excellent HE teachers would be very well placed to be represented on any group taking the TEF forward.

National Teaching Fellowship applications are prepared annually by HEIs seeking to gain recognition for their excellent teachers, and we propose that these applications could provide a readily-available set of case studies demonstrating impact and pedagogic leadership.

Question 3: What should be the objectives of a Teaching Excellence Framework (‘TEF’)? 

The TEF is a means by which individual and collective teaching excellence is to be recognised in universities and other institutions offering higher education, including Further Education Colleges, (hereafter termed HEIs) giving equal status to teaching to that accorded to research. In recent years there has been an imbalance in esteem, caused largely by HEIs over-focusing on the Research Excellence Framework, thereby giving perverse incentives for HEIs to overlook the importance of teaching.

a. How should a TEF benefit students? Academics? Universities? 

A TEF should benefit prospective students by allowing them to judge the relative teaching excellence of different departments or schools (rather than institutions), as they consider where they want to study. A TEF should benefit existing students, as institutions/departments strive to increase their teaching excellence as measured by the TEF. Furthermore, TEF and other initiatives being advanced by the funding councils can offer greater opportunities for students to engage meaningfully in and thereby influence quality assurance and enhancement activities.

A TEF could benefit academics whose teaching is excellent, giving them greater recognition for their work, and could incentivise HEI teachers who wish to enhance their teaching to place an increased focus on the importance of this aspect of their work. This could thereby counterbalance the current excessive prioritisation of research by some HEIs by emphasising the value of teaching and the scholarship of teaching (Boyer, 2014). A TEF could reinforce the importance of providing sound, base-level training for teaching in higher education, offering HEI teachers better levels of confidence and self-efficacy, as well as helping their managers to prioritise ongoing professional development opportunities. We welcome further moves towards professionalising teaching in higher education. Furthermore, we welcome the opportunity to enhance teaching through more research and evidence-informed approaches, and capturing excellence through pedagogic scholarship and CPD, so that the sector more widely can benefit.

A TEF could benefit Universities and the wider higher education community by providing more information to prospective students and other stakeholders including parents and carers (who frequently financially support them) about the quality of teaching in any institution. HEIs whose overall teaching excellence was found to be high, may then attract more students, and potentially more academically-able ones, to apply to study there. Through high emphasis on teaching excellence, HEIs that make student learning central to their missions could be rewarded for their endeavours, and this in turn could encourage universities to build a system of reward and recognition for teaching excellence.

It is proposed in the consultation that as a result of being highly recognised for teaching excellence, an institution may be allowed to charge higher fees: many NTFs feel this is a very bad idea.

b. What are the institutional behaviours a TEF should drive? How can a system be designed to avoid unintended consequences? 

We consider that institutional behaviours which could be involved in a TEF should include:

  1. That the HEI recognises and rewards excellent teaching, e.g. by supporting accreditation through the UK Professional Standards Framework run by the Higher Education Academy (HEA) which offers 4 categories of Fellowship, and by promoting a cadre of promoted staff on the grounds of their excellent teaching (in some HEIs these are termed Teacher Fellows).
  2. An indication of the proportion of professors who have achieved this status on the basis of their outstanding teaching, rather than just research.
  3. Evidence that students are involved in assuring and enhancing teaching at all stages, for example, through aiding curriculum design and working as partners in curriculum review, and that there are robust systems for training, developing, supporting, valuing and making good use of student representatives.
  4. Evidence that all new-to-HE staff are trained and supported through their early years of teaching (linked to probation) including graduate teaching assistants, sessional and fractional staff, and that career-wide continuing professional development is provided for all who teach, with take up of CPD monitored. A metric could include average institutional (or departmental/school) expenditure per head on staff development concerning teaching, learning and assessment, together with average hours used by staff engaging with pedagogic development.
  5. Evidence that scholarship and evidence-based practice are valued by the HEI, as indicated for example by the number and quality of peer-reviewed pedagogic publications about teaching, learning and assessment produced, including action-research and teaching-related consultancies.
  6. Data that show the number of staff at an HEI who have achieved National Teaching Fellowships over the 15 years the scheme has been running, and how the HEI engages them in enhancing the management of change in learning and teaching practices within and beyond their own institution.
  7. Evidence of quality assurance measures meeting expectations, which result in Quality Assurance Agency, and Professional, Statutory and Regulatory Body confidence.
  8. Data on the spend per full-time equivalent student on learning environments, libraries and virtual resources for students. Such metrics must be mode-neutral between full-time and part time students, and should recognise that many programmes are managed through distance learning.
  9. Evidence that international students are selected appropriately and supported throughout their studies.
  10. Evidence that students are satisfied with their learning experiences as indicated by a basket of measures, one of which could be NSS outcomes, although many acknowledge that measuring satisfaction and measuring excellence are very different processes.
  11. Evidence that outcomes for students are excellent as indicated by retention data at each level of undergraduate degree programmes, and on Masters programmes, and for PhD completions within reasonable periods.
  12. Data demonstrating that students are employed in graduate professions, or are creating successful portfolio/freelance careers in equivalent vocational professions within five years of graduation.
  13. Evidence that assessment systems have robust moderation in place to assure standards, with effective use of benchmarks and positive external examiners’ reports.
  14. Evidence that shows that steps are taken to advise students on what comprises plagiarism and poor academic conduct, and that poor practice is recorded, acted upon and monitored against national levels.
  15. Evidence that HEIs demonstrate investment in student engagement, welfare and pastoral care through student recruitment policies, and throughout the student lifecycle, showing a commitment to inclusivity and redressing all kinds of disadvantage, particularly in terms of demonstrating and monitoring Fair Access and Widening Participation activities.
  16. Inclusion of reviewed case studies demonstrating impact on the way that the discipline is taught in the sector. Case studies in relation to individual excellent teaching already exist in the three submissions that HEIs canmake each year for National Teaching Fellowships. Similarly case studies for sustained and effective educational leadership and strategic influence exist already in the successful Principal Fellowship applications that are accredited by the HEA.

Some caveats

Indicators such as first destination of students on graduation can be valuable but are notoriously hard to collect from students who choose not to keep in touch with their universities, and such indicators are therefore innately inaccurate. Furthermore, destination data tends to be aligned to local economic factors and students’ socio-economic backgrounds.

Indicators such as the proportion of students who get ‘good’ degrees i.e. Firsts and upper seconds are readily gameable indicators, and it is acknowledged that good degrees mean different things in different universities.

Similarly high retention figures could be an indicator of good teaching and learning support but also could be more representative of universities’ intakes, so wider contextual information is necessary, including potentially divergence from benchmarks. Students who enter universities with disadvantages have higher drop-out rates (Yorke, 2004), so caution must be exercised to avoid channelling universities into risk averse behaviours, by choosing only students from privileged backgrounds more likely to stay the course. There is ample evidence that retention and achievement rates not only vary between subjects but also reflect patterns of social and economic inequality.

We welcome universities being required to demonstrate widening participation, but this is not an indication of teaching excellence. Measures of learning gain for students from disadvantaged backgrounds achieving degrees and other HE awards can be a good indicator of excllent teaching but other factors are likely to be involved. As the Prime Minister acknowledged in his 2015 Conservative Party conference speech, inequalities and prejudices impact on student recruitment processes and these in turn impact upon graduate employment outcomes.

The avoidance of unintended consequences

We suggest that measuring teaching excellence is not a simple or straightforward task, and that the proposal to use only existing metrics in the first year of the scheme is likely to result in skewed outcomes.

Outcomes of the National Student Survey may be a useful indicator at a subject level, but they measure student satisfaction rather than student engagement or learning gain, and therefore are only a poor proxy for teaching excellence. NSS results tend to vary by discipline, with students consistently reporting higher levels of satisfaction with some subjects than others.

If indicators such as NSS, first destination employment data and retention data are to be key elements of the TEF, each indicator must be benchmarked to take account of student characteristics, discipline-level differences and other factors such as the region in which a university is based. This could level the playing field between institutions with different profiles and missions in a way that existing league tables fail to achieve. Additionally, the indicators that are available can be improved: revisions to the National Student Survey for 2017 are expected to include new questions that aim to measure student engagement. HEFCE is also undertaking 12 pilot projects to explore how learning gain might be measured. The provisions of the Small Business, Enterprise and Employment Act, passed earlier this year, should also provide more accurate data on graduate earning and employment.

Among measures we would not find useful, as they represent existing advantage among students on entry, are metrics which take into account salaries on graduation, as they implicitly advantage certain subject areas and disciplines as well as enforcing differences that are implicit at entry. For example some high paying blue-chip employers only recruit from a very limited pool of universities. Some disciplines like art and design rarely produce graduates with immediate high incomes, whereas others such as banking, finance and accounting are more likely to lead to initial high salaries.

Staff student ratios tend to be related to funding of universities, and therefore are not necessarily an indication of good teaching but could provide contributory evidence of the importance of afforded to it.

c. How should the effectiveness of the TEF be judged?

The effectiveness of the TEF could be judged by how well the measures of teaching excellence listed above are reflected. Any system adopted must be multi-dimensional but should not be resource-intensive and must be seen to be fair and equitable by the sector, if judicial review and expensive complaints processes are to be avoided.

Representatives of the National Teaching Fellowship community will have confidence that the TEF is trustworthy if outcomes are based on evidence rather than historic reputations, and if the means by which judgments of teaching excellence are made are transparent and demonstrably fair.

Since there are significant variations between the performances of different departments within any university, we would propose that measures of teaching excellence should be at a subject rather than an institutional level, recognising the importance of subject differences and signature pedagogies (Shulman, 2005) although this might be problematic if the TEF is to be linked to the increasing of fees for excellent teaching.

There is a range of ways in which these two alternative approaches (Institutional level and Subject level) could be combined. One option would be to require the institution as a whole to reach a certain threshold level of performance and then allow individual units to be entered into an assessment process. Such an approach has been compared to the process by which the Athena Swan gender equality award system works, with potentially a similar gold, silver and bronze departmental rating system for teaching excellence. This approach would allow for the recognition of achievements at the level at which academic staff and students tend to sit. It would also allow for a clear comparison between courses in the same subject at different universities. However, if the link to fees remains at the institutional level, problems aggregating departmental ratings would always remain.

Question 4. How should the proposed TEF and new quality assurance regime fit together? 

Representatives of the National Teaching Fellowship community argue that the TEF and any new quality assurance regime must be manageable, in that data prepared and presented by HEIs should only be required to be presented once. This means both systems must articulate fully, and arguably form part of a single cohesive system, whilt acknowledging the distinctions between quality assurance and teaching excellence. We would not wish to see an approach that mirrors former QAA subject review, which was arduous, time consuming and subject to gaming. Nor would we wish to see over-simplistic approaches that use only currently publicly available data and which therefore could not accurately reflect the dimensions of excellence for which we argue.

Representatives of the National Teaching Fellowship community have welcomed the support of HEFCE, DENI and HEFCW over the 15 years of our existence, and consider that HEFCE and its agencies would be well placed to offer a coherent approach to the administration of the TEF and QA regime.

Question 5. What do you think will be the main challenges in implementing a TEF? 

Representatives of the National Teaching Fellows community in our discussions agreed that deciding on the most appropriate metrics for teaching excellence is very challenging indeed.

Appropriately balancing the different elements of data used to reflect the teaching excellence of a whole institution seem to us impractical and we would therefore argue for a subject-based approach. Alternatively, working out how best to establish teaching excellence in particular disciplines, rather than whole institutions, where individual departments/schools are measured in terms of teaching excellence could be viable.

Question 6. How should the proposed connection between fee level and teaching quality be managed?

a. What should be the relationship between the TEF and fee level?

Representatives of the National Teaching Fellowship community feel it is extremely hazardous to link increases in fees above inflation to TEF scores until the process has been piloted and stress-tested, and hence we argue this should not be done in the first phase of TEF. Only when an approach is arrived at which is demonstrably fair should a linkage with higher fees be considered if at all.

b. What are the benefits or risks of this approach to setting fees?

Benefits of linking teaching excellence to increased fees might provide some students with clearer choices when selecting courses and universities. Additionally, as discussed above, the knock-on effects of such a linkage may drive a range of desired behaviours within institutions.

Risks of linking teaching excellence to increased fees include:

  • Potential failure to have confidence in the system as fair, thereby leaving universities open to legal challenges and judicial review;
  • Problems concerning different fees being set for different subjects in the same HEI, and the complexities of administering them;
  • Potential risks that such a regime might be perceived as allowing universities offering excellent teaching being only available to students who can afford them, consigning other students to what might be seen as ‘second best’. Only generous and accessible bursaries for students from disadvantaged backgrounds could redress this problem;
  • Risk that international students, who are currently recruited by most HEIs, considering that only those universities charging high fees are worth applying to;
  • Further reinforcement and hardening of existing prejudices about the relative qualities of different HEIs and different subject areas which are currently masked by equivalent fee regimes.

  

References

Boyer, E. L. (2014) Scholarship reconsidered: Priorities of the professoriate.

Shulman, L. (2005) The signature pedagogies of the professions of law, medicine, engineering, and the clergy: Potential lessons for the education of teachers. In Talk Delivered at the Math Science Partnerships (MSP) Workshop: “Teacher Education for Effective Teaching and Learning” Hosted by the National Research Council’s Center for Education, February (pp. 6-8).

Yorke, M. (2004) Leaving early: Undergraduate non-completion in higher education, London: Routledge.

 

The Association of National Teaching Fellows Annual Symposium 2014

The ANTF symposium took place on 28-29th April 2014 in Birmingham and included updates from the Higher Education Academy, Jisc TechDis, excellent keynote speakers, parallel sessions, Pecha Kucha and much discussion.

Thanks to Kirsten Hardie, Laura Ritchie who took photographs of the event, and more of these can be seen in addition to my own in our Flickr ANTF Symposium 2014 Album.

Macdonald Burlington Hotel Birmingham

Macdonald Burlington Hotel Birmingham – a lovely venue.

This was the first NTF symposium that I had attended and this is my personal account of it. It was a total buzz for two days, and being a relatively small group of around 60 delegates, many interesting and fruitful conversations could be had.

Pecha Kucha

The Pecha Kucha were 6 minute presentations made by national teaching fellows from a wide range of subject disciplines and backgrounds, and illustrated the high level of creativity in teaching practices within UK higher education. From crochet in the biosciences, pizza boxes as a pedagogic tool, peer-assessment (pier assessment groan), and the use of games in higher education, each and every one of the Pecha Kucha were outstanding. I was most touched by two of them, Pratap Rughani who is a documentary film maker and focuses on personal and community experiences of the world, addressing social and environmental issues. The other amazing story was from Anita Peleg who is a lecturer in marketing who talked about the artwork of her mother – Naomi Blake. Naomi survived the Holocaust as a child, and Anita talked about the transition in her artwork and sculptures over the years.

I hope to gather artefacts from each of the Pecha Kucha presenters to share back to the community at a later date because I really can’t do justice to each and every one of them here.

Drinks Reception and Book Launch

At the symposium, the fellows were invited to bring along any of their publications to display, and there were so many these will be listed separately on the BOOK page! The pre-dinner drinks session on Day 1 was kindly sponsored by De Montfort University and celebrated the success of the first ANTF book “For the Love of Learning” alongside an impressive range of titles from other fellows.

 

Perusing the National Teacher Fellow books

Perusing the books

 

Association of National Teaching Fellows Annual Symposium DAY 1

Engaging students in assessment for employability development – revealing (and resolving!?) our issues. Facilitator: Arti Kumar
OR
ANTF ideas: Towards new publication developments: including plans fora publication-producing, writer’s retreat. Facilitator: Tim Bilham

Unfortunately I couldn’t attend Arti’s session. In Tim’s spot he talked about the success of the NTF book and that another book sprint is planned for this year in July. Sales of the current book are going very well, and Tim’s session raised ideas for the next book, and probably many more after that!-

Communications at the HEA.
Facilitator: William Syms, Head of Marketing and Communications, HEA

William updated us on the redevelopment of the HEA website, a mammoth job comprising of over 165,000 web pages. I think we would all agree that the content / case studies and reports on the website are super but not very retrievable, so the new site will have added search functionality. I’ve just tried out the new site and I must say the search functions work exceedingly well, and the new site looks great.

For a quick tour of the HEA website –> https://www.youtube.com/watch?v=gZpNpes9ZME

“Blog log” – Dr. Vivien Rolfe
Chair: Dr. Laura Ritchie

Viv (me) ran a session on the ANTF website and what the teacher fellows wanted it for, how to grow our community, and what types of article would be appropriate to share? It was agreed that the blog was an informal space for news and sharing good practices, and that we would not expect items to be polished and of peer-review quality. The HEA would maintain the more official face of the NTFs on their website.

Keynote: “Moore’s Law. Friend or Foe?”
Ian Yorston, Director of Digital Strategy at Radley College

Ian presented an entertaining keynote challenging us to think about the use of technology in our education settings, and how the concept of bringing your device and even bringing your own technical ecosystem to your education should be encouraged. He talks about Moore’s Law states that the power of computers grows exponentially, therefore education providers need to change their monolithic approaches to keep up. Ian was a fantastic and inspiring keynote.

 

Wine reception

Wine reception

 

Association of National Teaching Fellows Annual Symposium DAY 2

Keynote: “Collaboration”
Professor Stephanie Marshall, Chief Executive, HEA

Stephanie gave an update on the status of the HEA and how she and her team are bringing forward their plans and ideas to become a self sustained organisation more quickly than anticipated. Stephanie invited all NTFs to contact her with ideas or should they wish to get involved in shaping the future of the organisation. Go to the HEA site for updates. Stephanie and other colleagues from the HEA including Rosa Spencer were around for the entire symposium, so it was lovely to speak with them, and the Association of National Teaching Fellows is committed to supporting the HEA through the present difficult times.

“From dissertation to publication”
Sally Brown, Emerita Professor LeedsMetropolitan University

Sally Brown

Sally Brown

I’d not met Sally before but I don’t think I’ve ever learnt so much from a conference workshop. Sally talked about publishing from your dissertation, but everything she said would apply to everyone involved in publishing. Visit Sally’s website for further information (http://sally-brown.net).

The top tips for me were:
1) Don’t have time to write? Yes you do! Use every spare space in your diary, and Sally finds that a 15 minute period can be time well spent.

2) Perhaps someone within the NTF community could mentor you? She said Granny and Grandpa NTFs, not me.

3) When submitting to journals read them to get a feel for the style of article and writing, and contact the editors rather than submit speculatively.

4) Think about other publications – quick articles such as comment or opinion pieces?

5) Become a reviewer yourself where you can learn by the feedback comments of other reviewers!

“Professional Frameworks” – Professor Julian Park, Principal Fellow HEA
OR
“Games and learning? It’s a doddle”Alex Moseley & Dr. Nicola Whitton

I was sorry to miss Alex and Nicola’s session but attended the one by Professor Julian Park and Dr Anne Crook from Reading University. They presented on the UK Professional Standards Framework (UKPSF).

This was a great session. There are currently around 830 Senior Fellows of the HEA and 160 Principal Fellows, and national teaching fellows form a relatively small number of these. People can apply for fellowships sometimes through their universities as part of accredited courses, or directly to the HEA. In round table discussions we talked about how perhaps NTFs could buddy up if they were considering going for a fellowship? Our table thought that the distinction between “senior” and “principal” was quite reflective of the individual’s present role and not necessarily their level of expertise and ability.

Dr Shirley Evans Jisc TechDis
Accessibility and Technological Support for Higher Education

TechDis is a small unit that works along side Jisc and the HEA to explore and provide advice to ensure inclusivity and accessibility within Higher Education, primarily through the use of technology, but also by recommending common sense and good practice. Shirley explained that around 200,000 students in higher education have a declared disability. From a show of hands in the audience it was surprising how many people had not heard of TechDis at all.

Their website (http://www.jisctechdis.ac.uk) is packed full of advice, not just about the technology but good academic practices to ensure we provide an inclusive approach to our work. The team regularly speak at conferences and run webinars, so there are plenty of opportunities to catch up with them.

Shirley talked about the “Voices” project that led to the development of student-friendly voices – Jack and Jess – for use with text to speech software. She talked about “Tbx Toolbox” – a series of resources on academic and personal skills useful for both staff and students. She also talked about Jisc and the “Discover Jisc” pages on the website to help locate the many tools, publications, toolkits and advice that Jisc have produced over the years.
A starting point for anyone wishing to review their understanding of accessibility would be “OASES“, an evaluation tool again located on the TechDis website.

For a quick tour of the TechDis and Jisc websites –> https://www.youtube.com/watch?v=SCXMA9XpGtk

Well there ended an inspirational and exhausting two days. What an amazing and inspiring group of people. I am very much looking forward to next year’s symposium. In light of feedback we are trying to find a more suitable date for the event and if you are an NTF please fill in our survey below (you can complete the survey on this page by scrolling down it or by visiting the SurveyMonkey site).

 

This blog post is by Viv Rolfe and entirely represents my own views and ways of expressing myself.