Higher education: what the research can and cannot settle

What the research actually asks

Higher-education research is three literatures that share a name. One is about systems: funding, participation rates, governance, the effects of policy instruments. One is about learning and teaching inside institutions: course design, assessment, what students actually do with their time. One is about the academic profession: careers, doctoral training, the conditions under which research is produced.

They rarely cite each other, and a claim that travels between them usually loses a qualifier on the way. A finding about a teaching method in one discipline at one institution becomes, three citations later, a statement about how students learn. Knowing which of the three a source belongs to is the first move in reading it.

Access, completion and the two different questions

Widening participation is measured by who enrols and judged by who graduates, and those are different questions with different answers. An institution can improve access figures and worsen outcomes for the students it admitted, if admission was not accompanied by anything else. The reverse also happens: high completion can indicate strong support or a narrow intake, and the number alone cannot distinguish them.

What follows is a habit rather than a finding. Any claim about access should be read with its denominator visible: of whom, out of how many, compared with which group, and measured at what point. Most disagreements in this area dissolve once both parties state their denominator, and the ones that survive are worth having.

The same discipline applies to internationalisation, which this domain's catalogue covered extensively. Recruitment figures measure income; they do not measure whether the education travelled.

Teaching in a system built for research

University teaching is done largely by people hired, promoted and evaluated for research, which explains more about teaching quality than any pedagogical variable. Constructive alignment, the most widely adopted framework in this area, is a response to exactly that condition: it gives a busy academic a checkable property (outcomes, activities and assessment sharing the same verbs) rather than a philosophy of teaching.

Two structural facts are worth carrying into any reform proposal. Student evaluations of teaching measure satisfaction and are affected by things unrelated to learning, so they cannot carry the weight promotion committees put on them. And the assessment format, not the syllabus, is what students respond to, which means a curriculum reform that leaves assessment untouched has changed the documents and not the course. Both points are argued in full in the curriculum design brief.

Leadership, policy and reform cycles

Higher-education systems reform on a cycle shorter than the time it takes to evaluate a reform. That single mismatch produces most of what practitioners experience as churn: an instrument is introduced, its early indicators are read as results, the next instrument arrives before the cohort it would have affected has graduated.

The research literature is consequently better at describing mechanisms than at settling whether a given reform worked. It can tell you what a funding formula rewards, how institutions respond to a ranking, and why a quality-assurance regime produces documentation rather than change. Those are useful, transferable findings. What it cannot usually give is a clean before-and-after, because nothing in a university system holds still.

Doctoral education as a system

Doctoral training is where the three literatures meet, and it is the part of higher education whose purpose has changed fastest without its structures changing much. A model built to reproduce academics is now the main route into research roles outside universities as well, while the apprenticeship at its centre — one candidate, one or two supervisors, several years — is largely as it was.

Three consequences show up repeatedly in the research. Supervision quality varies more than any other input, and it varies unaccountably, because it happens in private between two people with unequal power; this is why structured supervision agreements have spread even in systems that dislike bureaucracy. Completion times respond to structure rather than to exhortation, with cohort models, staged progression reviews and explicit milestones associated with shorter and less variable completions. And the labour-market destination shapes what the training should include, which is an argument for teaching the transferable craft (writing, project management, presenting to non-specialists) as content rather than as an optional extra.

The internationalisation of doctoral study, which this domain's catalogue covered directly, adds one more variable that is easy to miss: a candidate working in a second language and an unfamiliar academic culture is carrying a load that has nothing to do with their research ability, and the supervision arrangements that work for a domestic candidate often do not name it. The practical craft of getting the thesis written is treated in full in the dissertation writing brief.

Reading a claim about higher education

• Which literature is this? Systems, teaching, or profession. Claims do not transfer between them without loss.
• What is the denominator? Enrolment, completion, or a survey of the people who answered.
• At which level was it measured? Institution-level averages hide programme-level variation large enough to reverse the conclusion.
• How long did the study wait? A reform evaluated inside two years has been evaluated on its implementation, not its effect.

Those four questions are unglamorous and they retire a surprising share of confident claims, including several that circulate as settled.

Works referred to

1. Higher Education in Turmoil: The Changing World of Internationalisation — Sense Publishers, 2008.
2. Globalization and Its Impacts on the Quality of PhD Education — Sense Publishers, 2014.
3. Higher Education Management and Operational Research — Sense Publishers, 2013.
4. Greening the Academy: Ecopedagogy through the Liberal Arts — Sense Publishers, 2014.
5. Enhancing Teaching through Constructive Alignment — John Biggs, Higher Education, 1996.

Questions readers send

Do university rankings measure teaching quality?

Not in any direct way. Ranking composites are dominated by research output, reputation surveys and resource indicators, and the teaching components are usually proxies such as staff-to-student ratio. A ranking can tell you which institutions are research-intensive; it cannot tell you where a given subject is taught well.

Are student evaluations of teaching useless then?

They measure something real (student satisfaction with the experience) and that is worth knowing. What they do not measure reliably is how much was learned, and they are affected by factors unrelated to either. The error is not collecting them; it is treating them as an outcome measure in promotion decisions.

Does widening access lower standards?

The evidence does not support that framing. What it supports is narrower: admitting students with weaker preparation without changing anything else produces worse completion, and admitting them with adapted teaching and support does not. The variable is what the institution does afterwards, not the admission.

Why do higher-education reforms so rarely show clear results?

Because the evaluation window is shorter than the effect. A cohort takes three or four years to graduate, systems reform on a shorter cycle than that, and several initiatives usually run at once, so attribution is genuinely hard rather than merely unmeasured.

Is a doctorate still mainly training for an academic career?

It is designed as though it were and is used mostly for something else. In most systems a minority of graduates end up in permanent academic posts, which is an argument for teaching the transferable craft explicitly rather than treating non-academic destinations as a fallback.

The briefs that cover this in depth

1. Curriculum design Backward design, spiral sequencing and constructive alignment explained as a coverage-versus-depth trade, with the cost of each choice stated in teaching hours.
2. Dissertation writing A structural account of how to write a dissertation: the four failure modes that cause most late-stage trouble, and how each one is visible early.
3. Teacher development Which forms of teacher professional development change classroom practice, by how much, and how long the effect lasts before it decays.