# data[].attributes.attachment_link.readable_text
'Exams level the playing field'

Good assessment aims to provide a balanced and fair evaluation of every student. In my eyes, there is no fairer
way to test the knowledge and ability of students and to reward their efforts over time than through exams.

Exams level the playing field for everybody and give students the platform to show what they can and cannot do. It's the same questions - on the same day, under the same conditions - for everybody.

So, I never imagined that I would agree to there being no exams in an academic year, but it would be unrealistic and unfair to force exams to go ahead in 2021.

There are no easy decisions here. We completely understand why the government has taken the path it has to ensure children who have missed so much face-to-face education get a fairer deal. We're not in a normal year, so we welcome the news that students will be awarded grades based on the judgement of their teachers.

But we must see a return to exams next year if external factors allow.

Exams are the most equitable way to assess a student's academic performance. They are marked externally and put everybody into the same basket. No other methodology is as fair.

Our students have told us that they prefer exams, as they want to show what they are capable of through their own hard work. Children like to have something to work towards and they want to be in control of their own destinies. They also feel that if their grades are based on somebody's opinion, the process could be too subjective.

"What if my teacher has been away for a long time because of Covid?" one student asked me recently. "How will the teacher make a decision on my grades if they don't know me properly?"

Fortunately, as a large trust, we can do lots of moderation and use a consistent approach to awarding grades across all
our academies.

This is an understandable workaround for the current year. But it can't be the answer long term. Exams enable us to accurately test a student's breadth of understanding
of different topics.

There is also evidence that both studying for and sitting exams deepens learning and wider cognitive skills, as the process of retrieving information strengthens memory.

So, we know that learning is particularly strong when students are tested. Rather than our students passively reading and remembering, we want them to question and be questioned. Strengths and weaknesses can also be assessed through exams, and teachers can better understand where more attention in class may be needed.

So again, we strongly believe that students should sit them next year if conditions allow. Until then, we plan to do everything we can to give our students the best possible opportunities to progress on to the next stage of their careers.
Jane Millward is chief executive of E-ACT
'We need fundamental reform of the system'

As CEOs of a multi-academy trust, we believe the pandemic has made it glaringly obvious that exams are not the fairest way to assess the performance of all students. They inhibit social mobility and they are no longer an appropriate means of assessment in the 21st century.

Exams work well for those who are good at revising and remembering, but this is not a skill set shared by all students. Rote learning is not even particularly prized in our modern workplace or society.

What's more, revising and remembering can be enhanced by private tutoring, and so a focus on these skills further disadvantages young people without access to private tuition.

The Covid-19 pandemic has shown that our education system is not prepared for assessment without exams. But that is not an argument to maintain the current system. Instead, we need fundamental reform of the system, including high-stakes standardised tests, so that young people
are better equipped to thrive in the world beyond education.

The big issue facing schools, now students are back and things are fully operational, is not how to "catch up" students but that education as we remember it will not be fit for purpose for our young people. Welcoming them back to the classrooms of old simply will not do.

Restricting their capacity to explore ideas and applying their new-found creativity to the confines of the traditional classroom seems anathema. Restricting their learning to acquiring the knowledge set out in a national curriculum designed in the 1980s seems, frankly, absurd. The pandemic has provided a once-in-a-generation opportunity to reset the education system.

What has become evident to parents over the course of the pandemic is the value of school. By this we mean the value of the principle of school, not the content of school. Mass home education has somewhat dampened the attitude that "the education system was good enough for me, so it is good enough for my children".

Let's start a conversation about how to equip "generation creative" with the infrastructures and learning opportunities that they actually need to capitalise on their unique experiences of education to date.

Let's infuse and embed the curriculum with opportunities to work alongside real employers who are desperate to have meaningful engagement with young people.

The ideal replacement for exams already exists in academia. Anyone studying for a PhD has to present evidence of their work and then experience a detailed viva or interview.

Students at the end of Year 13 could present a portfolio of work, either across a range of subjects or related to a particular area such as medicine, showing their research, their understanding and their application of knowledge. Such an interview, presented to a mixed group of educationalists and business/industry leaders, would test the validity
of a student's work and also allow them to demonstrate many of the "future skills" required in today's workplaces.

The world today needs active problem solvers. Schools need to model this by rethinking and redesigning education.
Steve and Paula Kenning are co-founders and trustees of Aspirations Academies Trust This article originally appeared in the 19 March 2021 issue under the headline "Is this the chance for us to get rid of exams for good?"

# data[].attributes.attachment_link.title
Covid: Should we get rid of exams for good?

# data[].attributes.attachment_link.description
The cancellation of GCSE and A-level exams during the coronavirus pandemic was controversial, but it has also led many educators to question whether such exams are an outdated form of assessment. Two academy trust leaders go head to head on whether we should ditch end-of-year exams once and for all

# data[].attributes.attachment_link.title
ECC Self Assessment for English and Maths (EPYP) 1819

# data[].attributes.body
Download to see in full...

# data[].attributes.attachment_link.title
Copy of QIP English and Maths Department 2020-21 24 02 21 update

# data[].attributes.attachment_link.readable_text
Planned changes to the common inspection framework (CIF), and the way Ofsted inspects education and skills providers from September 2019, announced by the chief inspector, will shift the regulator's focus towards learning and training providers offering a broad, rich and deep curriculum, with less emphasis on headline outcomes data. Such an emphasis will have unintended consequences, most notably an overemphasis on narrowing teaching and assessment practice to focus on the end test.

While Ofsted's sentiments that it should "be a force for improvement" imply that it isn't perhaps achieving this, it's clear that there are issues within the current approach that the proposed changes will attempt to resolve.

So, what does the new inspection framework mean for training and learning providers? If Ofsted can deliver a new context that reflects its ambition, then there will be benefits to be had. In particular, as the focus on the quality of teaching pans out, and learner progress and how good a learning experience is being provided to learners (how "good learning experience" is defined is an interesting debate in its own right). What we are seeing is an inspectorate that is keen to listen and keen to undertake its remit based on evidence.

The proposed changes to the CIF have been widely shared. A single "quality of education" judgement will replace "outcomes for students" and "quality of teaching, learning and assessment". Also, while the curriculum will be core, Ofsted will, according to specialist adviser for apprenticeships Chris Jones, continue to remain focused on the wider development of FE learners, particularly their attitudes and behaviours in the classroom and the workplace, as well as the role and input of sector leadership and management.

Driven by schools

The inspection of safeguarding, which will sit under leadership and management, and hold the same weight across all remits, will be built around three core areas: are leaders and others identifying the right children and vulnerable adults, and how is this being done? What timely action do staff within the FE provider take, and how well do they work with other agencies? And third, how do responsible bodies and staff manage their statutory responsibilities, and how do they respond to allegations about staff and other adults?

As is always the case with changes to Ofsted inspection, the direction of travel appears largely driven by the inspection of schools. This is not a complaint, simply an observation. I do however wonder about the emphasis on behaviour with, for example, an apprenticeship provider working exclusively with adults on management and leadership programmes.

The timing for the proposed changes is good, with the number of apprentices undertaking end-point assessment (EPA) increasing month on month. While the introduction of EPA is a broadly positive step, I have often repeated my concerns about the curriculum being narrowed too early in an apprenticeship programme, as both the provider and employer's understanding of the tests increases. We must do our best to safeguard against this with Ofsted changes clearly designed to support such a position.

Quality of education

The debates will continue to rage about how judgements can be made about the quality of education, without inadvertently advocating a particular approach. What is positive is the debates are happening and the sector can do much to drive evidence-based practice from within, both in their own organisation and through support from the likes of the Education and Training Foundation's outstanding teaching, learning and assessment programme.

The new inspection framework is only at the proposal stage now, and will undoubtedly see changes and additions. Reflection, development and piloting for the new framework will now move ahead, with developments fed back by Ofsted to sector partners and consulting occurring over spring term 2019, before publication of the final framework in summer 2019.

But regardless of what happens, change is coming, and people need to plan for a different inspection landscape in future. We are particularly keen to see the reduction in duplication in the common inspection framework between the questions underpinning the judgements.

This affects providers when they undertake self-assessment and, understandably, align their approach to that of the CIF content. Indeed, we had as recently as August made the decision to create a framework for our clients in our self-assessment software module, which mapped to the CIF and reduced the duplication.

Quality assurance

Our position on the CIF remains steadfast - it is an incredibly useful tool to support FE and skills providers to develop sound internal quality-assurance practices. Our advice is therefore simple: focus time and effort on getting this right, and far less on unnecessarily worrying about the inspection process itself.

If there is one disappointment around the proposed changes, it's that there will be no consultation on removing the "outstanding" grade. The response from Ofsted is to say parents like it. This doesn't strike me as reason enough to keep it and I'm sure there's more to Ofsted's thinking.

With my governor's hat on, I recall a local school-improvement partner suggesting to me that it was a poison chalice. I've since come to understand what he means by that; the pressure to retain it and the perceived fall from grace if a provider doesn't is a problem. I draw parallels with the French chef who recently asked to be removed from the Michelin directory, citing the pressure it placed on himself and his staff.
Louise Doyle is a director at quality assurance and improvement specialist, Mesma

# data[].attributes.attachment_link.title
'Ofsted's new framework: what it will mean for FE'

# data[].attributes.attachment_link.description
Planned changes to the common inspection framework (CIF), and the way Ofsted inspects education and skills providers from September 2019, announced by the chief inspector, will shift the regulator's focus towards learning and training providers offering a broad, rich and deep curriculum, with less emphasis on headline outcomes data.

# data[].attributes.headline
Raising the level with threshold concepts

# data[].attributes.subject
Raising the level with threshold concepts

# data[].attributes.attachment_link.readable_text
Cross the threshold and seize a new and improved curriculum.
Many teachers will already be thinking about the significant curriculum changes just around the corner. I admit that I have met some previous curriculum changes with a response something like world-weariness - worsening with each successive hasty change and token relabelling. I am, however, intent on meeting this new national curriculum with more vigour and to seize the opportunity it presents, rather than simply complain and deride its origins. You realise after a few years teaching that national curricula don't fundamentally change or dictate teaching and learning, teachers change teaching and learning. We do not need to passively receive this latest curriculum iteration - we can actively shape it.

In our school, and in our English department, we have talked about raising the level of ' challenge' for a long time. When our department was faced with the latest curriculum changes we recognised it was an opportunity to make our curriculum more challenging, whilst ensuring every student could access the learning with skilled differentiation. One crucial mode of differentiation - to make the curriculum make greater sense for our students - is to create, and actively shape, a curriculum that is defined by ' big ideas' in something like a coherent whole.

When you being to explore the ideas of the ' big ideas' relevant to each subject area, as a group of teachers, you begin to consider the touchstones that define the important knowledge and skills required of our students. When considering the same for our English department I became interested in the idea of " threshold concepts". In many ways, ' threshold concepts ' are the defining 'big ideas' for teaching and learning in English...and in every other subject discipline.

So what are they all about?

"A threshold concept can be considered as akin to a portal, opening up a new and previously inaccessible way of thinking about something. It represents a transformed way of understanding, or interpreting, or viewing something without which the learner cannot progress."
Jan Meyer & Ray Land, ' Threshold Concepts and Troublesome Knowledge: Linkages to Ways of Thinking and Practising within the Disciplines '

Meyer and Land went about defining the characteristics of a ' threshold concept '. They summarise the following:

They are transformative. That is to say, once student grasps the ' threshold concept ' it changes the way a student thinks about a subject.

It is also highly likely to be troublesome for the student. It may seem counter-intuitive, or at best, really difficult to grasp. Yet, once understood, it can make subsequent learning feel more intuitive or ' easy '.

It is irreversible. Once grasped, the student would find it difficult to unlearn.

It is integrative. That is to say that once learned, the concept helps unify aspects of the subject that may not have appeared related to the student ( this is most important for me). It may completely shift the view that the student has towards the subject.

This new knowledge is enhanced by an extended and improved use of language. Grasping a ' threshold concept' will not happen in any twenty minute lesson snapshot. There is no simple shift from easy to hard overnight. It requires revisiting and reinforcement, but the rewards can be profound for our students.

In different subjects there will be crucial ' threshold concepts' that, once understood, can give students the essential foundation for learning. They begin to ' get ' Physics or Spanish or History - with many subsequent aspects of the subject becoming much more comprehensible for that crucial gateway knowledge. In Physics it may prove to be energy and heat transfer, in Economics it may prove to be the 'opportunity/cost' concept. Of course, I have little expertise in these areas - teachers of each curriculum subject should review their own subject for these central big ideas that matter.
What are the 'threshold concepts' in your subject discipline?
Considering these ' threshold concepts' in our English department led us to a ' less is more' approach - learning deeper to ensure real understanding, rather than being dictated by attempting to cover everything. We dropped units and will move to four deeper, knowledge filled units per year - see our draft model here. We are looking to help students connect their study of literature with a chronological approach to study. We are aiming to establish the underlying patterns (cognitive scientists would call these schemas) of language and the common plots, characters and myths that are the touchstones of our story culture. Our curriculum is focused on being integrative.

I see the ' threshold concepts' for English Literature as being the following: the role of myth and the story-teller; narrative structure; patterns of language, imagery and plot; comedy, tragedy and genre; the concept of power (character relationships, gender, class etc.). For English Language I view the following as the essential ' threshold concepts ': phonic decoding and vocabulary decoding (largely focused in Primary study, but not exclusively); recognition of sentence structures and a conscious manipulation of such structures for effect; generic writing structures; a recognition and application of complex patterns of language, imagery and plot.

We don't believe our current assessment model, outcomes or national curriculum levels help us with ensuring our students grasp the 'threshold concepts' in English so we are changing them. We are designing our own assessment and standards to best ensure our students can know the 'threshold concepts'. Vague numerical levels just don't do the job. There is an opportunity for every subject to do the same. We have nothing to lose but our chains!

What we must do as teachers in every subject discipline is start with these big questions for our curriculum redesign:
What are our 'threshold concepts'? What outcomes will best ensure the focused and repeated study of these 'threshold concepts'? What assessment criteria will best evidence student understanding? How can our curriculum design make understanding these concepts easier?
They may prove difficult - most things of value prove difficult to master. They may prove elusive for students, but we need to foreground them - revisit them with artful repetition in the structure of our curriculum. We should avoid looking for coverage of content at the expense of deeply knowing the core knowledge our students must grasp for long term understanding. We can help students understand these concepts by repeating their specialist language, without dumbing down our approach. We can reduce the difficulty (cognitive scientists would call this ' cognitive load ') by creating memorable patterns for which they can root their knowledge.

Debating about ' threshold concepts ' can certainly liven up a subject meeting. The debate is a healthy starting point for seizing our own curriculum as changes appear in our sights. We need only cross the threshold and make great changes!

If you are interested in finding out more, then here is a comprehensive bibliography for 'Threshold Concepts': http://www.ee.ucl.ac.uk/~mflanaga/thresholds.html#gen

# data[].attributes.attachment_link.title
Designing a New Curriculum - What are your 'Big Ideas'?

# data[].attributes.attachment_link.description
Cross the threshold and seize a new and improved curriculum. Many teachers will already be thinking about the significant curriculum changes just around the corner. I admit that I have met some previous curriculum changes with a response something like world-weariness - worsening with each successive hasty change and token relabelling.

# data[].attributes.body
Alex Quigley

# data[].attributes.attachment_link.readable_text
People have gone relatively quiet about the 'c' word... curriculum.

Whilst schools grapple with the challenges of having all pupils safely attend school, then everything has to be considered anew. The careful sequencing of curriculum still matters a great deal, but as schools are recommended to consider lopping great bits off, then new plans and considerations about curriculum emerge.

What are the keystone concepts of our curriculum? How is knowledge best organised in the curriculum model?

There is something about Stenhouse

Although the new challenges that attend curriculum seem utterly of this moment, useful insights can be found in the writing of educational thinkers from a lifetime ago.

Lawrence Stenhouse was writing about curriculum decades before I was born. In his seminal book, ' An Introduction to Curriculum Research and Development ' he squarely locates the key support factor for curriculum success:

"...curriculum development must rest on teacher development and that it should promote it and hence the professionalism of the teacher. Curriculum development translates ideas into classroom practicalities and thereby helps the teacher to strengthen her practice by systematically and thoughtfully testing ideas." (p24-25)

Stenhouse squarely rejects the notion of teachers being "docile agents". There will be no off-the-shelf 'recovery curriculum' that betters the agency of teachers as they make intelligent adaptations to their existing curriculum that is rooted in their understanding of history, science, literature, and their pupils.

Indeed, as questions remain in secondary schools about lopping off subjects and doubling up on English and maths, Stenhouse offers an important challenge:

"[Citing Peters (1966)] In history, science, or literature... there is an immense amount to know, and if it is properly assimilated, it constantly throws light on, widens, and deepens one's own view of countless other things."

Instead of secondary schools doing more English, perhaps we can instead double-down on high quality reading and writing instruction in history and science. It is not easy to weave together meaningful cross-curricular links, but nor is it easy to make more and more gains from more and more English.

Stenhouse, in the '70s presents a compelling image of an "extended professional". Indeed, it could describe many of the teachers who have been determined to develop their professional knowledge during the pandemic lockdown:

"In short, the outstanding characteristics of the extended professional is a capacity for for autonomous professional self-development through systematic self-study, through the study of the work of other teachers and through the testing of ideas by classroom research procedures." (p144)

He goes on to present a compelling notion of the teacher as researcher:

"A research tradition which is accessible to teachers and which feeds teaching must be created if education is to be significantly improved." (p165)

In the spirit of teacher as researcher, grappling with the complexities of new classes, some missed learning, new routines, and more, it leaves teachers and middle leaders with some compelling questions to steer their decisions around curriculum.

Here are some questions that may be useful for teachers and middle leaders considering intelligent adaptations to the curriculum and 'classroom practicalities':

Questions to help address curriculum challenges:

What 'classroom practicalities' need to be addressed that provide the platform to teach the curriculum successfully? [e.g. support for vulnerable pupils; behavioural routines; rooming & resourcing; seating arrangements; and countless new routines attending class feedback, marking etc.]

What topic/content sequencing will provide an autumn term 'smooth landing'? [e.g. is there a curriculum topic/s that teachers - and pupils - will be confident undertaking in Sept/Oct, as they establish an array of new routines and consolidate skills and knowledge that may have slipped due to natural learning loss]

What diagnostic assessments will best reveal what has been forgotten, misunderstood, and swiftly recalled? [e.g. discussion; concept mapping; quizzing]
Read Rob Coe's cracking EEF blogs on assessment are very helpful on this question - see HERE.

How will 1-2-1, or small group, targeted interventions (e.g. tutoring) be best synchronised with classroom teaching and curriculum planning? [E.g. sensitive diagnostic assessment; effective TA liaison; teacher training on intervention content]

What blended curriculum model would be most responsive to unplanned school closures? Is there an existing platform for the school curriculum, or can resources like Oak Academy be integrated into your curriculum planning? [Considerations of teacher workload, tech access etc. will continue to be important to mediate and mitigate]

How can routines and practices developed during partial school closures help enhance homework provision in the year ahead? [e.g. video and online curriculum content developed for closures could be reused, or recast, to provide an improved homework offer]

How do we best reconnect pupils with the 'big ideas' and basic building blocks of the curriculum? [E.g. we may need to retell the narrative that connects those big ideas; grapple once more with common misconceptions that have crept back in the thinking of our pupils; or re-establish and reteach the key vocabulary of the subject domain]

What rich opportunities will we provide that break beyond the parameters of the prescribed curriculum? What voices and perspectives are missing from our curriculum? [e.g. ensuring additional reading can enrich our pupils' understanding; many English teachers will rightly be considering their literary canon and how they can include more diverse voices]

How will we evaluate and monitor our curriculum adaptations? [e.g. peer curriculum planning, paired with reciprocal observations and curriculum plan adaptations; termly departmental feedback systems; moderation of pupils' work]

There are, of course, many questions beyond this short list, but that will be the work of teachers and middle leaders rigorously testing ideas, refining plans and developing the curriculum. It is the job of school leaders to create the culture and clear the way for this work to happen and to support teachers with great professional development.

# data[].attributes.attachment_link.description
People have gone relatively quiet about the 'c' word... curriculum. Whilst schools grapple with the challenges of having all pupils safely attend school, then everything has to be considered anew. The careful sequencing of curriculum still matters a great deal, but as schools are recommended to consider lopping great bits off, then new plans and considerations about curriculum emerge.

# data[].attributes.headline
Active Ingredients for Curriculum Development

# data[].attributes.subject
Active Ingredients for Curriculum Development

# data[].attributes.body
Education Endowment Programme

# data[].attributes.body
Curriculum Development

# data[].attributes.attachment_link.readable_text
"Curriculum development must rest on teacher development"
Lawrence Stenhouse, 'An Introduction to Curriculum Research and Development'
Paying attention to the careful, skilled development of your curriculum is essential business for every school. Though it has always been in view for teachers and school leaders, there is no doubt the focus has become more acute for schools since OFSTED's new inspection framework. For many school leaders and teachers, it is a welcome, intellectually nourishing focus for school improvement. That said, it is really, really hard to do well!

Now, I don't want to appear like some curriculum development Cassandra, but I think it is helpful to consider how and why curriculum development may run aground before we sink all of our efforts into it. Teacher workload, along with our pupils' outcomes, may even depend upon it.

My scepticism is rooted in decades of curriculum development defeats (see my TES article). Not only has curriculum development proven a near-insurmountable challenge in England, Scotland and the US, at pretty much every school phase, I have also had my own taste of bitter curriculum development failures. I have written about what was a beautifully 'knowledge-rich' KS3 English curriculum in 2013. I soon found out what Lawrence Stenhouse wrote about back in the 70s: there is no "teacher-proof curricula". A curriculum plan (our 'intent' in more recent parlance`) will only go as far as the professional development in place to support its implementation.

I learned from my particular 'knowledge-rich KS3' curriculum struggle that teacher knowledge, teacher habits and teacher beliefs, all need tending to (a lot) for any curriculum development to succeed. Crucially though, due to really variable ITT, followed by even more disparate and variable in-school training for developing teachers, that self-same teacher knowledge and practice will likely prove highly variable and hard to plan for and lead (ironically, there is no curricula for teacher learning).

Given this challenge then, as much as we should ask questions about the substantive knowledge, sequence and schema of is taught, we should ask at the same time, how are teachers aided with substantial professional development, coaching and follow on support to assimilate and enact these tricky habit changes?
The complex challenge of teacher development
As schools grapple with curriculum sequencing, whilst ensuring teachers have the requisite subject knowledge to plan and bring new curriculum innovations to life in the classroom, we should also get to grips with what type of teacher development may be necessary.

Offering teachers time for ongoing curriculum development is an obvious starting point. It is a necessary prerequisite, but it is insufficient in and of itself. A chastening truth is that even high quality subject-specific CPD may not be enough. In the US, maths teachers were given ample time and coaching on maths focused materials. And yet, despite between 68 and 110 hours of professional development being given over to teachers, the impact was negligible.

So, what are the 'active ingredients' of teacher professional development necessary to support curriculum development that may succeed? Time is a given. A subject specific emphasis would appear to be valuable (just how much, how specific, and how it relates to changing teacher habits, is still up for grabs). Teacher-facing tools will also matter.

Another curriculum seer, William Popham, stated back in 1969: "The education reformer who eloquently urges classroom teachers to change their practices may receive the accolades of the educational community, but the educational reformer who provides a set of useable curriculum materials for the teacher is more likely to modify what goes on in the classroom."

I would guess that unless teachers are supported with a batch of useable new curriculum resources, little of our curriculum changes will be enacted or really sustained. This includes assessments that are neatly aligned with whole-school data demands. Not only that, many of these resources will need to be developed (and adapted) together with teachers to ensure a greater understanding. Coaching, inquiry and peer collaborative planning will likely be necessary if we are to nurture a sustained habit change.

A handy short-cut is to consider all of our curriculum changes through the lens of a harried full-time teacher who is snatching at hard-earned windows of time in the corners of their full timetable. It rightly grounds our ambitions in the messiness of the real classrooms of tired, hustling teachers.

If we don't face head-on the significant challenge of teacher development, then our exciting curriculum debates and plans will come to nought.

Slowly, quietly, the majority of teachers in schools will close their classroom door and slip back into doing it as they always have done. Our best intent will simply wilt in the white-hot crucible of the classroom.

# data[].attributes.attachment_link.title
Curriculum Development and Teacher Development

# data[].attributes.attachment_link.description
"Curriculum development must rest on teacher development" Lawrence Stenhouse, 'An Introduction to Curriculum Research and Development' Paying attention to the careful, skilled development of your curriculum is essential business for every school. Though it has always been in view for teachers and school leaders, there is no doubt the focus has become more acute for schools since OFSTED's new inspection framework.

# data[].attributes.attachment_link.readable_text
If You Would Just Let Me Finish - Nancy Kline, The Observer (The Promise, Chptr 1 excerpt) An Interview with Nancy Kline - C Mumby © This article was published in Therapy Today, the journal of the British Association for Counselling and Psychotherapy Beyond-2o-Seconds: The Independent Mind of the Board- Mitzi Wyman How Can Leaders Ignite The Best Thinking In Others?- Soshan Soobramoney Thinking Caps Off - Nancy Kline, Catalytic Converter - Nancy Kline, Coaching At Work Positive Thinking For The Voluntary Sector - Nancy Kline Revealing Practices - Christina Breene, AMED Journal Working With Words - Helena Dolney, The Thinking Board - Fiona Dawe, Creating Time To Think - Neslyn Watson Druee, Public Service Review Time For A Rethink - Linda Aspey, Developing Leaders Breaking Glass - Women & Men: Changing Leadership - Nancy Kline, BBC Publications Games People Play At Work - Nancy Kline, Cosmopolitan The Thinking Environment Organisation - Nancy Kline Attention - The Stunning Catalyst - Nancy Kline Interruption As Assault - Nancy Kline Interest, Not Patience - Nancy Kline Importance Of Each Component - Nancy Kline Time To Think - An Imperative Of Behaviour, Not Time - Nancy Kline Experiencing The Unknowable - Nancy Kline Do What Works - Nancy Kline Why I Am Hopeful - Nancy Kline Equality - Nancy Kline Not Set In Stone - Nancy Kline Never Heard Music - Nancy Kline You Matter: Do Apple Products Create A Thinking Environment? - Nancy Kline The Thinking Environment: A Force For Positive Change - Dr Alex Goudie Sales In A Thinking Environment - Shirley Wardell Narcissistic Personality Disorder: Me, Myself And I - Christopher Spence The Growing Ideas Process: Innovation In A Thinking Environment - Dee Ramsayer Public Speaking In A Thinking Environment - Michael Charlesworth The Thinking Environment And Merger Integration - Alan H A Frost Incisive Questions, Remove Limiting Assumptions and Set The Mind Free - Lucy Faulkner, Bluekite Coaching Blog Mail & Guardian Online , Volunteering England Online Personnel Management

# data[].attributes.attachment_link.description
If You Would Just Let Me Finish - Nancy Kline, The Observer (The Promise, Chptr 1 excerpt) An Interview with Nancy Kline - C Mumby © This article was published in Therapy Today, the journal of the British Association for Counselling and Psychotherapy Beyond-2o-Seconds: The Independent Mind of the Board-

# data[].attributes.body
Don't interrupt me!

# data[].attributes.attachment_link.title
Let me finish: how to stop interrupting… and change the world

# data[].attributes.attachment_link.description
We are all guilty of interrupting – but if we just stopped and listened, argues Nancy Kline, we could radically change the way we live…

# data[].attributes.attachment_link.readable_text
Research into Education, and Development

To watch this inspiring short film of RED in practice, click the image or the link below:
Lesson observation reinvented - The RED System - East Coast College
East Coast College has been grade 3 or 4 at every inspection for the last 20 years, whether inspected by TSC, ALI or Ofsted. In February 2020, however, following a year in which its culture change through the introduction of The RED System, it has achieved its first grade 2. To read the new inspection report and to see what Ofsted thinks of RED, click here: 11-02-2020 East Coast College Ofsted report.

Typical inspection issues

Managers do not take sufficient action to ensure good-quality teaching, learning & assessment.

Teaching, learning and assessment are not of a consistently high standard.

Overall premise

"Outstanding learning is a symptom of outstanding planning and its realisation. For this, your need a collaborative, research-focused culture."
Tony Davis, Director CCQI
Why design an observation system that doesn't work?

Every system has an impact.

Do you know the impact your system is having?

Is it the one you want?

These fundamental questions can have a profound effect on education and training providers. Precisely what is the impact you want from your observation system, and is it delivering?

For many providers, the rhetoric is that they want their observation system to demonstrably improve learning outcomes. Providers like to feel that this is driven by the moral imperative to do the very best they can for their learners, but the reality is often quite different. More often than not, the real motivator behind the design of a lesson observation system has its roots firmly in Ofsted's Common Inspection Framework and it's requirement that providers should have 'robust performance management' (Ofsted, 2015).

In most instances, this leads providers to design an accountability system, but to call it (and even believe it to be) a learning system. The only test of whether a system is about either accountability or learning is to thoroughly understand the views of those being observed. If they believe it to be an accountability system, then it really doesn't matter what the architects think. And it is impossible for a system to truly be both.

The rot sets in with Ofsted's inappropriate and ironic use of the word 'robust'. It's ironic because Ofsted will never give a provider a strength for any of its quality systems, only the positive impact they have on learning. So on the ground, inspectors are not testing out whether or not a provider's system is robust, only that it is effective. In other words, you can have the most 'robust performance system' imaginable, but if it doesn't lead to sustained and sustainable improvements in the teaching and learning, then it will quite rightly be judged a weakness. The evaluative statement Ofsted really means to say, then, is that:

leaders and managers have an effective system for improving teaching and learning, that enables learners to achieve aspirational outcomes.

This is what we look for when we inspect.

A second motivator for designing an accountability system is the burden of proof - to Ofsted and to governors - that improvements are being made. The misconception here is that a lesson observation grade is a piece of reliable and useful data with which to evaluate the effectiveness of the system. It isn't. Here's one of many simple tests:

if a profile of lesson observation grades goes down, does this prove that the impact of teaching on learning has gone down?

Here's another:

does an observation grade, often based on no more than 30 minutes of a lesson, once a year, give a clear indication of the impact a teaching is having on learning?

Or:

are observers grading pedagogy or difference made to learning? If the former, then every grade is meaningless.

But real as this argument is, it effectively deflects us from the main issue with accountability-based observation systems. Take care here, as if you agree with the following premise, you may have to redesign your whole approach to lesson observation.

If an observation system can allow a teacher to go into denial about the observer's findings, it will not work.

For instance, if a teacher is given a grade lower than s/he expected (whether that be grade 4, 3 or 2 in the old Ofsted grading system), there is a very strong chance they will go into denial. Typical responses from a teacher in denial are:

"What does he know about teaching my subject?"

"She only stayed for the first half an hour, so she missed all of the impact on learning."

"He said I didn't cover equality and diversity. We did that just before he came in."

"The buses were all running late, so my lesson didn't start on time or as I'd planned."

And many, many more. The key issue with this, of course, is that these teachers are potentially the very ones the system aims and needs to improve; the very ones who, because of the system, won't.

So why design an observation system that doesn't work, and what should a lesson observation system look like if it is to achieve what leaders, governors and Ofsted all want? The short answer is: the opposite of the Ofsted model. The longer answer is: The RED System.

What follows is a statement of what The RED System is designed to achieve. If you agree, then you may wish to read on.

The intended impact of The RED System

To inspire teachers to invest in the development of their pedagogy, so that all learners achieve aspirational outcomes.

As a result of the RED system, teachers will be able to:

set out the outstanding level of impact they want their process to have on learners

ensure all learners exceed expectations by differentiating the support needed to enable all learners to achieve high grades rather than differentiating, and so lowering, expectations

judge the success of their experiments by evaluating the actual learning impact against the intended.

As a result of these impacts on teachers, every learner should:

find all lessons unmissable

progress from their individual starting points (A) to appropriately individual and challenging end points (B) in every lesson

become expert/independent learners

be inspired to study and explore their subject between lessons

be curious and engaged throughout every lesson

be part of a sustainable, high-performing social community within each class

treat 'mistakes' as a valuable learning experience

create strong visual, audio and physical anchors to sustain their learning

become effective reflective practitioners

look forward to their on-going challenges.

[For a full list of impacts on learning, see The Quality Standard for Teaching, Learning and Assessment.]
What the delegates say

For an overview of feedback on this session, please click here.

These comments can be invaluable when generating curiosity about forthcoming training.

# data[].attributes.attachment_link.title
The RED System: Transforming teaching observation

# data[].attributes.attachment_link.description
Research into Education, and Development To watch this inspiring short film of RED in practice, click the image or the link below: Lesson observation reinvented - The RED System - East Coast College East Coast College has been grade 3 or 4 at every inspection for the last 20 years, whether inspected by TSC, ALI or Ofsted.

# data[].attributes.attachment_link.readable_text
Cookies on GOV.UK

We use some essential cookies to make this website work.

We'd like to set additional cookies to understand how you use GOV.UK, remember your settings and improve government services.

We also use cookies set by other sites to help us deliver content from their services.

You have accepted additional cookies. You can change your cookie settings at any time.

You have rejected additional cookies. You can change your cookie settings at any time.

Accept additional cookies Reject additional cookies View cookies

Skip to main content

Ofsted is the Office for Standards in Education, Children's Services and Skills. We inspect services providing education and skills for learners of all ages. We also inspect and regulate services that care for children and young people.

Ofsted aims to improve lives by raising standards in education and children's social care. We inspect and regulate thousands of organisations and individuals providing education, training and care - from childminders to training providers, schools to local authorities - and we share what we find.

We report directly to Parliament, parents, carers and commissioners.

Who we are

We have around 1,800 employees across our 8 regions:

East Midlands

East of England

North East, Yorkshire and Humber

North West

South East

South West

West Midlands

London

We also directly contract with more than 2,000 Ofsted Inspectors to carry out inspections of schools and further education and skills provision. Our pen portraits contain biographical information about current His Majesty's Inspectors (HMIs).

Our responsibilities

We're responsible for:

Inspecting

maintained schools and academies, some independent schools, colleges, apprenticeship providers, prison education and many other educational institutions and programmes outside of higher education

childcare, local authorities, adoption and fostering agencies, initial teacher training and teacher development

Regulating

a range of early years and children's social care services, making sure they're suitable for children and potentially vulnerable young people

Reporting

publishing reports of our findings so they can be used to improve the overall quality of education and training

informing policymakers about the effectiveness of these services

Ofsted is a member of the National Preventive Mechanism which monitors and reports on places of detention.

Our values

We put children and learners first, and we are independent, evidence-led, accountable and transparent.

Our priorities

Our 2022-27 strategy sets out our strategic priorities:

Inspections that raise standards: our inspections help education and social care recover and improve.

Right-touch regulation: our regulation advances high-quality care, education and safeguarding for children.

Making the most of our insights: we share insights through our research and analysis. Our insights inform practitioners, policymakers and decision-makers and lead to improvements across the system.

The best start in life: we will develop the evidence base about early years education, including curriculum and pedagogy, and act on it.

Keeping children safe: we promote children's safety and welfare in everything we do.

Keeping pace with sector changes: we keep pace as the education and social care sectors evolve. We will continually review our approach and advocate for additional powers where required.

Accessible and engaged: we are open and accessible to our different audiences, understanding their needs.

A skilled workforce: we make sure we have the tools, knowledge and expertise we need to continue to be a force for improvement.

You can find out more about our priorities in the Ofsted 2022 to 2027 strategy.

Corporate information

Access our information

Jobs and contracts

Read about the types of information we routinely publish in our Publication scheme. Our Personal information charter explains how we treat your personal information. Read our policy on Social media use.

# data[].attributes.attachment_link.description
Ofsted is the Office for Standards in Education, Children's Services and Skills. We inspect services providing education and skills for learners of all ages. We also inspect and regulate services that care for children and young people.

# data[].attributes.attachment_link.title
Evaluation in Education: Meaning, Principles and Functions

# data[].attributes.attachment_link.description
ADVERTISEMENTS: After reading this article you will learn about:- 1. Meaning of Evaluation 2. Principles of Evaluation 3. Functions. Meaning of Evaluation: Evaluation is a broader term than the Measurement. It is more comprehensive than mere in­clusive than the term Measurement. It goes ahead of mea…

# data[].attributes.attachment_link.readable_text
I have a certain sympathy with the concept of accountability: we all want to know if our local school is any good and that our taxpayer contributions are spent effectively. But the way this straightforward desire has manifested itself in Ofsted - and the way some managers in schools have chosen (and it is a choice) to implement the inspectorate's criteria - has turned the entire process into a pointless, stressful, tick-box exercise.

I've been teaching in secondary schools for 16 years and have just been through my fifth Ofsted inspection. I never used to think much about inspections, but now they're seen as a life-altering, career-defining Armageddon. It's hard to identify a tipping point that led us to the current state of affairs, where colleagues try to redefine teaching and work idiotic hours to invent lessons that achieve the impossible. I saw one teacher sob uncontrollably in the staffroom because he'd been up until 4am preparing a lesson which wasn't inspected. A colleague and I tried to console him, but finding words of support did not come easily. I found myself angry and frustrated that educated adults and experienced professionals were being reduced to tears.

Driving home that day, I was determined this wouldn't be me. I've no problem with being held accountable for my students' achievements, but if I wanted to keep doing the job I love, I needed to find a new way of dealing with it.

So now I treat Ofsted inspections as a purposeless farce and never ask for feedback on my lessons. I care about my students' outcomes great deal, but making judgments about a lesson based on a spurious grid of phrases that defy consistent interpretation has become so lamentably futile there is nothing left to do other than laugh.

At my last school, I had a lesson inspection conducted by two assistant headteachers during a mocksted. The lesson was graded as "good" so I asked them what I could do to make it "outstanding". They looked blank and eventually suggested I should have spent a bit longer during a discussion section of the lesson. I pointed out that this would have reduced the time for plenary reflection - the latest targeted initiative - and they agreed. I never did get a clear answer on whether it was even possible to make the lesson "outstanding".

Fast forward a week and the headteacher dropped in on me unannounced. By pure chance I was teaching the same lesson (with some minor tweaks) to a parallel year group of almost identical ability range. The head deemed my lesson "inadequate". I pointed out that her two assistant heads, one of whom is in charge of teaching and learning, thought the same lesson was "good" - her reponse was to dispute whether the lesson was the same.

I could feel a sense of overwhelming frustration building up. I pointed out at some length that my GCSE and A-level results had been above the school average and that student uptake and retention had grown since my appointment, and then asked to be observed again to prove I know what I'm doing. She never came.

My current school was inspected by Ofsted late last year. As the meeting was called to tell us of the impending visit, I immediately reflected on how my previous experiences could help me. I decided that while I can't choose when or if I will be inspected - nor what the outcome will be - I can choose how I respond to it.

As expected, senior management went into overdrive with last-minute initiatives and tick-box exercises. But I had a fairly normal couple of days: I planned my lessons normally and went home at a normal time because I was meeting my mate Rob for a run. I told my department what I was doing and that we'd all be best prepared for the next few days with a decent night's sleep. I said that under no circumstances should they change their evening plans: I trusted their judgement about how best to plan their lessons, and that we would deal with the outcomes - good or bad - afterwards. I've no idea what senior management thought, but I assured my department we weren't going to be sacked for leaving before 9pm. Sure enough, the next day an inspector wandered in to see my year 11s. The lesson passed without hitch, and he invited me to see him at the end of the day for feedback.

I didn't go. What's the point? He wasn't a specialist in my subject and he was only going to tell me his interpretation of a grid of lesson descriptors that has changed virtually every year for the last decade. As far as I know, no-one has been fired or had their pay reduced directly as a result of one Ofsted lesson inspection alone. Some colleagues thought I was mad or disrespectful, but if the inspector had a problem with my teaching and results, he could have found me and said what was wrong and why. Any good teacher knows that students' progress is neither linear nor predictable, or consistent across subjects and time. Any good teacher also knows that building skills of resilience, humility, determination, awareness, ambition and curiosity cannot be measured by a grid. I'm not a maverick. Maybe other teachers are worried about the consequences of taking a different approach because some school managers continually "motivate" staff by waving a big Ofsted stick.

But it's my choice. I'm going to care less about Ofsted and put my energy into my students. I love my job and I don't want to waste energy resenting certain aspects of what it has become. I know at times this will be easier said than done, but to continue doing what I do best, I need to make sure I keep what is lacking in the current climate - perspective.

# data[].attributes.attachment_link.title
Secret Teacher: I see Ofsted for what it is - a purposeless farce

# data[].attributes.attachment_link.description
I have a certain sympathy with the concept of accountability: we all want to know if our local school is any good and that our taxpayer contributions are spent effectively.

# data[].attributes.headline
Discussion of the effectiveness of Ofsted

# data[].attributes.subject
Discussion of the effectiveness of Ofsted

# data[].attributes.attachment_link.readable_text
Dr Melanie Ehren, reader in educational accountability and improvement, UCL Institute of Education

"Scrapping Ofsted would leave us with a fragmented system"
Dr Melanie Ehren, reader in educational accountability and improvement, UCL Institute of Education
Many teachers and headteachers feel that Ofsted does more harm than good: "overhauling Ofsted" and "abolishing the watchdog" have become popular catch phrases. A quick glance at some headlines over the years explains some of the concerns: inaccuracy and inconsistency of Ofsted gradings; and inspectors who are poorly qualified and disrespectful in their communication with schools, or have a formulaic view of teaching and leadership. Ofsted is seen as one of the (or perhaps the major) causes of stagnating improvement as the fear of "being Ofsted-ed" has produced a risk-averse school system. It is no wonder that many would argue to abolish Ofsted altogether: it has clearly become a poisoned brand.

Why then should we not scrap Ofsted? Research across Europe shows the many positive effects of school inspections, in this country and others. School inspections can unlock valuable knowledge about the performance of individual schools, pointing towards great practice in high quality teaching and learning.

Alternative systems, such as self-evaluation or peer review, would not provide a national and comparative overview

They can also highlight areas that need to be improved, both in individual schools and on a system-level. Recent Ofsted reports on geographical inequalities within the country, or failing academy chains have done just that: they have highlighted where we need to do better and how our government needs to be smarter.

Scrapping Ofsted would leave us with a system that is fragmented in the information available about school quality, placing an even larger emphasis on external examinations and performance league tables. Other countries that have such test-based accountability, such as the US, have seen an over-reliance on data and tests, extensive teaching to test, narrow curricula focused on tested subjects and root-based learning.

School inspection, on the other hand, informs us about the quality of schools on a range of indicators so we can compare, track and contrast their performance over time. Alternative systems of quality assurance, such as school self-evaluation or peer review, would not provide that national and comparative overview.

It is therefore time to value the expertise and good of our Ofsted inspectors and make sure they work towards the benefit of our schools. Having said that, we also need to acknowledge that there is a problem; we need to get our inspection service right. A big step forward would be a more positive tone of voice from our watchdog when talking about the performance of our teachers, headteachers and school governors. The appointment of a new chief inspector in December will hopefully provide the momentum to change the narrative and ensure a more constructive inspection force.

"Ofsted does not help schools improve: a self-evaluation system would be more effective"

John MacBeath, professor emeritus, faculty of education, the University of Cambridge
The litmus test: does Ofsted empower teachers to become better professionals or does it disempower? It's been widely reported that Ofsted is a major cause of stress among teachers. There's a telling quote from Michael Wilshaw: "If anyone says to you that 'staff morale is at an all-time low' you will know you are doing something right."

The reward for getting the top grade in an Ofsted inspection is that you won't be inspected again for another few years. That says it all - it is clearly not an experience that schools look forward to, nor want again.

In a number of countries in which we've conducted research, such as New Zealand and Hong Kong, there's a form of external review or quality assurance which works from a different premise. Rather than "we're coming to inspect you and here are our criteria which you have to meet", the review team will ask staff and pupils what they consider a good school to be, how they evaluate their own school, the criteria they use and their sources of evidence. If a school is considered to be struggling, it will be given extra support to improve, not labeled as "requires improvement".

In the early 1980s in Scotland I was commissioned to help design an approach to self-evaluation. We met with teachers, parents, advisers and children aged five-18, and asked: what would make a good school for you? How would we know it was a good school? The consistency of answers from all groups provided a robust set of criteria.

As professionals, teachers have a pretty good idea of what makes a good classroom and what makes good learning. Unfortunately with Ofsted, teachers are too often disenfranchised because the things that are important to them aren't included in the inspection criteria. Ofsted focuses primarily on maths and English, and what is most easily measurable. They marginalise other creative aspects of school life, such as art, drama, sport or a range of student-led initiatives.

While there are many outstanding Ofsted inspectors, they are hamstrung in terms of what and how they have to inspect

To Ofsted's credit, it has - over the decades - changed, and changed again. Often these changes are ones many of us have long advocated, such as differential reviews in which some schools are given greater support than others.

Although I'm critical of Ofsted, the nub of the problem lies with the nature of what Ofsted has to inspect. Michael Gove's curriculum reform, for example, ignored the advice from his own experts - he knew from his own schooldays what a curriculum should look like and how it should be tested.

While there are many outstanding Ofsted inspectors, they are hamstrung in terms of what and how they have to inspect, living with the legacy of a process and protocols which are widely seen by teachers as neither fair nor effective.

Follow us on Twitter via @GuardianTeach. Join the Guardian Teacher Network for lesson resources, comment and job opportunities, direct to your inbox.
Topics

# data[].attributes.attachment_link.title
Should we scrap Ofsted? The pros, cons and alternatives

# data[].attributes.attachment_link.description
Dr Melanie Ehren, reader in educational accountability and improvement, UCL Institute of Education Dr Melanie Ehren, reader in educational accountability and improvement, UCL Institute of Education Many teachers and headteachers feel that Ofsted does more harm than good: "overhauling Ofsted" and "abolishing the watchdog" have become popular catch phrases.

# data[].attributes.headline
Evaluating alignment between curriculum, assessment and instruction

# data[].attributes.subject
Evaluating alignment between curriculum, assessment and instruction

# data[].attributes.headline
Evaluation for education, learning and change

# data[].attributes.subject
Evaluation for education, learning and change

# data[].attributes.attachment_link.readable_text
Evaluation for education, learning and change - theory and practice. Evaluation is part and parcel of educating - yet it can be experienced as a burden and an unnecessary intrusion. We explore the theory and practice of evaluation and some of the key issues for informal and community educators, social pedagogues youth workers and others. In particular, we examine educators as connoisseurs and critics, and the way in which they can deepen their theory base and become researchers in practice.

Contents: introduction · on evaluation · three key dimensions · thinking about indicators · on being connoisseurs and critics · educators as action researchers · some issues when evaluating informal education · conclusion · further reading and references · acknowledgements · how to cite this article

A lot is written about evaluation in education - a great deal of which is misleading and confused. Many informal educators such as youth workers and social pedagogues are suspicious of evaluation because they see it as something that is imposed from outside. It is a thing that we are asked to do; or that people impose on us. As Gitlin and Smyth (1989) comment, from its Latin origin meaning 'to strengthen' or to empower, the term evaluation has taken a numerical turn - it is now largely about the measurement of things - and in the process can easily slip into becoming an end rather than a means. In this discussion of evaluation we will be focusing on how we can bring questions of value (rather than numerical worth) back into the centre of the process. Evaluation is part and parcel of educating. To be informal educators we are constantly called upon to make judgements, to make theory, and to discern whether what is happening is for the good. We have, in Elliot W. Eisner's words, to be connoisseurs and critics. In this piece we explore some important dimensions of this process; the theories involved; the significance of viewing ourselves as action researchers; and some issues and possibilities around evaluation in informal and community education, youth work and social pedagogy. However, first we need to spend a little bit of time on the notion of evaluation itself.

On evaluation

Much of the current interest in evaluation theory and practice can be directly linked to the expansion of government programmes (often described as the 'New Deal') during the 1930s in the United States and the implementation of various initiatives during the 1960s (such as Kennedy's 'War on Poverty') (see Shadish, Cork and Leviton 1991). From the 1960s-on 'evaluation' grew as an activity, a specialist field of employment with its own professional bodies, and as a body of theory. With large sums of state money flowing into new agencies (with projects and programmes often controlled or influenced by people previously excluded from such political power) officials and politicians looked to increased monitoring and review both to curb what they saw as 'abuses', and to increase the effectiveness and efficiency of their programmes. A less charitable reading would be that they were both increasingly concerned with micro-managing initiatives and in controlling the activities of new agencies and groups. Their efforts were aided in this by developments in social scientific research. Of special note here are the activities of Kurt Lewin and the interest in action research after the Second World War.

As a starter I want to offer an orienting definition:

Evaluation is the systematic exploration and judgement of working processes, experiences and outcomes. It pays special attention to aims, values, perceptions, needs and resources.

There are several things that need to be said about this.

First, evaluation entails gathering, ordering and making judgments about information in a methodical way. It is a research process.

Second, evaluation is something more than monitoring. Monitoring is largely about 'watching' or keeping track and may well involve things like performance indicators. Evaluation involves making careful judgements about the worth, significance and meaning of phenomenon.

Third, evaluation is very sophisticated. There is no simple way of making good judgements. It involves, for example, developing criteria or standards that are both meaningful and honour the work and those involved.

Fourth, evaluation operates at a number of levels. It is used to explore and judge practice and programmes and projects (see below).

Last, evaluation if it is to have any meaning must look at the people involved, the processes and any outcomes we can identify. Appreciating and getting of flavour of these involves dialogue. This makes the focus enquiry rather than measurement - although some measurement might be involved (Rowlands 1991). The result has to be an emphasis upon negotiation and consensus concerning the process of evaluation, and the conclusions reached.

Three key dimensions

Basically, evaluation is either about proving something is working or needed, or improving practice or a project (Rogers and Smith 2006). The first often arises out of our accountability to funders, managers and, crucially, the people are working with. The second is born of a wish to do what we do better. We look to evaluation as an aid to strengthen our practice, organization and programmes (Chelimsky 1997: 97-188).

To help make sense of the development of evaluation I want to explore three key dimensions or distinctions and some of the theory associated.

Programme or practice evaluation? First, it is helpful to make a distinction between programme and project evaluation, and practice evaluation. Much of the growth in evaluation has been driven by the former.

Programme and project evaluation. This form of evaluation is typically concerned with making judgements about the effectiveness, efficiency and sustainability of pieces of work. Here evaluation is essentially a management tool. Judgements are made in order to reward the agency or the workers, and/or to provide feedback so that future work can be improved or altered. The former may well be related to some form of payment by results such as the giving of bonuses for 'successful' activities, the invoking of penalty clauses for those deemed not to have met the objectives set for it and to decisions about giving further funding. The latter is important and necessary for the development of work.

Practice evaluation. This form of evaluation is directed at the enhancement of work undertaken with particular individuals and groups, and to the development of participants (including the informal educator). It tends to be an integral part of the working process. In order to respond to a situation workers have to make sense of what is going on, and how they can best intervene (or not intervene). Similarly, other participants may also be encouraged or take it upon themselves to make judgements about the situation. In other words, they evaluate the situation and their part in it. Such evaluation is sometimes described as educative or pedagogical as it seeks to foster learning. But this is only part of the process. The learning involved is oriented to future or further action. It is also informed by certain values and commitments (informal educators need to have an appreciation of what might make for human flourishing and what is 'good'). For this reason we can say the approach is concerned with praxis - action that is informed and committed

These two forms of evaluation will tend to pull in different directions. Both are necessary - but just how they are experienced will depend on the next two dimensions.

Summative or formative evaluation? Evaluations can be summative or formative. Evaluation can be primarily directed at one of two ends:

To enable people and agencies make judgements about the work undertaken; to identify their knowledge, attitudes and skills, and to understand the changes that have occurred in these; and to increase their ability to assess their learning and performance (formative evaluation).

To enable people and agencies to demonstrate that they have fulfilled the objectives of the programme or project, or to demonstrate they have achieved the standard required (summative evaluation).

Either can be applied to a programme or to the work of an individual. Our experience of evaluation is likely to be different according to the underlying purpose. If it is to provide feedback so that programmes or practice can be developed we are less likely, for example, to be defensive about our activities. Such evaluation isn't necessarily a comfortable exercise, and we may well experience it as punishing - especially if it is imposed on us (see below). Often a lot more is riding on a summative evaluation. It can mean the difference between having work and being unemployed!

Banking or dialogical evaluation? Last, it is necessary to explore the extent to which evaluation is dialogical. As we have already seen much evaluation is imposed or required by people external to the situation. The nature of the relationship between those requiring evaluation and those being evaluated is, thus of fundamental importance. Here we might useful employ two contrasting models. We can usefully contrast the dominant or traditional model that tend to see the people involved in a project as objects, with an alternative, dialogical approach that views all those involved as subjects. This division has many affinities to Freire's (1972) split between banking and dialogical models of education.

Exhibit 1: Rowlands on traditional (banking) and alternative (dialogical) evaluation

Joanna Rowlands has provided us with a useful summary of these approaches to evaluation. She was particularly concerned with the evaluation of social development projects.

The characteristics of the traditional (banking) approach to evaluation:

1. A search for objectivity and a 'scientific approach', through standardized procedures. The values used in this approach... often reflect the priorities of the evaluator.

2. An over-reliance on quantitative measures. Qualitative aspects..., being difficult to measure, tend to be ignored.

3. A high degree of managerial control, whereby managers can influence the questions being asked Other people, who may be affected by the findings of an evaluation, may have little input, either in shaping the questions to be asked or reflecting on the findings.

4. Outsiders are usually contracted to be evaluator in the belief that his will increase objectivity, and there may be a negative perception of them by those being evaluated'.

The characteristics of the alternative (dialogical) approach to evaluation

1. Evaluation is viewed as an integral part of the development or change process and involves 'reflection-action'. Subjectivity is recognized and appreciated.

2. There is a focus on dialogue, enquiry rather than measurement, and a tendency to use less formal methods like unstructured interviews and participant observation.

3. It is approached as an 'empowering process' rather than control by an external body. There is a recognition that different individuals and groups will have different perceptions. Negotiation and consensus is valued concerning the process of evaluation, and the conclusions reached, and recommendations made

4. The evaluator takes on the role of facilitator, rather than being an objective and neutral outsider. Such evaluation may well be undertaken by 'insiders' - people directly involved in the project or programme.

Adapted from Joanna Rowlands (1991) How do we know it is working? The evaluation of social development projects, and discussed in Rubin (1995: 17-23)

_________

We can see in these contrasting models important questions about power and control, the way in which those directly involved in programmes and projects are viewed. Dialogical evaluation places the responsibility for evaluation squarely on the educators and the other participants in the setting (Jeffs and Smith 2005: 85-92).

Thinking about indicators

The key part of evaluation, some may argue, is framing the questions we want to ask, and the information we want to collect such that the answers provide us with the indicators of change. Unfortunately, as we have seen, much of the talk and practice around indicators in evaluation has been linked to rather crude measures of performance and the need to justify funding (Rogers and Smith 2006). We want to explore the sort of indicators that might be more fitting to the work we do.

In common usage an indicator points to something, it is a sign or symptom. The difficulty facing us is working out just what we are seeing might be a sign of. In informal education - and any authentic education - the results of our labours may only become apparent some time later in the way that people live their lives. In addition, any changes in behaviour we see may be specific to the situation or relationship (see below). Further, it is often difficult to identify who or what was significant in bringing about change. Last, when we look at, or evaluate, the work, as E Lesley Sewell (1966) put it, we tend to see what we are looking for. For these reasons a lot of the outcomes that are claimed in evaluations and reports about work with particular groups or individuals have to be taken with a large pinch of salt.

Luckily, in trying to make sense of our work and the sorts of indicators that might be useful in evaluation, we can draw upon wisdom about practice, broader research findings, and our values.

Exhibit 2: Evaluation - what might we need indicators for?

We want to suggest four possible areas that we might want indicators for:

The number of people we are in contact with and working with. In general, as informal educators we should expect to make and maintain a lot of contacts. This is so people know about us, and the opportunities and support we can offer. We can also expect to involve smaller numbers of participants in groups and projects, and an even smaller number as 'clients' in intensive work. The numbers we might expect - and the balance between them - will differ from project to project (Jeffs and Smith 2005: 116-121). However, through dialogue it does seem possible to come some agreement about these - and in the process we gain a useful tool for evaluation.

The nature of the opportunities we offer. We should expect to be asked questions about the nature and range of opportunities we offer. For example, do young people have a chance to talk freely and have fun; expand and enlarge their experience, and learn? As informal educators we should also expect to work with people to build varied programmes and groups and activities with different foci.

The quality of relationships available. Many of us talk about our work in terms of 'building relationships'. By this we often mean that we work both through relationship, and for relationship (see Smith and Smith forthcoming). This has come under attack from those advocating targeted and more outcome-oriented work. However, the little sustained research that has been done confirms that it is the relationships that informal educators and social pedagogues form with people, and encourage them to develop with others, that really matters (see Hirsch 2005). Unfortunately identifying sensible indicators of progress is not easy - and the job of evaluation becomes difficult as a result.

How well people work together and for others. Within many of the arenas where informal education flourishes there is a valuing of working so that people may organize things for themselves, and be of service to others. The respect in which this held is also backed up by research. We know, for example, that people involved in running groups generally grow in self-confidence and develop a range of skills (Elsdon 1995). We also know that those communities where a significant number of people are involved in organizing groups and activities are healthier, have more positive experiences of education, are more active economically, and have less crime (Putnam 1999). (Taken from Rogers and Smith 2006)

__________

For some of these areas it is fairly easy to work out indicators. However, when it comes to things like relationships, as Lesley Sewell noted many years ago, 'Much of it is intangible and can be felt in atmosphere and spirit. Appraisal of this inevitably depends to some extent on the beholders themselves' (1966: 6). There are some outward signs - like the way people talk to each other. In the end though, informal education is fundamentally an act of faith. However, our faith can be sustained and strengthened by reflection and exploration.

On being connoisseurs and critics

Informal education involves more than gaining and exercising technical knowledge and skills. It depends on us also cultivating a kind of artistry. In this sense, educators are not engineers applying their skills to carry out a plan or drawing, they are artists who are able to improvise and devise new ways of looking at things. We have to work within a personal but shared idea of the 'good' - an appreciation of what might make for human flourishing and well-being (see Jeffs and Smith 1990). What is more, there is little that is routine or predictable in our work. As a result, central to what we do as educators is the ability to 'think on our feet'. Informal education is driven by conversation and by certain values and commitments (Jeffs and Smith 2005).

Describing informal education as an art does sound a bit pretentious. It may also appear twee. But there is a serious point here. When we listen to other educators, for example in team meetings, or have the chance to observe them in action, we inevitably form judgments about their ability. At one level, for example, we might be impressed by someone's knowledge of the income support system or of the effects of different drugs. However, such knowledge is useless if it cannot be used in the best way. We may be informed and be able to draw on a range of techniques, yet the thing that makes us special is the way in which we are able to combine these and improvise regarding the particular situation. It is this quality that we are describing as artistry.

For Donald Schön (1987: 13) artistry is an exercise of intelligence, a kind of knowing. Through engaging with our experiences we are able to develop maxims about, for example, group work or working with an individual. In other words, we learn to appreciate - to be aware and to understand - what we have experienced. We become what Eisner (1985; 1998) describes as ' connoisseurs '. This involves very different qualities to those required by dominant models of evaluation.

Connoisseurship is the art of appreciation. It can be displayed in any realm in which the character, import, or value of objects, situations, and performances is distributed and variable, including educational practice. (Eisner 1998: 63)

The word connoisseurship comes from the Latin cognoscere, to know (Eisner 1998: 6). It involves the ability to see, not merely to look. To do this we have to develop the ability to name and appreciate the different dimensions of situations and experiences, and the way they relate one to another. We have to be able to draw upon, and make use of, a wide array of information. We also have to be able to place our experiences and understandings in a wider context, and connect them with our values and commitments. Connoisseurship is something that needs to be worked at - but it is not a technical exercise. The bringing together of the different elements into a whole involves artistry.

However, educators need to become something more than connoisseurs. We need to become critics.

If connoisseurship is the art of appreciation, criticism is the art of disclosure. Criticism, as Dewey pointed out in Art as Experience, has at is end the re-education of perception... The task of the critic is to help us to see.

Thus... connoisseurship provides criticism with its subject matter. Connoisseurship is private, but criticism is public. Connoisseurs simply need to appreciate what they encounter. Critics, however, must render these qualities vivid by the artful use of critical disclosure. (Eisner 1985: 92-93)

Criticism can be approached as the process of enabling others to see the qualities of something. As Eisner (1998: 6) puts it, 'effective criticism functions as the midwife to perception. It helps it come into being, then later refines it and helps it to become more acute'. The significance of this for those who want to be educators is, thus, clear. Educators also need to develop the ability to work with others so that they may discover the truth in situations, experiences and phenomenon.

Educators as action researchers

Schön (1987) talks about professionals being 'researchers in the practice context'. As Bogdan and Biklen (1992: 223) put it, 'research is a frame of mind - a perspective people take towards objects and activities'. For them, and for us here, it is something that we can all undertake. It isn't confined to people with long and specialist training. It involves (Stringer 1999: 5):

* A problem to be investigated.

* A process of enquiry

* Explanations that enable people to understand the nature of the problem

Within the action research tradition there have been two basic orientations. The British tradition - especially that linked to education - tends to view action research as research oriented toward the enhancement of direct practice. For example, Carr and Kemmis provide a classic definition:

Action research is simply a form of self-reflective enquiry undertaken by participants in social situations in order to improve the rationality and justice of their own practices, their understanding of these practices, and the situations in which the practices are carried out (Carr and Kemmis 1986: 162).

The second tradition, perhaps more widely approached within the social welfare field - and most certainly the broader understanding in the USA - is of action research as 'the systematic collection of information that is designed to bring about social change' (Bogdan and Biklen 1992: 223). Bogdan and Biklen continue by saying that its practitioners marshal evidence or data to expose unjust practices or environmental dangers and recommend actions for change. It has been linked into traditions of citizen's action and community organizing, but in more recent years has been adopted by workers in very different fields.

In many respects, this distinction mirrors one we have already been using - between programme evaluation and practice evaluation. In the latter, we may well set out to explore a particular piece of work. We may think of it as a case study - a detailed examination of one setting, or a single subject, a single depository of documents, or one particular event (Merriam 1988). We can explore what we did as educators: what were our aims and concerns; how did we act; what were we thinking and feeling and so on? We can look at what may have been going on for other participants; the conversations and interactions that took place; and what people may have learnt and how this may have affected their behaviour. Through doing this we can develop our abilities as connoisseurs and critics. We can enhance what we are able to take into future encounters.

When evaluating a programme or project we may ask other participants to join with us to explore and judge the processes they have been involved in (especially if we are concerned with a more dialogical approach to evaluation). Our concern is to collect information, to reflect upon it, and to make some judgements as to the worth of the project or programme, and how it may be improved. This takes us into the realm of what a number of writers have called community-based action research. We have set out one example of this below.

Exhibit 3: Stringer on community-based action research

A fundamental premise of community-based action research is that it commences with an interest in the problems of a group, a community, or an organization. Its purpose is to assist people in extending their understanding of their situation and thus resolving problems that confront them....

Community-based action research is always enacted through an explicit set of social values. In modern, democratic social contexts, it is seen as a process of inquiry that has the following characteristics:

It is democratic, enabling the participation of all people.

It is equitable, acknowledging people's equality of worth.

It is liberating, providing freedom from oppressive, debilitating conditions.

It is life enhancing, enabling the expression of people's full human potential. (Stringer 1999: 9-10)

The action research process

Action research works through three basic phases:

Look -building a picture and gathering information. When evaluating we define and describe the problem to be investigated and the context in which it is set. We also describe what all the participants (educators, group members, managers etc.) have been doing.

Think - interpreting and explaining. When evaluating we analyse and interpret the situation. We reflect on what participants have been doing. We look at areas of success and any deficiencies, issues or problems.

Act - resolving issues and problems. In evaluation we judge the worth, effectiveness, appropriateness, and outcomes of those activities. We act to formulate solutions to any problems.

(Stringer 1999: 18; 43-44;160)

We could contrast with a more traditional, banking, style of research in which an outsider (or just the educators working on their own) collect information, organize it, and come to some conclusions as to the success or otherwise of the work.

Some issues when evaluating informal education

In recent years informal educators have been put under great pressure to provide 'output indicators', 'qualitative criteria', 'objective success measures' and 'adequate assessment criteria'. Those working with young people have been encouraged to show how young people have developed 'personally and socially through participation'. We face a number of problems when asked to approach our work in such ways. As we have already seen, our way of working as informal educators places us within a more dialogical framework. Evaluating our work in a more bureaucratic and less inclusive fashion may well compromise or cut across our work.

There are also some basic practical problems. Here we explore four particular issues identified by Jeffs and Smith (2005) with respect to programme or project evaluations.

The problem of multiple influences. The different things that influence the way people behave can't be easily broken down. For example, an informal educator working with a project to reduce teen crime on two estates might notice that the one with a youth club open every weekday evening has less crime than the estate without such provision. But what will this variation, if it even exists, prove? It could be explained, as research has shown, by differences in the ethos of local schools, policing practices, housing, unemployment rates, and the willingness of people to report offences.

The problem of indirect impact. Those who may have been affected by the work of informal educators are often not easily identified. It may be possible to list those who have been worked with directly over a period of time. However, much contact is sporadic and may even take the form of a single encounter. The indirect impact is just about impossible to quantify. Our efforts may result in significant changes in the lives of people we do not work with. This can happen as those we work with directly develop. Consider, for example, how we reflect on conversations that others recount to us, or ideas that we acquire second- or third-hand. Good informal education aims to achieve a ripple effect. We hope to encourage learning through conversation and example and can only have a limited idea of what the true impact might be.

The problem of evidence. Change can rarely be monitored even on an individual basis. For example, informal educators who focus on alcohol abuse within a particular group can face an insurmountable problem if challenged to provide evidence of success. They will not be able to measure use levels prior to intervention, during contact or subsequent to the completion of their work. In the end all the educator will be able to offer, at best, is vague evidence relating to contact or anecdotal material.

The problem of timescale. Change of the sort with which informal educators are concerned does not happen overnight. Changes in values, and the ways that people come to appreciate themselves and others, are notoriously hard to identify - especially as they are happening. What may seem ordinary at the time can, with hindsight, be recognized as special.

Workarounds

There are two classic routes around such practical problems. We can use both as informal educators.

The first is to undertake the sort of participatory action research we have been discussing here. When setting up and running programmes and projects we can build in participatory research and evaluation from the start. We make it part of our way of working. Participants are routinely invited and involved in evaluation. We encourage them to think about the processes they have been participating in, the way in which they have changed and so on. This can be done in ways that fit in with the general run of things that we do as informal educators.

The second route is to make linkages between our own activities as informal educators and the general research literature. An example here is group or club membership. We may find it very hard to identify the concrete benefits for individuals from being member of a particular group such as a football team or social club. What we can do, however, is to look to the general research on such matters. We know, for example, that involvement in such groups builds social capital. We have evidence that:

In those countries where the state invested most in cultural and sporting facilities young people responded by investing more of their own time in such activities (Gauthier and Furstenberg 2001).

The more involved people are in structured leisure activities, good social contacts with friends, and participation in the arts, cultural activities and sport, the more likely they are to do well educationally, and the less likely they are to be involved even in low-level delinquency (Larson and Verma 1999).

There appears to be a strong relationship between the possession of social capital and better health. 'As a rough rule of thumb, if you belong to no groups but decide to join one, you cut your risk of dying over the next year in half. If you smoke and belong to no groups, it's a toss-up statistically whether you should stop smoking or start joining' (ibid.: 331). Regular club attendance, volunteering, entertaining, or church attendance is the happiness equivalent of getting a college degree or more than doubling your income. Civic connections rival marriage and affluence as predictors of life happiness (Putnam 2000: 333).

This approach can work where there is some freedom in the way that you can respond to funders and others with regard to evaluation. Where you are forced to fill in forms that require the answers to certain set questions we can still use the evaluations that we have undertaken in a participatory manner - and there may even be room to bring in some references to the broader literature. The key here is to remember that we are educators - and that we have a responsibility foster learning, not only among those we work with in a project or programme, but also among funders, managers and policymakers. We need to view their requests for information as opportunities to work at deepening their appreciation and understanding of informal education and the issues and questions with which we work.

Conclusion

The purpose of evaluation, as Everitt et al (1992: 129) is to reflect critically on the effectiveness of personal and professional practice. It is to contribute to the development of 'good' rather than 'correct' practice.

Missing from the instrumental and technicist ways of evaluating teaching are the kinds of educative relationships that permit the asking of moral, ethical and political questions about the 'rightness' of actions. When based upon educative (as distinct from managerial) relations, evaluative practices become concerned with breaking down structured silences and narrow prejudices. (Gitlin and Smyth 1989: 161)

Evaluation is not primarily about the counting and measuring of things. It entails valuing - and to do this we have to develop as connoisseurs and critics. We have also to ensure that this process of 'looking, thinking and acting' is participative.

Further reading and references

For the moment I have listed some guides to evaluation. At a later date I will be adding in some more contextual material concerning evaluation in informal education.

Berk, R. A. and Rossi, P. H. (1990) Thinking About Program Evaluation, Newbury Park: Sage. 128 pages. Clear introduction with chapters on key concepts in evaluation research; designing programmes; examining programmes (using a chronological perspective). Useful US annotated bibliography.

Eisner, E. W. (1985) The Art of Educational Evaluation. A personal view, Barcombe: Falmer. 272 + viii pages. Wonderful collection of material around scientific curriculum making and its alternatives. Good chapters on Eisner's championship of educational connoisseurship and criticism. Not a cookbook, rather a way of orienting oneself.

Eisner, E. W. (1998) The Enlightened Eye. Qualitative inquiry and the enhancement of educational practice, Upper Saddle River, NJ: Prentice Hall. 264 + viii pages. Re-issue of a 1990 classic in which Eisner plays with the ideas of educational connoisseurship and educational criticism. Chapters explore these ideas, questions of validity, method and evaluation. An introductory chapter explores qualitative thought and human understanding and final chapters turn to ethical tensions, controversies and dilemmas; and the preparation of qualitative researchers.

Everitt, A. and Hardiker, P. (1996) Evaluating for Good Practice, London: Macmillan. 223 + x pages. Excellent introduction that takes care to avoid technicist solutions and approaches. Chapters examine purposes; facts, truth and values; measuring performance; a critical approach to evaluation; designing critical evaluation; generating evidence; and making judgements and effecting change.

Hirsch, B. J. (2005) A Place to Call Home. After-school programs for urban youth, New York: Teachers College Press. A rigorous and insightful evaluation of the work of six inner city boys and girls clubs that concludes that the most important thing they can and do offer is relationships (both with peers and with the workers) and a 'second home'.

Patton, M. Q. (1997) Utilization-Focused Evaluation. The new century text 3e, Thousand Oaks, Ca.: Sage. 452 pages. Claimed to be the most comprehensive review and integration of the literature on evaluation. Sections focus on evaluation use; focusing evaluations; appropriate methods; and the realities and practicalities of utilization-focused evaluation.

Rossi, P. H., Freeman, H. and Lipsey, M. W. (2004) Evaluation. A systematic approach 7e, Newbury Park, Ca.: Sage. 488 pages. Practical guidance from diagnosing problems through to measuring and analysing programmes. Includes material on formative evaluation procedures, practical ethics, and cost-benefits.

Stringer, E. T. (1999) Action Research 2e, Thousand Oaks, CA.: Sage. 229 + xxv pages. Useful discussion of community-based action research directed at practitioners.

References

Bogden, R. and Biklen, S. K. (1992) Qualitative Research For Education, Boston: Allyn and Bacon.

Carr, W. and Kemmis, S. (1986) Becoming Critical. Education, knowledge and action research, Lewes: Falmer.

Chelimsky E. (1997) Thoughts for a new evaluation society. Evaluation 3(1): 97-118.

Elsdon, K. T. with Reynolds, J. and Stewart, S. (1995) Voluntary Organizations. Citizenship, learning and change, Leicester: NIACE.

Freire, P. (1972) Pedagogy of the Oppressed, London: Penguin.

Gauthier, A. H. and Furstenberg, F. F. (2001) 'Inequalities in the use of time by teenagers and young adults' in K. Vleminckx and T. M. Smeeding (eds.) Child Well-being, Child Poverty and Child Policy in Modern Nations Bristol: Policy Press.

Gitlin, A. and Smyth, J. (1989) Teacher Evaluation. Critical education and transformative alternatives, Lewes: Falmer Press.

Jeffs, T. and Smith, M. (eds.) (1990) Using Informal Education, Buckingham: Open University Press.

Jeffs and Smith, M. K. (2005) Informal Education. Conversation, democracy and learning 3e, Nottingham: Educational Heretics Press.

Larson, R. W. and Vera, A. (1999) 'How children and adolescents spend time across the world: work, play and developmental opportunities' Psychological Bulletin 125(6).

Merriam, S. B. (1988) Case Study Research in Education, San Francisco: Jossey-Bass.

Putman, R. D. (2000) Bowling Alone: The collapse and revival of American community, New York: Simon and Schuster.

Rogers, A. and Smith, M. K. (2006) Evaluation: Learning what matters, London: Rank Foundation/YMCA George Williams College. Available as a pdf: www.ymca.org.uk/rank/conference/evaluation_learning_what_matters.pdf.

Rubin, F. (1995) A Basic Guide to Evaluation for Development Workers, Oxford: Oxfam.

Schön, D. A. (1983) The Reflective Practitioner. How professionals think in action, London: Temple Smith.

Sewell, L. (1966) Looking at Youth Clubs, London: National Association of Youth Clubs. Available in the informal education archives: http://www.infed.org/archives/nayc/sewell_looking.htm.

Shadish, W. R., Cook, T. D. and Leviton, L. C. (1991) Foundations of Program Evaluation, Newbury Park C.A.: Sage.

Smith, H. and Smith, M. K. (forthcoming) The Art of Helping Others. Being around, being there, being wise. See www.infed.org/helping.

Acknowledgements and credits: Alan Rogers and Sarah Lloyd-Jones were a great help when updating this article - and some of the material in this piece first appeared in Rogers and Smith 2006.

The picture - Office of Mayhem Evaluation - is by xiaming and is reproduced here under a Creative Commons Attribution-Non-Commercial-Share Alike 2.0 Generic licence. Flickr: http://www.flickr.com/photos/xiaming/78385893/

How to cite this article: Smith, Mark K. (2001, 2006) 'Evaluation' in The encyclopedia of pedagogy and informal education, www.infed.org/biblio/b-eval.htm.

© Mark K. Smith 2001, 2006

Last Updated on

# data[].attributes.attachment_link.title
Evaluation for education, learning and change - theory and practice

# data[].attributes.attachment_link.description
Contents: introduction · on evaluation · three key dimensions · thinking about indicators · on being connoisseurs and critics · educators as action researchers · some issues when evaluating informal education · conclusion · further reading and references · acknowledgements · how to cite this article A lot is written about evaluation in education - a great deal of which is misleading and confused.

# data[].attributes.body
Reaction, Learning, Behaviour, Results

# data[].attributes.attachment_link.readable_text
Any time you deliver training to your team, you need to know how effective it's been. Are your people putting their learning into practice? And, is it positively impacting their role and the wider organization?

Kirkpatrick's Four-Level Training Evaluation Model can help you to answer questions like these. You can use it to objectively analyze the impact of training, to work out how well your team members learned, and to improve their learning in the future.

In this article, we'll explore Kirkpatrick's model and how to apply it. We'll also consider situations where it may not be appropriate.

Understanding Kirkpatrick's Four Levels

Donald Kirkpatrick, former Professor Emeritus at the University of Wisconsin, first published his model in 1959. He updated it in 1975, and again in 1993, when he published his best-known work, " Evaluating Training Programs."

Each successive level of the model represents a more precise measure of the effectiveness of a training program. It was developed further by Donald and his son, James; and then by James and his wife, Wendy Kayser Kirkpatrick.

In 2016, James and Wendy revised and clarified the original theory, and introduced the "New World Kirkpatrick Model" in their book, " Four Levels of Training Evaluation." One of the main additions is an emphasis on the importance of making training relevant to people's everyday jobs.

The four levels are Reaction, Learning, Behavior, and Results. We look at each level in greater detail, and explore how to apply it, below.

Level 1: Reaction

You want people to feel that training is valuable. Measuring how engaged they were, how actively they contributed, and how they reacted to the training helps you to understand how well they received it.

It also enables you to make improvements to future programs, by identifying important topics that might have been missing.

Questions to ask trainees include:

Did you feel that the training was worth your time?

Did you think that it was successful?

What were the biggest strengths and weaknesses of the training?

Did you like the venue and presentation style?

Did the training session accommodate your personal learning styles?

Were the training activities engaging?

What are the three most important things that you learned from this training?

From what you learned, what do you plan to apply in your job?

What support might you need to apply what you learned?

Identify how you want to measure people's reactions. Many people use employee satisfaction surveys to do this, but you can also watch trainees' body language during the session, or ask for verbal feedback.

Analyze the feedback, and consider the changes that you could make in response.

Level 2: Learning

Level 2 focuses on measuring what your trainees have and haven't learned. In the New World version of the tool, Level 2 also measures what they think they'll be able to do differently as a result, how confident they are that they can do it, and how motivated they are to make changes.

This demonstrates how training has developed their skills, attitudes and knowledge, as well as their confidence and commitment.

To measure how much your trainees have learned, start by identifying what you want to evaluate. Training sessions should have specific learning objectives, so make those your starting point.

You can measure learning in different ways, depending on the objectives. But it's helpful to measure these areas both before and after training.

Before the training begins, test your trainees to determine their knowledge, skill levels and attitudes. Then, when the training is finished, test your trainees a second time to measure what they have learned, or measure their learning with interviews or verbal assessments.

Note:

As a manager, you need to hold people accountable for improving their skills, and to offer them the support they need to do so.

Level 3: Behavior

This level helps you to understand how well people apply their training. It can also reveal where people might need help. But behavior can only change when conditions are favorable.

Imagine that you're assessing your team members after a training session. You can see little change, and you conclude that they learned nothing, and that the training was ineffective.

It's possible, however, that they actually learned a lot, but that the organizational or team culture obstructs behavioral change. Perhaps existing processes mean that there's little scope to apply new thinking, for example.

As a result, your people don't feel confident in applying new knowledge, or see few opportunities to do so. Or, they may not have had enough time to put it into practice.

Be sure to develop processes that encourage, reinforce and reward positive changes in behavior. The New World Kirkpatrick Model calls these processes "required drivers." If a team member uses a new skill effectively, highlight this and praise him or her for it.

Effectively measuring behavior is a longer-term process that should take place over weeks or months following the initial training. Questions to ask include:

Did the trainees put any of their learning to use?

Are trainees able to teach their new knowledge, skills or attitudes to other people?

Are trainees aware that they've changed their behavior?

One of the best ways to measure behavior is to conduct observations and interviews. Another is to integrate the use of new skills into the tasks that you set your team, so that people have the chance to demonstrate what they know.

Tip:

Managers need to be closely involved at this stage, assessing and coaching their team members in making behavior changes.

Level 4: Results

At this level, you analyze the final results of your training. This includes outcomes that you or your organization have decided are good for business and good for your team members, and which demonstrate a good return on investment (ROI). (Some adapted versions of the model actually have a Level 5, dedicated to working out ROI.)

Level 4 will likely be the most costly and time-consuming. Your biggest challenge will be to identify which outcomes, benefits, or final results are most closely linked to the training, and to come up with an effective way to measure these outcomes in the long term.

Modern trainers often use the Kirkpatrick model backward, by first stating the results that they want to see, and then developing the training that is most likely to deliver them. This helps to prioritize the goals of the training and make it more effective.

Here are some outcomes to consider, depending on the objectives of your training:

Increased employee retention.

Increased production.

Higher morale.

Reduced waste.

Increased sales.

Higher quality ratings.

Increased customer satisfaction.

Fewer staff complaints.

Make a series of short-term observations and measurements to check that changes in behavior due to training are making a worthwhile difference to your team's performance. The New World Kirkpatrick Model calls these "leading indicators."

Potential Pitfalls of Kirkpatrick's Model

Kirkpatrick's model remains popular, but it should be used with care. The basic structure is now more than 60 years old (despite its many updates), and the ways that people learn and organizations operate has changed radically in this time. Even the term "training" has been largely replaced by "learning and development."

Today, other, non-formal methods of workplace training are often more popular and effective (as shown by the 70:20:10 model). And, with the rise of personalized, user-directed learning, formal training is becoming less prominent. Kirkpatrick's model is not necessarily suited to this new approach to learning.

Another drawback is that Levels 3 and 4, which arguably yield the most useful information for the business, are time-consuming, resource-intensive, and expensive to implement. So the model may not be practical for all organizations, especially if you don't have a dedicated training or HR department to conduct the analysis. And, it's not ideal for all situations, such as one-off training.

Most importantly, organizations change in many ways, and these changes affect behaviors and results, as well as training. For example, measurable improvements in retention and productivity could result from the arrival of a new boss, or from a new computer system, rather than training. Or it could be a combination of these.

Kirkpatrick's model is great for evaluating training in a "scientific" way, but with so many possible variables, Level 4 may be limited in its usefulness.

Tip:

The New World Kirkpatrick Model seeks to address some of these challenges, by encouraging trainers and organizations to incorporate evaluation as part of the training design process.

Key Points

The Kirkpatrick Four-Level Training Evaluation Model is designed to objectively measure the effectiveness of training. The model was created by Donald Kirkpatrick in 1959, with several revisions made since.

The four levels are:

By analyzing each level, you can gain an understanding of how effective a training initiative was, and how to improve it in the future.

However, the model isn't practical in all situations, and measuring training effectiveness with it can be time-consuming and resource-intensive, so it should be used with caution.

# data[].attributes.attachment_link.title
Kirkpatrick's Four-Level Training Evaluation Model: Analyzing Training Effectiveness

# data[].attributes.attachment_link.description
Any time you deliver training to your team, you need to know how effective it's been. Are your people putting their learning into practice? And, is it positively impacting their role and the wider organization? Kirkpatrick's Four-Level Training Evaluation Model can help you to answer questions like these.

# data[].attributes.headline
Curriculum Development Strategy at East Coast College

# data[].attributes.subject
Curriculum Development Strategy at East Coast College

# data[].attributes.attachment_link.title
QD009 - Curriculum Development and Planning Strategy

# data[].attributes.attachment_link.readable_text
Understanding Classroom Dynamics: The Critical Incident Questionnaire The Classroom Critical Incident Questionnaire
Assessing learning, critical reflection, and quality educational outcomes: The Critical Incident Questionnaire by Donald Gilstrap and Jason Dupree, College and Research Libraries, 2008, Vol 69, No. 5.

By Their Pupils They'll Be Taught: Using Critical Incident Questionnaire as Feedback by Mary Ann Jacobs, Journal of Invitational Theory and Practice, 2015, Vol. 21, pp. 9-22.

The Critical Incident Questionnaire (CIQ): From Research to Practice and Back Again
By Jeffrey Keefer, Adult Education Research Conference Proceedings, 2009

A Regression Model of Predictor Variables on Critical Reflection in the Classroom: Integration of the Critical Incident Questionnaire and the Framework for Reflective Thinking D. Gilstrap & J. Dupree. Journal of Academic Librarianship, Vol. 34, No. 6, 2008

FORMATIVE EVALUATION: USING THE CRITICAL INCIDENT QUESTIONNAIRE IN A GRADUATE COUNSELING COURSE ON CULTURAL DIVERSITY By K.S. Linstrum, G. Ballard, T. Shelby Journal of Intercultural Disciplines, Vol. 10, 2012

A study of the uses of a blog-based critical incident questionnaire in further education Paul Smith, Doctoral Dissertation, University of Derby (UK), 2018

Connecting Critical Reflection and Group Development in Online Adult Education Classrooms
By Michelle Glowacki-Dudka and Nicole Barnett (PDF file)

Interrogating students' perceptions of their online learning experiences with Brookfield's Critical Incident Questionnaire
By Liam Phalen, Distance Education, Vol. 33, No. 1 (2012)

Using Critical Incident Questionnaires (CIQs) to Foster Critical Reflection and Formative Assessmentsimplified with Google Forms. By H. Brooke Hessler, Ph.D.

# data[].attributes.attachment_link.title
Critical Incident Questionnaire - Stephen D. Brookfield

# data[].attributes.attachment_link.description
Understanding Classroom Dynamics: The Critical Incident Questionnaire The Classroom Critical Incident Questionnaire Assessing learning, critical reflection, and quality educational outcomes: The Critical Incident Questionnaire by Donald Gilstrap and Jason Dupree, College and Research Libraries, 2008, Vol 69, No. 5.

# data[].attributes.attachment_link.title
The Critical Incident Questionnaire (CIQ) From Research to Pract

# data[].attributes.attachment_link.readable_text
This article will explain what curriculum development is, why it's important for an instructor's pedagogy and discuss the three different types of curriculum design.

Curriculum development can be defined as the step-by-step process used to create positive improvements in the courses offered by a school, college or university. The world changes every day and new discoveries have to be roped into the education curricula. Innovative teaching techniques and strategies (such as active learning or blended learning) are constantly being devised in order to improve the student learning experience. As a result, an institution has to have a plan in place for acknowledging these shifts and then be able to implement them in the school curriculum.

Want to learn how to make teaching fun and effective?

Subscribe to Top Hat's weekly blog recap:

What is curriculum development?

The word curriculum has roots in Latin. It originally meant "racing chariot" and came from the verb currere, "to run".

The way we understand and theorize about curriculum nowadays has altered significantly over the years. Today, the most simple definition of the word "curriculum" is the subjects comprising a course of study at schools, universities or colleges

Of course differences in course design exist-a math course taken at one university may cover the same material, but the educator could teach it in a different way-but the core fundamentals of curriculum development remain the same.

What are the models of curriculum development?

Current curriculum models can be broken down into two broad categories-the product model and the process model. The product model is results-oriented. Grades are the prime objective, with the focus lying more on the finished product rather than on the learning process. The process model, however, is more open-ended, and focuses on how learning develops over a period of time. These two models need to be taken into account when developing curriculum.

What is curriculum planning?

Curriculum planning involves the implementation of different types of instructional strategies and organizational methods that are focused on achieving optimal student development and student learning outcomes. Instructors might structure their curriculum around daily lesson plans, a specific assignment, a chunk of coursework, certain units within a class, or an entire educational program.

During the curriculum planning phase, teachers consider factors that might complement or hinder their lesson curriculum. These include institutional requirements. Each administrator at a university or college will have guidelines, principles and a framework that instructors are required to reference as they build out their curriculums. Educators are responsible for ensuring that their curriculum planning meets the students' educational needs, and that the materials used are current and comprehensible.

Educators should employ the curriculum process that best incorporates the six components of effective teaching. These components are applicable at both the undergraduate and graduate level:

To demonstrate knowledge of content;

To demonstrate the knowledge of students;

Select suitable instructional strategy goals;

To demonstrate knowledge of resources;

To design coherent instruction;

Assess student learning.

What is curriculum design?

Now that we've gone over curriculum development and planning, let's discuss curriculum design. Curriculum design is the deliberate organization of curriculum within a course or classroom. When instructors design their curriculums, they identity what will be done, who will do it and when, as well as what the objective of each course is. Remember that the curriculum contains the knowledge and skills that a student needs to master in order to move to the next level. By thinking about how their curriculum is designed, teachers ensure they've covered all the necessary requirements. From there, they can start exploring various approaches and teaching methods that can help them achieve their goals.

What are the types of curriculum design?

There are three basic types of curriculum design-subject-centered, learner-centered, and problem-centered design.
Subject-centered curriculum design revolves around a particular subject matter or discipline, such as mathematics, literature or biology. This type of curriculum design tends to focus on the subject, rather than the student. It is the most common type of standardized curriculum that can be found in K-12 public schools.

Teachers compile lists of subjects, and specific examples of how they should be studied. In higher education, this methodology is typically found in large university or college classes where teachers focus on a particular subject or discipline.

Subject-centered curriculum design is not student-centered, and the model is less concerned with individual learning styles compared to other forms of curriculum design. This can lead to problems with student engagement and motivation and may cause students who are not responsive to this model to fall behind.

Learner-centered curriculum design, by contrast, revolves around student needs, interests and goals. It acknowledges that students are not uniform but individuals, and therefore should not, in all cases, be subject to a standardized curriculum. This approach aims to empower learners to shape their education through choices.

Differentiated instructional plans provide an opportunity to select assignments, teaching and learning experiences, or activities. This form of curriculum design has been shown to engage and motivate students. The drawback to this form of curriculum design is that it can create pressure on the educator to source materials specific to each student's learning needs. This can be challenging due to teaching time constraints. Balancing individual student interests with the institution's required outcomes could prove to be a daunting task.

Problem-centered curriculum design teaches students how to look at a problem and formulate a solution. Considered an authentic form of learning because students are exposed to real-life issues, this model helps students develop skills that are transferable to the real world. Problem-centered curriculum design has been shown to increase the relevance of the curriculum and encourages creativity, innovation and collaboration in the classroom. The drawback to this format is that it does not always consider individual learning styles.

By considering all three types of curriculum design before they begin planning, instructors can choose the types that are best suited to both their students and their course.

Conclusion

Developing, designing and implementing an education curriculum is no easy task. With the rise of educational technology and the diverse types of students attending higher educational institutions these days, instructors have their work cut out for them. But by following the fundamental guidelines and framework of curriculum development, educators will be setting themselves - and their students - up for long-term success.

Amanda Stutt is a Vancouver-based writer and editor. Her stories have been published in the Village Voice, The Globe and Mail, Ottawa Business Journal, CBC News, Global News, The Vancouver Sun, and The Tyee. She is a returning guest lecturer for the Magazine Journalism program at Douglas College.

# data[].attributes.attachment_link.title
Curriculum Development and The 3 Models Explained | Top Hat

# data[].attributes.attachment_link.description
This article will explain what curriculum development is, why it's important for an instructor's pedagogy and discuss the three different types of curriculum design. Curriculum development can be defined as the step-by-step process used to create positive improvements in the courses offered by a school, college or university.

# data[].attributes.attachment_link.description
Joined up thinking from education to employment, further education, skills and apprenticeship news. Thought leadership by FE community experts, FE News.

# data[].attributes.attachment_link.title
Sir Ken Robinson: Changing education paradigms | TED Talk

# data[].attributes.attachment_link.description
In this talk from RSA Animate, Sir Ken Robinson lays out the link between 3 troubling trends: rising drop-out rates, schools’ dwindling stake in the arts, and ADHD. An important, timely talk for parents and teachers.

# data[].attributes.headline
Post-covid: Building a recovery curriculum

# data[].attributes.subject
Post-covid: Building a recovery curriculum

# data[].attributes.attachment_link.readable_text
Curriculum theory and practice.The organization of schooling and further education has long been associated with the idea of a curriculum. But what actually is curriculum, and how might it be conceptualized? We explore curriculum theory and practice and its relation to informal education.

Contents: introduction · curriculum as transmission · curriculum as product · curriculum as process · curriculum as praxis · curriculum and context · curriculum and informal education · further reading · links · how to cite this article

The idea of curriculum is hardly new - but the way we understand and theorize it has altered over the years - and there remains considerable dispute as to meaning. It has its origins in the running/chariot tracks of Greece. It was, literally, a course. In Latin curriculum was a racing chariot; currere was to run. A useful starting point for us here might be the definition offered by John Kerr and taken up by Vic Kelly in a standard work on the subject. Kerr defines curriculum as, 'All the learning which is planned and guided by the school, whether it is carried on in groups or individually, inside or outside the school. (quoted in Kelly 1983: 10; see also, Kelly 1999). This gives us some basis to move on - and for the moment all we need to do is highlight two of the key features:

L earning is planned and guided. We have to specify in advance what we are seeking to achieve and how we are to go about it.

The definition refers to schooling. We should recognize that our current appreciation of curriculum theory and practice emerged in the school and in relation to other schooling ideas such as subject and lesson.

In what follows we are going to look at four ways of approaching curriculum theory and practice:

1. Curriculum as a body of knowledge to be transmitted.

2. Curriculum as an attempt to achieve certain ends in students - product.

3. Curriculum as process.

4. Curriculum as praxis.

It is helpful to consider these ways of approaching curriculum theory and practice in the light of Aristotle's influential categorization of knowledge into three disciplines: the theoretical, the productive and the practical.

Here we can see some clear links - the body of knowledge to be transmitted in the first is that classically valued as 'the canon'; the process and praxis models come close to practical deliberation; and the technical concerns of the outcome or product model mirror elements of Aristotle's characterization of the productive. More this will be revealed as we examine the theory underpinning individual models.

Curriculum as a syllabus to be transmitted

Many people still equate a curriculum with a syllabus. Syllabus, naturally, originates from the Greek (although there was some confusion in its usage due to early misprints). Basically it means a concise statement or table of the heads of a discourse, the contents of a treatise, the subjects of a series of lectures. In the form that many of us will have been familiar with it is connected with courses leading to examinations - teachers talk of the syllabus associated with, say, the Cambridge Board French GSCE exam. What we can see in such documents is a series of headings with some additional notes which set out the areas that may be examined.

A syllabus will not generally indicate the relative importance of its topics or the order in which they are to be studied. In some cases as Curzon (1985) points out, those who compile a syllabus tend to follow the traditional textbook approach of an 'order of contents', or a pattern prescribed by a 'logical' approach to the subject, or - consciously or unconsciously - a the shape of a university course in which they may have participated. Thus, an approach to curriculum theory and practice which focuses on syllabus is only really concerned with content. Curriculum is a body of knowledge-content and/or subjects. Education in this sense, is the process by which these are transmitted or 'delivered' to students by the most effective methods that can be devised (Blenkin et al 1992: 23).

Where people still equate curriculum with a syllabus they are likely to limit their planning to a consideration of the content or the body of knowledge that they wish to transmit. 'It is also because this view of curriculum has been adopted that many teachers in primary schools', Kelly (1985: 7) claims, 'have regarded issues of curriculum as of no concern to them, since they have not regarded their task as being to transmit bodies of knowledge in this manner'.

Curriculum as product

The dominant modes of describing and managing education are today couched in the productive form. Education is most often seen as a technical exercise. Objectives are set, a plan drawn up, then applied, and the outcomes (products) measured. It is a way of thinking about education that has grown in influence in the United Kingdom since the late 1970s with the rise of vocationalism and the concern with competencies. Thus, in the late 1980s and the 1990s many of the debates about the National Curriculum for schools did not so much concern how the curriculum was thought about as to what its objectives and content might be.

It is the work of two American writers Franklin Bobbitt (1918; 1928) and Ralph W. Tyler (1949) that dominate theory and practice within this tradition. In The Curriculum Bobbitt writes as follows:

The central theory [of curriculum] is simple. Human life, however varied, consists in the performance of specific activities. Education that prepares for life is one that prepares definitely and adequately for these specific activities. However numerous and diverse they may be for any social class they can be discovered. This requires only that one go out into the world of affairs and discover the particulars of which their affairs consist. These will show the abilities, attitudes, habits, appreciations and forms of knowledge that men need. These will be the objectives of the curriculum. They will be numerous, definite and particularized. The curriculum will then be that series of experiences which children and youth must have by way of obtaining those objectives. (1918: 42)

This way of thinking about curriculum theory and practice was heavily influenced by the development of management thinking and practice. The rise of 'scientific management' is often associated with the name of its main advocate F. W. Taylor. Basically what he proposed was greater division of labour with jobs being simplified; an extension of managerial control over all elements of the workplace; and cost accounting based on systematic time-and-motion study. All three elements were involved in this conception of curriculum theory and practice. For example, one of the attractions of this approach to curriculum theory was that it involved detailed attention to what people needed to know in order to work, live their lives and so on. A familiar, and more restricted, example of this approach can be found in many training programmes, where particular tasks or jobs have been analyzed - broken down into their component elements - and lists of competencies drawn up. In other words, the curriculum was not to be the result of 'armchair speculation' but the product of systematic study. Bobbitt's work and theory met with mixed responses. One telling criticism that was made, and can continue to be made, of such approaches is that there is no social vision or programme to guide the process of curriculum construction. As it stands it is a technical exercise. However, it wasn't criticisms such as this which initially limited the impact of such curriculum theory in the late 1920s and 1930s. Rather, the growing influence of 'progressive', child-centred approaches shifted the ground to more romantic notions of education. Bobbitt's long lists of objectives and his emphasis on order and structure hardly sat comfortably with such forms.

The Progressive movement lost much of its momentum in the late 1940s in the United States and from that period the work of Ralph W. Tyler, in particular, has made a lasting impression on curriculum theory and practice. He shared Bobbitt's emphasis on rationality and relative simplicity. His theory was based on four fundamental questions:

1. What educational purposes should the school seek to attain?

2. What educational experiences can be provided that are likely to attain these purposes?

3. How can these educational experiences be effectively organized?

4. How can we determine whether these purposes are being attained? (Tyler 1949: 1)

Like Bobbitt he also placed an emphasis on the formulation of behavioural objectives.

Since the real purpose of education is not to have the instructor perform certain activities but to bring about significant changes in the students' pattern of behaviour, it becomes important to recognize that any statements of objectives of the school should be a statement of changes to take place in the students. (Tyler 1949: 44)

We can see how these concerns translate into a nicely-ordered procedure: one that is very similar to the technical or productive thinking set out below.

Step 1: Diagnosis of need

Step 2: Formulation of objectives

Step 3: Selection of content

Step 4: Organization of content

Step 5: Selection of learning experiences

Step 6: Organization of learning experiences

Step 7: Determination of what to evaluate and of the ways and means of doing it. (Taba 1962)

The attraction of this way of approaching curriculum theory and practice is that it is systematic and has considerable organizing power. Central to the approach is the formulation of behavioural objectives - providing a clear notion of outcome so that content and method may be organized and the results evaluated.

There are a number of issues with this approach to curriculum theory and practice. The first is that the plan or programme assumes great importance. For example, we might look at a more recent definition of curriculum as: 'A programme of activities (by teachers and pupils) designed so that pupils will attain so far as possible certain educational and other schooling ends or objectives (Grundy 1987: 11). The problem here is that such programmes inevitably exist prior to and outside the learning experiences. This takes much away from learners. They can end up with little or no voice. They are told what they must learn and how they will do it. The success or failure of both the programme and the individual learners is judged on the basis of whether pre-specified changes occur in the behaviour and person of the learner (the meeting of behavioural objectives). If the plan is tightly adhered to, there can only be limited opportunity for educators to make use of the interactions that occur. It also can deskill educators in another way. For example, a number of curriculum programmes, particularly in the USA, have attempted to make the student experience 'teacher proof'. The logic of this approach is for the curriculum to be designed outside of the classroom or school, as is the case with the National Curriculum in the UK. Educators then apply programmes and are judged by the products of their actions. It turns educators into technicians.

Second, there are questions around the nature of objectives. This model is hot on measurability. It implies that behaviour can be objectively, mechanistically measured. There are obvious dangers here - there always has to be some uncertainty about what is being measured. We only have to reflect on questions of success in our work. It is often very difficult to judge what the impact of particular experiences has been. Sometimes it is years after the event that we come to appreciate something of what has happened. For example, most informal educators who have been around a few years will have had the experience of an ex-participant telling them in great detail about how some forgotten event (forgotten to the worker that is) brought about some fundamental change. Yet there is something more.

In order to measure, things have to be broken down into smaller and smaller units. The result, as many of you will have experienced, can be long lists of often trivial skills or competencies. This can lead to a focus in this approach to curriculum theory and practice on the parts rather than the whole; on the trivial, rather than the significant. It can lead to an approach to education and assessment which resembles a shopping list. When all the items are ticked, the person has passed the course or has learnt something. The role of overall judgment is somehow sidelined.

Third, there is a real problem when we come to examine what educators actually do in the classroom, for example. Much of the research concerning teacher thinking and classroom interaction, and curriculum innovation has pointed to the lack of impact on actual pedagogic practice of objectives (see Stenhouse 1974; and Cornbleth 1990, for example). One way of viewing this is that teachers simply get it wrong - they ought to work with objectives. I think we need to take this problem very seriously and not dismiss it in this way. The difficulties that educators experience with objectives in the classroom may point to something inherently wrong with the approach - that it is not grounded in the study of educational exchanges. It is a model of curriculum theory and practice largely imported from technological and industrial settings.

Fourth, there is the problem of unanticipated results. The focus on pre-specified goals may lead both educators and learners to overlook learning that is occurring as a result of their interactions, but which is not listed as an objective.

The apparent simplicity and rationality of this approach to curriculum theory and practice, and the way in which it mimics industrial management have been powerful factors in its success. A further appeal has been the ability of academics to use the model to attack teachers:

I believe there is a tendency, recurrent enough to suggest that it may be endemic in the approach, for academics in education to use the objectives model as a stick with which to beat teachers. 'What are your objectives?' is more often asked in a tone of challenge than one of interested and helpful inquiry. The demand for objectives is a demand for justification rather than a description of ends... It is not about curriculum design, but rather an expression of irritation in the problems of accountability in education. (Stenhouse 1974: 77)

So what are the other alternatives?

Curriculum as process

We have seen that the curriculum as product model is heavily dependent on the setting of behavioural objectives. The curriculum, essentially, is a set of documents for implementation. Another way of looking at curriculum theory and practice is via process. In this sense curriculum is not a physical thing, but rather the interaction of teachers, students and knowledge. In other words, curriculum is what actually happens in the classroom and what people do to prepare and evaluate. What we have in this model is a number of elements in constant interaction. It is an active process and links with the practical form of reasoning set out by Aristotle.

Curriculum as process

Teachers enter particular schooling and situations with

an ability to think critically, -in-action

an understanding of their role and the expectations others have of them, and

a proposal for action which sets out essential principles and features of the educational encounter.

Guided by these, they encourage

conversations between, and with, people in the situation

out of which may come

They

continually evaluate the process and what they can see of outcomes.

Perhaps the two major things that set this apart from the model for informal education are first, the context in which the process occurs ('particular schooling situations'); and second, the fact that teachers enter the classroom or any other formal educational setting with a more fully worked-through idea of what is about to happen. Here I have described that as entering the situation with 'a proposal for action which sets out essential principles and features of the educational encounter'.

This form of words echoes those of Lawrence Stenhouse (1975) who produced one of the best-known explorations of a process model of curriculum theory and practice. He defined curriculum tentatively: 'A curriculum is an attempt to communicate the essential principles and features of an educational proposal in such a form that it is open to critical scrutiny and capable of effective translation into practice'. He suggests that a curriculum is rather like a recipe in cookery.

It can be criticized on nutritional or gastronomic grounds - does it nourish the students and does it taste good? - and it can be criticized on the grounds of practicality - we can't get hold of six dozen larks' tongues and the grocer can't find any ground unicorn horn! A curriculum, like the recipe for a dish, is first imagined as a possibility, then the subject of experiment. The recipe offered publicly is in a sense a report on the experiment. Similarly, a curriculum should be grounded in practice. It is an attempt to describe the work observed in classrooms that it is adequately communicated to teachers and others. Finally, within limits, a recipe can varied according to taste. So can a curriculum. (Stenhouse 1975: 4-5)

Stenhouse shifted the ground a little bit here. He was not saying that curriculum is the process, but rather the means by which the experience of attempting to put an educational proposal into practice is made available. The reason why he did this, I suspect, is that otherwise there is a danger of widening the meaning of the term so much that it embraces almost everything and hence means very little. For example, in a discussion of the so-called 'youth work curriculum' (Newman & Ingram 1989), the following definition was taken as a starting point: 'those processes which enhance or, if they go wrong, inhibit a person's learning'. This was then developed and a curriculum became: 'an organic process by which learning is offered, accepted and internalized' (Newman & Ingram 1989: 1). The problem with this sort of definition, as Robin Barrow (1984) points out, is that what this does is to widen the meaning of the term to such an extent that it just about becomes interchangeable with 'education' itself. More specifically, if curriculum is process then the word curriculum is redundant because process would do very nicely! The simple equation of curriculum with process is a very slap-happy basis on which to proceed.

We also need to reflect on why curriculum theory and practice came into use by educators (as against policy-makers). It was essentially as a way of helping them to think about their work before, during and after interventions; as a means of enabling educators to make judgments about the direction their work was taking. This is what Stenhouse was picking up on.

Stenhouse on curriculum

As a minimum, a curriculum should provide a basis for planning a course, studying it empirically and considering the grounds of its justification. It should offer:

A. In planning:

1. Principle for the selection of content - what is to be learned and taught

2. Principles for the development of a teaching strategy - how it is to be learned and taught.

3. Principles for the making of decisions about sequence.

4. Principles on which to diagnose the strengths and weaknesses of individual students and differentiate the general principles 1, 2 and 3 above, to meet individual cases.

B. In empirical study:

1. Principles on which to study and evaluate the progress of students.

2. Principles on which to study and evaluate the progress of teachers.

3. Guidance as to the feasibility of implementing the curriculum in varying school contexts, pupil contexts, environments and peer-group situations.

4. Information about the variability of effects in differing contexts and on different pupils and an understanding of the causes of the variation.

C. In relation to justification:

A formulation of the intention or aim of the curriculum which is accessible to critical scrutiny.

Stenhouse 1975: 5

There are a number of contrasts in this model of curriculum theory and practice as compared with the product model. First, where the product model appeals to the workshop for a model, this process model looks to the world of experimentation.

The idea is that of an educational science in which each classroom is a laboratory, each teacher a member of the scientific community... The crucial point is that the proposal is not to be regarded as an unqualified recommendation but rather as a provisional specification claiming no more than to be worth putting to the test of practice, Such proposals claim to be intelligent rather than correct. (Stenhouse 1975: 142)

Thus, in this sense, a curriculum is a particular form of specification about the practice of teaching. It is not a package of materials or a syllabus of ground to be covered. 'It is a way of translating any educational idea into a hypothesis testable in practice. It invites critical testing rather than acceptance' (Stenhouse 1975: 142).

Second, and associated with the above, given the uniqueness of each classroom setting, it means that any proposal, even at school level, needs to be tested, and verified by each teacher in his/her classroom ( ibid: 143). It is not like a curriculum package which is designed to be delivered almost anywhere.

Third, outcomes are no longer the central and defining feature. Rather than tightly specifying behavioural objectives and methods in advance, what happens in this model of curriculum theory and practice is that content and means develop as teachers and students work together.

Fourth, the learners in this model are not objects to be acted upon. They have a clear voice in the way that the sessions evolve. The focus is on interactions. This can mean that attention shifts from teaching to learning. The product model, by having a pre-specified plan or programme, tends to direct attention to teaching. For example, how can this information be got over? A process approach to curriculum theory and practice, it is argued by writers like Grundy (1987), tends towards making the process of learning the central concern of the teacher. This is because this way of thinking emphasizes interpretation and meaning-making. As we have seen each classroom and each exchange is different and has to be made sense of.

However, when we come to think about this way of approaching curriculum in practice, a number of possible problems do arise. The first is a problem for those who want some greater degree of uniformity in what is taught. This approach to the theory of curriculum, because it places meaning-making and thinking at its core and treats learners as subjects rather than objects, can lead to very different means being employed in classrooms and a high degree of variety in content. As Stenhouse comments, the process model is essentially a critical model, not a marking model.

It can never be directed towards an examination as an objective without loss of quality, since the standards of the examination then override the standards immanent in the subject. This does not mean that students taught on the process model cannot be examined, but it does mean that the examinations must be taken in their stride as they pursue other aspirations. And if the examination is a by-product there is an implication that the quality the student shows in it must be an under-estimate of his real quality. It is hence rather difficult to get the weak student through an examination using a process model. Crammers cannot use it, since it depends upon a commitment to educational aims. (Stenhouse 1975: 95)

To some extent variation is limited by factors such as public examinations. The exchange between students and teachers does not float free of the context in which it arises. At the end of the day many students and their families place a high premium on exam or subject success and this inevitably enters into the classroom. This highlights a second problem with the model we have just outlined - that it may not pay enough attention to the context in which learning takes place (more of this later).

Third, there is the 'problem' of teachers. The major weakness and, indeed, strength of the process model is that it rests upon the quality of teachers. If they are not up to much then there is no safety net in the form of prescribed curriculum materials. The approach is dependent upon the cultivation of wisdom and meaning-making in the classroom. If the teacher is not up to this, then there will be severe limitations on what can happen educationally. There have been some attempts to overcome this problem by developing materials and curriculum packages which focus more closely on the 'process of discovery' or 'problem-solving', for example in science. But there is a danger in this approach. Processes become reduced to sets of skills - for example, how to light a bunsen burner. When students are able to demonstrate certain skills, they are deemed to have completed the process. As Grundy comments, the actions have become the ends; the processes have become the product. Whether or not students are able to apply the skills to make sense of the world around them is somehow overlooked (Grundy 1987: 77).

Fourth, we need to look back at our process model of curriculum theory and practice and what we have subsequently discussed, and return to Aristotle and to Freire. The model we have looked at here does not fully reflect the process explored earlier. In particular, it does not make explicit the commitments associated with phronesis. And it is to that we will now turn.

Curriculum as praxis

Curriculum as praxis is, in many respects, a development of the process model. While the process model is driven by general principles and places an emphasis on judgment and meaning making, it does not make explicit statements about the interests it serves. It may, for example, be used in such a way that does not make continual reference to collective human well-being and to the emancipation of the human spirit. The praxis model of curriculum theory and practice brings these to the centre of the process and makes an explicit commitment to emancipation. Thus action is not simply informed, it is also committed. It is praxis.

Critical pedagogy goes beyond situating the learning experience within the experience of the learner: it is a process which takes the experiences of both the learner and the teacher and, through dialogue and negotiation, recognizes them both as problematic... [It] allows, indeed encourages, students and teachers together to confront the real problems of their existence and relationships... When students confront the real problems of their existence they will soon also be faced with their own oppression. (Grundy 1987: 105)

We can amend our 'curriculum as process' model to take account of these concerns.

Curriculum as praxis

Teachers enter particular schooling and situations with

a personal, but shared idea of the good and a commitment to human emancipation,

an ability to think critically, -in-action

an understanding of their role and the expectations others have of them, and

a proposal for action which sets out essential principles and features of the educational encounter.

Guided by these, they encourage

conversations between, and with, people in the situation

out of which may come

informed and committed action.

They

continually evaluate the process and what they can see of outcomes.

In this approach the curriculum itself develops through the dynamic interaction of action and reflection. 'That is, the curriculum is not simply a set of plans to be implemented, but rather is constituted through an active process in which planning, acting and evaluating are all reciprocally related and integrated into the process' (Grundy 1987: 115). At its centre is praxis: informed, committed action.

How might we recognize this? First, I think we should be looking for practice which does not focus exclusively on individuals, but pays careful attention to collective understandings and practices and to structural questions. For example, in sessions which seek to explore the experiences of different cultural and racial groups in society, we could be looking to see whether the direction of the work took people beyond a focus on individual attitudes. Are participants confronting the material conditions through which those attitudes are constituted, for example?

Second, we could be looking for a commitment expressed in action to the exploration of educators' values and their practice. Are they, for example, able to say in a coherent way what they think makes for human well-being and link this with their practice? We could also be looking for certain values - especially an emphasis on human emancipation.

Third, we could expect practitioners committed to praxis to be exploring their practice with their peers. They would be able to say how their actions with respect to particular interventions reflected their ideas about what makes for the good, and to say what theories were involved.

Curriculum in context

To round off this discussion of curriculum we do need to pay further attention to the social context in which it is created. One criticism that has been made of the praxis model (especially as it is set out by Grundy) is that it does not place a strong enough emphasis upon context. This is a criticism that can also be laid at the door of the other approaches. In this respect the work of Catherine Cornbleth (1990) is of some use. She sees curriculum as a particular type of process. Curriculum for her is what actually happens in classrooms, that is, 'an ongoing social process comprised of the interactions of students, teachers, knowledge and milieu' (1990: 5). In contrast, Stenhouse defines curriculum as the attempt to describe what happens in classrooms rather than what actually occurs. Cornbleth further contends that curriculum as practice cannot be understood adequately or changed substantially without attention to its setting or context. Curriculum is contextually shaped. While I may quibble about the simple equation of curriculum with process, what Cornbleth does by focusing on the interaction is to bring out the significance of context.

First, by introducing the notion of milieu into the discussion of curriculum she again draws attention to the impact of some factors that we have already noted. Of especial significance here are examinations and the social relationships of the school - the nature of the teacher-student relationship, the organization of classes, streaming and so on. These elements are what are sometimes known as the hidden curriculum. This was a term credited to Philip W. Jackson (1968) but it had been present as an acknowledged element in education for some time before. For example, John Dewey in Experience and Education referred to the 'collateral learning' of attitudes that occur in schools, and that may well be of more long-range importance than the explicit school curriculum (1938: 48). A fairly standard (product) definition of the 'hidden curriculum' is given by Vic Kelly. He argues it is those things which students learn, 'because of the way in which the work of the school is planned and organized but which are not in themselves overtly included in the planning or even in the consciousness of those responsible for the school arrangements (1988: 8). The learning associated with the 'hidden curriculum' is most often treated in a negative way. It is learning that is smuggled in and serves the interests of the status quo. The emphasis on regimentation, on bells and time management, and on streaming are sometimes seen as preparing young people for the world of capitalist production. What we do need to recognize is that such 'hidden' learning is not all negative and can be potentially liberating. 'In so far as they enable students to develop socially valued knowledge and skills... or to form their own peer groups and subcultures, they may contribute to personal and collective autonomy and to possible critique and challenge of existing norms and institutions' (Cornbleth 1990: 50). What we also need to recognize is that by treating curriculum as a contextualized social process, the notion of hidden curriculum becomes rather redundant. If we need to stay in touch with milieu as we build curriculum then it is not hidden but becomes a central part of our processes.

Second, by paying attention to milieu, we can begin to get a better grasp of the impact of structural and socio-cultural process on teachers and students. As Cornbleth argues, economic and gender relations, for example, do not simply bypass the systemic or structural context of curriculum and enter directly into classroom practice. They are mediated by intervening layers of the education system (Cornbleth 1990: 7). Thus, the impact of these factors may be quite different to that expected.

Third, if curriculum theory and practice is inextricably linked to milieu then it becomes clear why there have been problems about introducing it into non-schooling contexts like youth work; and it is to this area which we will now turn.

Curriculum as the boundary between formal and informal education

Jeffs and Smith (1990; 1999) have argued that the notion of curriculum provides a central dividing line between formal and informal education. They contend that curriculum theory and practice was formed within the schooling context and that there are major problems when it is introduced into informal forms of pedagogy.

The adoption of curriculum theory and practice by some informal educators appears to have arisen from a desire to be clear about content. Yet there are crucial difficulties with the notion of curriculum in this context. These centre around the extent to which it is possible to have a clear idea, in advance (and even during the process), of the activities and topics that will be involved in a particular piece of work.

At any one time, outcomes may not be marked by a high degree of specificity. In a similar way, the nature of the activities used often cannot be predicted. It may be that we can say something about how the informal educator will work. However, knowing in advance about broad processes and ethos isn't the same as having a knowledge of the programme. We must, thus, conclude that approaches to the curriculum which focus on objectives and detailed programmes appear to be incompatible with informal education. ( Jeffs & Smith 1990: 15)

In other words, they are arguing that a product model of curriculum is not compatible with the emphasis on process and praxis within informal education.

However, process and praxis models of curriculum also present problems in the context of informal education. If you look back at at our models of process and compare them with the model of informal education presented above then it is clear that we can have a similar problem with pre-specification. One of the key feature that differentiates the two is that the curriculum model has the teacher entering the situation with a proposal for action which sets out the essential principles and features of the educational encounter. Informal educators do not have, and do not need, this element. They do not enter with a clear proposal for action. Rather, they have an idea of what makes for human well-being, and an appreciation of their overall role and strategy (strategy here being some idea about target group and broad method e.g. detached work). They then develop their aims and interventions in interaction. And what is this element we have been discussing? It is nothing more nor less than what Stenhouse considers to be a curriculum!

The other key difference is context. Even if we were to go the whole hog and define curriculum as process there remain substantive problems. As Cornbleth (1990), and Jeffs and Smith (1990, 1999) have argued, curriculum cannot be taken out of context, and the context in which it was formed was the school. Curriculum theory and practice only makes sense when considered alongside notions like class, teacher, course, lesson and so on. You only have to look at the language that has been used by our main proponents: Tyler, Stenhouse, Cornbleth and Grundy, to see this. It is not a concept that stands on its own. It developed in relation to teaching and within particular organizational relationships and expectations. Alter the context and the nature of the process alters . We then need different ways of describing what is going on. Thus, it is no surprise that when curriculum theory and practice are introduced into what are essentially informal forms of working such as youth work and community work, their main impact is to formalize significant aspects of the work. One of the main outcome of curriculum experiments within youth work has been work, for example in the field of health promotion, which involve pre-specified activities, visiting workers, regular meetings and so on. Within the language of youth work these are most often called programmes or projects ( Foreman 1990). Within a school they would be called a course.

What is being suggested here is that when informal educators take on the language of curriculum they are crossing the boundary between their chosen specialism and the domain of formal education. This they need to do from time to time. There will be formal interludes in their work, appropriate times for them to mount courses and to discuss content and method in curriculum terms. But we should not fall into the trap of thinking that to be educators we have to adopt curriculum theory and practice. The fact that so many have been misled into believing this demonstrates just how powerful the ideas of schooling are. Education is something more than schooling.

Conclusion

We have explored four different approaches to curriculum theory and practice:

Curriculum as a body of knowledge to be transmitted.

Curriculum as an attempt to achieve certain ends in students - product.

Curriculum as process.

Curriculum as praxis.

In a number of respects these different bodies of curriculum theory and practice link to the four main forces in North American curriculum-making in the twentieth century: the liberal educators; the scientific curriculum makers; the developmental/person-centred; and the social meliorists (those that sought more radical social change) (after Kliebart 1987).

Guardians of an ancient tradition tied to the power of reason and the finest elements of the Western cultural heritage

Human life consists in the performance of specific activities. Education that prepares for life is one that prepares definitely and adequately for these specific activities.

The natural order of development in the child was most significant and scientifically defensible basis for determining what should be taught

Schools as a major, perhaps the, principal force for social change and social justice

Systematic development of reasoning power and the communication of 'the canon'.

Influenced by the rise of scientific management and notions of social efficiency. Focus on setting objectives (the statement of changes to take place in the students) and the organization of schooling to meet these.

Sought a curriculum in harmony with the child's 'real' interests, needs and learning patterns

Corruption and vice, inequalities of race and gender, and the abuse of privilege and power should be addressed directly. with the aim of raising a new generation equipped to deal effectively with these abuses.

We shouldn't push the similarities too far - but there are some interesting overlaps - and this does alert us both to the changing understanding and to shifting policy orientations over time.

For the moment we are having to operate within a policy environment that prizes the productive and technical. Furthermore, the discourse has become so totalizing that forms of education that do not have a curricula basis are squeezed. The temptation is always there to either be colonized by curriculum theory or adopt ways of describing practice that do not make sense in terms of the processes and commitments involved. Kleibart's analysis provides us with some hope - things will change. However, there is no guarantee that they will move in a more edifying direction.

Further reading and references

I have picked out some books that have the greatest utility for those concerned with informal education and lifelong learning.

Caffarella, R. S. (1994) Planning Programs for Adult Learners. A practical guide for educators, trainers and staff developers, San Francisco: Jossey-Bass. 248 pages. Just what the title says - but has the advantage of many manuals in this area in that the underlying model is dynamic and interactive and avoids some of the problems with linear planning models. Clearly written with plenty of worksheets etc.

Griffin, C. (1987) Curriculum Theory in Adult and Lifelong Education, London: Croom Helm. 218 pages. Explores the use of curriculum theory and practice in non-school settings. Particular attention is paid to Illich, Freire, Gelpi etc.

Grundy, S. (1987) Curriculum: Product or Praxis, Lewes: Falmer. 209 + ix pages. Good discussion of the nature of curriculum theory and practice from a critical perspective. Grundy starts from Habermas' theorisation of knowledge and human interest and makes use of Aristotle to develop a models of curriculum around product, process and praxis.

Houle, C. O. (1972) The Design of Education, San Francisco: Jossey-Bass. 323 pages. Influential statement of theory and practice with regard to a fundamental structure for program design. Identifies basic situations (eleven in all) in which programs are planned and discusses their operation.

Kliebard, H. M. (1987) The Struggle for the American Curriculum 1893 - 1958, New York: Routledge. 300 + xvii pages. A cracker of a book which charts the development of different curricula traditions and the political and social context in which they arose. He unpicks suspect notions such as 'progressive education' and demonstrates how Dewey in particular is positioned outside the main competing traditions. The movement between mental discipline, child centredness, scientific curriculum making (Taylorism) and social meliorism provides a very helpful set of insights into the theory and process of curriculum making within adult education.

Knowles, M. S. (1980) The Modern Practice of Adult Education. From pedagogy to andragogy 2e, Englewood Cliffs, N. J.: Cambridge/Prentice Hall. 400 pages. Pretty much the standard US work on practical program design in the 1970s and 1980s. Based around Knowles' assumptions concerning the way adults learn with some leanings to behaviouralism. Part one explores the emerging role and technology of adult education; Part two organizing and administering comprehensive programs of adult education; and Part three reflects on helping adults learn. Extensive appendices provide various exhibits and additional models. See also Knowles (1950) Informal Adult Education. A guide for administrators, leaders and teachers, New York: Association Press (272 pages) for an early but still useful review of program design and implementation within an NGO (Chicago YMCA).

Langenbach, M. (1988) Curriculum Models in Adult Education, Malibar: Krieger. 228 pages. Argues that adult educators must have a sound understanding of program design. Reviews different models of curriculum theory and practice (largely US) and assesses some specific areas of practice such as continuing professional education and literacy education.

Ross, A. (2000) Curriculum: Construction and critique, London: Falmer Press. 187 + xiii pages. Helpful overview of the history of curriculum development in Britain

Stenhouse, L. (1975) An Introduction to Curriculum Research and Development, London: Heinemann. 248 + viii pages. Classic statement of a process approach to the theory and practice of curriculum making. Chapters explore the nature of the curriculum problem; the content of education; teaching; the school as an institution; behavioural objectives and curriculum development; a critique of the objectives model; the process model; evaluation; a research model of curriculum development; the teacher as researcher; and the school and innovation.

Thornton, S. J. and Flinders, D. J. (eds.) (1997) The Curriculum Studies Reader, London: Routledge. 416 pages. Excellent collection of 30 readings that provides both a sample of enduring work and more recent material around curriculum theory and practice. Includes: Bobbitt, Dewey, Counts, Kliebard, Eisner, Jackson, Schwab, Greene, Freire, McLaughlin, Ravitch, Glazer, Apple, Lieberman and more.

Tyler, R. W. (1949) Basic Principles of Curriculum and Instruction, Chicago: University of Chicago Press. 128 pages. Important discussion of product-oriented curriculum building. The process is clear from the chapter titles: what educational purposes should the school seek to attain? How can learning experiences be selected which are likely to be useful in attaining these objectives? How can learning experiences be organized for effective instruction? How can the effectiveness of learning experiences be evaluated? How a school or college staff may work on curriculum building.

Wragg, T. (1997) The Cubic Curriculum, London: Routledge. 120 + x pages. Put aside the naff tittle - this book provides an accessible model of cur riculum building that attempts to incorporate a 'vision of the future'; a recognition that there are escalating demands on citizens, a belief that (children's) learning must be inspired by several influences; and lastly that it is essential to see the curriculum as much more than a mere collection of subjects and syllabuses. Wragg's 'cubic curriculum' has three dimensions: subject matter; cross-curricular themes and issues that influence children's general development; and the different methods of teaching and learning that can be employed. The concern is to provide a model for practice - so the book is a bit lightweight with regard to competing conceptualizations of curriculum and alternatives to curriculum thinking.

References

Aristotle (1976) The Nicomachean Ethics ('Ethics'), Harmondsworth: Penguin.

Barnes, J. (1976) 'Introduction' to Aristotle The Nicomachean Ethics ('Ethics'), Harmondsworth: Penguin.

Barrow, R. (1984) Giving Teaching back to Teachers. A critical introduction to curriculum theory, Brighton: Wheatsheaf Books.

Blenkin, G. M. et al (1992) Change and the Curriculu,, London: Paul Chapman.

Bobbitt, F. (1918) The Curriculum, Boston: Houghton Mifflin

Bobbitt, F. (1928) How to Make a Curriculum, Boston: Houghton Mifflin

Carr, W. & Kemmis, S. (1986) Becoming Critical. Education, knowledge and action research, Lewes: Falmer Press

Cornbleth, C. (1990) Curriculum in Context, Basingstoke: Falmer Press.

Curzon, L. B. (1985) Teaching in Further Education. An outline of principles and practice 3e, London: Cassell.

Dewey, J. (1902) The Child and the Curriculum, Chicago: University of Chicago Press.

Dewey, J. (1938) Experience and Education, New York: Macmillan.

Eisner, E. W. (1985) The Art of Educational Evaluation, Lewes: Falmer Press.

Foreman, A. (1990) 'Personality and curriculum' in T. Jeffs. & M. Smith (eds.) (1990) Using Informal Education. An alternative to casework, teaching and control? Milton Keynes: Open University Press. Also in the archives.

Freire, P. (1972) Pedagogy of the Oppressed, Harmondsworth: Penguin.

Grundy, S. (1987) Curriculum: product or praxis? Lewes: Falmer Press.

Jackson, P. W. (1968) Life in Classrooms, New York: Holt, Rinehart & Winston.

Jeffs, T. & Smith, M. (eds.) (1990) Using Informal Education. An alternative to casework, teaching and control? Milton Keynes: Open University Press.

Jeffs, T. J. and Smith, M. K. (1999) Informal Education. Conversation, democracy and learning, Ticknall: Education Now.

Kelly, A. V. (1983; 1999) The Curriculum. Theory and practice 4e, London: Paul Chapman.

Stenhouse, L. (1975) An introduction to Curriculum Research and Development, London: Heineman.

Newman, E. & G. Ingram (1989) The Youth Work Curriculum, London: Further Education Unit (FEU).

Taba, H. (1962) Curriculum Development: Theory and practice, New York: Harcourt Brace and World.

Tyler, R. W. (1949) Basic Principles of Curriculum and Instruction, Chicago: University of Chicago Press.

Usher, R. & I. Bryant (1989) Adult Education as Theory, Practice and Research. The captive triangle, London: Routledge.

Links

Acknowledgements: Picture: rubber bands by eek the cat. Sourced from Flickr and reproduced here under a Creative Commons Attribution-NoDerivs 2.0 Generic (CC BY-ND 2.0) Licence. http://www.flickr.com/photos/eek/76924263

How to cite this article: Smith, M. K. (1996, 2000) 'Curriculum theory and practice' The encyclopedia of pedagogy and informal education, www.infed.org/biblio/b-curric.htm.

© Mark K. Smith 1996, 2000

Last Updated on June 4, 2018 by infed.org

# data[].attributes.attachment_link.title
What is curriculum? Exploring theory and practice

# data[].attributes.attachment_link.description
Contents: introduction · curriculum as transmission · curriculum as product · curriculum as process · curriculum as praxis · curriculum and context · curriculum and informal education · further reading · links · how to cite this article The idea of curriculum is hardly new - but the way we understand and theorize it has altered over the years - and there remains considerable dispute as to meaning.

# data[].attributes.attachment_link.title
OECD iLibrary | Educational Research and Innovation

# data[].attributes.attachment_link.description
OECD's dissemination platform for all published content - books, serials and statistics