# data[].attributes.body
Weaknesses:
- A lot of evaluation questions which would potentially be confused and too broad for the purposes of one evaluation
- No references/bibliography etc.

Strengths:
- Clear depiction of the program overview and the models were put forward well.
- Identified the key issues and the existing approach
- Data collection methods

Natalia Notkina

# data[].attributes.body
Strengths:
Well organised and structured.

Weaknesses:
More than 7 evaluation questions.

# data[].attributes.body
Strengths:
- clarity and thoroughness in explanation of the logical model and theory of change.
- effective use of visual diagrams, flowcharts and tables to explain processes
- clear structure and detailed description of the stakeholders

Weaknesses:
- opportunity for more narrow scope in the evaluation questions - quite a lot, not necessarily specific
- No references

Michael

# data[].attributes.body
Strength
- Clearly describe the evaluand
- Key issues and the approach of the evaluation are clearly acknowledged

- Theory of change are briefly explained following the program logic

Weaknesses
- Evaluation questions are already aligned with the criteria for evaluation and the data source has been planned, but some questions have multiple purpose and could be done by other data collection and analysis methods
- The familiarity and relevance of professional and/or research literature were not supported

Diah

# data[].attributes.body
Strengths:
- Clear program overview provided, as well as appropriate logic a theory models.
- Questions have been identified and linked to criteria
- Data collection approach clearly identified and well set out.
Weaknesses:
- As previously stated, many evaluation questions, which address multiple purposes.
- Data collection methods identified but analysis is brief at times
- Ethical issues only briefly addresses
- Broad scope of study realistic?

Rebecca

# data[].attributes.body
- Length of the evaluation period - from November 19 - Feb 21. A long enough period to collect sufficient data.
- Two reports which served different purposes
- appears to have sufficient budget
- identifies the limitations and potential bias of the involvement of the internal person

Weaknesses
- the data collected from Moodle will be difficult to critically analyse
- some questions are vague, for example what is meant by satisfied?

Bernadette

# data[].attributes.body
Strengths
- Clearly identified models - theory of change and program logic
- Evaluand was clear
- Clearly laid out and easy to follow
- Collection methods were clearly and supported

Weaknesses
- Too many evaluation questions which was confusing at times
- No inclusion of references
- Seemed a little unbalanced without acknowledging the limitations
Leigh

# data[].attributes.body
Strengths:
- Identified the logic and theory model as the way of assessing.
- Gave a detailed description of the evaluand.
- Identified the key issues of the existing approach.
- Used headings to differentiate information.

Weakness:
- No references.
- The ethical concerns are not measured as much. more detail could have been given.
- A lot of of evaluation questions. would be better to narrow it down to make the evaluation more detailed and focused.

Claudia

# data[].attributes.body
Strengths:
- Identified theory of change and program logic model. Flowcharts and figures have been included.
- Clearly stated evaluation questions.
-Clear data analysis and collection methods. Identifying and supporting their mixed method approach.

Weaknesses:
- The 80% participation rate for the surveys is a bit of a high expectation.
-As mentioned, there are no references?
- More discussion around ethical concerns or measures.
-Looking at the criteria, the data analysis component doesn't answer all of the questions, doesn't address how it would be managed and kept/disposed of.
- Limitations?
-Budget for the evaluation, mentions it only briefly.

Millie

# data[].attributes.body
Strengths of the exemplar:
- Evaluation questions are listed
- CRITERIA for measurement!!!! :-D
- Theory of Change and Program Logic Model figures/flowcharts
- Plan for dissemination and timeline is detailed enough to help time management of evaluation
- Plans for metaevaluation by an external reviewer from NESA
- Audience for the evaluation is articulated

Weaknesses:
- Where are the REFERENCES? %-O
- Introduction - I think this would have been better served with a general overview of the evaluation, introduce the evaluand and set out its primary goal
- Evaluation questions - there are so many! Each would require a construct to measure, this would mean a lot of data with many disparate  measures. It would have been better to subsume some questions or refine them for improved utility.
- Signposting of the evaluation's purpose. I'm not sure that this is effectively done.
- Greater depth of discussion in data analysis is needed.
- Ethical conduct measures are not specific and need clarification; needs to talk about informed consent, right to rescind participation etc