# data[].attributes.body
Well we have finally come to the end of the course and it has been interesting to look back on the padlet entries and see where the journey has taken us. Returning to the first entry, I feel that I now have a solid theoretical foundation in evaluation; I feel more comfortable in the subject as a whole and feel as if I would be able to converse knowledgeably about how to develop an evaluation question or discuss the different data collection methods for example. However, as Christina and I both expressed in the discussion board this week, I feel that the text book didn’t place enough emphasis on how you would plan and carry out an evaluation as an internal evaluator and on a small scale.

The book did do a great job though of reminding us each chapter that our focus was on the stakeholder and the information they were seeking, to be able to change and develop programs. I found that helpful as it is a centralizing force and I found that it gave us the basis for creating our philosophy last week.

As we have submitted our final portfolio this week, I was proud of what we have achieved. I think we have covered and considered the many different aspects that the book describes and it has been beneficial to our learning to work on a different area each week to build the portfolio together. The peer reviews for us worked much better this week and I felt that I did better too, which was good!

In conclusion, although I still have lots of questions and would be very, very nervous about undertaking an evaluation to this level of detail on my own, I do feel now that I would be able to participate as a novice in a team. I can’t say I particularly enjoy it, but can definitely see the benefits of it and would be a willing, engaged and committed novice team member!

Thank you for all your help and advice along the way! It is much appreciated!

# data[].attributes.body
This week we have been reflecting on and changing our assignment in light of the information we have received throughout the course. As a result of making these changes, it has been interesting to see the growth that we have made; for example, we really struggled to write our evaluation questions back in Assignment 4 but during the revision, they came together so much more naturally.

We were a little daunted by the group philosophy initially as this felt a little ‘out of the blue’ but as we discussed it, we realized that we had had a group philosophy even though we hadn’t articulated it at the time. We realized that we had been focused on answering the questions that were practical and relevant to the stakeholders in a way that took into account the slightly more unique needs of our participants, which made the basis for our philosophy. However, we were surprised to see this included in our final portfolio and wonder whether we could at least start to think about it earlier on in the course in the future, rather than it suddenly appear at the end.

This week’s peer review worked better for us too in that the reviewer did an excellent job and gave us some clear and useful guidelines in which to improve our work; I felt that I did a better job too although I still find it a difficult balance between highlighting areas of improvement and whether I should make suggestions how to improve. One of my biggest concerns with it is that I worry that I haven’t understood what the writer was intending to say and so end up confusing them and me all at the same time. I don’t feel that within our group however, mainly because we have been working so closely together that we know what we were intending as a group and I know them well enough to text them too which helps enormously.

Looking forward to this last week!

# data[].attributes.body
I chose to look at the Contents of Communication and Reports for this week’s discussion as I was particularly struck by this comment in our text book: “complex [or] confusing… reports can result in a lack of understanding at the very least and at the worst, [result in] a misinterpretation of findings” (Russ-Eft & Preskill, 2009, p414). All throughout this course the textbook has been insistent that we focus on the stakeholders and that the evaluation is practical and relevant. This quote reminds us of this again, ensuring that the findings and analysis is presented in a way that the stakeholders can hear.

It sounds odd to admit to but I hadn’t thought of this, I had assumed that the evaluators would choose the method of communication and reporting, but it makes perfect sense and fits with the ethos of the book. Whilst the book has had limitations, I do feel that this emphasis has been its strength.

Finally, I realize that next week we are looking at ethics which I am interested to dig into as I feel that this is an as-yet unexplored but important factor of evaluation.

# data[].attributes.body
A significant part of my learning this week came not as a result of reading the text book or through the discussion board but with a dear friend as we walked around parts of Lake Washington on Thursday morning! This friend has a PhD and previously worked as a systems analyst for the MOD so she was a great person to discuss the differences between the various measures of central tendency and the challenges of knowing which ones to use.

Our discussion was triggered by our evaluation team spending quite some time early on Thursday morning debating whether we should use the mean or mode to analyze our data and whether / how we should convert our word descriptions to a number for evaluation. In the end we decided that the best (ie most accurate) representation of our data would be through using the mode but after having walked and talked about it for a couple of miles I am now in a good position to describe some of the challenges of using all three measures of central tendency in an evaluation! She was also the one to give me the book “How to lie with Statistics” that I referenced in my discussion this week.

I felt that I was only an average participant in the discussion board this week; this weekend is extremely busy for me in other areas of my life and so I have had to prioritize a bit. I have focused on writing my section for Assignment 7 as I don’t want to let my team down; we continue to work well together and are planning on studying together next term for the Comp exam in November, which is actually a relief to have organized already.

# data[].attributes.body
In an ice breaker this week I was asked to describe my week in weather terms; my description was “generally warm and sunny but with a sudden tornado mid-week”. Fortunately, although the tornado produced confusion in the early stages, it was soon evident that there was no long term damage for which I was thankful.

My description came after the email announcement this week that the program is not taking in new students this Fall. Whilst I appreciated the consideration that had been taken to ensure that current students would be able to complete the course, the initial email was not super clear and left me feeling very confused and upset; I had been excited with my own personal plan to complete the course and initially I was unsure whether this could now happen. Thankfully this was soon resolved but it did ‘upend’ me!! Thank you for your help in resolving it!

We continue to work well together in our team but I confess to feeling rather frustrated with some of the peer reviews. A couple of them were detailed and useful but it was evident that a couple of them hadn’t read the survey questions clearly or asked questions about content that is covered in depth in the earlier assignments, so made some unhelpful and irrelevant comments as a result. Furthermore, I realized that I had misunderstood and had only reviewed the survey and not the introduction of the assignments that I had been sent, which I felt bad about. Consequently, I think that the peer reviews for this week scored a meagre 3/10 for usefulness both for me and from me. Fortunately, our external reviewers have been much more helpful and have definitely enriched our survey as a result.

Finally, on a practical level, I had a very beneficial conversation with a parent attending my class this week which really highlighted the benefits of using UDL where possible. This Mum is an ESL speaker who attends with her husband and I was asking her how I could meet her needs better. She is able to understand most of what is being said and her husband will translate if he thinks she needs further clarification but we talked about making sure that she feels confident to ask me to stop and allow him to do this if she needs to. She appreciates the digital copies that I send to everyone each week and although I asked if she would like me to translate them into Korean she (and her husband) said that she has a high reading comprehension level and so having a physical copy in English in class is helpful to her too. In our evaluation, we have decided not to focus on qualitative data – and this short conversation clearly does not have the depth of a true evaluation – but it was helpful to get some understanding of things from her perspective none the less.

# data[].attributes.body
It was interesting this week to have a bit of cross over from the Coaching class to this one. The article by Leong and Huang (2008) that you gave us as supplemental reading for the Coaching class, although dense in Psychologist language, had a lot of good implications for our discussion in gathering data this week. I particularly felt that their emphasis on not making assumptions and really working to understand what the participants felt were the issues was very applicable as well as just raising an awareness of the challenges that working in a more global society brings.

One thing that I feel has come out of this week in particular is that there are so many different things to consider when deciding on the data collection method that it is easy to become a bit frozen and find it hard to make a decision. Knowing what the stakeholders’ value and the question that they are trying to answer certainly helps but there are still tough decisions to be made. Although our group seriously considered using Focus Groups to gather our data, we eventually went for a survey as we felt that this was the best fit but I acknowledge that some of the information that we could have gathered could be lost especially from our ESL parents.

I found that I challenged myself in my participation this week as I (hopefully gently) challenged one of my fellow students to explain and clarify their use of language. The person used this sentence: “When it comes to data collection, I find it to be both beneficial, but also intimidating for the interviewee and may not even get correct results.” and it was the word ‘correct’ that I found to be ambiguous and potentially limiting. Often I would ignore it and move on but I felt brave enough to challenge the use of the word which resulted in a deeper discussion and better understanding of the point they were making. They were making a great point but I felt that it did need clarification so I was glad that I had had the courage to push back a bit and ask for further information.

Leong, F. T., & Huang, J. L. (2008). Applying the cultural accommodation model to diversity consulting in organizations. Consulting Psychology Journal: Practice and Research, 60(2), 170.

# data[].attributes.body
I really appreciated the Blunt article this week; it made a lot of sense and was a good supplement to what we have been reading in our text book. I also appreciated your comments on my post about working with someone who complements your skills to create a more thorough and holistic evaluation. As I mentioned, I too have always leaned toward using a qualitative approach to research or evaluation both because I find meaning in words and value people’s comments but also because statistics are not my strong point and I would get lost in the numbers!  However, I do see value in the large statistical research too. John Gottman, who I think has one of the most powerful and wise voices on the relationship between parents after their children are born is a statistician and I value his work because he looks at thousands of couples and not just sees one or two. Similarly, there was an article this week on the low numbers of people participating in sleep studies of babies and small children and how these statistics, and therefore recommendations were skewed because of this.

I am finding that I have mixed feelings about the textbook though; on the one hand, I think it does a great job of explaining the theory about the work of evaluation but I am finding it difficult to see how you can apply it to real life unless you are actually working as an evaluator. I am struggling to see how you could apply this to real life situations, given the limitations of resources, in particular time, money and skill.

Finally, I found the concepts that we were working on in Assignment 5 challenging as I felt that we hadn’t covered the material in the book and wasn’t sure if I had missed something? I was grateful for my team and for the examples you gave us as this helped enormously.

# data[].attributes.body
This week our Team work to create an evaluation plan really got started in earnest. I am so enjoying working with these ladies – like many people, I have had good and bad experiences with working in teams both in work and in school but this is a good example of how working in a team can be a positive experience that increases learning and reduces stress. We met via zoom on Tuesday to thrash out our plan and then have been communicating well via google docs or text to complete our project and I think it is coming together well.

In relation to our weekly discussion, I feel as if I have participated well and have given some thoughtful comments to my peers; however, I feel as if I have gone down a rabbit warren a bit with my discussion with Jason. He asked a really good question about whether the pairing of mentor to mentee should be evaluated and on the face of it, it seems a really easy thing to say “yes” as a successful pairing is essential to a positive result; but then I started to think and wonder whether investigating this would take resources away from evaluating the actual result and I have been tying myself up in knots ever since! What do you think? I think I have concluded that in an ideal world, we should examine the mentor / mentee relationship but that if resources were limited (as they often are in the real world) then this would not be my priority.

I also really appreciated the insights of others in the group who suggested a more qualitative approach to the evaluation and who highlighted the need to consider culture and race when creating good mentor / mentee pairs. The book does a good job of presenting the differences but equal value of qualitative and quantitative methods of evaluation but I tend to think of companies wanting evaluations to be more quantitative or mixed methods rather than just qualitative so it is good to be reminded that this is not always the case.

Looking forward to week 4!

# data[].attributes.body
As many of us stated, there was a lot to digest in the readings last week, but it was interesting to see how we each applied the reading and what stood out to us personally. The Leach article certainly challenged assumptions and I am still processing how it would work in practice. On the one hand, I can see how it can be applied successfully in a setting like ours and how concepts such as UDL could both be a support to the ideas that Leach et al were proposing as well as being supported by these ideas. However, I do think that it needs to be stated somewhere that not all learning can be self-assessed and that there is a need for external assessment in some areas – driving a car or performing a medical procedure come to mind.

I also think that it must be quite difficult to do with students who have always received a more traditional view of assessment. If students were taught how to self-assess from an early age, it would be a skill that they would grow into and become more proficient in as they progressed through their education, which could result in some very meaningful learning as they reached adulthood. For myself, this week, I felt that I participated in the discussions about as well as I usually do which I was glad about as I was on vacation with the family and wasn’t sure how easy it was going to be to balance it. One thing that I find helpful though is to tell myself not to expect to read everyone’s post and to make a selection whose I choose to read. I keep a careful note of who I have read and commented on so that I alternate through the class over the quarter. This approach has its drawbacks but I do find that I respond more thoughtfully and more deeply if I do it this way.

This week’s class discussion for me also demonstrated how learning and meaning can be made through discussion with classmates. Tamar highlighted the limitations that can result through evaluation that has a clear end point in mind which totally made me rethink the “goal-free” evaluation process that Russ-Efft and Preskill outlined. Without her comments, I wouldn’t have gone back and re-evaluated it and whilst I can still see drawbacks to this approach, I can also see how this can benefit organizations.

Finally, I was surprised and relieved to see many of my fellow students expressing solidarity with my own problem of a terrible response to an online survey. I was grateful to learn that it wasn’t just me but at the same time, find it amazing that in this world where so much is done online that we have found an area that really doesn’t work so well online. This makes it particularly hard for courses such as this which is done completely online of course. I had already thought that, for the course that starts on Friday, I would revert back to having a paper evaluation at the end to try and make sure that I get responses from my clients and the experiences of my fellow classmates reaffirmed this decision.

# data[].attributes.body
It has been good to see so many familiar people in this class and I am excited to learn from them again. One of the benefits of working with the same group of people over many classes is that there is a deeper level of trust and therefore deeper learning. I know that my fellow students will challenge me and expand my understanding of the subject during this course and I am looking forward to seeing and reflecting on that progression through these learning logs.

I am delighted to see that many of the students in this class are completing their studies this quarter but I have to confess to being a bit envious as I have another year! Whilst taking 6 classes rather than 8 a year is the right decision for me and my family, next June feels like a long time away!!

I heard several people express concern about taking this class online, which made me reflect on a conversation that I had with a friend of mine last week about the strengths and weaknesses of taking an online course. I have found that the online classes I have taken at SU have given me a deep and rich learning experience. One explanation for this, I think, is because we return to the material several times a week rather than attending a class once a week which means that the material is digested in smaller chunks, with time for reflection and evaluation in between. This means that the subject matter is developed more slowly perhaps but that it leaves us with a deeper understanding. To an extent, this was suggested in the Stober and Grant (2006) book on Coaching this week too.

There are clearly drawbacks to taking classes online and the time it takes to type a good discussion is definitely one of them! I keep challenging myself to record my responses to the  discussions more and so I’ll see how this course goes! But despite this, I would strongly argue in favor of online classes for graduate students (although not for undergraduate students).

My concerns for this course remain about the process of evaluation. I understand the greater idea behind it but implementing it practically still concerns me. One issue that I hope that we tackle is how we encourage people to value, and therefore engage with, the evaluation process in order to get a full and honest understanding of the program that is being evaluated. I am looking forward to taking this class with you. Thank you!