My Learning Log by Benjamin Reed https://padlet.com/reedb21/twzn4urzd2y0 Made for the AEDT 5770 Course en-us 2017-03-28 00:17:03 UTC 2017-06-05 00:40:45 UTC hello@padlet.com https://padlet-assets.s3.amazonaws.com/icons/Manbag.png Week 1 Readings reedb21 https://padlet.com/reedb21/twzn4urzd2y0/wish/164224158 The first readings for this week were a good framework and refresher for approaching evaluation work. I have had the opportunity to do some of this work with the University of Washington as well as Seattle University and a lot of what this chapter talked about hit close to home. There have been lots of times that the evaluation of programs has felt tacked on at the end, and often have not been given the time and thought necessary. While the first thing most people point to is how little extra time educators have in their schedule, I firmly believe the fault lies higher in the chain of command. There is no incentive given at most colleges for completing evaluations, and often no requirement to do so except for a few programs.

What also strikes me is how little formal training most educators receive in this work. I have never encountered the kinds of evaluation listed in this chapter in my work, and I think they would have been incredibly useful to know. There are lots of things people hope to do with evaluations of their programs, and at the end of the day I think most people want to use evaluation to prove their work is successful, and are not thinking about using it to get an accurate picture of the impact of their work. So, in crafting evaluations there are not the right questions asked because of these intentions. I am looking forward into digging into these concepts further!]]> 2017-04-03 03:35:09 UTC https://padlet.com/reedb21/twzn4urzd2y0/wish/164224158 Module 2 reedb21 https://padlet.com/reedb21/twzn4urzd2y0/wish/165564066 So I realized that I focused on chapter 1 in the last post, and so this week I will be focusing on chapters 2 and 3, along with the supplemental readings and the discussion from this week. I would also like to take the opportunity here to mention that I did not do well with the frequency I was engaging with this course. I typically have to get used to online course posting schedules, since this is only my third online course. Next module is a new module!

The Leach, Neutze, and Zepke article from this week was a really excellent read, and I think your video lecture highlighted it well. One of the things the Student Development Administration program talks about often is critical theory as applied to education, and I think this article highlighted some of the big inequities. One of the the things that stood out to me was how standardized exams only work for a few learners. And this absolutely applies to adult learners, as certifications often require written exams, which can be a barrier to job access for many adults. 

I also really appreciated the responses to my question about the readings in the module discussion. It was a really useful perspective in challenging some of my own biases. Both responses focused on peer evaluations and the challenges around knowing your input would impact others. When I initially asked the question, I was not expecting this to be the focus, and while I found the answers surprising, the reasoning Shelbe and Maria gave was not super surprising. Anyway, I found it very thought provoking and it pushed me to think more critically about what evaluation methods are dis-empowering. ]]> 2017-04-09 17:25:19 UTC https://padlet.com/reedb21/twzn4urzd2y0/wish/165564066 Module 3 reedb21 https://padlet.com/reedb21/twzn4urzd2y0/wish/167701851             The reading for this week set me off feeling a little off-kilter. It felt technical to a certain degree, and it was approaching evaluation in a way that I hadn’t before. I mentioned this in the group postings, but over-all I was feeling a bit over my head. It was really nice to hear that some of my classmates were feeling the same way, if nothing else as it relates to the way the text was written. This was also one of the times that I found using Kindle ebooks to be a real limitation. For some reason the proposals in the readings for this week were formatted differently, and did not fit on the page well. I was able to make sense of it the second time through, but the first read through I felt like I was fighting it. I know I chose to use the Kindle instead of a traditional text book, but it is much easier to carry a tablet with me as a commute as opposed to a text book.
            Thankfully, the assignment really helped me clear up the readings for this week, and helped it feel much more grounded. As Heather and I got to talking about the assignment and reading through the text to help us answer questions about the text, I felt like the text was much more understandable. The assignments as they relate to the readings are very useful in this way, and will help me a lot in my work. At DigiPen there are some odd ways that the “business” side creeps into the educational work we do, and having the knowledge to write these proposals will help a lot in meeting these business needs. Unfortunately, this week was another busy week at work, which made some of my homework slightly stressful to get done. I have been promised that things will be lightening up soon, and I look forward to being able to spend some more time on coursework!
]]> 2017-04-23 18:49:01 UTC https://padlet.com/reedb21/twzn4urzd2y0/wish/167701851 Module 4 reedb21 https://padlet.com/reedb21/twzn4urzd2y0/wish/167706132 Before I type anything else up, I would like to extend an appology for not being very responsive or available this week. I was really quite ill for the first part of the week. I had a draft of my initial post sitting on my desktop for days, but between the fever and headache it was difficult to look at a computer screen long enough to edit it. But now I am on anti-botics, and feeling much better!

I really got a lot out of the readings this week, despite not being able to engage in the conversation too heavily. When I was working at UW Seattle one of the tings we talked about a lot as I was helping them with their evaluation model, was the different types of evaluation. It was difficult for a lot of the people I was working with the break free from the idea that all assessments had to be Likert-scale based post tests. I wish I had this chapter to point people towards at the time, because it might have given me the language necessary to break them of this! Having the widely accepted evaluation techniques talked about here couple could have really opened some doors to better evaluation in that office.

As with the previous week, the supplemental reading also gave me a lot to think about. I have often argued that in evaluating a program I get the most useful information from any opportunity for free-response. Coding free-responses can be incredibly time consuming, and honestly I understand why that is a barrier for people. This adjective-based evaluation technique though seems like it does a wonderful jobs of giving people a greater degree of variation in how they respond to a question, while structuring it enough to allow for easier processing of information.]]> 2017-04-23 19:55:12 UTC https://padlet.com/reedb21/twzn4urzd2y0/wish/167706132 Module 5 reedb21 https://padlet.com/reedb21/twzn4urzd2y0/wish/169081831 Can you believe we are already half-way through the quarter? Because I really cannot. This week's readings felt like some great forward momentum, because we are starting to delve into topics with some detail, and I have really appreciated it. Especially this week, since focus groups and interviews are not something that I have had to opportunity to facilitate.

Chapter seven felt a lot like the jumping off point for the rest of the textbook. I was kind of taken-aback by how many of the evaluation techniques were some form of test. While I certainly recognize that evaluating for skills or knowledge is relatively common and important, the fixed nature of how the authors think to do this is striking. There has been a lot of bias shown in tests, from IQ tests to the SATs, and so having theme feature so heavily in the text gave me pause. Creating a bias free test is extremely difficult, if not impossible, so if there are other ways to assess a person's learning, I feel like they ought to be explored.

Chapter seven on the other hand felt like a how to guide for focus groups and interviews, and it was great. Over all I found the advice to be useful, and it all felt pertinent to the technique being discussed. The only thing that gave me pause was how much the authors focused on compensating people for their time. While I agree that compensation makes it easier to secure participants, it can also chip away at the validity of your evaluation in the eyes of some people. I have seen people question results because they feared those results were "bought" instead of discovered through the analysis of data. It may sound nit-picky, but it seems to me that the more one can do to ensure others buy into their evaluation, the better.]]> 2017-04-30 21:30:34 UTC https://padlet.com/reedb21/twzn4urzd2y0/wish/169081831 Module 6 reedb21 https://padlet.com/reedb21/twzn4urzd2y0/wish/170415715 The reading for this week were really fantastic, and I think the discussions around them was also great. People had lots of anecdotes to share, especially around surveys and I think it was really great. I also found the more in-depth discussion around archival data to be really useful. 

What felt most useful this week however was the observation chapter. We tried to develop an observation instrument for our project, and this chapter was really helpful in the revisions. We also found that the revision notes we received from our experts to be really useful, even if just to note some of the obvious things we should have caught. Unfortunately, there was a mix up with the uploads, and Heather's review of another group's project was what the people who reviewed our project saw. Unfortunately we did not realize this until Sunday, the day we had set aside to polish this work. So, we made as many edits with the feedback we had received. It was a shame to miss the bulk of the feedback however.

This is a classic case of technical issues, and we do feel bad about it. As my discussion group has highlighted, there is a lot of expertise in this class, and having lost that opportunity for their ideas is kind of heartbreaking. We will be making the best of it though!]]> 2017-05-08 02:58:50 UTC https://padlet.com/reedb21/twzn4urzd2y0/wish/170415715 Module 7 reedb21 https://padlet.com/reedb21/twzn4urzd2y0/wish/171732865 Dr. Robbins, I am sorry that my ability to engage as frequently as I would like with this course has been so spotty. This quarter proved to be a perfect storm that kept me very busy. This week was Resident Assistant training, which I was in charge of running. All in all I was in the office and on campus for 75 hours this week, and then on top of that we moved in our summer international students Saturday, and today was their orientation, both full work days. Given the relative lack of time to spend on projects outside of work, I chose to prioritize the work on Assignment 7 over the discussion groups. This isn't meant to be an excuse, just an explanation of what happened this week.

Okay! Jumping into actual course material! I enjoy data analysis quite a bit, and so this week's readings were a lot of fun. Statistics is fascinating, especially in the way that they can be manipulated to show slightly different results with the same number. Sure, best practices and honest reporting of results means someone should explain their results thoroughly, but just the fact the same numbers can be used to show different ideas is interesting. Lies, damned lies, and statistics! ]]> 2017-05-15 02:38:29 UTC https://padlet.com/reedb21/twzn4urzd2y0/wish/171732865 Module 8 reedb21 https://padlet.com/reedb21/twzn4urzd2y0/wish/173053324 I mentioned this in my discussion group post, but this week's readings and assignments would have really come in handy when I was doing my assessment work at the UW. What I found most intriguing was the authors' acknowledgement that different stakeholders need to be communicated with differently. Even if those stakeholders are just individuals in the office, communication is never one-size fits all. Some in the office may find face-to-face chats crucial to their understanding of an evaluation process, while others are likely to find it tedious. I think one thing we haven't really talked about in detail yet in the course is how some people may view evaluation as a threat, and therefore may need to be communicated with differently.

I also think there were some parts of this book that show their age, especially in this setting. Lots of schools and businesses are integrating evaluation results into their websites to highlight triumphs, or even acknowledge areas of growth public to make the general public aware of efforts to improve. Lots of companies do this around sustainability initiatives as an example. While not every evaluation needs to be placed on a website, I think an acknowledgement of this new avenue of reporting would be useful. Come to think of it, I don't remember reading anything about the difficulties of surveys as feedback mechanisms as they are distributed online. The internet brings a lot of positives with it; ease of distribution, low cost, and quick processing times. However, it also makes the surveys more vulnerable to fraudulent entries, hacking, or manipulation. These drawbacks are preventable, but I think studying how best to prevent them would have been useful. The internet is here to stay, and it is featuring in schools, training programs, and everyday life more and more. I would love to learn more about this, Dr. Robbins, if you have any good resources!]]> 2017-05-22 04:29:46 UTC https://padlet.com/reedb21/twzn4urzd2y0/wish/173053324 Module 9 reedb21 https://padlet.com/reedb21/twzn4urzd2y0/wish/174151373 The readings and discussions around this week felt really topical. Unlike last week, where I felt there was lots of information not covered, this time I think the advice in this chapter is timeless! Managing the time and budget of evaluations will always be important. In my experience budgeting for evaluations isn't done well in many departments, and therefore it isn't well supported. Dr. Robbins, this is something we talked about a while ago, that without the money institutions of higher education won't do assessment well until there is real money tied to it. For a while I worked at the University of Puget Sound and they have an institutional research office whose whole role is to assist with evaluation. I actually had a chance to chat with Kate Cohn, who works in that office, recently about their work. 

She mentioned that their budget is growing year over year, because outside stakeholders are demanding more accountability. We talked about how this may be what actually drives colleges to begin taking evaluation more seriously. Donors want to know their money is being well spent, and the ever shrinking state funding, schools will be relying more on donors for their funding. With the value of higher education being thrown into question due to rising student debt, more and more schools will have to justify their costs and work. While it may be frustrating to some, I think it is an understandable change. Student affairs especially will be subject to increased scrutiny, and so many offices will have to start funding their assessment initiatives. ]]> 2017-05-28 18:37:18 UTC https://padlet.com/reedb21/twzn4urzd2y0/wish/174151373 Module 10 reedb21 https://padlet.com/reedb21/twzn4urzd2y0/wish/175144401 The final week! I cannot believe how quickly this quarter has gone, and how much we have covered in this course. It has been quite a ride, and I really wanted to thank you for a wonderful class. I have taken several online courses, and this was far-and-away the best course I've taken. So, honestly, from the bottom of my heart, thank you! 

On to actual course learning! I think ending with the ethics of evaluation is not a bad way to end the course, and I was very glad to see it included. There have been several times when I have seen people use the data from evaluations to manipulate the parties the information is reported to. This is frustrating, even when these people are manipulating results foe apparently good ends. History shows us, time and time again, how ignoring what people are actually experiencing only causes more problems. So an eye towards ethical evaluation is always for the best!]]> 2017-06-05 00:15:16 UTC https://padlet.com/reedb21/twzn4urzd2y0/wish/175144401