
Heading into Week 3 with my online GCTTL course, so my first assignments for each Subject are getting closer. I will spend my study time this week gathering material for them. I will upload the concept maps as soon as I do them ~:-)
I now have my readings sorted within a mango bright A4 binder, with recycled dividers (some are actually brand new but office staff at one of my jobs just throw them out!). The folder at this point has two main sections, one for each Subject for this semester.
My Study Schedules are at the beginning of each Subject, and glossary pages and or readings that aid in completing coursework. Each Subject has grey plastic dividers to separate weekly readings.
The readings are in alphabetical order, and the last page of each is folded to make a triangle (I could of used a paper clip at the top, to each their own), so that I can easily locate the beginning of each reading within a particular week.
Assignments 1 & 2 for each Subject have a bright coloured divider so that they stand out from the readings. This is where I can put the printed literature I collect, as well as notes from brainstorming, mind maps and concept maps etc.
Feeling quite organised ~:-) and consolidating the folder was something that I did whilst watching a DVD (as I need to do something when watching a movie, usually).
As the course notes for my Assessment and Evaluation Subject were put online last week I will reflect on them.
Mood:
Happy as a Ladybug; have spent the morning in the garden after some work with students.
- Definitional differences:
- Assessment
- Evaluation
- Measurement
- Testing
- Reporting
- How to conceptualise Quality Assessment?
My work experience background is heavily grounded in statistics~ though I still have heaps to learn! I have and do design questionnaires for NGOs and small businesses as well as undergrads and postgrads. I took this Subject because it is a topic I really enjoy and it is one in which I want to develop as a professional. Differentiating between the above concepts is not something I have taken the time to consider before. I like that this Subject will get me to slow down and to become more aware of what I don’t know.
- Assessment: Where am I/Others at prior to new knowledge?
- Evaluation: Where am I/Others at post exposure to new knowledge?
- Measurement: How will I/Others, Assess/Evaluate to answer what we want answered?
- Testing: Do I/Others understand what is needed to be known?
- Reporting: Summary of what is and isn’t known with recommendations, for I/Others.
Quality is a concept I was truly introduced to by Robert Pirisig; he writes about his reflections on the metaphysics of Quality. Love it. Very numinous as my friend Nix would say. When I was a TA I wrote up a Subject exam for the students in Developmental Psychology~ this consisted of choosing from set questions given to me in a folder. I was not impressed with the method.
As for individual assessment of students~ I was given no training for any of the classes I was a TA for, to grade a presentation or an essay etc. Neither were the other TAs. It was all very subjective, which drove me nuts as a psychologist~ I had just spent four years being trained in the ethics, reliability and validity of measurement~ to have it all ignored when I was “in the field”, training others to be examined on these topics! Talk about pointlessness.
For example, I had a marking schedule for the presentation~ but my determination for each section may have been widely different to that of TAs for the course in the past, the TA taking other classes for the same Subject, and I assume TAs in the future. Just crazy. And the students of course are so focused on the grade they get (in my School, this was encouraged), and yet that grade, was basically meaningless~ given the lack of standardisation across markers and marking schedules.
So, for me, a quality assessment for a university class would entail the same stringent and ethically bound criteria that would be used for developing a measurement tool for a research project:
- It would be based on relevant literature to provide a theoretical background
- It would be pilot tested for inter-rater (TA) reliability, construct, criterion and content validity etc
- TAs would be trained how to use it to achieve a standardised conceptualisation of what was being measured and how
- TAs would be evaluated on their use of the marking schedule and given feedback and opportunity for discussion on areas that they were seen as not fully understood
- The marking schedule would be evaluated periodically to ensure that it aligns with teaching and learning outcomes and relevant literature
In this way, students across time, year level, Subjects and tute classes could be assured that the margin for error had been greatly reduced. That their grade was actually meaningful~ not just based on the subjective assumptions of a TA.
- “good assessment” principles:
~~~~~~~~~- accurate, valid, fair, ethical
~~~~~~~~~- triangulated
~~~~~~~~~- analytical, formative, ongoing
~~~~~~~~~- criterion-referenced
~~~~~~~~~- support student development
These are not all the points listed in my handout, just the ones that I am particularly drawn to or that I have not considered before. My goal here is to learn new material, I already have ingrained in my mind that good assessment needs to be “Related to teaching and learning objectives”.
- Fair and accurate –> learner & teacher confidence + meaningful reporting
- Pre-determined, articulated criteria available to all (well before assessment)
I remember one exam as an undergrad, the Subject was run simultaneously in other city with another lecturer. There were a series of questions about material which we had not covered in my city! How unfair was that?! Myself and and another two students went to see our lecturer immediately after the exam. The lecturer was apologetic, and it was very clear they felt between a rock and a hard place~ it was mentioned that we were threatening Department funding with our complaint, that it was only a few question (at least 10!), and the body language of the lecturer said to me “feeling cornered”.
As by now I had a fair idea of “how things were done” within the School, I backed off (so then did the others cause as much as they “didn’t care” about the position the lecturer was in, they needed me as they certainly weren’t going to voice their disappointment to the lecturer’s face on their own). I had other exams to prep for and the time and political BS that I anticipated would be involved (based on experience and observation of others), were just not a priority for me at this time.
Also, I did care about the psycho-emotional state of this lecturer; the entire class was aware that some personal stuff had been going on for them. The lecturer was simply a cog in the wheel, not the one to be channeling anger at and unfortunately, not the type to take a stand even when aware of the unethical and unfair practices that existed (unless of course there were a bunch of others standing before and around them; again based on personal experience and observation).
So I made it a goal to get myself to a point where I could contribute to change of such an unfair and un-ethical system. And so, here I am…~:-)
The above example reveals the unreliability and invalidity of the exam we sat. It was unreliable because a similar set of participants (our colleagues taking the same Subject in another city) would get different results as they had been exposed to all the material being assessed, whilst we only partially.
Or maybe not, we would continue to not know about the topics we were not exposed to unless we went forth and made time to do our own research, though we would not be aware of what was “expected to get the point for that exam question”.
It was invalid as it did not measure our understanding of the material we had been exposed to, because some of it we had not been exposed to. Even if it were only one question about material we had not been exposed to it would not change my argument~ as the focus within the School was on “getting points” that would be one point that we would not have an equal chance of obtaining.
I’m not a Pac-woman collecting coloured discs (points), regardless of my grade I was aware of what I understood and did not understand. And being aware of my gaps let me know what to focus on during my holidays to be a competent and confident practitioner that could meet the needs and expectations of those who would seek my service.
Points, zoints as far as I was/am concerned.
OK, your turn…what are your experiences of accurate, valid, fair and ethical assessments? Of ones that were not? How would you define the terms above?