Webb-ing Learning & Media

This semester, I had the privilege of teaching a graduate course in Assessment, Evaluation and Criteria of quality for the second time. The learners were all art education MAT students. Early in the semester, we established a need on their part for a heavy integration of research skills into the course. From my perspective, this is a perfect match, since very similar skills are required for assessment and evaluation, as are required for pure research.

The only complaint I have about the course is that I find it very difficult to include both assessment and evaluation into only one course. On the assessment part, students expect to spend a lot of time discussing classroom assessment and developing skills in the classroom. It takes a lot of time and discussion to help students develop a vocabularly to discuss assessment in the terms that educational researchers and assessment-savvy educators use. Then suddenly, bam– let’s switch to program evaluation. Yes, in one sense it’s a micro to macro perspective, but I always feel like the evaluation portion is rushed.

The best part of the class is that it was very hands-on. Most of the students currently teach, so the project for formative assessment was for them to conduct action research whereby they thoughtfully integrated a formative assessment strategy into their classroom practice and reflected on the results and noted how it would impact their future practice. For the students who are not teachers, they did an inquiry into the assessment practices of someone who is currently teaching through interviews and observation.

Instructors should have aha moments too. My aha moment over the course of the semester and in tandem with my own work which is increasingly heavily assessment-oriented was that feedback is at the center of instructional practice. While some instructors understand this intuitively, others are not aware of how crucial it is. One of the students in the class, who teaches in an informal setting (thus making assessment that much more challenging) simply changed up the introduction to the lesson in a way that required the kids she was working with to reflect on what they were creating. Simply adding a question can radically change an entire lesson and the way someone else is thinking about their own work. That’s powerful.

Dr. Crazy’s post about difficulties teaching online is very insightful about the challenges of engaging students online.  Student motivation is key. Attrition in distance courses is a widely known issue and Dr. Crazy’s summary that students translate “online equals checked out” is right on the money. Or more like, online equals out of sight out of mind, and if I don’t check the course site, it doesn’t really exist.

But I am also curious about the course design. I’m not going to suggest that if students aren’t showing up and they’re not even doing the basic assignments required that the instructor can make miracles happen BUT I’m curious whether the course design builds an investment in each other on the part of the students.  I didn’t really see reference to that on the list of considerations in the post. Maybe if the course started out with peer interaction as a required assignment and the requirement to contact peers in the course was imperative? If students were required to post an email AND a phone number where classmates could contact them for this assignment? Have there been or could there be optional synchronous sessions for the instructor to just take questions from students who might be lost or not know where to start? In the future offering of the course, could there be a required synchronous session to start out the course?

These are just some suggestions, and it sounds like Dr. Crazy has been doing everything she can to motivate students. SO, maybe the biggest motivator would be an email sent out to floundering students that says: “As it stands at this moment, you have only completed x % of the required work to date. As of this moment, the highest grade you could expect to receive is a “?” C? F? Please contact me as soon as possible to discuss.” My guess is that some students have no idea they are failing the class to date and that might clarify things. But she did send emails saying they have 0s on assignments, one would think they could translate that to the big picture.

A faulty assumption might be that an email does the job, though I’m not suggesting every DL instructor can take the time to call students individually. It seems that at schools where students are part-time, their interest in logging into institutional email seems lacking. Again, we’re back at “what is the student’s responsibility.”

So maybe administrative support on the front end is required. Is there an orientation for distance students so they understand the type of dedication that is required to be successful online?

I want to thank Dr. Crazy for the candid account of the frustration of teaching online. I do believe the biggest responsibility is on the part of students to keep up with the work, and I completely agree that students assume that “online” will just fit into their life because it is flexible, without realizing that it generally takes more effort than f2f classes and a lot more self-direction. A lot of the responsibility to mitigate these incorrect assumptions lies at a program-level, where administrative types need to recognize the importance of this problem, and provide proactive, pre-emptive strategies to make sure the right students are signing up for distance for the right reasons.

If we’re worried that chacha will create cheaters, I’m worried that we’re asking students the wrong questions.  Or more to the point, let’s ask questions that require students to apply knowledge.

My post yesterday bordered on whining, but it felt good to express my frustration about students’ perception of the grading process. Though I dreaded the inevitable, “I worked so hard” that came after any kind of feedback, I don’t completely blame students. I blame the whole culture of “school.” Understanding the content at the heart of a subject is just the first step to becoming a master of an area, or even a competent contender.  And since most students I encountered thought the first resource that came up in a google search was the right answer, I am not too worried about the “cheating” aspect. The “cheating” part seems to be what’s happened to students who have learned throughout school that learning is all about “the right answer.” I would just ask, has the concept of what it means to “know” something shifted, and can’t we take that into account in the education process? If I can ask my friends and they can tell me the right answer, maybe the question wasn’t so great to begin with….

I understand why the call to create a “culture of assessment” has spread like wildfire in the title of workshops and in the rhetoric of the outcomes assessment movement. Just look here, and here, and here. I understand the concept is originating in the need for assessment in the process of considering the effectiveness of any assignments, courses, programs, and so on. But why the presumption that we don’t have an existing “culture of assessment?”

Isn’t what distinguishes the educational experience, or what makes it an educational experience is formal feedback and review from an expert/instructor and/or peers? And while I was drafting the beginnings of this lame and derivative discussion of what constitutes a culture of assessment, my brother emails me this idea:

I was thinking that it would be really neat to bring self-education and adult education into the 21st century a bit… to have a website which lays out an entire secondary and post-secondary education, delineated by courses, similar to high school and college work. So you have the syllabi, maybe videotaped lectures, problems, assignments, and tools for working through the material with others that have done it before in addition to volunteers that want to stick around and help with it. I think that the format for working through the material is something that will be evolving very rapidly, with people like you to blame for the innovations.

I think that its value would still be relatively small without some rigorous or at least semi-rigorous assessment. So, I’m thinking, you sit down and do a test… maybe it’s too hard to check if someone is cheating. That’s one problem to look at. But I think it might be possible to have others (multiple people) go through and correct/score the quiz/test/assignment. This is more straightforward, I think, with science/math, where it’s just a matter of getting the correct answers (though there’s the technical problem of how do you load it onto the website - do you require scanning it? faxing it in?). You have multiple people go through and score the test, and then you weight the scores depending on how reliable the scorer is. So the scorers themselves are scored.

Yes, the scorers would have to initially be scored by experts - perhaps real professors or students, either volunteer or working under a grant. The point of crowdsourcing the assessment is to allow the site to scale to serve millions without the need to add thousands or tens of thousands of paid experts.

First, I’m flattered he thinks I would be to blame for innovation. He’s even considered inter- and intra- rater reliability in the model. OpenCourseWare movement, I would like to introduce you to Crowdsourced assessment and then ask the question, why will college still cost $50k/year? Content + assessment will not be enough to create a holistic college-level education for the average user. What it’s missing behind it is the community of practice that accepts it as acceptable preparation. In other words, we still need a sponsor, a reputable one, to own the process and assure its rigor. But stepping back from the logistics of the whole process presented, and just on the assessment piece– why not some degree of crowdsourced assessment for the existing college classroom? We do it for screening for plagiarism (well, kind of, if we consider the artifacts of each student to be the “crowd”).

And it would tie in nicely with Downes’ Open Source Assessment model. State the problem or have students state the problem. Have a number of raters assess the success of the solution. People will definitely learn this way…but can they be credentialed (and should we care if they can’t ? )?

I am a sporadic blogger. I admit, and accept that. It’s not that I haven’t had relevant topics to discuss here, but every time I find a focus for my blog, my professional role seems to shift. And then it feels like I have to start all over again.

At any rate, I successfully moved from PA to MD. It’s all the same geographic region, but instead of suburban/rural, we’re right in Baltimore. I love it. I moved from Communications faculty back to Instructional Design with a heavy emphasis on assessment and faculty development. Now that I’m getting settled in and my focus is back on some big ideas, especially about assessment, learning versus schooling, program and curriculum development, distributed learning, and so on…

In my dream world, besides the world peace and solving of world hunger (and god knows we’re not even close to this), after these much more major issues, I dream of having a version of Adobe Captivate that works with the Mac OS. If that were to happen, I can’t think of a reason I’d ever have to touch a PC.

What’s happening to me?????

Tonight our Department debuted a Citizen Journalism website. Now that the technical structure is in place, I thought I’d comment on the design decisions that went into it.

First, back last summer, when this project was first introduced to me, we looked at a number of existing citizen journalism websites for some insight, such as Chi-Town Daily and MyMissourian. These sites are excellent examples of the genre. In the beginning, it wasn’t clear to me exactly what sort of content (e.g., what types of media) would need to be accommodated. And while to the end-user, the integration of audio, images, and video all look like they work together seamlessly, the back-end handling of these various media forms is not as easy. On many sites we looked at, the media seemed to drive the division of stories, so one would go to one section for video and another section for images. In this way, it seemed like the technology (the media) was driving the site organization rather than the content. In contrast, we wanted the story to drive the navigation, instead of it being driven by the media form.

Furthermore, although there are a number of open source platforms to choose from, when deciding on an appropriate platform, our criteria for selection became: 1) ease of use on the administrative side; 2) availability of plug-ins to handle images, audio, video; 3) and to a lesser extent availability of customizable templates. Based on these criteria, wordpress appeared to be the best option. Though it’s not traditionally a CMS, it is robust enough. We also considered drupal (but I hate the admin panel and the templates always seem to look drupal-y) and joomla (not an intuitive administrative panel).

That was the easy part. I will spare the details of the entire process, but the plug-ins that have been instrumental are:

• Vidavee:This plug-in allows for the uploading of video content, where the video is compressed and you can easily cut and paste code in the post that embeds the video for integrated playback. My only fear is that Vidavee’s terms of service will change.
  
• YAPB (Yet Another Photo Blog): This plug-in just literally saved the day because due to a server setting, the built-in resizing of images that comes with wordpress doesn’t work. I would recommend this plug-in anyway, because it nicely resizes images to a small image on the front of the site, a larger image within the full post, and an easily customizable sidebar in which to display thumbnails.
  
• Podpress: I swear this used to support audio upload, and it was a breeze. I might have imagined that. Nonetheless, after adding audio to the built-in wordpress media gallery, a user can cut and paste the url into Podpress and a snazzy audio player is embedded right into the post.

For a while I was also using the flickr rss plug-in. I really like that as well, but we didn’t want to add an additional layer of username and password for We-Town site users to have to go through to add pictures. Also, the group feature does not currently work with that plug-in. (It works if you are hard-coding the sidebar, but not when using widgets for some reason). I will also say that the YAPB plug-in is a little more flexible, and I like that we can change the width of photos as they appear. For someone already using flickr, this plug-in is excellent though. In fact, I use it on this site. For this same reason of not wanting to burden potential site users, we didn’t opt to demonstrate major overlap with YouTube either.

We are currently pulling together directions for using the site. I have started customizing the admin panel to reflect more specifically for a potential user what each section might be for. If podpress would just support uploading audio, it would make my life a lot easier as there would then be no need to point users to the media gallery.

Next, there are still a lot of process-oriented aspects to consider. I am also curious whether the “one image at a time” aspect is going to frustrate users. Certainly more sophisticated users can link to photobucket, flickr, or other photo sites, but then these images won’t show up in the sidebar. It will also be interesting to see how the categories across the top play out. They are designed to mirror the types of categories one might see in a newspaper. This is fine, but I wonder whether community users will see this as helpful. I can also start to imagine other features that would be good to add, including a way to add to a community calendar. The easier the better, I would think.

So I think we can fairly say that we’ve come to the end of Phase I - choosing the tool, customizing some key additional features, and choosing a look and feel (done by one of my Publication Graphics and Design classes). Phase II will be much more concentrated roll out of student-produced content and community events to help potential contributors get familiar with the process. We imagine student-designed workshops to do outreach to the community.

For the moment, I am in love. Due to some server issues, GD Utility was not working with Wordpress on a site I am working on for our department’s citizen journalism site. Thus, images were not resizing and life felt grim. Yet Another Photo Blog, YAPB has made my life easier within moments. Some additional customization will be necessary and widgetizing related photos for the sidebar is the future, but for the moment, I am elated that I don’t have to worry about how people will upload images and have them appear as reasonable sizes.

Steven Bell notes an article in the NY Times, The Professor as an Open Book. This article highlights one of the issues I have struggled with the most as an instructor at a small college that caters to traditional age students. My facebook profile, for example is not invisible but you have to be specifically searching for it to find it. Of course, since my students are facebook-addicted, it was inevitable that it was eventually found, but I told all my students that my policy was that I would not “friend” them until they graduate. I see them in a classroom several times a week, so it’s not like we need to develop rapport. It’s not an issue of being hip or accessible–there are just not enough levels of access in facebook. Someone is your “friend” or they are not, and although I don’t do anything of interest in facebook, I am actually linked to people I know. I know, I know, that’s SO old school. Even worse, many of my “friends” in facebook are family. Those are the people I have to worry about putting up suspect content that will reflect poorly on me, a la Doctorow’s analysis.

Actually facebook bores me these days, and there are plenty of other venues where I broadcast (this blog, another blog, twitter). They don’t require “friending” me to have access to my thoughts and interactions. I treat these as public representations of myself, but I have majorly cut down on how much of myself I put out there because of a new sense that I have a private identity to protect. Which is pretty funny if you think about it…I’m willing to say whatever if I know no one cares, but suddenly I am aware of a potential audience and I want to preserve my credibility with them…so I blog less.

But what is the responsibility of a teacher in this time of collapsed private versus public identity? How do other people navigate their online versus face-to-face identity with students?

The tenure system is a hot topic right now (or at least a few days ago, but I’m slow) in the academic blogosphere, specifically due to Andrea Smith’s tenure denial at the University of Michigan. The banter on the one hand reassures me that I am not the only person who finds this system perplexing, and on the other hand confuses me because I don’t know what would be a viable alternate system. To sum up some of the best posts on the Smith decision and personal experience with the tenure process:

• Tenured Radical’s post highlights the emotional impact of the tenure process;
• Tim Burke’s post notes that the process is necessarily confidential and mystical to protect professional identity;
• Oso Raro problematizes the aspect of “fit” in the tenure process and how fit might operate as a code word for sexism, racism, or just not liking someone else professionally.

And then the proposed solutions:

• The Constructivist suggests movement towards strong faculty unions; and
• Community College Dean (Dean Dad) suggests multi-year renewable contracts.

I want to believe that Dean Dad’s multi-year contract idea would work. I worked at an institution that had converted to that model (and was, as I understand it censured by the AAUP for doing so). Though there were some of the same issues that I see in tenure environments, I felt like my opinion as junior faculty mattered. A lot. On the other hand, I’m sure the 50% of the faculty body that were adjuncts might not have felt quite the same way.

A key issue is also portability. Tenure is the collateral of higher education. If some institutions move the cheese, while others do not, I suspect faculty will become even more invested in their “type” of institution than they already are. At the same time, at the community college level, for example, where the only requirement is effective and intensive teaching, and where tenure operates similarly to the public school system as in automatically after x years of service (and I’m sure I’ll get flak for this but seriously, why do public school teachers need tenure?!?!?!?), what is the point? (Please correct me if publications are required at some community colleges, but my experience was that the teaching load was so high that it was not realistic to expect.)

But the key elements Dean Dad calls for are transparency and reciprocity. I would call the review practice in my tenure track job pretty transparent. All department faculty and all students evaluate tenure-track people every year. Anonymously. Tenured professors get evaluated once every five years or something. That feels pretty transparent actually, though I might have enjoyed it more the first time around if I’d been informed of these processes before they happened. (YES, I realized I was going to be evaluated by students). But the stakes are high for the institution, because if they agree to tenure me, they (feel they are) stuck with me for the next thirty years of my career. It’s the all-or-nothing aspect of tenure that is the most inhumane, the rest-of-your-life or FIRED mentality, but I can also see how the abuse of a contract system or creating different tracks of faculty (tenured research versus untenured renewable contracts teaching) and other variations of unfairness might be just as inhumane.

And then reciprocity. What *is* the issue between administrative and faculty power? I’m not literally asking for an explanation, I just don’t understand why many in higher education feel that higher education is somehow exempt from planning and standardizing curriculum, identifying key objectives across courses, assessing outcomes–both student and institutional, supporting students in other areas besides their coursework, and so on. Higher education might not be corporate, but it is still an industry.

Ultimately though, I think this has been such a hot topic because the tenure system is already broken beyond repair. The only people who truly benefit are those who are tenured or on the tenure track. A recent study at least supports The Constructivist’s suggestion for unions in the sense that more faculty at institutions with unions are tenured or tenure track as opposed to adjunct. The army of adjuncts that do the majority of teaching speak to the true injustice in the attention given to one tenure decision of a top researcher.

The one theme I haven’t heard in any of these posts is what tenure means for students? I don’t know of any institution of higher education that exists without students. For the majority of institutions of higher education, teaching and working with students is still essential. Does the tenure system with jobs for life, painful rejection, and army of adjuncts help or hurt students? Thoughts?