Assessment and Evaluation in IT

As a part of my ed tech class, I’ve been asked to reflect how I might assess for creative problem-solving during maker-lessons. I’m not a teacher, but my IT team certainly performs constant assessment and alignment activities. Eric Isselhardt described the Green Street Academy’s (GSA) transition from traditional instruction to project-based learning (PBL). I was amazed by the degree to which this story mirrors our IT strategic planning journey. IT work is, quite literally, project-based. But is it strategic? Are we doing the right things? These are the sorts of organizational-level assessment questions the GSA faculty asked themselves.

Prior to moving to PBL, Isselhardt observed that, “They were using directive instruction modes designed to impart information and learning within a specific topic area, often in isolation from other topic areas, and they were having inconsistent student achievement results” (Isselhardt 2013). This sounds like IT. We’ve got a bunch of very talented individuals working across campus to achieve individual results. GSA faculty were guided by Common Core Standards; my team is guided by our strategic plan. As with GSA, our efforts were achieving inconsistent results.

We wanted to answer a central question: how do we measure whether we are actually accomplishing the mission? Our team amassed a ton of work, but we found it difficult to measure how well the work accrued to institutional objectives. Typical IT metrics are things like “projects completed on time”, “service uptime”, or various measures of efficiency. None of these, however, tell us if our IT work somehow helps students learn. Grant Wiggins tells us, “We can and do measure anything: critical and creative thinking, wine quality, doctors, meals, athletic potential, etc.” (2012).  Just today, Joshua Kim of Inside Higher Ed asked the question, “How do we measure judgement, honesty, bravery, integrity, and wisdom? The first step, I think, is to have an honest discussion about our goals…” (2015).

In keeping with Joshua Kim’s advice, we began by re-imaginging our objectives. GSA called it a “standards map”, and we called it our strategic objectives. Our old objectives were okay, but mirrored classic IT performance indicators of efficiency. We changed our objectives to map better to the campus mission: student achievement, diversity, affordability, and access. We mapped out the ways these objectives were overlapping and complimentary. With these in place, we developed our road-map (what GSA called “project route”). Our goal was to map the daily activities of the whole team to our lofty objectives.mission mapping

We are now in the process now that GSA called, “preparation for success”. This means mapping daily tasks to the mission objectives. This work is ongoing, but is beginning to allow us to construct mission-mapped performance goals for individual team members, making our work more relevant. We’re able to build our professional development programs around this as well. Best of all, we are beginning to engage the rest of the institution in our assessment and planning process.

process

Notice that process steps overlap. For example, each spring we are adjusting the coming year’s plan based on state budget allocations, while broadly gathering information about campus needs for the academic year following. This creates a cycle of continuous strategic planning (which I’ve blogged about before) that includes constant campus feedback about performance relative to the teaching, learning and research missions. This will hopefully generate the sort of real-team feedback that James Paul Gee tells us makes game-based learning so effective (2010). Of course, executing a multi-year strategic plan is a very slow-moving game, so the meaning of real-time feedback is relative.

In relating our work in IT to the work of GSA, I’ve come to realize that high performing IT needs a standards-based, team-wide PBL approach, just like a school. Interestingly, I think the hard work for IT revolves around mapping to standards because we’ve got the team-based project thing down pat. For schools, it may well be the opposite: standards are well understood but team-based project work is a whole new ballgame.

References

Edutopia. (2010, July 20). James Paul Gee on Grading with Games [Video file]. Retrieved from https://www.youtube.com/watch?v=JU3pwCD-ey0

Isslehardt, E. (2013, February 11). Creating Schoolwide PBL Aligned to Common Core [Web log comment]. Retrieved from http://www.edutopia.org/blog/PBL-aligned-to-common-core-eric-isslehardt

Kim, J., (2015, June 29). Learning Analytics in a Liberal Arts Context. Inside Higher Ed. Retrieved from https://www.insidehighered.com/blogs/technology-and-learning/learning-analytics-liberal-arts-context

Wiggins, G. (2012, February 3). On assessing for creativity: yes you can, and yes you should. [Web log comment]. Retrieved from http://grantwiggins.wordpress.com/2012/02/03/on-assessing-for-creativity-yes-you-can-and-yes-you-should/

Leave a Reply

Your email address will not be published. Required fields are marked *

Captcha * Time limit is exhausted. Please reload the CAPTCHA.