Nuts & Bolts July 2006 / ARCHIVE

NUTS & BOLTS

An electronic assessment newsletter
Springfield College in Illinois
-----------------------------------------

July 2006
Vol. 6 No. 11
-----------------------------------------

Editor's Note. It now looks like it'll be a while
before I can get SCI's assessment website up and
running again. In the meantime, I plan to publish the
newsletter by email and archive current issues on an
interim basis on my personal weblog at
http://www.teachinglogspot.blogspot.com/ ... back
issues through June 2006, as well as the teaching
blog, can be accessed from my faculty page at
http://www.sci.edu/classes/ellertsen/welcome.html

* * *

Stuff happens, to paraphrase (but not quote) a popular
bumper sticker. I had planned to put Nuts & Bolts on
hiatus while I reorganized parts of SCI's assessment
website, but there's information I think I should get
out to faculty on a timely basis. So this email
message will serve as a short version of Nuts & Bolts,
SCI's monthly assessment newsletter, updating you on:
(1) reminders, tips and links relating to fall
semester syllabi, which are due in late July and early
August; and (2) developments on the federal Commission
on the Future of Higher Education, which is
deliberating radical changes in the way we do
institutional assessment.

1. Syllabi

If you've taught before at SCI and/or Benedictine
University at SCI, you're in luck. You don't have any
changes in the syllabus format to wrestle with this
year. Mary Jo Rappe of the Academic Affairs Office is
sending out detailed instructions with deadlines for
SCI's traditional and adult accelerated programs, as
well the various Benedictine modules.

If you're new, Mary Jo's instructions will show you
how to format a syllabus. And your division chair will
be able to help you work with student learning
objectives, learning outcomes and the other details of
a college syllabus.

In either event, syllabi are to be submitted this year
to your division chairs for approval.

With government and other outside stakeholders
dictating more and more of what goes on in the
classroom, our syllabi may seem more complicated than
what you remember from when you were in school. But
once you get the hang of it, it'll make sense. And
you'll wonder what all the fuss was about.

As assessment coordinator, I will be happy to offer
informal advice on how to incorporate goals,
objectives and assessment criteria into your syllabi.
I can be reached by email at pellertsen@sci.edu ...
and we have on the SCI website a 45-page PDF document
entitled "Classroom Assessment for Continuous
Improvement" that walks you through SCI's Common
Student Learning Objectives and other details.

Published in 2005, the classroom assessment guide
summarizes some basic principles of quality
improvement planning and offers tips on how to carry
it out in the classroom by means of formative
assessment. Unlike other parts of the assessment
website at the moment, it can be reached from our
homepage at www.sci.edu ... click on the Quick Link to
"Faculty and Student Websites" and then on "Assessment
Program Goals and Objectives" in the website directory
that opens. That will take you to a new page headed
"Program Goals and Objectives." Scroll down to the
heading "Classroom assessment" and click on the link
thqat says "Guide for Instructors (pdf)." It's
important to keep scrolling down, because on most
browsers you won't be able to see the classroom
assessment links at first.

If your head's swimming from all these details,
remember all of this stuff is like walking, breathing
or riding a bicycle. It's a lot easier to just *do* it
than it is to try to explain it!

2. Federal politicking

The blue-ribbon Commission on the Future of Higher
Education, empaneled in September 2005 and due to
issue a report in September of this year, has released
a second draft report considerably less hostile to
classroom educators than its first draft. Assessment
is hardly even mentioned in this draft, at least it
isn't reflected in press coverage, but nationwide
standardized testing is still looming in the
background.

Reports the online newletter Inside Higher Ed:

"Taken together, the changes made in response to
commissioners’ criticisms of the initial report — many
of which focused on its tendency to favor
harsh-sounding and simplistic rhetoric and
recommendations over practical, well-conceived
analysis and answers — do not radically alter the
panel’s bottom line view: that higher education must
perform better in educating students and in proving
its value to the American public.

"And many if not most of the initial draft’s findings
and recommendations remain intact, a fact many college
officials will rue. The second draft, like the first,
calls for the creation of a national “unit records”
system to track students’ performance through their
academic careers and into the work place (though it
calls the proposal something else), and urges the
collection and publication of significantly more
information that colleges have either not collected
or, more often, held close to the vest.

"But in case after case, the second draft shuns the
instinct, so prevalent in the first, to “throw rocks”
at higher education, as one commissioner put it in
written comments to his colleagues. That doesn’t mean
the new report lets colleges off the hook or ignores
higher education’s real and serious problems; it just
does so in language that is more descriptive and less
inflamed."

Inside Higher Ed's story, dated July 17, can be
accessed at
http://www.insidehighered.com/news/2006/07/17/commission ...

The next day Inside Higher Ed's reporter Doug
Lederman, who has been following the issue all year
long, did a reaction story noting that members of the
commission were all over the map.

He quoted David Ward, president of the American
Council on Education (which represents college
presidents), as saying the second draft showed
"improvements in both tone and content" over the
first. But Ward added it "omitted the preamble that
contained the harshest rhetoric of the first draft,
and since 'these introductory comments will set the
tone for the rest of the report ... I am very anxious
to see what changes will be made in this area.'"

Lederman also quoted American Council of Trustees and
Alumni president Ann Neal as saying the second draft
dropped earlier criticism of "important curricular
issues - and their connection to the serious cultural
illiteracy that the commission recognizes." And
Richard Vedder, an adjunct scholar for a politically
conservative think tank, worried that "as we move to
maximize support within the commission [by toning down
the rhetoric], we run risk of making it more of a
pablum, inoffensive document that says relatively
little."

Lederman's headline, "Too Much Change, or Not
Enough?," catches the tone of things. His report is
available at
http://www.insidehighered.com/news/2006/07/18/commission

Media reaction to the draft, as with the commission's
other deliberations, ranged from muted to nonexistent.
But there were signs the political posturing isn't
quite over.

Writing on a blog titled "Phi Beta Cons: The *Right*
Take on Higher Ed" in the online edition of William
Buckley's National Review magazine, Candace de Russy
said "this draft’s regrettable dropping of focus on
declining undergraduate education should not surprise
us. There are too many higher education insiders
serving on the commission, and it is not in their
self-interest to demand serious curricular reform and
an end to grade inflation as well as to show
open-mindedness to innovative means for delivering
higher education."

She added, "Thus it’s the commission itself that ought
to be gutted and re-constituted with members with
(pardon the expression) real guts. Barring that, it is
likely that this entire exercise will in the end do
little or nothing to ameliorate higher education."

The permalink to de Russy's blog entry is http://phibetacons.nationalreview.com/post/?q=Zjk5NmQ0Yjc3YjJjMzU2MWQ3NjI5MzVlN2U4OThmMzg=

Also reacting to the new draft in the National
Review's higher ed blog was Charles Mitchell, program
director at the American Council of Trustees and
Alumni. He quoted ACTA president Neal's July 18
statement to Higher Ed Today: "In a time of global
competition and conflict, transparency and assessments
don’t matter if the product is not worthy. ... Access
and completion rates are simply irrelevant if the
education received is incoherent and fails to
guarantee the common ground of training and outlook on
which our society depends. Yet the commission remains
silent on these critical points."

Mitchell added, I think with good reason, "There is
certainly much more to come on this story."

Mitchell's permalink is http://phibetacons.nationalreview.com/post/?q=Nzg0ZmFiNmI3NzVlMTFkNDY3YzUzYWIyMDY0NWFlNzE=

National standardized testing

In the meantime, ETS has released a report calling for
"a broad national system to better understand student
learning in two- and four-year colleges and
universities." To do that, ETS specifically recommends
"a systematic, data-driven, comprehensive approach to
measuring student learning with direct, valid and
reliable measures."

The ETS report is titled "A Culture of Evidence:
Postsecondary Assessment and Learning Outcomes." It
notes the federal commission's deliberations and
recommends that the regional accrediting associations
develop a national plan for testing on "four
dimensions of student learning":

-- workplace readiness and general skills
-- domain-specific knowledge and skills
-- soft skills such as teamwork, communications and
creativity
-- student engagement with learning.

"Colleges and universities face continued pressure to
prove their effectiveness in an increasingly difficult
fiscal environment," said Mari Pearlman, Senior Vice
President of Higher Education at ETS, in a press
release posted to the MarketWire public relations
service. "We hope this paper will further the
discussion about how our system of higher education
might respond to this challenge."

The ETS press release, which contains a link to the
report in PDF format, is available at http://www.marketwire.com/mw/release_html_b1?release_id=145859 ...

I hope I don't sound cynical if I note that ETS
(originally known as the Educational Testing Service)
is a leader in the standardized test business. Its
products include the SAT, the GRE, the TOEFL and high
school advanced placement tests.

-- Pete Ellertsen is chairman of SCI's assessment
committee and editor of Nuts & Bolts.

Is this what the future looks like?

Proof, as if it were needed, that ideology has nothing to do with political meddling in the classroom comes today from Great Britain. It came in the form of a news report in The Guardian of a House of Commons committee hearing. Testifying was Alan Johnson, education secretary in Britain's Labour Party government. He defended the emphasis on standardized testing imposed by Labour's Office for Standards in Education ("Ofsted" for short). The Guardian reports:

Speaking to the House of Commons education select committee, Mr Johnson said staff at a school in Nottingham had told him recently that they would like to see league tables [ranking schools by test scores] scrapped.

"I accept the pressure it puts, and the extra intensity and stress it puts on teachers, but it's absolutely the right thing to do," he said.

Mr Johnson gave his backing to "the whole kit and caboodle" of accountability for schools - from Ofsted inspections to national tests and exams and league tables.

He added: "If anything, we need to intensify that rather than relax."

Mr Johnson said it was "fundamental" that children should leave primary school with a mastery of reading and maths.

Sound familiar?

Under Prime Minister Tony Blair, Labour has won elections since the late 1990s with a "New Labour" set of moderately liberal policies similar to former U.S. President Bill Clinton's. School reform; "league tables," well publicized lists of schools' aggragate test scores; and pressure on classroom teachers to raise test scores is part of the "whole kit and caboodle" New Labour offers to the voters. It sounds more than a little bit like our No Child Left Behind regimen of mandatory testing and ranking of schools by aggragate test scores, doesn't it?

British educators, like the teachers in Nottingham or Harvey Goldstein of the Institute of Education in London, argue the league tables can't help but measure factors like "sex, ethnic origin and social class background" that the schools can't be held responsible for.

Sometimes we tie the failure of NCLB to President Bush and the Republican Congress, but we forget the NCLB bill was co-sponsored by Sen. Edward Kennedy, D-Mass., and passed Congress with broad bipartisan support. Again, the similarity between Ofsted's accountability measures and NCLB is striking.

If anything, the British system is more hostile to good classroom teaching than our own. And it's good politics. At today's committee hearing, Johnson was kidded about his political ambitions. Here's how the exchange went:

The Conservative MP for Reading East, Rob Wilson, told Mr Johnson he had "a few quid" on the outcome.

Mr Wilson asked: "When Tony Blair steps down next year and you take over as prime minister will your priority be, as his was, 'education, education, education'?"

The Labour chairman of the committee, Barry Sheerman, suggested at this point that the minister might like to restrict his answer to education policy.

In response to Mr Wilson's question, Mr Johnson said: "Yes. I would probably classify it as 'learning, learning, learning', but it's the same thing."

And I would classify it as politics, politics, politics. Unfortunately, bashing classroom teachers looks like good politics on both sides of the water.

Resources on Native music

Cross-posted to music and teaching blogs for potential use in HUM 221 (Native American cultures) in the spring of 2007.

A valuable article in the Jan.-Feb. 2003 issue of Sharing Our Pathways, newsletter of the Alaska Rural Systemic Initiative at UA-Fairbanks. It's by Vivian Martindale, and it's titled "Native American Songs as Literature." In addition to an ANKN (Alaska Native Knowledge Netword) article on the Athabascan peoples, it mentions Joy Harjo, Canyon Records and other resources on Native cultures in the lower 48.

Says Martindale:

Classrooms don't have to be boring. Literature classes especially can be enhanced through the medium of song. In David Leedom Shaul's article "A Hopi Song-Poem in Context", he claims that the listener is similar to an audience during storytelling, in that the listener is also interacting with the music. The listener, as a participant, is not passive; the listener is hearing rhythms, words, patterns and much more. The listener does not have to understand the Native language in order to appreciate the song. Shaul calls attention to the genre called "song poems." These songs are in a category by themselves, separate from poetry and prose. "The text of song-poems in Hopi culture, like much poetry, seemingly create their own context by virtue of minimalist language" (Shaul 1992:230Ð31). Therefore it would be interesting to include the concept of song poems or poetry as music into a curriculum.

She quotes this from a Joy Harjo/Poetic Justice song called "My House is the Red Earth." (Poetic Justice is Harjo's band.):

My house is the red earth. It could be the center of the world. I've heard New York, Tokyo or Paris called the center of the world, but I say it is magnificently humble. You could drive by and miss it. Radio waves can obscure it. Words cannot construct it for there are some sounds left to sacred wordless form. For instance, that fool crow picking through trash near the corral, understands the center of the world as greasy scraps of fat. Just ask him. He doesn't have to say that the earth has turned scarlet through fierce belief, after centuries of heartbreak and laughter.

She also has tips and caveats on teaching traditional Native American music.

The Chronicle's take on Miller commission

Since The Chronicle of Higher Education usually hides its articles behind a subscription firewall, I haven't kept up with its coverage of the U.S. Education Department's blue-ribbon Commission on the Future of Higher Education. But every so often The Chronicle comes out from behind the firewall, and this week they've got a good takeout on the commission's draft report from the July 7 issue. Like practically everything else in The Chronicle, it's thorough and very well balanced.

Written by Kelly Field, the article catches the tone of the Commission's debate in the headline: "Draft Report From Federal Panel Takes Aim at Academe." A subhead notes the split between chairman Charles Miller and educators on the commission. It also details some of the substantive recommendations that have surfaced thus far:

A draft report released last week by the Commission on the Future of Higher Education called for overhauling the federal student-aid and accreditation systems, easing the process of transferring credits between institutions, and using testing to measure the "value added" by a college education.

The report, which the panel discussed during a closed meeting two days after it was released, also endorsed the creation of a national "unit record" system to track the educational progress of every college student in the United States.

With that on the record, Field goes on to sketch in the controversy on the commission over the tone of its deliberations. Some of the complaints are procedural, reflecting concern that the commission will railroad through a predetermined set of recommendations. Field says:

... several commission members were unhappy with both the substance and the tone of the preliminary report, which was written by an outside writer with assistance from commission staff members. Some said it favored the views of the consultants who drafted the commission's issue papers over the opinions of the commissioners themselves.

"This really reflects what the consultants put in the papers and what they would like the commission to say," said James J. Duderstadt, president emeritus of the University of Michigan at Ann Arbor. "It doesn't have any relationship to the kind of deliberations we had at the May meeting," when members began sifting through potential recommendations in an effort to reach an initial consensus.

David Ward, president of the American Council on Education, said the report was "based on a highly selective reading of testimony" and "in no way reflects the candid and creative discussions we have had during our yearlong process."

"I believe it is seriously flawed and needs significant revision," he wrote in a letter to college presidents.

On the other side of the issue, Field quoted Richard K. Vedder, an economist who writes for the right-wing American Enterprise Institute, who said the report represented "a good starting point," and Sara Martinez Tucker, president and chief executive of the Hispanic Scholarship Fund, who said she was "very pleased with the completeness of it." Field explains:

Ms. Tucker said she created a matrix of all the ideas that came out of the commission's task forces, cross-referenced it against the report, and found that only three of her colleagues' suggestions were missing.

"Some of the ideas may be buried, or not as prominent as people would want, but they're in there. You just have to look," she said, noting that the unit-record proposal — her No. 2 priority — is not mentioned until Page 22 of the 27-page report.

Still, there's this question of tone. It's dogged the Commission since day one, and it won't go away. Field reports:

Other panel members were troubled by the tone of the report, which began by noting that American higher education "has become one of our greatest success stories," but quickly turned to "the less inspiring realities of college life in our nation": the enrollment gap between rich and poor, the high use of remedial courses, rising costs, and a failure to prepare American workers for a changing global economy.

The report went on to describe colleges as "risk-averse, frequently self-satisfied, and unduly expensive," and blamed rising tuitions on colleges' "failure to seek institutional efficiencies and by their disregard for improving productivity."

Robert W. Mendenhall, president of Western Governors University, an online, nonprofit institution, called the report "overly negative and overly focused on the academy as the culprit." And Ms. Tucker said she worried that the report's get-tough tone could backfire, alienating, rather than engaging and inspiring, academe.

Mr. Miller defended the draft, noting that Secretary Spellings had called on the commission not to be "shy or mealy-mouthed." In an interview, he said panel members' repeated calls for "moderate" language have left him feeling "almost like I'm being censored."

Mr. Miller also stood by his decision to have the panel's outside writer produce a complete draft, rather than an outline or set of recommendations, as was initially planned. Several panel members who received the full report a week before it was released to the public said they had been surprised by the abrupt change in plans.

He called the idea of offering recommendations before documenting the problem "an Alice in Wonderland idea: 'answers first, questions later.'"

"My way is the honest way, the direct way," he said.

Note Miller's language. His way is not an honest way, it's the honest way, implying all other ways are something other than honest. Perhaps it's just a chance turn of phrase. Or perhaps Miller's tone is hostile and combative.

Field's article ends with a valuable list of specific recommendations so far on issues of Access; Affordability; Quality and Innovation; and Accountability (with its recommendations on who carry them out listed in parentheses). I'll quote the recommendations on accountabilty below:

• Require institutions to measure student learning using measures such as the National Survey of Student Engagement and the Community College Survey of Student Engagement, as well as the Collegiate Learning Assessment and the Measure of Academic Proficiency and Progress (states). Provide incentives for states, higher-education associations, systems, and institutions to develop outcomes-focused accountability systems (federal government).
  
  
• Make results of such measures available to students and report them publicly in the aggregate. They should also be included on transcripts and in national databases of accountability data. Institutions should make aggregate results publicly available in a consumer-friendly form.
  
  
• Administer the National Assessment of Adult Literacy every five years, instead of 10 (Education Department).
  
  
• Require the National Center for Education Statistics to prepare timely annual public reports on college revenue and expenditures, including analysis of the major changes from year to year, at the sector and state levels (secretary of education).
  
  
• Develop a national student unit-record tracking system to follow the progress of each student in the country, with appropriate privacy safeguards.
  
  
• Create a consumer-friendly information database on higher education that includes a search engine that allows parents, policy makers, and others to weigh and rank institutions based on variables of their choosing (Department of Education).
  
  
• Establish a national accreditation framework that contains a set of comparable performance measures on learning outcomes appropriate to degree levels and institutional missions, and that is suitable for accreditation, public reporting, and consumer profiles; that does not prescribe specific input and process standards; and that requires institutions to report progress relative to their national and international peers.
  
  
• Make accreditation more transparent. Make the findings of reviews easily accessible to the public, and increase the proportion of public representatives in the governance of accrediting organizations and members of review teams from outside higher education.

Footnote. The Chronicle does have posted to the World Wide Web a directory of stories it has written on the Miller commission. And we take The Chronicle at SCI's Becker Library. So if you want to read them in hard copy, you can print out the directory and take it to the library.

Snake oil and 'Texas-syle accountability'

I'm cross-posting this item to my blogs on newspapering and education, for reasons that should be obvious as we go along.

Are we seeing the beginning of an orchestrated effort to discredit American colleges and universities? After months of being mostly ignored by the news media, Charles Miller, the chairman of a blue-ribbon federal Commission on the Future of Higher Education gives an interview to his home-town paper. He blasts higher ed, and he blasts the members of his commission who dispute his rhetoric. Staff writer Ralph K.M. Haurwitz of The Austin (Tex.) American-Statesman reports in Friday's paper:

Charles Miller expected a fight from higher education administrators when he agreed to head a national panel for his old friend, Margaret Spellings, the U.S. secretary of education. He's getting one.

The Commission on the Future of Higher Education issued a draft report this week recommending academic and fiscal reforms. Some higher education leaders, including a few on the commission, have criticized the draft as overly harsh in tone and too quick to condemn academia.

Miller, speaking from Houston on Thursday, a day after the commission met to review the draft, didn't sound like someone interested in backing down on substance and perhaps not too much on tone.

"I've been advised to say things in moderate terms, to not criticize the academy," Miller said, declining to say who offered such advice. "It's almost like being censored. Some of the language ... could be toned down, but the real issue is putting responsibility on the higher education system for things it's not doing well. It has some really bad flaws."

The headline catches the tone of Miller's remarks: "Chairman defends panel's call for reforms in higher education." But his proposed reforms - which are not yet the commission's because they haven't been adopted yet - have been roundly questioned in The New York Times and a few papers like The Boston Globe in major metro areas where the commission has conducted hearings. (The Harvard Crimson, a student paper with an understandable ax to grind, has followed the commission more faithfully than any of the dailies.) So why does Miller answer his critics in a paper down in Texas and not The Times, The Globe, The Harvard Crimson or the papers that have covered the commission's debate? And why, for that matter, does The American-Statesman write up Miller's defense without interviewing his critics?

Now I'm going to assume The American-Statesman down in Austin got the story on its own. Miller is a former chairman of the University of Texas Board of Regents, and he might have mentioned it back home. Word might have gotten around town, and the paper might have decided to get to the bottom of it. Sometimes that's the way we got stories when I was on the courthouse beat. Of course Miller could have leaked it to a friendly paper, too, but I have no way of knowing that. So I won't speculate.

Miller's friend Margaret Spellings was in the news last week, too. At an international conference in Athens, Greece, she spoke on "higher education and the benefits of partnering with the private sector to prepare students for jobs in the 21st century." And by golly, she just happened to mention the Miller commission:

In launching this Higher Education Commission, we recognized that to remain a quality system we had to ask the tough questions and anticipate necessary changes that can and must be made if we are to have a robust system 50 years from now – especially as needs for all become greater.

As a nation, we spend more than $300 billion dollars a year on higher education – a third of which comes from the federal government. Yet, we have very little information on what we are getting in return for that investment. And what we do know is cause for action.

I can't find any evidence on the internet that the media picked up the story, but Secretary Spellings' remarks were helpfully posted on the U.S. Education Department website.

Miller noted the same statistical factoid in his interview. Here's how The American-Statesman reported his remark and put it in context:

The federal government covers a third of the nation's higher education spending but less than 10 percent of the K-12 investment. Yet the federal government exercises more control over primary and secondary education — through Texas-style accountability that Bush parlayed into a national policy — than it does over colleges and universities.

Miller, former head of the University of Texas System Board of Regents, said Thursday that he regards significant change as not only urgently needed but inevitable.

"If you have a very inefficient and very expensive enterprise, which higher education is now, and huge changes in technology and a cultural change in how people use this technology, that's almost a guarantee that some entity somewhere is going to develop a very effective way to deliver these skills at a much cheaper price," he said. "It could be in a country where they don't have a set of institutions to be angry about change.

"It'll have such demand that you'll have explosive growth that could sweep the higher education system like a tsunami. Supply creates a demand sometimes, not the other way around," Miller said, citing as an example the advent of personal computers and software to run them.

Miller said he knew from the start of the commission's work last year that some in higher education circles would be highly skeptical of his leanings. "I was from Texas and a businessman and worked on accountability and a Bush friend," he said. "I was in about the worst category you could be in."

On that note, the paper segued to Miller's recommendations: Streamling and increasing financial aid, better record keeping," encouraging "colleges and universities to develop new and better methods of controlling costs and improving productivity," and encouraging "states to require public colleges to measure student learning using tests, such as the Collegiate Learning Assessment, that examine critical thinking, reading, math and other skills."

All of this bears watching, but Miller's last points bear especially careful scrutiny. The membership of his commission is weighted toward industry and people with a vested interest in test prep and for-profit educational venures rather than academicians, and consistently he has touted one specific standardized testing product every time he mentions the subject of testing.

What it is that makes this old courthouse reporter think if Miller and his friends from Texas are peddling snake oil, and if they have their way, somebody, somewhere is going to make a big ole Texas-size pile of money as we move into the future of higher education?