Assessment Activities Summary Report

July 2004

Summary  

The assessment of students accepted by the Education Department as candidates preparing for licensure as elementary or secondary teachers is required by our colleges, by our national accreditor, by our state regulatory agency, and by an agency of the federal government.  During the past year we broadened our screening of prospective candidates’ for college-level academic skills in reading comprehension, following the conventions of writing, and in using mathematics.  Our affirmation of candidates’ writing skills, aside from small refinements in our testing procedures, remained unchanged.   

With respect to our prospective and accepted candidates’ academic skills, we find that…

  • At least three-fourths of our prospective candidates wrote essays equal to our minimum performance standard (read more about this finding beginning on page 9). 
  • Essays scored below that standard tend to include weak or ineffective forms of evidence to advance logically inconsistent or irrelevant arguments.  Such essays are more likely to reveal less effective patterns of organization (page 10).
  • Second year students having completed their year-long Symposium experience are as likely to write below-standard essays as are those First year students enrolled in that review college-level communication skills at the time of their assessment (page 11).
  • At least one-fifth of our prospective candidates perform below our standards on screening examinations in reading comprehension, using the conventions of writing, or in mathematics (see pages 12 through 16).

All who would be licensed as elementary, middle, or high school teachers by Minnesota’s Board of Teaching must verify the extent of their subject matter knowledge for the licensure areas in which they hope to teach.  Candidates must also verify through examination their understanding of pedagogy appropriate for their chosen licensure area.    

  • Examination performance suggest that candidates for elementary licensure often equal or exceed the performance of their peers in Minnesota and in the nation as a whole on most facets of Praxis II examinations over subject matter and pedagogy. 
  •  While their subject matter knowledge of Language Arts and Mathematics equals or exceeds scores earned by their Minnesota peers, our elementary candidates’ test performance in general science and social studies falls below that of candidates prepared by other Minnesota colleges (see pages 19 and 20).
  •  Most elementary candidates perform at or near the level of their Minnesota peers on the “Principles of Learning and Teaching” pedagogy examination.  The distribution of their scores, however, reveals patterns of weakness in “teacher professionalism” for those tested in 2001-2002 as well as knowledge of “students as learners” and “Instruction and Assessment” for those tested in the 2002-2002 (pages 21 and 22). 

Information provided by the Educational Testing Service concerning the test performance of candidates for licensure as social studies teachers in middle or high school settings is less comforting than data provided for our elementary candidates.

  • Social studies candidates’ overall test performance fell below their state or national peers in several content areas.  Our candidates’ scores were notably lower in world history (pages 23 and 24).

Our secondary candidates’ knowledge of teaching and learning, as described by their Praxis II test performance, is similar to that of candidates prepared by other Minnesota colleges and universities.

  • More secondary candidates completing their pedagogy exams in 2001-2002, however, earned lower scores in “teacher professionalism.”  More candidates completing their pedagogy exams during the following year (02-03) earned lower scores in “teacher professionalism” as well as in “students as learners” (see pages 25 through 27).

Nearly all candidates seeking elementary or secondary licensure exceeded Minnesota’s minimum qualifying scores on all relevant tests for their Minnesota licensure (see Appendix 2 on page 30). 

Program refinements guided by NCATE’s six accreditation standards and supported by our assessment of candidate performance are summarized in our annual report to that organization (see Appendix 3, pages 33 to 38).     

  • A study of novice “teachers of promise” nominated by staff employed by our partner schools revealed patterns of growth in subject matter knowledge and teaching skills that generally reflected our program mission and goals.
  • Following curricular review, we corrected a weakness in the design of our language arts specialty for elementary candidates that will increase their opportunities to know about using adolescent literature in their classrooms while improving their composition and speech communication skills. 
  • We continue to improve and expand candidates’ opportunities to observe and teach students from diverse racial, cultural, and ethnic backgrounds. 

The Context for the Assessment of Teacher Education

The Department of Education, jointly sponsored by College of Saint Benedict and Saint John’s University Education Department, annually prepares approximately 100 women and men for licensure by Minnesota’s Board of Teaching as elementary, middle, and high school teachers.  The Department’s faculty and curriculum are approved by the Minnesota Board of Teaching (Board), a regulatory agency established by Minnesota’s Legislature to manage the licensure of teachers and oversee their preparation by the 28 higher education institutions that prepare teachers in this state.  The Department’s licensure programs are accredited by the National Council for Accreditation of Teacher Education (NCATE), a voluntary association of teacher preparation programs bound together by the pursuit of common curricular and performance standards.  The United States Department of Education (DOE), under authority of Title II of the Higher Education Act, indirectly oversees the department through analysis of examination scores of our “program completers” used to rank our program relative to all other teacher preparation programs in Minnesota.  Each of these agencies requires regular annual reports and occasionally requests additional reports concerning curriculum, college or department policy, or student performance issues.  These agencies, more than the colleges that sponsor teacher education programs or their regional accreditors, arrange the conditions and set the contingencies for assessment of licensure candidates and review of their preparation.  An understanding of how assessment “works” in this context may help explain the approach we have taken.

State Approval. Minnesota’s Board of Teaching approves educational institutions to prepare teachers as well as their programs of study supporting each licensure area.  As a condition of such approval, an “institution” (an academic department, college, or school of education) must systematically review the performance of its candidates for licensure at each significant point in their preparation for teaching.  Facets of the comprehensive assessment system required to do so appear in several sections of the administrative “rules” guiding the Board’s approval process.

The Board requires that all institutions preparing teachers demonstrate that… 

Assessment and evaluation are integral components of the professional educational sequence and are used to monitor teacher candidate performance and program effectiveness (Minnesota Rules, 8700.7600, Subpart 5.A.3.  (http://www.revisor.leg.state.mn.us/arule/8700/7600.html).    

This integrated system of assessment and evaluation will use…

Multiple criteria and assessments…to identify candidates for admission who have potential to become successful teachers (5.D.2)

Board approved institutions will continue to use…

Authentic performance-based assessments and systematic procedures and timelines to determine whether candidates have the knowledge and skill needed to advance through the program (Subpart 5.E.4).

Approved institutions will affirm the readiness of their candidates for licensure by using… 

A systematic and comprehensive assessment design that is applied to all candidates throughout professional preparation (Subpart 5.F.2)

Candidates for licensure will demonstrate their attainment of… 

The program’s stated exit criteria and outcomes…through the use of multiple sources of data, for example a culminating experience, portfolios, interviews, videotaped and observed performance in schools, standardized tests, and course grades (5.F.4).

Reporting Requirements.  The Board reviews each of Minnesota’s 28 teacher preparation institutions every seven years.  Each licensure program supported by those approved institutions is also reviewed on a seven year cycle.  Institutional approval requires an extensive self-study focused on 64 standards set by the Board and established in legislative rule.  An on-site visit by Board staff and teacher education faculty members from other approved institutions provides opportunities to explore selected standards in detail.  Licensure programs are approved by the Board following favorable review by two or more content area experts.  Their review is supported by extensive documentation describing candidates’ performance and institutional resources for each licensure area.  Our institutional review will take place in October 2005.  Our eleven licensure programs will be reviewed during the fall of 2006.

The Board may also request supplemental documentation in response to its own needs or legislated mandates.  During the past year we described and documented the ways in which we provide our elementary education candidates with opportunities to know, practice, and be assessed on “scientific methods of reading instruction” (January 2004).  Working with other approved institutions, we reported the licensure, classroom experience, and school involvement of our faculty and staff (February 2004).

National Accreditation. State requirements for approved institutions and licensure programs are mirrored in the “standards” used by NCATE to offer initial or continuing accreditation to its member colleges.  All teacher preparation programs approved by that association must develop and use…

An assessment system that collects and analyzes data on applicant qualifications, candidate and graduate performance, and unit operations to evaluate and improve the unit and its programs. 

Standard 2: Program Assessment and Unit Evaluation,

Professional Standards for the Accreditation of Schools, Colleges, and Departments of Education, National Council for Accreditation of Teacher Education (Washington, D.C. 2001, p. 21 or currently at http://www.ncate.org/documents/unit_stnds_2002.pdf).  

Standard Two includes three sub-standards or “elements” concerning the design of the system itself, the ways in which information gathered and explored using that system, and the use of assessment information to support program improvement.  NCATE provides descriptive scoring guides or “rubrics” to help a visiting team estimate the extent of a program’s compliance with each element of the standard.

Reporting Requirements.  NCATE’s parent organization, The American Association of Colleges for Teacher Education (AACTE), sponsors an annual ingathering of enrollment and program information to sustain its Professional Education Data System (PEDS Parts A and B).  NCATE also requires an annual report to supplement this information (Part C).  Additional reports or surveys may be requested from time to time by either organization. 

Accreditation by NCATE is based on an extensive self-study documenting attainment of each of six global standards and related program requirements.  A Board of Examiners formed from college faculty and K-12 teachers uses the self-study, known as an Institutional Report, as the basis for its site visit to confirm and explore selected facets of the institution’s curriculum, faculty, and students.  NCATE’s institutional reviews, now on a five year cycle, will take place every seven years.  Our continuing accreditation visit will take place in October of 2005.

Federal Oversight. Legislation that reauthorized the Higher Education Act in 1998 included a “title” or section stipulating that all institutions receiving federal funds for any purpose must submit state licensure test data for those students who complete their teacher preparation programs.  Minnesota requires college students who seek to become teachers to confirm their academic skills by completing the “Pre-Professional Skills Test” (PPST or PRAXIS I) prior to their acceptance as candidates.  Upon completion of their preparation, students who persist in their preparation are required to pass examinations confirming their understanding of the body of knowledge they will teach (PRAXIS II content tests) and of the methods of instruction they will employ to do so (PRAXIS II Principles of Teaching and Learning).  Licensure cannot be granted to a candidate who has not passed all relevant tests. 

Reporting Requirements.  Since 1999 nearly all teacher preparation programs in the United States have been collecting, verifying, and reporting records of tests completed and passed or failed by students they prepare for licensure. They are required to share this information with prospective and current students in a variety of ways.  Our federally mandated test results are included in our promotional literature, noted in our colleges’ catalogue, and reported on our web site.

The Secretary of Education is required to annually report the information provided by these 15,000 institutions along with data provided by each state to the Congress.  The Secretary’s current report (2003) as well as his previous reports (2002, 2000) are available at the Department of Education’s internet site (http://www.ed.gov/about/reports/annual/teachprep/index.html).  

College Review.  With Federal encouragement, accreditation of American colleges and universities by regional associations requires documentation of what students attending these institutions are expected to learn as well as the success they experience in their efforts to do so.  Each institution, guided by the requirements of its regional accrediting association, follows its own approach in collecting and sharing this information.  

Reporting Requirements. We comply with the requirements of the College of Saint Benedict and Saint John’s University Academic Assessment Committee for an annual report of our assessment activity.  We submit documents prepared for our regulatory agencies which include information on the performance of students seeking to become licensed as teachers as well as studies we are asked to prepare or those we initiate to guide the evolution of our programs of study. 

Goals and Standards.  Higher education faculty are moving toward a working understanding of the language and implications of “assessment” with its narrowing focus on “learning goals” and “outcomes” describing students’ learning. At the same time higher education is moving away from reliance on descriptions of the “inputs” (teachers’ lectures) and “resources” (library holdings) instruction. 

Teacher education, following the policies and practices developed in K-12 schools, pursues a similar agenda through an interlocking maze of institutional and licensure program standards, content and pedagogical standards defining teachers’ practices, and learning standards guiding elementary and secondary students’ learning.  These related standards are in turn described by stipulated or developed “rubrics” or “scoring guides” which are in turn revealed through “benchmarks” or performance examples.  

In its guiding document on the design and assessment of teacher education, Professional Standards (2002), NCATE defines standards as “written expectations for meeting a specified level of performance” (p.57).  Rubrics are “written and shared criteria for judging performance that indicate the qualities by which levels of performance can be differentiated and that anchor judgments about the degree of success on a candidate assessment” (p.56).  Benchmarks are defined as “a description or example of candidate or institutional performance that serves as a standard of comparison for evaluation or judging quality” (p. 52).  

Responding to pressures from a variety of public and government sources, nearly every state now offers sets of standards that describe the knowledge, skills, and values which children from preschool through grade 12 are expected to acquire from their work “core” subject area as they complete each grade (see http://education.state.mn.us/mde/Academic_Excellence/Academic_Standards/index.html for Minnesota’s K-12 standards).  Their teachers, of course, must know and be able to guide them toward mastery of these standards.  Minnesota’s recently approved social studies standards, intended as a guide to the essential facts, concepts, and principles of history, the behavioral sciences, economics, and political science for students in all grades, are concisely described in only 80 pages.

Additional collections of standards, ranging from broadly sketched “learning goals” to more narrowly defined “objectives,” guide the preparation of all enrolled in most state’s teacher licensure programs.  Minnesota expects prospective and current teaches to have mastered 140 “Standards of Effective Practice” defining the process of teaching common to all subject areas (http://www.revisor.leg.state.mn.us/arule/8710/2000.html). 

More standards define the content and teaching practice of each licensure area. Rule 8710.3200, for example, defines the knowledge and skills required of those who would be licensed as elementary-level teachers (grades K-6: 96 core content standards).  Since Minnesota’s licensure requires all elementary teachers to balance their “generalist” knowledge of all subjects for the early grades with a “specialty” that would enable them to teach one subject to middle school students, all newly prepared elementary teachers must also know, apply, and be assessed on standards that define the content of their chosen specialization (grades 5-8 specialty: 96 standards plus Communication Arts, 6 standards; Mathematics, 38; Social Studies, 31; Science, 149; World Languages, 16).  Minnesota’s elementary-level teacher content standards are revealed in detail at http://www.revisor.leg.state.mn.us/arule/8710/3200.html.       

Department Learning Goals.  Our programs of study and practice are thus guided by and accountable for complex and sometimes overlapping clusters of state, national, and federal “standards” of many kinds.  We organize those programs around ten learning goals integrated within a conceptual framework that provides a sense of purpose and direction for our work with the women and men we accept as candidates for licensure.  These ten goals are congruent with Minnesota’s primary Standards of Effective practice and anchored in our evolving “knowledge base” of research and practice.  

  • Subject Matter - The candidates we prepare for licensure as Minnesota teachers understand the central concepts, tools of inquiry, and structures of the disciplines they are preparing to teach so that they will be able to make this subject matter meaningful for their students.
  • Student Learning - The candidates we prepare for licensure draw on their understanding of learning and developmental processes to choose optimal ways that encourage their students’ intellectual, social, and personal development.
  • Diverse Learners - Our candidates, recognizing how differences among students can influence their learning, make instructional decisions that reflect to their students’ backgrounds and exceptionalities.
  • Instructional Strategies - Our candidates use their knowledge of instructional strategies to decide upon and employ those which are most likely to encourage their students’ critical thinking, problem solving, and performance skills. 
  • Learning Environment - Our candidates for licensure use their knowledge and skills to create just, disciplined learning communities that can motivate students to achieve personal and academic success through positive social interaction and active engagement in their learning.  
  • Communication - The candidates we prepare for licensure as teachers use effective verbal, nonverbal, and  media communication techniques to foster their students’ learning.
  • Planning Instruction - Our candidates for licensure plan and effect instruction as they decide what content they will teach, to whom they will teach it, in what ways they will do so, and with what effect.
  • Assessment - Our candidates for teacher licensure use information provided through their use of formal and informal assessment methods to make instructional decisions that will support their students’ continuous development.
  • Reflection and Professional Development - Our candidates for licensure critically reflect on the effects of their instructional decisions on the performance of their students, on the practice of their colleagues, and on the actions of others in their learning communities, using those reflections to direct and sustain their professional renewal.
  • Collaboration, Ethics, and Relationships - The candidates we prepare for licensure as Minnesota teachers enhance their effectiveness as educators by working together with their colleagues, their students’ parents, and members of their school community to create and sustain a positive learning environment that can enhance students’ learning and well-being.

Evaluation Questions.  We have chosen to organize our assessment system, described in detail in documents prepared for and reviewed by both the Board and NCATE, around four central questions through which we examine progress toward our goals.

  • Do candidates prepared for licensure as teachers possess the academic skills that will sustain their learning while enrolled in our program of study and practice?
  • ·Do these candidates possess an integrated body of knowledge, skills, and values drawn from one or more of the disciplines central to their areas of licensure?
  • Do our candidates possess pedagogical knowledge, skills, and values appropriate for their areas of licensure?
  • Can these candidates teach knowledge and skills from their areas of licensure to others?

The following table outlines the major features of our “unit assessment system” organized by these four evaluation questions.  

Assessment Questions

Current Information Sources

Future Information Sources

1. Academic Skills

ACT and PPST sub-scores

  (reading, writing, math)

Department Writing Assessment

  (composition)

Speech Proficiency Test

College GPA

Department GPA

Student Teaching Portfolio

   (writing, math, speech)

High School GPA/Rank

Course Samples

   (embedded writing assessments)

Methods Course Work Sample

   (writing, math, speech)

2. Content Knowledge

PRAXIS II: Content

Student Teaching:

   -Performance Profile (goal 1)

   -Work Sample (unit lessons)

Major GPA

Methods Course Work Sample

   (content of lessons)

3a. Professional and pedagogical knowledge

PRAXIS II: Pedagogy

Student Teaching

   -Performance Profile

     (goals 2,3,5,9,10)

Management Plan

Human Relations Project

Methods Course Work Sample

3b. Pedagogical content knowledge

 

PRAXIS II: Content/Pedagogy

Student Teaching:

   -Performance Profile

     (Goals 1,4,6,7,8)

   -Work Sample (unit/lessons)

Methods Courses

  -Resource Portfolios (art/eled)

  -Work Sample

4. Teaching Others

 

Student Teaching

   -Performance Profile

     (Goals 3,5,9)

   -Work Sample

     (unit learning summary)

Methods Courses

   -Work Sample

     (lesson learning summary)

Findings

This report will focus on two of the four assessment questions we use to organize our review of students’ performance and program effectiveness.  We will review evidence gathered from selected sources to explore facets of these two questions from recent reports prepared for the department or its regulators, accreditors, and overseers.

Assessment Question 1:   Do candidates prepared for licensure as teachers possess the academic skills that will sustain their learning while enrolled in our program of study and practice?

We presume that all who would teach others must themselves be competent readers, skilled writers, and proficient in their use of mathematics.  We now rely on three indicators to identify those prospective education majors and minors whose academic skills can sustain meaningful leaning in our licensure programs.  Those indicators also reveal those of our students who may need developmental assistance in one or more of four skill areas. 

Writing Performance. The Education Department began systematic testing of prospective students’ composition skills in April of 1996 by asking prospective students to prepare persuasive essays on a topic in education that could confirm their ability to share their ideas with others through their writing.  Now the longest continuing assessment of student writing at our two colleges, testing procedures mirror the writing process advanced by Symposium guidelines followed by faculty teaching in this program.  Students are provided with information on the topic chosen for their essays and on the design of a persuasive essay.  All are invited to participate in workshops developed by the colleges’ Writing Center to refine their emerging essays.  Essays are scored holistically following accepted procedures by faculty and staff trained to perform their roles.

We expect that students who seek to become elementary or secondary teachers will begin their preparation with writing skills at “Level Two” or higher as defined by faculty who contributed to the scoring guide for this assessment.  That level, or performance standard, requires that writers of such essays will…

Take a clear stand or position on the topic but the essay may lack a unified argument, an arguable proposition, or three distinct reasons supporting that position.  The writer may offer a statement of personal preference, yet fail to go beyond an appeal to personal opinion as a reason for that preference.  While a “Score 4” paper will often reveal two or three logically developed reasons in support of a position on the topic, one or two may reveal flawed thinking.  Stronger forms of evidence will be used to support most reasons.  Mechanical or structural errors may be evident, but they will be less frequent than in a paper falling into the lower one-half range of scores (failing scores of 1, 2, or 3).  Essays scored as “4” suggest a writer working toward control of the persuasive task.

Education Department Writing Assessment Scoring Guide (Revised 4 February 2004)

The following table summarizes writing performance for this academic year compared with aggregated results from the previous 18 administrations of the Education Department Writing Assessment.

Table 1.  Writing Performance of Prospective Licensure Candidates              

Performance Level

 

Spring 1996

to

Spring 2003

Fall 2003

Spring 2003

 

Level 3

   45 (  5%)

 

 

 Level 2 

 585 (59%)

 41 (84%)

  25 (71%)

Level 1

 293 (30%)

   8 (16%)

  10 (29%)

Below 1

   61 (6%)

 

 

Totals

 984

 49

  35

Essays scored as Level Two provide a stronger argument, at least two logically distinct reasons in support of that argument, and stronger forms of evidence.  Although they are writing under some pressure, most Level Two writers observe the conventions of grammar, punctuation, spelling, and capitalization. 

Writers of essays scored at Level One are more likely to reveal weaknesses in the nature and forms of evidence as well as in the logic of their arguments.  Level One writers also reveal weaknesses in how they organize their arguments to support or oppose educational issues selected for their review.  Tables 2.A and 2.B present an analysis of failing essays drawn from the Department’s scoring guide for two recent administrations of our writing assessment.

Table 2.A.  Characteristics of Level One Education Essays: Fall 2002

Raw

Score

Task

Focus

Organization

Reasoning

Evidence

Mechanics

Sense of Audience

6

 

 

 

 

 

 

5

1 ( 8%)

 

 

 

 

 

4

5 (42%)

2 (17%)

2 (17%)

1 (8%)

8 (67%)

8 (67%)

3

5 (42%)

8 (67%)

8 (67%)

10 (83%)

4 (33%)

4 (33%)

2

1 ( 8%)

2 (17%)

2 (17%)

1 (8%)

 

 

1

 

 

 

 

 

 

Note: Raw Scores of from 1 to 3 on the six step AP/ED Essay rubric are equivalent to AP Level 1.

Twelve essays in this sample, taken on 19 October 2002, were rated and confirmed at Level 1.

Table 2.B.  Characteristics of Level One Education Essays: Spring 2003

Raw

Score

Task

Focus

Organization

Reasoning

Evidence

Mechanics

Sense of Audience

6

 

 

 

 

 

 

5

       

 

 

 

 

 1 6%)

4

13 (76%)

9 (53%)

9 (53%)

 1 (  8%)

14 (82%)

 

3

  4 (24%)

6 (35%)

6 (35%)

 5 (29%)

   2 (12%)

14 (86%)

2

 

2 (17%)

2 (14%)

 7 (41%)

   1 (6%)

  2 (14%)

1

 

 

 

  4 (24%)

 

 

Note: Raw Scores of from 1 to 3 on the six step AP/ED Essay rubric are equivalent to AP Level 1.

Seventeen essays in this sample were rated and confirmed as written at Level 1.

Although our sample is not easily extended to the experiences of all first and second year students who choose to enroll in our colleges, for those choosing to major or minor in education we find few meaningful differences between the essays written by first and second year students. We might expect more first year students enrolled in Symposium to write lower level essays than second year students who have completed that one year course in written and oral communication.  Tables 3.A and 3.B reveal the performance of two samples of prospective candidates that does not confirm this expectation.  Similar proportions of first and second year students wrote essays scored at Level One (21% and 23% respectively in fall 2002; 21% and 28% in spring 2003).  With somewhat greater variation, the same pattern holds for passing Level Two essays. 

Table 3.A.  Essay Performance by Year in College; Fall 2002

 Performance Levels:

First Year

Second Year

Third Year

Totals:

Level 3

  0 (0%)

  0 (0%)

0 (0%)

  0 (0%)

Level 2

15 (79%)

24 (77%)

6 (86%)

45 (79%)

Level 1

  4 (21%)

  7 (23%)

1 (14%)

12 (21%)

Totals:

19 (33%)

31 (54%)

7 (12%)

57 (100%)

Table 3.B.  Essay Performance by Year in College; Spring 2003

Performance Levels:

First Year

Second Year

Third Year

Totals:

Level 3

    3 (8%)

  2 (7%)

0 (0%)

  5 (7%)

Level 2

  28 (72%)

18 (64%)

    2 (50%)

48 (69%)

Level 1

    8 (21%)

      7 (25%)

    2 (50%)

17 (24%)

Totals:

39 (55%)

    27 (39%)

    4 (6%)

70 (100%)

Examination of the report providing the information abstracted for Table 1, the Education Department Writing Assessment: 2003-2004, will reward the reader with a description of the scoring process, the scoring guides defining performance levels, and selected essays illustrating each of those levels.  Additional information on candidates’ writing performance, described in the context of related academic skills, appears in an earlier report, Academic Skills of Prospective Education Students, 2002-2003.

Education Department Writing Assessment 2003-2004

Academic Skills of Prospective Education Students: 2002-2003

Writing Conventions. Skilled writers know and use the “mechanics” of English composition to encourage accurate communication of their thoughts.  While errors of this kind are easily made and difficult to detect, they can quickly destroy a writer’s ethos if evident in formal compositions.  Fortunately, the rules of grammar, capitalization, punctuation, and spelling are easily verified using fixed response test items focused at the knowledge or comprehension level of understanding.  Unfortunately, assessment of such recollections and habits is frequently used in place of a more costly and difficult examination of writing performance.

The department expects that students seeking acceptance as elementary education majors or secondary education minors will reveal their understanding of the conventions of English composition at or above Level Two as defined by the Educational Testing Service for use with the Academic Profile, a criterion-referenced test of college level general education.

In addition to performing successfully at Level One, a student who is proficient at Level Two also recognizes appropriate agreement among basic elements when they are complicated by intervening works or phrases. He avoids errors in relatively long and  complicated constructions. She is able to recast several simple clauses using a single more complex combination. Students performing at this intermediate level can recognize and use the conventions of good writing.

This Proficiency level description was adapted from pages 9 and 10 of The Academic Profile User’s Guide, published in1998 by the College Board and the Educational Testing Service.

Two other indicators provide information to supplement the Academic Profile’s estimates of prospective candidates’ knowledge of the conventions of English composition.  Those who matriculate with American College Testing (ACT) sub-scores in writing of 24 or above or students whose Pre-Professional Skills Test (PPST) writing scores equal or exceed 174 confirm their Level Two understanding of the rules of composition.  Information is now available for two of those three indicators.

Table 4 summarizes our students’ knowledge of writing mechanics as estimated by their Academic Profile scores.  A total of 127 students, more in their first or second year of study but some in their third or fourth, completed the test during the 2002-2003 academic year.  About one half of these students (Level One: 62 of 127, 49%) responded to the Profile’s questions as if their knowledge of writing conventions did not reach Level Two.  A few (4 of 127; 3%) tested below Level One.   

Table 4.  Academic Profile: Conventions of Writing

Performance Levels:

Spring 1996

       to

Spring 2001

Fall 2001

to

Spring 2002

Fall 2002

to

Spring 2003

Level 3

81 (12%)

   16  (14%)

   16  (13%)

Level 2

 259 (40%)

   31  (26%)

   45  (35%)

Level 1

 300 (44%)

   70  (59%)

   62  (49%)

Below 1

   15 (2%)

     1  (1%)

     4  (3%)

Subtotal

 654 (97%)

 118 (100%)

 127 (100%)

Reversals

   21 (3%)

     0

     0

Total Tested

675 (100%)

 118 (100%)

127 (100%)

While retaining the Academic Profile’s performance levels, the department elected to end use of that examination in the spring of 2003.  Statistical analyses at that time suggested that equivalent ACT and PPST scores could be used in place of the Profile to identify students who might need developmental assistance to correctly follow the rules of formal writing or to employ other academic skills.  The following table summarizes PPST scores for those completing one or more writing sub-tests during the 2002-2003 test year (1 Sep 2002-30 Aug 2003).  Unfortunately, these two indicators are not fully comparable.  Students whose test performance is described by ETS in Tables 5A and 5B may not be included in the previous summary of Academic Profile writing performance provided in Table 4 due to different patterns of test taking.  We hope to resolve this difficulty with refinement of our assessment data base.     

Table 5.A.  PPST Writing Examination: Group Performance: 2002-2003

Test Year

Number

Low Score

Median

High Score

25th-75th %tile

02-03

105 (64,642)

169 (150)

177 (175)

186 (190)

175-180 (172-178)

Table 5.B.  Level Two Equivalent Scores, PPST Writing Examination (174 and above)                    

PPST Writing Scores

150-168

169

170

171

172

173

174

175-180

          CSB/SJU Examinees

0

1

1

1

4

11

8

87

Using the PPST as an indicator of Level Two understanding of the conventions of English composition, we find that few of those tested appear to require further development of their writing skills (87 of 105, 83%).  At least 18 of the 105 who completed the test during the 2002-2003 test year, however, may profit from such assistance (17%).  Note that these 18 have PPST-Writing scores that fall into the first quartile (lower 25%) of a distribution of all CSB/SJU scores (169-174).  Because some of those students included in the PPST-Writing tables are not included in the summary of what appears to be a more challenging Academic Profile (Table 4), the two samples are not fully comparable.  

Reading Comprehension.  With efforts to improve reading comprehension emerging as a major theme among reformers of public education, we might profit from a careful look at the reading skills of those planning to become teachers.  Perhaps those who are themselves skilled readers might share the value and outcomes of that skill with their future students.  The Education Department adopted the Academic Profile in the spring of 1996 as one way to affirm the extent of prospective candidates’ reading comprehension.  In doing so, the Department also adopted “AP Reading Level Two” as the foundational performance standard expected of all students accepted as candidates for licensure.   

In addition to performing successfully at Level One, students who are proficient at Level Two can also gather information from different sections of a passage and recombine it.  These students recognize relationships that can be inferred but are not explicit.  They can recognize summaries and alternative ways of stating information, interpret figurative language, and recognize the point or purpose of a passage as a whole, or significant portions of a passage.

This proficiency level description was adapted from pages 9 and 10 of The Academic Profile User’s Guide, published in1998 by the College Board and the Educational Testing Service.

Table 6 summarizes the reading comprehension of prospective education majors and minors as described by their responses to the Academic Profile’s fixed response questions.  About one-half of those who completed this examination during the 2002-2003 academic year did not reach acceptable Level Two performance, an increase from about one-fourth tested with Level One skills in years past.

 Table 6.  Academic Profile: Reading Comprehension

  Performance Levels

Spring 1996

       to

Spring 2001

Fall 2001

to

Spring 2002

Fall 2002

to

Spring 2003

Level 3

107 (16%)

  0 (  0%)

17 (13%)

Level 2

377 (56%)

64 (54%)

55 (43%)

Level 1

168 (25%)

45 (38%)

50 (39%)

Below 1

     4 (1%)

  9 (8%) 

  5 (4%)

Subtotal

  656 (97%)

118 (100%)

127(100%)

Reversals

    19

     0

    0

Total Tested

675 (100%)

118 (100%)

127 (100%)

Candidates’ PPST performance as noted in Tables 7.A and 7.B suggest fewer students in need of developmental assistance to strengthen their understanding of what they read that did the Profile in 2002-2003.  The 106 for whom ETS provided test information include 21 prospective candidates whose PPST scores fell below the equivalent of Level Two (20%).  The scores from these 21 students together form the lowest (first) quartile of the distribution of our candidates’ PPST Reading scores (163-174).  

Table 7.A.   Group Summary: PPST Reading Performance

Test Year

Number

Low Score

Median

High Score

25th-75th %tile

02-03

106  (64,019)

163 (150)

180 (178)

188 (189)

175-182 (173-182)

Table 7.B.  Level Two Equivalent Scores, PPST Reading Examination (175 and above)                    

PPST Reading Scores

150-162

163

164

165

166

167

168

169

170

171

172

173

174

175-180

CSB/SJU Examinees

0

1

0

1

1

2

0

1

1

3

1

3

7

85

Anecdotal information gathered from diagnostic tests completed by some of those whose test performance on the PPST-Reading examination fell below Level Two, supplemented by faculty comments on the reading habits of prospective education students enrolled in “foundations” courses, suggests that reading comprehension in general, and inferential understanding of written information in particular, may be emerging as an area of weakness for more of our students. 

Mathematics.  Weaknesses in mathematics performance have long been documented through the test performance of American students.  We might this expect this deficiency to emerge in the performance of many of those same students who later prepare to become teachers.  If so, needed developmental instruction in the use of mathematics could become an important consideration in the selection of all who prepare for teaching careers.

We expect those who prepare for licensure to at least reach Level Two performance in mathematics.  To do so, a student should understand… 

number systems, including order, magnitude, and relationships of integers, functions, and decimals.  A student at this level can solve moderately difficult equations and inequalities, evaluate complex formulas, compare and apply information from more complex charts and graphs, and apply reasoning, geometry, and measurement skills in solving moderately complex intermediate level problems, including word problems.

This proficiency level description was adapted from pages 9 and 10 of The Academic Profile User’s Guide, published in1998 by the College Board and the Educational Testing Service.

Table 8 summarizes student performance in mathematics on the Academic Profile from April 1996 through the spring semester of 2003.  That table suggests that about one-fourth of those seeking acceptance as education majors or minors between April 1996 and March 2001 did not reach Level Two (28%) in mathematics.  A larger portion of those tested during the 2001-2002 (48%) and 2002-2003 (39%) academic years responded to the Profile’s multiple choice questions as if they did not understand or could not use mathematics at Level Two.

Table 8.  Academic Profile: Using Mathematics

 Performance Levels

April 1996

       to

March 2001

Fall 2001

to

Spring 2002

Fall 2002

to

Spring 2003

Level 3

143 (21%)

17 (14%)

20 (16%)

Level 2

316 (47%)

45 (38%)

57 (45%)

Level 1

170 (25%)

53 (45%)

47 (37%)

Below 1

   17 (3%)

  3 (3%)

 3 (2%)

Subtotal

646 (96%)

 118 (100%)

 127 (100%)

Reversals

  29 (4%)

     0

     0

Total Tested

675 100%

 118 (100%)

 127 (100%)

Summaries of our candidates’ performance on the Pre-professional Skills Test provided by the Educational Testing Service for the 2002-2003 test year, abstracted in Tables 9.A and 9.B, finds that six prospective and accepted students of the 104 who completed this test had mathematics test scores below 172 (6%), a score we believe to be the equivalent of Level 1 skill in mathematics.  The lowest one-quarter of mathematics scores fall below the scaled score of 178 on this examination for this test year, representing 20 students (20 of 106, 19%).

Table 9.A.  Group Summary: PPST Mathematics Performance, 2002-2003

Test Year

Number

Low Score

Median

High Score

25th-75th %tile

02-03

104 (65,609)

161 (150)

183 (178)

190 (190)

178-186 (171-183)

Table 9.B.  Level Two Equivalent Scores, PPST Math Examination (173 and above)                    

PPST Math Scores

150-160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

1  7

8

179-190

CSB/SJU Examinees

0

1

0

0

0

0

0

0

0

1

2

1

1

4

1

2

4

3

14

70

Note:  The first quartile in this distribution of scores, from 161 to 177, includes 20 scores representing only 19% of the total distribution.  The next score, 178, includes 14 students. Since the same score cannot be placed in two quartiles, by convention the 14 scores in the interval of 178 are placed in the next highest quartile.

While most of those who completed the Academic Profile during the 2002-2003 academic year also completed the PPST Mathematics exam in that same year, some did not.  Results from these two samples may not be fully comparable.

Additional information on the academic skills of our prospective students as revealed by their Academic Profile test performance appears in the previously noted Academic Skills of Prospective Education Students, 2002-2003.  Information on our students’ Praxis I or PPST basic skills test performance appears in Prospective and Accepted Candidates’ Performance on the Pre-Professional Skills Test, 2000 to 2003.

Academic Skills of Prospective Education Students: 2002-2003

Prospective and Accepted Candidates' Performance on the Pre-Professional Skills Test 2000-2003

Observations.  

This review of information gathered on behalf of candidates’ academic skills recalls the considerable effort many of us have invested in helping students improve those skills..  Such a review should encourage our reflection on policy and practice.  Do academic skills matter in college?  How important are these skills for the liberal and professional education of a prospective teacher?  One need not look too long before encountering a college faculty member lamenting on the reading or writing weaknesses among today’s students.  Some blame the public schools for not preparing those students for college work, while others look toward the passive learners who prefer to absorb images from films and skills from video games. 

Critics of teacher education often point to deficient academic skills of some teachers as evidence of their deficient preparation for their professional roles.  Can some one unable to follow the rules for punctuation, discern the implied meaning of passage from a poem?   Could some one who cannot calculate a percentage be expected to successfully teach such knowledge and skills to others?  While we could counter with the view that teaching and learning are far more complex processes that require a much broader and deeper range of talents and skills than memorized rules or recalled routines, such appeals to complexity fail to blunt the seductively obvious attacks by conservative critics who now control the agenda for reform of education.   

As a department we have advanced the assumption that such skills should be evident in the performance of all candidates.  Further, we have assumed that students blessed with deficient skills can correct their deficiencies if only they will invest enough time and effort in formal courses, work with a tutor, or complete a computer-managed course of study.  We routinely encourage all to work toward the required level of skill and knowledge without much note of the time it might take to do so or the probability of success.  Are these assumptions realistic and appropriate?

Perhaps solving an algebraic inequality, recalling the subtle distinctions in meaning between two similar words, recalling a punctuation rule, or writing an essay that advances a logically organized, well-reasoned, and solidly supported argument are academic skills that can be easily acquired by any motivated student.  But what of those students who do not easily acquire such skills?  Perhaps these skills and understandings are a manifestation of Spearman’s pervasive “g,” a foundational intelligence that supports the activities we ascribe to the formally educated and as such more difficult to develop than we realize.

If academic skills do matter, do some count for more than others?  We have favored verbal skills of reading and writing over mathematical skills.  Is this an appropriate bias?

How much of these more important skills need one have to prepare to become a competent teacher?  Our practice has been to encourage a “basic” Level Two that many of our students do not possess as they begin their study with us and hope for their progress beyond that level toward Level Three. 

In what ways, and for how long, should we encourage those students to pursue the development of deficient skills?  Our practice has been to encourage candidates to pursue development regardless of how long it might take them to reach an accepted level of performance or until they give up the quest. 

Do those who verify a basic level actually practice the skills they affirmed?  Many of us describe our frustration with students who, even after lengthy remediation, continue perform at below our expectations for them.

How can we best improve the academic skills of those candidates whose performance falls below what we believe to be fundamental for academic success?  We searched for parallel indicators of acceptable performance to reduce the number of tests we ask candidates to complete.  In doing so, have we denied the opportunity to acquire needed skills to some who could profit from developmental instruction?   

Assessment Question Two: Do our candidates possess an integrated body of knowledge, skills, and values drawn from one or more of the disciplines central to their areas of licensure?

The first of six standards used by the National Association for the Accreditation of Teacher Education (NCATE) for review of our teacher preparation program notes that…

Candidates preparing to work in schools as teachers…know and demonstrate the content, pedagogical, and professional knowledge, skills, and dispositions necessary to help all students learn.  Assessments indicate that candidates meet professional, state, and institutional standards.

Professional Standards for the Accreditation of Schools, Colleges, and Departments of Education. 2001, page 14.

NCATE further divides the knowledge required of candidates for licensure into three areas.

Teacher candidates have in-depth knowledge of the subject matter that they plan to teach as described in professional, state, and institutional standards.  They demonstrate their knowledge through inquiry, critical analysis, and synthesis of the subject (page 14).

Teacher candidates reflect a thorough understanding of pedagogical content knowledge as delineated in professional, state, and institutional standards.  They have in-depth understanding of the subject matter they plan to teach, allowing them to provide multiple explanations and instructional strategies so that all students learn.  They present the content to students in challenging, clear, and compelling ways and integrate technology appropriately (page 15).

Candidates reflect a thorough understanding of professional and pedagogical knowledge and skills delineated in professional, state, and institutional standards as shown in their development of meaningful learning experiences to facilitate student learning for all students.  They reflect on their practice and make necessary adjustments to enhance student learning.  They know how students learn and how to make ideas accessible to them.  They consider school, family, and community contexts in connecting concepts to students’ prior experience, and in applying the ideas to real-world problems (page 15).

Standard One also explores “dispositions” or value-oriented beliefs of those we prepare for teaching as well as their ability to encourage learning in the students they teach.  These areas are not directly examined through the Praxis II test series and not summarized for this report.    

The knowledge and skills of prospective teachers is also a concern of Minnesota’s Board of Teaching.  All candidates licensed as teachers must complete and pass a test of content knowledge for their subject area and a test of teaching knowledge appropriate for the grade levels within that area.  The Board has selected the Praxis II series of licensure examinations developed by the Educational Testing Service to enable candidates to meet this requirement.   These national recognized tests are “aligned” by weighting or excluding some items so that scaled scores awarded to examinees better reflect Minnesota’s content and pedagogy standards.

Elementary Candidates’ Content Knowledge.  The following table summarizes the performance of Saint Benedict’s and Saint John’s candidates for elementary licensure on the Praxis II content examination for grade K-6 licensure.  Parenthetical entries reveal the performance of all who completed this examination in the United States between 1 September and 31 August of each test year.   

Table 10.  Elementary Candidates’ Content Test Performance

Test Year

Number

Low Score

Median

High Score

25th-75th %tile

01-02

75 (27,205)

132 (100)

166 (163)

196 (200)

157-176 (150-175)

02-03

40 (25,994)

135 (100)

170 (162)

195 (200)

161-176 (149-175)

The 75 candidates who completed their content tests in the 2001-2002 test year earned a median scaled score of 166 as contrasted with the median score of 163 earned by all 27,205 test takers nationally.  The following year, 40 elementary candidates earned a median score of 170 while the 25,994 who completed this test nationally earned a median score of 162. 

Our candidates’ lowest scores appear to be well above the lowest observed score in the national sample, while the highest score earned by our sample of candidates in each test year fell just below the highest national score.  The score range for the middle one-half of our candidate samples is somewhat higher than the range for each of the national samples.  Overall, this pattern suggests good performance that equals or exceeds the national sample of all tested elementary candidates for licensure.

A closer look at our candidates’ core content sub-scores may tell us more.  ETS offers additional information describing performance on the categories of elementary level content knowledge included in this exam.  The following table reveals the mean or “average” percent of items correctly answered by our candidates, by all those who took the test in Minnesota, and by all who completed this examination in the United States during each of the two years.  Differences (+ or - %) between the CSB/SJU mean percent correct and that of all Minnesota candidates appear under the “CSB/SJU” column.  A similar set of observed differences between the Minnesota and the national samples is included in parentheses under the Minnesota column. 

Table 11.  Elementary Candidates’ Mean Percent Correct by Praxis II Content Area

Content Area

Year

Points

CSB/SJU

Minnesota

U. S.

Language Arts

01-02

30

77%  (+2)

75%  (+1)

74% 

 

02-03

30

82     (+3)

78     (+4)

74

Mathematics

01-02

30

78     (0)

78     (+7)

71

 

02-03

30

82     (+3)

79     (+8)

71

Social Studies

01-02

29

66     (- 2)

68     (+2)

66

 

02-03

29

67     (- 3)

70     (+1)

69

Science

01-02

30

64     (- 6)

70     (+3)

66

 

02-03

30

65     (- 5)

70     (+3)

67

This analysis suggests that more of our elementary candidates as a group correctly answered Praxis II questions affirming their knowledge of language arts and mathematics to a greater degree than did their Minnesota peers and the national sample.  Knowledge of social studies and general science as indicated by the mean percentage of correctly answered questions during each test year, however, suggest performance below that of other Minnesota elementary candidates and equal to or below national means for this indicator.  This pattern, should it persist, may require curricular review.

Additional patterns may emerge if we can look more carefully at the distribution of scores within each content area.  Table 12 presents a distribution of our candidates’ Praxis scores in one-quarter segments (quartiles).  These quartiles are derived by ETS from the range of all Praxis elementary content scores calculated for each content area and test year.  Since the quartile limits are based on national samples, differences between our candidates’ test performance and that of candidates included in those larger samples can be instructive.     

Table 12. Distribution of Elementary Candidates’ Praxis II Content Examination Scores

Content Area

Year

1st Quartile

2nd Quartile

3rd Quartile

4th Quartile

Tested

Language Arts

01-02

  8 (11%)

28 (37%)

24 (32%)

15 (20%)

75 (27,205)

 

02-03

  2 (  5)

  9  (23)

12 (30)

17 (43)

40 (25,994)

Mathematics    

01-02

  7 (  9)

16 (21)  

24 (32)

28 (37)

75

 

02-03

  2 (  5)

  8 (20)

10 (25)

20 (50)

40

Social Studies

01-02

10 (13)

27 (36)

28 (37)

10 (13)

75

 

02-03

  9 (23)

14 (35)

10 (25)

  7 (18)

40

Science

01-02

22 (29)

21 (28)

20 (27)

12 (16)

75

 

02-03

12 (30)

12 (30)

  9 (23)

  7 (18)

40

Note: Quartiles are set by ETS to capture the range of scores awarded to those candidates in the national sample for each Praxis test administered during each test year.  Scores earned by an institution’s candidates may thus not replicate the national distribution of 25% of all scores in each quartile.      

Looking to the language arts content area as noted in Table 12, we find that eight of our 75 candidates (11%) tested in 2001-2002 earned scores falling into the lowest one-quarter of this distribution of all Praxis test scores.  Fifteen of the 75 CSB and SJU candidates (20%) who completed their elementary content test during the 2001-2002 testing year earned scores which placed them in the top one-quarter of the scores received by the 27,205 who completed this test in that year.   More students earned scores in mathematics which fell into the highest national quartile for both test years, a strong pattern in a traditionally weaker area of content knowledge.

Our elementary candidates’ knowledge of social studies, as revealed by their performance on this test, may be weaker for 28 of the 75 candidates tested in 2001-2002 whose scores in that content area fell into the third quartile (37%).  Knowledge of general science appears to be the weakest of the four tested content areas for those completing this examination, with about 30% earning scores falling into the lowest national quartile for each test year (22 or 29% of 75 in 01-02; 12 or 30% of 40 in 02-03).  Should this pattern be affirmed, review of courses and field experiences that form these core areas would be warranted.  

Elementary Education Candidates’ Pedagogical Knowledge.  Minnesota requires those seeking licensure as elementary teachers of students enrolled in kindergarten through grade six to verify the extent of their knowledge of how to teach the facts, concepts, principles, and skills included in their licensure area.  Table 13 summarizes the performance of Saint Benedict’s and Saint John’s candidates for elementary licensure contrasted with, in parentheses, the performance of all who completed the Praxis II Principles of Teaching and Learning examination between 1 September and 31 August of each test year.

Table 13. Elementary Candidates’ Praxis II Pedagogical Knowledge Scores

Test Year

Number

Low Score

Median

High Score

25th-75th %tile

01-02

77 (34,640)

136 (100)

172 (173)

200 (200)

162-181 (165-180)

02-03

42 (29,329)

150 (102)

180 (175)

192 (200)

172-185 (166-182)

The 77 CSB and SJU students who completed this test in 2001-2002 earned a median score of 172 compared with a median score of 173 for the 34,640 in the national sample of examined candidates, while the highest score for our candidates in this year, 200, equaled the highest observed score in the national sample.  The median score for 2002-2003 exceeded the national median by five points (180).  These scores suggest a pattern of group performance for our elementary candidates that similar to that of the larger group completing this test each year.

All Praxis II “Principles of Learning and Teaching” (PLT) examinations follow the same general format for each grade level cluster, but shift the focus of test items to reflect those clusters (K-6, 5-9, and 7-12).   Each form of the test includes multiple choice “fixed response” test items, constructed or “free response” short answer questions.  Current versions also include case studies describing classroom situations in terms of teacher actions and student behavior appearing either in narrative form or as a set of documents to be analyzed to developing an extended response.   

Table 14 reveals the mean percent correct on each of several skill areas earned by our elementary candidates contrasted with all who took this test in Minnesota and with all who completed this examination in the United States for each of the two most recent test years. 

Table 14.  Elementary Candidates’ Mean Percent Correct on Skill Areas Included in the

                  Praxis II Principles of Learning and Teaching.

K-6 Pedagogical Skill Areas

Year

Points

CSB / SJU

Minnesota

U. S.

Students as Learners: Fixed Response

01-02

12

73%  (0)

73%  (+2)

71%

 

02-03

  8

71     (- 2)

73     (+3)

70

Instruction and Assessment: Fixed

01-02

11

70     (- 1)

71     (+3)

68

 

02-03

  8

76     (+4)

72     (+1)

71

Teacher Professionalism: Fixed

01-02

11

68     (- 4)

72     (+1)

71

 

02-03

  8

84     (- 2)

86     (+4)

82

Teacher Professionalism: Free Response

01-02

  8

79     (+1)

78     (+4)

74

All Skills: Free

01-02

36

71     (0) 

71     (  0)

71

Students as Learners: Case Histories

02-03

16

79     (+5)

74     (+3)

71

Instruction and Assessment: Cases

02-03

16

72     (- 2)

74     (+3)

71

Communication Techniques: Cases

02-03

  8

83     (+4)

79     (+3)

76

Teacher Professionalism: Cases

02-03

  8

81     (+2)

79     (+3)

76

Candidates who completed their “Principles of Learning and Teaching” examination during each of the two test years included in Table 14 appear to have a weaker understanding of “Teacher Professionalism” than do their Minnesota peers.  Occasional deviations for either test year in “Students as Learners” and “Instruction and Assessment may warrant further review should they continue or appear in other relevant indicators.  Our 2002-2003 candidates’ mean scores exceeded those of both their state and national peers, however, for case histories in “communication techniques” and “teacher professionalism,” but not in “Instruction and Assessment.”

Looking more closely at how our candidates’ scores compare with a national sample of elementary candidates, as revealed in Table 15, finds “teacher professionalism” to be a weaker area among the several included in this examination.   

Table 15.  Distribution of Elementary Candidates’ Praxis II Pedagogy Scores by Skill Area

K-6 Skill Areas

Year

1st Quartile

2nd Quartile

3rd Quartile

4th Quartile

Tested

Students as Learners: Fixed Response Items 

01-02

02-03

12 (16%)

  8 (19)

28 (37%)

15 (36)

24 (32%)

16 (38)

12 (16%)

  3 (7)       

76

42

Instruction and Assessment : Fixed      

01-02

02-03

10 (13)

  5 (12)

22 (29)

14 (33)

25 (33)

21 (50)

19 (25)

  2 (5)

76

42

Teacher Professionalism: Fixed

01-02

02-03

20 (26)

  6 (14)

20 (26)

17 (40)

20 (26)

19 (45)

16 (21)

  0 (0)

76

42

Teacher Professionalism: Free

 

01-02

 

10 (13)

 

20 (26)

 

32 (42)

 

14 (18)

 

76

All Skills: Free

01-02

17 (22)

26 (34)

17 (22)

16 (21)

76

Students as Learners: Case Histories

 

02-03

 

  5 (12)

 

  9 (21)

 

11 (26)

 

17 (40)

 

42

Instruction & Assessment: Cases

 

02-03

 

  9 (21)

 

12 (29)

 

18 (43)

 

  3 (7)

 

42

Communication Techniques: Cases

 

02-03

 

  5 (12)

 

  7 (17)

 

14 (33)

 

16 (38)

 

42

Teacher Professionalism: Cases

 

02-03

 

  4 (10)

 

12 (29)

 

26 (62)

 

  0 (0)

 

42

Scores from our elementary candidates included in this table suggest that more of those who completed their “Principles of Teaching and Learning” examination during the 2001-2002 test year earned higher sub-scores than did their CSB/SJU peers who completing the test in the following year.  Looking only at the three sub-scales common to both versions of the tests used during these two years, we find a greater portion of those tested in the first year in each of the three areas common to both tests appears in the fourth quartile than is the case for those of our candidates tested in 2003-2003; 16% versus 7% for knowledge of “Students as Learners,” 25% versus 5% for “Instruction and Assessment,” and 21% versus 0% for understanding “Teacher Professionalism.”

Moving to the other end of this distribution of sub-scale scores, about one-fourth of the 2001-2002 tested candidates earned scores in the first quartile for their knowledge of “Teacher Professionalism” (20 of 76, 26%), while fewer from the second year share that category with them (02-03: 6 of 42, 14%).  A smaller portion of those completing the examination had sub-test scores that fell into the fourth quartile for each year. This appears to be the weaker of the three core areas. 

By definition we should expect that 25% of the scores earned by all who completed this test would be included in the first quartile (observed scores of 100 to 165 from 34,640 tested in 01-02; scores of 100 to 166 for 29,329 tested in 02-03).  Scores earned by our candidates, however, need not reflect this national distribution.  Where we to provide our candidates with refined  opportunities to learn and practice the skills included in “teacher professionalism,” we might expect to see scores moving toward the fourth quartile

Secondary Social Studies Candidates’ Content Knowledge.  The Board of Teaching selected the Praxis II social studies subject matter examination to enable candidates in this licensure area to verify the extent of their knowledge.  The wide range of topics and disciplines folded into “social studies” requires candidates in this area, perhaps more so than those prepared for other licensure areas, to acquire and to be able to share an integrated understanding of their field of study.  That understanding must reflect both deeper and broader knowledge of each of the eight disciplines that contribute in some way to middle and high school students’ experiences in “social studies.”  Greater demands on social studies teachers would seem the necessary outcome, despite the field’s reputation as a haven for those who seek additional roles in secondary schools.  

A comparison of our students’ performance with a larger national population of tested social studies candidates reveals a general trend toward lower performance.  During both test years our candidates’ median and high scores lag their national peers.  Lower median scores may hint at areas of weakness within one or more of the several content areas included in this licensure test.

Table 16.  Social Studies Candidates’ Praxis II Content Test Performance

Test Year

Number

Low Score

Median

High Score

25th-75th %tile

01-02

 16 (  9,724)

137 (104)

160 (166)

180 (200)

150-172 (155-177)

02-03

 10 (11,059)

145 (107)

158 (165)

173 (200)

151-170 (155-177)

The middle range of scores (25th to 75th percentile) earned by those of our candidates who completed their social studies examination in both test years includes scores lower than the national sample’s low scores (150 versus 155 and 151 in 01-02 versus 155 for 02-03).   The high scores for our candidates within this middle range are also below those earned by their national peers.  This pattern may foreshadow a pattern of weaker test performance than might be evident for other licensure examinations completed by our secondary candidates.

ETS offers additional information through a comparison of the mean percent correct on this test earned by our secondary social studies candidates with the performance of all those who took the test in Minnesota as well as with all who completed this examination in the United States each year.

Table 17. Social Studies Candidates’ Mean Percent Correct on Content Areas Included

                 in the Praxis II Subject Matter Test

Social Studies Content Areas

Year

Points

CSB/SJU

Minnesota

U. S.

 U.S. History

01-02

29

59% (-  1)

60%  (- 4)

64%

 

02-03

29

55    (-  6)

61     (- 4)

65

World History

01-02

29

55    (-  7)

62     (+ 1)

61

 

02-03

29

43    (-15)

58     (- 3)

61

Political Science

01-02

21

69    ( 0)

69     (0)

69

 

02-03

21

58    (-  7)

65     (0)

65

Geography

01-02

20

75    (0)

75     (+ 7)

68

 

02-03

20

64    (-  5)

69     (+ 3)

66

Economics

01-02

19

63    (-  1)

64     (+ 5)

59

 

02-03

20

57    (-  4)

61     (+ 3)

58

Behavioral Sciences

01-02

13

70    (-  8)

78     (+ 4)

74

 

02-03

13

71    (-  1)

72     (+ 6)

66

The mean percentage correct earned by our candidates in both test years is lower for several of the social science test’s content areas.  While some of the differences between the scores earned by our candidates and those of other Minnesota candidates are small (-1), this table suggests that United States history and world history are two content areas with substantial discrepancies.  Relative to students prepared at other Minnesota colleges, our candidates earned lower mean scores in both United States history (55% correct versus 61% (-6%) for all Minnesota candidates in 2002-2003) and world history (7% below the Minnesota mean percent correct score for 2001-2002, 15% below our state mean for the 02-03 year).

Those of our candidates who were tested in 2002-2003, a year when passing test scores were required for licensure, performed below our state mean in political science (-7%), geography (-5%), and economics (-4%).  The mean percent correct for those tested in 2001-2002 in the behavioral sciences, a minor theme in the social studies examination, fell eight percent below their Minnesota peers (70% versus 78%).  Those completing their social studies content tests during the following year, however, reached a level of test performance nearly equal to their Minnesota peers with respect to their knowledge of the behavioral sciences.

A closer look at the distribution of our candidates’ scores by national quartile for each content area included in the Praxis II social studies examination may clarify this trend.  ETS set the quartiles to reflect the performance of the 9,724 who completed this test during 2001-2002 and the 11,059 who did so during the following test year.  While the small number of our candidates included in this distribution can hinder analysis, this table appears to confirm that our social studies candidates may have responded to questions included in this licensure test as if they could recall less about world and U.S. history than many candidates prepared by other Minnesota colleges and universities.

Table 18.  Distribution of Social Studies Candidates’ Praxis II Content Examination Scores by Subject Area

Subject:

Year

1st Quartile

2nd Quartile

3rd Quartile

4th Quartile

Tested

U.S History

01-02

  7 (44%)

  3 (19%)

  5 (31%)

  1 (6%)

16

 

02-03

  5 (50)

  4 (40)

  1 (10)

  0 (0)

10

World History    

01-02

  4 (25)

  6 (38)

  6 (38)

  0 (0)

16

 

02-03

  8 (80)

  2 (20)

  0 (0)

  0 (0)          

10

Political Science

01-02

  2 (13)

  9 (56)

  4 (25)

  1 (6)

16

 

02-03

  4 (40)

  4 (40)

  1 (10)

  1 (10)

10

Geography

01-02

  3 (19)

  4 (25)

  5 (31)

  4 (25)

16

 

02-03

  2 (20)

  4 (40)

  3 (30)

  1 (10)

10

Economics

01-02

  2 (13)

  4 (25)

  4 (25)

  6 (38)

16

 

02-03

  2 (20)

  4 (40)

  1 (10)

  1 (10)

10

Behavioral Sciences

01-02

  1 (6)

12 (75)

  3 (19)

  0 (0)

16

 

02-03

  1 (10)

  3 (30)

  4 (40)

  2 (20)

10

Looking at the first quartile, the pattern emerging in Table 17 seems to be confirmed.  Seven of the 16 who completed their social studies content test in 2001-2002 (44%) earned scores on the U.S. history sub-test which place them in the bottom 25% of the national distribution of all such scores. 

One-half of those tested during the following year also earned sub-scores falling into the same quartile (5 of 10, 50%).  While only one-forth of the candidates tested in the first year earned first quartile scores in world history (4 of 16, 25%),  eight of the ten tested during 2002-2003 reached this category (80%).   

There are also some stronger areas or performance in this score distribution.  More than one-third of those tested in 2001-2002 earned world history test scores that placed them in the third quartile of all social studies examinees in that year (6 of 16, 38%), although none reached the fourth quartile.  Looking at economics, as many tested that year earned sub-scores reaching into that fourth quartile (6 of 16, 38%).  Sub-scores from nearly one-third of these same 16 candidates reached the third quartile in U.S. history (5 of 16, 31%), helping to off-set the performance of their colleagues in the first quartile (7 of 16, 44%).

Other indicators should be examined to affirm the possible weaknesses enjoyed by those candidates seeking social studies licensure.  Those indicators may help us better understand how the experiences of our candidates may support their lower than expected performance.  A key question for such a review, difficult to answer without item analysis data, might focus on the congruence of licensure standards, courses and fieldwork designed to help candidates meet those standards, and the design of the licensure test. 

Secondary Education Candidates’ Pedagogical Knowledge.  Successful classroom teachers must join their knowledge of what to teach with their skills in how to teach the knowledge, skills, and values that states, districts, and their own professional standards might require.  The Board of Teaching and the Educational Testing Service used expert review of test items to “align” the Praxis II “Principles of Teaching and Learning” examinations to these standards through differential weighting of selected items and the deletion of others.

Most candidates for secondary licensure choose to complete the Praxis II examination of “Principles of Learning and Teaching” developed for teachers of grades 7 through 12.  The following table summarizes group performance on this test relative to all who completed it during the 2001-2002 and 2002-2003 test years.

Table 19: Secondary Candidates’ Praxis II 7-12 Pedagogy Test Performance

Test Year

Number

Low Score

Median

High Score

25th-75th %tile

01-02

27 (21,614)

162 (100)

175 (172)

186 (200)

170-179 (165-179)

02-03

28 (21,212)

156 (104)

173 (171)

195 (200)

166-177 (164-178)

ETS offers additional information describing knowledge of relevant pedagogical skills as revealed in candidates’ test performance.  The following table reports the mean percent correct for our candidates, for all those who took the test in Minnesota, and for all who completed this examination in the United States for each test year.  During the 2002-2003 test year ETS changed the format and number of items included in this examination, adding case histories and short answer items while deleting some fixed response questions, adjusting scoring to provide some comparability between the two versions.

Table 20.  Secondary Candidates’ Mean Percent Correct on Skill Areas Included in the Praxis II  Principles of Learning and Teaching (7-12) Examination

7-12 Pedagogical Skill Areas

Year

Points

CSB/SJU

Minnesota

U. S.

Students as Learners: Fixed Response

01-02

14

67%  (0)

67% (+ 4)

63%

 

02-03

14

73     (0)

73    (0)

70

Instruction and Assessment: Fixed

01-02

12

84     (+2)

82    (+4)

78

 

02-03

11

71     (- 3)

74    (+4)

70

Teacher Professionalism: Fixed

01-02

14

71     (- 3)

74    (+3)

71

 

02-03

14

78     (- 4)

82    (+5)

77

Teacher Professionalism: Free Response

01-02

8

81     (- 7)

86    (+5)

81

All Skills: Free

01-02

36

70     (+1)

69    ( 0)

69

Students as Learners: Case Histories

02-03

16

58     (0)

58    (+1)

57

Instruction and Assessment: Cases

02-03

36

69     (+2)

67    (+3)

64

Communication Techniques: Cases

02-03

8

73     (- 2)

75    (+3)

72

Teacher Professionalism: Cases

02-03

8

66     (- 4)

70    (+3)

67

Our candidates’ earned a mean percentage correct for “Students as Learners” that equaled those of their Minnesota peers for both test years (67% in 01-02; 73% in 02-03).   Results across the two test years are mixed for “Instruction and Assessment,” with those tested in the first year surpassing their peers by 2%, while those tested in 2002-2003 fell behind others tested in Minnesota (-3%). 

Concepts and principles that form the “Teacher Professionalism” skill area emerged as a weakness for those tested during both years (-3% in 01-02 followed by -4% in 02-03).  While not comparable due to changes in test format, candidates answering short answer questions on this topic fell behind their Minnesota peers by 7% in 2001-2002.  The case histories in this area completed by candidates during the following test year fell four percent below others tested in Minnesota that year (66% for CSB/SJU, 70% for Minnesota).

A closer look at the distribution of our candidates’ scores may clarify their relative understanding of each skill area.  ETS set the quartile limits to reflect the performance of all who completed this examination during each test year (21,614 in 01-02 and 21,212 in 02-03).   Table 21 provides this information.

Table 21. Distribution of Secondary Candidates’ Praxis II Principles of Learning and Teaching Scores by Skill Area

7-12 Pedagogy Skill Areas

Year

1st Quartile

2nd Quartile

3rd Quartile

4th Quartile

Tested

             Students as Learners:

Fixed Response Items 

01-02

02-03

  3 (11%)

  4 (14)

  6 (22%)

13 (46)

15 (56%)

  8 (29)

  3 (11%)

  3 (11)

27

28

Instruction and Assessment:  Fixed Response Items       

01-02

02-03

  2 (7)

  6 (21)

  3 (11)

10 (36)

21 (78)

12 (43)

  1 (4)

  0 (0)

27

28

Teacher Professionalism: Fixed Response Items

01-02

02-03

  7 (26)   )

  3 (11)

  8 (30)

13 (46)

11 (41)

  9 (32)

  1 (4)

  3 (11)

27

28

Teacher Professionalism:

 Free Response Items

 

01-02

 

  1 (4)

  5 (19)

 

16 (59)

 

  5 (19)

 

27

All Skill Areas:

 Free Response Items:

 

01-02

 

  4 (15)

 

12 (44)

 

  7 (26)

 

  4 (15)

 

27

Students as Learners:

Case Histories

 

02-03

 

  4 (14)

 

11 (39)

 

  6 (21)

 

  7 (25)

 

28

Instruction & Assessment: Case Histories

 

02-03

 

  1 (4)

 

10 (36)

 

  9 (32)

 

  8 (29)

 

28

Communication Techniques: Case Histories

 

02-03

 

  5 (18)

 

10 (36)

 

  4 (14)

 

  9 (32)

 

28

Teacher Professionalism: Case Histories

 

02-03

 

  5 (18)

 

13 (46)

 

  8 (29)

 

  2 (7)

 

28

A complex pattern emerges from this distribution. In general, the test performance of all candidates seeking secondary licensure favors the second and third quartiles over the first. Concepts included in “Teacher Professionalism” may have been poorly understood or incompletely recalled by about one fourth of our candidates as revealed by their sub-scale scores in 2001-2002 (1st quartile: 7 of 27 tested, 26%; 2nd quartile: 8 of 27, 30%).  More of those tested the following year earned scores falling into the second quartile for their knowledge of this same area (both fixed response and case history item categories, 13 of 28 tested, 46%). 

About one-half of the secondary candidates who completed this test in 2002-2003 earned sub-scale scores for “Students as Learners” (13 of 28 tested, 46%) and Teacher Professionalism (fixed response and case histories: 13 of 28, 46%), placing their test scores in the second quartile.  Sub-test scores for nearly one-third reached the fourth quartile for case histories of “Communication Techniques” (9 of 28, 32%).

A review of Licensure Candidates’ Performance on Praxis II Examinations: 2001-2002 and 2002-2003 offers additional information on the design of relevant Praxis examinations and the performance of those of our candidates who completed them.

Lisensure Candidates' Performance on Praxis II Examinations: 2001-2002 and 2002-2003


Observations.

Amidst strengths in their knowledge of how to teach their students as suggested by their Praxis II performance and confirmed by our assessment of their work as student teachers, our elementary candidates’ weakness two areas that will inform their practice deserves further exploration.  Elementary candidates’ apparent weakness in their knowledge of science is surprising in light of the investment our colleagues have made in developing and offering innovative courses designed to help students succeed in this area.  Perhaps our “hands on, minds on” approach to teaching elementary-level science, thought by many to be the best approach for younger learners, is not congruent with the design of the licensure examination.  Pervasive “science anxiety” may still hinder candidates’ ability to learn.  What steps can we take in the year ahead to better understand and perhaps begin to strengthen students’ science knowledge? 

Knowledge of the social science disciplines that inform social studies also emerged as a weaker area in the Praxis II test performance of our elementary candidates.  Given the emergence of Minnesota’s new social studies standards for K-6 students and the controversy surrounding their adoption, can we look more carefully at the patterns of courses and experiences that will support our candidates’ efforts to integrate those new standards into their teaching? 

Social studies knowledge emerged again as a significant weakness in the preparation of our secondary candidates preparing for licensure in that area.  Our curricular design for this licensure program may not provide sufficient opportunities to work in some of the disciplines formed into social studies, including world history.  Does the course of study we advise our secondary social studies candidates to pursue follow the design of the Praxis II test?  Our candidates, as they take one or more courses in each of the social science disciplines, may not be developing an integrated understanding of how those disciplines fit together to encourage a richer, more complex image of our world.  Can we find a way to balance breadth of knowledge with depth of understanding that fits the expectations set by licensure standards for this program?

D. Leitzman

July 2004

 

Appendices

Appendix 1.  Board of Teaching Reports

1. A.  Reading Strategies.   The Minnesota Board of Teaching, responding to an legislative initiative, required all approved teacher preparation programs to provide additional evidence of how their candidates learned about, practiced, and were assessed on their use of “scientifically proven” and “research-based” reading strategies. 

Minnesota Statute 2002, 122A.18 Subdivision 2a, Reading Strategies:

a) All colleges and universities approved by the board of teaching to prepare persons for classroom teacher licensure must include in their teacher preparation programs reading best practices that enable classroom teacher licensure candidates to know how to teach reading, such as phonics or other research-based best practices.

b) Board-approved teacher preparation programs for teachers of elementary education must require instruction in the application of comprehensive, scientifically based, and balanced reading instruction programs.

We prepared an extensive review of our efforts to provide elementary and secondary candidates with a “balanced” approach to reading instruction which included opportunities to discover, use, and be assessed on phonics, whole language, and other relevant methods of encouraging children to read.  The following table is drawn from our report 2 January report to the Board.

Reading Strategies

 

Required

Courses

Learning Opportunities

a) Reading “best practices” that enable all classroom teacher licensure candidates to know how to teach reading, such as phonics or other research-based best practices.

 

EDUC 354 Middle Level Literacy and Pedagogy

(All K-12 

and 5-12 candidates)

Content Area Literacy: Days 13 and 14, (see course syllabus, pages 4 & 5)

 -Reading Comprehension: levels, processes, influences, strategies

 -Vocabulary Development: content area instruction, teaching strategies

Assessed via group tasks and SEP-based portfolio (see course assignments, pages 3 & 7)

English Language Learners and Literacy: Day 15  (syllabus, page 5)

 -Optimal Learning Environment

 -Strategies for Optimal Learning and Interacting

Assessed via analysis of Culture Day Lesson (see assignments, pages 1and 2)

Licensure Standards: Candidates seeking a license to teach middle level learners support their preparation by exploring the meaning and value of content area literacy within a context of a developmentally responsive pedagogy (3.B.2, 3.B.3; Syllabus, Days 12, 13, and 14).  Candidates’ reflections on middle level learners reading comprehension in content areas introduce the need to “use a variety of appropriate strategies, techniques, and skills for developing comprehension” in those areas (3.B.5).  Observations in a middle level school (Practicum 1, Assignments, p.2-3), an interdisciplinary “Culture Day” lesson prepared with peers and taught to middle level learners (Assignments, p. 1-2) and ongoing reflections concerning SEP 4 (Assignments, p. 3) provide evidence of “use of developmentally appropriate techniques for augmenting the listening, speaking, reading, and writing vocabularies” of  middle level learners (3.B.4). 

NB: This course invites all students preparing to teach middle level learners to explore ways of adapting pedagogical content knowledge from their disciplines to better meet the emerging needs and talents of middle level learners (grades 5-8).  Candidates encounter an integrated approach to the review, selection, and use of best practices in literacy (reading, writing, vocabulary development) with an emphasis on how they can use those practices to enhance learning in their content areas.  They test the efficacy of these practices in lesson and unit plans.  Current and emerging literacy methods are reviewed so that they might be incorporated in candidates’ plans and reflections, with an emphasis on how those techniques might best fit NCTE and state standards.

 

1.B.  Faculty Experience in K-12 Teaching.  House File 934 became law during the Minnesota Legislature’s 2003 session as an amendment to the state’s higher education bill. 

The Minnesota Association of Colleges of Teacher Education is requested to collect data from each of its member institutions that measure the involvement of teacher education programs and their faculty with Minnesota K-12 schools.  The data shall include at least:  current Minnesota licensure status of faculty, K-12 teaching experience of college faculty under that licensure within the last five years, descriptions of college and faculty collaborations with K-12 teachers and students, and information on other projects involving higher education in K-12 schools. The data shall be presented to the education policy and finance committees of the legislature by February 15, 2004.

Working together with Minnesota’s other teacher preparation programs through the Minnesota Association for Colleges for Teacher Education (MACTE), a voluntary state-wide federation of schools, colleges, and departments engaged in teacher preparation, we provided extensive information on the K-12 licensure status of our faculty and staff and their active participation as consultants, volunteers, and supports of public education.   Information about MACTE and a summary of our joint report to the legislature is available at http://www.mnteachered.org/.


Appendix 2.  U. S. Department of Education: Title II Report

Teacher Candidates’ Licensure Examination Performance:  2002-2003 Academic Year

The federal government’s Higher Education Act of 1998 requires all institutions preparing teachers for public schools to publish the performance of their graduates on all examinations required for licensure by the states in which those colleges and universities are located.  Minnesota’s Board of Teaching compelled all who sought licensure as elementary or secondary teachers to successfully demonstrate their academic skills using the Pre-Professional Skills Test (PPST or PRAXIS I), to verify their knowledge of how to instruct students in the subject they will be licensed to teach (PRAXIS II: Principles of Teaching and Learning), and to affirm their knowledge of that subject (PRAXIS II: content knowledge).  The following table summarizes performance on these tests for the 66 women and men completing our program during the 2002-2003 academic year. 

Aggregated Licensure Examination Performance:  2002-2003

                           CSB / SJU                    

MINNESOTA

 

Assessment

Students Assessed

Students

Passing

Pass Rate

Students Assessed

Students Passing

Pass Rate

Basic Skills

66

66

100%

3590

3473

97%

Professional Knowledge

66

65

98%

3564

3535

99%

Content   Knowledge

62

61

98%

2951

2908

99%

Totals

66

64

97%

3757

3601

96%

Note:  Passing scores were not set for candidates seeking licensure as teachers of foreign languages to students in grades K-12.  Four candidates who completed their preparation for licensure in this area through our program and completed the appropriate examination are thus not included in this tabulation.  Totals for Minnesota include candidates who sat for and passed more than one content or pedagogy examination.


Appendix 3.  Accreditation Reports: AACTE and NCATE Annual Reports

3.A.  AACTE Professional Educational Data System: Part A

Part A recalls the major administrative features of our two colleges for the American Association of Colleges for Teacher Education: Mailing addresses, leaders, degree programs, institutional control and classification, accrediting groups, and contact persons for follow-up information.  A copy is in the Education Department for review.

3.B.  AACTE Professional Educational Data System:  Part B

Part B of our annual AACTE report, on file in the Education Department, summarizes student enrollment in our colleges and our licensure programs.  It also includes information on the number and assignments of our faculty.  The following tables are reproduced from our 2003 report.

Table 3.B.1.  Undergraduate Students Whose Major Field of Study was Education

                     As of 15 October 2002

Racial or Ethnic Group

Current Year: 2002-2003

Women              Men

Prior Year: 2001-2002

Women          Men

Asian/Pacific Islander

 

         1

 

 

White, non-Hispanic

   175

       41

    212

     36

Hispanic

       2

         1

 

 

International (non-resident alien)

 

         7

 

 

TOTALS:

   177

       42

    220

      36

Table 3.B.2.  Undergraduate Students Whose Major Field of Study Was NOT Education

                      But Who Were Formally Accepted as Candidates Preparing for Licensure

                      As of 15 October 2002

Racial or Ethnic Group

Current Year: 2002-2003

Women             Men

Prior Year: 2001-2002

Women          Men

Asian/Pacific Islander

      1

 

 

 

White, non-Hispanic

    88

        1

     92

 

Hispanic

 

      56

 

    74

International (non-resident alien)

      1

 

       1

 

TOTALS:

    90

      57

     93

    74

Table 3.B.3  Degrees Awarded by Licensure Program and Candidate Groups; 2001-2002

 

Black

White

Unknown

Program Totals

Licensure Programs

Men 

Women

Men

Women

Men

Women

Men

Women

Art

 

 

 

 

 

 

 

 

Biology

 

 

 

 

 

1

 

1

Chemistry

 

 

 

 

1

 

1

 

Elementary

 

1

7

32

 

 

7

33

English

 

 

3

1

 

 

3

1

Mathematics

 

 

3

1

 

 

3

1

Music

 

 

2

5

 

 

2

5

Physics

 

 

1

 

 

 

1

 

Gen Science

 

 

2

1

 

 

2

1

Social Studies

 

 

2

5

 

 

2

5

Foreign Languages

 

 

 

 

 

 

 

 

Education Studies

 

 

4

 

 

 

4

 

Group Totals

 

1

24

45

1

1

25

47

Table 3.B.4.  Full Time Faculty in Professional Education

 

Tenured Faculty

Tenure Track

Casual

Academic Rank

2002-03

2001-02

2002-03

2001-02

2002-03

2001-02

Professors

4

4

 

 

 

 

Associate Professors

3

2

1

 

 

 

Assistant Professors

 

 

 

3

 

 

Instructors

 

 

 

 

1

1

No Rank

 

 

 

 

5

4

TOTALS

7

6

1

3

6

5

Table 3.B.5.  Undergraduate Teaching Load  

 

2002-2003

2001-2002

Full Time Faculty

       14

       14

Total Credit Hours Taught

     149

     123

Total Courses Taught

       39

       30

3.C  Professional Education Data System: Part C.  Each year we add a narrative summary of our work on each of the six standards that guide our voluntary accreditation by the National Council for the Accreditation of Teacher Education.  The following report was included as “Part C” of our 2003 Professional Education Data System summary.

Section 1: Institutional Information

Institution: College of Saint Benedict and Saint John’s University

Unit: Education Department

Next Accreditation Visit: Fall 2005

Last Accreditation Visit: Spring 2001

Deadline to submit final version of Part C: 1 October 2003

Section 2: Individual Contact Information

Corrected CEO: Henry Smorynski, Provost

Corrected CEO Phone: 320-363-5503

Corrected CEO Fax: 320-363-5050

Corrected CEO Email: hsmorynski@csbsju.edu

Section 3: NCATE Standards Categories and Weaknesses

Section A: Conceptual Framework. The conceptual framework establishes the shared vision for a unit’s efforts in preparing educators to work effectively in P-12 schools.  It provides direction for programs, courses, teaching, candidate performance, scholarship, service, and unit accountability.  The conceptual framework is knowledge based, articulated, shared, coherent, consistent with the unit and institutional mission, and continuously evaluated.

Our conceptual framework continues to guide our work with young women and men who seek to become teachers whose informed, intentional, and reflective decision-making will guide their K-12 students’ learning.  In the spring of 2004 we will begin another round of review of our framework as a prelude to our preparation for NCATE and Board of Teaching program review in the fall of 2005.

We continue to draw upon our knowledge base of empirical research integrated with practical wisdom to guide our work with our candidates.  Pending the outcome of our conversations on the role and value of our conceptual framework, we anticipate extensive revision of that knowledge base during the summer of 2004.

Section B: Candidate Performance

Standard 1: Candidate Knowledge, Skills, and Dispositions. Candidates preparing to work in schools as teachers or other professional school personnel know and demonstrate the content, pedagogical, and professional knowledge, skills, and dispositions necessary to help all students learn.  Assessments indicate that candidates meet professional, state, and institutional standards.

Assessments embedded in courses and structured clinical experiences indicate that most candidates acquire the knowledge, skills, and values that will serve as their foundation for successful student teaching in field settings. Analyses of candidates’ work samples prepared during their student teaching experiences continue to confirm their successful encouragement of their students’ learning.  An ongoing study of novice “teachers of promise,” nominated by their faculty mentors who serve our K-12 partner schools, reveals growth in knowledge, teaching skills, and professional judgment consistent with the broad dimensions of our program.

Minnesota’s second year of Praxis II testing found all but two of our 67 “program completers” meeting or exceeding state standards for knowledge of their field of licensure and knowledge of pedagogy.  All program completers found to have academic skill deficiencies early in their preparation were able to resolve them so as to complete Praxis I and our requirements.

We have corrected a weakness in the design of our language arts licensure for elementary candidates seeking a middle level (grades 5 to 8) specialty in that area by restructuring its design to incorporate more substantial work in speech communication, composition, and adolescent literature.  Work continues on the design of an ESL/ELL specialty that candidates could add to their licensure.

We have opened negotiations with representatives from Intel’s “Teacher to the Future” curricular initiative.  This approach to integrating instructional technology at all levels of our program may provide opportunities to build on our strengths in this area. While limitations of this program may hinder our full participation, we foresee an attractive model that might shape our and our candidates’ uses of instructional and information technology.

Standard 2: Assessment System and Unit Evaluation.  The unit has an assessment system that collects and analyzes data on applicant qualifications, candidate and graduate performance, and unit operations to evaluate and improve the unit and its programs.

Refinement of our unit’s assessment system continues, although at a slower than expected pace which has delayed the full implementation of our performance database.  The practical demands of coding, recording, and using multiple indicators for each of Minnesota’s 140 Standards of Effective Practice encouraged the introduction of technological innovations that could ease this burden.  Pilot testing of the MVal behavioral analysis system, software for hand-held computers linked to Minnesota’s standards, offers a promising alternative to manually recording and entering information in candidates’ performance profiles.

Screening of prospective candidates’ academic skills continues. A review of candidates’ entry test results found that scores on Praxis I and ACT sub-tests correlated with Academic Profile performance levels, encouraging the use of either of these two indicators as an alternative to the Profile.  We have thus reduced by one the number of examinations completed by prospective candidates while continuing to help those with deficient skills select and complete needed developmental opportunities. We continue to assess writing performance for those who seek preparation for licensure through our program, encouraging development of needed skills for those whose performance falls below our expectation.

Section C: Unit Capacity.  

Standard 3: Field Experiences and Clinical Practice.  The unit and its’ school partners design, implement, and evaluate field experiences and clinical practice so that teacher candidates and other school personnel develop and demonstrate the knowledge, skills, and dispositions necessary to help all students learn.

Our efforts to initiate and maintain productive partnerships with elementary and secondary schools continue to provide our candidates with useful opportunities to observe, shape, and refine through practice those skills and values that will prove fundamental to their teaching practice.  These experiences begin in their first semester of “foundations,” continue through their acceptance as candidates in their second or third year of study, and conclude with 16 weeks of student teaching in two school settings.

While partnerships with nearby schools are essential for short term clinical work, and while more of our nearby partner schools are enrolling an ever more diverse population of learners as our larger community becomes more diverse, our work with inner city Minneapolis schools continues to offer our students exciting experiences in very diverse settings.  These efforts respond to one student’s view that, “watching how other teachers do things gives me greater confidence to try them myself.”  Added another, “you can only learn so much by being told how to do it.  I need to see how it’s done before I can do it.”  

Prospective elementary candidates in their first year of study join with their peers for a one week immersion in a school enrolling more than 80% non-white k-8 students. Reviewing her experiences in this setting, one first year student noted that, “Risen Christ was totally different from what I expected.  I have never been in so diverse a school site as I was there.  It was so different from my own small town school experience.” 

Another, completing her review of the same experience, found that, “Risen Christ seems so limited in the resources they have.  There’s no ESL/ELL teacher to go to.  But their kids’ needs are met.  Their teachers have to know how to make up the difference in resources.  I found it frustrating at times, but it was a realistic experience that helps me realize what I have to prepare for.”

Building on the success of this elementary “block,” we are at work on revising a cluster of five courses to form a “mini-block” for prospective secondary level candidates so that they might have an opportunity to experience outcomes similar to those enjoyed by elementary candidates.

Standard 4: Diversity.  The unit designs, implements, and evaluates, curriculum and experiences for candidates to acquire and apply the knowledge, skills, and dispositions necessary to help all students learn.  These experiences include working with diverse higher education and school faculty, diverse candidates, and diverse students in P-12 schools.

Areas of Improvement related to Standard 4 cited as a result of the last NCATE review: Candidate experiences with diverse faculty, candidates, and student populations are limited.

We continue to revise and refine our program so that we can provide every candidate with meaningful opportunities to observe and teach children from diverse racial, economic, cultural, and ethnic backgrounds.  Our collaboration with Risen Christ, a Minneapolis

K-8 parochial school enrolling 80% of its students from minority groups, is an important partner in our efforts to realize that promise. Given the success of this partnership, we are exploring additional relationships with other urban schools serving diverse student populations, including San Miguel Middle School, Ascension School (K-8), and St. Bernard’s School (K-12).

We are at work on curricular revisions which will extend our commitment to provide experiences with diverse K-12 learners to those of our candidates pursuing secondary licensure.  While the block of time we provide secondary candidates will take a different form, we are confident that it will offer meaningful learning that can open them to diverse learners’ needs and opportunities.

Our efforts to attract more faculty from diverse racial or cultural groups to our program, while more intense than in past years, were not successful.  In addition to traditional methods, our use of formal and informal networking with communities of color in Minneapolis and Saint Paul produced a wider pool of candidates than in years past. Unfortunately, none offered the experience and skills we sought for these two short term positions. 

We continue to enrich our understanding of cultures different from our own through workshops designed to meet the needs of our faculty and those of K-12 faculty teaching in our partner schools.  Responding to the dramatic increase in families emigrating to Saint Cloud from Somalia in the past three years, in May of 2003 we planned and hosted an afternoon and evening retreat that introduced us to the work of Minnesota’s Center for Victims of Torture.  Using funds secured from the colleges’ we invited five educators from Saint Paul and Minneapolis public schools, four of whom were Somali, to help us begin to understand the refugee experience that scars many of these new students who continue to arrive in our schools.

As many of our prospective candidates for licensure participate in service learning programs during their spring break, we have begun to explore school settings in urban areas for such programs.  We hope to identify and sponsor extended service learning opportunities in diverse schools during our newly inaugurated May term.

We also hope to meet with our colleagues in Student Development and Admissions in the year ahead to explore ways in which we might appeal to those from communities of color who may seek to become teachers.  We hope to work with our colleagues to find ways to highlight the teaching profession, and college enrollment, as a viable option for culturally or racially diverse students who hold that dream while enrolled in high schools serving communities of color. 

Sister Ann Marie Biermaier, OSB, was invited to travel to Tanzania, Africa, in May-June 2003 to determine the possibility of setting up a partnership with St. Agnes School, Chipole, Tanzania. She spent three weeks visiting the convent and school while meeting with individuals about the prospect of such an endeavor. Although a study partnership in the school may prove difficult because of dissimilar schedules, she will continue to probe other possibilities for working with the orphans, elementary and secondary students who reside at this school.

Standard 5: Faculty Qualifications, Performance, and Development.  Faculty are qualified and model best professional practices in scholarship, service, and teaching, including the assessment of their own effectiveness as related to candidate performance.  They also collaborate with colleagues in the disciplines and schools.  The unit systematically evaluates faculty performance and facilitates professional development.

Areas of Improvement related to Standard 5 cited as a result of the last NCATE review: Faculty have limited involvement in scholarly activities at the regional and national levels.

We expect a high level of teaching performance from each other as we work toward our unit mission.  That expectation demands a high investment of each instructor’s time and effort.  Further, teaching in a values-focused liberal arts college offers opportunities to invest in the personal and intellectual development of our candidates, modeling the investment we expect them to make in the lives of their students.  Forming and maintaining productive relationships that can mediate students’ intellectual and moral growth requires the further commitment of our time and energy. 

We realize that we must also model the value of scholarship through its practice, not only to inform and strengthen our own teaching but also to demonstrate its value for our candidates.  We strive to keep teaching, service to our school partners, and scholarship in dynamic balance that reflects our needs and resources. We have worked to include greater attention to scholarship in that balance. 

In January of 2003 Brother Doug Mullin, OSB, summarized his work on developing “A Framework for Assessing Teacher Candidate Dispositions” during the AACTE Annual Meeting in New Orleans.  In the following month Brother Doug shared his findings on this topic in a presentation for the Minnesota Teacher Education Congress held in Minneapolis.  Now on sabbatical, Doug has begun an exploration of promising novice teachers’ beliefs and practices.  Later this year he will accept an invitation to share his research into the dispositions and practices of novice teachers as well as his perceptions of current issues in American Education with the faculty of Beijing Normal University and with alumni of Fu Jen University, also in Beijing.  His travel and work in China will offer opportunities to study that nation’s educational system, the instructional practices of its teachers, and its history.

While others in our department cannot invest as much of their time and energy in scholarly work without the benefit of a sabbatical, we have enjoyed some success in sharing knowledge and experience through our applied research and writing.  Sister Ann Marie Biermaier OSB and Professor Deanna Lamb have proposed a review of our efforts to assure opportunities for every candidate to observe and teach K-12 students from diverse cultures and races for the 2004 AACTE Annual Meeting in Chicago. Professor Arthur Spring has prepared three book reviews for Bridges: An Interdisciplinary Journal of Technology, Philosophy, History, and Science. Professor Edmund Sass has prepared lessons plans accepted by Ask ERIC for its electronic database. Professor Bruce Dickau continues his collaboration with SCIMATH Minnesota/TRN, a state-wide organization investing in research on how teachers incorporate science standards and best practices into their classroom instruction.  Dr. David Leitzman, a member of the editorial board of Performance Improvement Quarterly, has reviewed several articles during the past year for that publication.  In late August of this year Sister Ann Marie Biermaier joined 100 participants attending the Minnesota Catholic Education Association’s annual convention, representing catechetical ministries in all six of Minnesota dioceses, to help them explore their core beliefs and work toward consensus on strategies which could advance education in the faith.

Standard 6: Unit Governance and Resources.  The unit has the leadership, authority, budget, personnel, facilities, and resources, including information technology resources, for the preparation of candidates to meet professional, state, and institutional standards.

We have the resources we need to accomplish our mission. With the beginning of this school year Sister Ann Marie Biermaier returned to the role of department chairperson and unit head.  We continue to enjoy the talents of a cohesive, committed faculty and staff who work well together as they prepare competent, caring teachers who are able to help each of their students learn.

Program Completers during the 2002-2003 Academic Year:  67

Enter the name of the person filling out the report:  David F. Leitzman

 

M:\Assessment\Reports\Interim Assessment Report AAC AASR SP04.doc