Please update your web browser or disable Compatibility View.

Licensure Candidates’ Performance on Praxis II Examinations

2001-2002 and 2002-2003

Overview

The College of Saint Benedict and Saint John’s University jointly sponsor the Department of Education to prepare those who are called to the vocation of teaching. The Department provides state approved programs to prepare elementary generalists (grades K-6) having a middle level specialty (5-8) in communication arts and literature, social studies, mathematics, or natural science or in world languages (K-8). Additional programs of study and practice prepare secondary teachers (grades 7-12) of mathematics, natural science, communication arts and literature, and social science. The Department also offers approved licensure programs for those who seek to teach vocal or instrumental music, visual art, and French, German, or Spanish (K-12). A non-licensure program prepares teachers of theology for grades five through twelve.

All who would be licensed as elementary, middle, or high school teachers by Minnesota’s Board of Teaching must verify the extent of their knowledge of the licensure areas in which they hope to teach as well as their understanding of pedagogy appropriate for those areas. The Board selected the Praxis II series of examinations developed by the Educational Testing Service for this purpose. ETS offers annual summaries of all Praxis tests completed by ten or more students from the same program in a test year. Information included in these annual summaries offers one indicator of our candidates’ content knowledge and pedagogical understanding as they near the conclusion of their preparation for licensure.

Results of examinations summarized by the Educational Testing Service and reviewed for this report suggest that candidates for elementary licensure often equal or exceed the performance of their peers in Minnesota or in the nation as a whole on some facets of Praxis II examinations over subject matter and pedagogy. Exceptions to this generalization are also evident.

  • While their test scores in Language Arts and Mathematics equal or exceed scores of their Minnesota peers, our elementary candidates’ test performance in general science and social studies is below that of candidates prepared by other Minnesota colleges (see Tables 1.2 and 1.3 beginning on page 5).
  • Most elementary candidates perform at or near the level of their Minnesota peers on the “Principles of Learning and Teaching” pedagogy examination. The distribution of their scores, however, reveals patterns of possible weakness in “teacher professionalism” for those tested in 2001-2002 (fixed response items) and “students as learners” (fixed) and “Instruction and Assessment” (case histories)” for those tested in the 2002-2002 (see Table 1.6 on page 8).

Content examination results for those seeking secondary licensure are less comforting for social studies candidates hoping to teach in middle and high school settings.

  • Social studies candidates’ overall test performance fell below their state or national peers in several areas (see Table 2.1 on page 9). Our candidates’ scores were notably lower in world history (Table 2.2 on page 7).

Our secondary candidates’ knowledge of teaching and learning, as described by their Praxis II test performance, is similar to that of candidates prepared by other Minnesota colleges and universities.

  • More secondary candidates completing their pedagogy exams in 2001-2002, however, earned lower scores in “teacher professionalism” (see Table 2.6 on page 13). More candidates completing their pedagogy exams during the following year (02-03) also earned lower scores in “teacher professionalism” as well as in “students as learners” (see Table 2.6 on page 13).

Nearly all candidates seeking elementary or secondary licensure exceeded Minnesota’s minimum qualifying scores on all relevant tests for their Minnesota licensure (see Table 3.0 on page 14).

Context

Responding to state and federal demands for such examinations, The Educational Testing Service (ETS) developed and refined the Praxis series of tests during the previous decade (http://www.ets.org/praxis/index.html).  Tests of content knowledge included in this series reflect the guidelines offered by professional and scholarly associations as to what teachers should know to successfully perform their roles. Praxis II examinations of pedagogy include fixed and free response items that explore the ten standards and their respective indicators developed by the Interstate New Teacher Assessment and Support Commission or “INTASC” (http://www.ccsso.org/projects/Interstate_New_Teacher_Assessment_and_Support_Consortium/Projects/Standards_Development/).

Guided by Minnesota’s legislature, in the mid-1980’s the Board of Teaching began to require teachers seeking a license to verify the extent of their academic skills in reading, writing, and mathematics by completing the Pre-Professional Skills Test (PPST or Praxis I) developed by the ETS. Subsequent legislation required additional testing to affirm teachers’ knowledge of the subjects they would teach and their understanding of teaching and learning before receiving their teaching licenses (see Minnesota Rule 8710.0500 as revised on 31 July 2000 at http://www.revisor.leg.state.mn.us/arule/8710/0500.html).    

When the Board of Teaching selected ETS’ Praxis II series to meet this additional requirement, it invited licensed teachers and teacher educators to review these examinations for agreement with Minnesota’s licensure rules. Working under the guidance of testing specialists employed by ETS, these teams “aligned” relevant tests in this series to better fit Minnesota’s content and pedagogy standards. Thus the “Principles of Learning and Teaching” examinations developed for each of three grade level groupings (K-6, 7-9, or 5-12) reflect both INTASC and Minnesota’s Standards of Effective Practice (http://www.revisor.leg.state.mn.us/arule/8710/2000.html).  Praxis II content examinations were also “aligned” to more closely agree with the legislated rules defining the knowledge expected of teachers in each licensure area (for example, elementary licensure as defined at http://www.revisor.leg.state.mn.us/arule/8710/3200.html).

With the completion of this review process in 2000, prospective teachers seeking licensure during the 2000-2001 testing year (1 September 2000 to 31 August 2002) were required by the Board of Teaching to complete relevant Praxis II tests without penalty of failure to provide data enabling the Board to set minimum qualifying scores for each licensure area. Careful examination of test data encouraged the Board to set qualifying scores that would not unreasonably discriminate against those whose first language was not English or those who were nurtured in a minority culture. These qualifying or “cut” scores were set for each licensure test to reflect a level of candidate performance equal to the minimal level of knowledge and understanding necessary for teachers to competently perform in their roles. Candidates earning scores below this point, if licensed, may have inadequate content knowledge or insufficient understanding of teaching that could harm the educational progress of students in their charge.

All seeking a Minnesota teaching license after 1 September of 2002 were required to complete approved Praxis II content and pedagogy examinations and to earn scores at or above each test’s minimum qualifying score for tests related to their area of preparation. Licensure tests are thus “high stakes” examinations for those whose performance fails to reach the qualifying score.

Praxis I and II test results are also gathered and reported by ETS to the federal government in response to the 1998 reauthorization of the Higher Education Act. Working from the assumption that all teachers were poorly prepared and unable to respond to the demands placed on them with the 2002 reauthorization of the Elementary and Secondary Education Act, popularly referred to as the “No Child Left Behind” or NCLB act (http://www.ed.gov/nclb/landing.jhtml?src="http://www.csbsju.edu/education/accreditation/ncate2005/pb), all teacher preparation programs must verify that their “program completers” meet state standards. Teacher preparation programs thus also view licensure examinations as “high stakes” tests when their candidates fail to reach qualifying scores.

Praxis II Results

While the portion of our students who “pass” their licensure exams always attracts our immediate attention owing to the consequences of failure, the relatively low minimum scores set by the Board of Teaching enables most candidates to successfully overcome this potential barrier. Additional Praxis II test information provided by the Educational Testing Service offers a more useful perspective from which to judge some facets of our candidates’ preparation for their professional roles in Minnesota’s schools. Analysis of their performance on these tests, taken in context, may reveal areas of curricular strength and weakness and thus encourage our search for ways to refine the courses, clinical work, and student teaching experiences that together provide a foundation on which new teachers might begin to build their practice.

The size of our program, however, limits such analyses to those tests completed by ten or more candidates during any test year. Candidates for licensure as teachers of elementary education and those seeking to become secondary teachers of social studies completed content tests in sufficient number to enable ETS to report test performance. ETS also prepared analyses of pedagogy tests completed by all elementary and all secondary candidates in each test year. These reports, while helpful, are limited to summaries of group performance and comparisons with state and national samples. We unfortunately do not have access to more detailed item analyses which might encourage deeper understanding of the nature of students’ test performance, an important limitation hindering generalizations we might hope to advance from the information offered in the following report.

Elementary Education Candidates: Content Knowledge.About two-thirds of our candidates seek licensure as elementary teachers who will serve students from kindergarten through the sixth grade. They must be prepared to offer instruction in all subjects specified by the district that will employ them upon graduation from our program and by state standards in core subjects. While some of Minnesota’s school districts may provide teachers of music, art, physical education, and perhaps foreign languages to offer their students specialized instruction in those areas, all elementary teachers prepared as generalists must be able to plan and offer age-appropriate instruction embedding K-12 student learning standards set for the language arts (reading, writing, speaking), the social sciences, mathematics, and the physical sciences. Appendix 1 on page 15 offers an outline of the content included in this examination.

Legislated rules define related standards that must be evident in the practice of elementary teachers which form the core of our licensure programs. Elementary content standards, found at http://www.revisor.leg.state.mn.us/arule/8710/3200.html, and related pedagogical standards described at http://www.revisor.leg.state.mn.us/arule/8710/2000.html, are the basis for teachers’ instruction and examination. The Board of Teaching set the minimum qualifying score for the elementary content test at 140. Candidates for elementary licensure earning lower scores must complete the examination again in the hope that they will attain a score at or above 140 if they wish to teach in one of Minnesota’s public schools.

1.1. Content Test Performance. The following table summarizes the performance of Saint Benedict’s and Saint John’s candidates for elementary licensure. Parenthetical entries reveal the performance of all who completed this examination in the United States between 1 September and 31 August of each test year.

Test Year

Number

Low Score

Median

High Score

25th-75th %tile

01-02

75 (27,205)

132 (100)

166 (163)

196 (200)

157-176 (150-175)

02-03

40 (25,994)

135 (100)

170 (162)

195 (200)

161-176 (149-175)

The 75 candidates who completed their content tests in the 2001-2002 test year earned a median scaled score of 166 as contrasted with the median score of 163 earned by all 27,205 test takers nationally. The following year, 40 elementary candidates earned a median score of 170 while the 25,994 who completed this test nationally earned a median score of 162. Our candidates’ lowest scores are well above the lowest observed score in the national sample, while the highest score earned by our candidates each year fell just below the highest national score. The score range for the middle one-half of our candidate groups is somewhat higher than the range for each of the national samples. Overall, this pattern suggests good performance that equals or exceeds the national sample of all tested elementary candidates for licensure. A closer look at our candidates’ core content sub-scores may tell us more.

1.2. Mean Percent Correct by Content Areas. ETS offers additional information describing performance on the categories of elementary level content knowledge included in this exam. The following table reveals the mean or “average” percent of items correctly answered by our candidates, by all those who took the test in Minnesota, and by all who completed this examination in the United States during each of the two years. Differences (+ or - %) between the CSB/SJU mean percent correct and that of all Minnesota candidates appear under the “CSB/SJU” column. A similar set of observed differences between the Minnesota and the national samples is included in parentheses under the Minnesota column.

Content Area

Year

Points

CSB/SJU

Minnesota

U. S.

Language Arts

01-02

30

77% (+2)

75% (+1)

74%

02-03

30

82 (+3)

78 (+4)

74

Mathematics

01-02

30

78 (0)

78 (+7)

71

02-03

30

82 (+3)

79 (+8)

71

Social Studies

01-02

29

66 (- 2)

68 (+2)

66

02-03

29

67 (- 3)

70 (+1)

69

Science

01-02

30

64 (- 6)

70 (+3)

66

02-03

30

65 (- 5)

70 (+3)

67

This analysis suggests that more of our elementary candidates as a group correctly answered Praxis II questions testing their knowledge of language arts and mathematics to a greater degree than did their Minnesota peers and the national sample. Knowledge of social studies and general science as indicated by the mean percentage of correctly answered questions for each test year, however, are below our state means and equal to or below national means. This pattern, should it persist, may encourage curricular review of the two weaker areas.

1.3. Score Distribution by Content Area. The following table reveals distributions of our candidates’ scores placed into one-quarter segments (quartiles) derived from the range of test scores calculated by ETS for each content area and test year. Since the quartile limits are based on the national sample, our candidates’ test performance may preset a different pattern.

Content Area

Year

1st Quartile

2nd Quartile

3rd Quartile

4th Quartile

Tested

Language Arts

01-02

8 (11%)

28 (37%)

24 (32%)

15 (20%)

75 (27,205)

02-03

2 (5)

9 (23)

12 (30)

17 (43)

40 (25,994)

Mathematics

01-02

7 (9)

16 (21)

24 (32)

28 (37)

75

02-03

2 (5)

8 (20)

10 (25)

20 (50)

40

Social Studies

01-02

10 (13)

27 (36)

28 (37)

10 (13)

75

02-03

9 (23)

14 (35)

10 (25)

7 (18)

40

Science

01-02

22 (29)

21 (28)

20 (27)

12 (16)

75

02-03

12 (30)

12 (30)

9 (23)

7 (18)

40

Note: Quartiles are set by ETS to capture the range of scores awarded to those candidates in the national sample for each Praxis test administered during each test year. Scores earned by an institution’s candidates may not replicate the national distribution of 25% of scores in each quartile.

Looking at the language arts content area, we find that eight of our 75 candidates (11%) tested in 2001-2002 earned scores falling into the lowest one-quarter of this distribution of all Praxis test scores. Fifteen of the 75 CSB and SJU candidates (20%) who completed their elementary content test during the 2001-2002 testing year earned scores which placed them in the top one-quarter of the scores received by the 27,205 who completed this test in that year.

The information presented in Table 1.2 now emerges in more detail. Social studies knowledge appears limited for 28 of the 75 candidates tested in 2001-2002 whose scores in that content area fell into the third quartile (37%). Knowledge of general science appears to be the weakest of the four tested content areas for those completing this examination, with about 30% earning scores falling into the lowest national quartile for each test year (22 or 29% of 75 in 01-02; 12 or 30% of 40 in 02-03). Should this pattern be affirmed, review of these core areas may be warranted.

Elementary Education Candidates: Pedagogical Knowledge.Minnesota requires those seeking licensure as elementary teachers of students enrolled in kindergarten through grade six to verify the extent of their knowledge of how to teach the facts, concepts, principles, and skills included in their licensure area. The Board of Teaching selected “Principles of Learning and Teaching” from the Praxis II series of examinations to verify licensure candidates’ pedagogical knowledge, setting the minimum qualifying score for this test at 152. Those earning scores below 152 cannot be licensed to teach in Minnesota’s public elementary schools. Appendix 3 on page 17 provides a content outline for this examination.

1.4. Pedagogy Test Performance. The following table summarizes the performance of Saint Benedict’s and Saint John’s candidates for elementary licensure contrasted with, in parentheses, the performance of all who completed this examination in the United States between 1 September and 31 August of each test year.

Test Year

Number

Low Score

Median

High Score

25th-75th %tile

01-02

77 (34,640)

136 (100)

172 (173)

200 (200)

162-181 (165-180)

02-03

42 (29,329)

150 (102)

180 (175)

192 (200)

172-185 (166-182)

The 77 CSB and SJU students who completed this test in 2001-2002 earned a median score of 172 compared with a median score of 173 for the 34,640 in the national sample of examined candidates, while the highest score for our candidates in this year, 200, equaled the highest observed score in the national sample. The median score for 2002-2003 exceeded the national median by five points (180). This suggests a pattern of group performance for our elementary candidates that is consistent with the national sample.

1.5. Mean Percent Correct by Pedagogical Skill Areas. The Praxis II “Principles of Learning and Teaching” (PLT) examinations follow the same general format for each grade level cluster, shifting the design of test items to reflect those clusters. The Principles of Learning and Teaching Study Guide published for prospective examinees by ETS in 2001 defined the Students as Learners skill area to include questions exploring a candidate’s knowledge of “student development and the learning process,” the nature of “students as diverse learners,” and aspects of “student motivation and the learning environment.” ETS developed the Instruction and Assessment skill area to verify candidates’ knowledge of “instructional strategies,” ways of “planning instruction,” and of “assessment strategies” that might be used with their future students.

Communication Techniques includes knowledge of “verbal and nonverbal communication,” understanding of “cultural and gender differences” that can influence communication, and awareness of ways to encourage “discussion and responses in the classroom.” The fourth skill area included in this examination, “Profession and Community,” allows elementary licensure candidates to demonstrate their understanding of how a teacher might perform as a “reflective practitioner” and their recognition of teaching as taking place within a “larger community” setting formed by families, colleagues, the school’s district, state and federal government, and a variety of advocacy groups (pages 11, 29 and 30).

Responding to calls from critics of such tests for more “authentic” assessments of a prospective teacher’s knowledge, ETS changed the PLT in the 2002-2003 test year to include a greater variety of item formats. Tests used during both years included in this analysis featured multiple choice “fixed response” test items and constructed or “free response” short answer questions. Case studies focused on teacher actions and student behavior and which may appear in the form of a narrative or as a set of documents were added for 2002-2003. These constructed response items include both relevant and unhelpful information in a different forms. Brief essays written in response to a set of related prompts are holistically scored for key ideas. Writing skills revealed through case study responses are not directly assessed as part of “Principles of Learning and Teaching.”

Table 1.5 below reveals the mean percent correct on each of several skill areas earned by our elementary candidates, by all who took this test in Minnesota, and by all who completed this examination in the United States for each test year.

K-6 Pedagogical Skill Areas

Year

Points

CSB / SJU

Minnesota

U. S.

Students as Learners: Fixed Response

01-02

12

73% (0)

73% (+2)

71%

02-03

8

71 (- 2)

73 (+3)

70

Instruction and Assessment: Fixed

01-02

11

70 (- 1)

71 (+3)

68

02-03

8

76 (+4)

72 (+1)

71

Teacher Professionalism: Fixed

01-02

11

68 (- 4)

72 (+1)

71

02-03

8

84 (- 2)

86 (+4)

82

Teacher Professionalism: Free Response

01-02

8

79 (+1)

78 (+4)

74

All Skills: Free

01-02

36

71 (0)

71 (0)

71

Students as Learners: Case Histories

02-03

16

79 (+5)

74 (+3)

71

Instruction and Assessment: Cases

02-03

16

72 (- 2)

74 (+3)

71

Communication Techniques: Cases

02-03

8

83 (+4)

79 (+3)

76

Teacher Professionalism: Cases

02-03

8

81 (+2)

79 (+3)

76

Candidates who completed their “Principles of Learning and Teaching” examination during each of the two test years included in Table 1.5 may have a weaker understanding of “Teacher Professionalism” than their Minnesota peers. Occasional deviations for either test year in “Students as Learners” and “Instruction and Assessment may warrant further review. Our 2002-2003 candidates’ mean scores exceeded those of both their state and national peers for case histories in “communication techniques” and “teacher professionalism,” but not in “Instruction and Assessment.”

1.6. Score Distribution by Pedagogical Skill Areas.; The following table reveals distributions of our candidates’ scores for each skill area by test year. ETS set the quartiles to reflect the performance of all who completed this examination during the test year.

K-6 Skill Areas

Year

1st Quartile

2nd Quartile

3rd Quartile

4th Quartile

Tested

Students as Learners: Fixed Response Items

01-02

02-03

12 (16%)

8 (19)

28 (37%)

15 (36)

24 (32%)

16 (38)

12 (16%)

3 (7)

76

42

Instruction and Assessment : Fixed

01-02

02-03

10 (13)

5 (12)

22 (29)

14 (33)

25 (33)

21 (50)

19 (25)

2 (5)

76

42

Teacher Professionalism: Fixed

01-02

02-03

20 (26)

6 (14)

20 (26)

17 (40)

20 (26)

19 (45)

16 (21)

0 (0)

76

42

Teacher Professionalism: Free

01-02

10 (13)

20 (26)

32 (42)

14 (18)

76

All Skills: Free

01-02

17 (22)

26 (34)

17 (22)

16 (21)

76

Students as Learners: Case Histories

02-03

5 (12)

9 (21)

11 (26)

17 (40)

42

Instruction & Assessment: Cases

02-03

9 (21)

12 (29)

18 (43)

3 (7)

42

Communication Techniques: Cases

02-03

5 (12)

7 (17)

14 (33)

16 (38)

42

Teacher Professionalism: Cases

02-03

4 (10)

12 (29)

26 (62)

0 (0)

42

Scores from our elementary candidates included in this table suggest that more of those who completed their “Principles of Teaching and Learning” examination during the 2001-2002 test year earned higher sub-scores than did their CSB/SJU peers who completing the test in the following year.  Looking only at the three sub-scales common to both versions of the tests used during these two years, we find a greater portion of those tested in the first year in each of the three areas common to both tests appears in the fourth quartile than is the case for those of our candidates tested in 2003-2003; 16% versus 7% for knowledge of “Students as Learners,” 25% versus 5% for “Instruction and Assessment,” and 21% versus 0% for understanding “Teacher Professionalism.”

Moving to the other end of this distribution of sub-scale scores, about one-fourth of the 2001-2002 tested candidates earned scores in the first quartile for their knowledge of “Teacher Professionalism” (20 of 76, 26%), while fewer from the second year share that category with them (02-03: 6 of 42, 14%).A smaller portion of those completing the examination had sub-test scores that fell into the fourth quartile for each year. This appears to be the weaker of the three core areas.

By definition we should expect that 25% of the scores earned by all who completed this test would be included in the first quartile (observed scores of 100 to 165 from 34,640 tested in 01-02; scores of 100 to 166 for 29,329 tested in 02-03). Scores earned by our candidates, however, need not reflect this distribution. If we provide our candidates with opportunities to learn and practice the skills included in an area such as “teacher professionalism,” we might expect to see scores skewed toward the fourth quartile, an indicator of stronger candidate performance.

Secondary Social Studies Candidates. Those serving middle and high school students as social studies teachers are licensed to instruct their students in United States history, world history, political science, geography, economics, and faces of the behavioral sciences of psychology, sociology, and anthropology. Legislated “content” rules that guide coursework for this licensure program appear at http://www.revisor.leg.state.mn.us/arule/8710/4800.html.  The concepts and principles described in this rule provide incomplete support for all facets of Minnesota’s newly adopted social studies learning standards for students in all grades approved this spring by our state legislators (http://education.state.mn.us/html/038364.htm).

The Board of Teaching selected the Praxis II social studies examination to verify the extent of candidates’ knowledge for this area, setting the qualifying score at 145. The wide range of topics and disciplines folded into “social studies” requires candidates in this area, perhaps more so than those prepared for other licensure areas, to acquire and to be able to share an integrated understanding of their field of study. That understanding must reflect both deeper and broader knowledge of each of the eight disciplines that contribute in some way to middle and high school students’ experiences in “social studies.” Greater demands on social studies teachers would seem the necessary outcome, despite the field’s reputation as a haven for those who seek other roles in secondary schools. Appendix 2 on page 16 offers a content outline for this test.

2.1. Social Studies Content Test Performance. A comparison of our students’ performance with a larger national population of tested social studies candidates reveals a general trend toward lower performance. During both test years our candidates’ median and high scores lag their national peers. Lower median scores may hint at areas of weakness within one or more of the several content areas included in this licensure test.

Test Year

Number

Low Score

Median

High Score

25th-75th %tile

01-02

16 (9,724)

137 (104)

160 (166)

180 (200)

150-172 (155-177)

02-03

10 (11,059)

145 (107)

158 (165)

173 (200)

151-170 (155-177)

The middle range of scores (25th to 75th percentile) earned by those of our candidates who completed their social studies examination in both test years includes scores lower than the national sample’s low scores (150 versus 155 and 151 in 01-02 versus 155 for 02-03). The high scores for our candidates within this middle range are also below those earned by their national peers. This pattern, unique to the social studies exam, foreshadows a pattern of weaker test performance than might be evident for other licensure examinations.

2.2. Mean Percent Correct by Social Studies Content Areas. ETS offers additional information through a comparison of the mean percent correct on this test earned by our secondary social studies candidates with the performance of all those who took the test in Minnesota as well as with all who completed this examination in the United States each year.

Social Studies Content Areas

Year

Points

CSB/SJU

Minnesota

U. S.

U.S. History

01-02

29

59% (-1)

60% (- 4)

64%

02-03

29

55 (-6)

61 (- 4)

65

World History

01-02

29

55 (-7)

62 (+ 1)

61

02-03

29

43 (-15)

58 (-3)

61

Political Science

01-02

21

69 (0)

69 (0)

69

02-03

21

58 (-7)

65 (0)

65

Geography

01-02

20

75 (0)

75 (+ 7)

68

02-03

20

64 (-5)

69 (+ 3)

66

Economics

01-02

19

63 (-1)

64 (+ 5)

59

02-03

20

57 (-4)

61 (+ 3)

58

Behavioral Sciences

01-02

13

70 (-8)

78 (+ 4)

74

02-03

13

71 (-1)

72 (+ 6)

66

The mean percentage correct earned by our candidates in both test years is lower for several of the social science test’s content areas. While some of the differences between the scores earned by our candidates and those of other Minnesota candidates are small (-1), this table suggests that United States and world history are two content areas with substantial discrepancies. Relative to students prepared at other Minnesota colleges, our candidates earned lower mean scores in both United States history (55% correct versus 61% (-6%) for all Minnesota candidates in 2002-2003) and world history (7% below the Minnesota mean percent correct score for 2001-2002, 15% below our state mean for the 02-03 year).

Our candidates who were tested in 2002-2003, a year when passing test scores were required, performed below our state mean in political science (-7%), geography (-5%), and economics (-4%).

The mean percent correct for those tested in 2001-2002 in the behavioral sciences, a minor theme in the social studies examination, fell eight percent below their Minnesota peers (70% versus 78%). Those completing their social studies content tests during the following year, however, were nearly equal to their Minnesota peers with respect to their knowledge of the behavioral sciences.

2.3. Score Distribution by Social Studies Content Areas.The following table reveals distributions of our candidates’ scores by national quartile for each content area included in the Praxis II social studies examination for each test year. ETS set the quartiles to reflect the performance of the 9,724 who completed this test during 2001-2002 and the 11,059 who did so during the following test year. While the small samples of our candidates included in this distribution can be misleading, this table generally confirms that our social studies candidates may have responded to this licensure test as if they recalled less about world and U.S. history than did many candidates prepared by other Minnesota colleges and universities.

Content Area

Year

1st Quartile

2nd Quartile

3rd Quartile

4th Quartile

Tested

U.S History

01-02

7 (44%)

3 (19%)

5 (31%)

1 (6%)

16

02-03

5 (50)

4 (40)

1 (10)

0 (0)

10

World History

01-02

4 (25)

6 (38)

6 (38)

0 (0)

16

02-03

8 (80)

2 (20)

0 (0)

0 (0)

10

Political Science

01-02

2 (13)

9 (56)

4 (25)

1 (6)

16

02-03

4 (40)

4 (40)

1 (10)

1 (10)

10

Geography

01-02

3 (19)

4 (25)

5 (31)

4 (25)

16

02-03

2 (20)

4 (40)

3 (30)

1 (10)

10

Economics

01-02

2 (13)

4 (25)

4 (25)

6 (38)

16

02-03

2 (20)

4 (40)

1 (10)

1 (10)

10

Behavioral Sciences

01-02

1 (6)

12 (75)

3 (19)

0 (0)

16

02-03

1 (10)

3 (30)

4 (40)

2 (20)

10

Looking at the first quartile, the pattern emerging in the previous table (2.2) is confirmed. Seven of the 16 who completed their social studies content test in 2001-2002 (44%) earned scores on the U.S. history sub-test which place them in the bottom 25% of the national distribution of all such scores. One-half of those tested during the following year also earned sub-scores falling into the same quartile (5 of 10, 50%). While only one-forth of the candidates tested in the first year earned first quartile scores in world history (4 of 16, 25%), eight of the ten tested during 2002-2003 reached this category (80%).

There are some brighter aspects to this distribution. More than one-third of those tested in 2001-2002 earned world history test scores that placed them in the third quartile of all social studies examinees in that year (6 of 16, 38%), although none reached the fourth quartile. Looking at economics, as many tested that year earned sub-scores reaching into that fourth quartile (6 of 16, 38%). Sub-scores from nearly one-third of these same 16 candidates reached the third quartile in U.S. history (5 of 16, 31%), helping to off-set the performance of their colleagues in the first quartile (7 of 16, 44%).

Other indicators should be examined to affirm the possible weaknesses enjoyed by those candidates seeking social studies licensure.  Those indicators may help us better understand how the experiences of our candidates may support their lower than expected performance. A key question for such a review, difficult to answer without item analysis data, might focus on the congruence of licensure standards, courses and fieldwork designed to help candidates meet those standards, and the design of the licensure test.

Secondary Education Candidates: Pedagogical Knowledge.Successful classroom teachers must join their knowledge of what to teach with their skills in how to teach the knowledge, skills, and values that states, districts, and their own professional standards might require. Minnesota’s expectations of its licensed teachers’ pedagogical knowledge appear as the Standards of Effective Practice for Teachers (http://www.revisor.leg.state.mn.us/arule/8710/2000.html).  The Board of Teaching and the Educational Testing Service used expert review of test items to “align” the Praxis II “Principles of Teaching and Learning” examination to these standards through differential weighting of selected items and the deletion of others. Appendix 3 on page 17 includes a summary outline of this examination.

2.4. Grade 7-12 Pedagogy Test Performance. Most candidates for secondary licensure choose to complete the Praxis II examination over “Principles of Learning and Teaching” for grades 7 through 12. The following table summarizes group performance on this test relative to all who completed it during the 2001-2002 and 2002-2003 test years.

Test Year

Number

Low Score

Median

High Score

25th-75th %tile

01-02

27 (21,614)

162 (100)

175 (172)

186 (200)

170-179 (165-179)

02-03

28 (21,212)

156 (104)

173 (171)

195 (200)

166-177 (164-178)

2.5. Mean Percent Correct by Pedagogical Skill Areas. ETS offers additional information describing knowledge of relevant pedagogical skills as revealed in candidates’ test performance. The following table reports the mean percent correct for our candidates, for all those who took the test in Minnesota, and for all who completed this examination in the United States for each test year. During the 2002-2003 test year ETS changed the format and number of items included in this examination, adding case histories and short answer items while deleting some fixed response questions, adjusting scoring to provide some comparability between the two versions.

7-12 Pedagogical Skill Areas

Year

Points

CSB/SJU

Minnesota

U. S.

Students as Learners: Fixed Response

01-02

14

67% (0)

67% (+ 4)

63%

02-03

14

73 (0)

73 (0)

70

Instruction and Assessment: Fixed

01-02

12

84 (+2)

82 (+4)

78

02-03

11

71 (- 3)

74 (+4)

70

Teacher Professionalism: Fixed

01-02

14

71 (- 3)

74 (+3)

71

02-03

14

78 (- 4)

82 (+5)

77

Teacher Professionalism: Free Response

01-02

8

81 (- 7)

86 (+5)

81

All Skills: Free

01-02

36

70 (+1)

69 (0)

69

Students as Learners: Case Histories

02-03

16

58 (0)

58 (+1)

57

Instruction and Assessment: Cases

02-03

36

69 (+2)

67 (+3)

64

Communication Techniques: Cases

02-03

8

73 (- 2)

75 (+3)

72

Teacher Professionalism: Cases

02-03

8

66 (- 4)

70 (+3)

67

Our candidates’ earned a mean percentage correct for “Students as Learners” that equaled those of their Minnesota peers for both test years (67% in 01-02; 73% in 02-03). Results across the two test years are mixed for “Instruction and Assessment,” with those tested in the first year surpassing their peers by 2%, while those tested in 2002-2003 fell behind others tested in Minnesota (-3%).

Concepts and principles that form the “Teacher Professionalism” skill area emerged as a weakness for those tested during both years (-3% in 01-02 followed by -4% in 02-03). While not comparable due to changes in test format, candidates answering short answer questions on this topic fell behind their Minnesota peers by 7% in 2001-2002. The case histories in this area completed by candidates during the following test year fell four percent below others tested in Minnesota that year (66% for CSB/SJU, 70% for Minnesota).

2.6. Score Distribution by 7-12 Pedagogical Skill Areas.The following table reveals distributions of our secondary candidates’ scores by national quartile for each pedagogical skill area and for each test year. ETS set the quartile limits to reflect the performance of all who completed this examination during each test year (21,614 in 01-02 and 21,212 in 02-03). Table 2.6 places scores earned by our candidates within the distribution formed by the national samples for these two test years.

7-12 Pedagogy Skill Areas

Year

1st Quartile

2nd Quartile

3rd Quartile

4th Quartile

Tested

Students as Learners:

Fixed Response Items

01-02

02-03

3 (11%)

4 (14)

6 (22%)

13 (46)

15 (56%)

8 (29)

3 (11%)

3 (11)

27

28

Instruction and Assessment: Fixed Response Items

01-02

02-03

2 (7)

6 (21)

3 (11)

10 (36)

21 (78)

12 (43)

1 (4)

0 (0)

27

28

Teacher Professionalism: Fixed Response Items

01-02

02-03

7 (26)

3 (11)

8 (30)

13 (46)

11 (41)

9 (32)

1 (4)

3 (11)

27

28

Teacher Professionalism:

Free Response Items

01-02

1 (4)

5 (19)

16 (59)

5 (19)

27

All Skill Areas:

Free Response Items:

01-02

4 (15)

12 (44)

7 (26)

4 (15)

27

Students as Learners:

Case Histories

02-03

4 (14)

11 (39)

6 (21)

7 (25)

28

Instruction & Assessment: Case Histories

02-03

1 (4)

10 (36)

9 (32)

8 (29)

28

Communication Techniques: Case Histories

02-03

5 (18)

10 (36)

4 (14)

9 (32)

28

Teacher Professionalism: Case Histories

02-03

5 (18)

13 (46)

8 (29)

2 (7)

28

A complex pattern emerges from this distribution. In general, the test performance of all candidates seeking secondary licensure favors the second and third quartiles rather than the first. Concepts included in “Teacher Professionalism” may have been poorly understood or incompletely recalled by about one fourth of our candidates as revealed by their sub-scale scores in 2001-2002 (1st quartile: 7 of 27 tested, 26%; 2nd quartile: 8 of 27, 30%). More of those tested the following year earned scores falling into the second quartile for their knowledge of this same area (both fixed response and case history item categories, 13 of 28 tested, 46%).

About one-half of the secondary candidates who completed this test in 2002-2003 earned sub-scale scores for “Students as Learners” (13 of 28 tested, 46%) and Teacher Professionalism (fixed response and case histories: 13 of 28, 46%), placing their test scores in the second quartile. Sub-test scores for nearly one-third reached the fourth quartile for case histories of “Communication Techniques” (9 of 28, 32%).

3.0. Licensure Examination Pass Rates. Legislated demands for increasing “accountability” from schools, colleges, and departments of education preparing students for teaching careers are in turn focusing greater attention on those students who do not pass one or more licensure examinations. The following table summarizes the number and percentage of those completing an examination in a testing year whose scores on one or more attempts did not reach the qualifying score set for them by Minnesota’s Board of Teaching. Note that classification errors in ETS Praxis II score reports used for this summary include all completed tests by all candidates who declared themselves to have received “relevant training” for their roles through our program. Performance of candidates submitted to the Secretary of Education under Title II of the 1998 reauthorization of the Higher Education Act has been “cleaned” to include only our graduates and only those individuals’ highest score on an examination. Current Title II reports are reported at our Title II webpage.

Examinees Reporting Preparation for Licensure at CSB and SJU Who Did
Not Attaining Minnesota Qualifying Scores on PRAXIS II Examinations

Praxis II Licensure Tests

Qualifying Score

Proportion Below Qualifying Score

Elementary Content Knowledge: 01-02

140

5 of 75 (7%)

02-03

140

2 of 40 (5)

Elementary Pedagogy K-6: 01-02

152

2 of 77 (3)

02-03

152

1 of 42 (2)

Social Studies Content Knowledge 5-12: 01-02

145

2 of 16 (13)

02-03

145

0 of 10 (0)

Secondary Pedagogy 7-12: 01-02

153

0 of 27 (0)

02-03

153

0 of 28 (0)

Note: Examinees may not complete one or more tests in the year of their graduation from CSB and SJU. Test results reported in this table may include more than one attempt to pass one or more tests by the same individual in any one test year. Results may thus not agree with state and federal Title II reports, which only include the highest score when a candidate makes more than one attempt to pass a test. Title II reports include scores from verified program completers, while PRAXIS reports may include scores from students who were not prepared for licensure through our program.

Appendix 1: Overview of the Elementary Education Content Knowledge Test

I. Language Arts: 25% of 120 multiple choice test items.

A. Understanding literature: Narratives, Nonfiction, Poetry, Resource/research material

B. Text structures and organizers: Structural elements in text, Organizational patterns in text

C. Language in writing: Grammar and usage, Sentence types and structure, Orthography and morphology, Semantics

D. Literacy acquisition and reading instruction: Theories and concepts concerning reading development, Children’s literature, Strategies for word solving, Strategies for comprehension, Study skills and tools.

E. Communication Skills: Stags of the writing development, stages of the writing process, Spelling development, Elements of speaking , Elements of listening (pp. 16-35)

II. Mathematics: 25%.

A. Number systems and number sense: Meaning and use of numbers, Standard algorithms for the four basic operations, Appropriate computation strategies and reasonableness of results. Methods of mathematical investigation

B. Algebraic concepts: Basic algebraic methods and representations, Additive and multiplicative inverses, Special properties of zero and one, Equalities and inequalities, Patterns, Algebraic formulas

C. Informal geometry and measurement: Properties and relationships in figures and shapes in two and three dimensions, Angles and the Pythagorean theorem, Transformations, Geometric models, Nets, Standard measurement units

D. Data organization and interpretation: Visual displays of quantitative information, Simple probability, Outcomes and events, Statistics. (pp. 40-49)

III. Social Studies: 25%

A. Geography: The world in spatial terms, Places and regions, Physical systems, Human systems, Environment and society, Uses of geography.

B. World History: Prehistory and early civilizations, Classical civilizations (Egypt, Greece, Rome), Non-European civilizations, Rise and expansion of Europe, Twentieth-century developments and transformations.

C. United States History: European exploration and colonization, The American Revolution and the founding of the nation, Growth and expansion of the Republic, Twentieth-century developments and transformations.

D. Political Science: Nature and purpose of government, Forms of government, United States Constitution, Rights and responsibilities of citizens, State and local governments.

E. Anthropology, Sociology, and Psychology

F. Economics: The market, Individuals and the market, Economics’ effect on populations and resources, Government’s role in economics and economics’ impact on government, Economic systems, Impact of technological developments on economy, International economics. (pp. 54-70)

IV. Science: 25%

A. Earth Science: Structure of the Earth, Process of the Earth system, Earth history, Earth and universe

B. Life Sciences: Structure and function of living systems, Reproduction and heredity, Regulation and behavior, Biological evolution, Interdependence of organisms

C. Physical Science: Structures and properties of matter, Forces and motions, Energy, Interactions of energy and matter.

D. Science as Inquiry: Inquiry process, Science, technology, and society.

E. Science in Personal and social perspectives: Personal health, Historical perspectives in science, Science as a career

F. Unifying processes: Systems theory, Evidence, models, and explanation, Evolution and equilibrium, Form and function, Change, constancy and measurement. (pp.75-83)

The preceding outline is drawn from Elementary Education: Content Knowledge Study Guide published in 2001 by the Educational Testing Service of Princeton, New Jersey.

Appendix 2: Overview of the Secondary Social Studies Content Knowledge Test

I. United States History: 22% of 130 multiple-choice questions

  • Physical geography of North America,
  • Native American peoples,
  • European exploration and colonization,
  • Establishing a new nation,
  • Early years of the new nation,
  • Continued national development,
  • Civil War era,
  • Emergence of the modern United States,
  • Progressive era through the New Deal,
  • Second World War and postwar period,
  • Recent developments

II. World History: 22%

  • Human society to 3500 B.C.E.,
  • Development of city civilizations,
  • Ancient empires and civilizations,
  • Disruption and reversal (500-1400 C.E.),
  • Emerging global interactions,
  • Political and industrial revolutions,
  • Nationalism,
  • Conflicts, ideologies, and revolutions in the 20th Century,
  • Contemporary trends

III. Government and Civics: 16%

  • Basic political concepts,
  • United States political system,
  • Systems of government/international politics

IV. Geography: 15%

  • The world in spatial terms,
  • Places and regions,
  • Physical systems,
  • Human systems,
  • Environment and society,
  • Uses of geography

V. Economics: 15%

  • Fundamental concepts,
  • Microeconomics,
  • Macroeconomics,
  • International economic concepts,
  • Current issues and controversies

VI. Behavioral Sciences: 10%

Principles and concepts from:

  • Sociology,
  • Anthropology, and
  • Psychology

The preceding outline is drawn from pages 3 and 17 of Social Studies: Content Knowledge published in 2001 by the Educational Testing Service in Princeton, New Jersey.

Appendix 3: Content Overview for Principles of Learning and Teaching (all grades)

I. Students as Learners: 35% of 4 case histories, each with 3 free response questions and 24 multiple choice items.

A. Student development and the learning process

  • Theories of development
  • Contributions of key theorists
  • Terminology
  • Human development in the physical, social, emotional, moral, and cognitive domains

B. Students as diverse learners

  • Differences in the ways students learn and perform
  • Areas of exceptionality in student learning
  • Legislation and institutional responsibilities relating to exceptional learners
  • Approaches for accommodating various learning styles, intelligences, or exceptionalities
  • The process of second language acquisition, and strategies to support the learning of students for whom English is not a first language
  • How students’ learning is influenced by individual experiences, talents, and prior learning as well as language, culture, family, and community values

C. Student motivation and the learning environment

  • Theoretical foundations of human motivation and behavior
  • Terminology
  • How knowledge of human motivation influences strategies for organizing and supporting individual and group work in the classroom
  • Factors and situations that are likely to promote or diminish students’ motivation to learn
  • How to help students become self-motivated
  • Principles of effective classroom management and strategies to promote positive relationships, cooperation, and purposeful learning

II. Instruction and Assessment: 35% of the total test score.

A. Instructional strategies

  • Major cognitive processes associated with student learning
  • Major categories of instructional strategies
  • Principles, techniques, and methods associated with various instructional strategies
  • Methods for enhancing student learning

B. Planning instruction

  • Techniques for planning instruction to meet curriculum goals, including the incorporation of learning theory, subject matter, curriculum development, and student development
  • Techniques for creating effective bridges between curriculum goals and students’ experiences

C. Assessment strategies

  • Measurement theory and assessment-related issues

III. Communication Techniques: 15%.

A. Basic, effective verbal and nonverbal communication techniques

B. Effects of cultural and gender differences on communication in the classroom

C. Types of questions which can stimulate discussion in different ways for particular purposes

IV. Profession and Community: 15%

A. Reflective practitioner

  • Resources available for professional development and learning
  • Role of personal reflection on teaching practice and approaches that can be used to reflect and evaluate

B. The larger community

  • The role of the school as a resource for the larger community
  • Factors in the students’ environment outside of school that may influence their lives and learning
  • Basic strategies for involving parents/guardians and leaders in the community in the educational process
  • Major laws related to students’ rights and teachers’ responsibilities

The preceding outline was drawn from pages 3 and 11 through 30 of Principles of Learning and Teaching Study Guide, published in 2001 by the Educational Testing Service in Princeton , New Jersey.  Item types and proportions are based on the design of the test as used after 1 September 2002.

D. Leitzman, June 2004

M:\NCATE2005\Standard1\PRAXIS II Test Performance sp04.doc