Development, validation, and implementation of the elementary mathematics motivation inventory (EMMI): examining motivational constructs in elementary mathematics

Abstract

This article details the development, validation, and implementation of the elementary mathematics motivation inventory, a survey instrument designed to measure motivational constructs related to mathematics in elementary students. Early elementary years are critical in students’ development of a wide range of attitudes toward schooling, including their academic motivation. Currently, there is limited research on elementary students’ motivation for mathematics since this construct was difficult to measure in young children. During Year One of this 2-year study, a 17-item survey was developed based on theoretical constructs of motivation and piloted with a small sample of third grade students (N = 79). Revisions were made based on pilot data and the subsequent survey was administered to a larger sample of first through fifth grade students (N = 1018) during Year Two. In addition to describing the development and validation process for the elementary mathematics motivation inventory (EMMI), the authors present a discussion of the findings from Year Two of this study. Future research directions and implications for classroom practice are also discussed.

Introduction

Measuring psychological constructs in young children is often a challenging process. Estimating constructs such as motivation in elementary students can prove to be problematic and inconsistent [50]. As a result of these challenges, motivation research has focused on older students, usually at the secondary level. However, early elementary years are thought to be critical in students’ development of a wide range of attitudes toward schooling, including their academic motivation [31, 37]. These early years have the potential to affect students’ subsequent development, setting a foundation for either success or failure in later grades. The formative nature of the elementary grades signals the need for more effective methods of measuring motivational constructs in young children, especially when high motivation has been found to correlate with positive academic outcomes [16, 20]. The current study details the development, piloting, and implementation of an instrument designed to measure these critical motivational constructs specifically in elementary mathematics students.

While multiple instruments exist to measure general motivational constructs [18, 29] there are limited resources available to researchers and educators desiring to measure content-specific constructs of motivation in mathematics. For example, the patterns of adaptive learning survey [29] can be adapted to a specific domain to measure motivational constructs across broad academic domains (i.e., math, science). However, there is a need to drill down further into student efficacy and value for specific mathematical strands, such as number and operations. A primary goal in the development of the EMMI was to provide researchers and teachers this level of specificity in measuring student motivational components at these micro-levels.

Many current instruments contain complex sentence structures that are difficult for young children to process [18, 29, 57]; the EMMI uses simple, concise statements written at a level aimed to reduce cognitive load to allow elementary age students to comprehend and respond reliably to each item. In addition, most self-report instruments rely on a numerical Likert scale or a system of “emoticons” in which students identify their agreement based on a scale (e.g., 1–5 with 1 representing a low level of agreement and 5 representing the highest level of agreement with a given statement). These scales are often too abstract for young children and result in low reliability for self-report measures in elementary grades [38]. Consequently, our goal was to create a developmentally appropriate scale that allowed each child to relate to the response in a concrete way. The EMMI’s response options rely on a first-person perspective (i.e., “Just like me” and “Not like me”) in order to support concrete thinking as students process and respond to each item.

Theoretical framework

Motivation and student achievement

Motivation is a psychological construct that is fundamental to students’ academic success [45]. Motivation can be defined as “the processes that energize, direct, and sustain behavior” [44, pp. 414]. Highly motivated students may be more likely to engage in behaviors that enhance academic performance [15, 16, 31, 54], including effective goal-setting, focusing effort, and persisting in academic challenges [35, 51]. Highly motivated students also are more likely to view academic tasks as valuable and important [27, 56].

Students’ levels of cognitive engagement and use of metacognitive strategies have also been connected to their motivation [39]. Students who display high motivation for a task may employ more efficient cognitive strategies, thereby encoding new information more effectively [31, 43]. These students also employ critical thinking skills in problem-solving situations and integrate prior knowledge with new information. Highly motivated students may also utilize more effective metacognitive strategies such as planning how to approach a new learning task, evaluating progress, and monitoring comprehension of new material [40, 41]. In other words, motivated students are better equipped to learn than unmotivated students.

Research also indicates student motivation to be domain-specific, varying across curriculum areas [14, 26, 35, 49]. As early as third grade, students have demonstrated the ability to differentiate between subject areas in relation to motivation [2]. Specifically, student levels of math motivation have been associated with subsequent achievement in mathematics. For example, research by DiPerna et al. [16] indicated a significant positive correlation between motivation and math academic achievement for primary students. This study also revealed increased engagement with mathematical tasks for highly motivated students. Other research suggests that students who have high intrinsic math motivation may also have a greater conceptual knowledge of math topics [20, 34, 48]. In short, previous research points toward a link between student motivation and achievement. This connection reinforces the need for an instrument to reliably measure motivation in young children.

Motivational theories: survey development

Several key theories in motivation guided the development of the elementary mathematics motivation inventory, the motivational survey detailed in the present study. Items were written to prompt a student judgement related to the following constructs: self-efficacy, efficacy × value, and math anxiety. These constructs are described below.

Self-efficacy

Bandura [8, 9] noted that self-efficacy can be predictive of an individual’s motivation, affect, and behavior. In addition, studies have found that self-efficacy is a strong predictor of students’ course selections and career choices [6, 13]. Self-efficacy is a central concept to the development of students’ academic motivation [6, 10]. Students with high self-efficacy for a task have confidence in their ability to perform the task effectively. In contrast, low self-efficacy is marked by a lack of confidence in one’s abilities to succeed at a given task or domain [40, 41].

Studies indicate that self-efficacy is positively correlated with student achievement [15, 16, 31, 47, 54]. Students who believe that they can perform well in a specific academic domain make healthier attributions for both success and failure, consequently supporting learning strategies that are associated with higher student achievement [53]. Since self-efficacy has been identified as a domain-specific construct [26, 35, 49], students may have high self-efficacy for some academic tasks and lower self-efficacy in other areas.

Expectancy × value

Expectancy-value theory posits that motivation is a function of an individual’s expectancy for success for a given task and the individual’s value for the task [17, 56]. Within this model, expectancies for success and task value are the two primary constructs related to an individual’s motivation. Interestingly, these constructs have been studied together and in isolation; research indicates that task value is often predictive of an individual’s choices or decisions while expectancies for success are more predictive of performance [17, 56]. Research also highlights the possible connections between task value and goal orientations [23]. For example, a student who holds a mastery orientation for learning mathematics may be more likely to perceive mathematics as useful and enjoyable.

In addition to students’ expectancies for success [8], motivation can also be affected by a student’s value for the domain or task. Task value is central to the expectancy-value motivational theory [17, 19]. Task value is generally discussed in terms of utility value, intrinsic value, attainment value, and cost [55, 56]). Utility value refers to the student’s perception of how useful a given task or domain is in his or her life. Intrinsic value references the student’s enjoyment of the task or domain. Attainment describes the perceived importance of succeeding at a task, while cost refers to the effort needed to complete a task. Task value can be predictive of students’ motivation for the given task or, in relation to the current study, science. Studies indicate that intrinsic and utility task value are predictive of students’ effort in academic settings [6, 19, 56].

Math anxiety

Many students exhibit a negative emotional reaction in response to mathematical tasks; this emotional response can be described as math anxiety [5]. Research indicates that anxiety may develop in response to experiences such as perceived or actual failures, social comparisons, real or imagined academic pressures, or physiological predispositions to anxiety [4, 28, 52]. Math anxiety is correlated with multiple negative student outcomes such as poor performance on standardized tests, avoidance of challenging mathematical tasks, and the development of negative beliefs about math abilities [4, 5]. Math anxiety often results in a negative feedback loop in which anxiety produces low academic achievement in math and in turn further strengthens the anxiety associated with mathematics [28]. This cycle often leads to far-reaching effects of math anxiety on other math-specific motivational components such as efficacy, value, and goal orientation.

The present study details the development, piloting, revision, and implementation of a scale to measure math motivation in young children, specifically elementary grades. Since motivation has been demonstrated to be domain-specific [26, 35, 49], this survey was written specifically to address constructs of motivation in relation to mathematical concepts. This study explores the following research question: do significant differences exist between student motivation in mathematics across grades one through five?

Methods: Year One

Year One of the current study consisted of the following: survey development, piloting the survey, and survey revision. The following sections detail this process and provide information pertaining to the validation of the EMMI.

Survey development

Initially, we consulted the literature in mathematics education and educational psychology in order to identify salient constructs in motivation to form the theoretical basis of the survey. Specifically, we focused on motivational constructs that previous research has found to correlate with student achievement in mathematics. A review of literature on student motivation and achievement indicated correlations between student achievement in mathematics and the following constructs: math anxiety, efficacy for mathematics, performance orientation and mastery orientation toward learning mathematics, and value for mathematics (as described in the “Motivational theories” section). Therefore, items in the instrument were written to prompt student judgements related to these underlying theoretical constructs; (1) math anxiety [46, 52], (2) self-efficacy [7], and (3) task value [56] and interest [22]. Since limited research has been conducted on young children and motivational constructs in mathematics, literature reflects studies including primarily secondary mathematics students.

The mathematics content reflected in this survey represents concepts drawn from the number and operations strand of the national standards as developed by National Council of Teachers of Mathematics [32, 33]. Including content from the other strands in these national standards would have necessitated a survey length that would have been developmentally inappropriate for young children. In addition, the number and operations standards comprise approximately half of the content taught each year at the elementary level [32, 33]. Subsequently, the need for additional scales reflecting content from alternate strands will be discussed as an area for future research.

After identifying key motivational constructs from previous research in mathematics, a 17-item survey was designed to measure these motivational constructs in elementary students. Differing items contained positively or negatively phrased statements of disposition toward math, prompting students to respond with a four-point Likert-type scale (“Just like me,” “Sort of like me,” “Not really like me,” and “Definitely not like me”). Items were then reviewed by a measurement expert for clarity and appropriateness and the scale was evaluated for reading level. The survey was determined to have a Flesch–Kincaid reading level of 3.1. This measure indicates that the readability level of the EMMI is approximately on a third-grade level. Table 2 provides a list of all items in the EMMI as well as factor loadings for each item, as described in the following section.

Pilot study

Following survey development, a pilot study was conducted to examine the reliability and validity of the instrument. The survey was piloted with elementary students (N = 79) at a school in the southeastern United States. These students (37 girls and 42 boys) were from six third grade classrooms and participation was contingent upon parental consent. Ages of participants ranged from 8 to 10 years. In order to control for reading level and maintain consistency in implementation, the survey was administered orally by one of the lead researchers on this study. Following survey administration, negatively phrased items were coded positively and all data were analyzed using measures of reliability, item analysis, and factor analysis.

Results: Year One

Reliability

Reliability for the math motivation scale was estimated by computing the Cronbach’s Alpha (α = .91). This coefficient demonstrated high reliability for the scale. Further, an examination of “Cronbach’s Alpha if Item Deleted” suggested that all items should be retained.

Construct validity: factor analysis

The dimensionality of the 17 items from the motivation scale was analyzed using principle components factor analysis (PCA). A varimax rotation yielded four factors with eigenvalues greater than one, and each factor yielded an interpretable factor solution. Seven items loaded on more than one factor. Factor 1 (which accounted for 45% of item variance) was defined by eleven of the scale items, which may explain the overlap of items on more than one factor. Because these items were related to varying motivational constructs, Factor 1 was labeled math motivation. Factor 2 (which accounted for an additional 9% of item variance) was defined by eight of the scale items and was labeled self-efficacy. The third Factor was defined by five of the scale items, accounted for 7% of item variance, and was labeled performance orientation. Factor 4 (6% of variance) was defined by two scale items and was labeled expectancy × value. However, due to the low number of participants in the pilot survey (N = 79), this factor analysis was only used as a cursory examination of the survey items to determine if any should be removed for the validation study. Although two items did not load on to any of the factors, they were retained to examine further with a larger sample.

Survey revisions

Results from the pilot study indicated high reliability for the survey (α = .91). Analysis of pilot data indicated no increase in reliability by eliminating specific survey items, therefore all 17 original items were retained in the motivation scale. Survey administration did reveal student requests for clarification on one specific item (“I do poorly in math.”), so this statement was reworded for clarity (“I make good grades in math.”). The survey was also reviewed again by the measurement expert in order to ensure construct validity. Recommendations from this review led to revisions on an item reading: “I know I can learn new things in math.” This item was rephrased to read, “I feel confident that I can learn new things in math.” This revision was made in order to better align the intent of the statement with the construct of self-efficacy.

Methods: Year Two

Year Two consisted of the following: participant selection, data collection and analysis, and validation of the survey instrument. The revised survey from Year One was administered to a larger sample to compare levels of mathematics motivation among first through fifth grade students. The following section details the methods for Year Two.

Participants

Participants for Year Two were identified through criterion sampling techniques. First through fifth grade students from five elementary schools in southeastern United States were recruited for this study. The five schools were partners in a 4-year, state-funded grant committed to improving teacher quality through the implementation of an inquiry-based K-5 mathematics curriculum. During each grant year, teachers in these five settings implemented a portion of the curriculum and participated in the concurrent ongoing professional development model. This study was conducted in the spring during the third year of the grant. Due to the transient nature of enrollment at these five schools, it is unclear if all participants were exposed to the curriculum prior this school year.

The intent of this study was to compare levels of mathematics motivation for students at different grade levels. Therefore, researchers limited participant selection to these five elementary schools in an effort to control confounding variables. Teachers were all using the same curriculum and had participated in the same professional development model. The demographics of students in each school were also very similar as displayed in Table 1. Researchers worked with the math coaches at each school to recruit participants for the study. Each math coach asked teachers to send home a consent letter to inform parents or guardians about the study. While parent consent was required, student participation was still voluntary. A total of 1018 students participated in the study. The breakdown of participants at each grade level was as follows: grade one (137 students), grade two (209 students), grade three (241 students), grade four (200 students), and grade five (231 students).

Table 1 Demographics of Year Two participating schools

Context: curriculum implementation

Teachers in these settings were in the process of implementing a K-5 mathematics curriculum designed with four interrelated strands that connect vertically across grade levels and horizontally across mathematics content areas [30]. Lessons in this constructivist-based curriculum are designed around a four-step learning cycle where students engage in meaningful tasks that foster sense making in mathematics [24]. Teachers in the five participating schools began implementing the curriculum during the first year of the grant. Teachers started with the algebra and data strand of the curriculum and continued adding additional strands each year. In the school year when this study was conducted, teachers in these settings had implemented three out of the four strands and used their traditional textbook to cover their remaining standards. Prior to implementing each strand, teachers participated in a 2- or 3-day content-focused professional development experience where facilitators exposed teachers to a vertical articulation of the mathematics content from kindergarten through fifth grade.

During these experiences, teachers also had the opportunity to become immersed in the lessons while the facilitator modeled how constructivist learning and teaching should look in the classroom. By experiencing the lessons as students would, teachers were able to recognize their role in fostering the types of discourse that could occur in a heterogeneously grouped collaborative learning environment. In addition to these content-focused experiences, teachers participated in reflective professional development sessions throughout the school year to examine student work and discuss issues and success related to curriculum implementation.

Data collection and analysis

Data collection for this study took place over the course of 2 weeks in May of the school year in which the survey was administered. Researchers distributed the 17-item surveys to the math coach at each of the five schools who, in turn, distributed them to every first through fifth grade teacher. Teachers were asked to administer the survey in an effort to increase the level of trustworthiness for students. Models of assessment for early childhood and elementary education recommend that students complete assessments in a natural setting, such as their classroom, with an assessor with whom they have an established relationship, such as their teacher [12].

All participants completed the survey in their regular classroom setting. Teachers were given a specific protocol to read prior to administering the survey to establish a high level of validity related to instrument implementation. However, researchers were unable to ensure that the protocol was followed correctly in every classroom. First through third grade teachers read each item aloud to participants to control for difficulties associated with reading comprehension. These participants were allotted 10–15 s to complete each item before the next one was read aloud. Students who decided not to participate were asked to put their names on the survey and then to leave it blank. Teachers collected all surveys and returned them to the mathematics coach. Researchers collected the completed surveys from the mathematics coaches.

Two objectives were established for data analysis: (1) to examine validity and confirm the overall constructs of the survey instrument, and (2) to examine levels of mathematics motivation in elementary age students. Internal consistency of the scale was established through an analysis of the Cronbach’s Alpha. PCA was conducted to examine construct validity [36]. Once factors were confirmed, analysis of variances (ANOVAs) were conducted to determine if differences in levels of motivation existed between grade levels and the factors confirmed through PCA.

Year Two: Results

Reliability

Reliability for the mathematics motivation scale was estimated by computing the Cronbach’s Alpha (α = .83). This coefficient demonstrated high reliability for the scale. Further, an examination of “Cronbach’s Alpha if Item Deleted” suggested that the Cronbach’s Alpha increases to .84 upon deletion of two items in the scale. However, these items were retained since this increase was not significant enough to warrant their removal from the scale.

Construct validity: factor analysis

The dimensionality of the 17 items from the motivation scale was analyzed using PCA. A varimax rotation yielded three factors with eigenvalues greater than one, and each factor yielded an interpretable factor solution. Three items loaded on more than one factor. Factor 1 (which accounted for 29% of item variance) was defined by six of the scale items. Because these items were related to varying constructs of mathematics anxiety, Factor 1 was labeled mathematics anxiety. Factor 2 (which accounted for an additional 11% of item variance) was defined by seven of the scale items and was labeled mathematics self-efficacy. The third Factor was defined by five of the scale items, accounted for 8% of item variance, and was labeled value of mathematics. Table 2 outlines these results for Year Two.

Table 2 Factor loadings of EMMI survey items

Analysis of variance

A one-way analysis of variance was conducted to evaluate the relationship between grade level and each of the following factors: (1) mathematics anxiety, (2) mathematics self-efficacy, and (3) value of mathematics. The independent variable, grade level, included five levels, represented by grades one through five. The dependent variable was the student score for each factor on the motivational scale, identified through factor analysis.

The ANOVA for grade level and mathematics anxiety was significant, F(4,1013) = 13.554, p < .001. Follow-up tests were conducted to evaluate pairwise differences among the means. Levene’s statistic revealed equal variances, resulting in the use of Bonferroni’s post hoc comparison. Students in grade one reported significantly lower mathematics anxiety than grades three, four, and five. Grade two also reported significantly lower mathematics anxiety than grades four and five.

The ANOVA for grade level and mathematics self-efficacy was also significant, F(4,1015) = 5.928, p < .001. Bonferroni’s post hoc was selected to evaluate pairwise differences among the means, since variances were equal. Students in grade one reported higher mathematics self-efficacy than students in grades three, four, and five. There were no additional significant differences between self-efficacy and grade level.

The ANOVA for grade level and value of mathematics was significant, F(4,1013) = 3.215, p < .001. Finding equal variances, Bonferroni’s post hoc was conducted to examine differences among the means. Students in grade five reported significantly lower value for mathematics than students in grades two and three.

Discussion

Instrument development

The development of the EMMI represents an attempt to provide a reliable, valid measure for motivational constructs related to mathematics in young children. Throughout a series of revisions to the original scale, a developmentally appropriate measure has emerged featuring three subscales, which assess the following constructs: mathematics anxiety, mathematics self-efficacy, and value of mathematics. During the piloting of the original instrument, four subscales were identified using principal components factor analysis (PCA). However, this pilot was conducted with a relatively small sample (N = 79). In the subsequent implementation of this instrument with a larger sample (N = 1018), a three-factor solution emerged through PCA. This three-factor solution using the larger sample provides a more powerful analysis of the distribution of items into construct-specific subscales. In addition, the three-factor solution provided subscales that were more closely aligned with the literature on self-efficacy, anxiety, and task value. Consequently, we made the decision to define the EMMI as a measure comprised of three subscales, with the intention of conducting confirmatory factor analyses in future studies with this instrument to further test this distribution of items.

Measures of internal consistency also support the reliability of this instrument. In the large-scale study during Year Two, reliability for the instrument was high (Cronbach’s Alpha = .833), yet in a range that suggests discriminate validity in items within separate subscales. Obtaining these results in a large sample of children ranging from grade one through grade five suggests the strength of this instrument to provide a reliable measure of mathematics motivation in young children.

Instrument implementation

Results from the Year Two study indicate that students in the lower grades at these five schools report less mathematics anxiety and higher self-efficacy in mathematics as is consistent with other studies examining mathematics motivation at levels beyond elementary grades (e.g., [1, 42]). These results raise questions about possible reasons for this discrepancy, including the need to investigate issues of increased math anxiety as a result of testing pressure or rising levels of difficulty in content. The correlation existing between math anxiety and achievement in mathematics has been well documented in the research literature [3, 11]. The results from this study also provide continuing evidence of the possible correlation between mathematics anxiety and reduced self-efficacy in mathematics in young children as is consistent with other research (e.g., [21, 25]).

These results also raise questions about factors at the schools that may account for the reduced levels of motivation at the higher grades such as classroom environment, teacher quality, or fidelity of implementation. Fifth grade students at these schools report a significantly lower value of mathematics than students in other grades. Value is essential for students to be motivated in mathematics [16, 17]. It is necessary to explore how the role of the teacher and the curriculum account for such a change in attitudes. It would also be beneficial to compare the motivation of upper elementary students whose teachers are using an inquiry-based curriculum to students whose teachers are using a more traditional mathematics curriculum where procedural knowledge is emphasized over the development of conceptual understanding. While students at the upper elementary grades report less motivation for mathematics than students in lower grades, it is unclear if they have more motivation than students in other schools where a traditional mathematics curriculum is implemented.

Implications

Implications for instrumentation

The EMMI has the potential for use with researchers and practitioners. This measure offers a reliable, valid measure of mathematics motivation in young children, and is distinct from other mathematics motivation scales which are typically designed for secondary mathematics students. Consequently, this instrument represents an advancement in the field of content-specific motivation, as these constructs have previously been challenging to measure in young children. In addition, practitioners can use this scale as a form of assessment of student motivation in mathematics in elementary grades. Research suggests that these early years are critical to students’ development of attitudes regarding mathematics [37]. As a result, teachers need a tool to reliably assess these constructs in their students. For example, identifying high mathematics anxiety in a student may provide a teacher the opportunity to focus individualized attention on mathematics content that may be the root of the anxiety. In addition, teachers who notice a low task value for mathematics within their classroom may develop strategies for making the mathematics more meaningful to students and making connections between mathematics content and real-world applications.

Future research using the EMMI will include additional work to further evaluate the current three-subscale model of the instrument and to measure reliability of the instrument with different student populations. In addition, we plan to develop scales focusing on mathematics content strands such as algebraic reasoning, geometry, measurement, data analysis and probability, and problem solving. Since the current instrument focuses on number and operations, future scales will provide measures of students’ motivation related to other strands of mathematics. Since self-efficacy has been demonstrated to be content-specific, we conjecture that it may also be concept-specific, possibly varying between various strands in mathematics.

Implication for practice

The Year Two study provides insight into students’ motivation in mathematics in relation to grade level. These data may be helpful in informing curriculum decisions and instructional practices that may sustain students’ motivation as they progress from early-to-late elementary grades. Special attention should also be given to increasing levels of mathematics anxiety as students advance through the grade levels. Identifying and reducing this anxiety in students in the mid-to-late elementary years may result in more positive experiences with mathematics in the middle school years. In addition, a drop in reported value for mathematics during fifth grade may suggest a need for authentic experiences that help make mathematics meaningful for these students.

The EMMI instrument can be a tool for continuing research on student motivation in mathematics. By identifying levels of motivation in elementary students, researchers can then qualitatively examine reasons why students report high or low levels of motivation for mathematics. Researchers can also examine the relationship between motivation and other factors such as student achievement and understanding of mathematics, student discourse, teacher beliefs about teaching mathematics, and teacher content knowledge and pedagogical content knowledge in mathematics. These explorations are necessary to build a comprehensive understanding of the complexity that exists when examining motivation.