Introduction 

Proof-writing is a core disciplinary practice of mathematicians and a crucial skill of all mathematics majors and future mathematics teachers. Developing students’ facility and comfort with proofs is an important objective of undergraduate Geometry for Teachers (GeT) courses (An et al., in press; Grover & Connor, 2000). GeT courses cultivate robust and flexible knowledge and skills such as writing, analyzing, and critiquing proofs that allow prospective secondary teachers (PSTs) to support their students’ learning of geometry (AMTE, 2017; González & Herbst, 2006). 

Technology plays an important role in supporting PSTs’ engagement with proof. Dynamic Geometry Environments like GeoGebra, Geometer’s Sketchpad, and others provide opportunities to explore geometrical properties and make and test conjectures (Jones, 2000; Mariotti & Baccaglini-Frank, 2018). These tools have been beneficial for fostering PSTs’ attitudes towards proof and comfort with proof and teaching it to secondary students (Abdelfatah, 2011; Kardelen & Menekse, 2017). However, they offer little support for writing a deductive proof of a conjecture or a theorem. 

With the advancements in artificial intelligence and machine learning, new tools have been created that support automated and interactive proof writing and verification (e.g., Lodder et al., 2021; Matsuda & VanLehn, 2005; Wang & Su, 2017). FullProof is a software developed to advance students’ proof-writing skills in Euclidean Geometry. Using FullProof, students write a step-by-step two-column proof, using an interactive diagram, an equation editor, and a library of theorems/postulates and definitions. The software checks the proof and provides feedback and/or hints to improve their work. 

Our project aimed to explore the potential benefits of FullProof for GeT students; our research question was: How and to what extent does engaging GeT students with the FullProof software affect their competence with, and beliefs about geometry proofs? 

Methods

Research Setting

In Fall 2021, we conducted an exploratory study by integrating FullProof into our GeT courses, which all have a large portion of the curriculum devoted to Euclidean Geometry from a synthetic perspective. Two GeT courses had a mixed student body of PSTs and STEM majors; one GeT course only had PSTs. Thirty students agreed to participate in the study across the three GeT courses. 

FullProof served as a support system for writing proofs in class and homework. Each instructor used about 20 proof problems from FullProof ‘s collection, on a variety of topics, at three levels of difficulty: high, medium, and low. Figure 1 shows a screen capture of a sample problem (medium difficulty) about triangle midsegments. 

Each element in the figure can be highlighted or marked when pointed to, and auxiliary lines can be added as needed (Fig. 2). The right side of Figure 1 shows the givens and the statement to be proven. The students write the proof by typing statements in the numbered lines and justifying them. The justifications can be searched by keywords or by browsing the FullProof justifications library. 

At any point in the process, the students may ask for a hint by clicking the hint button. The system will produce hints in order of increased specificity from a vague “try using triangle congruence,” to suggesting a certain theorem to use, to proposing a particular step–like “try proving ∆BFO ≅ ∆EGO.”

Once finished, the students click the “check” button, and in a few seconds, the algorithm checks the proof and provides feedback. Correct proof lines get a green check mark. Incorrect or partially correct lines are marked down with an explanation of the mistake. Figure 2 shows a solution with two mistakes – a missing proof step and a missing justification for the last step. Clicking on the notifications will show a student what the missing step was (here ∆BFO ≅∆EGO by angle-side-angle theorem) and how many points were deducted for each mistake.

Clicking the “try again” button allows a student to improve their work, resubmit, and receive more feedback. The number of hints and submission attempts is unlimited (except for the test mode), allowing the students to achieve a perfect score eventually. The information about the number of hints and submission attempts is stored in the system and is available on the instructor’s interface (Fig. 3). 

Data Collection and Analysis

To respond to our research question, we administered pre- and post-surveys via Qualtrics, which took about 20 minutes to complete. Most questions were Likert-type on mathematical identity and comfort level with writing proofs. The items were adapted from the literature (e.g., Kaspersen & Ytterhaug, 2020; Stylianou et al., 2015). Here, we focus on responses to the bolded open-ended questions, developed by the researchers for the post-survey, about student perceptions of FullProof (Fig. 4). We analyzed the data qualitatively, using open coding and thematic analysis (Patton, 2002) to reveal recurring themes in students’ responses, specifically the types of positive and negative appraisals about using FullProof. To capture all themes, comments that included multiple ideas were coded multiple times.  

Figure 4. Open-ended Items from the FullProof Survey.

Student Appraisals of the FullProof Platform 

The analysis of the positive and negative appraisals in students’ written comments on the post-survey revealed three main themes that emerged from the 61 positive appraisals of FullProof identified across all seven open-ended questions. The largest category (41%) described the affordances of FullProof for supporting students’ writing and understanding of proofs, including the searchability of reasons for proof steps, clear structure that supports communication and comprehension, interactive feedback, and the ability to pursue multiple solution paths. One student wrote, “FullProof has helped me a lot when writing proofs. I like how I can search for reasons if I am not completely sure about a reason/theorem.” Another student wrote, “FullProof made writing proofs easier not only to write, but also to understand.”  

The second theme (26%) described the advantageous technical features of FullProof, such as hints which help them move forward if stuck, interactive feedback pointing to errors, visual clarity, and color-coded elements of a diagram. The third theme (25%) described the affordances of FullProof as a pedagogical tool for teaching others. In this theme, the participants highlighted the elements of the software that would support them as teachers in the geometry classroom. Students wrote that FullProof presents “a good instructional strategy to implement in the classroom,” and the platform “makes it easier for the students to see what their errors were.” 

The 23 negative appraisals constituted 29% of the codes and were distributed rather uniformly across five themes. The main critique (7 out of 23 appraisals, 30%) concerned the discrepancy between the wording of the theorems in the FullProof platform and in class. For example, FullProof does not have the angle-angle-side triangle congruence theorem, so students need to complete an extra step and use the angle-side-angle congruence theorem. Another category of critiques (22%) described the variation in standards of rigor employed by FullProof vs. classroom instructors. The students wrote that the software allowed them to skip steps and earn full points for their solution, while they were aware that their instructor would have likely deducted points. One student wrote: “It allows some things to slide, that should be wrong.” 

Additional patterns emerged from the analysis of specific survey questions. When asked, “Has FullProof changed the way you write proofs,” the responses split almost evenly between yes and no. But in the written comments clarifying the forced choice, 83% of comments were positive regarding FullProof. A similar pattern was observed in the question: “How has FullProof changed your understanding of reasoning and proof?” Sixty-one percent of students wrote that FullProof positively influenced their understanding, but interestingly, 91% of their comments contained some specific description of the positive effect of FullProof. When asked, “Would you use FullProof in your future classrooms,” 14 out of 17 students who answered this question responded positively citing the advantages of the platform. When asked to describe some of the successes and challenges students had with FullProof, 65% of participants’ comments described challenges, as opposed to 35% that described successes. However, the overall impressions of FullProof were very positive with 72% of comments containing positive reviews of the tool. These results show positive appraisals of the FullProof platform outweigh the critiques of the tool. 

Discussion

Our quasi-experimental research design has several limitations. Due to the lack of randomization and a single experimental group, potential confounding variables were not controlled (Johnson & Christensen, 2012). Each course was taught by a different instructor, with no common curriculum or textbook across the institutions. There was also some variation in how instructors used FullProof in their courses, with one institution teaching in a hybrid mode due to the Covid-19 pandemic. Because of these variations, our study design emulates natural conditions of how instructors might use any technological tool within the unique constraints of their institutional environments. Against this backdrop, we find it encouraging that the qualitative analysis showed more positive appraisals (69) than negative appraisals (27) about using the platform which indicates the overall positive impact of using FullProof in the GeT course. Moreover, since our analyses did not distinguish between PSTs and other STEM majors, the observed potential advantages of FullProof may apply to all GeT students.

As instructors, we note several advantages to using FullProof in our GeT courses. The availability of multiple proof problems at various difficulty levels and the feedback provided by the FullProof platform allowed assigning a greater number of challenging problems, compared to the past. Since FullProof offers hints and multiple submissions, students were able to get assistance from the software itself. However, this also presented a challenge; some students abused this functionality to get hints on every proof step. In the future, we plan to limit the number of hints and submission attempts. Another limitation of FullProof was the observed lack of rigor in its auto-grading system. We observed cases that FullProof issued students three stars (excellent) on their proof when certain necessary proof steps were missing or the reasons were not matching the statements. This limitation makes FullProof more suitable as a learning tool instead of a formal assessment tool. However, the discrepancies in the grading standards between the software and the instructor can be leveraged to engage students in analyzing and critiquing proofs, which is an important learning objective of GeT courses.

Based on what we learned from the pilot study, we modified the research design and are currently collecting the second round of data in our GeT courses. Some changes include shortening the surveys and assigning a few common FullProof problems across courses. While we feel FullProof is overall an efficient tool in facilitating the learning of proof, we want to further study how and to what extent it improves GeT students’ proof learning experience.    

References

Abdelfatah, H. (2011). A story-based dynamic geometry approach to improve attitudes toward geometry and geometric proof. ZDM-Mathematics Education 43, 441–450. doi.org/10.1007/s11858-011-0341-6

Association of Mathematics Teacher Educators. (2017). Standards for Preparing Teachers of Mathematics. https://amte.net/standards   

An, T., Boyce, S., Brown, A., Buchbinder, O., Cohen, S., Dumitrascu, D., Escuadro, H., Herbst, P., Ion, M., Krupa, E., Miller, N., Pyzdrowski, L., J., Sears, R., St. Goar, J., Szydlik, S., Vestal, S. (in press). (Toward) Essential student learning objectives for teaching geometry to secondary pre-service teachers. AMTE Professional Book Series, Volume 5: Reflection on Past, Present and Future: Paving the Way for the Future of Mathematics Teacher Education.

González, G. and Herbst, P. (2006). Competing arguments for the geometry course: Why were American high school students supposed to study geometry in the twentieth century? International Journal for the History of Mathematics Education, 1(1), 7-33.

Grover, B., & Connor, J. (2000). Characteristics of the College Geometry Course. Journal of Mathematics Teacher Education, 3, 47–67.

Hanna, G., Reid, D. A., & De Villiers, M. (Eds.). (2019). Proof Technology in Mathematics Research and Teaching. Springer International Publishing.

Johnson, B., & Christensen, L. (2012). Educational research: Quantitative, qualitative, and mixed approaches (4th ed.). SAGE Publications, Inc. 

Jones, K. (2000). Providing a foundation for deductive reasoning: Students’ interpretations when using dynamic geometry software and their evolving mathematical explanations. Educational Studies in Mathematics, 44(1), 55–85.

Kardelen, G., & Menekse, S. T. B. (2017). Views of pre-service teachers following teaching experience on use of dynamic geometry software. Educational Research and Reviews, 12(24), 1208-1219.

Kaspersen, E., & Ytterhaug, B. O. (2020). Measuring mathematical identity in lower secondary school. International Journal of Educational Research, 103, 101620. https://doi.org/10.1016/j.ijer.2020.101620

Lodder, J., Heeren, B., Jeuring, J., & Neijenhuis, W. (2021). Generation and Use of Hints and Feedback in a Hilbert-Style Axiomatic Proof Tutor. International Journal of Artificial Intelligence in Education, 31(1), 99-133. 

Mariotti, M. A., & Baccaglini-Frank, A. (2018). Developing the Mathematical Eye Through Problem-Solving in a Dynamic Geometry Environment. In Broadening the Scope of Research on Mathematical Problem Solving (pp. 153-176). Springer.

Matsuda, N., & VanLehn, K. (2005). Advanced Geometry Tutor: An Intelligent Tutoring System for Proof-Writing with Construction. Proceedings of the Japan National Conference on Information and Systems in Education, (Vol. 125, pp. 443-450). 

Patton, M. Q. (2002). Qualitative research & evaluation methods. Sage.

Stylianou, D. A., Blanton, M. L., & Rotou, O. (2015). Undergraduate students’ understanding of proof: Relationships between proof conceptions, beliefs, and classroom experiences with learning proof. International Journal of Research in Undergraduate Mathematics Education, 1(1), 91–134.

Wang, K., & Su, Z. (2017). Interactive, intelligent tutoring for auxiliary constructions in geometry proofs. arXiv:1711.07154v1