Re: Regarding Maths --- An Idea

Re: Regarding Maths --- An Idea

[David Arnold]

Hans,

Whew! You really want to do this right!:-)

Let me share a few more thoughts with you on how our math department at the
College of the Redwoods might use such a package. Please forgive my
rambling style.

First, I don't believe that the department is ready for complete online
testing. We really believe "showing the work" is most important. However,
we also believe that some multiple choice of "fillin" testing has its
place. For example, I typically design tests that use multiple choice to
test little quick calculations, like "what is the x-coordinate of the
vertex of the parabola whose equation is given by y=x^2-2x-3?" However,
about half of my exams still require free response, where I expect to see
all of the work. For example,

5. Consider the equation y=-x^2-4x-5.

(a) Use the method of completing the square to place the equation in
"vertex form." Identify the coordinates of the vertex.

(b) Find the x and y-intercepts of the parabola. Show your work.

(c) Sketch the graph of the parabola on a sheet of graph paper. Label the
vertex, x and y-intercepts with their coordinates. Draw the axis of
symmetry and label it with its equation.

6. Consider the equation e^{-0.12x}=x+1.

(a) Use a graphing calculator to draw the graph of each side of this
equation. Select a viewing window that captures the point of intersection
of the two graphs and make a copy of your view screen on your examination
paper.

(b) Use the intersect utility of your graphing calculator to find the point
of intersection. Label the point of intersection in part (a) with the
coordinates found by the intersect utility.

etc., etc.

Also, we just don't have the facilities or hardware to do massive online
testing. Consequently, in class examinations will remain an essential
ingredient of our curriculum for quite some time. 

However, there are a number of ways that we would take advantage of online
testing. Let me explain a few.

1. Placement testing. Our college testing center requires that students
enrolling for the first time take placement examinations to help guide the
students into a proper level according to their ability. Placement exams in
both english and mathematics are required. Typically, these exams are
completely multiple choice and are machine scored. At the moment, these
exams have to go through a validation process, so we would not be able to
replace these immediately with new efforts. However, on our web site we
could design pre-placement tests for students who may be thinking about
attending our college. They could be similar to the actual placement tests
given by the testing center and would provide online readers an opportunity
to see how they stand before taking the exam in the testing center.
Currently, our testing center offers a booklet for purposes of review.
Online pre-tests could help students review material at a level they think
they would like to enter our curriculum. Now, in such cases, we wouldn't
care at all if students "cheated" on the exam. What we definitely would
want in this situation is one single exam, and a scoring button at the end
which tells the online viewer the result when finished. Moreover, we would
want some method of marking the correct and incorrect answers, and we would
want the ability of allowing the online viewer the opportunity to click a
correct answer and be provided with a complete worked out solution that
they could compare with their notes to see if they understand the concept. 

2. Pre-tests. A pre-test would be something that the students could work on
in preparation for an actual inclass examination. See, for example, see
exam #5 at the bottom of page

http://online.redwoods.cc.ca.us/instruct/darnold/ColAlg/exams.htm

Again, in this case, we don't care at all about "cheating." Typically, this
is a hyperlinked question-answer pair where we present the question and a
link to the answer, with the answer containing a link back to the question.
We would want on onscreen version and the ability to select a version
suitable for printing. It would be important to disable printing of the
onscreen or online version. This format was very successful last semester
for two reasons. First, the students really liked it. Second, I really
liked it because I was able to share the workload with my colleague. I
would type up the questions, and he would typeset the answers. This sort of
coordination on exams also helped us pace our classes and collaborate in a
large number of ways that we never would have realized if we didn't work
together on the examinations.

3. Tutorials. In the event that students are having difficulty with a
particular topic, say the parabola, we can design online tutorials for
student improvement. Again, we wouldn't care much if students "cheated" on
these tutorials, because what's the point? However, here we would very much
like to record student responses and tally results for two important
reasons. First, if we have a record of how each student did on each
question, we can counsel him in office hours. We can say, "Look, you had
difficulty with this type of problem. Go to such and such page, try these
problems, then go back and try the tutorial again." Second, if we have an
overall tally on how all students did who took the tutorial, we would be
able to adjust our curriculum or our class presentations to emphasize and
address areas of weakness. Cumulative data would also help us in studying
our class offerings and curriculum globally, and afford us an opportunity
to rethink our class design.

Now let's discuss another level of testing, actual inclass testing. This
brings two areas to my mind.

1. Test Banks. We have some experience in the construction of test banks at
the College. I wrote an intermediate algebra text for Addison Wesley and we
designed test banks using the Scientific Workplace and its utility called
the ExamBuilder. This was back in the days of version 2.0 of SW.
Essentially, we created a template, and the template simply picked a random
multiple choice question from a database. For example, suppose a question
called for finding the x-intercepts of a parabola. Then we would write 6-10
questions similar to "Find the x-intercepts of the parabola whose equation
is y=x^2-2x-3." The template would choose one of these questions at random
from a file of 6-10 similar questions. A typical exam would have 10-15
multiple choice questions. The second part of the exam would have several
"essay" questions, similar to #5 and #6 above. The template would also
select these at random. So, consider #5 above, we would have a file
containing 6-10 questions that were similar, all residing in the same TeX
file, and the ExamBuilder would simply select one of them at random. The
result would be a test that was part multiple choice, part essay, it was
printed and administered in class. This test bank was used by most of the
teachers teaching our intermediate algebra course. I will now describe some
of the difficulties we had. 

A. Because teachers administered the exams on different days and different
hours, test security was a problem. We worked around this by asking our
teachers not to pass back the exams until all sections were administered
the exam during the exam week. However, students will talk, so the
randomness in selecting questions when an individual teacher "ran" a test
creation was important. Also, because some teachers had more than one
section, typically a teacher wanted more than one test per run, and he
wanted them to be significantly different, so truly random was not desired.
What was desired is that the second run select none of the problems
selected during the first run. However, there was no avoiding some
repetitition of exam questions when generating exams for 10-12 sections of
the class. We didn't feel too badly about some minimal repetition because
of our agreement to hold back returning the exams until all students were
tested. Also, each semester, we would add more  samples to each question.
Consequently, the file holding #5 might have 6 similar questions the first
time the test was administered, but the next semester it would be appended
and have 12. It grew and grew.

B. Some teachers wanted to ask a different number of multiple choice or
essay. For example, one teacher might feel uncomfortable asking 13 multiple
choice ("that's too many for my students") and want fewer. Consequently,
our basic template might call for 15 multiple choice and 5 essay questions,
but some would edit the template and choose 13 multiple choice and 4
essays, etc. Now, because of this decision being made by individual
teachers, what was important was a TOC, a list or description of the
possible questions to choose from, a preview if you like.

C. Finally, some teachers wanted to add a question or two of their own
design "I really want to ask about symmetry on this test and you don't have
any questions on symmetry in the test bank." So, it was important that
teachers be able to insert a question into the template when they desired.
This was easily done in TeX by simply typing in the question in the spot
they wanted to see it. 

We've moved away from the test bank idea in recent years, but I don't think
anyone would object to returning to it. It was successful and well
received, especially when individual teachers learned they could customize
the exams to meet their individual teaching needs. That really sold them on
using the test bank. Our department is currently discussing competency
based testing in our developmental (remedial) math courses , where students
must pass a set of 5 module exams or they don't pass. Students will be
given several tries to pass each module, with some penalty assessed for
each retake. There is talk of perhaps one day developing a testing center
to administer all such examinations. So, Context could be a lifesaver in
desigining and implementing the goals of this program.

2. Currently, I work by creating one exam with my colleague, part multiple
choice, part essay, then after the exam is administered, I simply post it
on our web site for future students to use for purposes of preparation for
upcoming exams. 

At 11:02 AM 10/9/99 +0100, you wrote:
>I cc to matthew, because he may be interested in doing the analysis
>part. His wife is an english teacher, and I would like to extend such an
>effort to humanities / languages as well. No reason to limit this to
>math. 
>
>David Arnold wrote:
> 
>> Now regarding your proposal of an update for Maths, once Taco gets his
>> macros together. I am very much interested in a project I would like to do
>> that would benefit both of us. If you can design a Context interface to
>> mimic that in intalg2.pdf for designing exams and tutorials, then I will
>> write an example  using the interface tha uses all of the math constructs
>> that Taco constructs. This, I think, would be a valuable service to users
>> of Context.
>
>So you want a system for student to click on answers, collect the data,
>calculate the score, etc etc. Actually, this is what the next alacarte
>(former exfiles) will be. Actually, I have something like that, but lack
>of time and application withheld me from going on with it. -)
> 
>> There are some specifics I think you should consider in the design of such
>> an examination module.
>> 
>> 1. There should be a way of identifying the student taking the quiz, exam,
>> tutorial, by creating some sort of sign-on in the beginning that includes
>> the student's name, identification number, and the class for which the exam
>> is being taken.
>
>Ok, trivial. 
> 
>> 2. The exam should be self correcting when the student finishes and report
>> a score and a grade. There should be some way of identifying right and
>> wrong answers. See attached test3.pdf.
>
>Sure, but in a more clever way I suppose. More shared js code, somehow
>isolating cq. disguising answers, for now a student with exchange can
>simply look at the javascipt to see what is right, even worse, run a
>perl script and get all answers right. 
> 
>> 3. It would be nice if we could include some way of allowing free response
>> to some questions.
>
>That's what I've bees thinking of for some time. Given that js has
>regexp's .... I was (really) thinking of a proper perl plug in.  
> 
>> 4. Finally, we need some way to get the data from the pdf into some sort of
>> database for teacher use. This is most important. Here is what I think we
>> need:
>
>Not that hard. 
> 
>> A. A record that student Hans took Exam # for class ? and his score.
>
>The same. 
> 
>> B. Ideally, there should be some way of keeping which problems the student
>> got right and which he got wrong. This would be for purposes of counseling
>> during office hours. A teacher could glance at the problems missed and make
>> some recommendations for improvement by assigning particular problems in
>> areas of weakness.
>
>Analyzing data afterwards. 
> 
>> C. It would be a good thing if there were available a summary of how all
>> students did on the quiz on a per problem basis. For example, 45% of the
>> students missed problem 32. That way, a teacher could look at general
>> weaknesses and design strategies for improvement of the class as a whole.
>> More time on this topic, less on this, etc.
>
>The same. 
> 
>> D. If you look through Donald Story's webeqman, also attached, you will
>> note a forpaper option that he designed. This was a HUGE timesaver that my
>> colleague and I used to design examinations last semester. By changing
>> exactly four lines of code, we were able to move from on onscreen version
>> to a printed version that we could actually distribute in class. Once the
>> actual inclass printed version was administered, we simply changed back the
>> four lines and we had on onscreen document to archive for student study in
>> the future. What a beautiful idea!
>
>Sure, but that's tex anyway, isn't it? 
>
>  texexec --mode=paper examn12
>
>with \startmode[paper] ... \stopmode stuff. 
>
>This fits in our plans for extending context with a computer assisted
>instruction module. It would also mean, a proper database approach, so:
>random examns with validated items. (Do you have such a collection? Long
>ago I assisted in colleges concerning data-analysis etc, a course on
>making proper questions/mpc answers, analysis and so. It's that kind of
>assistancy that gave me the opportunity to learn to program -)
> 
>> This is a project that I could really sink my teeth into. If we could get
>> the interface going, than I would be happy to write an examination in
>> college algebra, trig, or even calc that takes advantage of all the maths
>> designed by Taco.
>
>A proper database is indeed needed. 
> 
>> Let me know if you want to do this. I know you have plenty on your plate
>> already.
>
>Sure, but this is not that hard to do.  
>
>Hans
>
>
>-----------------------------------------------------------------
>                                          Hans Hagen | PRAGMA ADE
>              Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
>      tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.nl
>-----------------------------------------------------------------
>
>
>
David Arnold
College of the Redwoods
Mathematics Department
7351 Tompkins Hill Road
Eureka, CA 95501
(707) 476-4222

My Home Page
http://online.redwoods.cc.ca.us/instruct/darnold/index.htm

Ordinary Differential Equations Using Matlab
http://www.prenhall.com/books/esm_0130113816.html