Session T3H

Non�Competitive Programming Contest Problems as the Basis for Just-in-time

Teaching

Owen L. Astrachan Department of Computer Science

Ouke Unilrcnily, Dluham, NC 27708 olaecs.duke.edL\

A bstTact -We report on the successful use of small program­ming assignments that augment a just-in-time approach to teach­ing programming. and problem solving in the first year of com­puter science. The emphasis in these assignments is on algo­rithmic problem:solving rather than on object-oriented design and programming. Students are given a terse explanation of a problem and several non-exhaustive test cases. A programmed solution uses no 110, but is tested via a web-based submission system that reports success or failure for between 10 and 40 test cases. Students compile. test, and debug via the online web­based system. There is no limit on the number of times students can test their solutions. After submission for grading. solutions are judged solely on whether they pass tests. all of which can be seen via the online testing mechanism, and not on the so-called quality of the code submitted.

Keywords - Programming, Algorithms. Active Learning. Problem Solving

INTRODUCTION

Although many universities and schools are now basing the first year of instruction on object-orientation ([I. recent com­mentary [2] indicates a nascent, yet growing dissatisfaction with a diminished emphasis on algorithmic problem-solving due to an increased emphasis on object-oriented design and its con­comitant emphasis on event-driven programming [31. GUls, and Design Patterns [4], [51, [61.

In this paper we report on the successful use of small (one method/one class) programming assignments that augment a

just-in-time approach [7] to teaching programming. algorithms. and problem solving. We use these small assignments. which we call weeklies, to augment our more lengthy object-oriented pro­gramming assignments. The small assignments emphasize al­gorithmic problem-solving and functional decomposition rather than object-oriented design. They employ the same kind of test­by-failure found in unit-testing [8] software like JUnit [9].

Students are given a terse explanation of a problem and three­to-six test cases. A programmed solution uses no liD (except for aid in debugging). but is tested via an online, web-based mech­anism that reports success or failure for ten-to-forty test cases. Students compile, test, and debug via the online web-based sys­tem and only after extensive testing submit their solution for credit. Solutions are judged solely on whether they pass tests, all of which can be seen via the online testing mechanism, and not on the so-called quality of the code submitted. However, we do use student solutions as the basis for subsequent class discus­sion. so the quality of code is part of the learning process, just not the grade awarded to these assignments. We have required

students to solve one to three of these problems each week in our first year programming courses for the past two years. Problem genres include, but are not limited to, no-loop programs, array processing, two-dimensional flood-fill with backtracking, use of sets and maps, and depth-first search in graphs. We have based this approach on how problems are presented and solved in the online Topcoder contests (http://www.topcoder.com). but we have removed all competitive aspects from the problem­solving process.

This approach bas been successful on many levels. The prob­lems are usually simply explained, though not always simple to solve. We discuss potential approaches, techniques, and solu­tions as part of our interactive lectures. The solutions provide us with a foundation for our lectures and motivate student inter­est. Students are very receptive to these problems with enthusi­asm expressed across gender and ability levels, though we have not measured this precisely. Although nol conclusive, informal surveys and analysis show that when students are given these algorithmic problems to solve we find they develop into better programmers as measured by ability to work on the larger pro­gramming projects we also require. We have used these prob­lems successfully in workshops with high school teachers learn­ing Java and are in the process of making them available to the general academic community by partnering with both academic and commercial enterprizes like Topcoder.

WHY ALGORITHMIC PROBLEM SOLVING?

Our introductory programming courses are similar to first­year courses at most universities following recommendations from ACM and IEEE guidelines for the first year of computer science. From 1994-2004 we used C++ in our first two courses, increasing the emphasis on object-orientation each year. Be­ginning in the Fall 2004 semester we will move these courses to Java. As our emphasis on object-orientation increased, we found students were less able to solve simple programming problems at the functional level. Although in some ways a class invariant is similar to a loop invariant, skills in thinking at the class level do not simply transfer to developing loops.

Our programming assignments usually require writing parts of several classes that fit into a larger framework [10]. To complete these assignments students must understand a per­assignment API to see how the code they write will be called-by and will call methods in other classes supplied as part of the assignment. For example. students will sometimes be asked to write the model in a model-view-controller (MVC) design. Our assignment write-ups are typically five or more pages with de­tailed instructions on how to use the per-assignment API, how to test, how the assignment will be graded, and how to submit it.

0-7803-7691-61031517.00 @Z004 IEEE October 20 - 23, 2004 Savannah, GA 34th ASEEIJEEE Frontiers in Education Conference

T3R-20

These details are necessary given the complexity of the assign­ments, but they do not offer practi� in algorithmic problem­solving.

As our students became more adept at reading and using these APIs. they had less practice in designing loops, writing func­tions, and using algorithmic techniques to solve simple prob­lems. We investigated integrated test-and-practice frameworks such as the Code Lab testing framework provided as a commer­cial service supported by TUring's Craft [11]. but the problems posed there are too constrained because of the methodology em­ployed to test student code. The problems are typically designed to test language-features rather than problem-solving as we see it. For example, two of the their array-mastery problems are

duplicat�d below (copyright [11 J);

Assume that an array o/integers named a has been declared and initialized. Write a single statementlhal assigns a new value to theftrst element of the array. The new value should be equal 10 twice the value stored in Ihe fast element of the array.

The problem above does test understanding of the syntax and semantics of arrays, but does not test the kind of algorith­mic problem-solving skills we want our students to develop. Problem-solving is related to debugging and error interpreta­tion. As an example, consider how the error messages differ in a CodeLab environment and our testing environment. An er­ror message in the Code Lab system is shown in Figure I for an

incorrect submission to the problem statement "Assume that an integer array named a has been declared and initialized Write a statementlhal adds 1010 theftrSI element of the array.". The error is produced in response to the submission of this code frag­ment : a [OJ += 5; which adds the wrong value to the first element of the array.

Logic or Runtiml! Errors

Our MaIyIll Of Your Cod.: 1. You t .... "�d!tIt rii}ht!!!!I ,lem.nt. bill usig""d � tho

..... ng .-aIuo. 2. M ygUidiling the p10PIll �Iu,?

Your code:

a(01 _ 5:

FIGURE 1 ERROR MESSAGE FROM CODELAB

Session T3H

The problem below shows how constrained some of the prob­lem statements are because of the nature of the online testing mechanism.

Assume that an array of ints named a that contains exactly five elements has been declared and initialized In addition, an in' variable j has also been declared and initialized to a value somewhere between 0 and 3. Write a single statement that assigns a new value to element of the array indexed by j. This new value should be equal to twice the value stored in the next element of the array (i.e. the element after the element indexed by j).

A more complex example might require students to write a loop to sum the elements of an array. However, since students write code fragments rather than complete methods, the prob­lem will be phrased so that students must use k as a loop-control variable, a as the name of the array, and total as the name of the variable holding the final value. A submitted solution is ei­ther correct or wrong, with no edge-cases or other test scenarios as part of the problem. Although we see the value in this kind of practice, we are interested in problems that are more algorithmic and rather less focused on simple language constructs.

te"t1nq • • .

1 t .. 11 e�pec\;ed. ["13", I gO\; ["eO, I : ["J", :a t .. 11 expecl;;ed. ["U", I gOI;; ["6", I : t"l", 3 fail 1I!1CJ;:III!ctll!d ["1N'., ] qat ["'.2:"', ) : 1"'-3",

FIGURE 2 ERROR MESSAGE FROM OUR SYSTEM

Problems and Test Data

At the same time that we were becoming increasingly dissat­isfied with our student's skills in simple, functional problem­solving and decomposition, our best students were doing very well in an online programming contest called Topcoder [12] . The Topcoder contest is an individual contest, but similar in spirit to the team-oriented, international ACM programming contest. In addition to being an individual contest, all problems from recent Topcoder contests are online and testable - that is one can submit a proposed solution and have it tested against the data used during the online contests.

We saw the value in the Topcoder problems not from a com­petitive viewpoint, but from a pedagogical viewpoint: algorith­mic problems with complete test data are provided on a roughly weekly basis with additional access to more than 100 past con-

This is most likely a helpful error for a student first learning tests. Because these problems are posed as part of a compe­about arrays. The error message produced for a similar ptob- titian, and because of our experience with programming con­lem in our system is shown in 2 (we programmed our system tests [13], we are confident that the test data provided is exhaus­to essentially process the same problem, we most likely would tive. HoweVer, rather than using Topcoder's practice facilities, not use this problem in our system). Here the student is left to we used their intellectual property (problems and test data) in determine the possible source of the error given the input, the our own online, web-based system. Our prototype was devel­expected output, and the output produced by the student's code. oped for face-to-face teaching purposes, but we are patterning The more prescriptive error produced in the CodeLab environ- with Topcoder to make materials available to a wider audience. ment is directive, and likely helpful to a student who is having Topcoder has given us permission to make these problems avail­trouble with arrays. In our system the student must determine able for educational purposes and is working with a team of ed-the source of the error given the intended output. ucators to provide access via the Topcoder website to a database

0-7803-7691-6/03/$17.00 (92004 IEEE October 20 - 23, 2004 Savannah, GA 34th ASEEIIEEE Frontiers in Education Conference

T3H-21

of problems which they could administer with faculty guidance and support.

Testing and Feedback

The Topcoder practice rooms allow access to previous con· tests, but not on a per-problem basis. Instead, a registered par­ticipant can choose to see and solve all the problems from one contest at a time. After submitting code for a problem, a test run indicates success or failure, but test runs stop after one fail­ure. We wanted a different model for our algorithmic problem­solving problems, one that combines testing in a competitive environment with the unit-testing ideas from JUnit. The JUnit test philosophy is to run all tests and report the success or failure of every test. ,We incorporated this idea into our testing frame­work showing all test-runs and data from those runs that fail. We elaborate on this testing mechanism below, the results students see for an incorrect submission are shown in Figure 2.

EXAMPLES OF ALGORITHMIC PROBLEM

SOLVING

In this section we provide complete elCamples of two prob­lems and how we have used them in our courses. We also pro­vide summaries of some of the other examples and how they have increased the range of problems our students solve during the course of a semester. In discussing the examples, we show how they fit into our just-in-time teaching (JiTI) process mod­elled after [7] which describes the practice as follows. "The essence of JiTI is feedback: students are asked to work in preparation for each class, and faculty use this work to improve the class itself. In tum, faculty consider the successes and fail· ures of each class, and use this information to adjust subsequent assignments."

From Sorting to Maps

The problem statement below comes from our Spring 2004 courses, has also been used in high school workshops, and is licensed under a Creative Commons [14] license for further dis­semination. The frequency with which dala occurs is sometimes an impor­tant statistic. In this problem you'll determine how frequently strings occur and return an array representing the frequencies of each different/unique string. The array returned contains as

many frequencies as there are unique strings. The returnedfre­quencies represent an alphabetic/lexicographic ordering of the unique words, so the first frequency is how many times the al­phabetically first word occurs and the last frequency is the num­ber of times the alphabetically last word occurs. Consider these strings (quotes for clarity. they're not part of the strings). ["apple", "pear", "cherry", "apple",

"cherry", "pear", "apple", "banana")

The array returned is [3, 1, 2, 2 J since the alphabetically first word is "apple" which occurs 3 times; the second word alpha­betically is "banana" which occurs once, and the other words each occur twice.

The class and method are specified as follows.

Session T3H

clasD SortedFreqa

{ public intI! freqs[String(! data)

{ II fill in code here

We pose this problem as our students are writing classes and methods to find all anagrams in a list of words. Although they do not see the similarities at first, students eventually see the connection between finding words with similar canonical forms, (e.g., ''tar'', "art" and "rat" all share a sorted-canonical form of "art·) and counting occurrences of words. After students have submitted solutions to the SortedFreqs problem, we use the student-generated solutions as the basis for an inclass discussion of big-Dh complexity and the tradeoffs in approaches. When students are in control of the material used in class they are more attentive and often do better at analyzing the solutions than if instructors develop alternatives [IS]. This is also an essential part of the JilT process we are trying t o use.

This problem is the basis for a discussion about solutions that are simple to write: in this case we sort and look for runs of equal words. We compare this solution to one that is slightly more code, but that in some situations is faster: form a set of the unique words and count the occurrences of each set-element in the Original array. The first solution is O(n log n) and the latter is O(n log k + kn) where k is the number of unique strings. When we discuss maps, we return to this problem and show how to solve it in O(n) time using hashing.

This problem shows how a simple algorithmic problem can form the basis fot an interactive, active class discussion of al­gorithmic and design tradeoffs while leading students to a better understanding of a more in-depth programming assignment. We use the next problem in similar ways.

From Loups to Sets

This example is copyright Topcoder [121. We have used this to introduce Java to experienced students and high school teach­ers as well as in our courses (in which we use both c++ and Java versions of the problem). A health club chain allows its members to visit any of its many health club locations an unlimited number of times per day. The only constraining rule is, a customer can only visit one health club location per day, eVen though he or she may relurn to that location an unlimited number of times for the rest of that day, Although the honor system has always worked quite well. the club wants to run some tests to see how many people really fol­low the rules. You are to write a program that takes the entrance logfiles from three different clubs (all logging the same day) and return a sorted list of the people who are not honest and went to more than one health club location in the same day. The log files are represented as String arrays: St ring [J ,

where each element is the member name of a customer who en­tered that day. For example, if a customer showed up three times to one of the club locations that day, the member's name would appear three times in the corresponding String [].

0-7803.7691-6/03/$17.00@2004 IEEE October 20 - 23,2004 Savannah, GA 341h ASEE/IEEE Frontiers in Educiltion Conference

T3H-22

public c1688 MemberCheck { public Stringl)

whoaDlahone8t(Stringl) clubl. String II club2. string I) clubl)

II fill in code here

Student solutions to this problem typically duplicate code in looping over one array to see if its elements are found in the other arrays. We use these solutions to discuss functional de­composition when there is duplicate code, but we do not grade the solutions based on the quality of the code, only on their correctness. We emphasize an AgilefXP software process [16] where simple solutions with duplicate code are acceptable pro­vided the students are aware of the choice when they choose to duplicate code. We then use this problem as a springboard to discussing set intersection and sorting which students often use in their solutions.

We have also used this problem when introducing experi­enced students and high school teachers to Java. After they have submitted their solutions, we discuss the straight-line solution below which leverages the Java Collections API. No student has ever submitted this solution, but all are appreciative of it after at­tempting on their own to duplicate the functionality of the meth­ods shown.

Set cheaters . new freeSet(): Set 81 - new freeSet(Arrays.asLiat(cl»;

Set ala . new TreeSet(Arrays.a8List(cl»; Set &2 • new TreeSetIArrays.asList(c2»; Set 03 - new TreeSetIArrayo.asList(C3»;

a1.retainAII(s2); /1 inte�aection of 0:1 and c2 sla.retsinAll(s31; II intersection of c1 and cJ s2.retainA11(aJI; /1 intersection of c2 and c3

cheaters.addAll(al) ; cheaters.addAlllala); cheaters.addAll(s2J;

return (Stringl) cheaters.toArray(new String [0) I:

Showing this solution without the benefit of having students tried to develop a solution on their own is not as successful in generating discussion and in helping students to understand how to use the standard Java API in a problem-solving context.

Flood Fill and Recursion

In our first course for majors we usually cover recursive flood­fill algorithms in some detail. In the past, we have asked students to solve a problem using flood-fill techniques, but we have not been successful in getting all students to master the technique. This year we used three of the weekly problems described in this paper to test understanding of the technique. The prob­lems were non-trivial, requiring recursive flood-fill code to solve backtracking problems that were related, but far from similar to problems discussed in class. All students solved one of the prob­lems, and many students solve more than one (only one was re­quired). The instant-feedback of the online testing was cited by students as one reason for the success.

Session T3H

How TO SOLVE IT BY PROGRAMMING

We show students how to solve these problems as part of our process-oriented classroom presentations. For example, con­sider this simple problem statement that we use in our intro­ductory course. The square length of a string is the square of the number of characters in the string. The square length of "hello" is 25, the square length of "a" is I, and the square length of "apple

sauce" is 121. W rite the method length that returns a string's square length.

public class SquareLengtb

( public int length{Strin9 str)

( 1/ fill in code here

We first show students how to write a one-line method body and how to test this solution via our web-based submission mechanism. Here is the code we write in class.

public int length{String str)

{ return Oi

} This emphasizes a stub-based functional decomposition and leads to the webpage report shown below which is extracted from the webpage returned by the submission system (the strings are not shown in their entirety because of space­constraints, but are shown completely in a scrollable webpage).

testing 1 fail expected 25 got 0 : hello 2 pass 3 fail expected 1 got 0 : a

4 fail expected 4 got 0 : at

5 fail expected 64 got 0 : elephant 6 fail expected 1225 got 0 : now is th ...

7 fail expected 1 got 0 : I 8 fail expected 8100 got 0 : 12345 ... 9 fail expected 32400 got 0 : 12345 ... 10 fail expected 25 got 0 : apple

Students immediately see how many test cases there are and quickly (for this problem) develop a solution that leads to the following output.

testing ... 1 pass 2 pass 3 pass

4 pass 5 pass 6 pass

7 pass 8 pass 9 pass 10 pass

When students get an "all-pass" there is typically a whooplhurrahlshout that results. This simple form of success

0-7803-7691-6/03/517.00 @2004 IEEE October 20 - 23, 2004 Savannah, GA 34th ASEEfIEEE Frontiers in Education Conference

T3H-23

is achievable by nearly all our students. Since we do not grade based on style, students know immediately what their grade is. Since we can use their solutions as the basis for inclass dis­cussions about style we are able discuss style in the context of correct programs using student-generated solutions. This is the essence of just-in-time teaching and has been succcssfiJI from our standpoint in creating better problem-solvers and program­mers.

Time on Task

Submitted programs in our courses must be accompanied by an estimate of how much time students have spent on the as­signment. Across all weeklies the average time per weekly is re­ported as less than two hours. Greater than ninety-five percent of all submissions ate estimated to require fewer than three hours to code. This shows that incorporating weeklies into an already crowded course should not require an inordinate amount of new work from students. Since the programs provide the basis for some inc lass discussion, they can decrease instructor-time spent on developing teaching materials.

We have no self-reported cases of cheating with respect to these weekly programs and no suspicious similarities as re­

ported by Moss [17]. However, we emphasize collaboration on our larger programs and attempt to inculcate a sense of respon­sibility and honor with respect to individual assignments.

CONCLUSION

We report on a web-based submission system and suite of short problems that are part of a just-in-time teaching process in our first year of programming courses. These problems em· phasize algorithmic problem solving although our courses con· centrate on object-oriented programming and design. Although our model for these problems is an online programming contest, we do not rely on competition to foster interest [18]. We em­phasize the process of problem-solving as its own reward [19] and encourage all students to succeed by trying as many times as necessary to solve each problem we pose.

Although we have not measured success or interest by gender, the philosophy and approach we use are similar to techniques espoused in [20] as conducive to fostering interest by women in the programming process.

ACKNOWLEDGMENTS

This work was supported by NSF CAREER 9702550 and an

IBM Eclipse Grant. We gratefully acknowledge the support of Topcoder, the Duke Computer Science system administrators, and the education group in Computer Science at Duke who have embraced this approach.

REFERENCES

[I) M.A. Perez-Quiilones, Steven Edwards, Claude Anderson, Doug Baldwin, James Caristi, and Paul J. Wagner, "Transitioning 10 an objects-early three· course introductory sequence: Issues and expectations," in The Pupers of Ihe Thirty.Fifth SlGCSE Technical Symposium on Computer Science Education. 2004, pp. 499-500, ACM Press.

(2) Stuart Reges, "Resolved: Objects have railed us," ht.tp, //listserv.acm.org/archives/wa.cgi?A2_ind0302C&Lc sigcse-members&P.'R1697&m�3844.

Session T3H

(3) Kim Bruce, Andrea Danyluk, IIId Tom Murtagh, "I!vent-4iriven progmm­ming ean be limple el\ollgh for CI I," in Procudillgs O/IM Sulh COli'

forence 011 11IIIOWJUI1n and Technology ill CompulV Science Educalion (mCSE).lool, pp. 1-4, ACM pras,

(4) Owen Allnl:ban, ''00 overkill: When limple il better than not," in TIle I'IIpen o/Ihe Thlrty-&conti SlGCSE T�lrnical $Jimpo1/um Oil Compuler Science EduCdllolI. 2001, pp. 302-306, ACM !'retl.

[S] Erich G&rIlJm, Ri,bard Hebn, JWpb Jolm$on, and Jobo V1i .. idu, De­sign Patle,.",: ElemenlJ of Rewab/e Objec/·OrlePlled So.fir.'tut. Adciison­Wesley, 1995.

(6) C. A1p� P. V�tura, S. WOllll, IIIId D.X. NIIIY�. "Teach;"B design pattenu ;" Cl1/esl witb killer examples," io � f'opvI of lhe Thlrty­Fourth SIGCSE TwudCDI Symposium on Compuru Science Education, 2003.

(7) Gregor Novak, Andrew Garnn, Wolf88nB Chrl.tian, and Evelyn Palterlon, ./wI-In-11m' �/JIg.' Blending Aclm lAomlng wltll We/l Teclrnology, Prentice Hall, ) 999.

(8) Slephen H. BdwatdI, "UlinS software talins to move ltudenb From trial-and-=or 10 reflection.in-aaion. � in 11te Ibpe,.. o/Ihe Thlrly-Fl./Ih SlGCSE 1«hn1C<11 SympollunJ Oil CO"'l'IIt'r Sclm" EtJt,c"t/on. 2004, pp. 26-30, ACM Prell.

(9) "Juni\: Unit teatins," http://www . junit .org. [10] O. Aalnl:han and D. Rced, "AAA IJId C5-1: The applied apprenticnhip

approach to CS I," in Th. Papen o/Ih. 'l'Wellty-Sulh SIGCSE Technical Synrpwlum all ComJ1U"r Sclm" Educat/oll. MlJeh 1995, pp. I-S, ACM J'reg, SIGCSE Bulletin V. 27 N 1.

[11) David Arnow, "Turing's c:raft," http://www.turing8craft.com. [12) Topooder Contests, ''Topcoder,'' http://www.topcoder.com. [13) Owen Aalnl:ban, Vivek Khera, and David Katz, ''The duke internet pro­

gramming contest: A rqlOrt and philDSOPhy," in TIre Pupers oflh. 'l'Wen/y­Fourth SIGCSE Teclrnlcal SymposlunJ on ComplllU Sciellc. Educalioll. March 1993, pp. 48-'2, ACM J'reg, SIGCSE BuUetio V. 2S N I.

[14) Oweo Astrachan. "Creatiye commons license," http: / /creati vecommonB. org, attribution,share.aJike.

[IS) James B. Zull, TIre Art o/Changing the Brain, Srylus PublishinB, 2002. [16) "Manifesto for agile software dcvelopmen�" http,//

agilealliance . org/. [17) Alex Aiken, "A ,ystem for detecting software plagiarism," http'll

www.c8.berkeley.edu/-aiken/moS8.html. (18) Eric Roberu, "StrBlcgies for encounging individual achievement in in­

trodUC:lory compultf sdence courses:' in The Papers I)/Ihe Tlrirty.First SIGCSE Techniccz/ Symposium 011 Computer Science Education. 2000, pp. 295-299, ACM Press.

[191 Alfie Kohn, Punished by Rcwanis: The trouble with Gold Stars. Incenlive Plans. A', Praise. dIId OIher Briber. Houghton Mifflin. 1993.

[20] Jane MBlgolis and Allan Fisher, UII/octillg Ihe C/u/>hoUJe, MIT Press, 2001.

0-7803-7691-6103/$17.00 @2004 IEEE October 20 - 23, 2004 Savannah, GA 34th ASEEIIEEE Frontiers in Education Conference

T3H-24