I’m currently taking the Coursera Programming Languages course which starts with ML. Or more specifically SML/NJ.
Why I was looking at unit testing in SML
The instructor gave a test file with statements like:
val foo = function_to_write(1) = 2;
This is good in that it clearly shows the API and some of the expectations. It’s not so good because you have to go thru the output to look whether all the values are set to true. We were provided with 11 tests and told to write more. I had 57 by the time I was done. And I haven’t looked at the two challenge problems yet. The instructor was very direct in saying we shouldn’t compare SML to languages we know because it impedes learning. Well, I didn’t. I compared the infrastructure. And I learned more SML learning how this works than covered in week 1 of the class so I think I honored the spirit of this.
Choosing a SMLUnit implementation
I found two options in search
- SML Sharp‘s SML Unit – I found more references to this one on the internet. However it requires steps to install and looks like it takes longer to get started with.
- SML Unit – This is the #1 hit on google for SML Unit and appears to be something written by a university student. It’s very good though and I recommend it.
SMLUnit comes with good documentation and examples. It also comes with a runner that uses Python to kick off the tests and report on the status. This can’t be done purely in SML because reporting on the results requires state.
I’m sure it is a good runner, but I didn’t notice it existed. (I started on the code page with the assertions not the main page that explains all this.) Since I went to the (minor) trouble to create a runner in UNIX, I’m posting it in case anyone wants to test SML on a machine that doesn’t have Python. Not likely I know – most machines have Python.
The runner feeds your unit tests to SML and then reports on whether there were any failures.
# Since sml is a functional language, I couldn't figure out a way to output a summary # at the end as to whether any tests failed. Wrapping in a shell script to avoid # this problem. [realized afterwards it comes with a Python runner] # # Usage: # smlunit_runner.sh name_of_tests.sml # # Assumes the sml test suite has "use" statements for all dependencies including smlunit if [ $# -eq 0 ] then echo "SML test file is mandatory" exit fi OUTPUT=`cat $1 | /usr/local/smlnj/bin/sml` echo "$OUTPUT" NUM_FAIL=`echo "$OUTPUT" | grep "^FAIL$" | wc -l` NUM_PASS=`echo "$OUTPUT" | grep "^OK$" | wc -l` echo "" echo "" echo "Test summary: (also look at last output to make sure not a stack trace)" echo "$NUM_FAIL tests failed" echo "$NUM_PASS tests passed"
What do the tests look like?
Code – helloWorld.sml
fun hello_world() = "hello" fun add(a:int, b:int) = a + b
Tests – helloWorldTest.sml
use "asserts.sml"; <em id="__mceDel">use "helloWorld.sml";</em> assertTrue true "assert true"; assertEqual "hello" (hello_world()) "compare two values"; assertEqual 3 (add(1,2)) "compare two values again";
And what about the asserts?
asserts.sml is a direct copy of smlunit.sml
(* from https://github.com/dellsystem/smlunit/blob/master/lib/smlunit.sml *) fun roundish (x:real) = Real.realRound(x * 100000000000.0) / 100000000000.0; fun assert expr1 expr2 (desc:string) function = (print ("*********** " ^ desc ^ " ***********\n" ); (if function (expr1, expr2) then print "OK\n" else print "FAIL\n" ); [expr1, expr2]); fun assertEqual expr1 expr2 (desc:string) = assert expr1 expr2 desc (fn (x, y) => x = y); fun assertRealEqual (expr1:real) (expr2:real) (desc:string) = assert expr1 expr2 desc (fn (x, y) => Real.== (roundish(x), roundish(y))); fun assertTrue expr desc = (assertEqual expr true desc); fun assertFalse expr desc = (assertEqual expr false desc); (* Functions for comparing a lot of things in one list *) fun assertReals f  (desc:string) =  | assertReals f ((input, output, expl)::t) (desc:string) = (assertRealEqual (f input) output (desc ^ ": " ^ expl)) @ (assertReals f t (desc)); fun assertRegulars f  (desc:string) =  | assertRegulars f ((input, output, expl)::t) (desc:string) = (assertEqual (f input) output (desc ^ ": " ^ expl)) @ (assertRegulars f t (desc));
I have less than 5 hours experience with SML (I did know LISP in college and recursion certainly isn’t new.) So I’m sure there are naming conventions and the like that I’m not following. I look forward to hearing comments on what I did.
Today I discovered the idiom for testing an exception is thrown: (functionToTest(); print(“\nFAIL\n”)) handle IllegalMove => print(“\nOK\n”)
A better way suggested in the class as a way to test it:
val actual = (functionToTest());
handle IllegalMove => true)
assertTrue actual “message”
I guess I’m a bit too late for this year’s course, but I have looked into SML testing frameworks and also written my own: https://github.com/emilwall/DescribeSML
I’ve listed a few in the appendix of my master thesis as well: https://github.com/emilwall/exjobb/blob/master/exjobb_thesis_emilwall_appendix.pdf?raw=true
Thanks for the blog entry. It is still useful, years later! And yes, also for the great PL/Grossman class.
BTW is it me or does smlunit not handle functions that take options as arguments? I did never get that working.