Today I propose to improve our
Testsframework a bit.
As all here surely recall, the current Tests framework is something that I just ginned up quickly to provide a consistent way to write tests. I find that when I have decent tools, I tend to use them to my benefit, and without tools, I am more included to just hack around. With testing, that usually comes down to programming with little or no organized testing.
Today I want to adopt an idea from CodeaUnit, a very nice little testing framework, written by Jake Sankey, for Codea Lua, an iPad product. He has a very nice assertion format. Let me write a test using the new feature that I propose to implement. I’ll put the new tests into the tests-samples file that contains the existing ones.
function Tests:test_expect_is()
result = 54
self:expect(result).is(54)
end
If this goes well, there will be other verbs in addition to is, such as isnt, has, hasnt, and throws. If it doesn’t go well, you will not be reading this.
What could expect return that can respond to tings like is? the short answer is that it will return a table of functions, with names like is.
I only have to run the tests in my samples file to realize why I don’t want to work in it: it has a handful of tests that intentionally fail, That makes it dump a lot of info that I don’t want to deal with.
I’ll create a new file. And I get the message I expect:
Error: /Users/jr/Desktop/repo/test-expect.lua:24:
attempt to call missing method 'expect' of table. Tests:test_expect_is
Now we can get to work. In the end, if all goes well, we will put our new code into the file tests.lua but since we have the class available to us in test-expect, we can make tentative changes here without needing to version and revert the main tests.lua.
I add a second test, so as to check the fail side:
function Tests:test_expect_is_fail()
result = 54
self:expect(result).is(53)
end
And implement:
function Tests:expect(actual)
local function is(expected, epsilon)
self:assert_equals(actual, expected)
end
return {
is = is,
}
end
The expect method defines a local function that uses the existing assert_equal method for now. We’ll see about improving the whole thing after we have a lot more of this working. Will we replace the existing way of checking with this new one? We just might. I’m far from sure about that.
The tests operate as intended:
Actual: 54, Expected: 53. Tests:test_expect_is_fail
All Tests: Pass 2 Fail 1 Ignored 0. Tests:run_tests
CodeaUnit has you put your explanatory message, if you want one, in the expect, like this:
function Tests:test_expect_is_fail()
result = 54
self:expect(result, "expected to fail").is(53)
end
We try this:
function Tests:expect(actual, message)
local function is(expected, epsilon)
self:assert_equals(actual, expected, message)
end
return {
is = is,
}
end
And we get this:
Actual: 54, Expected: 53. [expected to fail]
Tests:test_expect_is_fail
Perfect. Well, as intended, anyway.
We have an issue with the epsilon, and with tables. CodeaUnit, if I recall, does not check table contents but we do. So we’ll need to deal with those cases. First epsilon:
function Tests:test_close_enough()
result = 55
self:expect(result).is(54,1)
end
We expect this to pass because 54 is within 1 of the desired answer. It will not pass:
Actual: 55, Expected: 54.
Tests:test_close_enough
We fix it:
function Tests:expect(actual, message)
local function is(expected, epsilon)
if epsilon then
self:assert_nearly_equal(actual, expected, epsilon, message)
else
self:assert_equals(actual, expected, message)
end
end
return {
is = is,
}
end
This is going nicely.
I am beginning to wish for a way to assert that a test should fail. I make a note of the idea. It would be useful for me, perhaps not for anyone else.
Let’s do tables while we are at it:
function Tests:test_table_is()
result = {1, 2, 3}
expected = {1, 2, 3}
self:expect(result).is(expected)
end
This might actually work, since we use assert_equals, which handles tables. Let’s see. Yes, it passes. Change the expected table to see it fail.
Actual: table[4] is missing,
Expected: [4]=4.
Tests:test_table_is
Perfect!
Let’s do isnt.
function Tests:test_isnt()
unexpected = 100
result = 100
self:expect(result, "better not be 100").isnt(unexpected)
end
This might be a bit more tricky, because we do not have a not_equal assertion in Tests, so we’re on our own to build it.
Turns out it’s not too bad, so far:
local function isnt(expected, epsilon)
if epsilon then
else
if not self:tally(actual ~= expected) then
local m = string.format('Actual: %s Unexpected.', tostring(actual))
self:report(m, message)
end
end
end
My test fails properly:
Actual: 100 Unexpected. [better not be 100]
Tests:test_isnt
So far so good! Do epsilon now:
function Tests:test_isnt_epsilon()
unexpected = 100
result = 99
self:expect(result, "stay away from 100").isnt(unexpected, 2)
end
This should fail because 99 is within 2 of 100. Implement:
local function isnt(expected, epsilon)
if epsilon then
ok = not (expected - epsilon <= actual and actual <= expected + epsilon)
if not self:tally(ok) then
local m = string.format('Actual: %s within %s of %s',
tostring(actual), tostring(epsilon), tostring(expected))
self:report(m, message)
end
else
if not self:tally(actual ~= expected) then
local m = string.format('Actual: %s Unexpected.', tostring(actual))
self:report(m, message)
end
end
end
Some duplication there that we might want to deal with. But the thing works.
I don’t think isnt has much value, so we’ll not give it a special feature for table inequality.
I do think that has might be useful, and perhaps not hasnt. has checks to see if a given value is in a table result. It doesn’t handle keys, just values.
function Tests:test_has()
result = {1, 2, 3, 4}
self:expect(result, "checking table").has(5)
end
Implement:
local function has(expected)
local found = false
for _, v in actual do
if v == expected then
found = true
break
end
end
if not self:tally(found) then
local m = string.format('Table: %s did not contain %s',
tostring(actual), tostring(expected))
self:report(m, message)
end
end
Works:
Table: table: 0x0000000140017b50 did not contain 5 [checking table] Tests:test_has
Our tests are all running. We have implemented these new forms:
self:expect(result).is(54)
self:expect(result).is(54)
self:expect(result).is(54,1)
result = {1, 2, 3}
expected = {1, 2, 3}
self:expect(result).is(expected)
self:expect(result, "better not be 100").isnt(unexpected)
self:expect(result, "stay away from 100").isnt(unexpected, 2)
self:expect(result, "checking table").has(3)
I’m not sure that I like including the message as an optional parameter. It might be better to say something like:
self:expect(result).message("checking table").has(3)
But fact is, we only slip the message in rarely, so maybe its OK as is. And you can always give the test itself a more meaningful name as well, since that’s printed as part of the result.
I think it comes down to whether we want to say
self:assert_equals(actual, expected)
Or
self:expect(actual).is(expected)
And of course, when we need has, the new form is available and there is no useful way to deal with it otherwise.
I think that if we want this “new and improved” syntax, we should cut over to it and use it exclusively. So far, I think the only person much affected by that change would be me, though Dizzi might have a few tests written.
We’ll let the idea perk, but I will put the features into the tests file that I use, and push it to the web site. It’s harmless, as nothing in the original was changed today.
This was fun and perhaps useful. Safe paths!