Having added some CodeaUnit features to our own Tests framework, I’m deciding whether to port it to SLua, or whether to improve our existing Tests along similar lines.
Let’s look at some of the comparisons between our Tests framework and CodeaUnit:
Tests treats each test as a method in the Tests class itself. We might be able to pollute the Tests class less with CodeaUnit. That said, we might be able to reduce the pollution within Tests without going to CodeaUnit.
CodeaUnit provides the notion of a “suite” of tests. This includes the ability to easily specify setup and tear-down for a group of tests. Adding that capability to Tests would change things a lot but would probably not be difficult.
CodeaUnit is rather clean code. We could clean up ours to be as good, and that might be worth doing for the exercise.
CodeaUnit relies on two features of Codea Lua that we will not have available in SL, namely the ability to read a project’s source code, and to load and execute code. We’ll have to use a more explicit way of defining tests than CodeaUnit now has, similar to what we do now.
Ha. I started out, 15 minutes ago, thinking that we’d port CodeaUnit. And now I realize that, no, my thing is not to rewrite and replace wholesale, it is to improve my code incrementally, shaping it to be more and more what it needs to be.
We will improve Tests to be more like CodeaUnit. (And we will feel free to borrow ideas from CodeaUnit, as we have done already.)
Provide a test suite capability, including a suite description, set up, and tear down.
Pollute the Tests namespace less, ideally not at all.
It is not an objective to make existing tests run as they do: requiring conversion of all existing tests will be allowed. Extra credit will be granted, however, if conversion is not required or, failing that, quite easy.
We will follow the nested function design of CodeaUnit, described below.
That is all for now but new objectives will come to mind and be added below as they come up.
Here is the CodeaUnit example test file:
function testCodeaUnitFunctionality()
CodeaUnit.detailed = true
_:describe("CodeaUnit Test Suite", function()
_:before(function()
-- Some setup
end)
_:after(function()
-- Some teardown
end)
_:test("Equality test", function()
_:expect("Foo").is("Foo")
end)
_:test("Negation test", function()
_:expect("Bar").isnt("Foo")
end)
_:test("Containment test", function()
_:expect({"Foo", "Bar", "Baz"}).has("Foo")
end)
_:test("Thrown test", function()
_:expect(function()
error("Foo error")
end).throws("Foo error")
end)
_:ignore("Ignored test", function()
_:expect("Foo").is("Foo")
end)
_:test("Failing test", function()
_:expect("Foo").is("Bar")
end)
end)
end
You will notice that there is just one top-level function,, testCodeaUnitFunctionality, and that it really only calls one method, _:describe, passing a string, and a function that includes all the rest of the test.
_The variable _ is set to be the CodeaUnit instance that is running the tests. I propose to borrow that notion. The effect is that test, for example, is a method of the unit testing framework. And, if we look at an example of test in use:
_:test("Equality test", function()
_:expect("Foo").is("Foo")
end)
We see that each check in a suite of tests always includes a descriptive string, such as “Equality test”, and a function which will be called by the framework to run that test.
In Tests as presently implemented, we define a function, perhaps named test_equality, and Tests inspects its own namespace to find all the methods named with ‘test_’ and call them. In the new scheme, we call test and the framework executes the function. Similar but different.
It is possible that we can keep both schemes working, allowing for a smooth transition from one scheme to the other.
Because we can’t read and search our own source file, I think we’ll start by defining our suites as methods in Tests. Rather than call them test-something, let’s call them suite-something to keep them separate. And we’ll set up the underbar right away.
Let’s get started, finding out way forward.
I think I’ll work with a fresh test file, not the existing Bezier one. Here’s my first version:
local Tests = require('./tests')
local _ = Tests
function _:suiteTestSuite()
_:describe("Initial Test Suite", function()
_:test("Equality Test", function()
_:expect("Foo").is("Foo")
end)
end)
end
Tests:execute()
The existing run call is run_tests, so I’ll use execute here, to keep things separate for now.
I think that execute needs to look for suite methods and do whatever needs doing. Run them, basically.
I start with something a bit less difficult:
function Tests:execute()
for name, func in self do
if name:sub(1, 5) == 'suite' and type(func) == 'function' then
print(name)
end
end
end
When I run the file I expect it to print my suite name and nothing else.
That actually happens, once I start editing the copy of tests.lua from the repo, not the one in the downloads file on the github site. Took me a while to realize that.
CiodeaUnit just calls the suite function when it finds it, so I’ll do the same, expecting a failure on describe:
function Tests:execute()
for name, func in self do
if name:sub(1, 5) == 'suite' and type(func) == 'function' then
print(name)
func()
end
end
end
Sure enough that’s what happens. Now we need to do a describe method. Its real job includes a lot of record keeping and such but mainly it wants to run the function provided:
function Tests:describe(description, all_tests)
all_tests()
end
That, of course, errors looking for the test method. We provide:
function Tests:describe(description, all_tests)
tests_passing = 0
tests_failing = 0
tests_ignored = 0
all_tests()
end
function Tests:test(description, check)
local ok, message = pcall(check)
if not ok then
self:tally(ok)
self:report(string.format('Error: %s.', message))
end
end
I changed my test to expect “Bar” instead of “Foo”, to generate a failure, and I get this:
suiteTestSuite
Actual: Foo, Expected: Bar. Tests:
[Finished in 32ms]
So that’s actually working. We’re not getting the summary output yet, and we have duplication in the setup.
Let me duplicate some more, putting the summary info into describe.
With a bit of fiddling, and an added test, I get this output:
Suite: Initial Test Suite
Actual: Bar, Expected: Foo. Tests:
All Tests: Pass 1 Fail 1 Ignored 0. Tests:Initial Test Suite
Here’s the suite function:
function _:suiteTestSuite()
_:describe("Initial Test Suite", function()
_:test("Equality Test", function()
_:expect("Foo").is("Foo")
end)
_:test("Expected to Fail", function()
_:expect("Bar").is("Foo")
end)
end)
end
And here’s all the added code:
function Tests:execute()
for name, func in self do
if name:sub(1, 5) == 'suite' and type(func) == 'function' then
func()
end
end
end
function Tests:describe(description, all_tests)
print(`Suite: {description}`)
tests_passing = 0
tests_failing = 0
tests_ignored = 0
all_tests()
self.current_name = description
local m = string.format('All Tests: Pass %s Fail %s Ignored %s.',
tests_passing, tests_failing, tests_ignored)
self:report(m)
end
function Tests:test(description, check)
local ok, message = pcall(check)
if not ok then
self:tally(ok)
self:report(string.format('Error: %s.', message))
end
end
We have plenty to do … but we have plenty done, and it has been easy so far. I’m not sorry that I decided to evolve the existing Tests forward rather than try to port CodeaUnit.
Now time for a break. Today is Saturday, after all.
Safe paths!