ruby-core

Dean Wampler <deanwampler gmail.com> writes:
> The redundancy part (or DRY in Ruby parlance) I
> was referring to is the goal of only writing the requirements once and
> it's better to write them in an executable format so they have
> enforcement power.

But to go back to the problems I was pointing out:

a) In the real world, the requirements do not constitute the entirety of 
the supported functionality.

b) In the real world, actually unit testing the complete scope of the 
requirements requires unreasonable amounts of time and effort.

Consider '+'. Requirements are that it add any two things which 
constitute numbers. Trouble is, you can't even test every possible pair 
of floating point numbers, let alone the integers and rationals. Any 
test of N pairs that you do will be statistically insignificant.

> So, the acceptance tests used by customers to confirm that
> requirements are met may be written in a documentation tool from which
> test code is generated.

Right. Unit tests are not the requirements; they are acceptance tests 
used to confirm that the requirements have been met. There's a subtle 
but very important difference.

Similarly, requirements are not specifications. The specifications are 
(hopefully) derived from the requirements, but they typically describe a 
superset of the functionality strictly required, and almost always 
describe a superset of the functionality tested.

For example:

Requirements: method to add two vectors of Floats.

Specification (as implemented): method adds two vectors of objects which 
support '+'.

Unit tests: take a sample file of vectors of floats, add them in pairs, 
compare against the file of correct output.

All perfectly reasonable. However, the unit tests do not tell you 
whether anything other than floats will work with the method. That's 
perfectly reasonable, because working with something other than floats 
wasn't a requirement. However, it's also perfectly reasonable that the 
first implementation happens to work with integers, rationals, and so 
on--because the power of Ruby is that generic programming doesn't take 
extra effort. In fact, it would take extra effort to prevent the code 
from working with other types.

So, if you only provide unit tests and no documentation, users have to 
guess. They might look at the unit tests, and guess that the method only 
works with floats. End result: code bloat because they go and 
unnecessarily find or implement an integer vector add.

Or, they might look at the code, and see that it works with Complex. 
Then they go and write a ton of scientific visualization software that 
relies on the fact. Then you want to refactor the method to use CBLAS 
for speed, and it breaks NASA, and there are howls of anguish.

Hopefully I'm getting the problem across better this time around. I am 
*not* discussing unit tests vs documentation, or unit tests vs 
contracts, or unit tests vs anything else, or documentation vs anything 
else. At no point have I said that unit tests are anything other than 
essential. My point is simply that a documented API is *also* essential, 
and at a minimum it should describe the functionality that is 
*supported*, as opposed to that which is tested or that which happens to 
work. Leaving people to guess the supported API from the unit tests (or 
worse, the code) is a flawed approach which has led to problems refactoring.

In a nutshell:

documentation = what is *supported* or *supposed to work*
unit tests = what is *actually tested*
code = what *happens to work*

What is *tested* != what is *supported*, because you can't test 
everything. I can't feasibly test every possible RSS feed against my RSS 
parser, for example.

What is *supported* != what *happens to work*, not least because Ruby 
code tends to be generic by default. Even ignoring the generic issue, 
though, just because my RSS parser accepts a particular malformed feed 
today, doesn't mean I'm necessarily going support that malformed feed 
after I refactor the parser.

And when I'm using libraries, though I do also care about what's tested, 
my primary concern is what's supported. I care about knowing that 
something I use today won't change in an unexpected way and break my 
application. e.g. Rake, Yaml, ...

mathew
[ And as usual, as soon as I get into a discussion, the mailing list 
software stops sending me mail. ]

Thread

Prev Next

In This Thread

Prev Next