Home | Trees | Index | Help |
---|
Module nonmockobjects |
|
How to tell if you need this module: You want to run automated tests on your code, but you have a relatively complicated data model, perhaps a nicely normalized database. The bulk of your tests consist of setting up this relatively complicated data model, and the tests all break whenever you change your model. Constructing the data has become so difficult (and breaks so often) that you've just stopped testing entirely.
nonmockobjects.py is a framework I built to solve this problem. Some test systems seem to provide some limited support for very simple homogenous test data, like some small set of rows in one table, unrelated to anything else; this module is designed for the rapid creation of hierarchial, heterogenous (or even graph-oriented) data at the relevant level of detail for your current test, without sacrificing the power to reach down four layers and set a certain setting for your test, and avoiding "mock objects" to the extent possible in favor of real objects.Suppose we have a standard database application which includes Users, Accounts, Permissions, and Resources. For simplicity, suppose Permissions belong to Users, and Users and Resources belong to Accounts.
Suppose the following constraints must hold, and are all tested upon object creation:Suppose you want to test an Account. In order to create an Account, you must have created an Resource and a User. In order to create a User, you must have a Permission.
If you write the most obvious test code for your Account, your tests will have to create a User, a Resource, and a Permission. If you later change how your Users and Permissions interact, your Account test will break, because even if the interface to User and/or Permissions is compatible at the class level, your tests are still tightly coupled to the entire object tree. (Obviously, in the real world with real models this problem is even more acute.)
If you try to abstract this out into a function, you will, bit by bit, refactor your way into a model where each level of the object graph basically has its own function, and you'll eventually refactor your way into being able to pass arguments down from a high level to a low-level freely.
Then, you can say something like newAccount =
data.newAccount()
and get a correctly-configured Account
object, while still being able to change the user's name with
something like: newAccount = Objects.newAccount(use_user:
{'name': 'Bob'})
, or so on, even deeper.
First, find a "leaf object" in your code. This is an object that can exist without reference to any other object.
In the example, this would be a Permission; it is the only object of the four that can exist in isolation.
Create a function that can create this leaf object, using whatever relevant parameters you want. For simplicity, Permissions only take a 'name':def createPermission(data, name = 'testpermission'): return Permission(name)
The first parameter is the nonmockobjects.Objects object that you are using to call this function.
Now, register this function with nonmockobjects:from nonmockobjects import register @register def createPermission(data, name = 'testpermission'): return Permission(name)
data = nonmockobjects.Objects()Now, you can create a permission via the method on data corresponding to the name of your function:
permission = data.createPermission() # is named 'testpermission'You can override the name with either normal or keyword args (note this will change later):
permission = data.createPermission('test2') # is named 'test2' permission = data.createPermission(name='test3') # is name 'test3'At this point, we've done nothing you couldn't already do, of course.
This is where it starts to get interesting.
Find an object that is the next layer up from your leaf object. In this example, it is the User object, which must have a permission, which must be passed in as a list of permissions:@register def createUser(data, name = 'testuser%(inc)s', email = lambda: 'email sample', permission = createPermission): return User(name, [permission])
There are some special services nonmockobjects is providing for
you, as documented in register
. Since email
is
callable, it will actually be called each time to generate a default
email, if you don't pass one in. permission =
createPermission
indicates that we want to create a new
permission if one isn't passed in, or that we want to use certain
arguments to construct a new permission.
user = data.createUser(permission = permIHave)Passing in an existing permission causes the user to use that.
user = data.createUser(use_permission = {'name': 'new permission'})This creates a new Permission as if you called data.createPermission(name = 'new permission'), then creates a new User with that permission.
user = data.createUser()
Now when you need to test a user, you can create a user with the necessary level of detail, and no more. If what you are testing has nothing to do with permissions, you can just create one. If you need the user to have a certain permission, you can do that too.
Note that normal concerns about the mutability of Python keyword arguments apply.Repeat Step 3 as needed for all objects you want to test.
If you continuing creating functions, eventually you'll end up with a function "data.createAccount()", which creates multiple objects and ties them together for you, but allows you the ability to override any part of that process. For instance, to create a permission with a specific name:account = data.createAccount(use_user = {'use_permission': {'name': 'specific permission' } } )Feel free to add new creation or manipulation methods as needed. For instance, if this were an app I would have created an addUserToAccount method:
@register def addUserToAccount(data, account, user = createUser): account.addUser(user)
This allows you to either add a user to an account you already have in hand, or allow the nonmockobjects system to simply come up with a new user. (Creating functions that can either use existing objects or create them on the spot is more useful when you have more parameters and more complicated functions; the true power of this approach really can't be demonstrated in sample code.)
In the end, you'll end up with a series of functions that match your system's structure, and you can create tests that depend only on what you're testing. Thus, if the way Users and Permissions works changes, your calls to data.createAccount() need not be changed, except in tests directly affected by the changes. No matter how complex your object system may get, if you want to test a "leaf" object, all you have to do is "objects.leafObject()".
Using this module makes it practical to test very complicated objects, and to write tests that use data as close to production data as possible (since this takes the pain out of creating large test structures).Functions that you @register are used to create methods on the
nonmockobjects.Objects class. That's why they always take
data
as the first argument; in some sense that's actually
self
, but that would be deceptive I think. (You can use it
if you like, of course.)
Objects
object's documentation for the
details, but as a quick note, the Objects object takes any keyword
arguments and sets them as attributes on itself. Thus, if your test
functions need access to a database connection, you can:
data = nonmockobjects.Objects(db_conn=db_conn)
and all test creation objects can access the database connection by getting the db_conn attribute from their first parameter.
It is probably a bad idea to use this to communicate amoung creation functions, but knock yourself out.Sometimes it is useful to test several variations of a given object; this can be useful to test that invariant hold, that a "real account" is created for any of several combinations of options, or other things.
NonMockObjects provides a way to quickly generate numerous
combinations of parameters for creating an object using
Choose
and ChooseArgs
.
def permission(data, name = Choose("perm%(inc)s", "SpecialPerm")): return Permission(name)
When you call this as data.permission()
, NonMockObjects
will automatically take the first choice as the "default" for
the object, resulting in a permission that uses the automatic
incrementer to create a name.
However, if you call permissions =
data.variations_permission()
, you get an iterator that will
yield a series of Permission objects, created by making calls to the
permission
function, using each choice once. In this case,
it will return two permissions, one using the default incrementer, and
one being "SpecialPerm", which you may want to test for some
reason.
(A different method name is used based on variations_*
,
because returning an object is fundamentally different than returning
an iterator. Explicit is better than implicit.)
ChooseArgs
is basically the same as Choose, except that
instead of providing a default value, it provides a series of
sub-argument specifications. For instance:
def user(data, name = "User%(inc)s", permission = ChooseArgs(permission, {'name': 'perm1'}, {'name': 'perm2'})): return User(name, permission)
The first argument is the function to use to create the sub-object
(same as specifying the function without ChooseArgs), and the remainder
of the arguments are a series of dicts to use as the sub-object
parameters. In this case, data.variations_user()
would
produce two users, one with the perm1
Permission and one
with the perm2
Permission.
Choose
or
ChooseArgs
in a function definition. In that case,
NonMockObjects will create one object for every combination of
choices:
def user(data, name = Choose("Bob", "Jane", "Spock"), permission = ChooseArgs(permission, {'name': 'perm1'}, {'name': 'perm2'})): return User(name, permission)
This returns six User objects, using each combination of name and permissions.
Because of the danger of the combinations growing rapidly, this functionality is not done recursively. If you need that, you should explicitly create a new top-level function that dispatches arguments as appropriate.
TheseChoose[Args]
-based attributes can be overridden
as normal in a call:
users = data.user(name="Me")
with the previous user
function will only return two
User objects, both named Me but one with perm1
and one
with perm2
.
def account(data, name = "Account%(inc)s", user = user): return Account(name, user)you can call:
accountsToTest = data.variations_account(name=Choose('a', 'b'))
to an iterator which will produce two accounts, each with a new user, one named "a" and one named "b".
You can also use this to manually limit a choice of many objects to a choice of fewer objects, to cut down on the number of choices generated.Fundamentally, every difference between your production code and your test harness is a place for bugs to live. Every mock object you add is a difference between your test harness and your real system, and is therefore a place where bugs can live.
I've been down that road and I didn't like it. It was better than nothing, but there were too many things I missed. A half-assed combination of mock objects, mostly to satisfy constraints like those discussed above, and test-specific hacks to fix a certain test in a way not used anywhere else in the system, let alone our production code, was getting on my nerves.
This structure allows you to minimize the number of hacks in your testing code by taking the pain out of constructing real objects.
Mock objects and hacks may still be necessary for performance and replicability purposes, but I think they ought to be considered a last resort or a performance optimization, not the first thing you reach for in a test situation. The exact balance will of course depend on your situation.You should create a solid naming style for these test functions. I've been using "newSomething" for a function that creates something from scratch, and "addSomething(target)" for a function that adds a something to the given target, where the target must be the first argument.
It is a common pattern to have a function that just sets the default parameters for a call to some other function, like so:def createSomething(data, a = 1, b = "test%(inc)s", c = 4): return Something(a, b, c)As the number of parameters grows and the names get long, it is bad to have to retype them in the Something call. Consider this instead:
def createSomething(data, a = 1, b = "test%(inc)s", c = 4): return Something(**all_args())
NonMockObjects provides all_args, which is just like "locals()" except it is smart enough to pop off the first argument, which is the Objects instance, which you don't need.
It is possible to put the creation functions for a class in the class itself, like:class User(object): def __init__(self): ... @register def newUser(data, name): return User(name)
But note that at the time the register decorator is run, it has no way (to my knowledge) to know that newUser is in a class specification; register recieves a function object, not an unbound method. As a result, register can not treat the function specially in any way, so it must act just like any other registered function.
On the one hand, it may be convenient to bundle all test creation functions in with the class specification. On the other hand, while 'newUser' does not really belong to User (it's really a method on nonmockobjects.Objects), the User class will still end up with a newUser method, which may screw up introspection, may be confusing if accidentally called, etc. I don't recommend sticking them in classes.
The reason I went with functions as the final creation mechanism, and not some metadata specification or an attempt to label a method/classmethod as the 'creation' technique, or try to introspect the class itself, is that you will find that in practice, your real functions will end up more complicated than the deliberately-simple examples shown here. I use this in a well-normalized-database environment (in Django), so I do know it does at least some real-world tasks correctly, even relatively complicated ones.
Write powerful functions, but try to minimize their length; anything that can be moved up into the class probably should be. Even so, the 'core' test code left behind can be surprisingly large. Production code should not use any nonmockobjects creation functions; if you are tempted to do so, that means you have functionality to move into a separate method or function. I find the test code often teaches me what convenience methods to add to my code early on, when the test code may do something five times but I've only gotten one 'real' use.
If you have many slightly different functions, do not be afraid to use "register_as" in a loop:for specialName in [...]: def testFunction(data, arg1, arg2): # do things, do something different based on specialName register_as('testFunc%s' % specialName)(testFunction)
Repetitiveness in test code is just as evil as it is anywhere else.
To take the names of functions and create the "use_function = {}" parameter names, nonmockobjects calls nonmockobjects.use_prefix(methodName). The default function implements the default Python naming policy, which will change 'permission' into 'use_permission', for instance. Feel free to override that function (as early as possible) to implement your own local coding standards, whatever they may be. The best thing to do may be to create a small module that wraps nonmockobjects and overrides that function, then use that module instead of nonmockobjects directly.
This system helps make it easier to start with a truly fresh test database, if you've got a database application, as creating test accounts of any complexity is just a function call.
The new Python 2.5 'partial function application' support can be really useful with the code you'll produce with this module.Classes | |
---|---|
Choose |
Implements a selection of choices for a given parameter. |
ChooseArgs |
Implements a selection of choices to feed to a sub-function. |
Objects |
Objects is the object you instatiate to get access to your creation functions. |
Chooser |
A superclass for the multiple-choice choosers. |
Function Summary | |
---|---|
This gets all the local args of the calling stack frame, but filters out the Object instance parameter. | |
Protects callables that you really want to pass as a function by wrapping it in a function that will return your callable. | |
Register a function as a non-mock-object creation function. | |
Longer form of register that takes a name_override to set the Objects name of the registered function to. | |
Standard use_prefix function: Implements 'use_' + name. | |
Standard variations_prefix function: Implements 'variations_' + name. |
Variable Summary | |
---|---|
count |
INCREMENTER = count(0)
|
Function Details |
---|
all_args(*exclude)This gets all the local args of the calling stack frame, but filters out the Object instance parameter. |
call_protect(callable_object)Protects callables that you really want to pass as a function by wrapping it in a function that will return your callable.
|
register(func=None)Register a function as a non-mock-object creation function. Can be used as a decorator. register examines the default arguments you provide for the given function by using Python introspection, and performs the following manipulations when calling your function:
call_protect function:
@register def call_func(function: call_protect(lambda x: x + 1)): return function(44)
|
register_as(name_override=None)Longer form of register that takes a name_override to set the Objects name of the registered function to. (This can be necessary when the name you want for the method conflicts with another object by the same name; just create a function with a throwaway name, and pass in a name_override to register_as.).
|
use_prefix(name)Standard use_prefix function: Implements 'use_' + name.
|
variations_prefix(name)Standard variations_prefix function: Implements 'variations_' + name.
|
Variable Details |
---|
INCREMENTER
|
Home | Trees | Index | Help |
---|
Generated by Epydoc 2.1 on Thu Mar 1 14:03:21 2007 | http://epydoc.sf.net |