Module nonmockobjects

Summary

How to tell if you need this module: You want to run automated tests on your code, but you have a relatively complicated data model, perhaps a nicely normalized database. The bulk of your tests consist of setting up this relatively complicated data model, and the tests all break whenever you change your model. Constructing the data has become so difficult (and breaks so often) that you've just stopped testing entirely.

Example Of The Problem

Suppose we have a standard database application which includes Users, Accounts, Permissions, and Resources. For simplicity, suppose Permissions belong to Users, and Users and Resources belong to Accounts.

• A User must have at least one permission in order to be valid.
• An Account must have at least one user to be valid.
• An Account must have at least one resource to be valid.

Suppose you want to test an Account. In order to create an Account, you must have created an Resource and a User. In order to create a User, you must have a Permission.

If you write the most obvious test code for your Account, your tests will have to create a User, a Resource, and a Permission. If you later change how your Users and Permissions interact, your Account test will break, because even if the interface to User and/or Permissions is compatible at the class level, your tests are still tightly coupled to the entire object tree. (Obviously, in the real world with real models this problem is even more acute.)

If you try to abstract this out into a function, you will, bit by bit, refactor your way into a model where each level of the object graph basically has its own function, and you'll eventually refactor your way into being able to pass arguments down from a high level to a low-level freely.

Then, you can say something like newAccount = data.newAccount() and get a correctly-configured Account object, while still being able to change the user's name with something like: newAccount = Objects.newAccount(use_user: {'name': 'Bob'}), or so on, even deeper.

How To Use This Module (/Tutorial)

Step 1

First, find a "leaf object" in your code. This is an object that can exist without reference to any other object.

In the example, this would be a Permission; it is the only object of the four that can exist in isolation.

    def createPermission(data, name = 'testpermission'):
        return Permission(name)

The first parameter is the nonmockobjects.Objects object that you are using to call this function.

    from nonmockobjects import register
    @register
    def createPermission(data, name = 'testpermission'):
        return Permission(name)

Step 2

    data = nonmockobjects.Objects()

    permission = data.createPermission() # is named 'testpermission'

    permission = data.createPermission('test2') # is named 'test2'
    permission = data.createPermission(name='test3') # is name 'test3'

Step 3

This is where it starts to get interesting.

    @register
    def createUser(data,
                   name = 'testuser%(inc)s',
                   email = lambda: 'email sample', 
                   permission = createPermission):
        return User(name, [permission])

There are some special services nonmockobjects is providing for you, as documented in register. Since email is callable, it will actually be called each time to generate a default email, if you don't pass one in. permission = createPermission indicates that we want to create a new permission if one isn't passed in, or that we want to use certain arguments to construct a new permission.

• Create a user using a permission you already have in hand (here bound to 'permIHave'):

      user = data.createUser(permission = permIHave)
  

  Passing in an existing permission causes the user to use that.
• Create a user and simultaneously create a permission with given arguments:

      user = data.createUser(use_permission = {'name': 'new permission'})
  

  This creates a new Permission as if you called data.createPermission(name = 'new permission'), then creates a new User with that permission.
• Finally, just create a new user and let the system take care of all the necessary Permission creation:

      user = data.createUser()
  

Now when you need to test a user, you can create a user with the necessary level of detail, and no more. If what you are testing has nothing to do with permissions, you can just create one. If you need the user to have a certain permission, you can do that too.

Step 4

Repeat Step 3 as needed for all objects you want to test.

    account = data.createAccount(use_user = {'use_permission': {'name':
                                  'specific permission' } } )

    @register
    def addUserToAccount(data, account, user = createUser):
        account.addUser(user)

This allows you to either add a user to an account you already have in hand, or allow the nonmockobjects system to simply come up with a new user. (Creating functions that can either use existing objects or create them on the spot is more useful when you have more parameters and more complicated functions; the true power of this approach really can't be demonstrated in sample code.)

In the end, you'll end up with a series of functions that match your system's structure, and you can create tests that depend only on what you're testing. Thus, if the way Users and Permissions works changes, your calls to data.createAccount() need not be changed, except in tests directly affected by the changes. No matter how complex your object system may get, if you want to test a "leaf" object, all you have to do is "objects.leafObject()".

How This Works

Functions that you @register are used to create methods on the nonmockobjects.Objects class. That's why they always take data as the first argument; in some sense that's actually self, but that would be deceptive I think. (You can use it if you like, of course.)

   data = nonmockobjects.Objects(db_conn=db_conn)

and all test creation objects can access the database connection by getting the db_conn attribute from their first parameter.

Variations

Sometimes it is useful to test several variations of a given object; this can be useful to test that invariant hold, that a "real account" is created for any of several combinations of options, or other things.

NonMockObjects provides a way to quickly generate numerous combinations of parameters for creating an object using Choose and ChooseArgs.

   def permission(data, name = Choose("perm%(inc)s", "SpecialPerm")):
       return Permission(name)

When you call this as data.permission(), NonMockObjects will automatically take the first choice as the "default" for the object, resulting in a permission that uses the automatic incrementer to create a name.

However, if you call permissions = data.variations_permission(), you get an iterator that will yield a series of Permission objects, created by making calls to the permission function, using each choice once. In this case, it will return two permissions, one using the default incrementer, and one being "SpecialPerm", which you may want to test for some reason.

(A different method name is used based on variations_*, because returning an object is fundamentally different than returning an iterator. Explicit is better than implicit.)

   def user(data, name = "User%(inc)s",
            permission = ChooseArgs(permission,
                                    {'name': 'perm1'},
                                    {'name': 'perm2'})):
       return User(name, permission)

The first argument is the function to use to create the sub-object (same as specifying the function without ChooseArgs), and the remainder of the arguments are a series of dicts to use as the sub-object parameters. In this case, data.variations_user() would produce two users, one with the perm1 Permission and one with the perm2 Permission.

   def user(data, name = Choose("Bob", "Jane", "Spock"),
            permission = ChooseArgs(permission,
                                    {'name': 'perm1'},
                                    {'name': 'perm2'})):
       return User(name, permission)

This returns six User objects, using each combination of name and permissions.

Because of the danger of the combinations growing rapidly, this functionality is not done recursively. If you need that, you should explicitly create a new top-level function that dispatches arguments as appropriate.

   users = data.user(name="Me")

with the previous user function will only return two User objects, both named Me but one with perm1 and one with perm2.

   def account(data, name = "Account%(inc)s", user = user):
       return Account(name, user)

   accountsToTest = data.variations_account(name=Choose('a', 'b'))

to an iterator which will produce two accounts, each with a new user, one named "a" and one named "b".

Why Do This?

Fundamentally, every difference between your production code and your test harness is a place for bugs to live. Every mock object you add is a difference between your test harness and your real system, and is therefore a place where bugs can live.

I've been down that road and I didn't like it. It was better than nothing, but there were too many things I missed. A half-assed combination of mock objects, mostly to satisfy constraints like those discussed above, and test-specific hacks to fix a certain test in a way not used anywhere else in the system, let alone our production code, was getting on my nerves.

This structure allows you to minimize the number of hacks in your testing code by taking the pain out of constructing real objects.

Hints, Tips, and Tricks

You should create a solid naming style for these test functions. I've been using "newSomething" for a function that creates something from scratch, and "addSomething(target)" for a function that adds a something to the given target, where the target must be the first argument.

    def createSomething(data, a = 1, b = "test%(inc)s", c = 4):
        return Something(a, b, c)

    def createSomething(data, a = 1, b = "test%(inc)s", c = 4):
        return Something(**all_args())

NonMockObjects provides all_args, which is just like "locals()" except it is smart enough to pop off the first argument, which is the Objects instance, which you don't need.

    class User(object):
        def __init__(self): ...

        @register
        def newUser(data, name):
            return User(name)

But note that at the time the register decorator is run, it has no way (to my knowledge) to know that newUser is in a class specification; register recieves a function object, not an unbound method. As a result, register can not treat the function specially in any way, so it must act just like any other registered function.

On the one hand, it may be convenient to bundle all test creation functions in with the class specification. On the other hand, while 'newUser' does not really belong to User (it's really a method on nonmockobjects.Objects), the User class will still end up with a newUser method, which may screw up introspection, may be confusing if accidentally called, etc. I don't recommend sticking them in classes.

The reason I went with functions as the final creation mechanism, and not some metadata specification or an attempt to label a method/classmethod as the 'creation' technique, or try to introspect the class itself, is that you will find that in practice, your real functions will end up more complicated than the deliberately-simple examples shown here. I use this in a well-normalized-database environment (in Django), so I do know it does at least some real-world tasks correctly, even relatively complicated ones.

Write powerful functions, but try to minimize their length; anything that can be moved up into the class probably should be. Even so, the 'core' test code left behind can be surprisingly large. Production code should not use any nonmockobjects creation functions; if you are tempted to do so, that means you have functionality to move into a separate method or function. I find the test code often teaches me what convenience methods to add to my code early on, when the test code may do something five times but I've only gotten one 'real' use.

    for specialName in [...]:
        def testFunction(data, arg1, arg2):
            # do things, do something different based on specialName

        register_as('testFunc%s' % specialName)(testFunction)

Repetitiveness in test code is just as evil as it is anywhere else.

To take the names of functions and create the "use_function = {}" parameter names, nonmockobjects calls nonmockobjects.use_prefix(methodName). The default function implements the default Python naming policy, which will change 'permission' into 'use_permission', for instance. Feel free to override that function (as early as possible) to implement your own local coding standards, whatever they may be. The best thing to do may be to create a small module that wraps nonmockobjects and overrides that function, then use that module instead of nonmockobjects directly.

This system helps make it easier to start with a truly fresh test database, if you've got a database application, as creating test accounts of any complexity is just a function call.

   @register
   def call_func(function: call_protect(lambda x: x + 1)):
       return function(44)