The growth of interest in business rules (BR) management has spurred an increasing number of attempts to implement ways in which rules can be operationally automated. From both theoretical and vendor perspectives, there is a perception that BR automation will be well received by most user communities, even though there may need to be some advocacy and education about the benefits of a BR approach. This perception seems to be justified when users are engaged in discussions about BR management. Many users are genuinely interested in being able to define and implement rules within the information systems they are work with. In particular they often want to implement incremental enhancements or change their systems to match gradual evolution of their businesses. It seems to these users that the difficulties of dealing with IT departments to have such changes implemented in the traditional way outweigh the benefits to be gained, and they would welcome any mechanism to implement their own business rules.

Beyond incremental changes to systems, users and major software vendors are also looking for ways to make systems integration easier. Everyone seems to agree that it would be much more efficient (and probably a lot cheaper) to allow users to implement rules that enable large-scale software products to be successfully integrated with the business processes and existing systems of an enterprise. This goal may still be some way off, but it no longer seems to be unattainable.

If there is so much genuine interest and demand for BR automation, could there possibly be any resistance on the part of users to it? Unfortunately, for those users that have actually participated in BR automation projects there can be a distinct unease that is sometimes manifested as resistance. This reaction occurs as a result of what can be called the "Black Box Problem," and this problem deserves close attention from the BR community if BR automation is to be a success.

The problem arises when users have to work with a piece of software into which they enter business rules, and which then receives inputs that are transformed into outputs. To the users this software is like a "black box" -- they can see rules they have created, the outputs that are produced, and perhaps the inputs. However, they cannot "see" the execution of the rules inside the black box, which may lead to feelings of uncertainty and loss of control, and even to a resistance to BR approaches.

Some Personal Experiences

For over 10 years I have been building software applications for structured finance. A large part of this has involved building interfaces that allow users to define their own business rules, which then run as part of the application.

The world of finance is extremely ripe for BR automation because it consists of many complex deals. These deals fall into many classes, ranging from things like real estate partnerships to the securitization of automobile leases. Within each class of deal there is considerable similarity, but not all related deals are identical.

It is also true that, at a high level of abstraction, all the deals within a particular class need to run identical processes (like the distribution of profits to partners or aggregation of monthly collateral positions). Yet at a detailed level these processes differ from one deal to the next. These slight differences arise as a result of financial innovation, different goals of the parties involved, changing legal and regulatory environments, and other factors. It is simply impossible to build one system that can manage a family of similar deals without making allowances for the variations between them -- and these variations cannot be known in advance. In my experience, the only real way to overcome this problem (and prevent an impossible support burden) is to allow users to define their own business rules that run as part of the system.

However, I found early on that although users recognized the value of this approach and used it, they behaved in ways that indicated they did not trust it. The Black Box Problem had manifested itself.

One very surprising finding was that users who eagerly adopted a BR automation solution would also run a shadow set of spreadsheets that replicated the rules they input into the system. When a particular business process ran, the users would then repeat the entire process in their spreadsheet models. This generated a great deal of extra work for the users involved, although they kept quiet about what they were doing so their senior management was not fully aware of it. When I asked the users why they were doing this large amount of extra work, they said it was to be sure that the results they were getting from the rules-based software were accurate and to double-check the calculations that were going on inside it.

I have found that this experience has been shared by the few other colleagues whom I know build software applications that permit user-defined business rule automation. Their users also found ways to duplicate what the rules automation software was doing, in order to check the results it was providing.

How Not What

A second symptom of the Black Box Problem manifested itself early on in my BR career. I found that I was called in on a number of occasions to explain how certain numbers were calculated, based on rules that the users themselves had defined. For these users it was important to understand how the rules were working. They were not interested in the technical architecture of the application as such but the processes at work within the rules themselves.

In such circumstances it was not helpful to tell users that the rules executed whatever it was that the users themselves had instructed the rules to do. It turned out that the users had varying degrees of uncertainly about whether they had written the rules correctly. Even though the application demanded that the rules were highly atomic -- and they were expressed in English terms -- the users have the following concerns:

• When a user selected a database column to be included in a rule, was this column really the one they meant? Even when the application displayed agreed logical names for the data, they worried that they could be choosing the wrong column.
• Where the rule was a calculation (which many of our rules were), what was the order of operations within the rule?
• What was the order in which rules fired? This was particularly important where a result produced by one rule became an input into another rule.

These problems manifested themselves as users tried to correlate the rules they had written in the BR automation software with the spreadsheets they had built to shadow the rules application. It was only when the users could see exactly how a particular output number was derived from the inputs provided that they agreed with the rule.

And significant problems arose when the users felt that there was a discrepancy between an expected output number and the number generated by the rules application. These problems sometimes required a great deal of research and fell into four classes:

1. The expected result had been incorrectly calculated by whatever means the users used to shadow the rules application (so there was really no problem).
2. There were data quality problems with the input data.
3. The users had written the rule incorrectly when they input it into the application
4. There was a bug in the application itself.

It was very rare to find a bug in the rules application itself. This is because, by its very nature, rules automation software is highly leveraged -- that is, the same code is used repeatedly to generate many different rules. Thus, any bugs that may exist become apparent in a very short time. The most common findings were that there was not a problem, or that the users had written a rule incorrectly.

What Are the Rules?

There is another problem that can become bound up with the Black Box Problem, even though it is not really part of it. This is the issue that users may not be fully able to articulate and define the rules they use in their business processes -- and hence do not include them in the rules they write into their BR automation software.

In the financial world, this is usually manifested when the users observe that the rules automation application is producing a number that they consider not to be incorrect. Once again, the question is asked -- "How is this number is being produced?" It is not unusual that, after all the steps in the production of the number are retraced, the finding is that a rule is missing. This is often because a rarely-encountered condition has not been taken into account, and one or more rules need to be included to deal with that condition.

There have been many articles written about fully defining rules associated with business processes, and there are a number of good methodological approaches (and even tools) that permit the results of this analysis to be stored and tracked. Unfortunately, in a production BR scenario, all problems seem to be viewed by users through the prism of the Black Box Problem, and the complaint is that if they could only "see" what is happening inside the rules automation software they would have increased confidence in the BR approach.

Inflexibility Beyond Rules

If a user forgets to include a rule it is not really the fault of the software provider. However, there is also a danger that the software provider may, perhaps unwittingly, include inflexible hard-coded rules in the BR automation software itself. For anyone building a rules-based application there is always the temptation to include certain functions "to make things easier for the user."

However, these functions may not always be quite what the user wants, and in trying to use these functions the user may never be able to write a particular rule to work in the way that he or she wants. For instance, in some financial rules it is necessary to determine the number of days between two dates. Sometimes the dates are inclusive while under other circumstances they are not. Thus a simple date difference function is insufficient.

If the users are constrained by functions to be able to do things only in a certain way, they again feel that the BR automation software is behaving like a black box that they cannot control. This is a particularly devastating criticism for software that is designed to provide users with the flexibility to write their own rules. The only way to avoid it is to try to anticipate how users may want to override the working of each function and make sure that this capability is built in.

Reliance on IT

Now, it may be argued that any piece of software put into production is a black box from a user's perspective. However, in traditional systems development, users have developers as partners. Usually, the principal responsibilities of the users are to confirm that their business requirements have been gathered, perhaps to validate that a proposed design will meet these requirements, and to undertake user acceptance testing. If something goes wrong with the implemented system, the users may need to confirm that there is a genuine problem, but they typically will call in IT staff (or equivalent technical resources) to diagnose the bug and to fix it.

With BR automation software, the users are placed in a position where they must be more self-reliant. By using such software the users avoid the problems of dealing with IT staff to build and implement a solution. However, there is no IT support available in the same way as there would be for a traditional systems development project. The perception that there is not a partner who must take responsibility for technical support can be very unsettling to users.

This is true even if the support they have received from IT staff has traditionally been very poor, and even if the BR automation software provider is prepared to provide some degree of support. The fact is that the users understand that they will have to be responsible for the rules they write and that they are operating a black box where they cannot "see" what is happening. Users everywhere tend to shy away from having responsibilities for anything that is technical -- and for good reason.

Yet if they are dealing with a rules automation black box, they are never quite sure of where the boundary between technology and business lies. The result is a feeling that they have acquired additional responsibilities that they are not equipped to deal with.

Seeing Is Believing

Having encountered the Black Box Problem on a number of levels, I have attempted to mitigate it in the BR automation software that I have written using several techniques.

1- Report the Rules

The first technique is to provide reports of the rules themselves. These reports state (in English) what operations the rules are carrying out. It is necessary to supply a well-documented data model with such reports because, if the users are to understand the rules, they must understand the data model they are dealing with.

In complex financial transactions this approach does seem to work. Users in this arena are often highly educated and quick to adapt to new techniques. They are used to modeling deals and welcome the representation of their data. What is more difficult is explaining the cardinalities that exist between related entities, but this is necessary since many rules involve the aggregation of numbers from a child entity to a parent, or the pro-rating of a number in a parent amongst instances within child entities. Users need to understand cardinalities in order to write these kinds of rule, which are very common in financial applications.

2- Document the Flow

Even good documentation of individual rules and a clear description of a data model is not sufficient to explain how rules have been set up in a BR automation application -- users also demand to know the sequences in which rules fire. The best technique I have found in this area is to produce a flow diagram that shows the sequence of rules. It is not very productive to explain to users that the rules engine can itself determine the sequence in which rules fire by working out dependencies; they want to actually have it described to them.

The flowcharting approach, which can be implemented by (for example) driving Visio from a web-enabled database, is also important in certain financial transactions where the sequence in which rules fire must be controlled by users. For instance, in a real estate partnership the monthly revenue may need to be used first to pay off loss-making investments. Any money left over must then be used to pay accumulated fees charged for managing currently-owned properties. If there is money left over from that then it must be used to provide partners with a 10% return on their invested capital.

This kind of "cash waterfall" is extremely common in structured finance and is an ideal business application for BR automation. Users of these applications really want to see the waterfall depicted in the reports of their business rules, and a visual flowcharting method is usually preferred. Producing these flowcharts goes a long way to removing the Black Box Problem as an issue that has to be dealt with.

3- Provide for Auditability

But producing reports about how rules are structured and how they will run only goes so far. Users typically demand that the results produced can be audited by them. This means that as each rule fires information about it must be recorded. The information is typically the name of the rule, the values that are input to it, and the outputs that are produced by it. After all the rules in a business process have been run, a report can be printed containing this information and the users can then see how any numbers they may question were produced.

Including such auditability may come at a price. It often slows down the execution process of the rules software and generates a great deal of output. Making the running of the audit functionality a user option can mitigate this. Implementing this approach has made an enormous difference for my users and has greatly reduced my support requirements. It provides the transparency that is required to really overcome the Black Box Problem.

4- Remember Flexibility

Lastly, when building rules automation software it is necessary to continuously bear in mind that nothing must be done to introduce inflexibility or "hard-coded rules" into the application, as they will defeat the purpose for which it is being built. In reality, especially with fixed deadlines, this can be a very difficult goal to meet, but it is essential for any successful BR automation software.

Conclusion

The Black Box Problem is something that should be taken seriously by the business rules community. Users who opt for implementing BR automation software should not be put in a position where they feel they are dealing with something whose inner workings are "mysterious" and for which they have limited or no support. The best ways to mitigate this problem is to provide detailed and useful reports on the rules the users have written and the sequence in which they fire. Sequence may be especially important to certain users whose rules involve specifying precedence. Beyond this, it is necessary to provide users with an optional audit trail of what the rules do as they fire in the execution of a business process. Additional support materials may be useful, especially a data model accompanied by an explanation thorough enough to allow users to understand what they are doing when they work with data in writing their rules. Paying attention to these aspects can improve the success of BR automation activities by bringing transparency to "Black Boxes."

# # #