1  Background

A recent software development project for a New Zealand Government Agency has provided locally-based IT consultancy, Equinox, with an opportunity to test a business rules approach.  Business rules are applied throughout the agency's core business processes in order to:

• control capture of application data,
• determine allocation of cases,
• support decisions made during assessment of each case,
• select approved providers,
• identify appropriate services based on the type of case,
• drive the contents of letters to providers and clients, and
• check claims for payment of provider services against guidelines.

Equinox was engaged in 2008 to re-develop and enhance the software application supporting these key processes.  Two of the key business drivers for this work were:

1. "The Agency needs to be able to implement business changes with low risk and in accordance with  their current technology Principles" and
   
   
2. "Administrators need to improve the consistency and quality of granting decisions."

With these aspirations in mind, a decision was made to make use of a business rules engine to de-couple business logic wherever possible from other code.  This decision was made in the early stages of selecting the architecture for this re-development.

To make the most of the flexibility that a business rules engine could provide, Equinox was aware that the team would need to apply an analysis approach that would complement this technology.  In particular, it would need to:

• identify those rules that would be both in-scope for the software application and would be most likely to change in the future, and
  
  
• document the rules in a way that could be readily converted into the business rules engine syntax.

1.1  Case Study Features

The project was further characterised by the following factors:

• Iterative techniques employed within the framework of a Unified Process-based method.
  
  
• Use cases used to structure and document functional requirements. 
  
  
• Thirteen Equinox staff, one Microsoft data migration consultant, and five Agency staff directly involved at some time, over a period of 18 months.
  
  
• Extensive use made of Sparx Enterprise Architect modelling tool to document requirements, business rules, and architecture.

2  Issues

2.1  Volume of Rules

Between two-hundred and two-hundred-fifty business rules were within the system's scope.  Sources of these rules existed in many different forms, including legislation, Agency policy documents, and regional procedures.  Left to a single analyst to elicit and document, in a fashion appropriate for translation to rules engine syntax, would create an unnecessary bottleneck and a high risk of contributing to project overruns.

Many of these business rules were likely to be re-used by a number of use cases — for example, eligibility rules used during the Assess Application function would also be applied during Assess Amendment.  This represented a risk of inconsistencies, if documented in multiple use cases.  So, embedding rules in use case documentation was never an option.

2.2  Rules Elicitation Method

Equinox recommended, as always, that high-level solution requirements and business rules be identified in the context of a redesigned set of business process flows.

For a number of reasons, the client was not able to accommodate this approach.  There was a belief that the project was essentially a 're-development' of the existing system.  Also, a separate business process review by another party was already planned, and the client was looking to move forward as quickly as possible into the development phase.

Therefore, the business rules to be embedded in the system needed to be gathered and documented in parallel with elicitation and analysis of detailed software requirements.

2.3  Rules Engine

Some business rules engines are purported to provide a user-friendly syntax and rules-definition facilities that 'empower' business users to make business rule changes directly into the system.  Others support the parsing of documents in order to harvest the business rules directly from legislative and other documentation.

These are definitely at the Rolls Royce end of the business rules spectrum at the moment, with price tags to match.  The business rules engine employed on the Agency's Management System project was closer to the Volkswagen end of the spectrum — cheap but reliable and performs remarkably well.

The rules engine used comes free with the Workflow component of MicroSoft's .Net 3.5 framework.  It requires an object oriented-like syntax and data 'wrappers' to be built in order to make the relevant data available to each rule set.  So, any user attempting to make changes to rules needs to understand the classes and attributes that provide the link to data.  In short, creation and maintenance of rules require the user to have some technical background.

The tool therefore required us to make manual translations from the original rule source — with an inherent risk of misinterpretation by the analyst — and then for these rule definitions to be coded by a developer into the engine — with an additional risk of inaccurate translation.  Thus, the nature of the technology used created a very real risk of inaccurate implementation of business rules.

Having said this, the advantages of the rules engine that justified this approach included:

• Support of separation of conditional logic code from main business logic layer.  So, business rules are coded in one place and reused from multiple points (in the code).
  
  
• Support of 'forward chaining' of rules (one rule triggering another) and therefore support of 'Conditional' action assertion rules.
  
  
• Support of 'backward chaining' of rules (a rule that references the results of other rules) and therefore support of decision trees.
  
  
• Capability for 'effective dating' — that is, rules can be tested and implemented into production but do not take effect until a specified date.
  
  
• Separate front end applications available to support the creation and management of rules, thereby eliminating the need to develop rules administration software.

I should also add that we have had no issues with the implementation or execution of this rules engine and performance looks good so far.

3  Solutions

3.1  Theory — What, Where, and How

To address these issues, we hoped to be able to merge elements of a business rules approach, including categorisation of rules and use of a separate rules repository, into our existing Unified Process-based software development approach.  In essence, the intent was to apply business rules analysis to help identify what was rule versus functional requirement, where a rule would be implemented, and how to best document each.

3.1.1  What

When eliciting business rules within the context of detailed software requirements gathering, we've found there is a tendency to confuse system rules with business rules and sometimes to embed business rules into functional requirements.

To avoid these traps and ensure that business rules were isolated in a separate repository, to be offered as candidate business rules engine entries, the following basic tests were planned:

1.  Would this rule still apply if this system did not exist?

For example, a rule that states that "the application receive date should be the date the application arrives at an Agency office" is true whether or not this date is entered in a system.  Therefore, this is a business rule and may change if policy changes.  By contrast, the rule stating that the application date must be entered as DD/MMM/YYYY is a validation that ensures data is captured and stored consistently in the system; it would not exist if the system was not built.  This is a data validation rule, less likely to change, and is more appropriately implemented in the presentation layer of the system.

2. Does the requirement (or part of the requirement) fit one of the business rule categories?

If not, it is probably a functional or non-functional requirement rather than a rule.

For example, the requirement that states "The system must notify Team Leaders when a case allocated to one of their staff is withdrawn" is actually hiding a functional requirement:  "The system must provide automatic notification of key staff members when selected events occur" and a business rule:  "Team Leaders must be made aware of cases when the grants for those cases are withdrawn and the case is allocated to a member of their team" which fits the 'action enabler' category within the action assertion group of rules categories.

We would expect the functional requirement represented by automated notification to be covered by a use case workflow.  The data validation rule represented by the date format is a system rule and would be documented as a special requirement within the related use case (or within supplementary requirements, if the system rule was common to a number of use cases).

An early work of the Business Rules Group provides a useful categorisation approach that groups business rules into structural assertions, action assertions, and derivations.^[2]  The IIBA's BABOK version 1.6^[3] included a variation of this where 'action enablers' are treated as a separate rule type rather than a special kind of 'conditional'.  This appeared to be a simpler approach for the novice business rules analyst.

Accordingly, on this project the categories used — and definitions applied — were as follows:

• Structural Assertion

• Term  — business concept or entity (Client, Invoice, Invoice Claim)

• Fact  — relationship between entities (an Invoice Claim is related to one Invoice) or additional information about entities (Invoice Number, Invoice Date)

• Action Assertion

  • Conditional  — causes another rule to be applied

  • Action Enabler  — causes some action to be performed

  • Integrity Constraint  — determines whether some combination of facts is acceptable
        Only invoices for which all invoice claims are valid may be approved. — i.e., If the status of all claims related to an invoice is valid then the invoice is acceptable.

  • Authorisation  — action that specified business roles are authorised to perform

• Derivation  — an expression that determines the value of a fact
      Invoice Total = sum of all invoice claims + GST

3.1.2  Where

Categorising also helps to determine where the business rules will be implemented and therefore where best to document them.  This was used to identify those rules to concentrate on when building the list of rules to be provided to the development team as candidate business rules engine entries.

Terms and facts will eventually be implemented as tables, columns, and referential integrity constraints.  From a requirements perspective, we document data requirements using a conceptual domain model, which evolves into a logical data model.  Accordingly, these rules would not be included in the business rules list.

Authorisations to be supported by software are maintained using the security administration functions, so from a software requirements perspective they add little value — as long as functional security requirements (such as the need for role-based or row-based security) are identified. There was therefore no intention to elicit and document security rules as part of the software requirements set.

The remaining rules would normally be coded into business logic.  The plan was to concentrate on the elicitation and documentation of these rules in a separate rules repository, as a distinct component of the software requirements package.

Figure 1.  Business Rule Categories and Requirements Models

In effect, we are specifically focusing on documenting those rules that will change when the organisation's policies or legislation change.  These are the Derivations and the Action Assertions (except for Authorisations that can be implemented later, by a security administrator).  The Structural Assertions (Terms and Facts) would be implemented in the database as tables and referential integrity constraints and would, therefore, be documented in the Domain Model.

3.1.3  How

In order to minimise the translation required from rules documentation to rules engine syntax, we planned to document different categories of rules using different formats, as follows:

• Conditional rules documented as IF rule expression THEN check rule
  
  
• Integrity Constraints documented as IF rule expression THEN Accept
  
  
• Action Enablers documented as IF rule expression THEN Perform xyz
  
  
• Derivations documented as Fact = rule expression

Here, a "rule expression" is a combination of facts or other rules, forming a Boolean expression.  Facts are implemented as attributes and so, at the elementary level, these are Boolean expressions related to data.

Where rules expressions rely on the results of other rules, the intention was to make use of the hierarchical element feature of the modelling tool.  For example, a requirement element BR 1.6.1, the result of which is used in BR 1.6, is added as a child element of BR 1.6 (shown in Figure 2).

Figure 2.  BR 1.6.1 — a Child Element of BR 1.6

3.2  Practice

Since it was not possible to identify business rules during a business process re-design exercise, rules were identified during use case workshops and then documented off-line by subject matter experts — effectively, a bottom-up approach to documenting just those rules that affect the system.

In practice, the rules elicitation and documentation process worked like this:

1. Business rules are identified during use case workshops.

Since we have employed use case workshops to identify the reference points for rules within use cases, this was business-as-usual for us.  If anything, our issues were with communication between the subject matter experts and the analysis and development team.  Hand-drawn screens and diagrams (albeit followed up with modelling tool documentation) just didn't cut it.  Later, we used a UI mock-up tool instead of drawing screens on the whiteboard.  In the future we intend to support validation of requirements with virtual walkthroughs (using Expression Blend 3 and SketchFlow), but that's another story.

Identification of "decision points" in the use case work flow worked well, and our subject matter experts had no issues being involved in this process.

2. Use Case workflow steps are written to include references to business rules (using a business rule number and title).

For example, a basic flow step might include:  "The system checks BR 3.1 Valid Service Claim, for each invoice line, determines that the invoice is valid, and prompts the user to authorise …"

There were also no issues raised relating to this technique, so I took no news to be good news.  The fact that the subject matter experts referred to certain rules being used at a specific point in the flow was also a good sign that this approach was workable.  We keep our alternate flows self-contained and our basic flows clean (i.e., source, condition, and destination are all documented in the alternate flows or extension use cases only), so adding a set of references to business rules did not clutter the use case work flows.

Of more impact to the project as a whole was the fact that not all business rules could be determined by the project SMEs alone.  Often, consultation with the wider client group was required.  In these cases, the related business process flow needed to be redesigned so, not surprisingly, we were having a "bottom up" impact on business processes.  The results of these discussions sometimes had a flow-on effect to other process flows and business rules — and therefore to software development — mostly causing changes to rules and functionality already implemented.

All this could have been avoided if business processes had been redesigned well, before software requirements and development began, and business rules collected and defined in this context.

3. Skeleton business rule requirement types are created within the appropriate Use Case element (within the Sparx Enterprise Architecture project repository), then moved to the appropriate location within the business rules package, using the Sparx EA Move External function. 

This creates a separate requirement element (stereotyped as 'Business Rule') for the rule — making it available to be linked to other use cases — and creates a link from the use case it was "discovered" in.

There were definitely some efficiency gains in documentation due to this.  In addition, presenting requirements for validation was simplified as discussed below.  While the developers tended to read a separate rules document created from this rules "repository," business reviewers preferred to see these rules documented within the use cases that referenced them.  Use of the modelling tool in the manner described above supports both needs, with minimal additional effort.

4. Skeleton business rule text is added to each rule element with comments as reminders of the intent.

The skeleton rules were formatted as If rules expression Then accept data change, execute another rule, perform function (depending on the type of action assertion) or, for derivations, Fact = rules expression.

We found this format decreased the degree of translation required by developers, and our SMEs assigned to the project team were comfortable with the approach.

5. Subset documents with these skeleton rules are generated from the repository and distributed to project team subject matter experts who complete the documentation, often working with other business experts.

This eliminated the risk of analyst misinterpretation of these rules but placed a large work load on the SMEs who were part of the project team.  Since there were more subject matter experts than analysts, the rules documentation bottleneck was eased, but not removed totally.

The resulting rules were not always atomic.  Often, rules came back with multiple If … Then expressions.  I suspect this reflected the subject matter experts' need to express rules as a related set (or, in Decision Model-speak, as a "rules family").  There was an implied sequence to the application of these rules, which sometimes needed clarification but, in general, could be derived from the context.

For example, the following excerpt is from the rule to determine that a case should be transferred to another office:

If the Application Type is A or E then the case should be transferred to the Christchurch Office. If the Application Type is R then the case should be transferred to the Auckland Office. ... If the application type is S then transfer the case to the Wellington Office.

Resulting rules specifications were not always precise, as little attempt was made by subject matter experts to create correct nesting or grouping and data was referred to using business terminology.  However, in the few cases where there was ambiguity, a quick discussion between developer and subject matter expert resolved the intent.

Expanding the glossary to include a reference to the related attribute name may have alleviated some of this ambiguity, although in practice this would have required the same conversations between the developers and subject matter experts that were required to implement the rules.

In most cases, the rules expressions came back from subject matter experts in a more English format than the original skeleton, as this obviously suited the business experts better.  In hindsight, it also helped provide a clear business context for the rules that were documented as part of system requirements.  (Note that explicit traceability was built in the modelling tool between layers of requirements — from use case to features to needs and rules to use cases — but not between rules and business processes and activities.)

For example, the rule for setting the integrity constraint for finalising a case started life as

If Case Finalisation criteria Then Allow Finalisation.

but was re-documented as

If  this is the final Invoice for the Case and the final invoice has been processed and there are no outstanding Invoices or Reversals or Credit Notes related to the Case, and there is no unprocessed communication related to the Case and there are no outstanding Requests logged against the Case Then the case may be finalised.

Since the developers knew the context of this rule and the data it represented, this description required no further explanation.

Other example results were:

Example Integrity Constraint

Example Conditional plus Derivation

In summary, the subject matter experts were comfortable with an IF … Then … construct, replacing the rules expression within this with English.  They didn't care whether a rule should have been "…Then Accept" or "… Then perform" (etc.), so only two constructs were meaningful:  If expression Then  and Fact = expression.

This implies that business people are not interested in, and there is no reason to expose them to, business rules analysis techniques like categorisation.

6. Fully-documented rules are copied back into the business rules repository.

This makes the rules available to be listed in stand-alone business rule documents or in use case specifications (as references or with complete text).

The client asked to see the full business rule text in any use case the rule was referenced by.  This did not increase the amount of effort but made the use case documents unnecessarily large.  In the future, I would create a range of views of the requirements and rules.  This could be, for example, a version of the use case report with only a list of related business rule numbers and text (and possibly not even this) for the development team and, for the wider client group, a version of the use case reports that includes only the description, business rule, and user interface mockup sections.

7. Developers translate rules into rules engine syntax and hook these into appropriate reference points, based on use case documentation.

In the end, some of these were implemented as stored procedures, particularly where it made sense to return a list.  These will always require developer skills to change, but at least they are coded only once.  Most of the business rules identified, as well as a number of system rules, were coded into the business rules engine.

4  Summary of Findings

• With a low-end business rules engine, rules are not able to be maintained by business users, but there is still value in isolating conditional code, due to:
  • ease of development (each rule is developed in one place and re-used wherever necessary),
  • ease of maintenance (each rule is maintained in one place without the need to touch code), and
  • the ability to effective-date the implementation of business rule changes in the production environment.
    
    
• Not all business rules are implemented using the business rules engine (some become stored procedures) and not all logic implemented using the business rules engine are business rules (for example, screen flow rules).
  
  
• Use of a business rules engine introduces a third change control category.  Now there is:
  1. Static data change — often performed by system administrator,
  2. Functional change — requiring expensive code change and test,
  3. Business rule change — requiring less expensive, and faster, business rule change via the business rules engine user interface.
     
     
• Use of a business rules engine requires identification and careful documentation of rules — therefore, business rules analysis is important.
  
  
• During software requirements elicitation and analysis, if business rules have not already been captured, categorisation can help to focus analysis on the critical business rules and to test requirements in order to find embedded rules.
  
  
• Business rules analysis allows rules to be separated into a central repository and this avoids duplication.
  
  
• Use cases help to locate business rule reference points (aka decision points) within user–system interactions (i.e., workflow).
  
  
• Use cases help to ensure coverage in terms of business rules selection, whether  business rules are harvested during business modelling or during detailed software requirements elicitation.
  
  
• Appropriate skeleton formats for rules allow users to document rules in a manner that reduces the translation required to write business rules engine syntax.  This means that specification of rules can be distributed to the people who are in the best position to perform this work.  This, in turn, improves the degree of accuracy of the software solution, with regard to business needs.
  
  
• Rules clarity should not be traded for precision.  It is better to leave rules in English with inherent ambiguity than to re-write in pseudo code in order to gain precision, at least until translation into business rules engine syntax.

5  Conclusions

The success of the project in terms of the five key advantages of the Business Rules Approach can be summarised as follows:

• Agility — Unknown.  This is yet to be tested as the system is still in User Acceptance Test.  Prior to deployment, there will be a discussion with the client regarding who will update the business rules.  Because the selected rules engine exposes the business rules administrator to system classes and attributes and a "technical" syntax, it is unlikely that business users will be able to update rules directly.  It is hoped that one or more of the client's "power users" will be able to perform this role, however.

Some effort has already been saved due to the approach of documenting and coding rules in one place.  Some rules have already required updates during development (due to change requests), and these were significantly easier to make than changes to the main code base.

• Consistency — Yes.  A common business process was identified and implemented for different application types by separating business rules (particularly for data capture, assessment, amendment, and invoicing).
  
  
• Decision Making (support) — Yes.  This was especially the case with the complex guidance that was now able to be provided to grants officers during application eligibility assessment, through the use of the rules engine.
  
  
• Compliance — Yes.  Particularly in the key business activities of application assessment and review, the system will guide appropriate eligibility decisions.  Also, when providers are assigned to cases, business rules prevent incorrect assignments — i.e., assigning solicitors or barristers who do not have appropriate approvals and experience levels to cases.  Rules are also used to drive compliance with the claimable hours guidelines related to provider services.
  
  
• Transparency — Yes.  It is now possible, if required, to publish the list of rules that have been implemented in the system, as a separate list from functional requirements.  In addition, a business rules user interface is available to administrators, to review the actual "code" of those rules implemented in the system.

These outcomes are in large part due to the use of a business rules engine, but the results would have been difficult to achieve without the adoption of a specialised approach to the elicitation, analysis, and specification of business rules.  The application of a set of business rules categories was a key ingredient in this approach to business rules analysis.

References

[1]  Semantics of Business Vocabulary and Business Rules, (2005), accepted for finalisation stage of standardisation by OMG.  

[2]  Business Rules Group, Defining Business Rules ~ What Are They Really? 4th ed., July 2002.  Originally published as the GUIDE Business Rules Project Report, 1995.  Available at http://www.BusinessRulesGroup.org.  

[3]  IIBA Business Analysis Body of Knowledge, version 1.6.  

# # #