Fitting Acceptance Tests with DSLs

Reading Ayende's post about Fit prompts me to say that I, too, haven't been overly pleased with the Fit/FitNesse experience in the .NET world; Java maybe a different story...The Fit book was/is excellent at demonstrating what can be done with Fit and why it's valuable to have customer driven/written acceptance tests, but the framework just doesn't seem to have matured enough on the .NET framework.

I've started writing a DSL for acceptance testing based on the ideas presented by Dan North on his blog which I found while looking at Behave#. I believe the Behave# project has good intentions, but the idea is too low-level; customers, business analysts, etc. aren't going to understand what's going on there, especially when asked to use anonymous methods.

Dan North's ideas seem to provide a good context for establishing boundaries around what constitutes a given function of the application, how it is expected to be used/work, and how to verify the application behaved as expected given a number of situations (or scenarios, test cases, what-have-you). And, perhaps most importantly, the language he and his team uses as a template for story writing provides a solid foundation as the basis for an extensible, customer-friendly, acceptance testing DSL. Let's take a look at one of the examples Dan provides:

Story: Account Holder withdraws cash

As an Account Holder

I want to withdraw cash from an ATM

So that I can get money when the bank is closed


Scenario 1: Account has sufficient funds

Given the account balance is $100

And the card is valid

And the machine contains enough money

When the Account Holder requests $20

Then the ATM should dispense $20

And the account balance should be $80

And the card should be returned


Scenario 2: Account has insufficient funds

Given the account balance is $10

And the card is valid

And the machine contains enough money

When the Account Holder requests $20

Then the ATM should not dispense any money

And the ATM should say there are insufficient funds

And the account balance should be $20

And the card should be returned


Scenario 3: Card has been disabled

Given the card is disabled

When the Account Holder requests $20

Then the ATM should retain the card

And the ATM should say the card has been retained


Scenario 4: The ATM has insufficient funds

…

The Story heading presents the title of the story/functionality, followed by three important pieces: the role, desired functionality, and the reason why the functionality is important. These are represented by the As a[n], I want, So that prefixed statements. While it is arguable whether these statements represent any valuable testing information for developers and/or testers, it does provide them with important contextual and business-need information about the desired functionality. From there, each scenario describes the background/"setup" context (the Givens), the action or functionality to execute (the When), and the desired outcome (the Thens).

Given these ideas, here's a DSL that could accomplish this and also yield us (the developers) automated test fixtures for verification during continuous integration cycles:

Story "Account Holder withdraws cash":
    Role "Account Holder"
    Feature "Withdraw cash from ATM"
    Benefit "Access money when the bank is closed"

    Scenario "Account has sufficient funds":
       Given:
          That "The account balance is $100":
            CreateAccount "012324", 100
          And "The card is valid":
            AddATMCardToAccount "012324", "32123456879", Valid
          And "The machine contains enough money":
            SetupAtm 10000
       When "The Account Holder Requests $20":
          UseATM "32123456789", Withdrawal, 20
       Verify:
          That "The ATM should dispense $20":
            VerifyATM LastTransaction, Withdrawal 20
          And "The account balance should be $80":
            VerifyAccount "012324", Balance 80
          And "The card should be returned":
            VerifyATM State NoCardPresent

What if the DSL processor produced the following:

[TestFixture]
public class StoryFixture_Account_Holder_withdraws_cash
{
[Test]
public void Scenario_Account_has_sufficient_funds()
{
  Account a = new Account("012324", 100);
  a.AddAtmCard(new AtmCard("32123456879", AtmCardStatus.Valid));
  AutomatedTeller m = new AutomatedTeller(10000);

  m.Withdrawl(a.AtmCard, 20);

  Assert.AreEqual(Operation.Withdrawl, m.LastTransaction.Operation);
  Assert.AreEqual(20, m.LastTransaction.Amount);
  Assert.AreEqual(80, a.Balance);
  Assert.AreEqual(AtmState.NoCardPresent, m.State);
}
}

Not all of the syntactic sugar has been worked out here, but I think the idea of the foundation DSL extended with the application specific language is clear. While this is better than having to write lower-level C# with anonymous methods, I think users - as in customers, business analysts, etc. - would be better served with a UI that provided them the necessary "guidance", if you will, to create their scenarios as necessary. Something like:

Would then output the DSL file. Now note that none of that UI works, it's all just show right now and very little of the DSL parsing is functional as of yet. With more time, I hope to have this publicly available for contributions, etc. if there's interest.