Maintainable JMeter Scenarios

One of the problems I have with JMeter is that its scenarios are hard to maintain.

I have already found that I can introduce variables to parametrize a scenario but it is kind of cumbersome - when application changes a bit I had to re-run the scenario in Firefox with Firebug, hunt for parameters' values and set their new values.

This approach is slow and prone to errors. Definitely no way for my colleagues that do not know JMeter at all or only a little.

Recently I have had to solve it and I have come with following solution - let's transform the original scenario to a template and fill it with data gathered by some transparent proxy during manual re-play of the scenario.


Recording The Session

I found Paros proxy (or its fork Andiparos) to be a good match for my needs, although the proxy is made for vulnerability testing of web applications. To record a session follow these steps:

1. Run Paros on your workstation.
   
   1. Open Tools - Options and set:
      1. Local Proxy - domain to localhost, port to 9009  
      2. Trap - check "Use exclusive filter" and insert pattern to exclude unwanted requests : .*\.(png|js|css)
   2. Open Tools - Filter and check "Log request and reponse into file (filter/messages)"
      
      
2. Run Firefox :
   

   firefox -P -no-remote

   This will open Firefox Profile manager. Create a new profile, named e.g. test. Run Firefox with this profile and set manual HTTP proxy to the values set previously in Paros.
   
   
3. In the browser go through your scenario. You can close the browser after that.
   
4. Get recorded session from Paros directory - messages.txt file.

 The Template

When you have the recorded session, open you JMeter scenario - a file with jmx suffix - and replace all parameter values with custom place holders. The file contains XML, here is a part of it with %HOST% and %PORT% place holders :

<ConfigTestElement guiclass="HttpDefaultsGui" testclass="ConfigTestElement" 
                   testname="HTTP Request Defaults" enabled="true">
  <elementProp name="HTTPsampler.Arguments" elementType="Arguments" 
               guiclass="HTTPArgumentsPanel" testclass="Arguments" 
               testname="User Defined Variables" enabled="true">
     <collectionProp name="Arguments.arguments"/>
   </elementProp>
   <stringProp name="HTTPSampler.domain">%HOST%</stringProp>
   <stringProp name="HTTPSampler.port">%PORT%</stringProp>
</ConfigTestElement>

The last thing you need is a Python script to extract the parameters' values and fill the template with them:

#!/usr/bin/python

import sys
import argparse
import string
import re

if __name__ == '__main__':

    host_pattern = re.compile("^Host: (.*)")
    host_value = None

    # -------------- get cmdline args -----------------
    parser = argparse.ArgumentParser(prog='jmfiller')
    parser.add_argument('session_record_file', help='session record file')
    parser.add_argument('template_file', help='template file')

    args = parser.parse_args()
 
    session_record_file_name = args.session_record_file
    template_file_name = args.template_file

    # -------------- collect data ----------------------
    for line in open(session_record_file_name,'r'):
       # get HOST    
       if host_value is None:
          match = host_pattern.search(line)
          if match is not None: host_value = match.group(1)

    print  >> sys.stderr, "HOST:", host_value

    # -------------- insert in the template ------------

    host_placeholder                    = "%HOST%"

    for line in open(template_file_name,'r'):       
        output_line = line
        output_line = string.replace(output_line, host_placeholder, host_value)

        print output_line

Moving to Selenium 2 on WebDriver, Part No.4

Conditional Waits

Majority part of web UI tests I fixed and keep fixing has the same flaw - there is either no wait for dynamically loaded parts of page or the wait is unconditional. As driver.get() is blocking this is clearly problem of an AJAX calls.

WebDriver provides two APIs for conditional waits - com.thoughtworks.selenium.Wait and org.openqa.selenium.support.ui.Wait.


Class com.thoughtworks.selenium.Wait

Simpler of the two and IMHO better suited for ad-hoc waits is this abstract class. It has only two methods - wait() and until().  You have to implement method until() and it should return true when the condition is met.  

Example of wait for pop-up of given name and switching to it:

public void waitForPopUp(final String windowName, int timeoutMs)
{
   new Wait()
   {
      @Override
      public boolean until()
      {
         try
         {
            driver.switchTo().window(windowName);
            return true;
         }
         catch (SeleniumException ignored) { }
         return false;
      }
   }.wait(String.format("Timed out waiting for %s. Waited %s",windowName, timeoutMs), timeoutMs);
}

Interfaces and classes from org.openqa.selenium.support.ui

More sophisticated approach is represented by interfaces Wait, implemented by WebDriverWait class, and  ExpectedCondition (extending com.google.common.base). Class implementing the ExpectedCondition interface must define method apply() returning true or not-null when a condition is met.

Example of wait expecting JavaScript expression evaluate to true:

int timeoutSeconds = 10;
Wait wait = new WebDriverWait(driver, timeoutSeconds);
JavascriptExecutor js = (JavascriptExecutor)driver;

public boolean waitForJsCondition(final String script, String timeoutMessage)
{
    boolean result = false;

    try
    {
        result = wait.until
        (
            new ExpectedCondition<Boolean>()
            {
               public Boolean apply(WebDriver driver)
               { return (Boolean)js.executeScript("return " +  script); }
            }
        );
    }
    catch (Throwable t)
    { handleWaitTimeout(t,timeoutMessage); }

    return result;
}

As you can see the advantage is that this Wait returns value. You can wait for an element to become visible and return it, thus keeping you code cleaner and more resilient. Even better, expected conditions returning the same type are interchangable. That not only adds to flexibility, it also helps to fight code duplication.