May 28, 2011 by mensming.

Latency: Why You Should Worry and What Up You Can Do About It
Philip Tellis, Yahoo! Inc.
2010 Velocity Conference
June 22-24, 2010
(15 min, 40 sec)

Here are my notes from watching the presentation:

• Bandwidth is easy - It is not the problem…
• Bandwidth vs. Latency - How bad is it?
• More resources on a page, latency more of a problem
• It’s Still the Latency, Stupid http://www.stuartcheshire.org/rants/latency.html
• Javascript adds latency

What to Do About Latency?

• How did CPUs solve latency? Cache, Parallelise, Predict
• These apply to the web as well…
• Pre-fetch content is a way to predict

How to measure?

• Announcing Boomerang http://github.com/yahoo/boomerang
  
• Measures page load time from the unload of the last page to this page is loaded
• Measures bandwidth
• Measures latency
• Include a script on the page, include a beacon on your server to send the data…
• BSD license

Posted in performance, test tools, conferences | 1 Comment »

March 19, 2011 by mensming.

During my day to day testing, I often need to generate large blocks of text. I often find the text of such public domain classics like Moby Dick or The Adventures of Tom Sawyer. Then there is the old stand by - Lorem Ipsum. The site www.lipsum.com is a useful site to generate different lengths of this text.

My new favorite site for this type of work is www.blindtextgenerator.com. Not only does it provide lorem ipsum it provide the following additional options: cicero (latin and english), Li Europan lingues (Occidental - an international auxiliary language and english), Far far away, Werther, Kafka, Pangrams, a-z A-Z 123.

Check it out.

Posted in test tools | No Comments »

September 18, 2010 by mensming.

I must admit, I have not done much with JUnit 4 - even though it has been out for a very long time. I came across a blog post by John Ferguson Smart entitled Testing Exceptions in JUnit 4.7. This is a great tutorial on how to JUnit 4 is evolving to make testing thrown exceptions easier.

Not only is this a good tutorial, it has inspired me to take a closer look at JUnit 4.

Posted in unit testing, test tools | No Comments »

September 11, 2010 by mensming.

A couple of weeks ago, I realized I needed to combine multiple testng-results.xml files into to do some consolidated reporting. I did some research and came up with a solution that works for me. Recently, the testng-users Google group had a thread that asked the same thing. This post explains how I did it - not necessarily the best way but works for me now.

Basically, I use an XSLT stylesheet to merge the various XML files into a single file. As you will see below, the solution I used only merges 2 files at a time so I create a loop that merges each file into a master file. Let’s start with the master ant task:


...
<taskdef resource="net/sf/antcontrib/antlib.xml"/>
...
<target name="merge-results">
    <delete dir="${report.dir}/testng_summary_report"/>
    <delete file="${report.dir}/start-merge.xml" />
    <delete file="${report.dir}/testng-merge.xml" />

    <mkdir dir="${report.dir}/testng_summary_report"/>
    <copy
        file="empty-testng-results.xml"
        tofile="${report.dir}/testng-merge.xml" />

    <for param="xmlFile">
        <path>
            <fileset dir="${report.dir}" >
                <include name="**/testng-results.xml" />
            </fileset>
        </path>


        <sequential>

            <copy
                file="${report.dir}/testng-merge.xml"
                toFile="${report.dir}/start-merge.xml" />
                <xslt style="testng-merge.xsl"
                    destdir="${report.dir}"
                    in="${report.dir}/start-merge.xml"
                    out="${report.dir}/testng-merge.xml">

                    <param name="with" expression="@{xmlFile}" />
                </xslt>
            <delete file="${report.dir}/start-merge.xml" />
        </sequential>
    </for>
</target>



After some initial cleanup, the ant task copies the "starting" XML file (empty-testng-results.xml). It looks like this:


<?xml version="1.0" encoding="UTF-8"?>
<testng-results>
    <reporter-output>
    </reporter-output>
</testng-results>


The task then uses the antlib for command to loop through all of the testng-results.xml files and applies the XSLT transform to each in turn. I did not write the XSLT stylesheet. I found it at http://www2.informatik.hu-berlin.de/~obecker/XSLT/merge/merge.xslt.html. For convenience, I include it here:


<!--
   Merging two XML files
   Version 1.6
   LGPL (c) Oliver Becker, 2002-07-05
   obecker@informatik.hu-berlin.de
-->
<xslt:transform
    xmlns:xslt="http://www.w3.org/1999/XSL/Transform"
    xmlns:m="http://informatik.hu-berlin.de/merge"
    version="1.0"
    exclude-result-prefixes="m">

<!– Normalize the contents of text, comment, and processing-instruction
     nodes before comparing?
     Default: yes –>
<xslt:param name="normalize" select="’yes’" />

<!– Don’t merge elements with this (qualified) name –>
<xslt:param name="dontmerge" />

<!– If set to true, text nodes in file1 will be replaced –>
<xslt:param name="replace" select="false()" />

<!– Variant 1: Source document looks like
     <?xml version="1.0"?>
     <merge xmlns="http://informatik.hu-berlin.de/merge">
        <file1>file1.xml</file1>
        <file2>file2.xml</file2>
     </merge>         
     The transformation sheet merges file1.xml and file2.xml.
–>
<xslt:template match="m:merge">
   <xslt:variable name="file1" select="string(m:file1)" />
   <xslt:variable name="file2" select="string(m:file2)" />
   <xslt:message>
      <xslt:text />Merging ’<xslt:value-of select="$file1" />
      <xslt:text />’ and ’<xslt:value-of select="$file2" />’<xslt:text />
   </xslt:message>
   <xslt:if test="$file1=” or $file2=”">
      <xslt:message terminate="yes">
         <xslt:text>No files to merge specified</xslt:text>
      </xslt:message>
   </xslt:if>
   <xslt:call-template name="m:merge">
      <xslt:with-param name="nodes1" select="document($file1,/*)/node()" />
      <xslt:with-param name="nodes2" select="document($file2,/*)/node()" />
   </xslt:call-template>
</xslt:template>

<!– Variant 2:
     The transformation sheet merges the source document with the
     document provided by the parameter "with".
–>
<xslt:param name="with" />

<xslt:template match="*">
   <xslt:message>
      <xslt:text />Merging input with ’<xslt:value-of select="$with" />
      <xslt:text>’</xslt:text>
   </xslt:message>
   <xslt:if test="string($with)=”">
      <xslt:message terminate="yes">
         <xslt:text>No input file specified (parameter ’with’)</xslt:text>
      </xslt:message>
   </xslt:if>

   <xslt:call-template name="m:merge">
      <xslt:with-param name="nodes1" select="/node()" />
      <xslt:with-param name="nodes2" select="document($with,/*)/node()" />
   </xslt:call-template>
</xslt:template>

<!– ============================================================== –>

<!– The "merge" template –>
<xslt:template name="m:merge">
   <xslt:param name="nodes1" />
   <xslt:param name="nodes2" />

   <xslt:choose>
      <!– Is $nodes1 resp. $nodes2 empty? –>
      <xslt:when test="count($nodes1)=0">
         <xslt:copy-of select="$nodes2" />
      </xslt:when>
      <xslt:when test="count($nodes2)=0">
         <xslt:copy-of select="$nodes1" />
      </xslt:when>

      <xslt:otherwise>
         <!– Split $nodes1 and $nodes2 –>
         <xslt:variable name="first1" select="$nodes1[1]" />
         <xslt:variable name="rest1" select="$nodes1[position()!=1]" />
         <xslt:variable name="first2" select="$nodes2[1]" />
         <xslt:variable name="rest2" select="$nodes2[position()!=1]" />
         <!– Determine type of node $first1 –>
         <xslt:variable name="type1">
            <xslt:apply-templates mode="m:detect-type" select="$first1" />
         </xslt:variable>

         <!– Compare $first1 and $first2 –>
         <xslt:variable name="diff-first">
            <xslt:call-template name="m:compare-nodes">
               <xslt:with-param name="node1" select="$first1" />
               <xslt:with-param name="node2" select="$first2" />
            </xslt:call-template>
         </xslt:variable>

         <xslt:choose>
            <!– $first1 != $first2 –>
            <xslt:when test="$diff-first=’!'">
               <!– Compare $first1 and $rest2 –>
               <xslt:variable name="diff-rest">
                  <xslt:for-each select="$rest2">
                     <xslt:call-template name="m:compare-nodes">
                        <xslt:with-param name="node1" select="$first1" />
                        <xslt:with-param name="node2" select="." />
                     </xslt:call-template>
                  </xslt:for-each>
               </xslt:variable>
      
               <xslt:choose>
                  <!– $first1 is in $rest2 and 
                       $first1 is *not* an empty text node  –>
                  <xslt:when test="contains($diff-rest,’=') and not($type1=’text’ and normalize-space($first1)=”)">
                     <!– determine position of $first1 in $nodes2
                          and copy all preceding nodes of $nodes2 –>
                     <xslt:variable name="pos" select="string-length(substring-before( $diff-rest,’=')) + 2" />
                     <xslt:copy-of select="$nodes2[position() &lt; $pos]" />
                     <!– merge $first1 with its equivalent node –>
                     <xslt:choose>
                        <!– Elements: merge –>
                        <xslt:when test="$type1=’element’">
                           <xslt:element name="{name($first1)}" namespace="{namespace-uri($first1)}">
                              <xslt:copy-of select="$first1/namespace::*" />
                              <xslt:copy-of select="$first2/namespace::*" />
                              <xslt:copy-of select="$first1/@*" />
                              <xslt:call-template name="m:merge">
                                 <xslt:with-param name="nodes1" select="$first1/node()" />
                                 <xslt:with-param name="nodes2" select="$nodes2[position()=$pos]/node()" />
                              </xslt:call-template>
                           </xslt:element>
                        </xslt:when>
                        <!– Other: copy –>
                        <xslt:otherwise>
                           <xslt:copy-of select="$first1" />
                        </xslt:otherwise>
                     </xslt:choose>
      
                     <!– Merge $rest1 and rest of $nodes2 –>
                     <xslt:call-template name="m:merge">
                        <xslt:with-param name="nodes1" select="$rest1" />
                        <xslt:with-param name="nodes2" select="$nodes2[position() &gt; $pos]" />
                     </xslt:call-template>
                  </xslt:when>

                  <!– $first1 is a text node and replace mode was
                       activated –>
                  <xslt:when test="$type1=’text’ and $replace">
                     <xslt:call-template name="m:merge">
                        <xslt:with-param name="nodes1" select="$rest1" />
                        <xslt:with-param name="nodes2" select="$nodes2" />
                     </xslt:call-template>
                  </xslt:when>

                  <!– else: $first1 is not in $rest2 or
                       $first1 is an empty text node –>
                  <xslt:otherwise>
                     <xslt:copy-of select="$first1" />
                     <xslt:call-template name="m:merge">
                        <xslt:with-param name="nodes1" select="$rest1" />
                        <xslt:with-param name="nodes2" select="$nodes2" />
                     </xslt:call-template>
                  </xslt:otherwise>
               </xslt:choose>
            </xslt:when>

            <!– else: $first1 = $first2 –>
            <xslt:otherwise>
               <xslt:choose>
                  <!– Elements: merge –>
                  <xslt:when test="$type1=’element’">
                     <xslt:element name="{name($first1)}" namespace="{namespace-uri($first1)}">
                        <xslt:copy-of select="$first1/namespace::*" />
                        <xslt:copy-of select="$first2/namespace::*" />
                        <xslt:copy-of select="$first1/@*" />
                        <xslt:call-template name="m:merge">
                           <xslt:with-param name="nodes1" select="$first1/node()" />
                           <xslt:with-param name="nodes2" select="$first2/node()" />
                        </xslt:call-template>
                     </xslt:element>
                  </xslt:when>
                  <!– Other: copy –>
                  <xslt:otherwise>
                     <xslt:copy-of select="$first1" />
                  </xslt:otherwise>
               </xslt:choose>

               <!– Merge $rest1 and $rest2 –>
               <xslt:call-template name="m:merge">
                  <xslt:with-param name="nodes1" select="$rest1" />
                  <xslt:with-param name="nodes2" select="$rest2" />
               </xslt:call-template>
            </xslt:otherwise>
         </xslt:choose>
      </xslt:otherwise>
   </xslt:choose>
</xslt:template>

<!– Comparing single nodes: 
     if $node1 and $node2 are equivalent then the template creates a 
     text node "=" otherwise a text node "!" –>
<xslt:template name="m:compare-nodes">
   <xslt:param name="node1" />
   <xslt:param name="node2" />
   <xslt:variable name="type1">
      <xslt:apply-templates mode="m:detect-type" select="$node1" />
   </xslt:variable>
   <xslt:variable name="type2">
      <xslt:apply-templates mode="m:detect-type" select="$node2" />
   </xslt:variable>

   <xslt:choose>
      <!– Are $node1 and $node2 element nodes with the same name? –>
      <xslt:when test="$type1=’element’ and $type2=’element’ and local-name($node1)=local-name($node2) and namespace-uri($node1)=namespace-uri($node2) and name($node1)!=$dontmerge and name($node2)!=$dontmerge">
         <!– Comparing the attributes –>
         <xslt:variable name="diff-att">
            <!– same number … –>
            <xslt:if test="count($node1/@*)!=count($node2/@*)">.</xslt:if>
            <!– … and same name/content –>
            <xslt:for-each select="$node1/@*">
               <xslt:if test="not($node2/@* [local-name()=local-name(current()) and namespace-uri()=namespace-uri(current()) and .=current()])">.</xslt:if>
            </xslt:for-each>
         </xslt:variable>
         <xslt:choose>
            <xslt:when test="string-length($diff-att)!=0">!</xslt:when>
            <xslt:otherwise>=</xslt:otherwise>
         </xslt:choose>
      </xslt:when>

      <!– Other nodes: test for the same type and content –>
      <xslt:when test="$type1!=’element’ and $type1=$type2 and name($node1)=name($node2) and ($node1=$node2 or ($normalize=’yes’ and normalize-space($node1)= normalize-space($node2)))">=</xslt:when>

      <!– Otherwise: different node types or different name/content –>
      <xslt:otherwise>!</xslt:otherwise>
   </xslt:choose>
</xslt:template>

<!– Type detection, thanks to M. H. Kay –>
<xslt:template match="*" mode="m:detect-type">element</xslt:template>
<xslt:template match="text()" mode="m:detect-type">text</xslt:template>
<xslt:template match="comment()" mode="m:detect-type">comment</xslt:template>
<xslt:template match="processing-instruction()" mode="m:detect-type">pi</xslt:template>

</xslt:transform>


At the end of this process, the merged results will be found in a file named testng-merge.xml.

Posted in test tools | No Comments »

June 21, 2010 by mensming.

One of the neat things about TestNG is the ability to annotate individual tests as belonging to zero or more groups. At execution time, it is possible to tell the test runner to run all tests in a particular group. In addition, tests in certain groups can be excluded so you can run all tests in a particular group except for those that are marked as broken.

The Apache Harmony project apparently uses TestNG. On their wiki, they describe their testing conventions for using groups. They have groups to identify tests that are operating system specific, processor architecture specific, feature specific, environment requirements, current state and test type. Worth checking out.

Posted in test tools | No Comments »

May 29, 2010 by mensming.

I have recently started to learn TestNG. There are two reasons for this. The first was an intriguing discussion I had with a test engineer candidate about a Selenium based system he had built on top of TestNG. More recently, a colleague suggested we convert from JUnit to TestNG for our Selenium tests to better allow us to use Selenium Grid.

My initial impressions are positive. However, I am just beginning. This is my initial set of reference materials:

• The TestNG web site and references on the web site
• The testng-users Google Group
• Next Generation Java Testing: TestNG and Advanced Concepts
• StackOverflow questions related to TestNG

Posted in unit testing, test tools | No Comments »

February 20, 2010 by mensming.

Software Engineering Radio some time ago had a great podcast covering the basics of continuous integration with Chris Read. Highly recommended for both the beginner and the experienced needing a reminder of the basics. Recommended. The episode can be found here.

Posted in process improvement, test tools | No Comments »

January 30, 2010 by mensming.

This is another in a series of posts of my reactions to watching the videos from the Google Test Automation Conference 2008 held in Seattle. The talk The Value of Small Tests (38 minutes long) presented by Christopher Semturs on October 24, 2008 was not what I expected. The slides from the talk can be found here.

I was hoping for a discussion around the broader test case / test suite management and the intersection with test automation. Instead, it discussed the value of creating small tests with limited dependencies and using dependency injection, mocking, etc. Not to say that it is a bad talk - just a quick introduction for these topics. I would say the talk is aimed at getting developers writing and enabling automated testing.

If you are already familiar with dependency injection and mocking, I would not spend the time watching. However, if these concepts are new, it may be valuable.

Posted in test tools, conferences | No Comments »

January 23, 2010 by mensming.

Even if you are not working on the Microsoft platform, the December 2009 MSDN Magazine has some articles which can be of use to all testers.

In particular, Pairwise Testing with QICT by James McCaffrey provides enough insight into the QICT tool that it can be easily ported to any platform. While Automated Unit Tests for Legacy Code with Pex by Nikhil Sachdeva is specific to Microsoft technologies, Pex is a tool that all testers should become familiar. Finally, Using Agile Techniques to Pay Back Technical Debt by David Laribee has tips that are useful to most test organizations.

Posted in professional development, process improvement, test tools | No Comments »

January 16, 2010 by mensming.

I am in the process of watching the videos from the GTAC (Google Test Automation Conference) held in Seattle in October 2003. This post contains my notes for the presentation Taming the Beast - How to Test an AJAX Application (1 hour 1 minute) by Markus Clermont & John Thomas held on October 23, 2008.

I am not recommending spending time watching this video. The presentation is not bad. However, it does not translate well to video. There is a lot of interaction with the audience - which is great when you are present at the presentation. However, in the video the audience cannot be heard and the presenters do not repeat many of the questions / responses. In addition, the talk is misnamed. While an AJAX application is used as the example throughout the application, most of the talk is not about how to test an AJAX application. The talk is a good discussion of how to tame automation - automation of any application.

Here are the slide titles:

• AJAX: A Different Beast
• An Example GWT Application - (GWT = Taming the Beast - How to Test an AJAX Application)
• Some Statistics - (At this point there are questions to the audience regarding their automated testing experience.)
• System Architecture
• Small Medium Large - (Essentially how much of the application is under test)
• Testing Layer Pairs
• Are we done?
• Comparison
• Conclusion

Posted in training, test tools, web testing, conferences | No Comments »