January 16, 2010 by mensming.

I am in the process of watching the videos from the GTAC (Google Test Automation Conference) held in Seattle in October 2003. This post contains my notes for the presentation Taming the Beast - How to Test an AJAX Application (1 hour 1 minute) by Markus Clermont & John Thomas held on October 23, 2008.

I am not recommending spending time watching this video. The presentation is not bad. However, it does not translate well to video. There is a lot of interaction with the audience - which is great when you are present at the presentation. However, in the video the audience cannot be heard and the presenters do not repeat many of the questions / responses. In addition, the talk is misnamed. While an AJAX application is used as the example throughout the application, most of the talk is not about how to test an AJAX application. The talk is a good discussion of how to tame automation - automation of any application.

Here are the slide titles:

• AJAX: A Different Beast
• An Example GWT Application - (GWT = Taming the Beast - How to Test an AJAX Application)
• Some Statistics - (At this point there are questions to the audience regarding their automated testing experience.)
• System Architecture
• Small Medium Large - (Essentially how much of the application is under test)
• Testing Layer Pairs
• Are we done?
• Comparison
• Conclusion

Posted in training, test tools, web testing, conferences | No Comments »

December 20, 2009 by mensming.

A little while ago, I needed to run some Selenium processes against a different server which was https only with a self signed certificate. When Selenium fired up, I saw the SSL certificate warnings followed by the test failing. Even when I inserted a pause into my script so I would have time to add the security exception, the test still failed.

A search on the web led to Elliot Smith’s post entitled Dealing with self-signed SSL certificates when running Selenium server with Firefox. At the time, the post was only a few days old. It was exactly what I needed and worked like a champ. Thank you Elliot!

I would add to Elliot’s procedure that the cert_override.txt and cert8.db files can be added to source control. Even if the person executing the tests has never followed the procedure regarding creating a Firefox profile, the process will still work.

Finally, a minor warning. If you find that you need to add additional self-signed certificates, you will need to do that as part of a full-fledged Firefox profile - not the limited version created here. In that case, just like the first time, you will need to delete all of the Firefox created files and directories except cert_override.txt and cert8.db just like the first time it was setup. (I suppose you could also copy the 2 cert*.* files from a different full fledged profile - as long as you were confident regarding the contents of those files.)

Posted in test tools, web testing | No Comments »

May 4, 2009 by mensming.

Screen capture tools are required for any tester working on a system with a UI. Lifehacker recently asked their readers to nominate the 5 best screen capture tools. The results can be found here.

Here is the list of winning applications:

1. Snipping Tool (Windows)
2. FastStone Capture (Windows)
3. Jing (Windows/Mac)
4. Skitch (Mac)
5. Snagit (Windows)

Posted in test tools | No Comments »

April 20, 2009 by mensming.

I just got this in my inbox — quite a deal if you are looking to tryout JIRA or Confluence.

For this week only, we’re offering a special 5-user “starter” license of JIRA and Confluence for only $5 each. We’re calling it the Atlassian Stimulus Package and it’s our way of supporting small teams and small businesses in this difficult economic environment. Best of all, we’re going to donate every penny to charity, so please help us spread the word!

The Atlassian Foundation is donating all proceeds to Room to Read, a charity that helps the world’s future entrepreneurs by building libraries and schools for children in developing nations.

Get all the details at www.atlassian.com/starter. Hurry, offer ends on 24 April 2009.

Posted in IT, test tools, jira, bug tracking | No Comments »

February 7, 2009 by mensming.

While working through my backlog of reading, I came across the results of Software Test & Performance magazine’s Testers Choice Awards for 2008. This list is of no use to me.

First, look at most of the winners - LoadRunner, TestDirector, components of IBM Rational suite and other high priced, commercial products. Working at a startup now (and several in the past), there is rarely budget for these tools. Oftentimes, the tools imply a process or a level of maturity that do not exist in the first couple of years of a startup’s existence. A more useful breakdown would be a set of testers choice awards for open source tools and a set of testers choice awards for commercial tools. (There is a category for free test/performance tool but that is about as broad as giving an award for best vertebrate - there is just too much variation.)

Which is actually the second point, because of the diversity of the software development space, a generic list such as this does not shed light on tools that may be applicable to my particular area. This is similar to an example James Surowiecki used in The Wisdom of Crowds regarding the generic top 10 lists in music. There may be only 1 or no songs in the top 10 list for the genre I am interested in and most likely I am already familiar with the song. If I am interested in that genre, the generic top 10 list only tangentially touches my interests and does not supply any new information. If I am looking for new songs like the one I like in the generic list, the other entries may not help. However, a top 10 list of songs in the specific music type (the more specific the better) may lead me to new songs which I will enjoy. The same is true when it comes to tools. A “testers choice award for java code coverage tools” might point me to tools I am not familiar with while the category “java test/performance” does not point me to any tools that I am not already familiar. Knowing which code coverage tool for java others consider the best is much more useful than knowing LoadRunner won the java test/performance category for 2008.

Posted in test tools | No Comments »

December 14, 2008 by mensming.

While it has been a while since I needed to do installer testing, I recently researched a product that may be helpul in this task. Sandboxie strives to isolate programs from making changes to the rest of the operating system. The Sandboxie website describes it as:

Sandboxie runs your programs in an isolated space which prevents them from making permanent changes to other programs and data in your computer.

and lists as a benefit:

Windows Stays Lean: Prevent wear-and-tear in Windows by installing software into an isolated sandbox.

Sandboxie adds a third approach to installer testing. The first approach, to install over and over on the same instance, is expediant but not necessarily the most realistic. The second approach is to re-image / restore the system. When I first started, this meant using a program such as Ghost, now owned by Symantec. This approach has been supplanted using virtual machines. Virtual machines have made this process much faster.

Using Sandboxie, it is possible to have the cleanliness afforded by virtual machine or images plus the expediency of installing over and over on the same system. On a single system, multiple sandboxes could be created - one for each install test. This approach seems reasonable to me. It is still necessary to do installer testing on native system, but this can be done after many bugs our found and resolved.

Pricing is very reasonable for a commercial license. Check it out.

Posted in test tools, system testing | No Comments »

April 30, 2008 by mensming.

A lot of developers I have worked with have been amazed to see that I expand a suite of tests they developed by a factor of four. This is almost always because I am stressing error handling which is often not tested at all. Unfortunately, these tests will often cause many warnings. Sometimes I want to verify the exact warning. Other times it just clutters the output being sent to the screen. This can be very dangerous - it may lead someone running the test to believe there is a newly introduced bug.

Luckily, perl has a well defined mechanism for overriding the warn behavior. If you want to suppress warnings from being displayed in a section of tests do the following:

Once the scoping ends, whatever warning handler was in effect before will be active again.

Posted in test tools, system testing, web testing | 2 Comments »

April 15, 2008 by mensming.

One of the best conferences I have ever attended was the 2007 VMWorld conference held in San Francisco. Part of this was probably because I was new to VMWare. The quantity and quality of the information shared was uniformly high. For a vendor sponsored conference, this is saying a lot.

However, most of the sessions I attended were presented by VMWare employees. I would like to see more user based presentations. In particular, VMWare experiences outside of the standard server consolidation usage pattern. With this being said, VMWare has issued a Call for Papers for the 2008 VMWorld to be held in Las Vegas on September 15-18, 2008.

Posted in test tools | No Comments »

March 24, 2008 by mensming.

I use VMWare extensively in my daily job. Most of these are Windows virtual machines (VMs) that get refreshed on a fairly regular basis. We use the clone feature along with a customization script to refresh a VM. Add a perl script which uses the API to drive this process and it is fairly painless. If a change needs to be made to the VMs, we make the change to the template and the next time the VMs are refreshed, the change is propagated to all of the instances.

Recently, I needed to add a routine scheduled maintenance task to the VMs. Easy enough — I thought. I created the script that would perform the task. Tested it locally. All works. Tested that it works as a scheduled process. Great. Transferred it to a test VM. Performed the same tests. All is going according to plan. I add the script to the templates. I schedule the script to run at the specified times. Double check my work. Now just wait until we refresh the VMs.

After the refresh of the templates, I go to verify that the script is behaving as expected. Nope. Is the script there? Yes. Is the scheduled task there? Yes. Can the scheduled task run? No. Since our customization script creates new security IDs during the clone, the security ID associated with the scheduled tasks (which was valid in the template) is not valid in the cloned instance.

The work around was rather simple. In the customization script, there is an option to set a "Run Once" command that will execute the first time the new virtual machine is powered on. We already use this feature to invoke a batch file to download our latest code and do some configuration based on the machine name. I modified this batch file to use the "at" command to schedule the script. Test again. All looks good. Perform test clone. All looks good. Wait for VMs to refresh. Problem solved.

Posted in test tools | 2 Comments »

March 10, 2008 by mensming.

Scott Hanselman, on his Hanselminutes podcast interviewed Quetzal Bradley of Microsoft in show 103. I enjoyed the discussion because it covers two topics I find myself explaining to less technical managers time and time again:

• Why test coverage is a necessary but insufficient measure of when testing is done
• Why 100% automation is not necessarily a good thing

It is a quick listen - recommended.

Posted in test tools, system testing | No Comments »