February 20, 2010 by mensming.

Software Engineering Radio some time ago had a great podcast covering the basics of continuous integration with Chris Read. Highly recommended for both the beginner and the experienced needing a reminder of the basics. Recommended. The episode can be found here.

Posted in process improvement, test tools | No Comments »

January 30, 2010 by mensming.

This is another in a series of posts of my reactions to watching the videos from the Google Test Automation Conference 2008 held in Seattle. The talk The Value of Small Tests (38 minutes long) presented by Christopher Semturs on October 24, 2008 was not what I expected. The slides from the talk can be found here.

I was hoping for a discussion around the broader test case / test suite management and the intersection with test automation. Instead, it discussed the value of creating small tests with limited dependencies and using dependency injection, mocking, etc. Not to say that it is a bad talk - just a quick introduction for these topics. I would say the talk is aimed at getting developers writing and enabling automated testing.

If you are already familiar with dependency injection and mocking, I would not spend the time watching. However, if these concepts are new, it may be valuable.

Posted in test tools, conferences | No Comments »

January 23, 2010 by mensming.

Even if you are not working on the Microsoft platform, the December 2009 MSDN Magazine has some articles which can be of use to all testers.

In particular, Pairwise Testing with QICT by James McCaffrey provides enough insight into the QICT tool that it can be easily ported to any platform. While Automated Unit Tests for Legacy Code with Pex by Nikhil Sachdeva is specific to Microsoft technologies, Pex is a tool that all testers should become familiar. Finally, Using Agile Techniques to Pay Back Technical Debt by David Laribee has tips that are useful to most test organizations.

Posted in professional development, process improvement, test tools | No Comments »

January 16, 2010 by mensming.

I am in the process of watching the videos from the GTAC (Google Test Automation Conference) held in Seattle in October 2003. This post contains my notes for the presentation Taming the Beast - How to Test an AJAX Application (1 hour 1 minute) by Markus Clermont & John Thomas held on October 23, 2008.

I am not recommending spending time watching this video. The presentation is not bad. However, it does not translate well to video. There is a lot of interaction with the audience - which is great when you are present at the presentation. However, in the video the audience cannot be heard and the presenters do not repeat many of the questions / responses. In addition, the talk is misnamed. While an AJAX application is used as the example throughout the application, most of the talk is not about how to test an AJAX application. The talk is a good discussion of how to tame automation - automation of any application.

Here are the slide titles:

• AJAX: A Different Beast
• An Example GWT Application - (GWT = Taming the Beast - How to Test an AJAX Application)
• Some Statistics - (At this point there are questions to the audience regarding their automated testing experience.)
• System Architecture
• Small Medium Large - (Essentially how much of the application is under test)
• Testing Layer Pairs
• Are we done?
• Comparison
• Conclusion

Posted in training, test tools, web testing, conferences | No Comments »

December 20, 2009 by mensming.

A little while ago, I needed to run some Selenium processes against a different server which was https only with a self signed certificate. When Selenium fired up, I saw the SSL certificate warnings followed by the test failing. Even when I inserted a pause into my script so I would have time to add the security exception, the test still failed.

A search on the web led to Elliot Smith’s post entitled Dealing with self-signed SSL certificates when running Selenium server with Firefox. At the time, the post was only a few days old. It was exactly what I needed and worked like a champ. Thank you Elliot!

I would add to Elliot’s procedure that the cert_override.txt and cert8.db files can be added to source control. Even if the person executing the tests has never followed the procedure regarding creating a Firefox profile, the process will still work.

Finally, a minor warning. If you find that you need to add additional self-signed certificates, you will need to do that as part of a full-fledged Firefox profile - not the limited version created here. In that case, just like the first time, you will need to delete all of the Firefox created files and directories except cert_override.txt and cert8.db just like the first time it was setup. (I suppose you could also copy the 2 cert*.* files from a different full fledged profile - as long as you were confident regarding the contents of those files.)

Posted in test tools, web testing | No Comments »

May 4, 2009 by mensming.

Screen capture tools are required for any tester working on a system with a UI. Lifehacker recently asked their readers to nominate the 5 best screen capture tools. The results can be found here.

Here is the list of winning applications:

1. Snipping Tool (Windows)
2. FastStone Capture (Windows)
3. Jing (Windows/Mac)
4. Skitch (Mac)
5. Snagit (Windows)

Posted in test tools | No Comments »

April 20, 2009 by mensming.

I just got this in my inbox — quite a deal if you are looking to tryout JIRA or Confluence.

For this week only, we’re offering a special 5-user “starter” license of JIRA and Confluence for only $5 each. We’re calling it the Atlassian Stimulus Package and it’s our way of supporting small teams and small businesses in this difficult economic environment. Best of all, we’re going to donate every penny to charity, so please help us spread the word!

The Atlassian Foundation is donating all proceeds to Room to Read, a charity that helps the world’s future entrepreneurs by building libraries and schools for children in developing nations.

Get all the details at www.atlassian.com/starter. Hurry, offer ends on 24 April 2009.

Posted in IT, test tools, jira, bug tracking | No Comments »

February 7, 2009 by mensming.

While working through my backlog of reading, I came across the results of Software Test & Performance magazine’s Testers Choice Awards for 2008. This list is of no use to me.

First, look at most of the winners - LoadRunner, TestDirector, components of IBM Rational suite and other high priced, commercial products. Working at a startup now (and several in the past), there is rarely budget for these tools. Oftentimes, the tools imply a process or a level of maturity that do not exist in the first couple of years of a startup’s existence. A more useful breakdown would be a set of testers choice awards for open source tools and a set of testers choice awards for commercial tools. (There is a category for free test/performance tool but that is about as broad as giving an award for best vertebrate - there is just too much variation.)

Which is actually the second point, because of the diversity of the software development space, a generic list such as this does not shed light on tools that may be applicable to my particular area. This is similar to an example James Surowiecki used in The Wisdom of Crowds regarding the generic top 10 lists in music. There may be only 1 or no songs in the top 10 list for the genre I am interested in and most likely I am already familiar with the song. If I am interested in that genre, the generic top 10 list only tangentially touches my interests and does not supply any new information. If I am looking for new songs like the one I like in the generic list, the other entries may not help. However, a top 10 list of songs in the specific music type (the more specific the better) may lead me to new songs which I will enjoy. The same is true when it comes to tools. A “testers choice award for java code coverage tools” might point me to tools I am not familiar with while the category “java test/performance” does not point me to any tools that I am not already familiar. Knowing which code coverage tool for java others consider the best is much more useful than knowing LoadRunner won the java test/performance category for 2008.

Posted in test tools | No Comments »

December 14, 2008 by mensming.

While it has been a while since I needed to do installer testing, I recently researched a product that may be helpul in this task. Sandboxie strives to isolate programs from making changes to the rest of the operating system. The Sandboxie website describes it as:

Sandboxie runs your programs in an isolated space which prevents them from making permanent changes to other programs and data in your computer.

and lists as a benefit:

Windows Stays Lean: Prevent wear-and-tear in Windows by installing software into an isolated sandbox.

Sandboxie adds a third approach to installer testing. The first approach, to install over and over on the same instance, is expediant but not necessarily the most realistic. The second approach is to re-image / restore the system. When I first started, this meant using a program such as Ghost, now owned by Symantec. This approach has been supplanted using virtual machines. Virtual machines have made this process much faster.

Using Sandboxie, it is possible to have the cleanliness afforded by virtual machine or images plus the expediency of installing over and over on the same system. On a single system, multiple sandboxes could be created - one for each install test. This approach seems reasonable to me. It is still necessary to do installer testing on native system, but this can be done after many bugs our found and resolved.

Pricing is very reasonable for a commercial license. Check it out.

Posted in test tools, system testing | No Comments »

April 30, 2008 by mensming.

A lot of developers I have worked with have been amazed to see that I expand a suite of tests they developed by a factor of four. This is almost always because I am stressing error handling which is often not tested at all. Unfortunately, these tests will often cause many warnings. Sometimes I want to verify the exact warning. Other times it just clutters the output being sent to the screen. This can be very dangerous - it may lead someone running the test to believe there is a newly introduced bug.

Luckily, perl has a well defined mechanism for overriding the warn behavior. If you want to suppress warnings from being displayed in a section of tests do the following:

Once the scoping ends, whatever warning handler was in effect before will be active again.

Posted in test tools, system testing, web testing | 1 Comment »