Archive for the conferences Category

PNSQC CFP Deadline Extended - May 1, 2008

In a prior post, I mentioned the call for papers (CFP) for the Pacific Northwest Software Quality Conference (PNSQC). I received a notice last week that the deadline has been extended to May 1. More details can be found here.

 

Pacific Northwest Software Quality Conference CFP

A call for papers (CFP) for the Pacific Northwest Software Quality Conference (PNSQC) is currently active. This year’s conference (the 26th annual) will be held October 13-15, 2008 in Portland, OR. Presenting at a conference is an excellent way to sharpen your ideas and subject them to a rigorous examination by your peers. I have presented twice a PNSQC and found it to be a professional stretching activity.

Detailed information on the call can be found here. Abstracts must be submitted by April 20, 2008.

 

VMWorld - Day 3 - September 13, 2007

I was at VMWorld last week. Unfortunately, my network connectivity was limited so I was unable to post my notes during the conference.

Keynotes - Mendel Rosenblum, Chief Scientist & Co-Founder, VMWare

Demo - Storage VMotion (future)

Demo - Virtual Appliance - streaming download of image "instant on" (future)

Demo - Continuous Availability (future)

VMWorld 2008 - Las Vegas (9/18 - 9/20)

Fault Analysis and Methodology

Mike Valley, Systems Engineer, VMWare

esxcfg-boot -q boot
Currently used ram disk

esxcfg-boot -q vmkmod
List configured modules

service mgmt-vmware restart
will restart service console, hostd

hostname - use FQDN, configure /etc/hosts

vmkping - use vmkernel to ping a device

No eth0 - use vswif
Use esxcfg-vswif for configuring network interface

Collecting log files - export diagnostic data via VI client or from start menu

vm-support -S -i 10 -d 600 - capture performance snapshots

-X shows mapping between internal ID and name of VM

Virtual Center
——————-
vpxd -S - dumps to console to see what is wrong

LM Tools - License manage tools


How to License MS Servers for Your Virtualized Environment

Eric Jewett, Lead Product Manager, Windows Server Marketing, Microsoft

This presentation focused on server products and not desktops. The reaction of the audience was downright hostile.

ISV Licensing in a Virtualized Environment

Amy Kanary, Research Director, Software Pricing & Delivery, IDC

Common Models:

  • Per system / # of servers (10% of worldwide revenue)

  • Capacity based licenses (per cpu, per core, etc.) (12% of worldwide revenue)
  • User based licenses (51% of worldwide revenue)

Concerns:

  • Require purchase of more software than needed
  • Restrict vertical deployment & mobility
  • Allow flexibility only for best customers

Innovative Approaches:

  • Move away from processor / core / socket
  • Subscription licenses
  • Value based metrics
  • Pay per use

Navigate Complexity

  • If possible, purchase via volume licenses (usually more flexibility)
  • Research via web
  • Work with channel partners / resllers
  • Tell ISV know how importance standardization is to you

Panel:
Dr. Stephen Hess, BEA
Roman Bukary, SAP

VProbes: Diagnostics for Production Software

Keith Adams, Sr. Staff Engineer, VMWare
Robert Benson, Engineer, VMWare

Alex Migorodskiy, Technical Staff, VMWare

VProbes allow you to insert probes at various levels of architecture (App, Guest OS, VMM, etc)

Does not require a restart
Are "read only" - cannot change values

OS Independent
Currently pre-alphs

Define a script that ties into an event

Could be used to see root kits since it exists outside of the guest

VAssure: Write Your Own Bug Detectors Under VMWare Record and Playback

Min Xu
Dmitry Grinberg

Currently a research process

Since it uses record and playback (assertions execute during playback) it should not slowdown execution or impact the program

Logically:

if (replay) {
assert(…)
}

VAssert SDK Methods: VAssert, VLog, VWatchPoint

VMWorld - Day 2 - September 12, 2007

I was at VMWorld last week. Unfortunately, my network connectivity was limited so I was unable to post my notes during the conference.

Keynotes - John Chambers, CEO Cisco

John Chambers message was basically talking about how IT enables business and how to present that case to business executives.

Keynotes- Energy Panel

Quentin Hardy, Panel Moderator, Forbes Magazine
David Brooks, Harvard
Ray Cline, EDS
Amdrew Fanara, EPA
John Gibson, HSBC Bank

ESX Server CPU Scheduling

Andrei Drofeev, VMWare

CPU Scheduling Internals

  • SMP Virtual Machines
  • Load Balancing & Migrations
  • Hyperthreading Support
  • User Worlds - VMWare term for whole VM
  • Interrupts
  • NUMA

Performance Tips

  • Idle VMs still receive timer interrupts
  • Avoid VM CPU affinity
  • SMP - use as few virtual CPUs as possible
  • 64 bit guest generally better performance
  • Avoid running programs in service console
  • Don't fully commit CPU with reservations

ESXTOP Utility

  • High %RDY time - VM was ready for CPU time but had to wait
  • High %Wait / %Idle - workload is not CPU intensive

Future Directions

  • Scalability - # CPUs per host
  • Power mgt
  • Better multi-core support


VM Infrastructure 3 - Best Practices for Performance

Jeff Buell, Staff Engineer
Deraki Kulkarni, Sr. MTS, VMWare

Sources of Virtualization Overhead

  • CPU
  • Memory
  • Devices
  • Resource Management

CPU Performance:

  • Keep in mind OS timer interrupts. For awhile, Linx was using a 1000 Hz which was too often. Latest linux is set to 250 Hz which is OK. Windows is 100 Hz.
  • 64 bit guests give better performance
  • DIsable unused controllers / devices (see KB 1290)

Memory Performance

  • Page sharing - pages which are the same between VMs can be shared between VMs
  • Memory ballooning - require VMWare tools
  • Avoid active memory over commit (eliminate ESX memory swapping)
  • Right size guest OS memory

Networking

  • Ensure there is enough CPU to process networks
  • Use vmxnet network driver from VMWare tools

Storage Performance

  • Guest driver setting can affect performance (KB 9645697) - Increase the size of writes
  • Recommend using Fibre channel SAN
  • NFS or iSCSI - use more CPY than SAN
  • Better performance if use Virtual Center to create partitions
  • Increase VMs max outstanding disk requests if needed (KB 1268)

Using the Secure Technology Implementation Guide (STIG) with VMWare Infrastructure 3

DISA - Defense Infromation Systems Agency

STIG - general guide for securing systems

Vulnerability Categories
Category I - Worse level. Allows access tot he machine
Category II - Provides information that could lead to access (high potential)
Category III - Provide information that could lead to access

5 STIGs related to VMI3 (http://iase.disa.mil/stigs/index.html)

  • Virtual Computing STIG
  • Unix STIG (ESX Server, VMs)
  • DB STIG (Virtual Center)
  • Win OS STIG (Win VM)
  • Web Server STIG and checklist
  • Other => SRR scripts, associated checklists, vulnerabilities management systems

Run SRR scripts (system readiness review - ./start-SRR)

See slides for many details of current findings and their meanings.

VMWare Infrastructure 3: Advanced Diagnostic Log Analysis

Mostafa Khalil, VCP, VMWare Product Support Engineer

ESX Server Boot Process: Boot Loader -> initrd -> vmkernel -> vmnix -> /sbin/init ->init scripts -> vmware init scripts

Collecting logs:

  • UI via VI Client (right click on item and select export diagnostics data)

  • Multiple logs on each ESX server

VMKernel Log:

/var/log
All events generated by vmkernel (warnings are also written to seperate log)
Logs rotates. All events since last load are also in memory at /proc/vmware/log.

General Log msg: timestamp, hostname, msg source, uptime, <instance>, device, src line #, msg

Translating vmkernel error codes: already listed in msg in ESX 3

Message Log:

Like linux messages
Console events, logon events, iSCSI authentication events

hostd.log:

vi client events
Events done on behalf of various services

vpxa.log

Events of interactions with Virtual Center

esxcfg-firewall

rule events

oldconf files

backup of config files modified by VC, VI Client or esxcfg-* scripts

esxupdate.log

Updates from esxupdates

vmkernel-version

Whenever kernel is loaded / updated


Security Architecture Design and Hardening of VMWare Infrastructure 3

Kick Larsen - Engineering - Product Security Officer
Banjot S. Chanana - Product Mgr, Platform Security
Brian Cosker-Swerske - Senior Consultant

Service Console

Actually a VM based on RedHat Linux (own Virtual CPU)
Ports 902, 80, 443 and 22 open by default

Hardening:
Disable ctrl-alt-del
Require password for single user mode

service console network parameters
login banners

Secure Networks
Password policy for local user accounts
Password complexity
/etc/security/access.conf

Limit root access using securetty
enable syslog
Change snmp community string from the default

Securing VMs

Same as a physical box
Remove unnecessary functions / services
Disable cut and paste

VMWorld - Day 1 - September 11, 2007

I was at VMWorld last week. Unfortunately, my network connectivity was limited so I was unable to post my notes during the conference.

Keynote - Diane Green / President and CEO of VMWare

The most interesting part of the keynote for me was the introduction of ESX Server 3i. The existing ESX Server has a 2 GB footprint. ESX Server 3i has a footprint of 32MB which can be embedded in flash memory by hardware manufacturers. Virtual Center will also be fed hardware status information including CPU, Power and temperature information. The chief marketing officer for Dell, Mark Jarvis, came on stage to demonstrate 3i on some new hardware Dell plans on shipping in November. Other vendors announcing support for 3i include Dell, IBM, HP, Fujitsu / Siemans and NEC.

Other announcements included disaster recovery and desktop consolidation products.

Keynote - Pat Gelsinger, Sr. VP and General Manager, Digital Enterprise Group, Intel

Virtualization is disaggregating the OS. It breaks the 1 to 1 relationship between the operating system and hardware.

Intel is viewing support for virtualization in 4 areas:

  • Processors- VT-x / VT-i architecture
  • Core Platform - Intel VT for directed IO
  • Networking & Storage - Intel VT for Connectivity
  • Standards

VMDq Network Architecture: Allows hardware to queue packets per virtual machine (instead of ESX figuring it out) which should improve network throughput.

Keynote- Hector de J. Ruiz, PhD, Chairman and CEO, AMD

Barcelona quad core architecture was released the prior day. In addition, includes better support for virtualization, especially VMotion.

AMD Senior Fellow Leendert Van Doorn can on stage to discuss the architecture in more detail.

Virtual Center Administration: Top Ten

1. Overview of Virtual Center Architecture

2. Virtual Center Updates

All updates are full releases
Basic procedure: Shutdown virtual center, backup database, apply update

When Virtual Center is updated, the Virtual Center agent is pushed out to each ESX Server. For this to succeed, the directory /tmp/vmware-root directly must exist. Unfortunately, there is a cron job that will remove the directory. There is a script,
CreateTmpDirs.vbs, that will create these directories but the script will only work it ssh root access is enabled.

3. What is new in Virtual Center 2

2 patch / maintenance release in past year.

2.0.1
Patch 2

  • Came out in February, build #40644
  • Improve statistics with rolling process (TempDB growth issue)
  • Server recovery options
  • License directory instead of 1 file

2.0.2

  • Came out in July, build #50618

  • Support SQL Server 2005 SP2
  • Installer enhancements with better database advisor
  • Alarm on VM heartbeat
  • Performance improvements

4. Virtualizing Virtual Center

VC can be run in a VM and is fully supported.

Since license server usually runs in VC, need to get a host based license for ESX server running VC.
Database should be kept in a seperate VM or physical center

5. Virtual Center Availability

Options:

Run VMWare HA, VC in a VM
MS Clustered Server

Note: VC will shutdown on DB connectivity issues
Prior to 2.0.1 patch 2, VC reported a non-error exit code when shutting down so service control manager thought it was an expected shutdown.

6. Virtual Center Database

DB schema is often modified with VC updates

A DB restart will require a VC restart

Statistics collection level can be set to 1-4. Default is 1. Do not run more than 2

New sizing calculator available

Recovery mode should be set to simple if not backing up transaction logs

7. Virtual Center Diagnostics

Look up slides from last year VMWorld on VC diagnostics

VC log files found in c:\windows\temp

8. Understand VMotion CPU compatibility

9. VC-Support Scripts

Used when contacting tech support — zips up various system configuration and log files for sending to support

May want to run on a regularly scheduled basis to track trends.

10. Common VC Support Issues

Port conflicts
Administrator lockout (can disable authorization checks temporarily)

VI3 Resource Management and DRS - Performance Use Cases

Aravind Pavuluri, VMWare
Chirag Bhatt, VMWare

DRS - Distributed Resource Scheduler

A VM will only start if its reservations can be guaranteed (CPU, memory, etc.)

Reservations - Minimum requirements
Limits - Maximum usageShares - Allow VMs to compete for resource pools.

RM - Resource Manager (single host)

Better performance generally occurs with reservations

If no resource pools are specified, the default pool is used

Internal memory swapping - avoid if possible. 2 types of swapping, within the guest and at the ESX server

Memory balooning - requires VMWare tools be installed in guest OS. Will swap out guest memory which is not in use when ESX server needs additional memory.

Performance Benchmarking in Virtual Environments

Hemant Guidhani, Technical Marketing Manager, VM Ware

Common mistakes

  • Physical to virtual comparisons but settings are not comparable
  • Using different virtual products (VMWare server vs ESX server)

Benchmarking

  • Standard benchmarking guidelines still apply
  • No tuning should be required for VI3 (ESX should be optimal out of the box)
  • Virtual overhead depends on work loads

General Guidelines

  • Always use release or GA builds
  • Use hardware on the compatibility list
  • When benchmarking specific components, make sure other system resources are not constrained

Creating VM

  • Install latest version of VM Tools
  • Disable unused components

Recommendations:

  • Set minimum reservation to the working set of the application
  • Do not disable memory ballooning or page sharing
  • Set appropriate caching and IO policy (Read & write cache. Write back and NOT write through policy)
  • Create virtual disk from Virtual Center to ensure disk is aligned
  • Use vmxnet network adaptor from VMWare tools for network performance
  • Use esxtop to monitor VM / ESX interactions

IT Service Management - A technical overview

Eddue Dinel, Product Manager, VMWare

As a system progresses through the SDLC process:

  • Lots of copies of the system in development (Lab Manager)
  • Fewer copies in staging and other pre-production environments
  • Single system in production

Lab Manager

  • Optimized for high churn environment

  • All copies of VM in lab manager are linked clones (not full copies)
  • Network fencing - can duplicate IP addresses, machine names, etc.

Stage Manager

  • Product in development to support pre-production environment
  • Work on servers as a group (as a system, not individual VMs)
  • Access control
  • Integrate workflow
  • Service operations: start, stop, clone, promote, demote, archive


Design Implementation and Management of Computing Security Lab Environment

Brian Hag, University of Alaska, Fairbanks

I had hoped that this would be a discussion of how to study real world security exploits in a safe environment. Instead it ended up being about a lab where students could learn how to scan systems, sniff traffic and look for vulnerabilities in their own projects.

VMWorld - Day 0 - September 10, 2007

I was at VMWorld last week. Unfortunately, my network connectivity was limited so I was unable to post my notes during the conference.

Due to a long line, I actually missed the first lab I had scheduled for the day. That lab was on performance benchmarking (which I later heard a talk on so I think I made out about even…)

Lab06 - VI Perl Toolkit Scripting to Administer VMWare Infrastructure

Overall this lab was a nice introduction to the perl API. However, if you had any experience at all with the API, there probably would not be a whole lot new learned during this lab. Some of the items I took away from the lab:

  • Perl API can be executed against Virtual Center or ESX server
  • Realtime information can be gathered directly from the ESX server
  • "If you can do it in VirtualCenter, you can do it with the perl toolkit"

Generic Perl Script Pattern:

  • Import VI runtime modules
  • Customize options (datacenter, virtual center, etc.)
  • Parse & validate options
  • Connect to server
  • Obtain, process and display results
  • Close the server connection

Perl Script Repository:

http://www.vmware.com/communities/content/developer/viperlscripts.html

Currently just sample scripts but will eventually be expanded to include user generated scripts

SDK Reference Guide:
http://www.vmware.com/download/sdk/

Forum: http://www.vmware.com/community/forum.jspa?forumID=393

Trouble Shooting:

Use "Managed Object Browser" (MOB) - web interface to get object names and what responses will look like. Viewable via web interface on Virtual Center.

Pacific NW Software Quality Conferences

I am always looking for conferences / training opportunities close to where I work. A staple is the Pacific Northwest Software Quality Conference held in Portland. This years conference will be the 25th annual event. PNSQC recently announced the keynote speakers for the 2007 conference (October 8-10, 2007): Johanna Rothman and Hugh Thompson. As usual, I recommend this conference highly.

I recently saw an announcement for a conference that will be held in Bellevue, WA on July 9-11, 2007. The 2nd annual Conference of the Association for Software Testing will have Esther Derby, Keith Stobie, Lee Copeland and Harry Robinson as keynote speakers. Looks like it could be a good one.

 

Seattle Area Software Testing Related Organizations

I was recently asked by a coworker about local testing related organizations and I put together the following in my response. I thought it would be useful to capture the information in one spot so I can refer people to it in the future:

Seattle Area Software Quality Assurance Group
http://www.sasqag.org/

WSA Quality Assurance special interest group
http://www.wsa.org/events/sig/detail.asp?sigID=47

Seattle Eastside Area Software Process Improvement Network (SeaSpin)
http://www.seaspin.org/

American Society for Quality (ASQ) Seattle Section
http://www.asq-seattle.org/

IEEE Seattle Section
http://www.ieee-seattle.org/

 

Conference Call for Papers

Two conferences which I have participated in have recently issued call for papers.

The Pacific Northwest Software Quality Conference, this year being held October 9-10, 2007 in Portland, requests 300 - 600 word abstracts be submitted via the website by March 31, 2007. Details can be found here.

The Software Division of the American Society for Quality puts on the International Conference on Software Quality, this year being held October 16-17, 2007 in Lakewood, CO. The deadline for proposals is April 16, 2007. The form for technical presentations can be found here and the form for tutorials can be found here.

Conference Papers Posted

Some time ago, I presented papers at three different conferences. These papers have been safely tucked away inside the conference proceedings never to be seen again. Until now… (That is not exactly true — I placed the papers on the home page provided by my home ISP but that page has never been indexed, no one has linked to it, etc. so it was not necessarily discoverable.) These papers are now part of the website I am putting together as a companion to this blog, www.meesqa.com.

The first conference I presented at was ASQ’s Eighth International Conference on Software Quality held in conjunction with the 1998 Pacific Northwest Software Quality Conference. (This was the second conference I had ever attended.) The paper, Hierarchical Organization of Test Cases, is the result of my experience at two different companies that had no rhyme or reason as to how they prioritized / selected test cases for execution.

As a result of working with a team of recent college graduate developers, my coworkers and I developed a review approach that worked very well within that team. We started by doing a peer review where the developer walked us through the non-error path of their design (the happy path). After teasing out the basic functional issues in the design, the developer started to step us through the design again. This time, the review team threw every obstacle they could think of into the flow of the design. We started to call the first iteration the walk through and the second iteration the stagger through. The result of this experience was Walk and Stagger Through Review Process which I presented at Quality Week 2001 in San Francisco.

My final presentation was the result of the work I had been doing in how best to capture test cases and test results. I had struggled using text documents, HTML, Excel, Word, etc. and was never satisfied. There always came a time when I needed to tweak the presentation for either a functional or political reason. Like all companies at the time, we were actively involved in XML development and I decided to put the raw data in an XML document and use XSLT to transform it into the presentation format I wanted. I presented this at the 2002 Pacific Northwest Software Quality Conference as Using XML for Test Case Definition, Storage and Presentation.

I have not presented since but would like to start doing it again on a regular basis. Time will tell.

|