September 16, 2007 by mensming.

I was at VMWorld last week. Unfortunately, my network connectivity was limited so I was unable to post my notes during the conference.

Keynotes - Mendel Rosenblum, Chief Scientist & Co-Founder, VMWare

Demo - Storage VMotion (future)

Demo - Virtual Appliance - streaming download of image "instant on" (future)

Demo - Continuous Availability (future)

VMWorld 2008 - Las Vegas (9/18 - 9/20)

Fault Analysis and Methodology

Mike Valley, Systems Engineer, VMWare

esxcfg-boot -q boot
Currently used ram disk

esxcfg-boot -q vmkmod
List configured modules

service mgmt-vmware restart
will restart service console, hostd

hostname - use FQDN, configure /etc/hosts

vmkping - use vmkernel to ping a device

No eth0 - use vswif
Use esxcfg-vswif for configuring network interface

Collecting log files - export diagnostic data via VI client or from start menu

vm-support -S -i 10 -d 600 - capture performance snapshots

-X shows mapping between internal ID and name of VM

Virtual Center
——————-
vpxd -S - dumps to console to see what is wrong

LM Tools - License manage tools

How to License MS Servers for Your Virtualized Environment

Eric Jewett, Lead Product Manager, Windows Server Marketing, Microsoft

This presentation focused on server products and not desktops. The reaction of the audience was downright hostile.

ISV Licensing in a Virtualized Environment

Amy Kanary, Research Director, Software Pricing & Delivery, IDC

Common Models:

• Per system / # of servers (10% of worldwide revenue)

• Capacity based licenses (per cpu, per core, etc.) (12% of worldwide revenue)
• User based licenses (51% of worldwide revenue)

Concerns:

• Require purchase of more software than needed
• Restrict vertical deployment & mobility
• Allow flexibility only for best customers

Innovative Approaches:

• Move away from processor / core / socket
• Subscription licenses
• Value based metrics
• Pay per use

Navigate Complexity

• If possible, purchase via volume licenses (usually more flexibility)
• Research via web
• Work with channel partners / resllers
• Tell ISV know how importance standardization is to you

Panel:
Dr. Stephen Hess, BEA
Roman Bukary, SAP

VProbes: Diagnostics for Production Software

Keith Adams, Sr. Staff Engineer, VMWare
Robert Benson, Engineer, VMWare

Alex Migorodskiy, Technical Staff, VMWare

VProbes allow you to insert probes at various levels of architecture (App, Guest OS, VMM, etc)

Does not require a restart
Are "read only" - cannot change values

OS Independent
Currently pre-alphs

Define a script that ties into an event

Could be used to see root kits since it exists outside of the guest

VAssure: Write Your Own Bug Detectors Under VMWare Record and Playback

Min Xu
Dmitry Grinberg

Currently a research process

Since it uses record and playback (assertions execute during playback) it should not slowdown execution or impact the program

Logically:

if (replay) {
assert(…)
}

VAssert SDK Methods: VAssert, VLog, VWatchPoint

Posted in conferences | No Comments »

September 16, 2007 by mensming.

I was at VMWorld last week. Unfortunately, my network connectivity was limited so I was unable to post my notes during the conference.

Keynotes - John Chambers, CEO Cisco

John Chambers message was basically talking about how IT enables business and how to present that case to business executives.

Keynotes- Energy Panel

Quentin Hardy, Panel Moderator, Forbes Magazine
David Brooks, Harvard
Ray Cline, EDS
Amdrew Fanara, EPA
John Gibson, HSBC Bank

ESX Server CPU Scheduling

Andrei Drofeev, VMWare

CPU Scheduling Internals

• SMP Virtual Machines
• Load Balancing & Migrations
• Hyperthreading Support
• User Worlds - VMWare term for whole VM
• Interrupts
• NUMA

Performance Tips

• Idle VMs still receive timer interrupts
• Avoid VM CPU affinity
• SMP - use as few virtual CPUs as possible
• 64 bit guest generally better performance
• Avoid running programs in service console
• Don't fully commit CPU with reservations

ESXTOP Utility

• High %RDY time - VM was ready for CPU time but had to wait
• High %Wait / %Idle - workload is not CPU intensive

Future Directions

• Scalability - # CPUs per host
• Power mgt
• Better multi-core support

VM Infrastructure 3 - Best Practices for Performance

Jeff Buell, Staff Engineer
Deraki Kulkarni, Sr. MTS, VMWare

Sources of Virtualization Overhead

• CPU
• Memory
• Devices
• Resource Management

CPU Performance:

• Keep in mind OS timer interrupts. For awhile, Linx was using a 1000 Hz which was too often. Latest linux is set to 250 Hz which is OK. Windows is 100 Hz.
• 64 bit guests give better performance
• DIsable unused controllers / devices (see KB 1290)

Memory Performance

• Page sharing - pages which are the same between VMs can be shared between VMs
• Memory ballooning - require VMWare tools
• Avoid active memory over commit (eliminate ESX memory swapping)
• Right size guest OS memory

Networking

• Ensure there is enough CPU to process networks
• Use vmxnet network driver from VMWare tools

Storage Performance

• Guest driver setting can affect performance (KB 9645697) - Increase the size of writes
• Recommend using Fibre channel SAN
• NFS or iSCSI - use more CPY than SAN
• Better performance if use Virtual Center to create partitions
• Increase VMs max outstanding disk requests if needed (KB 1268)

Using the Secure Technology Implementation Guide (STIG) with VMWare Infrastructure 3

DISA - Defense Infromation Systems Agency

STIG - general guide for securing systems

Vulnerability Categories
Category I - Worse level. Allows access tot he machine
Category II - Provides information that could lead to access (high potential)
Category III - Provide information that could lead to access

5 STIGs related to VMI3 (http://iase.disa.mil/stigs/index.html)

• Virtual Computing STIG
• Unix STIG (ESX Server, VMs)
• DB STIG (Virtual Center)
• Win OS STIG (Win VM)
• Web Server STIG and checklist
• Other => SRR scripts, associated checklists, vulnerabilities management systems

Run SRR scripts (system readiness review - ./start-SRR)

See slides for many details of current findings and their meanings.

VMWare Infrastructure 3: Advanced Diagnostic Log Analysis

Mostafa Khalil, VCP, VMWare Product Support Engineer

ESX Server Boot Process: Boot Loader -> initrd -> vmkernel -> vmnix -> /sbin/init ->init scripts -> vmware init scripts

Collecting logs:

• UI via VI Client (right click on item and select export diagnostics data)

• Multiple logs on each ESX server

VMKernel Log:

Message Log:

hostd.log:

vpxa.log

esxcfg-firewall

oldconf files

esxupdate.log

vmkernel-version

Security Architecture Design and Hardening of VMWare Infrastructure 3

Kick Larsen - Engineering - Product Security Officer
Banjot S. Chanana - Product Mgr, Platform Security
Brian Cosker-Swerske - Senior Consultant

Service Console

Securing VMs

Posted in conferences | No Comments »

September 16, 2007 by mensming.

I was at VMWorld last week. Unfortunately, my network connectivity was limited so I was unable to post my notes during the conference.

Keynote - Diane Green / President and CEO of VMWare

The most interesting part of the keynote for me was the introduction of ESX Server 3i. The existing ESX Server has a 2 GB footprint. ESX Server 3i has a footprint of 32MB which can be embedded in flash memory by hardware manufacturers. Virtual Center will also be fed hardware status information including CPU, Power and temperature information. The chief marketing officer for Dell, Mark Jarvis, came on stage to demonstrate 3i on some new hardware Dell plans on shipping in November. Other vendors announcing support for 3i include Dell, IBM, HP, Fujitsu / Siemans and NEC.

Other announcements included disaster recovery and desktop consolidation products.

Keynote - Pat Gelsinger, Sr. VP and General Manager, Digital Enterprise Group, Intel

Virtualization is disaggregating the OS. It breaks the 1 to 1 relationship between the operating system and hardware.

Intel is viewing support for virtualization in 4 areas:

• Processors- VT-x / VT-i architecture
• Core Platform - Intel VT for directed IO
• Networking & Storage - Intel VT for Connectivity
• Standards

VMDq Network Architecture: Allows hardware to queue packets per virtual machine (instead of ESX figuring it out) which should improve network throughput.

Keynote- Hector de J. Ruiz, PhD, Chairman and CEO, AMD

Barcelona quad core architecture was released the prior day. In addition, includes better support for virtualization, especially VMotion.

AMD Senior Fellow Leendert Van Doorn can on stage to discuss the architecture in more detail.

Virtual Center Administration: Top Ten

1. Overview of Virtual Center Architecture

2. Virtual Center Updates

All updates are full releases
Basic procedure: Shutdown virtual center, backup database, apply update

When Virtual Center is updated, the Virtual Center agent is pushed out to each ESX Server. For this to succeed, the directory /tmp/vmware-root directly must exist. Unfortunately, there is a cron job that will remove the directory. There is a script,
CreateTmpDirs.vbs, that will create these directories but the script will only work it ssh root access is enabled.

3. What is new in Virtual Center 2

2 patch / maintenance release in past year.

2.0.1
Patch 2

• Came out in February, build #40644
• Improve statistics with rolling process (TempDB growth issue)
• Server recovery options
• License directory instead of 1 file

2.0.2

• Came out in July, build #50618

• Support SQL Server 2005 SP2
• Installer enhancements with better database advisor
• Alarm on VM heartbeat
• Performance improvements

4. Virtualizing Virtual Center

VC can be run in a VM and is fully supported.

Since license server usually runs in VC, need to get a host based license for ESX server running VC.
Database should be kept in a seperate VM or physical center

5. Virtual Center Availability

Options:

Run VMWare HA, VC in a VM
MS Clustered Server

Note: VC will shutdown on DB connectivity issues
Prior to 2.0.1 patch 2, VC reported a non-error exit code when shutting down so service control manager thought it was an expected shutdown.

6. Virtual Center Database

DB schema is often modified with VC updates

A DB restart will require a VC restart

Statistics collection level can be set to 1-4. Default is 1. Do not run more than 2

New sizing calculator available

Recovery mode should be set to simple if not backing up transaction logs

7. Virtual Center Diagnostics

Look up slides from last year VMWorld on VC diagnostics

VC log files found in c:\windows\temp

8. Understand VMotion CPU compatibility

9. VC-Support Scripts

Used when contacting tech support — zips up various system configuration and log files for sending to support

May want to run on a regularly scheduled basis to track trends.

10. Common VC Support Issues

Port conflicts
Administrator lockout (can disable authorization checks temporarily)

VI3 Resource Management and DRS - Performance Use Cases

Aravind Pavuluri, VMWare
Chirag Bhatt, VMWare

DRS - Distributed Resource Scheduler

A VM will only start if its reservations can be guaranteed (CPU, memory, etc.)

Reservations - Minimum requirements
Limits - Maximum usageShares - Allow VMs to compete for resource pools.

RM - Resource Manager (single host)

Better performance generally occurs with reservations

If no resource pools are specified, the default pool is used

Internal memory swapping - avoid if possible. 2 types of swapping, within the guest and at the ESX server

Memory balooning - requires VMWare tools be installed in guest OS. Will swap out guest memory which is not in use when ESX server needs additional memory.

Performance Benchmarking in Virtual Environments

Hemant Guidhani, Technical Marketing Manager, VM Ware

Common mistakes

• Physical to virtual comparisons but settings are not comparable
  
• Using different virtual products (VMWare server vs ESX server)

Benchmarking

• Standard benchmarking guidelines still apply
• No tuning should be required for VI3 (ESX should be optimal out of the box)
• Virtual overhead depends on work loads

General Guidelines

• Always use release or GA builds
• Use hardware on the compatibility list
• When benchmarking specific components, make sure other system resources are not constrained

Creating VM

• Install latest version of VM Tools
• Disable unused components

Recommendations:

• Set minimum reservation to the working set of the application
• Do not disable memory ballooning or page sharing
• Set appropriate caching and IO policy (Read & write cache. Write back and NOT write through policy)
• Create virtual disk from Virtual Center to ensure disk is aligned
• Use vmxnet network adaptor from VMWare tools for network performance
• Use esxtop to monitor VM / ESX interactions

IT Service Management - A technical overview

Eddue Dinel, Product Manager, VMWare

As a system progresses through the SDLC process:

• Lots of copies of the system in development (Lab Manager)
• Fewer copies in staging and other pre-production environments
• Single system in production

Lab Manager

• Optimized for high churn environment

• All copies of VM in lab manager are linked clones (not full copies)
• Network fencing - can duplicate IP addresses, machine names, etc.

Stage Manager

• Product in development to support pre-production environment
• Work on servers as a group (as a system, not individual VMs)
• Access control
• Integrate workflow
• Service operations: start, stop, clone, promote, demote, archive

Design Implementation and Management of Computing Security Lab Environment

Brian Hag, University of Alaska, Fairbanks

I had hoped that this would be a discussion of how to study real world security exploits in a safe environment. Instead it ended up being about a lab where students could learn how to scan systems, sniff traffic and look for vulnerabilities in their own projects.

Posted in conferences | No Comments »

September 16, 2007 by mensming.

I was at VMWorld last week. Unfortunately, my network connectivity was limited so I was unable to post my notes during the conference.

Due to a long line, I actually missed the first lab I had scheduled for the day. That lab was on performance benchmarking (which I later heard a talk on so I think I made out about even…)

Lab06 - VI Perl Toolkit Scripting to Administer VMWare Infrastructure

Overall this lab was a nice introduction to the perl API. However, if you had any experience at all with the API, there probably would not be a whole lot new learned during this lab. Some of the items I took away from the lab:

• Perl API can be executed against Virtual Center or ESX server
• Realtime information can be gathered directly from the ESX server
• "If you can do it in VirtualCenter, you can do it with the perl toolkit"

Generic Perl Script Pattern:

• Import VI runtime modules
• Customize options (datacenter, virtual center, etc.)
• Parse & validate options
• Connect to server
• Obtain, process and display results
• Close the server connection

Perl Script Repository:

http://www.vmware.com/communities/content/developer/viperlscripts.html

Currently just sample scripts but will eventually be expanded to include user generated scripts

SDK Reference Guide:
http://www.vmware.com/download/sdk/

Forum: http://www.vmware.com/community/forum.jspa?forumID=393

Trouble Shooting:

Use "Managed Object Browser" (MOB) - web interface to get object names and what responses will look like. Viewable via web interface on Virtual Center.

Posted in conferences | No Comments »