Testing Interview Questions-2

What is SEI? CMM? ISO? IEEE? ANSI? Will it help?

· SEI = ‘Software Engineering Institute’ at Carnegie-Mellon University; initiated by the U.S. Defense Department to help improve software development processes.

· CMM = ‘Capability Maturity Model’, developed by the SEI. It’s a model of 5 levels of organizational ‘maturity’ that determine effectiveness in delivering quality software. It is geared to large organizations such as large U.S. Defense Department contractors. However, many of the QA processes involved are appropriate to any organization, and if reasonably applied can be helpful. Organizations can receive CMM ratings by undergoing assessments by qualified auditors.

       Level 1 - characterized by chaos, periodic panics, and heroic

                 efforts required by individuals to successfully

                 complete projects.  Few if any processes in place;

                 successes may not be repeatable.

       Level 2 - software project tracking, requirements management,

                 realistic planning, and configuration management

                 processes are in place; successful practices can

                 be repeated.

       Level 3 - standard software development and maintenance processes

                 are integrated throughout an organization; a Software

                 Engineering Process Group is is in place to oversee

                 software processes, and training programs are used to

                 ensure understanding and compliance.

       Level 4 - metrics are used to track productivity, processes,

                 and products.  Project performance is predictable,

                 and quality is consistently high.

       Level 5 - the focus is on continouous process improvement. The

                 impact of new processes and technologies can be

                 predicted and effectively implemented when required.

      Perspective on CMM ratings:  During 1997-2001, 1018 organizations

      were assessed.  Of those, 27% were rated at Level 1, 39% at 2,

      23% at 3, 6% at 4, and  5% at 5.  (For ratings during the period 

      1992-96, 62% were at Level 1, 23% at 2, 13% at 3, 2% at 4, and 

      0.4% at 5.)  The median size of organizations was 100 software 

      engineering/maintenance personnel; 32% of organizations were 

      U.S. federal contractors or agencies.  For those rated at 

      Level 1, the most problematical key process area was in 

      Software Quality Assurance.

· ISO = ‘International Organization for Standardization’ - The ISO 9001:2000 standard (which replaces the previous standard of 1994) concerns quality systems that are assessed by outside auditors, and it applies to many kinds of production and manufacturing organizations, not just software. It covers documentation, design, development, production, testing, installation, servicing, and other processes. The full set of standards consists of: (a) Q9001-2000 - Quality Management Systems: Requirements; (b)Q9000-2000 - Quality Management Systems: Fundamentals and Vocabulary; (c)Q9004-2000 - Quality Management Systems: Guidelines for Performance Improvements. To be ISO 9001 certified, a third-party auditor assesses an organization, and certification is typically good for about 3 years, after which a complete reassessment is required. Note that ISO certification does not necessarily indicate quality products - it indicates only that documented processes are followed.

· IEEE = ‘Institute of Electrical and Electronics Engineers’ - among other things, creates standards such as ‘IEEE Standard for Software Test Documentation’ (IEEE/ANSI Standard 829), ‘IEEE Standard of Software Unit Testing (IEEE/ANSI Standard 1008), ‘IEEE Standard for Software Quality Assurance Plans’ (IEEE/ANSI Standard 730), and others.

· ANSI = ‘American National Standards Institute’, the primary industrial standards body in the U.S.; publishes some software-related standards in conjunction with the IEEE and ASQ (American Society for Quality).

· Other software development process assessment methods besides CMM and ISO 9000 include SPICE, Trillium, Tick IT and Bootstrap.

What is the ’software life cycle’?
The life cycle begins when an application is first conceived and ends when it is no longer in use. It includes aspects such as initial concept, requirements analysis, functional design, internal design, documentation planning, test planning, coding, document preparation, integration, testing, maintenance, updates, retesting, phase-out, and other aspects.

Will automated testing tools make testing easier?

· Possibly. For small projects, the time needed to learn and implement them may not be worth it. For larger projects, or on-going long-term projects they can be valuable.

· A common type of automated tool is the ‘record/playback’ type. For example, a tester could click through all combinations of menu choices, dialog box choices, buttons, etc. in an application GUI and have them ‘recorded’ and the results logged by a tool. The ‘recording’ is typically in the form of text based on a scripting language that is interpretable by the testing tool. If new buttons are added, or some underlying code in the application is changed, etc. the application can then be retested by just ‘playing back’ the ‘recorded’ actions, and comparing the logging results to check effects of the changes. The problem with such tools is that if there are continual changes to the system being tested, the ‘recordings’ may have to be changed so much that it becomes very time-consuming to continuously update the scripts. Additionally, interpretation of results (screens, data, logs, etc.) can be a difficult task. Note that there are record/playback tools for text-based interfaces also, and for all types of platforms.

· Other automated tools can include:

·              code analyzers - monitor code complexity, adherence to

·                               standards, etc.

·          

·              coverage analyzers - these tools check which parts of the

·                               code have been exercised by a test, and may

·                               be oriented to code statement coverage,

·                               condition coverage, path coverage, etc.

·          

·              memory analyzers - such as bounds-checkers and leak detectors.

·          

·              load/performance test tools - for testing client/server

·                               and web applications under various load

·                               levels.

·          

·              web test tools - to check that links are valid, HTML code

·                               usage is correct, client-side and

·                               server-side programs work, a web site’s

·                               interactions are secure.

·                                                  

·              other tools - for test case management, documentation

·                               management, bug reporting, and configuration

·                               management.