Other Software QA and Testing Resources
- Top Resources
- Software QA and Testing-related Organizations/Certifications/Conferences
- Links to QA and Testing-related Magazines/Publications
- General Software QA and Testing Resources
- Agile Testing Resources
- Test Automation Resources
- Mobile Testing Resources
- Web QA and Testing Resources
- Web Security Testing Resources
- Web Usability Resources
Stickyminds.com - Comprehensive software testing resource site from Techwell Corp., with articles, news, information on software testing and quality engineering, books, tools, conferences, message boards.
Thoughtworks Testing Portal - Testing articles and blogs from Thoughtworks, Inc.
James Bach's Blog - Large collection of thoughtful postings/articles on Jame's Bach's Satisfice web site; organized into a wide selection of categories.
Association for Software Testing - A nonprofit professional organization dedicated to advancing the understanding and practice of software testing. For scholars, students, and software development practitioners. Sponsors conferences, publications, web sites, newsletter.
SEI - Web site of Software Engineering Institute (at Carnegie Mellon University); info about SEI technical programs, publications, bibliographies, some online documents, SEI courses and training, links to related sites.
American Society for Quality - American Society for Quality (formerly the American Society for Quality Control) web site; geared to quality issues in general, not just Software QA. ASQ is the largest quality organization in the world, with more than 100,000 members. Provides a wide variety of general quality-related certifications, as well software-specific ones such as the CSQE (Certified Software Quality Engineer).
SPIN - Software Process Improvement Network, for those interested in improving software engineering practices. Organized into regional groups called "SPINs" that meet and share their experiences initiating and sustaining software process improvement programs. Annual meeting at the Software Engineering Process Group (SEPG) Conference, which is co-sponsored by the SEI and a regional SPIN. Web site lists links to regional SPINs worldwide.
SQuAD - Software Quality Association of Denver - software QA monthly meetings and an annual conference, in Denver, Colorado. Site has links to many past meeting presentations.
TCQAA - Twin Cities Quality Assurance Association of Minneapolis and St. Paul, Minnesota. Has monthly meetings. Site has link to past presentations
Southern California Quality Assurance Association - Software QA organization in Orange County Califiornia near Los Angeles.
VANQ - Vancouver Canada area association for software quality assurance and testing. Site has links to videos of some past meetings.
KWSQA - Software QA/Testing group with monthly meetings in Kitchener-Waterloo region (Ontario, Canada).
Northern Virginia Test Automation Interest Group - Monthly meetings on software test automation in the Washington DC area. Site has links to many past meeting presentations.Note: The above is only a partial listing. There are many other local QA- and Testing-related meetings in cities around the world. Check with local software-related professional organizations or sites such as meetup.com for information about current local meetings in your location.
Certification Information for Software QA and Test Engineers:
ISTQB Certified Tester - The International Software Testing Qualifications Board, based in Belgium, was initially a part of the European Organization for Quality - Software Group. Certifications and testing are administered by ISTQB organizations in each of a number of countries around the world. Multiple certification levels are available.
CSQE - ASQ (American Society for Quality) CSQE (Certified Software Quality Engineer) program - information on requirements, outline of required 'Body of Knowledge', listing of study references and more.
ISEB Software Testing Certifications - The British Computer Society Information Systems Examinations Board (ISEB) maintains a program of several levels of testing certifications. Some levels are equivalent to the ISTQB Certifications.
ASTQB - The American Software Testing Qualifications Board is a branch of the ISTQB Certifications are based on experience and a written test. Multiple types of certifications are available. The ISTQB Body of Knowledge, Terms, and sample exams are available as free downloads.
CSQA/CSTE - QAI Global Institute's program for CSQA (Certified Software Quality Analyst), CSTE (Certified Software Tester), and Certified Software Project Manager (CSPM), Certified Associate in Software Quality (CASQ), and many other certifications.
Software QA and Testing Conferences:
Conferences - Conferences listing at Kerry Zallar's 'Software Testing Stuff' web site.
LogiGear Magazine - Online testing magazine from Logigear; site includes archived articles from past issues by year and by category (such as Test Methods & Metrics, Agile, Mobile, etc.). Published 4-6 times per year.
Tea Time With Testers Magazine - Free online testing magazine available as PDF file; web site includes back issues.
Professional Tester Magazine - Bi-monthly testing publication from Professional Tester Inc. Free access to PDF version, archive, individual articles and news and events agenda.
Software Quality Professional Magazine - Published by the American Society for Quality; web site includes table of contents and abstracts of all articles, and full text of selected articles.
Better Software Magazine - Online magazine with an orientation to software testing, published 4 times year by TechWell Corp. Latest issue downloadable as a free PDF.
Methods and Tools - Software Methods and Tools e-newsletter web site by Martinig and Associates; regular articles are included on process improvement, testing, modeling, management, etc. Site includes current issues and past issues with full text of all articles; as well as extensive additional information and resources.
(Note: also see the 'Books' section for a listing of books on Software QA, Testing, and related subjects.)
Google Testing Blog - Public blog site for Google's testers. Includes archives going back to 2007.
The Lean Mindset - Blog site of Mary Poppendieck, author of the book 'Lean Software Development'.
Heuristic Test Strategy Model - A 5 page checklist-based approach to test strategy by James Bach. Essentially a set of things to consider for a testing strategy, in 4 categories. Includes checklists for 'General Test Techniques', 'Quality Criteria Categories', 'Project Environment', 'Product Elements'.
Software Testing Mindmaps - Useful way to view various testing strategies such as 'How to test text fields', 'Web Security Testing Checklist', 'Mobile Testing in a Nutshell', or 'Web App Testing Toolkit'. More than 100 'mindmaps' in categories 'General', 'Mobile', and 'Web'. Can be helpful in considering various test strategies and tools.
Pass vs. Fail: Is There a Problem Here? - Interesting blog article on testing vs checking and the issues in 'pass/fail' type testing and reporting, from 2009 in Michael Bolton's DevelopSense blog.
The Seven Basic Context-Driven Principles - Article about the 'context-driven' approach to testing on the Association for Software Testing web site, by Cem Kaner and James Bach. Lists the main principles and includes description, examples.
The Value of Checklists and the Danger of Scripts - Presentation at CAST 2008 by Cem Kaner.
XP in a Safety-Critical Environment - Interesting article by Mary and Tom Poppendieck concerning the applicability of XP practices in safety-critical software development.
Health Exchange Mess - Post-mortem on the 2013 implementation problems of the Maryland State Health Exchange, authored by Charles Hayward, a retired US Government Accountability Office auditor. The April 2014 article is in 2 parts: 'Health Exchange Mess Part I: Failures in leadership', and 'Health Exchange Mess Part II: Assigning blame, recouping money.' Also see LA Times articles summarizing problems with implementations including those in Hawaii, Massachusetts, and Oregon in a March 2014 2-part series by Maeve Reston: 'States that have struggled with healthcare sites consider lawsuits' .
Software Negligence and Testing Coverage - Article by Cem Kaner contains an old but still very informative list of 101 types of testing coverage measures; shows the complexities in any discussion of 'testing coverage'. Selected quotes of interest from the article: "Even if you achieve complete coverage for a given population of tests (such as, all lines of code tested), you have not done complete, or even adequate, testing." and "The decision as to whether to try for 1%, 10%, 50% or 100% coverage against any given population is non-obvious. It involves tradeoffs based on thoughtful judgment."
What is DevOps? - Article from New Relic discribing the common themes, tools, and ideas behind 'DevOps', a widely-used term having widely-varying interpretations including it's relation to testing.
Software Engineering: An Idea Whose Time Has Come and Gone? - An interesting 2009 article from 'IEEE Software' by Tom DeMarco in which he indicates, among other things, that his early ideas and advice regarding quantified work, project planning, and metrics for software projects - such as those in his 1982 book 'Controlling Software Projects: Management, Measurement, and Estimation', may have been wrong.
Exploring Exploratory Testing - Article by Cem Kaner and Andy Tinkham from 2003 about the exploratory testing approach to software testing; includes discussions of questioning strategies and heuristics.
Exploratory Testing Explained - Article by James Bach on exploratory testing; includes attributes of a software project and tester that impact decisions on testing approaches, exploratory testing examples, etc.
Cem Kaner's software testing site - Cem Kaner's site contains a large selection of his articles about software testing, legal issues, test management, and more (see the 'Publications' section of the site). Also see his 'badsoftware.com' website , a consumer and legal-issues orientation to software quality issues.
They Write the Right Stuff - Summary of the original article 'How to write near-perfect software' by Charles Fishman that was in Fast Company magazine - about how software was developed for the U.S. Space Shuttle. "The group's most important creation is not the perfect software they write -- it's the process they invented that writes the perfect software."
What is a Test Architect? - Discussion re test architects by Microsoft's Alan Page.
RBCS Testing articles - RBCS Consulting Services web site's collection of software testing articles on a wide variety of testing-related subjects.
Errors in Scientific Software - Article titled 'The T experiments: errors in scientific software' by Les Hatton; old but still alarming article from 1997. For a more recent article on software bugs slipping unknowingly into scientific research (2010), sometimes resulting in retraction of published papers, see 'Computational science: ...Error: …why scientific programming does not compute' by Zeeya Merali in Nature magazine.
Certifications - Cem Kaner's perspective on software testing certifications from a 2007 article - includes a discussion of a proposed 'Open Certification Process'; section 3 of the paper has a long discussion of "Project Manager’s Perspective: Problems With the Current Certification System"
'Good programmer' definitions/discussions - Since testers and developers often need to work closely together, and since many testers also do some programming, it is helpful to get some perspective on 'what is a good programmer'. Also see the discussion in the old 'Joel on Software' forum..
WSR Consulting Group publications - Good collection of QA and Testing related articles from WSR consulting, a computer crisis/litigation consulting company. The articles have an emphasis on proper management of problem projects and engineering-customer relationships for software projects.
Where is the Science in Computer Science? - Article in Communications of the ACM (from October 2012) by Vinton Cerf (VP at Google, past winner of the Turing Award, one of the acknowledged 'Fathers of the Internet', and president of ACM). In the article he states, among other things, '....Even though we design software systems and ought to have some clues about how these systems behave and perform, we generally do not have a reliable ability to anticipate the states these systems can get into, their vulnerabilities, their performance, and ability to adapt to changing conditions.' He also goes on to note our generally poor ability to predict how long it will take to find and fix bugs, or to have an idea how many new bugs will be created by fixes.
Software Estimation - Interview about software metrics and estimation from the IT Metrics and Productivity Journal of December 2015.
Testing Education Articles - Collection of articles on software testing and the teaching of software testing from the Florida Institute of Technology funded by the U.S. National Science Foundation.
'Software Experts' site - Software engineering site oriented to microcontroller/embedded system environments, by Eberhard De Wille and Dana Vede. Site has sections on design, coding, refactoring, process, and a large section on testing.
ITIL - Originally the 'IT Infrastructure Library' - a set of best-practices guides on the management and provision of operational IT Services, covering 5 main topics: Service Strategy, Service Design, Service Transition, Service Operation, and Continual Service Improvement. ITIL was initially developed by of the British Office of Government Commerce and the itSMF, the 'IT Service Management Forum' a UK-based organization comprosed of 1000 companies and government organizations worldwide; ITIL is now owned by AXELOS, a joint venture of the British Cabinet Office of the UK government and Capita plc. The ISO/IEC 20000 standard is based on ITIL. There is a good summary of the ITIL approach in Wikipedia.
Big Ball of Mud - Outstanding essay on the 'de-facto standard software architecture', by Briane Foote and Joseph Yoder of the U. of Illinois at Urbana-Champaign. The 'Big Ball of Mud' architecture is defined as 'a casually, even haphazardly, structured system. Its organization, if one can call it that, is dictated more by expediency than design....The overall structure of the system may never have been well defined. If it was, it may have eroded beyond recognition.' They discuss why this architecture is so popular, advantages and disadvantages, and what can be done to improve such systems.
Satisfice.com Web Site - James Bach's Satisfice.com Web Site with a great collection of his articles on various aspects of software testing.
Search Software Quality - TechTarget's software QA and testing site with a collection of articles, tutorials, blogs, and news.
Illustrative Risks to the Public in the Use of Computer Systems - Enormous list of software, system, and related problems compiled by Peter Neumann/SRI International. Organized by categories such as space, defense, medical, stock market, elections, insurance, cryptography, etc. Includes related book list, other information. (Also see 'Risks Digest' listed below.)
ARIANE 5 Flight 501 Failure Report by the Inquiry Board - A rare and instructive detailed public analysis of a major software failure - the 1996 launch failure of the new (At the time) Ariane 5 rocket. This is the official report of the inquiry board appointed by the French National Center for Space Studies and the European Space Agency. Also see the article 'Design by Contract: The Lessons of Ariane' which includes a discussion of the code reuse issues brought to light by the Ariane 5 failure.
Risks Digest - Digest of the 'Forum on Risks to the Public in Computers and Related Systems'. Includes latest issue and archives covering software and system problems, vulnerabilities, disasters; based on the comp.risks newsgroup.
SEI Capability Maturity Models - Software Engineering Institute's CMMI web site, with info and documentation downloads on the CMMI models for for Development, Services, Acquisition, People, and Data Management.
Construx Software Resources - Site with many useful resources, estimation info and resources, various checklists, and Steve McConnell's 'Software Survival Guide' website.
Software Engineering Resources - Large collection of useful information and links to many other sites and resources, all related to the SW engineering process including agile processes, project planning and management, metrics, risk analysis, programming methods, OO SW engineering, testing, QA, CM. From R.S. Pressman, author of the book 'Software Engineering, A Practitioner's Approach'.
Software Test Coverage Analysis article - Article containing a good discussion of test coverage analysis from Bullseye Testing Technology, maker of "C-Cover Test Coverage Analyzer" tool.
Embrace Technical Debt - Article by Eric Ries (author of book 'The Lean Startup') from his Startup Lessons Learned site. Among other things, he discusses how technical debt concerns might vary depending on context. For example when a design or business model has a high likelihood of changing (such as in a startup), technical debt concerns might be moderated compared to more stable contexts. He discusses how to take a disciplined approach to these context variations.
Object-Oriented Concepts - Basics of object-oriented programming concepts, from Oracle's (formerly Sun's) Java site. Good quick intro.
Manifesto for Agile Software Development - The origin of the 'Agile' approach and the twelve guiding principles of agile software development.
Agile Methodologies - Martin Fowler's online discussion of 'agile' methodologies (XP, Scrum, Crystal, FDD, DSDM, etc.) includes summaries of various approaches as well as reference information, and factors to consider in choosing these approaches.
A Sprint Framework for Testers - Suggested processes and practices to consider for testers in an Agile environment - from an October 2015 blog post by Conner Roberts from Oct 2015.
'Agile Testing and Quality Strategies: Discipline Over Rhetoric' - Introductory agile testing article by Scott Ambler.
Agile Testing - Key Success Factors - Chapter 21 from the book 'Agile Testing: A Practical Guide for Testers and Agile Teams' by Lisa Crispin and Janet Gregory; book excerpt provided by InfoQ.
Perils and Pitfalls of Agile Adoption - Article by Matt Heuser at InformIT site, includes discussion of risks such as that agile methods are easy to misunderstand, that it's easy to think you're doing Agile right, and be wrong, and that agile methods make value (or lack of value) visible.
Agile Testing - What is it? Can it work? - PDF version of an article by Bret Pettichord that summarizes considerations and issues in testing in agile environments.
An Uncomfortable Truth about Agile Testing - Article by Jeff Patton on the StickyMinds site about some of the potential difficulties of testing on an Agile project.
Ron Jeffries Site - Agile/Scrum/XP - Large collection of articles and posts from Ron Jeffries about Agile, Scrum, and Extreme Programming, including a discussion of how QA fits into the XP approach, XP Magazine archives, and many testing-related articles with relevance to XP/Scrum/Agile. Also see 'The Rules of Extreme Programming' at the www.extremeprogramming.org web site.
Scrum Resources - Web site of Scrum Alliance with a resources section that includes resources and articles such as 'Tranistioning to Agile', 'The Scrum Primer', 'Lean Agile: The Ethical Choice', and much more.
GTAC - Web site for the Google Test Automation Conferences includes links to past conferences and slides and videos for each, going back to 2006.
NoVaTAIG Test Automation Resources - Test automation presentations, articles and resources from the Northern Virginia Test Automation Interest Group monthly meeting summaries ('Recent past meetings' link on site main page).
7 Deadly Sins of Test Automation - Slideshare by Adrian Smith - the 'seven deadly sins' discussed are a set of common anti-patterns found to erode automation's value, resulting in long term maintenance issues and reducing teams' ability to respond to change and continuously deliver.
The Forgotten Layer of the Test Automation Pyramid - Short article by Mike Cohn (author of 'Succeeding with Agile' book) about the Services testing layer of the pyramid. Also see Justin Rohrman's article Getting Started with API Testing article in the Gurock Quality Hub site.
Seven Steps to Test Automation Success - Old but still surprisingly relevant introductory article on considerations in successful automated testing; by Bret Pettichord.
How to Grade Your Selenium Tests - Slideshare of a Dave Haefner presentation on how to judge how good/bad your selenium test automation is. Provides good guidance on selenium test design and coding.
Effective Performance Testing articles - Extensive collection of how-to and other information on performance testing at Scott Barber's web site.
GUI Testing Design - Short discussion of automated UI testing issues, includes discussion of methods of identifying a component in a GUI hierarchy for use in developing automated test scripts.
See the Softwareqatest.com Bookstore section on Automation for books on test automation.
Mobile Testing - Sept 2013 online LogiGear Magazine issue with a focus on Mobile Testing
Android application testing with the Android test framework - Long tutorial by Lars Vogel re how to test Android apps with different Android testing frameworks. Includes test automation, what/how to test, mocking, more.
Test Automation Interfaces for Mobile Apps - Article in LogiGear Magazine by Julian Harty discusses mobile automation considerations from the perspective of 3 automation stages - Discovery, Design, and Execution. There is discussion of the mobile app's interfaces which he groups into a) Human–computer Interaction ( HCI) - touch, proximity, movement, sound, light, controls; and b) Sensor interfaces such as accelerometers, magnetism, GPS, Orientation, Camera, etc. Possible automation interfaces discussed include the GUI, code (unit testing), API's, accessibility capabilities, etc.
Browser compatibility: viewports - Mobile device/browser compatibility information from Peter-Paul Koch's web site.
Testing Mobile Web Apps with WebDriver - From the Open Source at Google blog - discussion of how to write automated tests to test a site when viewed from an Android or iOS browser. The WebDriver web testing framework includes a touch API that allows a test to interact with a web page through finger taps, flicks, finger scrolls, and long presses. It can rotate the display and provides an API to interact with HTML5 features such as local storage, session storage and application cache.
Mobile Application Testing blog - Mobile app testing site and blog by Anurag Khode.
QuirksBlog - Mobile - Articles re mobile browser compatibility and responsive issues from blog of Peter-Paul Koch.
See the 'Mobile Web/App Testing Tools' section of the Web Test Tools List page for mobile testing tools.
Ultimate Website Launch Checklist - Useful comprehensive checklist that can be useful to help drive testing strategy.
Web Site Testing Checklist - More of a web site development checklist, but it is extensive and still useful as a way to generate ideas for testing a web site.
Performance Testing Guidance for Web Applications - Online 18-chapter guide for an end-to-end approach for implementing web performance testing. Part of the Performance Testing Guidance Project web site. Covers: managing and conducting performance testing in both Agile and structured environments; load testing, stress testing, and other types of performance related testing; identifying objectives, designing tests, executing tests, analyzing results, and reporting. Published 2007 but still useful.
Web Site Performance Testing - A collection of useful information on various aspects of performance testing, from Scott Barber's web site. Topics include: "Pinpointing and Exploiting Specific Performance Bottlenecks", "Common Performance Testing Challenges", "How Fast is Fast Enough", and "Introduction to Performance Testing". Although some of the information is not specifically oriented to web performance testing, it is still highly applicable.
Keynote Systems Resources Page - Useful collection of articles and information on web site and mobile performance testing. Also see the related site performance indices which lists a variety of business, consumer, government, and other web sites along with their 'performance index'.
Evaluating Web Sites for Accessibility - Articles and information on the World Wide Web Consortium web site's 'Web Accessibility Initiative' section on how to assess and test web sites for accessibility issues.
Automated local accessibility testing using WAVE and WebDriver - Interesting article from WatirMelon Testing Blog by Alister Scott; refers to some other interesting related resources.
OWASP - The Open Web Application Security Project (OWASP) is dedicated to finding and fighting the causes of insecure software. Everything available in site is free and open source. 'How To' section includes 'Guide to Building Secure Web Applications and Web Services', a book-length 'Testing Guide', 'Code Review Guide', etc. 'Cheat sheets' section includes a 'Web Application Security Testing Cheat Sheet'. Also security news, articles such as 'How to Write Insecure Code', tools, code, filters, downloads, and more.
StopBadware.org - Web site security guidelines and information from the StopBadware site, formerly based at Harvard University’s Berkman Center for Internet & Society and subsequently spun off as an independent non-profit.
Computer Audit FAQ - Good introductory information from IsecT Ltd. on 'Computer Audit', which refers to the analysis of computer systems and networks by examining the effectiveness of their technical and procedural controls (information security control systems) to minimise risks. Also has links to other resources, and some articles such as 'Strategic Approach to Information Security Management'.
SANS Security Resources - Web site of SANS (SysAdmin, Audit, Network, and Security Institute), a cooperative research and education organization for sysadmins, security professionals, and network administrators for sharing lessons learned and solutions. Includes an Intrusion Detection FAQ; more than 1500 white papers on security; webcasts; security trends, top security risks, and much more are freely available.
SAMATE - Software Assurance Metrics And Tool Evaluation site at the U.S. National Institute of Standards. Has information on web application security issues and technical/security/other vulnerabilities information, lists of Web Application Vulnerability Scanners, source code analysis tools, links to other lists, etc. Also: information on other related security resources, publications, conferences, and more.
CVE - Searchable, downloadable, and on-the-web 'Common Vulnerabilities and Exposures' list hosted by Mitre Corp. CVE goal is to standardize the names for all publicly known vulnerabilities and security exposures, so that security information can be efficiently shared and handled. Many security test tools are utilizing or planning on utilizing this standardized naming/numbering system.
Common Attack Pattern and Enumeration - CAPEC is a publicly available, community-developed list of common attack patterns (descriptions of common methods for exploiting software systems), with a comprehensive schema and classification taxonomy. By Mitre Corp.
W3 Security Resources - Large collection of information and resources on web security, including an FAQ, hosted by the W3C Consortium (the folks who set web standards/protocols, etc.)
Computer Emergency Response Team site - CERT's internet security web site; includes web server security information, blogs, podcasts, publications; hosted by the Software Engineering Institute at Carnegie Mellon University.
See the 'Web Tools' section for web security test tool listings.
See the Softwareqatest.com Bookstore section on Security Testing for books on software security testing.
Prioritizing Web Usability - PDF chapter from 2006 book on Web Usability by Jakob Nielsen and Hoa Loranger.
Usability.gov - Web site with a large collection of web usability resources, information, and guidelines. Although the site was developed by the U.S. federal government for use by various federal agencies, the site is a resource available to anyone.
User Interface Engineering - Web site of User Interface Engineering Inc., founded by Jared M. Spool. Many articles on web site and product usability, such as 'Web Application Form Design', 'Seven Common Usability Testing Mistakes', '5 Things to Know about Users', and more.
Jakob Nielsen's articles re interface usability and website design - Jakob Nielsen's web usability/design collection with such articles as 'How Users Read on the Web", 'Costs of User Testing', and 'Differences between Print Design and Web Design'.