CSI 5118 Winter 2008
Automated Verification and Validation of Software
News:
o
Exam date/time:
Tuesday April 22: 13:00 to 16:00 Location: CBY B012
o Exam
information and review slides
o
Course project: due April 18
o Submission
instructions and evaluation criteria
o
Assignment 2: Due Thurs. April 3, noon.
o SDL alternative: WaterMeasure.zip
§ SDL model viewable in browser
o SPIN
alternative: WaterMeasure.pml.txt
o
Assignment 1: Due Mon. March 10, noon.
o Automated Test Execution Introduction (January 10)
o JUnit (January 15 / 17)
o Code coverage / instrumentation (January 22 / 24)
o Eclipse Test and Performance Tools Platform (TPTP) (January 29 / 31)
o Mock Objects (January 31, February 5)
o Database Test Tools (February 7 / 12)
o Web Application Testing (February 12/14)
o State space exploration (February 26 / 28, March 4)
o SPIN Tutorial (March 6)
o Automated Test Generation (March 11 / 13)
o Capture and Replay / GUI Testing (March 18 / 20)
o UML Testing Profile (March 25)
o OMG standard document: UML Testing Profile, version 1.0 (PDF)
o TTCN (March 27 / April 1)
o Security Testing (April 1 / April 3)
o Summary (April 8)
o JUnit demonstration (Jan. 14):
o Coverage Tools, shown January 24:
o TPTP, shown January 29 :
o Mock objects, shown January 31 / February 5
o Database testing, shown February 7 / 12:
o Web application testing, shown February 12 / 14. Eclipse project .zip files:
o Capture and replay / GUI testing demo: (March 18 / 20)
o UML test profile example (March 25)
Code
Library
o Apache Commons Logging (1.1.1)
o Cobertura (1.9)
o DbUnit (2.2)
o EasyMock (2.3)
o HttpUnit (1.6.2)
o Jetty (5.1.12)
o jMock (2.4.0)
o JUnit (4.4)
o MySQL JDBC Connector (5.0.7)
o SPIN
o Tcl/Tk (get the ActiveTcl “standard” distribution; it’s free)
o Jacareto (0.7.12) download installer from SourceForge
o Marathon (1.1.3)
o Abbot (1.0.1)
o Eclipse automated GUI testing plug-in (4.4.1)
o Eclipse: integrated development environment
o Eclipse Test and Performance Tools Platform (TPTP)
o Emma code coverage tool:
o Standalone: Emma
o Eclipse plug-in: EclEmma
o Cobertura coverage tool
o Coverlipse coverage tool (not recently updated; won’t work with Java 6 or Eclipse 3.3)
o EasyMock mock objects framework
o jMock mock objects framework
o DbUnit database test tool
o HttpUnit web page test tool
o MySQL database
o Jetty small web server
o Apache Tomcat web server
o SPIN state-space exploration, logic model checking program
o Jacareto capture and replay tool for Java/Swing
o Marathon capture and replay tool for Java/Swing
o Abbot GUI testing framework for Java [versions for Swing or SWT]
o TPTP automated GUI testing plug-in, for Eclipse / SWT based applications
o Nmap, port mapping tool
o Wireshark, network sniffer tool
o Open Web Application Security Project (OWASP) home page, including
o WebScarab security proxy tool
o Alan Williams
o e-mail: awilliam@site.uottawa.ca (this is the preferred contact method)
o Office:
o Phone: (613) 562-5800 x6914
o Office hours: Wednesdays: 13:00 – 14:30
o Tuesdays 13:00 – 14:30, Thursdays 11:30 – 13:00
o
Location:
o First lecture: Tuesday Jan. 8
o
Last lecture:
Tuesday Apr. 8.
o
No lectures: Tuesday Feb. 19 or Thurs.
Feb. 21, Thurs. April 10
o Course web page: http://www.site.uottawa.ca/~awillliam/csi5118_w08
o Course presentation material will be posted there.
o Suggested references (all are optional):
o J.B. Rainsberger, JUnit Recipes (2005)
o V. Massol, JUnit in Action (2004)
o C. Wysopal, L. Nelson, D.Dai Zovi, E. Dustin, The Art of Software Security Testing (2007)
o There is no specific pre-requisite for the course other than an undergraduate background in computer science or software engineering or equivalent experience. The material is complementary to CSI 5111 (Software Quality Engineering), but it is not necessary to have taken that course. Knowledge of the Java programming language would be useful for this course, as well as some familiarity with web applications.
o There will be two assignments and a project for the course, as well as a final exam. The assignments will each be worth 15% of the final grade, and the project will be worth 25%, while the exam will be worth 45%. The assignments will be due near the end of February and March, and the project will be due at the end of the term.
o The official calendar description: Topics in formal test derivation methods, test management, high-level, CASE-based verification and validation, data-flow & control-flow measures and metrics for assessing quality of designs and code, regression analysis & testing.
o The intention is to investigate issues related to software test automation, what capabilities are (and are not) available in current test tools, and what is the potential for new capabilities. The course takes a software engineering approach and provides the opportunity for some “hands-on” work with test tools. After taking this course, the goal is to be an intelligent user of software test tools, and provide the background to design and improve such tools.
o Theme for this offering of the course: tools for web and security testing
o Topics to be covered:
o Introduction: what is automated verification and validation, when is it feasible (and not feasible)?
o Types of testing, and the impact on tools.
o Test execution systems:
o General functions: issues with test execution and management
o JUnit: what it does, and how it works.
o Eclipse Test and Performance Tools Platform
o Performance measurement tools.
o Issues with testing graphical user interfaces.
o Coverage measurement and tools:
o What is being measured?
o Approaches for measuring coverage.
o Tools that measure coverage: Emma, Cobertura, CodeCover, Clover
o Tool issues
o Web application testing:
o Applications that use databases, and the DbUnit tool.
o Testing on web application servers, and the Cactus tool.
o Testing web page responses, and the Http/HtmlUnit tools.
o Other web components: XML files, scripts, tag libraries, etc.
o Web page verifiers
o Security testing:
o Security issues
o Tools to analyze software footprint and internals.
o Tools to generate invalid input and protocol messages.
o Verification tools:
o State-space exploration
o Issues with verification tools
o SPIN and its derivatives
Most recent update to this page: Tuesday April 8, 2008 12:05