Software requirements, design and test strategy review

More documentation is not better.  Better living documentation is better. The purpose of this review is to verify that the artifacts have sufficient content to develop and test the code. This review applies especially if you have agile development.  It's expected that the artifacts will be living documents. Artifacts that may/will be reviewed are:

  • Top down traceability from requirements to design to code to test plans.  People often trace the requirements to tests but often forget to design to the requirements and code to the design.  Even if you have Agile or incremental development it's still important.  Traceability can be accomplished even with living document.
  • Bottom up traceability.  Is there design code that doesn't implicitly or explicitly satisfy the design or requirements?  To much is an indication of gold plating or dead code.
  • Applicable state, logic and timing diagrams.  Pictures are more effective than words.  In many industries the traceability and documentation requirements are so exhaustive that they don't provide for pictures that can say 1000 words.  Even if your process requires words - you can still have informative pictures.
  • Interface diagrams, contracts and tables.  In todays complex systems this is a must have.  Particularly when some code is being developed by a different company or in a different city.
  • Use cases - this is a timeless must have
  • White box test cases - Contrary to myth software engineers aren't generally either willing or good at testing their own code and should not be allowed to test their code in whatever way they want to.
  • Black box test cases - Testers often focus on volume of test cases instead of quality of test cases.  Do they actually test exactly what the end user will do? All day? Without turning off their computers?  Are they looking for non-obvious but serious faults? Are they testing in order of risk or are they testing the riskiest code last and causing a schedule delay while the defects are getting corrected?
  • Testability of requirements and design. Because of rubber stamped one shot requirements reviews, people often find out during development that they don't really know enough to write the code or test it. This causes schedule delays while clarifications are requested or it causes rework when both software engineers and testers guess when they shouldn't guess.
  • Inclusion of top level and low level exception handling.  Many software reliability issues are due to the code not being able to identify system and software failures.  Whether the requirements say so or not - the software is the brains of the system and must be able to detect and recover from system and software faults.  Pretending they won't happen doesn't make them go away.
  • Identification of the "shall nots" - what the software should not or cannot do.  Yes, these can be tested if you identify the failure space.
  • Unwritten assumptions