The validation plan in an important document for an architect.
Often a solution will be configured by a number of different providers, personnel, and external contractors
The lead architect still has a level of responsibility to ensure the design phases pass through to the actual configuration.
It should be possible to plan, prior to implementation, a suitable method to prove the solution, when presented to the client, complies to the physical design.
The approach I took for this area in my successful VCDX documentation was to create a validation strategy document for the solution that I could use to prove each of the key areas from the VCDX blueprint.
This approach is more than just a selection of tests in a table for each of the settings I gave as design choices in the architectural document.
It was my attempt of showing evidence of ensuring solution quality and how as a lead architect I would plan this phase of work with a number of testers.
Areas to consider
- Scope of testing
- Configuration items (i.e. settings applied etc)
- Performance (Load, transactions)
- Recovery (RTO, RPO,)
- Security (Pen testing, Scans)
- Risks (i.e. professional testers, testers are the implementers etc).
- Responsibility (who tests what, owns the faults, and remediation tasks)
- Entry criteria (What is required prior to testing – Design docs, implementation sign off?)
- Exit Criteria – This is the statement of quality, how many faults and how severe can they be, i.e.> S1- Show-stopper a crash. data loss, S3 – Minor issue – workaround.
- Test Suspension – Major faults which suggest meaningful testing cannot take place
- Test deliverables – Test plans, Reports, Fault logs etc
Ensure each of the track blueprint silos are covered and review the requirements to ensure each one can be proven.
Present each test case in a easy to read format which covers ;
- Reference No
- Silo Area (i.e. MVCNS, Performance, DR)
- Test description
- Expected Result
- Actual Result
Validation and the defence
Understanding and knowing these tests and your validation approach will help you in the defence.
Q: Are you sure you can recover from a failure? How are you sure the RPO, RTO, SLA is met?
A: This was validated using the documented test plan with successful recovery by a number of testers.
Test plan reports were reported back to the client and remedial steps taken to ensure there were as detailed in the physical design.
A Workaround for X number of test faults were documented as standard operating processes and logged as part of regular health check reports.
Q: Can you walk me through your process for this?
A: High level process on whiteboard, “This is also documented in my test plan and was reported back to the project team as part of the operational readiness phase.
Useful Book for concepts – aimed at networking silo