Choosing a Penetration Testing Vendor: Scope, Evidence and Reporting

Penetration testing quality varies more than almost any other security service, and the variation is hard to see from a proposal. Two firms quoting similar prices can deliver work of entirely different value, and you generally find out only after you have paid.

One artefact predicts quality better than anything else in the process: the redacted sample report.

Reading the sample report

Ask every vendor for one. A firm that will not provide a redacted sample is telling you something.

Reproduction steps are the single best quality signal. A finding you cannot reproduce is a finding your engineers will argue with and eventually ignore. A good report lets a developer reproduce the issue in minutes.

The attack narrative is the second. A findings table sorted by CVSS is scanner output with formatting. A narrative describing how the tester moved from initial access to the objective is what a human produced, and it is the part that changes how you think about your estate.

Business context separates consultants from technicians. "This SQL injection allows retrieval of the full patient table" is more useful than "SQL injection, CVSS 9.8".

Who actually does the work

Certifications are an imperfect but useful signal. Practical, hands-on certifications carry more weight than multiple-choice ones, and a tester with public research, tooling or disclosures behind them is usually a good sign.

Scoping discipline

A vendor who scopes carelessly will test carelessly. Watch how they behave during scoping — a good firm asks awkward questions before quoting:

• What are you actually worried about?
• What would a successful attack look like for your business?
• What is out of bounds, and why?
• What is the state of your vulnerability management — should we expect to spend the first days on known issues?
• Are there systems where testing could affect safety or continuity?

A vendor who quotes from a one-line request without asking any of these will deliver a generic engagement, because that is what they scoped.

The terms to fix in writing

1. Scope — exact ranges, URLs, roles, and exclusions
2. Methodology referenced explicitly
3. Testing window, and constraints around it
4. Escalation path if something breaks, or if a critical finding emerges mid-test
5. Immediate notification for critical findings, rather than waiting for the report
6. Retesting — included, and for how long afterwards
7. Deliverables — report, executive summary, attestation letter if your customers need one
8. Data handling — what they collect, where it is stored, when it is destroyed

Point five matters more than it looks. If a tester finds a critical vulnerability on day two, you want to know on day two, not in the report three weeks later.

The penetration testing RFP template sets these out in a form vendors can bid against consistently, which is what makes prices comparable. See also what a test should cost.

Red flags

• A quote without scoping questions
• Pricing far below the market — you are buying a scan
• No sample report available
• Certifications advertised for the firm rather than the testers
• Findings promised in advance — "we always find X"
• No retest included, and reluctance to discuss it

Where to start

Request a redacted sample report from three vendors before discussing price. Fifteen minutes with each tells you more about what you will receive than any amount of proposal reading, and it reframes the conversation from cost to quality.

GuardsArm provides sample reports on request and scopes against a written specification. See security testing or book a scoping call.