The demand for penetration testing and security assessment services worldwide has been growing year-on-year. Driven largely by Governance, Risk, and Compliance (GRC) concerns, plus an evolving pressure to be observed taking information security and customer privacy seriously, most CIO/CSO/CISO’s can expect to conduct regular “pentests” as a means of validating their organizations or product’s security.

An unfortunate circumstance of two decades of professional service oriented delivery of pentests is that the very term “penetration testing” now covers a broad range of security services and risk attributes—with most consulting firms provide a smorgasbord of differentiated service offerings—intermixing terms such as security assessment and pentest, and constructing hybrid testing methodologies.

For those newly tasked with having to find and retain a team capable of delivering a pentest, the prospect of having to decipher the lingo and identify the right service is often daunting—as failure to get it right is not only financially costly but may also be career-ending if later proven to be inadequate.

What does today’s landscape of pentesting look like?

All penetration testing methodologies and delivery approaches are designed to factor-in and illustrate a threat represented by an attack vector or exploitation. A key differentiator between many testing methodologies lies in whether the scope is to identify the presence of a vulnerability or to exploit and subsequently propagate an attack through that vulnerability. The former is generally bucketed in the assessment and audit taxonomy, while the latter is more commonly a definition for penetration testing (or an ethical hack).

The penetration testing market and categorization of services is divided by two primary factors—the level of detail that will be provided by the client, and the range of “hacker” tools and techniques that will be allowed as part of the testing. Depending upon the business drivers behind the pentest (e.g. compliance, risk reduction, or attack simulation), there is often a graduated-scale of services. Some of the most common terms used are:

• Vulnerability Scanning – The use of automated tools to identify hosts, devices, infrastructure, services, applications, and code snippets that may be vulnerable to known attack vectors or have a history of security issues and vulnerabilities.
• Black-box Pentest – – The application of common attack tools and methodologies against a client-defined target or range of targets in which the pentester is tasked with identifying all the important security vulnerabilities and configuration failures of the scoped engagement. Typically, the penetration scope is limited to approved systems and windows of exploitation to minimize the potential for collateral damage. The client provides little information beyond the scope and expects the consultant to replicate the discovery and attack phases of an attacker who has zero insider knowledge of the environment.
• Gray-box Pentest – Identical methodology to the Black-box Pentest, but with some degree of insider knowledge transfer. When an important vulnerability is uncovered the consultant will typically liaise with the client to obtain additional “insider information” which can be used to either establish an appropriate risk classification for the vulnerability, or initiate a transfer of additional information about the host or the data it contains (that could likely be gained by successfully exploiting the vulnerability), without having to risk collateral damage or downtime during the testing phase.
• White-box Pentest (also referred to as Crystal-box Pentest) – Identical tools and methodology to the Black-box Pentest, but the consultants are supplied with all networking documentation and details ahead of time. Often, as part of a White-box Pentest, the client will provide network diagrams and the results of vulnerability scanning tools and past pentest reports. The objective of this type of pentest is to maximize the consultants time on identifying new and previously undocumented security vulnerabilities and issues.
• Architecture Review – Armed with an understanding of common attack tools and exploitation vectors, the consultant reviews the underlying architecture of the environment. Methodologies often include active testing phases, such as network mapping and service identification, but may include third-party hosting and delivery capabilities (e.g. domain name registration, DNS, etc.) and resilience to business disruption attacks (e.g. DDoS, Ransomware, etc.). A sizable component of the methodology is often tied to the evaluation and configuration of existing network detection and protection technologies (e.g. firewall rules, network segmentation, etc.)—with configuration files and information being provided directly by the client.
• Redteam Pentest – Closely related to the Black-box pentest, the Redteam pentest mostly closely resembles a real attack. The scope of the engagement (targets and tools that can be used) is often greater than a Black-box pentest, and typically conducted in a manner to not alert the client’s security operations and incident response teams. The consultant will try to exploit any vulnerabilities they reasonably believe will provide access to client systems and, from a compromised device, attempt to move laterally within a compromised network—seeking to gain access to a specific (hidden) target, or deliver proof of control of the entire client network.
• Code Review – The consultant is provided access to all source code material and will use a mix of automated and manual code analysis processes to identify security issues, vulnerabilities, and weaknesses. Some methodologies will encompass the creation of proof-of-concept (PoC) exploitation code to manually confirm the exploitability of an uncovered vulnerability.
• Controls Audit – Typically delivered on-site, the consultant is provided access to all necessary systems, logs, policy-derived configuration files, reporting infrastructure, and data repositories, and performs an audit of existing security controls against a defined list of attack scenarios. Depending upon the scope of the engagement, this may include validation against multiple compliance standards and use a mix of automated, manual, and questionnaire-based evaluation techniques.

The Hybrid Pentest Landscape

In recent years the pentest landscape has evolved further with the addition of hybrid services and community-sourcing solutions.

• Bug Bounty Programs – Public bug bounty programs seek to crowdsource penetration testing skills and directly incentivize participants to identify vulnerabilities in the client’s online services or consumer products. The approach typically encompasses an amalgamation of Vulnerability Scanning and Black-box Pentest methodologies—but with very specific scope and limitations on exploitation depth. With (ideally) many crowd-sourced testers, the majority of testing is repeated by each participant. The hope is that, over time, all low-hanging fruit vulnerabilities will be uncovered and later remediated.
• Purple Team Pentest – This hybrid pentest combines Redteam and Blueteam (i.e. the client’s defense or incident response team) activities into a single coordinated testing effort. The Redteam employs all the tools and tricks of a Redteam Pentest methodology, but each test is watch and responded to in real-time by the client’s Blueteam. As a collaborative pentest, there is regular communication between the teams (typically end of day calls) and synching of events. The objectives of Purple Team pentesting is both assess the capabilities of the Blueteam and to reduce the time typically taken to conduct a Redteam Pentest—by quickly validating the success or failure of various attack and exploitation techniques, and limiting the possibility of downtime failures of targeted and exploited systems.
• Disaster Recovery Testing – By combining a Whitebox Pentest with incident response preparedness testing and a scenario-based attack strategy, Disaster Recovery Testing is a hybrid pentest designed to review, assess, and actively test the organization’s capability to respond and recover from common hacker-initiated threats and disaster scenarios.

Given the broad category of “pentest” and the different testing methodologies followed by security consulting groups around the globe, prospective clients of these services should ensure that they have a clear understanding of what their primary business objectives are. Compliance, risk reduction, and attack simulation are the most common defining characteristics driving the need for penetration testing—and can typically align with the breakdown of the various pentest service definitions.