Yvonne Enselman

Advanced Security Testing: Processes Part 1

As in so many other areas of testing in general as we get deeper into the process of security testing, we focus on specific areas. Definition, planning, design, execution, evaluation, and maintenance are the natural next steps as security testing is implemented strategically.

First, we need to define what we need to do in order to determine how to do it.

This is also a lifecycle activity. Implementation of practices and proper validation of them are needed throughout the project. This is an area where alignment between development types and testing objectives is crucial. The organization’s testing risks and needs will be unique to the nature of the company, the software development process, and the business risks. Then we align the testing process to the particular lifecycle model accounting differently for sequential, iterative, commercial off the shelf (aka 3rd party) and open source.

Security test planning needs to focus on 2 aspects; verify the designed security defenses are implemented and function, as designed and verifying no vulnerable, are introduced.

We need to identify who should perform the testing. What is the schedule and is the information being used to determine it realistic. What tasks are involved and how long will they take. What environment will this be testing in and does it mimic production closely enough to be accurate. Lastly what authorizations are needed.

Design phase has different ways to be approached. Risk analysis, threat model, or ad hoc origin categorization of risks are all valid basis and depending on the type of project of these may be needed. Attributes needed are prioritized by risk and threat models, traced to requirements, known intended audience, define known security defect profiles, and automation.

The execution needs to closely analyze the environment more than in other areas of testing.

Isolated from other environments and the malware risk is crucial. Completeness includes systems and applications under test, operating systems, networking, middleware, client/server configuration, mobile concerns, databases, access rights, browsers and plug-ins, co-existing applications, data. Planning and approval is also heightened in this area. Compliance and regulatory laws plus the business risk of an improperly identified intrusion attempt can be devastating.

Evaluation reports need to be as detailed as possible and this is an area reporting cannot be overlooked. All of this feeds into the maintenance of these tests. Given all the work involved they need to be reused and to evolve with the …

Advanced Security Testing: Planning Strategically and Determining End Goals

As I have mentioned many times, there is no such thing as exhaustive testing and one of the main principles of testing is QA shows defects are absent from the cases run not that no bugs exist. However, without proper testing and analysis, we can be certain we have no protection from attacks. What is imperative is that we perform as much testing and validation as possible within the constrains of our business processing and activities. Importantly, the protocols discussed in this series of articles show due diligence and attention to the issues involved which by being prioritized help ensure security is in place as verified by the testing.

Security is considered a specialist role, and while it is getting more visible on the IBMi side it still isn’t the first area infosec teams are concerned with.


Part of this is due to the securable nature of the system. However, as we know, the box doesn’t ship secure. We need to have the correct system values and authority limitations in place. Further, like every interface, the IFS is a vulnerability. So we first have to analyze the context of our systems. Ironically, PC systems with less critical data are frequently more tested as the security risks are well known, while the I with the mission imperative might be overlooked. Determining the testing effort needs to be based on the importance of the components. Lastly, security needs to baked into our systems, patching after the fact will never be as reliable as the innate settings.

Once we have aligned effort with the context of the system importance we look at our actual objectives.


This is an area where the difference between information assurance and actual testing is important to understand. To be clear; information assurance is, “measures that protect and defend information and information systems by ensuring their availability, integrity, authentication, confidentially, and non-repudiation. These measures include providing for restoration of information systems by incorporating protection, detection, and reaction capabilities.” [NISTIR 7298]. Whereas Security Testing is, “A process used to determine that the security features of a system are implemented as designed and that they are adequate for a proposed application environment.” [MDA1]. What does this mean in the real world? IA (aka Information Assurance) are the rules that pertain to everything in our environment, the specifications, and requirements we work from, and what we base our designs and …

Advanced Security Testing: Moving to Standardized Security Testing

We have been talking about routine vectors that can be exploited to access any system, including IBM via the IFS. Importantly, as environments have become more multifaceted other areas of the business may have vulnerability. Protecting the organization requires we have a testing approach that accounts for all the factors that can breach our security. In the next set of articles, we are going to look at advanced security tests and how to organize them.

It is important to always begin with assessing what we care about and why, and how we are doing things today as well as what we have identified as existing procedures.


Testing is all about mitigating risks, since we can never eliminate everything entirely, and identifying the same to the stakeholders so relevant decisions can be made. Building on the general risk management covered earlier we move into security risk assessments. Start with what is important to the business, the mission, function, image, and reputation. An ironic fact of QA is some of the issues that have the least impact on the actual business and customers are the most embarrassing and ones that no one ever sees are the most damaging. We want to know what can hurt our assets, individuals, other entities, etc.


Next, we need to link these areas to loss of confidentiality, integrity, information, or system access.


Organizing the above would involve a matrix combining risk on a scale of impact and the importance of the artifact. From there we move into existing security policies and procedures. Knowing what we believe we are protecting and preventing, and proving that correct is tremendously important for a baseline.  The business needs to define things such as acceptable use, minimum access, network access, remote access, internet access, managing users, data protection and classifications, configuration and change management, server security, mobile, guests, physical security, and user practices (don’t put passwords under your keyboard), passwords policy (long beats difficult and changing often isn’t proven to make an appreciable difference), malware protection, incidents, auditing, software licensing (out of date software is a known danger), electronic monitoring and privacy, security procedures. The list seems daunting, however, if we look at that last sentence as a way to organize our efforts the overall test plan starts to look like an elephant we can eat, albeit one bite at a time.

Next entry portion is the auditing component.

Load and Stress Testing for Security Concerns

Continuing our discussion of security testing infrastructure, we are up to Denial of Service attacks. Since this focuses on resource depletion, load and stress testing is the primary proactive technique to counter.


Interestingly, it is easy to confuse these two, or at least it was for me.


Load testing is evaluating the behavior of the system with increasing load, looking at parallel users and transactions. There are 2 ways to look at this type of testing. First, we can steadily increase what we are simulating mimicking peak usage. Second, it can be very difficult to load tests given the differences between test and production environments and the limitations of testing. Especially sans automation tools.


Think realistically about the ebb and flow of users and processes on your systems. There are excellent performance tools on IBM i that can indicate this for you. From there, scale your tests to the size of your environments in an equal ratio test to production. Again, for Load Testing, we want to simulate both the user/transaction LOAD and the increases you see in production in relevant percentages. Stress testing on the other hand is conducted to evaluate a system or component at or beyond the limits of its anticipated or specified workloads, or with reduced resources.


Now with LPARs being common and the ability to have separate but equal environments, this type of testing has become far easier in the last 15 years. However, the tests need to be designed to push the infrastructure to the limit of both their intended use and what could happen if your business goes crazy. I doubt companies making PPE thought they would need to ramp up the way they did in 2020. Another way to differentiate between these 2 test types, both of which are important, is load testing looks at what the system does via application development and stress testing looks at how it is done from a configuration/admin POV.


Of course, DoS attacks range from the prank variety to governmental or business espionage. It is easy to see how disruption can hurt a business or reputation. However, we also know that at times of responding to chaos, organizations are less likely to catch malevolent behavior, back door insertion, unauthorized access, modifications that haven’t gone through proper change management, etc.


This can be frustrating for quality assurance professionals. Having an exact replication of the production system and

Security Testing for Denial of Service Attacks

Security Testing is a component of reliability verification and nothing is more viable as a failure than when systems can’t be accessed. One interesting and frustrating fact about this threat is that it has been used as pranks all the way through the first stage of major threats to governmental agencies. The intent of these attacks is to cause resource depletion to a website causing it to fail. Unfortunately, it doesn’t need to be complicated to be successful and can bring a business down quickly without specific intrusion that can be monitored for. Much like malicious data encryption, the hacker doesn’t need to want to do anything with your system. If they can prevent you from running your business, you need to immediately deal with them and the threat.…