Automated ZAP Scans for Orchard Core Apps

ZAP is a great tool if you’re doing pentesting hands-on, but for us at Lombiq it’s even more valuable that it can be thoroughly automated. This is why we’ve just released a tool to run ZAP security scans automatically as part of a UI test suite for Orchard Core apps.

Orchard Core and ZAP

Orchard Core is an open source ASP.NET Core web framework and Content Management System. It’s a great one, as well, but due to spending the last 12 years with it, I may be a bit biased. Anyway, if you’re a .NET web developer, I highly recommend you check it out so you don’t need to reinvent the wheel in your own projects for things like content/user/media management, audit trail, workflows, and much more.

So, where do ZAP and Orchard Core meet? Now, as part of the open source Lombiq UI Testing Toolbox for Orchard Core you can run ZAP scans for Orchard Core apps, right in a UI test suite, simply by using the .NET configuration API or completely custom Automation Framework plans even.

Running ZAP from a UI Test

This is what such a ZAP scan-running UI test looks like, written in C#, using the xUnit testing framework:

[Fact]
public Task BasicSecurityScanShouldPass() =>
    ExecuteTestAfterSetupAsync(context =>
        context.RunAndAssertBaselineSecurityScanAsync());

Not much to it, right? The ExecuteTestAfterSetupAsync method sets up the test app, and then runs the test itself. The RunAndAssertBaselineSecurityScanAsync method is the one that runs the ZAP Baseline scan. It’s a one-liner (well, broken into multiple lines for readability here), but it’s also possible to run a completely custom Automation Framework plan or use the .NET configuration API to configure various properties of the plan:

[Fact]
public Task SecurityScanWithCustomConfigurationShouldPass() =>
    ExecuteTestAfterSetupAsync(
        context => context.RunAndAssertBaselineSecurityScanAsync(
            configuration => configuration
                .ExcludeUrlWithRegex(".*blog.*")
                .DisablePassiveScanRule(
                    10037,
                    "Server Leaks Information via \"X-Powered-By\" HTTP Response Header Field(s)")
                .DisableScanRuleForUrlWithRegex(
                    ".*/about",
                    10038,
                    "Content Security Policy (CSP) Header Not Set")
                .SignIn(),
            sarifLog => sarifLog.Runs[0].Results.Count.ShouldBeLessThan(34)));

This one is a bit more involved. It configures the scan to exclude URLs containing “blog”, disables a passive scan rule, disables a scan rule for a specific URL, and also signs in to the app with a test user before running the scan. Finally, it also has a custom assertion (instead of the default “no alerts”) to verify that the scan finds less than 34 issues. As you may have guessed from the sarifLog variable name, the assertion uses the .NET representation of the SARIF report generated by ZAP. However, a human-readable HTML report is also generated by default.

Behind the scenes, ZAP is run in a Docker container. So, the tests can run equally well, without any manual installation, under all major desktop operating systems, as well as in CI systems like GitHub Actions.

Conclusion

This was just a short teaser, but be sure to check out the UI Testing Toolbox’s security scanning documentation, because we tried to summarize everything necessary to get you going there, including samples that you can just copy-paste.

ZAP, integrated with the UI Testing Toolbox, now ensures that the apps we build for our clients and ourselves are secure. It’s also an important addition to the toolbox of the Orchard Core community; perhaps we’ll run ZAP scans for Orchard Core itself too. So, we couldn’t be happier with it.