Utilizing unit testing frameworks as a vulnerability scanner
Understanding the problem
Finding vulnerabilities is a complex process and what's more complex is exploiting the vulnerabilities across multiple targets in different configurations. This is the reason there are many frameworks like Metasploit and different vulnerability scanners like nuclei exist. So many frameworks and tools yet exploit development is still done in simple scripts written in different languages like python and golang. Why is that? The answer is simple, flexibility.
Writing own scripts is a better choice obviously and security researchers enjoy writing code in python and golang these days. However there are a few problems with just plain scripts. For example, when you have hundreds of scripts and you gotta find one specific script to exploit one part based on the detections. For this, I have seen security researchers writing their own tools having simple functionalities like detect and exploit methods and then run the code in multiple threads. I think you might have tried this. Maybe you already have this setup or maybe you are utilizing Nuclei to accomplish the same. This is easy and gets job done indeed, but is there a better way? Yes. there is. Unit testing.
What is Unit testing and Why use it for vulnerability scanning?
Unit testing is a methodology to test small units or individual piece of code. You can test a small function and check if the result is as expected or not. Exactly what a vulnerability scan does. Run a small piece of code, check if it is correct result or not.
The reason why you should be using unit tests instead of a vulnerability scanner, is simple. You get the flexibilty of writing code in your choice of programming language as well as the flexibility of system level operations. You can also simply import and test a small piece of code. You can do web pentesting with headless browsers and HTTP client libraries, you can do binary exploitation using system level functions, you can also do static analysis and dynamic analysis over code. If you are write your own scripts, you will have to write a multithreaded application that can load scripts as an addon/plugin/extension which in itself is a time consuming project. The best part of using unit tests for exploit detection is that you can share the code with the developers easily for regression testing (the idea of testing code again and again with each iteration of development so that old code does not break). This kind of flexibility is hard to accomplish in vulnerability scanners.
So how does unit testing works?
In unit testing, we basically have an existing code that needs to be tested. We add a testing framework and write a class or function called a "Test" that has methods to assert some assumptions. Then you can use the testing command like npm test, python -m nose2, go test, dotnet test or anything you may have in your choice of test framework. If you need to learn or brush up your unit testing skills, find the appropriate tutorials for your choice of language and framework. Here are some good tutorials:
From here on, I will focus on xUnit as I am using it as my vulnerability scanner for a while. But you can find ways to accomplish the same in your choice of framework easily with some searching on the internet.
xUnit as vulnerability scanner
To build a vulnerability scanner out of unit testing needs two things only. A way to run code multithreaded and a way to log things. Aside this, just a simple project to keep all the code. Create a new xunit project with,
dotnet new xunit -o VulnScanner
It should create a new directory with a proj file and a basic Unit test called UnitTest1.cs.
Now lets make it multithreaded. In our project, create a new file called xunit.runner.json with the following content:
{
"$schema": "https://xunit.net/schema/current/xunit.runner.schema.json",
"parallelizeAssembly": true,
"maxParallelThreads": 40
}
Here in maxParallelThreads you can define how many threads you want to use to run the tests. $schema is optional and only helps in code completion and highlighting in Visual Studio (Code). Now, we gotta tell compiler to copy our configuration to build directory. Open your proj file and add the following in <ItemGroup> tag
<Content Include="xunit.runner.json" CopyToOutputDirectory="PreserveNewest" />
Your proj file should look like this by now:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net8.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<IsPackable>false</IsPackable>
<IsTestProject>true</IsTestProject>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.6.0" />
<PackageReference Include="xunit" Version="2.4.2" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.4.5">
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
<PrivateAssets>all</PrivateAssets>
</PackageReference>
<PackageReference Include="coverlet.collector" Version="6.0.0">
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
<PrivateAssets>all</PrivateAssets>
</PackageReference>
<Content Include="xunit.runner.json" CopyToOutputDirectory="PreserveNewest" />
</ItemGroup>
</Project>
And we are done. You can now start writing your code as tests in UnitTest1.cs or any other test file and it will run multithreaded.
dotnet test
Now there is one more thing to do, logging. It does not matter where you log. You just need to capture the log output of xUnit and then use it however you like. Capturing is quite simple and you can still ship the test. We capture the output by injecting the ITestOutputHelper into our test's constructor.
using Xunit;
using Xunit.Abstractions;
public class UnitTest1
{
private readonly ITestOutputHelper output;
public UnitTest1(ITestOutputHelper output)
{
this.output = output;
}
[Fact]
public void Test1()
{
string target = Environment.GetEnvironmentVariable("TARGET");
getallUrls(target);
Assert.Contains("/haproxy-status", target);
output.WriteLine("Some logging here");
}
}
Writing to output will send your log to the debug logs of xUnit which you can read with the following command.
dotnet test --logger "console;verbosity=detailed";
If you want to log to some other source, you can build a logger. However, a logger might introduce a third party dependency so it is best done in an abstract class and let your test extend it. While distributing your exploit, you can simply remove the inheritance to remove this logging dependency.
Another problem that you might face is inputs. But it is quite easy with environment variables.
using Xunit.Abstractions;
namespace ScanV;
public class UnitTest2: Test
{
public UnitTest2(ITestOutputHelper output) : base(output) { }
/// <summary>
/// Second test
/// </summary>
[Fact]
public void Test2()
{
string? target = Environment.GetEnvironmentVariable("TARGET");
if(target != null) {
output.WriteLine($"Attacking {target}");
}
}
}
In bash like shells you can pass the input to above test with
TARGET="example.com" dotnet test --logger "console;verbosity=detailed";
In powershell,
& { $env:TARGET="example.com"; dotnet test --logger "console;verbosity=detailed"}
and now you can use it as vulnerability scanner. Only catch here is that you will need to fail a test to confirm a vulnerability and pass a test means not a vulnerability. I will add more things to this article soon but you got the main idea. Have fun scanning.