CyberBench
CyberBench is an AI assisted workbench for investigation and incident report analysis where all inference processes need to occur locally, avoiding pushing sensitive data to the cloud or your competitor. Further, it is important that any findings that have been generated using Machine Learning are clearly marked as such. CyberBench uses CycloneDX BOMs and Attestations to preserve all the metadata, decisions, prompt inputs and outputs, digitally signed to ensure the chain of custody is maintained during and after an investigation.
CyberBench is the security-domain workbench in the same platform as Tanuki; same inference engine, same workflow runtime, a different user experience and a much stricter record.
What a case looks like
Every artefact in a case is hashed, timestamped, and stamped with the method that produced it.
When working on a case you simply pull the repo containing the evidence or create one fresh and then drag in your evidence. The evidence then is pre-processed in the background to allow lag free responses when interrogating the data. Audio, text and documents are processed down to structures that help accelerate your work.
When you have found an insight in the data that insight is tagged and signed and prepared for managerial review. Once approved the insight becomes a finding that can be added to a final report. Every inference run is logged hashed and signed, allowing other users to see what models were used, what configuration parameters were set and what the precise prompts and outputs were.
Reading what the scanners give you
Turn giant vulnerability reports into actionable intel in seconds.
Cyber incident scanners provide rich and thorough documents that describe the state of your software and systems, but their complexity can make it difficult to quickly focus on the problems at hand. CyberBench lays out the important information that drives decision-making, using machine learning to provide human language summaries of the situation and severity to cut through the noise.
Our technical assets; our apps, our container images, websites, everything we build is constantly exposed to all manner of cyber risks. And a rich and thorough ecosystem of scanners exists to continually ensure that our technology can withstand the assault.
But these reports are very difficult to digest, and there are no standards so understanding a Trivy report doesn’t really help you understand an AWS Inspector report. This is where cyberBench reporter shines. Drag in your scanner results, optionally enrich them with lock files and BOMs and let your local, private inference summarise the results.
See in seconds which problems need attention right now, then dive through the detail visually rather than squinting at 30000 lines of JSON.
Why it runs on your hardware
It’s easy to fall into the trap of thinking local ML inference is not up to tasks frontier models are doing, but this turns out not to be true in all cases. You don’t hire a new BDM and expect them to water the plants or do the annual performance reviews, call us crazy, but we think the same applies to the software that drive our businesses.
Firstly, in this model your inference runs without any token metering. You can run inference tasks on your own hardware all day without any risk of bill shock.
Secondly, smaller models are often superior to frontier models when it comes to literal text processing. We can tune our models locally with far more precision than ChatGPT will let you. We can do things like turn down the ‘temperature’ of the responses, helping to keep the model focused only on the content that is passed to it.
We discovered this while designing Tanuki, we found that small, open weight models often do a better job of document summarisation than hyperscale models.
Finally comes energy consumption. We’ve load tested the Tanuki Network Appliance under serious adversarial conditions, the findings show that Apple Silicon can perform inference tasks continuously, all day, without thermal runaway and all for the same energy consumption as a single light-bulb. Who said AI and ML workloads needed to literally boil the ocean?
Asking again is nearly free
The first step in running an inference task is converting the userspace content into the interconnected matrix of tokens that the machine learning models can actually process. This is an expensive task, and it is the cause of slow time-to-first-token in most local inference solutions.
So we built the MLXPromptCache module and made it open source. CyberBench adds a pre-heat pass over the evidence an analyst is about to review, so the model keeps what it has already read and never re-reads it.
During our testing we created complex cases of log files, the most token dense format and found that on an m4 Max Studio, the time to first token for a pre-warmed case is always under 2 seconds.
You don’t need to work with us to benefit from this. The module is open source, and you can use it inside your own application.
Nothing happens to a case off the record
The case lifecycle explorer at sec.hypermedia.au, running the verifier over a worked case.
We used CycloneDX to build the bill-of-materials documents and attestations that record how machine learning was used in a case, and various ISO standards on collecting and processing evidence to shape the custody model underneath it.
The model is public, and so is the verifier. You can walk a worked case, or drop your own BOMs and attestations in, at sec.hypermedia.au/lifecycle. It is the same supply-chain thinking as Your Stack, pointed at evidence instead of deployments.
What it deliberately doesn’t do
CyberBench is not a scanner, not a SIEM and not a detection engine. Once a model has been pulled to your workstation, no network access is required, no data is pushed anywhere.
We believe your work belongs to you, so our tool sends no data back to us for any reason at all.
Where we are at
CyberBench has moved out of MVP status currently is approaching V1 status.
The case management capabilties are complete and allow a user to interrogate case evidence locally with their choice of models. Case closure is nearing completion, with the BOM Spec complete and operational.
CyberBench reports is approaching V1 status, we currently scan Trivy, Nessus AWS Inspector with Google SCC reports currently being implemented.
Get ahead of the competition, start analysing your environments and reports locally with your choice of open models and no token metering.