CyberBench

When we created Tanuki we didn’t expect we would use it as our own tool daily. It’s an indispensable pipeline that we can use to process our thoughts, and simplify repeatative documentation processes. Its most impressive feature is its workflow pipeline.

But while we were beta testing we were speaking to some experts in the Cyber-Security space who wanted a local inference solution. The feeling within the space is using hyperscale AI wont be an option for much longer, for the obvious geopolitical and technical reasons.

So we decided the third branch of the Tanuki platform would be CyberBench. While it’s currently in its MVP stage and being beta tested we are thrilled by what we see it’s already capable of.

Local inference across logs is hard

The first step in running an inference task is converting the userspace content into the interconnected matrix of tokens that the machine learning models can actually process. This is a very expensive task and is the cause of slow Time to First Token metrics in most local inference solutions.

So we build the MLX-Prompt-Cache module and open sourced it, CyberBench is different to Tanuki Desktop, so we have a pre-heat process built in that takes the evidence that the analyst needs to review. By doing this we reduce the Time to First Token to less than a second, this makes working with a local model seem significantly faster than a cloud based alternative.

You don’t need to work with us to benefit from this, you can just use our module inside your application.

Provenance is even harder

The Australian Government has announced that it will move ahead with mandatory reporting rules for Inference. We saw this coming and are close to V1 of our CyberBench Provenance Stack. We used CycloneDX to create a set of Bill of Materials documents and Attestations that allow us to cryptographically seal and prove how Machine Learning was used in case analysis.

We’ve used various ISO standards on collecting and processing evidence to build this tool, and are confident that we will be one of the first solutions that can meet a courts evidenciary bar.

This work is ongoing, and we will share our progress in our blog. If provenance is on your radar or roadmap but you don’t know where to start, get in touch.