precision
How we measure precision.
A false alarm costs the team's time and its trust in the tool. That is why precision is codafort's first measure: of the alerts it raises, how many are real problems. Every release is measured before it is published.
what we measure
Two measures, in this order.
Precision
Of the alerts codafort raises, how many are real problems. It is the measure that cannot get worse from one release to the next.
Recall
Of the flaws that exist in the code, how many codafort finds. It grows from release to release, and only what costs no precision gets in.
where we measure
On cases other people wrote, with the answer already known.
Public tests
Test suites maintained by the security community, in which each case already says whether it has a flaw, separated by language.
Held-out cases
Part of the cases is never used to tune codafort. It shows whether an improvement holds beyond the cases we already know.
Real flaws already fixed
The code of public vulnerabilities in real projects, before and after the fix.
when we measure
With every release, before it ships.
The measurements run on the whole release. If precision drops on any suite, the release is not published until the cause is fixed.
what the number does not say
Precision measured with every release.
Pre-launch: codafort is not available to install yet. Join the waitlist →