Coverage
A source code audit reports what it tested for, not only what it found.
While the agents analyze your code, they also walk a methodology checklist: a list of security checks a reviewer would work through on an application like yours. Every check on that list gets an answer in the report, and a finding is only one of the answers it can get.
Why this exists
Before this, a report could show nothing for a whole class of vulnerability without telling you whether anyone had looked. An empty section read the same way in three very different situations: the class was checked and your code is clean, the class does not apply to your application, or the class was never examined at all.
Coverage separates them. The absence of a finding and the absence of a test are now different lines on the page.
The five answers
Your report carries a Testing Methodology section: one row per check, with the answer it got. These are the five answers a row can carry.
| Answer | What it means |
|---|---|
| Tested | At least one audit task examined your code for this check. Findings, where there are any, appear in the findings section. |
| Analyzed | The audit's code reading and architecture mapping answered this check. That work runs on every assessment. |
| Not applicable | Your codebase does not contain the surface this check targets. |
| Requires runtime | The check needs a running application, which a source code audit does not start. |
| Not completed | The audit did not settle this check. The report claims nothing for it. |
Put plainly: a check is either examined here, answered by the reading the audit does anyway, out of scope because your application has nothing to test, out of reach because source alone cannot settle it, or openly unanswered. Nothing on the list is left blank.
The section opens with the totals across all five, so you can see the shape of the walk before reading a single row.
When "Not completed" appears
Two things produce it, and both are the report refusing to overclaim.
An audit task can fail. The run records that one failed and not which checks it was carrying, so a single failure anywhere pulls every answered check down to Not completed for the whole report, and a note under the totals says so. That is deliberate: the task that died may be exactly the one that would have found something, and a report cannot tell you which.
The walk itself can also leave a check unanswered. The audit goes back over what is left, and keeps going while each round closes at least one more check. The moment a round closes none, the rest are recorded as unsettled rather than quietly dropped.
The checklist is always there
Four methodology packs run on every code audit. They cover web applications, mobile applications, infrastructure as code, and compiled binaries.
AISafe does not ask you to choose between them, because real projects are mixed. A mobile client talks to a web backend, the backend deploys through Terraform, and somewhere in the tree there is a compiled component. Picking one pack up front would mean deciding what your application is before reading it, and getting that wrong is how a whole surface goes unexamined.
So every check runs, and the ones that do not fit your application come back as not applicable. They are answered, not hidden.
Where a whole pack turns out to have nothing to test, its rows collapse to one summary line with the counts, rather than padding a backend report with a hundred inapplicable mobile rows. The counts still state the whole truth.
What "not applicable" means
It means the audit looked for the surface a check needs and your application does not have it: the repository ships no native code for a binary check to examine, say.
It never means the audit ran short of time, budget, or patience. That is what Not completed reports, and the two labels are kept apart on purpose.
Checks that code cannot answer
Some checks depend on how a deployed system behaves rather than on what the source says. Reading code cannot settle them, however carefully you read.
The report marks such a check Requires runtime rather than passing over it in silence. That tells you where a code audit stops. If those checks matter to you, follow the audit with a service that touches a live target: a Black-Box Pentest or a White-Box Pentest.
Regulated regimes are opt-in
When you start an audit you can tick one or more regulated regimes: the General Data Protection Regulation (GDPR), the Health Insurance Portability and Accountability Act (HIPAA), or the Payment Card Industry Data Security Standard (PCI DSS).
Ticking one adds two things at once: that regime's checks, and the recognition ability that makes them answerable. A check that asks where personal data travels arrives together with the ability to recognize personal data in your code. You cannot select one without the other, and there is no reason to want to, because either half alone answers nothing.
Leave all three unticked and the audit still runs the four always-on packs.
What a regulatory pack adds
Ticking GDPR does not turn the run into "a GDPR audit". It adds checks anchored to the technical articles a code audit can speak to:
- security of processing
- erasure of personal data
- integrity and confidentiality
- data protection by design and by default
It deliberately leaves out the obligations answered by a document rather than by code. Staff training, processor contracts, and breach-notification timelines are real duties under the regulation, and a code audit cannot read any of them. Your report will not mention them, and no result in it should be read as covering them.
HIPAA and PCI DSS work the same way. The checks speak to the technical controls that live in your source. The organizational and procedural controls sit outside what reading code can reach, and the audit says nothing about them.
What the scan read
Coverage has a second half: before any check runs, the audit reports how much of the attached source it actually read.
Most files a repository holds are not source code — images, documents, build output — and skipping them is correct, so they are never reported as a gap. What is reported is source the scan should have read and could not: files too large to admit, files that would not parse, files the walk could not open.
- If almost nothing was lost, the results page notes it in one quiet line.
- If enough was lost to change what you received — say a tenth of your source files — the results page says so prominently, with the exact counts.
- If the attached source holds no readable code at all, the audit is refused before it starts and before anything is charged, and the message says what was found instead so you can attach the right branch or archive.
A repository of Terraform or other configuration is still a real audit target: configuration is analysed for infrastructure, dependency, and secret issues even though it carries no application code.
What this is not
An audit is not an assessment of compliance, and AISafe is not an assessor.
What you get is narrower and more useful than that: findings mapped to control frameworks (see Compliance mapping), plus a record of which methodology checks the audit walked and what each one answered. That record is evidence you or your auditor can read, question, and check against the code.
It is not a judgment about your application's standing under any standard, and nothing in it supports one. Deciding that question is your auditor's job, and the coverage record is one of the things they can read while they do it.
Standing rules
Every audit also runs against a set of standing rules — guidance our security team writes about how each class of weakness actually shows up in source code. Each run records the rules assigned to its stages and the rules cited by the work it produced. These records help review the audit's coverage. Assignment alone does not prove that a stage finished or that a rule was applied.