Skip to content

Python API

import complydoc as cd

Everything on this page is public. Anything in the package that is not on this page is internal.

class Observation

An open, then finished, cd.observe block.

After the block: report is the report, path where it was written, and summary() a few lines on what each stage did. error says why there is no report when observing itself failed.

Field Type What it is
location str | None = None Where the report is: its path here, or its address in a bucket.
summary() -> str

What each stage did, a line each, and where the report was written.

class StageIdentifier

An identifier in what a stage passed on, or, for an embedding stage, sent.

Field Type What it is
fingerprint str —
label str —
masked str —
severity str —
occurrences int —
evidence str = 'pattern' As a match’s: model where the name model found it, which a stage scanned by patterns alone could not have, so it is not missing from such a stage.
items list[int] = field(default_factory=list) For a vector store’s step, which of the items it stored hold this identifier, as positions in the step’s ids: what to delete to take it out of the index.
class Trace

Schema 17, with schema 18’s documents, warnings and tracebacks on each stage: an ingestion pipeline observed with cd.observe, stage by stage.

Field Type What it is
name str —
scan str patterns, full or off.
seconds float The block’s wall time.
overhead_seconds float What observing cost: reading settings as stages ran, and scanning at the end.
stages list[TraceStage] = field(default_factory=list) —
kind str = 'pipeline' pipeline, observed with cd.observe. The viewer makes an audit’s own trace from each document’s timing, with the kind audit.
connections_outside list[str] = field(default_factory=list) Lookups and connections made in the block but outside any stage.
libraries dict[str, str] = field(default_factory=dict) The libraries whose classes were observed, with their versions.
error str | None = None What the block raised, if it did.
unscanned dict[str, str] = field(default_factory=dict) Schema 19: the kinds of identifier the run could not look for, by label, with why: names, where no name model is installed. They are neither counted nor masked.
class TraceStage

One step of an observed ingestion pipeline: a loader, a splitter, a transformer, an embedding call, or a function of the caller’s own marked with @cd.stage.

Field Type What it is
index int —
kind str load, transform, split, embed, store or custom.
component str The class, or the function’s name.
module str —
method str —
seconds float Time spent inside the stage, not counting what the caller did between items of a stage read lazily.
tags list[str] = field(default_factory=list) —
parameters dict[str, Any] = field(default_factory=dict) The component’s settings, such as a splitter’s chunk_size, read from its attributes. Values that look like secrets are left out and long text is cut.
documents_in int | None = None —
documents_out int | None = None For an embedding stage, the texts it was given are its input, and it has none out.
characters_in int | None = None —
characters_out int | None = None —
sources list[str] = field(default_factory=list) The documents named in what the stage passed on, as the report names them.
scanned str = 'off' How the output was scanned: full, with the name model, patterns, or off.
identifiers list[StageIdentifier] = field(default_factory=list) Identifiers in the output, or in the texts sent for an embedding stage.
hidden int | None = None Hidden or instruction-like passages of medium severity or above. None unscanned.
metadata_keys list[str] = field(default_factory=list) —
metadata_keys_added list[str] = field(default_factory=list) —
path_keys list[str] = field(default_factory=list) Metadata keys whose values are absolute file paths.
connections list[str] = field(default_factory=list) Lookups and connections made while the stage ran.
hosts list[str] = field(default_factory=list) —
ids list[str] = field(default_factory=list) For a vector store, the ID it stored each item under, in the order it was given them. An ID that holds a path or an identifier is kept as an empty string, so the report does not carry the value; empty where the store gave none.
vectors int | None = None —
dimensions int | None = None —
chunks int | None = None For a split stage, its place in the report’s chunks.
parent int | None = None The stage this one ran inside: a directory loader’s loader for one file, say.
started float = 0.0 Seconds from the start of the block to when the stage was called.
tokens_in int | None = None —
tokens_out int | None = None Tokens in what the stage was given and passed on, counted with the headline model’s encoding; for an embedding stage, tokens_in is what it sent.
usd float | None = None What the stage cost, where it sent text to a priced model: an embedding call.
usd_basis str | None = None estimated from the price table, or unpriced for a model the table does not list.
previews list[StagePreview] = field(default_factory=list) —
finished bool = True False for a stage read lazily whose items were not all taken before the block ended.
error str | None = None —
traceback str | None = None Schema 18: where the error was raised, file paths shortened and identifiers masked.
documents list[StageDocument] = field(default_factory=list) Schema 18: what the stage handled of each document.
warnings list[StageWarning] = field(default_factory=list) Schema 18.
observe(
name: str = 'pipeline',
out: str | os.PathLike[str] | None = '.complydoc',
scan: Scan = 'patterns',
reveal: bool = False,
config: Config | None = None,
extracted_text: bool = True,
models: Sequence[str] | None = None,
previews: int = 20,
) -> Observation

Observe the ingestion pipeline run inside the with block.

name names the pipeline: its runs are grouped under it, as an experiment’s are. The report is written to out, .complydoc in the working directory by default, as <name>-<time>.json; out=None keeps it in memory only. out may be a bucket folder, s3://bucket/folder, with the s3 extra installed: the report is then copied there, and location says where it went.

scan sets what is looked for in each stage’s output when the block ends: patterns, the default, reads identifiers by pattern at every stage and runs the name model on the documents loaded and on the last stage; full runs everything at every stage; off records counts, settings, timings and connections only.

extracted_text keeps each loaded page’s text, masked, so the viewer can show it with the chunks drawn over it. previews keeps that many of each step’s items, masked, so the viewer can show what the step passed on; 0 keeps none. A library is observed wherever it is imported, before the block or inside it.

stage(
name: str | Callable[..., Any] | None = None,
kind: str = 'custom',
) -> Callable[..., Any]

Mark a function of your own as a stage of the pipeline.

@cd.stage("dedupe")
def dedupe(documents): ...

The first argument, when it is a list, is taken as what the stage was given, and a list returned as what it passed on. Library calls made inside it are part of it, not stages of their own. Outside a cd.observe block the function runs as it is.

aiter_audit(
target: str | os.PathLike[str],
components: Sequence[str] = COMPONENTS,
**options: Unpack[AuditOptions],
) -> AsyncIterator[DocumentReport | SkipRecord]

iter_audit for asynchronous code, reading each document in a worker thread.

check_facts(
report: AuditReport,
facts: Iterable[Fact | str],
threshold: float = FUZZY_THRESHOLD,
) -> list[FactCheck]

Whether each fact appears in the report’s extracted text.

The report must have been produced with extracted_text=True. Results are keyed by the loader’s name for loader output, and by report otherwise.

compare_chunkers(
chunkers: Mapping[str, Any],
documents: Any,
**options: Any,
) -> ChunkComparison

inspect_chunks for each named splitter on the same documents.

compare_loaders(
loaders: Mapping[str, Any] | Sequence[Any],
config: Config | None = None,
components: Sequence[str] = COMPONENTS,
reveal: bool = False,
extracted_text: bool = True,
models: Sequence[str] | None = None,
allow_network: bool = False,
paths: str | os.PathLike[str] | Sequence[str | os.PathLike[str]] | None = None,
facts: Iterable[Fact | str] | None = None,
fact_threshold: float = FUZZY_THRESHOLD,
cache_dir: str | os.PathLike[str] | None = None,
verify_with: str | VisionModel | None = None,
verify_scope: str = 'flagged',
formats: Mapping[str, Iterable[str]] | None = None,
page_images: bool = False,
) -> AuditReport

Run several loaders on the same input and report where their output differs.

loaders maps a name to a loader, or is a sequence of loaders, each named after its class. Anything inspect_documents accepts as a source is accepted as a loader, including documents already loaded. The first is the baseline the report’s findings are built from.

The comparison is in report.loader_comparison. Each document’s extractions lists every loader’s reading of it, first the baseline’s, and with extracted_text on, each page carries the other loaders’ text in readings.

allow_network applies to every loader. Each one’s connections are recorded in its row of report.loader_comparison.loaders.

With paths, a folder, a file or a list of files, each loader is a callable taking a file path, such as a loader class or a complydoc.loaders.parsers preset, and runs once per file. Files a loader raises on are recorded in its row.

formats maps a loader’s name to the file types it is meant for, as extensions (.pdf) or format names (docx), and it is given only those files. Parser presets and well-known loader classes, such as PyPDFLoader, already know theirs; any other loader is given every file.

cache_dir stores each loader’s output per file when paths is given, so a later run does not parse unchanged files again; see complydoc.loaders.cache.

facts are passages the documents are expected to contain. Each is checked against every loader’s text; see complydoc.extraction.facts.

verify_with reads the baseline’s pages again with a vision model of the caller’s, as full_audit does, rendered from the files the loaders read. The vision reading joins each page’s readings, so it can be put beside any loader’s, with what each one cost.

page_images, with paths, puts a picture of each page of every file read beside its text, rendered by complydoc from the file, with the identifiers found on the page blacked out, as full_audit(page_images=True) does. A page with one that cannot be placed has no picture; with reveal, the pictures are as the pages are.

cost_audit(
target: str | os.PathLike[str],
**options: Unpack[AuditOptions],
) -> AuditReport

Estimate what these documents would cost an LLM to read.

Per model and per architecture — the text layer, the text layer with OCR behind it, and every page sent as an image.

extract_text(
target: str | Path,
config: Config | None = None,
mask: bool = True,
reveal: bool = False,
ocr: bool = True,
recurse: bool = True,
password: str = '',
extractor: str | None = None,
model: str | None = None,
max_tokens: int | None = None,
) -> TextResult

Read a folder’s text, masked, with token counts and what went wrong.

ocr is on by default here, unlike the audit, so scanned pages are read.

mask=False returns the text as the page says it, with no scan run at all. reveal=True runs the scan and reports the counts while leaving the values in place.

full_audit(
target: str | os.PathLike[str],
**options: Unpack[AuditOptions],
) -> AuditReport

Audit a file or folder on every component.

Cost, readiness and the identifier scan, with the global readiness score and the quick wins that follow from all three.

inspect_chunks(
splitter: Any,
documents: Any = None,
name: str | None = None,
config: Config | None = None,
model: str | None = None,
min_tokens: int = 20,
max_tokens: int | None = None,
facts: Iterable[Fact | str] | None = None,
fact_threshold: float = FUZZY_THRESHOLD,
questions: Iterable[Question | Mapping[str, Any] | Sequence[str]] | None = None,
top_k: int = 5,
) -> ChunkReport

Split documents with splitter and inspect the chunks.

With documents omitted, splitter is taken to be the chunks themselves. max_tokens enables the oversized flag. Token counts use model’s encoding, or the headline model’s.

inspect_documents(
source: Any,
name: str | None = None,
config: Config | None = None,
components: Sequence[str] = COMPONENTS,
reveal: bool = False,
extracted_text: bool = True,
models: Sequence[str] | None = None,
allow_network: bool = False,
) -> AuditReport

Report on the documents a loader produced.

source is a loader, a callable returning documents, a list of documents, or a single document. A loader is run inside the network guard, and any connection it attempts is recorded in report.loader.network_attempts; a loader that fails because a connection was refused produces a report with no documents and the failure in report.loader.error. Any other exception a loader raises is not caught.

allow_network=True lets the loader’s connections through, for loaders that call a hosted service. Each lookup and connection is still recorded, the report states that network access was allowed, and everything complydoc does after loading stays behind the guard.

extracted_text is on by default here, unlike full_audit.

iter_audit(
target: str | os.PathLike[str],
components: Sequence[str] = COMPONENTS,
**options: Unpack[AuditOptions],
) -> Iterator[DocumentReport | SkipRecord]

Yield each document’s report entry as it is read, and each skipped file.

Takes the options of full_audit. The network guard is armed while a document is read and released between documents. Raises FileNotFoundError for a missing path when called.

readiness_audit(
target: str | os.PathLike[str],
**options: Unpack[AuditOptions],
) -> AuditReport

Measure how ready these documents are to extract data from.

report.documents[i].readiness.signals holds the signals. A signal that could not be measured is reported as not applicable.

security_audit(
target: str | os.PathLike[str],
**options: Unpack[AuditOptions],
) -> AuditReport

Find personal and financial identifiers, and nothing else.

Values arrive masked. report.documents[i].sensitive.matches carries each one with its severity and its evidence tier — confirmed where a checksum passed, down to model for a statistical guess.

count_tokens(
text: str,
model: str | None = None,
config: Config | None = None,
) -> TokenCount

Tokens in text for model, or for the headline model when none is given.

find_hidden(
text: str,
config: Config | None = None,
reveal: bool = False,
) -> list[ContentFinding]

Invisible characters and instruction-like passages in text.

A string has no rendering to check, so visibility is not_measured except for characters that are invisible by definition, such as Unicode tag characters.

mask_text(text: str, config: Config | None = None) -> MaskedText

text with each identifier replaced, strongest evidence first.

scan_text(text: str, config: Config | None = None, reveal: bool = False) -> TextScan

Personal and financial identifiers in text, masked unless reveal.

diff_reports(
old: AuditReport,
new: AuditReport,
score_tolerance: float = 0.5,
) -> ReportDiff

Changes from old to new. Score changes smaller than score_tolerance are ignored.

expect(report: AuditReport) -> Expectation

Start a chain of checks on report.

launch_ui(
*sources: str | Path,
port: int = DEFAULT_PORT,
open_browser: bool = True,
block: bool = False,
dist: Path | None = None,
host: str = HOST,
read_only: bool | None = None,
allowed_hosts: tuple[str, ...] | list[str] = (),
uploads: str | Path | None = None,
) -> ViewerServer

Open the report viewer on the reports in sources, served from this machine.

sources are report files or folders to search, .complydoc when none is given. The server listens on 127.0.0.1 only and makes no outbound connection. A source may also be a bucket folder, s3://bucket/folder, which is the one case where it does: it copies that folder’s reports here to serve them, and looks again as the page is reloaded. With block=False, the default, it runs in the background and the returned server’s stop() ends it, which is what a notebook wants.

host is the address to listen on. Any but this machine’s own serves the reports to whoever can reach it, with no sign-in, so the viewer is then read-only unless read_only=False says otherwise. It answers only to this machine’s names and to allowed_hosts: the name a proxy serves it under goes there, and "*" accepts every name.

uploads is a folder for documents dropped on the viewer, which it audits and then lists. It is taken only by a viewer that serves this machine and may write.

load_config(config_dir: Path | None = None) -> Config

Load the three config files from config_dir, defaulting to the shipped set.

load_report(source: str | os.PathLike[str] | Mapping[str, Any]) -> AuditReport

An AuditReport from a JSON file written by write_json, or from its parsed data.

Raises ValueError for a schema_version not in READABLE_SCHEMA_VERSIONS. Fields missing from an older report take their default values.

write_json(
report: AuditReport,
path: str | os.PathLike[str],
detail: Literal['summary', 'full'] = 'summary',
) -> Path

Write the report as JSON, and return where. Same shape the CLI writes.

detail="summary" leaves out the price of every document on every model and, unless the run asked for page images, the page geometry the viewer draws pages with, keeping the folder’s cost per model. detail="full" writes every field; use it for anything that reprocesses reports, since a summary reads back without the parts it left out.

class DropHiddenPassages

Removes invisible characters and instruction-like sentences from the text.

With drop_document, a document containing either is dropped instead. This works on text: hidden formatting in the source file is detected by an audit, not here. Invisible characters include zero-width joiners, which some scripts use.

Field Type What it is
name — = 'drop_hidden_passages' —
class MaskIdentifiers

Replaces identifiers in the text, and in string metadata values when metadata.

Field Type What it is
name — = 'mask_identifiers' —
class Step

Base class: subclasses implement apply on one document’s text and metadata.

Field Type What it is
name — = 'step' —
apply(text: str, metadata: dict[str, Any]) -> tuple[str, dict[str, Any]] | None

The new text and metadata, or None to drop the document.

class StepChange
Field Type What it is
step str —
document str | None The source named in the document’s metadata.
detail str —
class StripPathMetadata

Replaces absolute file paths in metadata values with the file name.

Field Type What it is
name — = 'strip_path_metadata' —
class Detector

Finds spans of one kind in a page’s text.

A detector may also define prepare(texts, context), called with every page of a document before the first find, to read them all ahead in one batch.

Field Type What it is
id str —
class DetectorContext

Everything a detector is given about the category it is running for.

Field Type What it is
category_id str —
config CategoryConfig —
class Engine
Field Type What it is
id str —
name str —
set_threads(count: int | None) -> None

Limit the engine’s own threading, before it reads its first page.

class Extractor

Reads the text layer of one page.

Field Type What it is
id str —
name str —
provides_tables bool —
provides_raw_chars bool —
granularity Granularity —
available() -> bool

Whether this extractor can run here at all.

class Finding

A candidate span within one page’s text. Carries no value.

Field Type What it is
start int —
end int —
confidence float | None = 1.0 The detector’s own score, or None where the detector has none to give.
context_term str | None = None Set when a generic pattern was only reported because of a nearby label.
class Loader

What every format loader must provide.

Field Type What it is
extensions tuple[str, ...] —
format DocumentFormat —
class LoaderSpec

How to build a loader for one file, and what running it involves.

Field Type What it is
name str —
factory Callable[[str], Any] Called with a file path; returns a loader or a list of documents.
network bool = False True when the parser sends documents to a hosted service.
price_key str | None = None Entry under parsers in pricing.yaml, for the cost per page.
tags tuple[str, ...] = () Framework and library names shown beside the loader in reports.
formats tuple[str, ...] | None = None File extensions the parser reads, such as .pdf, or format names such as docx. In a comparison over a folder, it is given only these files. None gives it every file.
class Measurement

What a signal returns.

Field Type What it is
value float | int | bool | None = None —
display str = '' —
detail dict[str, Any] = field(default_factory=dict) Supporting numbers, shown under the row so the value can be checked.
not_applicable_reason str | None = None —
class Signal

Implemented by every module under readiness/signals/.

Field Type What it is
id str —
name str —
unit str | None —
why str The default plain-English sentence. readiness.yaml can override it.
applies_to frozenset[DocumentFormat] —
all_engines() -> list[Engine]
all_extractors() -> list[Extractor]

Presets for document parsers, for use with compare_loaders(..., paths=...).

import complydoc as cd
report = cd.compare_loaders(
{
"pymupdf4llm": PyMuPDF4LLMLoader,
"docling": cd.parsers.docling(),
"llamaparse": cd.parsers.llamaparse(tier="cost_effective"),
},
paths="./contracts",
allow_network=True,
)

Each preset returns a LoaderSpec: a factory that builds a loader for one file, whether the parser sends documents to a hosted service, and the parsers entry in pricing.yaml used to estimate its cost per page. The parser libraries are not dependencies; each preset imports its library when it runs and says what to install when it is missing.

A hosted preset only runs with allow_network=True.

parsers.docling(
export: Literal['markdown', 'chunks'] = 'markdown',
**options: Any,
) -> LoaderSpec

Docling through langchain-docling, run locally.

export="markdown" returns one document per file; "chunks" returns Docling’s own chunks. options are passed to DoclingLoader.

parsers.opendataloader_pdf(**options: Any) -> LoaderSpec

OpenDataLoader PDF through langchain-opendataloader-pdf, run locally.

Java 11+ must be on the PATH. options are passed to OpenDataLoaderPDFLoader; its default uses the local Java engine.

parsers.markitdown(**options: Any) -> LoaderSpec

Microsoft MarkItDown through the markitdown package, run locally.

Returns one Markdown document per file. MarkItDown does not split pages, so the document carries no page metadata. options are passed to MarkItDown.

parsers.unstructured(
api: bool = False,
api_key: str | None = None,
strategy: Literal['fast', 'hi_res', 'ocr_only', 'auto'] = 'fast',
**options: Any,
) -> LoaderSpec

Unstructured through langchain-unstructured.

api=True partitions through the hosted Unstructured API. options are passed to UnstructuredLoader.

parsers.llamaparse(
api_key: str | None = None,
tier: Literal['fast', 'cost_effective', 'agentic', 'agentic_plus'] = 'cost_effective',
**options: Any,
) -> LoaderSpec

LlamaParse, hosted, through the llama-parse package.

tier selects the price entry; parsing options such as the mode are passed in options to LlamaParse.

parsers.azure_document_intelligence(
endpoint: str,
api_key: str,
model: str = 'prebuilt-layout',
mode: Literal['markdown', 'page', 'single'] = 'markdown',
**options: Any,
) -> LoaderSpec

Azure AI Document Intelligence, hosted, through Azure’s own SDK.

mode="markdown" returns one Markdown document per file, "single" one plain text document, and "page" one document per page, numbered from 1 as page_number. options are passed to begin_analyze_document, such as features.

This went through langchain-community’s loader until LangChain archived that package in June 2026. It calls the SDK that loader wrapped, with the same request, so the text is the same.

register_detector(instance: Detector) -> Detector

Add an identifier detector to this process.

A category uses it when its detector is this detector’s id; add the category to sensitive.categories, for example with Config.override.

register_engine() -> type
register_extractor() -> type

Class decorator. Instantiates the extractor once and registers it by id.

The same arrangement the signals use, so the two read alike.

register_instruction_classifier(classifier: Classifier | None) -> None

Score passages with a classifier as well as the patterns.

classifier takes a passage and returns how likely it is to be an injected instruction, from 0 to 1. A passage at or above instructions.classifier_threshold in hidden.yaml is reported at the model tier. Pass None to remove it.

complydoc ships and downloads no model. The classifier runs inside the network guard, so it has to load from files already on the machine. It is registered in this process only: audits with jobs above 1 run their documents in worker processes that do not have it.

register_loader(loader: Loader) -> Loader
register_signal(instance: Signal) -> Signal

Add a readiness signal to every run in this process.

The signal is measured and shown unrated until readiness.signals.<id> is configured, for example with Config.override.

supported_extensions() -> tuple[str, ...]
class ConfigError

Raised when a configuration file is missing or fails validation.

class ExpectationError

A report did not meet an expectation.

class LoaderError

A file could not be opened or parsed. Always caught; never fatal to a run.

class NetworkAccessError

Raised when any code attempts to open a network connection.

class UnknownModelError

A model was asked for by name and is not in the pricing config.

class AuditOptions

Everything the four entry points accept beyond the folder itself.

One set for all of them. An option that does not apply to the components being run is ignored.

Field Type What it is
config Config | None A loaded configuration. load_config() is used when this is absent.
ocr bool Read pages with no text layer. Slower, and finds what a scan is hiding.
reveal bool Put identifiers in the report in full. Off, and the report says which it was.
recurse bool —
password str —
monthly_volume int | None —
models Sequence[str] | None Price against these models instead of the configured comparison.
extractor str | None —
compare_extractors Sequence[str] —
compare_engines Sequence[str] —
sample int | None Audit this many documents, keeping each file type’s share of the folder.
jobs int Worker processes. 1 by default; 0 reads the folder and decides.
timeout float | None Seconds to give each document. A document still being read when the time passes is stopped and reported as skipped. Enforced by reading in a worker process, so a run with a timeout always uses one.
page_images bool —
extracted_text bool Keep the text read off each page. It is the document, so it is off by default.
offline_guard bool Block outbound sockets for the duration. On, and only off for a caller who knows their process needs the network while this runs.
progress Callable[[int, int, Path], None] | None Called with (finished, total, path) as each document completes.
verify_with VisionModel | str | None Read pages again with a vision model of your own, and report where it disagrees. A callable taking a VisionPage and returning a VisionReading or the text, or a "vision:module:function" spec naming a factory for one. The page images go wherever that code sends them, and the report names the hosts. A callable keeps the run in one process; a spec crosses into workers. Nothing is verified by default.
verify_scope Literal['flagged', 'all'] flagged, the default: the pages routing sent to vision, the pages with no usable reading, and the pages two readers disagreed about. all: every page.
ignore_file str | os.PathLike[str] | None Findings to set aside, each with a reason. Without it, .complydoc-ignore.yaml at the top of the folder is read when there is one.
categories_file str | os.PathLike[str] | None Identifier categories to switch off or re-grade. Without it, .complydoc-categories.yaml at the top of the folder is read when there is one.
concepts_file str | os.PathLike[str] | None Your own things to look for. Without it, .complydoc-concepts.yaml at the top of the folder is read when there is one.
judge_concepts str | Callable[[str, str, str], float] | None Put the concepts marked judge to a judgement model, page by page: "jev", which sends page text to TypeSafe, or a function of your own taking a label, a description and a page’s text and returning a probability. None by default.
class AuditReport
Field Type What it is
run RunMetadata —
documents list[DocumentReport] = field(default_factory=list) —
skipped list[SkipRecord] = field(default_factory=list) —
cost FolderCostEstimate | None = None —
aggregate Aggregate | None = None —
limitations list[Limitation] = field(default_factory=list) —
staleness_warnings list[str] = field(default_factory=list) —
signal_weights dict[str, float] = field(default_factory=dict) Printed in the report whenever a score is shown.
config_masking MaskingConfig | None = None Echoed so the report can state how much of a value is shown.
overall OverallReadiness | None = None Global readiness: content, cost and exposure combined. Both this and quick_wins are derived from a finished report, so their modules import this one. The annotations are resolved only by a type checker, which keeps the dependency one-way at runtime.
quick_wins list[QuickWin] = field(default_factory=list) What to do next, ranked. See complydoc.report.quickwins.
loader LoaderRun | None = None Set when the documents came from an external loader.
loader_comparison LoaderComparison | None = None Set by compare_loaders. loader is then the baseline’s run.
routing RoutingSummary | None = None Pages per route for the folder, and what that mix costs. See complydoc.report.routing.
verification VerificationSummary | None = None How many pages an independent vision read agreed with. None unless --verify.
ignores IgnoreSummary | None = None The ignore file this run read, and what each entry set aside. None without one.
concepts ConceptSummary | None = None The custom concepts this run looked for. None without a concepts file.
categories CategorySummary | None = None Schema 21: the categories this run looked for otherwise than as shipped. None without a categories file.
evals EvalSummary | None = None Schema 24: the evals file this run read, and what each reading kept of it. None without one.
chunks list[ChunkReport] | None = None Schema 17: each text splitter complydoc chunks ran, and what it made of the text. None for a run that split nothing. A chunks run has no per-document entries.
trace Trace | None = None Schema 17: set for a pipeline observed with cd.observe.
thresholds Thresholds = field(default_factory=Thresholds) Schema 17: the bands and the similarity line this report was judged by.
to_pandas(table: str = 'documents') -> Any

One table of this report as a pandas DataFrame.

Table names are in complydoc.report.tables.TABLES. Requires the notebook extra.

class Change
Field Type What it is
area str documents, identifiers, metadata, hidden, readiness, signals, similarity, global, facts, loaders or limitations.
kind str added, removed or changed.
subject str —
worse bool —
document str | None = None —
before Any = None —
after Any = None —
class Chunk

One piece of a document’s text, and what is known about it.

Field Type What it is
document str Path relative to the folder that was asked for.
page int —
part int 1 unless the page was split to fit max_tokens.
text str —
tokens int —
token_fidelity str ‘exact’, ‘approximate’ (a real encoding, another provider’s) or ‘estimated’.
encoding str —
masked int How many identifiers were covered over in this chunk.
masked_confirmed int How many of those passed a checksum. The difference between the two is the part of the masking that is best effort. A chunk where they are equal had nothing guessed in it.
source str ‘native’ where the text was on the page, ‘ocr’ where it was recognised.
class ChunkComparison
Field Type What it is
reports dict[str, ChunkReport] —
to_pandas() -> Any

One row per chunker. Requires the notebook extra.

class ChunkReport
Field Type What it is
chunker str —
chunks list[InspectedChunk] —
stats ChunkStats —
token_encoding str —
token_fidelity str —
flag_counts dict[str, int] = field(default_factory=dict) —
flag_meanings dict[str, str] = field(default_factory=lambda: dict(FLAG_MEANINGS)) What sets each flag, so a reader of the report words them as complydoc does.
repeated_identifiers dict[str, int] = field(default_factory=dict) Identifiers found in more than one chunk, with the number of chunks.
facts list[FactLocation] = field(default_factory=list) —
retrieval list[QuestionResult] = field(default_factory=list) One result per question, when questions were given.
top_k int = 5 —
retrieval_hit_rate float | None Share of questions retrieved within top_k. None without questions.
mean_reciprocal_rank float | None Mean of 1/rank for questions retrieved within top_k, 0 for the rest.
to_pandas() -> Any

One row per chunk. Requires the notebook extra.

class ChunkStats
Field Type What it is
count int —
tokens_total int —
tokens_min int —
tokens_median float —
tokens_p95 float —
tokens_max int —
class CleanResult

What was written, and what was done to it.

Field Type What it is
source Path —
format str —
output Path | None = None —
masked int = 0 Identifiers replaced by their masked form.
masked_confirmed int = 0 The subset of those that passed a checksum.
metadata_removed list[str] = field(default_factory=list) Metadata keys and parts taken out of the copy.
unscanned_categories dict[str, str] = field(default_factory=dict) Categories nothing was looked for, so none of that kind were masked.
changes list[CleanChange] = field(default_factory=list) Each identifier covered over, with where it was.
notes list[str] = field(default_factory=list) What the copy does not cover, in the caller’s own terms.
skipped str | None = None Set when no copy was written, with the reason.
class ContentFinding

A passage that is hidden from a reader, reads as an instruction to a model, or both.

Field Type What it is
page int | None —
visibility str visible, not_measured, suspected (one signal) or confirmed.
instruction str confirmed (decoded from hidden characters), pattern, model or none.
severity str —
excerpt str The passage, identifiers masked unless the run used reveal, cut at 240 characters.
characters int —
hidden_reasons list[str] = field(default_factory=list) —
instruction_reasons list[str] = field(default_factory=list) —
score float | None = None The registered classifier’s score, when there is one.
in_loader_output bool | None = None For loader output: whether the loader’s text contains this passage.
fingerprint str = '' The same for this passage in every document and run; what an ignore file names.
class Document
Field Type What it is
path Path —
sha256 str —
format DocumentFormat —
pages list[Page] = field(default_factory=list) —
encrypted bool = False —
decrypted_with_empty_password bool = False —
acroform_fields int = 0 —
producer str | None = None —
page_count_known bool = True False for formats with no fixed pagination until they are rendered.
load_warnings list[str] = field(default_factory=list) Non-fatal problems. These become entries in the report’s limitations.
class DocumentFormat
Field Type What it is
PDF — = 'pdf' —
IMAGE — = 'image' —
DOCX — = 'docx' —
XLSX — = 'xlsx' —
PPTX — = 'pptx' —
HTML — = 'html' —
MARKDOWN — = 'markdown' —
TEXT — = 'text' —
EMAIL — = 'email' —
OTHER — = 'other' For a third-party loader of a format complydoc does not know. The registry is public, so a third party can teach complydoc a format without changing it. They need a name for what they are loading, and inventing enum members for formats we have never seen is not possible — so there is one to share. The reports group by this, so a folder of them appears together as “other”.
class DocumentReport
Field Type What it is
path Path —
relative_path str —
sha256 str —
format DocumentFormat —
page_count int —
page_count_known bool —
load_warnings list[str] = field(default_factory=list) —
cost DocumentCostEstimate | None = None —
readiness ReadinessReport | None = None —
sensitive ScanResult | None = None —
routing DocumentRouting | None = None The route each page needs: its text layer, OCR, or a vision model.
previews list[PagePreview] = field(default_factory=list) Per-page wireframes. Geometry only — never document content.
timing DocumentTiming | None = None —
extracted_text list[PageText] = field(default_factory=list) The text itself. Only populated with –extracted-text: it is the document.
extractions list[ExtractorReading] = field(default_factory=list) One per extractor the run was asked for. The first is the one kept.
metadata_findings list[MetadataFinding] = field(default_factory=list) Identifiers in the metadata a loader returned. Empty for files read directly.
content_findings list[ContentFinding] = field(default_factory=list) Hidden passages and instruction-like text. See complydoc.hidden.
concept_findings list[ConceptFinding] = field(default_factory=list) Pages a judgement model said hold one of your concepts, on a run that asked one.
concepts_unjudged int = 0 Questions about this document’s pages the judge failed to answer.
verification DocumentVerification | None = None Pages read again by a vision model. None unless the run used --verify.
visibility_checked bool | None = None Whether hidden text could be checked for. None when the scan did not run.
visibility_note str | None = None —
ignored list[IgnoredFinding] = field(default_factory=list) Findings an ignore file set aside. Not in sensitive.matches or content_findings.
path_exposures list[str] = field(default_factory=list) Metadata keys whose value is an absolute filesystem path. Not an identifier category, and no detector would flag it, but a home directory path names the account it belongs to and the layout around it.
extractors_disagree bool Whether the extractors read this document differently enough to say so. Measured against the one whose output was kept. A tenth of the text is the line: below that the difference is line endings and whitespace, and reporting it would be noise on every document.
disagreement str | None The kind of disagreement between extractors, or None.
class Expectation
Field Type What it is
collect — = collect Gather results instead of raising, so every check runs. Used by policies.
no_identifiers(
severity: str | None = None,
evidence: str | None = None,
categories: Iterable[str] | None = None,
) -> Expectation

No identifiers in text or metadata at or above severity and evidence.

no_hidden(severity: str = 'medium') -> Expectation

No hidden or instruction-like passages at or above severity.

readiness_at_least(score: float) -> Expectation

Every scored document has a readiness score of at least score.

facts_found(
facts: Iterable[Fact | str] | None = None,
threshold: float = FUZZY_THRESHOLD,
) -> Expectation

Every fact found by every loader.

Without facts, uses the facts of a compare_loaders report.

no_network() -> Expectation

Nothing in the run reached the network: no loader, and no classifier.

The loader attempts were the whole of this check, which made it pass on a run that sent every judged passage to a hosted service — the one case where a gate asserting “no network” most needs to fail.

no_identifiers_sent(severity: str | None = None) -> Expectation

No step of a pipeline observed with cd.observe sent text holding an identifier, at or above severity, to a host.

Raises ValueError for a report with no pipeline: it cannot say what was sent.

only_hosts(hosts: Iterable[str]) -> Expectation

No step of an observed pipeline reached a host other than hosts.

Raises ValueError for a report with no pipeline.

no_failures() -> Expectation

No loader failed on a file, and no file was skipped.

no_regressions(
baseline: AuditReport | str | os.PathLike[str],
score_tolerance: float = 0.5,
) -> Expectation

Nothing worse than in baseline, a report or a path to its JSON.

class Extraction

One page as one extractor read it.

Field Type What it is
text str = '' —
blocks list[TextBlock] = field(default_factory=list) —
granularity Granularity = 'word' —
raw_chars str = '' Characters before unicode normalisation, where the extractor exposes them. Empty when it does not: the garbled-character signal needs un-normalised text and reports as not measured without it.
tables list[TableInfo] = field(default_factory=list) —
tables_searched bool = False False when this extractor cannot look for tables at all, which is not the same as looking and finding none.
notes list[str] = field(default_factory=list) —
coverage_pct(width: float, height: float) -> float | None

Share of the page covered by text boxes, or None if none were returned.

A reader that returns text without geometry has no coverage measurement.

class ExtractionWarning

Something the caller needs to know before using the text.

Field Type What it is
kind str —
detail str —
document str | None = None —
page int | None = None —
hides_content bool Whether this warning means text is missing from the result.
class Fact

A passage a document is expected to contain.

Field Type What it is
text str —
document str | None = None File name or relative path to check. None checks every document.
kind str = 'text' text, a passage the reading must hold; value, a number it must hold however it is grouped; absent, a passage it must not hold.
label str | None = None For a value, the words it follows, such as Total due.
page int | None = None The page it must be on, counted from 1. None looks on every page.
class FactCheck

Whether one expected fact appears in the text of each loader.

Field Type What it is
fact str —
document str | None The document the fact was checked in, or None for every document.
found dict[str, str | None] Loader name to exact when the fact held, fuzzy for a near miss, or None. For a passage that must not appear, holding is not being found: pages and documents then say where a reading that failed it holds it.
scores dict[str, float] Loader name to the best similarity found, from 0 to 1.
pages dict[str, int | None] —
documents dict[str, str | None] Loader name to the document the fact was found in.
nearest dict[str, str | None] = field(default_factory=dict) Loader name to the passage that came closest, where the fact was not found exactly. What a missing fact looks like in that loader’s text, which is usually the reason it went missing.
kind str = 'text' text, value or absent, as complydoc.extraction.facts.Fact has them.
label str | None = None For a value, the words it was looked for after.
class FactLocation
Field Type What it is
fact str —
status str whole when one chunk contains it, split when only the joined document text does, missing otherwise.
chunks list[int] —
class IdentifierDifference

An identifier in some loaders’ output and not in others’.

Matched by where it was found, its category and its masked value, so the same identifier read by two loaders is one row. Metadata findings are matched regardless of key name, because loaders spell the same field differently: producer in one, Producer in another.

Field Type What it is
document str —
category str —
label str —
severity str —
evidence str —
value str Masked, unless the run used reveal.
location str text or metadata.
keys list[str] The metadata keys it was found under. Empty for text.
found_by list[str] —
missed_by list[str] —
class IngestOptions
Field Type What it is
ocr bool = False Run local OCR on pages with no usable text layer.
ocr_min_chars int = 40 Below this many native characters, a page counts as having no text layer.
render_dpi int = 150 Resolution used when rasterising a page for OCR or skew measurement.
max_render_pages int = 50 Cap on how many pages of one document are rasterised, to bound memory.
extract_tables bool = True —
extractor str = 'complydoc' Which extractor’s output the report is built from.
compare_extractors tuple[str, ...] = () Others to run alongside, for comparison only. They never change a finding.
compare_engines tuple[str, ...] = () OCR engines to read every rasterised page with, beside the one in use.
keep_readings bool = False Whether to keep each reader’s text for the page.
password str = '' Tried on encrypted files before falling back to an empty password.
ocr_compare bool = False Also OCR pages that already have a text layer, so the two can be compared.
render_all_pages bool = False Rasterise every page, including pages no signal needs. Set when the report is going to show the page next to what was extracted from it. Off by default: rasterising costs time and memory, and the default report carries no page images.
class InspectedChunk
Field Type What it is
index int —
document str | None The source file named in the chunk’s metadata.
page int | None —
characters int —
tokens int —
identifiers list[str] Identifiers in the chunk, as label: masked value.
hidden int Hidden or instruction-like passages of medium severity or above.
flags list[str] —
metadata_keys list[str] —
preview str The start of the chunk with identifiers masked.
start int | None = None Schema 17: where the chunk starts in its page’s text, in characters. None where the splitter changed the text, so it cannot be found there, or no page text was given.
end int | None = None —
page_characters int | None = None How long the page’s text was, so a reader of the report can tell whether a text it holds for the page is the one these places are in.
class LoaderComparison

Where several loaders’ output differed, measured against the first.

Field Type What it is
baseline str —
loaders list[LoaderSummary] —
identifier_differences list[IdentifierDifference] = field(default_factory=list) —
metadata_keys dict[str, list[str]] = field(default_factory=dict) Keys not returned by every loader, matched ignoring case, and which returned them.
documents dict[str, list[str]] = field(default_factory=dict) Documents not returned by every loader, and which loaders returned them.
facts list[FactCheck] = field(default_factory=list) Expected facts, checked against every loader’s text.
recommended str | None = None The loader to use, where the run could tell. None when it could not.
verdict str = '' What decided the recommendation, or what stopped it being decided.
ranked list[str] = field(default_factory=list) Every loader, best first, by what the run could measure.
formats list[FormatComparison] = field(default_factory=list) The same comparison for each file type in the run, with its own recommendation.
baselines dict[str, str] = field(default_factory=dict) Documents measured against a loader other than the first, and that loader. A document’s baseline is the first loader, in the order given, that returned it. The first loader may skip a file type or fail on a file.
class LoaderRun

What an external loader did when complydoc ran it.

Field Type What it is
name str —
documents_returned int —
seconds float | None None when documents were passed in already loaded.
network_attempts list[str] = field(default_factory=list) Connections the loader tried to open: refused by the guard, or made, when network_allowed is true.
error str | None = None —
metadata_keys list[str] = field(default_factory=list) Every metadata key the loader returned, across all documents.
failures dict[str, str] = field(default_factory=dict) Files the loader raised on, with the error, when it ran over several files.
cached_files int = 0 Files whose output came from the cache instead of the loader.
network_allowed bool = False The caller passed allow_network=True, so the loader’s connections went through. complydoc’s own processing stays behind the guard either way.
tags list[str] = field(default_factory=list) The framework and library the loader comes from, such as LangChain and pypdf.
skipped list[str] = field(default_factory=list) Files not given to the loader, because they are not of a type it is meant for.
formats list[str] | None = None The file extensions the loader is meant for. None when it was given every file.
class LoaderSummary

One loader’s totals, in a comparison of several.

Field Type What it is
name str —
documents int —
pages int —
characters int —
seconds float | None —
network_allowed bool —
network_attempts list[str] —
error str | None —
metadata_keys list[str] —
identifiers_in_text int —
identifiers_in_metadata int —
documents_with_paths int Documents with at least one metadata key holding an absolute path.
readiness_score float | None —
global_score float | None —
text_path_usd float | None —
failures dict[str, str] = field(default_factory=dict) Files the loader raised on, with the error.
cached_files int = 0 —
facts_found int | None = None Expected facts found in this loader’s text, when facts were given.
parser_usd float | None = None Estimated parser cost for these pages, from parsers in pricing.yaml.
tags list[str] = field(default_factory=list) The framework and library the loader comes from, such as LangChain and pypdf.
skipped list[str] = field(default_factory=list) Files not given to the loader, because they are not of a type it is meant for.
formats list[str] | None = None The file extensions the loader is meant for. None when it was given every file.
class MaskedText

A string with its identifiers replaced by mask characters.

Field Type What it is
text str —
masked int Identifiers replaced.
masked_confirmed int Of those, how many passed a checksum.
unscanned dict[str, str] = field(default_factory=dict) —
complete bool False when a category could not be scanned, so none of its values were masked.
class MetadataFinding

An identifier found in a document’s metadata.

Loaders attach metadata to every document they return, and it is usually stored beside the content — in a vector store, next to each chunk — so an identifier there travels exactly as far as one in the text.

Field Type What it is
key str —
category str —
label str —
severity str —
evidence str —
masked str —
revealed str | None = None —
page int | None = None —
significant bool Whether this finding raises limitations and quick wins. A low-severity category found by the name model is excluded. Loaders put the producing software in metadata — “ReportLab PDF Library”, “Microsoft Word” — and the model labels it an organisation, which would otherwise flag every PDF. It stays in the list; it does not become a limitation or a quick win on its own.
class Page
Field Type What it is
number int 1-indexed.
width_pt float —
height_pt float —
rotation int = 0 —
text str = '' —
text_source TextSource = 'none' —
text_blocks list[TextBlock] = field(default_factory=list) —
image_blocks list[ImageBlock] = field(default_factory=list) —
tables list[TableInfo] = field(default_factory=list) —
fonts set[str] = field(default_factory=set) —
embedded_fonts bool | None = None —
ocr_text str = '' What OCR read, kept separately from text. Normally OCR only runs where there is no text layer, and its output becomes text. With ocr_compare it runs on every page as well, so the text layer and the recognised text can be put side by side — which is how you tell a document that extracts badly from an extractor that reads it badly.
ocr_confidence float | None = None The engine’s mean confidence in what it read, 0 to 1. None if it did not run.
ocr_seconds float | None = None How long OCR took on this page, on the machine that ran it. None if it did not run.
invisible_text_share float | None = None Share of the page’s text objects that are drawn invisibly, 0 to 1. An OCR layer laid over a scan is drawn with the render mode that paints nothing, so a share near 1 means the text is a machine’s reading of the picture, and a share near 0 means the text was typeset. None when it was not measured, which is whenever pdfium was not given the page.
raw_chars str = '' Page characters as stored, before any unicode normalisation. text comes from the extractor’s own text assembly, which NFKC-normalises — turning a fi ligature into two plain letters. That is usually helpful, but it destroys exactly the evidence the garbled-character signal needs, so the un-normalised characters are kept alongside. Word spacing is absent here, so this is only useful for character-level questions.
extractions list[ExtractionSummary] = field(default_factory=list) One per extractor asked for, including the one whose output was kept.
readings dict[str, str] = field(default_factory=dict) What each extractor and OCR engine made of this page, by name. Only populated for the readers a run was asked to compare, and only when the run is keeping text at all. It is the page’s words several times over, so it is the largest thing a comparison adds to a report.
raster Image | None = None Populated only for pages that need to be looked at as pixels.
notes list[str] = field(default_factory=list) —
size_key tuple[int, int] Rounded page size, for counting distinct sizes within one document.
estimated_dpi() -> float | None

Effective scan resolution, from the largest embedded image on the page.

Only meaningful where the page really is a scan; a page whose images are small logos will report a number that means nothing, so callers check image coverage first.

class PageSource

The handles a page can be read through.

One per library, opened once by the loader. An extractor takes the handle it understands and ignores the rest, so adding a third does not mean opening the file a third time for the two that were already there.

Field Type What it is
plumber Any = None —
pdfium Any = None —
pypdf Any = None The reader the loader already opened, decrypted if the file was.
number int = 1 Which page this is, one-based, for readers that index a whole document.
class Question

A question and a passage that answers it.

Field Type What it is
text str —
fact str A passage the answering chunk contains, matched as expected facts are.
document str | None = None File name or relative path holding the answer. None accepts any document.
class QuestionResult
Field Type What it is
question str —
fact str —
status str retrieved, ranked_low, split or missing.
rank int | None Rank of the best-placed chunk holding the fact, from 1. None when no such chunk shares a word with the question.
answer_chunks list[int] Indexes of the chunks holding the fact.
top_chunks list[int] Indexes of the top_k best-ranked chunks, best first.
class Recognised

What an engine read, and how sure it was.

An engine reports a confidence for every box it recognises. Throwing those away would leave OCR text asserted with the same authority as a native text layer, which it has not earned.

Field Type What it is
text str —
confidence float | None —
boxes int —
class Rect
Field Type What it is
x0 float —
y0 float —
x1 float —
y1 float —
class ReportDiff
Field Type What it is
old_version str —
new_version str —
changes list[Change] = field(default_factory=list) —
to_pandas() -> Any

The changes as a DataFrame. Requires the notebook extra.

summary() -> str

One line per change.

class SkipRecord

A file complydoc could not open, or could not finish. Reported; the run continues.

Field Type What it is
path Path —
reason str —
detail str | None = None —
class TextBlock
Field Type What it is
text str —
bbox Rect —
class TextResult

Everything read, in order, with what could not be read alongside it.

Field Type What it is
chunks list[Chunk] = field(default_factory=list) —
warnings list[ExtractionWarning] = field(default_factory=list) —
masked bool = True —
documents_read int = 0 —
documents_skipped int = 0 —
text str Every chunk joined, for the common case of wanting the lot.
complete bool Whether everything in the folder reached this result. False when a file or a page could not be read.
all_categories_scanned bool Whether every configured identifier category ran. True means no category was skipped. Categories found by a model can still miss values; MASKING_BEST_EFFORT says so every time masking runs.
guaranteed list[Chunk] The chunks whose identifiers were all confirmed by a checksum. Empty unless mask ran. Use it where a wrong answer is expensive enough that best effort is not good enough — and read all_categories_scanned first.
class TextScan

Identifiers found in a string.

Field Type What it is
matches list[SensitiveMatch] —
unscanned dict[str, str] = field(default_factory=dict) Categories that could not be scanned, with the reason.
class Trail

Everywhere a value was found, newest run first.

Field Type What it is
label str | None What kind of identifier it was found as, such as IBAN. None when not found.
masked str | None The value as reports show it.
fingerprints list[str] —
runs list[RunSighting] —
reports_searched int —
class VisionModel

Anything called with a page that returns what it read.

class VisionPage

One page, rendered, as the model is given it.

Field Type What it is
document str The file’s name, for a model that logs what it was sent.
number int 1-indexed.
image bytes The page as an image, encoded as media_type.
width_px int —
height_px int —
media_type str = 'image/png' —
class VisionReading

What a vision model read off one page.

Field Type What it is
text str —
input_tokens int | None = None The provider’s own count, from the response. Makes the cost actual.
output_tokens int | None = None —
usd float | None = None What the call cost, where the caller already knows. Taken as given.
model str | None = None The model that answered, where it differs from the one the reader named.
clean_document(
path: str | Path,
out_dir: str | Path,
config: Config | None = None,
rasterise: bool = False,
) -> CleanResult

Write a safe copy of path into out_dir, and say what was done to it.

rasterise applies to PDFs only: each page is rendered to an image and the file rebuilt from those, so the text layer does not survive.

find_value(value: str, *sources: str | Path, config: Config | None = None) -> Trail

Everywhere value was found in the reports in sources, .complydoc when none is given.

value is an identifier as it is written, such as an account number or a name, or a finding’s fingerprint. Spacing, dashes and case do not matter.

sha256_of(path: Path) -> str