Structured data
Defined fields with types and constraints. Data that does not fit is rejected when it is written.
- Rows and typed columns
- Enforced by the database
- Queried with SQL
- Aggregates are exact and fast
Guide · AI data modeling
Structured vs unstructured data: where the schema lives, and who reads it.
Structured data carries its schema in the database. Unstructured data carries its meaning in language, layout and pixels — and most company knowledge lives there.
This guide defines the three kinds of data precisely, places real formats on a single scale, shows how each is stored and queried, gives the cited figures on how much data is unstructured, and walks through how AI reads values out of documents into typed tables a database can enforce.
Updated · 6 sources
Structured: schema fixed before the data arrivesUnstructured: meaning carried by language, layout or pixels
1234567
- Relational tableEvery row has the same typed columns, enforced on write.
- CSV exportColumns by position; types are implied, not enforced.
- JSON or XML messageSelf-describing keys and nesting; the shape can vary record to record.
- EmailStructured headers — sender, date, subject — around a free-text body.
- Invoice PDFFields and a line table, but only as positions on a page.
- ContractObligations and dates expressed in clauses of prose.
- Scan, call audio, videoPixels and sound; text must first be recognised.
What is the difference between structured and unstructured data?
Structured data follows a schema defined before it is stored — rows and typed columns a database enforces and SQL can query. Unstructured data has no predefined data model: emails, contracts, PDFs, scans, audio and video carry their meaning in language and layout. In structured vs unstructured data, the practical difference is who does the interpreting: the database for the first, a person or an AI model for the second. Semi-structured data — JSON, XML — sits between them, describing itself with keys and tags.
Side by side
Structured vs unstructured data, compared dimension by dimension.
The two kinds differ in when the schema is applied, what a query looks like, and what it costs to get a reliable number out.
Unstructured data
Content whose meaning is expressed in language, layout, images or sound. Nothing checks its shape on the way in.
- Documents, messages, media
- Stored as files or objects
- Found by search, read by people or models
- Values must be extracted before they can be counted
Before writing — schema on writeWhen the schema appliesWhen it is read — or never
Relational databases, warehousesTypical storesFile shares, object storage, content systems
SQL: filter, join, aggregateHow it is queriedKeyword and semantic search; reading
“Total overdue by supplier this quarter”A question it answers well“Which contracts allow termination on 30 days’ notice?”
Types, keys and constraintsQuality controlReview by people; extraction checks
Natural-language questions over tablesWhat AI addsReading the values out in the first place
Semi-structured data — JSON (RFC 8259) and XML (W3C XML 1.0) — carries its own labels. A program can parse it without guessing, yet two records in the same file may have different keys, so the schema is enforced by whoever reads it.
Anatomy of a document
Most “unstructured” documents hold structured data inside them.
An invoice is unstructured to a database and highly structured to an accounts clerk. The values are all there; they are just expressed as positions, labels and sentences on a page instead of columns.
One supplier invoicea PDF the database sees as a single file
Header fieldsinvoice number, date, supplier — key-value pairs by position
Line tablerows and columns, drawn with lines rather than stored as rows
Totals blocksubtotal, tax, total due — numbers that must reconcile
Payment terms“Net 60, 2% discount within 10 days” — a rule in prose
Stamps and signaturesapproval evidence that exists only as pixels
Covering emailcontext that changes a value, such as a corrected PO number
“Unstructured” describes the storage, not the content. A database sees one file; the page holds dozens of typed values. The work of document AI is to recover that structure and give each value a type and a source.
The figures
How much enterprise data is unstructured.
Estimates come from analyst firms and vary with what is counted — new data or all stored data, files or bytes. These are the figures with a traceable report behind them.
How to read these numbers. Both reports were sponsored by storage or content vendors — Box and Seagate — and the widely repeated “80% unstructured” line is usually quoted without a named report. Every estimate agrees that most company data is unstructured; the exact share depends on definitions, so quote a named report rather than a rounded slogan.
Where it lives
Where structured and unstructured data are stored and queried.
Each store is built around when the schema is applied. The newer designs exist to bring the two kinds closer together.
From strict schema to none
01
Relational databasetyped tables for transactions; constraints checked on every write
schema on writeSQL02
Data warehousestructured data modelled for analysis across systems
facts and dimensions03
Lakehouseopen file formats on object storage with warehouse features — transactions, schema enforcement — added on top
CIDR 202104
Document storeJSON records whose shape may vary; indexes on chosen keys
semi-structured05
Data lake and file sharesraw files in any format, interpreted when read
schema on read06
Search and vector indextext split into passages, indexed by word and by meaning
keywordembeddingsAWS’s own comparison sums up the two ends: large collections of structured stores are data warehouses, large collections of unstructured stores are data lakes. The lakehouse design, set out by Armbrust and colleagues at CIDR 2021, argues for one copy of the data in open formats with warehouse-grade management layered over it.
Unstructured to structured
How AI turns unstructured data into structured data.
The conversion is a pipeline, and each stage has its own failure mode. Skipping validation or source tracking is how extracted data loses the trust that made the table worth building.
- Capture and classify
Bring in files and messages, recognise text in scans, and decide what each document is — invoice, contract, claim form — so the right schema applies.
- Extract to a schema
A model reads the document and returns named, typed fields that match a declared schema.
- Validate
Check types, required fields and business rules — totals reconcile, dates are in range.
- Review the uncertain
Low-confidence or failed values go to a person, with the source page open.
- Load
Typed rows land in a table, each value linked back to the page it came from.
The extraction stage is covered in depth in structured data extraction with LLMs; the review stage in straight-through processing.
Decision guide
Structured vs unstructured data: which approach a question needs.
Most questions a business asks of its documents fall into one of three shapes. The shape, not the file format, decides the approach.
What do you need to do with the information?
Count, sum or compare itExtract the values into a structured table first. A model cannot reliably total 40,000 invoices by reading them at question time.
Find and cite a passageIndex the documents for keyword and semantic search and let an agent answer with citations. Extraction is optional.Read more →
Act on a value in a documentExtract it, validate it and route the uncertain cases to a person before a system acts — paying, approving, filing.
Values are named fields
Extract once, look upA policy number or an expiry date: extract and store it, then look it up.extraction
Extract, then aggregateSpend by supplier, exposure by counterparty: only a table answers this exactly.structured table
Values are free textSearch and readOne clause in one contract: search finds it, a person or agent reads it.search
Search with an agentThemes across hundreds of complaints: retrieval plus a model that reads and summarises.agentic RAG
Answer lives in one placeAnswer spans many documentsOn Botminds
How Botminds keeps structured and unstructured data in one model.
On Botminds the same data model serves both kinds: documents are read into typed records, and the documents themselves stay searchable for agents.
One data modellabels the AI reads as instructions and the database enforces as a schema
Processing collectionsdocuments flow in and agents read values out
Structured collectionstyped rows in real database tables
Nine field typestext, numbers, dates, yes / no, choice, links
Schema from a fileupload a spreadsheet; a person reviews the proposed types
A source for every valueits page, its words and a confidence score
Knowledge collectionsdocuments chunked, embedded and indexed
Meaning and keyword searchagents search by meaning, and by keyword when it is switched on
Same names everywhereon screen, in agents and in the API
Read the full design in AI data modeling, and learn the collection kinds in the Academy lesson on collection types.
Questions people ask
Structured vs unstructured data: common questions.
The technology behind itAI data modelingFrom documents to typed, searchable records→What is the main difference between structured and unstructured data?
In structured vs unstructured data, structured data follows a schema defined before it is stored, so a database can enforce types and SQL can query it. Unstructured data has no predefined model — its meaning is in language, layout, images or sound — so it must be searched or read, by a person or an AI model.
What are examples of structured and unstructured data?
Structured: a customer table, a general ledger, an inventory list, sensor readings in a time-series table. Unstructured: contracts, emails, scanned forms, call recordings, chat transcripts, slide decks and photos of damage on an insurance claim.
What is semi-structured data?
Data that labels itself without a fixed schema, such as JSON and XML. Keys and tags make it parseable, but records can differ in shape, so consistency is checked by the program reading it rather than by the store.
Is a PDF structured or unstructured data?
Stored as a file, a PDF is unstructured: a database cannot query its contents. Many PDFs — invoices, statements, forms — hold highly structured content that AI extraction can recover into typed fields.
How much of the world’s data is unstructured?
An IDC white paper sponsored by Box (2023) puts it at 90% of company data. Figures vary with definitions, and the commonly quoted 80% rarely names its source, so cite a specific report.
How is unstructured data used in AI?
Two ways. Retrieval indexes documents so a model can find and cite passages when answering. Extraction reads named values out of documents into structured records that systems can total, check and act on.
What is schema on read vs schema on write?
Schema on write checks data against a defined structure as it is stored — the relational approach. Schema on read stores raw data and applies a structure only when a query or program reads it — the data lake approach.
Can structured and unstructured data be stored together?
Yes. Lakehouse designs keep both in open formats on object storage, and document platforms keep each extracted value linked to the file it came from, so the record and its source travel together.
Sources
- IDC white paper sponsored by Box, “Untapped Value: What Every Executive Needs to Know About Unstructured Data” (August 2023), as reported by Boxblog.box.com
- IDC, “Data Age 2025: The Digitization of the World” (sponsored by Seagate)seagate.com
- AWS: The difference between structured and unstructured dataaws.amazon.com
- RFC 8259: The JavaScript Object Notation (JSON) Data Interchange Formatrfc-editor.org
- W3C: Extensible Markup Language (XML) 1.0w3.org
- Armbrust et al., “Lakehouse: A New Generation of Open Platforms…”, CIDR 2021cidrdb.org
Read next
Structured data extraction with LLMsSchemas, structured outputs, validation layers and citations→Taxonomy vs ontologyHierarchies of terms against formal models a machine can reason over→Semantic layer vs ontologyGoverned metrics over tables against a formal model of what things are→All guidesEvery explainer, by topic→
See it on
your own work.
Bring one process and its documents. We’ll show you the agents, the controls and the record, running on your data.