Evaluate · evidence of review

FINRA rule on email review: a lexicon hit is not a review

The FINRA rule on email review, Rule 3110(b)(4), requires firms to review correspondence and internal communications on risk-based principles, by a registered principal, with written evidence of each review. A lexicon hit proves only that a word appeared. This guide reads the rule and FINRA's guidance on lexicon review, shows how much keyword searches miss, and explains how Electronic Communications Surveillance records a review an examiner can test.

Updated · 13 min read

The FINRA rule on email review is Rule 3110(b)(4): a member's supervisory procedures must provide for the review of incoming and outgoing written, including electronic, correspondence and internal communications relating to its securities business, "appropriate for the member's business, size, structure, and customers", with each review conducted by a registered principal and evidenced in writing (FINRA Rule 3110). Firms may use risk-based principles to decide how much to review, and most use lexicons to choose. The trap is to treat the lexicon's output as the review. A hit proves a word appeared; a review proves a qualified person read the communication in context and decided something about it. Botminds' Electronic Communications Surveillance is built around the second: every close and escalation carries a reviewer, a date, the communication and a written reason.

A lexicon hit and a review are different records

A lexicon hitA review under Rule 3110
What it proves

A lexicon hit: A listed word or phrase appeared in a message.

A review under Rule 3110: A qualified person read the communication and decided what it meant.

Who

A lexicon hit: A rule in the lexicon, owned by whoever last edited it.

A review under Rule 3110: A registered principal, or a delegate whose work the principal supervises and evidences.

Context

A lexicon hit: One line, often without the rest of the conversation.

A review under Rule 3110: Enough of the conversation, in a language the reviewer reads, to judge it.

What is recorded

A lexicon hit: The term matched, the message ID, a time.

A review under Rule 3110: The reviewer, the communication, the date and the action taken on any significant issue.

What it misses

A lexicon hit: Anything said without the listed words: in an image, an attachment, slang or another language.

A review under Rule 3110: Whatever was never flagged or sampled, which is why random review and testing complement it.

Left: the output of lexicon-based screening as FINRA's guidance describes it. Right: Rule 3110(b)(4) and Supplementary Material .07 on evidence of review.Source: FINRA Rule 3110; FINRA RN 07-59

What does FINRA Rule 3110(b)(4) require?

Rule 3110(b)(4) requires procedures for reviewing two kinds of communication, each for a purpose. Incoming and outgoing correspondence must be reviewed "to properly identify and handle in accordance with firm procedures, customer complaints, instructions, funds and securities, and communications that are of a subject matter that require review under FINRA rules and federal securities laws". Internal communications must be reviewed to identify those of a subject matter that requires review (FINRA Rule 3110(b)(4)).

The rule then fixes who and how: reviews are conducted by a registered principal and evidenced in writing, on paper or electronically. Three supplementary paragraphs fill in the detail. Rule 3110.06 sets the risk-based approach; 3110.07 says what the evidence must show; 3110.08 allows the principal to delegate functions to unregistered staff while staying responsible. Rule 3110.09 sends retention to SEC Rule 17a-4(b).

The rule applies to FINRA members. An investment adviser that is not a broker-dealer is not bound by Rule 3110, but its compliance rule requires policies reasonably designed to prevent violations. An adviser may adopt FINRA's structure for its own email review, and should then say so in its own procedures rather than rely on the rule. Messages sent to more than 25 retail investors are not correspondence at all but retail communications, approved before use under Rule 2210, as who signs each piece under the Marketing Rule and Rule 2210 explains.

What is risk-based email review?

Risk-based review means the firm decides, and writes down, how much correspondence beyond the required subject matters it reviews, rather than reading everything. Rule 3110.06 leaves the firm to decide, on risk-based principles, how far its review procedures reach beyond the required subject matters, and attaches a condition: a firm that does not review all correspondence before it is sent must provide training, document that training, and carry out "surveillance and follow-up to ensure that such procedures are implemented and followed" (Rule 3110.06).

The history explains the design. FINRA's predecessors once required members to review all correspondence of their registered representatives about securities transactions; in 1998, as email made that impractical, they let firms design review procedures to fit their business (FINRA, Regulatory Notice 07-59). Risk-based review was the answer to volume. The cost of that flexibility is that the firm must be able to show its chosen method works.

The lexicon method of review, as FINRA describes it

FINRA's 2007 guidance on electronic communications remains the most detailed regulatory description of lexicon review. It defines lexicon-based reviews as those "based on sensitive words or phrases, the presence of which may signal problematic communications", and asks firms using them to use an appropriate lexicon, keep it confidential and "periodically evaluate the efficacy of the lexicon" (RN 07-59). A good system, it says, can add and delete terms, read attachments, restrict access to the word list and exclude the firm's own disclaimers, which would otherwise flag every email.

Two sentences in that guidance shape how firms run their queues. First, a firm "may determine that it is not necessary to review each and every lexicon 'hit' in order to maintain an effective review system", provided the rationale is kept in its procedures. Second, the firm is responsible for the system working, "especially if the system is that of a vendor", and should consider a supplemental random review if it lacks confidence in the lexicon.

FINRA was candid about the limits. Lexicons could not read password-protected or encrypted documents; image files "can be used to pass information through lexicon filters undetected"; and a representative determined to evade the system could simply avoid the trigger words. Its advice was to combine methods: "Given the strengths and weaknesses of any single review tool, members should consider complementary review techniques."

How much does a keyword search miss?

More than the people running it usually believe. Two research programmes measured it on legal document review, the closest well-studied cousin of lexicon surveillance. In a study of a full-text retrieval system used to defend a large corporate lawsuit, Blair and Maron found that the lawyers' searches retrieved about 20% of the relevant documents, while the lawyers believed they had found at least 75% (Blair and Maron, Communications of the ACM, 1985). Two decades later, the Text Retrieval Conference's legal track found that a Boolean query negotiated by two lawyers matched just 22% of the relevant documents on average, with precision of 0.29 (Oard et al., Artificial Intelligence and Law, 2010).

Keyword searches find less than their users think

Recall the lawyers believed they had reached (STAIRS study)
75%+
Recall actually measured (STAIRS study)
~20%
Recall of a lawyer-negotiated Boolean query (TREC legal track, 2007)
22%
Precision of that query: share of matches that were relevant
29%
Recall is the share of all relevant documents a search found; precision is the share of what it found that was relevant. STAIRS: Blair and Maron (1985), restated by Blair (1996). TREC: Oard, Baron, Hedin, Lewis and Tomlinson (2010), reporting the 2007 legal track. Both are peer-reviewed. Document review in litigation, not surveillance of communications; no regulator publishes equivalent figures for surveillance lexicons.Source: Blair and Maron, CACM 1985; Oard et al., AI and Law 2010

Neither study measured a surveillance lexicon, and a firm should not borrow their numbers. What they show is the shape of the problem: words are a poor proxy for meaning, recall falls when people do not use the expected words, and users overestimate how much they found. Blair later explained that the lawyers could not recall the exact words unique to the documents they wanted, only their gist. A trader avoiding a lexicon does the same on purpose.

What counts as evidence of review?

Evidence of review is a record, made at the time, that names the reviewer, identifies the communication, dates the review and shows what was done about any significant regulatory issue. Rule 3110.07 says the evidence "must be chronicled either electronically or on paper" and must show "the actions taken by the member as a result of any significant regulatory issues identified during the review"; it adds, in its last sentence, that opening a communication does not count as reviewing it (Rule 3110.07).

FINRA's guidance says the evidence can be "a log or other record from the electronic communication system that identifies the reviewers", and that firms must be able to "reasonably demonstrate that such reviews were conducted" (RN 07-59). A pick-list code on a closed alert identifies a reviewer and a date. It does not, on its own, demonstrate that the message was understood. A written reason, one or two sentences on what the message meant and why it is or is not a concern, does.

The retention rule makes the reviewer's identity permanent: "The names of the persons who prepared outgoing correspondence and who reviewed the correspondence shall be ascertainable from the retained records" (Rule 3110.09). A review whose author cannot be shown later is, for an examiner, no review.

Context belongs in the evidence too. A reviewer who closed a flagged line after reading the conversation around it made a better decision than one who read the line alone, but only the record can show which happened. Listing what was read, and what was set aside as unrelated and why, lets an examiner see the review's scope as well as its outcome.

The scope of a review, written down: what the thread left out, and why

A side panel titled Left out of the thread, 31, Thread agent, same people plus or minus 48 hours. It explains that the agent searched every channel for the trader and the broker from October 12 to 16 and kept only items about the issuer or the call; the rest were left out as unrelated and each is still in the archive. A table lists each item left out with its time, channel, sender and subject, such as a morning run sheet of Brazil sovereign levels, a weekly LatAm credit newsletter, a lunch order and a settlement confirmation, and the reason: no issuer, order or call in common. A button lets the reviewer add an item to the thread.
Fictional sample data. Thirty-one messages between the same two people were set aside, each with its reason, and each remains in the archive; the reviewer can add any of them to the thread.

Measure reviews, not hits

Lexicon reports count hits, alerts and closures. None of those measures whether the reviews were right. Three measures do. A blind re-review: a second reviewer, or a checking process, re-reads a random sample of closed alerts without seeing the first decision, and the disagreement rate is the error estimate. Seeded messages: test communications written to trip each risk the lexicon should catch, sent through the real channels, to measure what the system flags and what reviewers then decide. And outcome checks: whether escalations led anywhere, and whether anything later found by other means had been closed as noise.

Measure each by reviewer and by language, because averages hide the weak spots. FINRA's 2007 guidance supports both habits: periodic evaluation of the lexicon's efficacy, and re-review of selected communications by compliance or legal, for example during branch inspections (RN 07-59).

Measuring the reviews: a blind re-read of closed alerts

QA re-sampling for September: 9,772 closed alerts, of which 9,104 false positives and 668 coaching notes; 489 re-read blind, 5%, at least 2 per reviewer and language; 482 agreed, 98.6%; 7 reopened, 1.4%, 2 escalated, 2 kept, 3 open. The reopened list includes a German message, 'the numbers only come out Thursday, but they are good', closed as public news although the issuer reported two days later; a Portuguese call-back closed against a recorded call with a different broker; and a Mandarin line about golf with a broker after a bond allocation, a gifts and entertainment item. Each has buttons to keep closed or escalate, for a second reviewer.
Fictional sample firm. The QA agent re-reads 5% of closes blind and reopens 1.4%; one reopened alert, a German 'the numbers only come out Thursday, but they are good', had been closed as public news two days before the issuer reported.

FINRA email retention policy: how long, and in what form?

Rule 3110.09 requires members to keep correspondence and internal communications for the period and accessibility set by SEC Rule 17a-4(b): at least three years, with the first two in an easily accessible place (17 CFR 240.17a-4(b)). The communications covered are "Originals of all communications received and copies of all communications sent (and any approvals thereof)" relating to the business, so the approval, which is the review, is kept with the message.

Electronic records must meet Rule 17a-4(f): kept either in a non-rewriteable, non-erasable format or in a system with a complete audit trail of every change, the time it was made and who made it. FINRA's 2026 report lists both under the books and records obligations, and finds firms failing to test their third-party recordkeeping vendors; its effective practice is to simulate a regulator's request and see whether the records come back (FINRA, 2026 report).

FINRA email retention, in three numbers

3years

minimum retention for business communications under Rule 17a-4(b)

2years

of that in an easily accessible place

2ways

to store electronically: non-rewriteable, non-erasable, or a full audit trail

17 CFR 240.17a-4(b) and (f), current eCFR text, applied to FINRA members' correspondence and internal communications by FINRA Rule 3110.09. Firms may keep records longer under their own policies or under other rules that apply.Source: 17 CFR 240.17a-4; FINRA Rule 3110.09

What FINRA found when it examined email review in 2026

FINRA's 2026 oversight report lists, among its books and records findings, firms "reviewing electronic communications without selecting adequate samples or using targeted key word searches", and failing to review communications in non-English languages in which they do business (FINRA, 2026 report). Both are failures of the method rather than of the archive: the messages were kept, but the review could not have found what mattered in them.

The same report faults procedures that do not say how a firm will find business conducted on unapproved platforms, and lists as an effective practice the frequent revision of key words used to detect off-channel use. The off-channel communications guide covers what those key words are for.

A checklist for an email review an examiner can test

Nine things a defensible email review leaves behind

  1. A written risk-based rationale

    Which correspondence is reviewed before and after sending, which lexicon hits are not reviewed, and why.

  2. A lexicon with an owner and a change history

    Additions, deletions and disclaimer exclusions, each dated and approved.

  3. A periodic test of the lexicon’s efficacy

    Seeded messages and a look at what it missed, at least yearly.

  4. A random sample alongside the lexicon

    FINRA sets no minimum percentage, but the sample must be reasonable for the firm, and higher for people under special supervision.

  5. Review in every business language

    With independent interpretation where the reviewer is not fluent.

  6. A decision record for every alert

    Reviewer, communication, date, action and a written reason.

  7. Evidence of the principal’s supervision of delegates

    What was delegated, how it was checked, and the escalation protocol.

  8. A blind re-review rate by reviewer and language

    The error estimate that a hit count can never give.

  9. Retention that keeps the approval with the message

    Three years minimum, two accessible, under Rule 17a-4(b) and (f).

Drawn from FINRA Rule 3110(b)(4) and Supplementary Material .06 to .09, FINRA Regulatory Notice 07-59 and SEC Rule 17a-4.Source: FINRA Rule 3110; FINRA RN 07-59

How Electronic Communications Surveillance records evidence of review

In Electronic Communications Surveillance, every close and escalation carries a reason code, a written reason, the reviewer's name and the date and time, linked to the communication reviewed. The written reason is drafted by the agents for the reviewer to check, edit or reject, so the evidence is complete without the reviewer typing it from nothing. Noise is closed in batches grouped by reason, but each alert in the batch keeps its own reason, and the reviewer ticks that they reviewed the agents' reasons before approving.

Two controls address what lexicons miss. The QA agent re-reads 5% of closes blind each month and a different reviewer decides each reopen; and a proposed lexicon change is back-tested on last month before the head of surveillance approves it. The reasons an agent can propose, and the decisions it may never take, are set out in who may close an alert.

Questions people ask

Does FINRA require every lexicon hit to be reviewed?

No. FINRA's 2007 guidance says a firm may decide it is not necessary to review every lexicon hit to keep an effective review system, provided the rationale is kept in its policies and procedures. The firm remains responsible for showing the system works.

What percentage of emails must be reviewed under FINRA rules?

FINRA prescribes no minimum or fixed percentage. The amount reviewed must be reasonable for the firm's size, business, customers and individual circumstances, and a person with a disciplinary history or under special supervision may warrant a higher share.

Who can review emails under Rule 3110?

Reviews are conducted by a registered principal, who may delegate functions to people who need not be registered. The principal stays responsible, must ensure delegated work is done properly and must evidence that supervision. No one may review their own communications unless the firm has no reasonable alternative.

Is a pick-list disposition enough evidence of review?

It records who, what and when, which Rule 3110.07 requires. It may not show that the message was understood or what was done about an issue. A short written reason closes that gap and is far easier to defend when an examiner picks an alert at random.

Does Rule 3110 apply to text messages and chat?

Yes. FINRA treats the duty to supervise as following the content and audience of a message, not its form, and its 2026 report names email, instant message, text message and chat among the business communications that must be kept. A channel the firm cannot capture should not be used for business.

How long must FINRA members keep emails?

At least three years, the first two in an easily accessible place, under SEC Rule 17a-4(b) as applied by Rule 3110.09, stored either in a non-rewriteable, non-erasable format or with a full audit trail under Rule 17a-4(f).

Electronic Communications Surveillance.
See it on your own files.

An agentic solution from Botminds. Reviews flagged emails, chats and calls in any language and closes each alert with reasons Book a demo, or explore how it works.