Full DART Search Syntax Guide

IN THIS ARTICLE:

This article contains confidential information, which may include trade secrets and other proprietary information. The reader acknowledges that this information has been developed by Lighthouse Global as valuable trade secrets. All information contained herein shall remain the exclusive property of Lighthouse Global and shall be disclosed only to persons who have a need to know. The recipient agrees not to copy or reproduce in any form any information supplied herein without prior written permission from an authorized representative of Lighthouse Global. The recipient further agrees to provide security for this document to a reasonable degree so that unauthorized disclosure is prevented.

Words and Phrases / Exact Match

By default, DART search matches the exact text. Searching for a word or a phrase returns exact matches of that word or phrase. Different forms of the same word or phrase, spelling corrections/variations, and conceptually related text are not matched. For example:

advertise => Matches the exact text "advertise" within a document. Does not match text such as "advertising," "advertised," "advertisement," "marketing," "sales," etc.

world health organization => Matches the exact text "world health organization" within a document. Does not match text such as "worldwide health," "healthy world," "organization world health," "world health center," "world health org," "W.H.O.," etc.

Phrases do not need to be inside quotation marks. Running a phrase such as  world united does not return documents that simply contain both words or that contain those two words in the reverse order; it returns only those documents that contain the exact phrase "world united."
DART search syntax is case insensitive.

WHO => Matches text such as "WHO," "who," or "Who" within a document.

who => Matches text such as "WHO," "who," or "Who" within a document.

Default Stream

Words and phrases run over the "default stream," which is the designated set of document fields over which all search terms run by default in DART. The default stream contains the body of the document as well a set of key metadata fields: email_subject, email_from, email_to, email_cc, email_bcc, attendees, attach_names, author, file_name, doc_subject, and title. All other metadata fields (such as file_path, doc_type, file_extension, custodian, etc.) are not included in the default stream.
You may request customized changes to the default stream for your project.

Booleans

Boolean operators connect and define relationships among the elements in your search terms. (Elements are defined as individual words, phrases, terms, conditions, grouped combinations, etc. within a search term.) The three basic Boolean operators are AND, OR, and NOT. Using ORbroadens your search results, while using AND or NOT narrows your search results.

Brackets (alternative syntax for OR)

Brackets vs. OR

When Boolean OR is used instead of brackets, search terms can quickly become long and difficult to read. Brackets are a faster, easier, and more reader-friendly way to add precise variation to a word or concept within a search term:
[ad{s, vertisement{s}}, commercial{s}, market segment{s}]
versus
ad OR ads OR advertisement OR advertisements OR commercial OR commercials OR market segment OR market segments

Optional Space

Curly brackets can also be used to represent an optional space or optional special character/symbol (such as a hyphen) inside a word. For example, the search term  e{ }mail matches all of the following strings of text:

• email
• e-mail
• e mail

Wildcards and Stemming

A wildcard operator is a special symbol that matches one or more other characters in the text. Stemming identifies the predictable grammatical variations associated with a root word. Wildcards and stemming allow you to add variation to a word in a way that is faster and easier than using brackets or Boolean OR. However, wildcards and stemming do not let you specify or control the exact variation that is or is not captured, and they may capture unanticipated and/or undesired variation.

Exercising Caution with Wildcards

It is important to exercise caution and use your discretion with wildcards, especially with the unlimited wildcard. The unlimited wildcard presents the highest risk of matching unanticipated and/or undesired variation, as well as the highest risks related to the computational expense. In situations where you need to use a wildcard operator at the beginning of a word, embedded within a word, or at the end of a word that consists of three or fewer letters, consider using the single-character or digit wildcard instead of the unlimited wildcard.

Proximity (Basic)

Proximity operators allow you to specify the number of tokens that may occur between two or more elements in your search term.

Tokens

The search index defines a token as a word containing letters and/or numbers. Tokens are separated from each other by white space, and the search index defines all non-alphanumeric characters (punctuation, symbols, special characters, etc.) as white space. The token count can be different from the word count within a string of text. For example, the following string of text contains 19 words:

"Hi! Please email this to John at jmartin@enron.com (he's out of the office now, but he'll be back soon)."

The search index tokenizes this text into 23 tokens for search:

"Hi Please email this to John at jmartin enron com he s out of the office now but he ll be back soon"

Additional Considerations for Proximity

Token count:

For bi-directional and uni-directional proximity, zero to N tokens may occur between the elements. The elements in the term are not included in the token count. The search term  approv* /5 proposal* matches the following text:
"The proposal was reviewed by Maria for approval."

For window proximity, the elements are included in the token count. The search term  w(size=5, approv, proposal*) allows for only three additional words in the window of five words. It does not match the text above, and it does not match the following text:

"The project has been approved by marketing, and the proposal for next steps is ready."

Bi-directional expansions:

When a search term contains two or more bi-directional proximity operators, the search does not match every possible proximity combination within a certain window size. For example, the search term  budget w/10 cost* w/10 proposal* results in four combinations, or expansions:

• budget* f/10 cost* f/10 proposal*
• proposal* f/10 budget* f/10 cost*
• cost* f/10 budget* f/10 proposal*
• proposal* f/10 cost* f/10 budget*

The bi-directional operators in this search term are treated from left to right, resulting in the above set of four expansions (for more information on this topic, see Grouping).

To match every possible proximity combination, you should use the window operator. For example, the following text is matched by   w(size=20, budget, cost*, proposal*) but is not matched by budget* w/10 cost* w/10 proposal*:

"The cost of this proposal is not within the budget."

"The budget is low, and the costs associated with these materials are high. Please account for this in your proposal."

Different proximity sizes within the same search term:

While the window operator is a better option when you want to match every possible proximity combination within a certain window size, the bi-directional and uni-directional proximity operators are better options when you want to be able to specify the proximities between particular elements in a given search term. Using different proximities within a search term can help you model and target relationships between words. For example:

meet* w/5 Anderson w/50 Baker f/2 account*

The search term above captures language related to meetings and "Anderson" when they have a closer linguistic relationship, perhaps within the same clause. It captures "Baker" in more of a contextual relationship with the Anderson meeting, perhaps in the same paragraph. It ensures that "Baker" occurs before "account," likely as an adjective, and it allows for a few words to occur between "Baker" and "account" to capture possible variations like "the Baker and Harper accounts."

Metadata

The metadata operator allows you to search outside of the default stream and allows you to target content in a specific metadata field. You may use the metadata field's long name or short name in the syntax. For example, the search terms  m/email_from: John and m/from: John both match "John" in the email_from field. For a list of all the field names in DART, go to File in the DSR menu and choose Metadata Info.

Offset Metadata

Email Segment offset metadata

There is a fifth type of offset metadata for email segments, which combines aspects of all four offset metadata types listed above into a single search operator. The syntax for email segment search is detailed in a separate guide: “ Email Segment Sub-Streams and Syntax.”

Quotation Marks

Quotation marks are helpful when you want to search for a word that without quotation marks would act as a search operator, such as OR, AND, NOT, or empty. Quotation marks can also be helpful if there are symbols/special characters in your search term that are part of DART syntax, but you want the search term to run as an exact text match. DART syntax no longer works as search syntax when enclosed in quotation marks; it is treated as whitespace instead. For example, consider the following search term enclosed in quotation marks:  "review w/10 [account*, deal{s}]"

This search term matches the following text: 

"review w 10 account deal s"

Previously Executed Searches

The History tab in DSR records a temporary history of the search terms you have executed in the selected search scope. The history persists for seven days (or you may delete history items at any time). Each search term saved in the history is assigned a number. You can reference these history numbers in DART syntax, which allows you to represent a previously executed search term by a number rather than rewriting the query text into a new search term.

NOTIN

Proximity may be used on either side of NOTIN: (communicat* w/10 confidential*) NOTIN w(size = 50, [e{ }mail , communication, message], [privileged, confidential], [intended, error, indicated, designated])

Grouping

Grouping is the use of parentheses to resolve the search ambiguity that can exist in complex search terms.

Operator Precedence

The default precedence for DART operator syntax is:

1. Grouping (parentheses)
2. OR, brackets
4. Uni-directional Proximity
3. Bi-directional Proximity (smaller first, or if the same size, left to right)
5. AND
6. NOT
7. NOTIN

Additional Considerations for Grouping

Metadata Operator:

Everything that follows the metadata operator is included in the scope of the  m/ operator, unless grouping is used. For example:

• m/custodian_all: Meyers NOT Lee => Returns documents that contain "Meyers" in the custodian_all field, and that do not also contain "Lee" in the same field.

versus

• (m/custodian_all: Meyers) NOT Lee => Returns documents that contain "Meyers" in the custodian_all field, and that do not also contain "Lee" anywhere else in the document (anywhere in the default stream, to be exact).

• m/repeated: e{ }mail AND privileged => Returns documents that contain both "email" and "privileged" within the repeated content in the body of the document.

versus

• (m/repeated: e{ }mail) AND privileged => Returns documents that contain the word "email" within the repeated content in the body of the document, and that also contain the word "privileged" anywhere else in the document (anywhere in the default stream, to be exact).

Bi-directional Proximity:

When a search term contains two bi-directional proximity operators, the operator with smaller proximity takes precedence:
investigat* w/50 close~ w/5 account* runs as investigat* w/50 (close~ w/5 account) =>

• investigat* f/50 close~ f/5 account*
• close~ f/5 account* f/50 investigat*
• investigat* f/50 account* f/5 close~
• account* f/5 close~ f/50 investigat*

To change this operator precedence, use grouping:  (investigat* w/50 close~) w/5 account* =>

• investigat* f/50 close~ f/5 account*
• account* f/5 investigat* f/50 close
• close~ f/50 investigat* f/5 account*
• account* f/5 close~ f/50 investigat*

When a search term contains two bi-directional proximity operators of the same size, the operator on the left takes precedence:
budget* w/10 cost* w/10 proposal* runs as (budget* w/10 cost*) w/10 proposal* =>

• budget* f/10 cost* f/10 proposal*
• proposal* f/10 budget* f/10 cost*
• cost* f/10 budget* f/10 proposal*
• proposal* f/10 cost* f/10 budget*

To change this operator precedence, use grouping:  budget* w/10 (cost* w/10 proposal*) =>

• budget* f/10 cost* f/10 proposal*
• cost* f/10 proposal* f/10 budget*
• budget*f/10 proposal* f/10 cost*
• proposal* f/10 cost* f/10 budget*

If more than two bi-directional operators are used in the same search term, grouping must be used to specify the desired operator precedence.

Proximity and Booleans:

When combining proximity and Boolean operators in the same search term, some grouping combinations are not allowed.

Proximity + AND:

The following grouping combination of proximity and AND is not allowed because it can be difficult to ascertain the intent of the underlying search logic:

• marketing w/5 (San Francisco AND New York)

The search term above should be rewritten so that the intent of underlying search logic is more readily apparent:

• (marketing w/5 (San Francisco OR New York)) AND (San Francisco AND New York)
• ((marketing w/5 San Francisco) AND New York) OR ((marketing w/5 New York) AND San Francisco)
• (marketing w/5 San Francisco) AND New York
• (marketing w/5 New York) AND San Francisco
• (marketing w/5 New York) AND (marketing w/5 San Francisco)

Proximity + NOT:
The following grouping combination of proximity and NOT is not allowed because it can be difficult to ascertain the intent of the underlying search logic:

• marketing w/5 (Washington NOT Washington D{ }C)

The search term above should be rewritten so that the intent of underlying search logic is more readily apparent:

• (marketing w/5 Washington) NOT Washington D{ }C

NOT excludes at the document level. If the intention of the search term is to exclude at the level of individual hits, NOTIN should be used instead of NOT. In most cases, NOTIN can be used instead of NOT to exclude at the hit level rather than at the document level, but there are some exceptions when proximity and grouping are involved (see below).

Proximity + NOTIN:

The following grouping combination of proximity and NOTIN is not allowed because of the computationally expensive nature of NOTIN:

• marketing w/5 (Washington NOTIN Washington D{ }C)

However, the search term above can be rewritten slightly (see below), and the rewritten version returns nearly the same results that would be returned by the search term above:

• (marketing w/5 Washington) NOTIN (marketing w/5 Washington D{ }C)

Word Order:

Grouping can trump word order when used with bi-directional proximity in certain combinations:

• bank account w/10 investigation questions =>
  • bank account f/10 investigation questions
  • investigation questions f/10 bank account

versus

• bank (account w/10 investigation) questions =>
  • bank account f/10 investigation questions
  • bank investigation f/10 account questions

Case Sensitivity

These operators allow you to run searches with sensitivity to case.

By default, DART syntax is not case sensitive. For example, the following search terms do not contain case sensitive operators; therefore, they all match text without sensitivity to case, and they all return exactly the same results:

• BUDGET AND COST
• Budget AND Cost
• budget and cost

Diacritics

Without this operator, with the right foreign language settings in place, search terms are blind to diacritics and match both the word with diacritics and the word without diacritics. For example, with the right foreign language settings in place, the search term  está matches the words "está" or "esta." The search term diacritic(está) matches only the word "está," and the search term diacritic(esta) matches only the word "esta."
Note: If the correct foreign language settings are  not in place in the search index, the diacritic operator does not work, and any letters in the text with diacritics on them become part of the ignore class of characters and are treated as white space. For example, without the right foreign language settings, the word "baño" within the text is treated as "ba o" during search, and the word "está" is treated as "est" during search.

Proximity (Advanced)

Variant Sets

A “variant set” is a special element in DART syntax that acts as a stand-in or shorthand for a specific bracketed set of elements.For example, a variant set named “key people” is created and contains the following people: Chen, Goldman, Kapoor, Navarro, and Parker. Using variant set syntax, the search term  meet* w/10 <<key people>> runs as meet* w/10 [Chen, Goldman, Kapoor, Navarro, Parker]. This variant set can be used across multiple search terms, and it represents the same set of elements across all search terms.

Variant sets can also be updated at any time. For example, a few weeks after this variant set is created and in use, Chen is no longer considered a key person, and Carter is added to the list of key people. The underlying syntax for the  <<key people>> variant set is then updated to [Carter, Goldman, Kapoor, Navarro, Parker]. All existing and new search terms with this variant set now run with the updated name list. So meet* w/10 <<key people>> now runs as meet* w/10 [Carter, Goldman, Kapoor, Navarro, Parker].

For more information on how to create, use, and maintain variant sets, see the separate DART guide "Variant Sets."

Token Classes

A "token class" is a special element in DART syntax that allows you to search for a set of tokens that have been defined as belonging to the same class. There are nine token classes available in DART syntax: number, currency, percent, date, time, symbol, email, credit card number, and social security number.

Although the search index ignores all non-alphanumeric characters and treats them as whitespace, this does not apply to token classes. Token classes can search for text that includes symbols, punctuation, and/or special characters.

Note: The chart below contains brief descriptions and examples that attempt to explain at a basic level how each token class works. However, the full syntax for token classes is not necessarily represented below, and the full capability of each token class operator can be more complex than what is represented below.

Computational Expense

Token classes can be computationally expensive because they search for a large variety of tokens that tend to have very high occurrence rates across documents. For example, searching for every number, every mention of currency, or every email address across a large number of documents would likely generate a very high number of hits. Search terms that generate a very high number of hits result in slow search execution and drain system resources. Running a token class on its own is also generally too broad and returns too many results.
For most scenarios, token classes are best used as an element within a search term. For example:

• (inventory w/10 tc(number > 50)) AND shipment*
• projected revenue w/50 tc(currency > 100k)
• (m/email_from: tc(email, domain = bofa)) AND (m/email_dist: tc(email, domain = enron))
• stash* w/5 tc(symbol = $)

Custom Token Classes

You may request customized token classes for your project. An administrator will need to assess the feasibility of creating and implementing the request (using regular expressions, etc.). Examples of custom token classes that could be created and implemented for a project include:

• tc(phone number): Could match either of the following token patterns, where N is any digit:
  • (NNN) NNN-NNNN
  • NNN-NNN-NNNN
• tc(hashtag): Could match any token that contains # as its first character.

Flags

Flags are used in DART to "mark" a document (often referred to as "flagging" or "tagging"). Flags are organized in DSR under flag folders. Flags can be applied at the document level, or they can be used to highlight sequences of text in the document. When flags are used to highlight sequences of text, they are called annotations. The flag operator allows you to search for documents marked by flags at the document level or the annotation level. This syntax also allows you to search for documents marked by any flag under a specified flag folder.

The flag operator syntax is relatively robust and can get complex. Use the Query Builder in DSR for help building out a complex search term using the flag operator.

Document Flag vs. Annotation Flag

By default, the flag operator returns both documents marked at the document level and documents marked at the annotation level. Additional syntax can be used to specify those marked at the document level (d) or those marked at the annotation level (a). For example:

• f(flag = Accounting) returns documents marked by the Accounting flag either at the document level, the annotation level, or both.
• f(flag = Accounting, d) returns documents marked by the Accounting flag at the document level.
  • Returns a document that is marked by the Accounting flag both at the document level and the annotation level. Does not return a document marked only at the annotation level.
• f(flag = Accounting, a) returns documents marked by the Accounting flag at the annotation level.
  • Returns a document that is marked by the Accounting flag both at the document level and the annotation level. Does not return a document marked only at the document level.
• f(flag = Accounting, d) NOT f(flag = Accounting, a) returns documents marked by the Accounting flag at the document level, only when the document is not also marked by the Accounting flag at the annotation level.
  • Does not return a document that is marked by the Accounting flag both at the document level and the annotation level.

Annotation Status and Text

Additional conditions can be included to specify criteria unique to annotations: the annotation status (status), or the text highlighted by the annotation (text). For example:

• f(flag = Accounting, Status = Accepted) returns documents marked by the Accounting flag at the annotation level, and where the status of the annotation is Accepted.
• f(flag = Accounting, Text: audit* w/10 question*) returns documents marked by the Accounting flag at the annotation level, and where this annotation highlights text that is matched by the search term audit* w/10 question*.

Flag History

Additional syntax can be used to search the flag history (fh). Searching the flag history returns any document that has ever been marked by the specified flag, even if the flag has since been removed. For example:

• f(flag = Maybe, fh) returns any document that has ever been marked by the Maybe flag. This includes documents that are still marked by the Maybe flag, as well as documents that were once but are no longer marked by the Maybe flag.
• f(flag = Maybe, fh) NOT f(flag = Maybe) returns documents that were once but are no longer marked by the Maybe flag.

Vetting Count and Stack

"Vetting" refers to the process of reviewing a document for responsiveness, and marking the document as Responsive (R), Not Responsive (N), or Maybe (M). A special set of flags called "categories" are used in DART for vetting. Categories contain R, N, and M flags, and these flags represent the responsiveness designation applied to a document. A document that does not contain an R, N, or M flag is called "un-vetted."

Category flags have two extra search conditions that can be used in the flag operator: vetting count and stack (these conditions are not available for regular/non-category flags in DART). Vetting count and stack track each time a document's vetting is changed by a different user. This helps facilitate the vetting cross-check and maybe resolution processes. Vetting count and stack can be used to identify when two users disagreed on the responsiveness designation of a single document, or when one user resolved a Maybe designation on a document applied by another user. However, vetting count and stack do not track when a single user changed their mind while vetting, or when a single user resolved their own Maybe designation.

Vetting count and stack must be used in conjunction with the flag history syntax because they search the history of a flag. Vetting count must be used with flag folder syntax, and stack must be used with flag syntax. For example:

• f(folder = Enron, fh, vcount => 3) returns documents where the vetting has been changed by a different user three or more times.
  • Returns a document where User A vetted the document M, then User B changed the vetting on this document to N, and then User C changed the vetting on this document to R.
  • Returns a document where User A vetted the document R, then User B changed the vetting on this document to N, and then User A changed the vetting on this document back to R.
  • Does not return a document where User A vetted the document R, then User B changed the vetting on this document to N, and then User B changed the vetting on this document back to R (User B changing his/her mind)
• f(flag = M-Enron, fh, stack = 1) AND f(flag = N-Enron, fh, stack = 2) returns documents where the first vet was M, and then the second vet was N by a different user.
  • Returns a document where User A vetted the document M, and then User B changed the vetting on this document to N.
  • Does not return a document where User A vetted the document M, and then User A changed the vetting on this document to N (User A resolving his/her own Maybe designation).

Notes

Notes can be added to a document in DART. The notes operator allows you to search for documents associated with specified note conditions.
The notes operator syntax is relatively robust and can get complex. Use the Query Builder in DSR for help building out a complex search term using the notes operator.

Collections

Collection Operator vs. Collection Search Scope

There are at least two collections involved when the collection operator is used in a search term: a collection associated with the collection operator, and a collection associated with the search scope. The results returned by a search term using a collection operator are always a subset of the documents that exist within the search scope (you can think of it as searching for a "collection within a collection"). For example:

• Collection_01 contains five documents: 101, 102, 103, 104, and 105.
• Collection_02 contains five million documents. Documents 101, 102, 103 are contained in Collection_02, but documents 104 and 105 are not.
• The following search term is run over Collection_02: c(name = Collection_01).
• The search term returns the following results: documents 101, 102, and 103.

Lace Jobs

Additional Lace and Excerpts Features in DSR

Running a search term that contains a Lace operator pulls up two additional features in DSR: Lace View and Term Hits Tab. These additional features display information related to the Lace Job ID referenced in the Lace operator.

When excerpts are generated for a Lace Job, adding an 'e' to the Lace operator syntax pulls up additional features in DSR related to excerpts generated by the job: Excerpts View, Excerpts Tab, and excerpts highlighting.

In the examples below, excerpts have been generated for Lace Job 275:

• L(job = 275) pulls up the Lace View and the Term Hits Tab.
• L(job = 275, e) pulls up the Lace View, the Term Hits tab, the Excerpts View, the Excerpts Tab, and excerpts highlighting.

In the examples below, excerpts have not been generated for Lace Job 275:

• L(job = 275) pulls up the Lace View and the Term Hits Tab.
• L(job = 275, e) prompts an error message that reads "Excerpts have not yet been generated for this job."

Specifying additional features in Lace Job results

There are four additional features related to Lace Job results that can be specified in this syntax:

• L(job = 275, terms =< 3) specifies that a document must be hit by 3 or fewer different terms in this Lace Job.
• L(job = 275, hits > 5) specifies that a document must be hit more than 5 times (by any term) in this Lace Job.
• L(job = 275, nodecount = 1) specifies that a document must be hit by terms in only one Term Management node in this Lace Job.
• L(job = 275, expansions > 10) specifies that a document must be hit by more than 10 expansions (from any term) in this Lace Job.