Running reviews

In a legal team, reviewing rules are typically built by senior lawyers or knowledge managers. (It is often a good idea to even hide the Manage reviewing rules from the team members.)

Once built and tested, they are meant to be shared with the rest of the team.

You can also turn reviewing rules into checklists, to allow for a more manually-oriented workflow.

You can run the rules by clicking on the Review document icon on ClauseBuddy's home page.

Note that this icon may be missing on your homepage, either because your administrator disabled it, or because there are no reviewing rules defined yet to which you have access.

---

Step 0: Opening a Word-document

Because ClauseBuddy will interact with your currently opened MS Word document (e.g., in order to rewrite text, highlight paragraphs or lead you to certain paragraphs), it is crucial that you open a Word-document and then start ClauseBuddy in it.

If you want to review a PDF-file, you need to first convert it into MS Word. You can do so easily from within ClauseBuddy by clicking on the Upload PDF area in the bottom-left corner, or by dragging & dropping your PDF file on that area. ClauseBuddy will then convert the PDF-file into a DOCX file and open it in a new window for you; you can then open ClauseBuddy in that new window.

---

Step 1: Selecting a Review Set

In the first step, you need to select which rules you want your opened Word-document to check against. Simply click on the Select button next to the one you want and press Next.

---

Step 2: Selecting the rules to include

By default, all rules within the review set you selected will be included. You can leave out those you don't want.

Interestingly, you can also add an Addhoc rule. This allows you to specify a temporary requirement, on top of the ones that are stored in (i.e., predefined by) the rule set.

The first box contains the title (meant for short pieces of text), in the second box you can be more elaborate towards the LLM on what it should check for.

---

Step 3: Answer questions

In this third step, you can optionally answer certain questions referred to by the rules you checked in the previous step.

The answers to these questions can sometimes be crucial for the LLM to correctly review the document. For example, if some rule would state that only the state courts of the country of delivery are acceptable, then it may need to be explicitly stated what that country of delivery is.

Remember: other than generally known public information, the information available to the LLM is limited to what is explicitly written in the document.

Step 4: Perform the review!

In this step, you will finally send the document and reviewing rules to the LLM.

However, before doing so, consider the following:

• If multiple LLMs are installed in your account, you may need to make a selection of the relevant LLM first (by pressing the right triangle), otherwise the previously chosen LLM will be used:

• In the upper-right corner, you can choose the Quality setting through the "..." menu.

Usually, you can leave the quality to Medium. Only move this setting to High if you notice that the LLM:

• is losing precision across large documents (e.g., good results in the beginning, then deteroriating towards the middle)

• is unpredictable in whether it seems to understand certain complex rules (e.g., one time the LLM does seem to perfectly grasp what's being asked, the next time it doesn't)

With the High setting, ClauseBuddy re-sends the document multiple times, in order to get better results. Usually this is not necessary, but it may be worth to try it.

The High setting is much slower, and uses up to 5x as much capacity. Be aware that in most subscriptions of ClauseBuddy you are charged by your usage.

---

Step 5: Evaluating the results

The results will be dripping in from the LLM, ready for you to inspect:

The following color codes are assigned by the LLM:

• Dark green means "requirement is fully met"

• Lighter green means "requirement is mostly met", i.e. most aspects are met

• Orange means "partially met", i.e. only some aspects

• Red means "not met"

• Grey means "irrelevant"

You can hover over the info-icon at th right side of each reviewing result to see the body of the rule:

You can set the Status of a review result to anything you like. This allows you to write down your own assessment of what the LLM is telling.

Below the assessment, you will usually see the original paragraph(s) that were deemed relevant by the LLM for the rule. You can click on the paragraph to directly jump towards that location in your document and select the corresponding paragraph.

You can also click on the black Rewrite selected text button in order to rewrite the text that is currently selected in the document. ClauseBuddy will then take that selected text and send it to the Redraft module, where it will be automatically redrafted in order to fit the contents of the rule (and/or the additional redrafting hint that may have been specified in the ruleset).

Feel free to select the relevant text in your opened document before clicking on the black Rewrite button.

If the ruleset author also attached one or more clauses to your rule, then they will be shown at the bottom. Similar to how the Quality Library works, you can insert clauses into your current document through the + button in the top-left corner of the clause.

Step 6: Exporting the results

You can export the review results to a report by clicking on the Export button and choosing either the default export format, or (when that's configured by your administrator) one of your branded custom templates.

You can also export the review results to a Checklist, and continue your manual review on the basis of the results initially prepared by the LLM.