Task Testing
Part of AI research tools
Comparing document-grounded research with open-web research
Choose between selected documents and open-web sources, compare their gaps and reconcile conflicting answers.
Use a defined document collection when the question is what those approved documents say. Use open-web search when you need public information beyond the collection, or a check for change. Some decisions need both: establish the internal position, find relevant public developments, then have the document owner reconcile any conflict.
Set the source boundary
Write the question before choosing a mode. “What does our approved service policy permit?” is a document question. “Have the provider’s published requirements changed since that policy was approved?” needs a current public-source check. Combining them without identifying which source controls each part can obscure a conflict.
Record a document’s owner, version and effective date. For web material, record its original publisher, publication or update date where given, and access date. An approved document may govern an internal process while being too old to describe an external product. A newer provider page may show a change but does not itself amend an internal policy.
| Issue | Selected documents | Open web |
|---|---|---|
| Best use | Answer from a named, authorised collection | Discover or check public material |
| Main gap | Missing or superseded documents | Missing, stale, inaccessible or weak pages |
| Evidence check | Open the cited document passage | Open the original publisher’s page |
| Stop point | The collection lacks or conflicts on the answer | No suitable public source settles the point |
Check what each product actually searches
For Gemini Notebook, consult the current help pages for chat and source discovery, then record the source-selection options and settings actually used. Do not describe an answer as limited to selected documents unless the active mode and its behaviour establish that boundary.
For Perplexity, consult current help material on Internal Knowledge Search and file uploads. Record the active source mode and any stated limits; do not assume organisation files, web sources or full-file coverage without verifying them.
For ChatGPT Deep Research, consult current help material and account or workspace settings before defining its source boundary. Record the sources actually available and used where the interface makes them visible; do not infer completeness.
Key Source Verification Practices
- Gemini Notebook: Check active source mode in help docsVerify settings before claiming document boundary
- Perplexity: Confirm Internal Knowledge Search statusCheck file upload and coverage limits
- ChatGPT Deep Research: Review account/workspace settingsRecord visible sources used; don’t assume completeness
Compare without changing the evidence problem
Prepare an authorised, non-sensitive document set containing a clear answer, an exception and one fact it cannot establish. Ask the document workflow to answer from that set and identify the gap. Ask a web workflow a separate public question and name the kind of primary page that could settle it. Do not give one workflow extra facts and call the result a better interpretation.
For each material claim, record whether it is supported by the selected documents, supported by a current primary web page, conflicting or unresolved. An honest “the documents do not answer this” may be the correct document-mode result. Check whether a web answer imports the wrong region, edition or historical page.
For Australian business files, check applicable privacy obligations and internal approval before using personal information in any research workflow.
Finish with two statements: what the approved collection establishes and what public research adds or calls into question. Record the public check’s date and scope so later reviewers can tell what it did—and did not—cover.
