Website

Find public pages, review them, and train the selected pages before the agent can use them

Website#

A website source in Agent Studio → Data sources is one lifecycle. The source row, the page list, and training all use the same status. Finding pages only saves addresses. The agent cannot answer from a page until that page is extracted and the source is trained.

Add a website#

Open Agent Studio → Data sources.
Choose Add source, then Website.
Choose how to find addresses:
- Get Links starts from the public address you enter.
- Sitemap uses a sitemap address.
- Individual Link saves one page address.
For Get Links, open Advanced options if you need to limit how many links are discovered or which paths are included or excluded.
Start discovery. This does not download page content and does not train the agent.
When the status is Ready to train, open Manage pages, review the addresses, and choose Train agent.

Use a public address. Password-protected pages are not included. Documents that are not web pages belong in Files. Options for reading text in images are in Training options after links have been found.

Website status#

Status	What it means
Finding pages	Discovery is still running. Addresses found so far are not downloaded or trained.
Ready to train	Discovery finished. Review the pages, then choose Train agent. Content has not been downloaded yet.
Queued	Work is waiting. Do not start it again.
Extracting pages	Selected pages are being downloaded.
Building knowledge	Extracted pages are being added to the agent's knowledge.
Trained	Extracted pages are available. Check coverage, then test.
Retry queued	A retry is already scheduled. Wait unless it stops recovering.
Recovery queued	Discovery stopped reporting progress and will be picked up again.
Sync paused	Processing stopped and is not retrying on its own. Retry the sync.
Needs attention	Open the error and any failed pages, then use the offered recovery action.
Removed	The website is out of active knowledge. Restore it, or permanently delete it.

While the status is Finding pages, Queued, Extracting pages, or Building knowledge, follow that progress instead of starting the same work again. You can stop discovery, stop training, or stop the website sync while that work is running.

Page list#

Manage pages is the same lifecycle, not a second status. It shows how many addresses were discovered, which are selected, which have been extracted, and which failed.

A failed page does not mean the whole website failed. Open Review failed pages, inspect that address, and retry the page when the action is offered. If the page moved or was deleted, change the site or the path filters instead of retrying it forever.

During a refresh, do not assume new pages are usable because they were found. Follow the status above and the extracted coverage before treating the new pass as ready.

After the first train#