Scrunch | How-to guides - How to identify website optimization opportunities for AI search

How to identify website optimization opportunities for AI search

Site auditing

Use this guide if you need to:

Contents

🚧 Scrunch recently shipped major product updates, so some parts of this guide may not match what you see in our platform today. We'll be updating the guide to match, but if you have questions in the meantime, please:

Quick answer

Identify website optimization opportunities for AI search from Scrunch's Site Maps tab.

Website optimization for AI search starts with knowing which pages are working and which aren't. Here's how to find out:

Start in the Site Maps tab for page-level performance metrics:

Drill down in the Page Details page for deeper analysis and actions:

Track changes over time: Use the date selector (default: last 12 weeks) to monitor how page performance evolves across any time period.

Bottom line: Knowing where to start with website optimization is half the battle—prioritize pages where AI is already showing up but not following through with citations.

Start here

Why you should identify website optimization opportunities for AI search

Website optimization—aka fixing the technical and content issues that prevent AI from reading and recommending your content—is how you turn monitoring into action. If AI bots can't access or understand your content, you won't show up in AI answers. To optimize AI search performance, you need to know which webpages are working, which aren't, why, and where to start.

Here's how to do it using Scrunch.

Before you begin:

Step 1: Map website architecture and view performance metrics in the Site Maps tab

Log in to Scrunch and navigate to the Site Maps tab.

This page is your command center for evaluating and optimizing website performance in AI search.

Scrunch automatically crawls your site and maps out webpages and related subpages the way AI user agents see them. Performance metrics for each page are refreshed throughout the day as new data is collected.

At the top of the page you’ll see options for Domain Insights and Page Audits & History.

Scroll down the page and you’ll find a table featuring every mapped page on your website. The number of pages you can map is dependent on your plan level.

The table is displayed in a hierarchical view based on your website structure, but you can change the view using the buttons directly above the table. This includes the ability to display pages in a view sortable by specific metrics or rolled up under certain page types.

You can also use the search bar above the table to locate specific pages.

In addition to page title and URL path, the table shows you:

You can see page performance over time by using the date selector at the top of the table (it’s set to the last 12 weeks by default).

Click the date selector button to view your options or set a custom date range.

This data makes it easy to quickly assess site-wide AI visibility health and identify high-impact optimization opportunities minus manual analysis.

Use it to explore your website structure and prioritize pages in need of review.

💡 Pro tip: Besides low Audit Scores, pages with high AI agent traffic but low citations are prime candidates for further investigation and optimization.

Once you spot priority pages, it’s time to drill down to see detailed performance, run page-level audits, optimize content, and improve content delivery.

Step 2: Explore performance and take actions in Page Details pages

Click into a page to open its Page Details page.

At the top of the page you’ll see options for:

You’ll automatically land in the Performance section. This shows you how this page stacks up across different AI platforms for Agent Traffic, Citations, and AI Referrals.

Navigate to the Diagnostics section.

At the top of the page, you’ll see:

💡 Pro tip: Unless the content on your site is monetized, Scrunch recommends giving AI bots access to your site. The more that LLMs know about your brand, products, and services, the likelier it is that they’ll represent them accurately in AI responses.

Farther down you’ll see a visual rendering of what humans see when they visit the page versus the text content that AI agents see when they visit the page.

Navigate to the Prompts & Responses section.

This section surfaces tracked prompts that have recently generated AI responses that cited this page, as well as prompts that have not resulted in AI citations for this page but should.

You can filter the prompts based on which ones currently cite or don’t cite the page using the dropdown menu in the top right corner.

💡 Pro tip: Scrunch recommends prioritizing high-intent, non-branded prompts that represent core product categories and audience demand. Identify and fill content gaps by updating or creating content that exceeds the quality of content currently being cited as a source by AI platforms.

Click the “Add filter” button at the top of the page to view your filter options.

You can navigate to the prompts on this page in the Prompts Monitoring tab at any time for a deeper dive.

Navigate to the Deep AI Audit section.

This section allows you to run an audit of the page to see:

This provides specific, actionable insights around AI bot accessibility, technical problems, and content issues.

All audited pages can be found in Page Audits & History.

What’s next

Start optimizing your website:

  1. Run a site audit: Fix access, delivery, or content issues hurting your visibility.
  2. Prioritize technical updates: Make sure AI bots can access and read website content before focusing on content updates or creation.
  3. Update or create content: Review competitive sources and update or create content to exceed content quality.