Managing optimization projects for large e-commerce sites is one of the tasks we encounter most often at Zeo. Coordinating a team responsible for on-site SEO and improving a site of that scale takes considerable effort. Fortunately, the right tools can reduce the amount of manual work involved.
For any site with more than 100 pages, crawl analysis tools make it much easier to see the overall picture. They help you assess how well your pages meet SEO requirements in less time and without checking each page manually. In this article, I will introduce several tools that we frequently use in client projects at Zeo.
Some of these tools are paid and others are free, but I see them as complementary rather than interchangeable. Using them together helps cover the gaps in each tool and can produce a more effective analysis. This gives you more control over your on-site SEO work and helps you build stronger optimization strategies.
Moz Pro
Moz Pro lets us review the on-site SEO condition of projects of any size through reports written in clear, accessible language. I call the reports "understandable" because even someone without advanced SEO knowledge can understand the site's current condition.
Moz Pro also offers competitor analysis, ranking tracking, and link analysis alongside its on-site SEO features. For this article, however, I will stay focused on on-site SEO.
We begin by creating a campaign and entering the details of the website we want to analyze. The site then needs time to be crawled. This can take a few days, particularly for large e-commerce sites with many subpages. Once the analysis is complete, we reach one of the most important stages: evaluating the results accurately and viewing the site from Google's perspective. That evaluation determines which changes we need to make.
The "Crawl Diagnostics" tab lists all analyzed pages and their status. The example shown contains results from a medium-sized e-commerce site. This page explains the errors found and the areas that need attention. On the right, you can see which issues occurred and approximately how many pages they affected. The menus under the "Show" button let you review issues individually or group them under "error", "warning" and "notification".
Moz classifies the issues that definitely need attention as 'errors'. These include 4xx errors, 5xx errors, missing or unused headings, double page content, double page headings and pages that are blocked by robots.txt. The list may look extensive at first, but with the right strategies, you can watch these problems gradually decline. If any error definition is unclear, Moz provides a dedicated page with further details.
Moz continues running crawl analyzes periodically until you end the campaign, keeping you informed about the site's status and progress. You can also export this progress in PDF or Excel compatible form and use the report to guide further analysis and improvements.
Screaming Frog is a truly indispensable program to fully understand your website's performance in organic searches and to ensure that Googlebots can easily navigate your site thanks to the changes you make.
Screaming Frog is a user-friendly desktop program that is fully suitable for SEO processes. Especially for large projects, it has an important role, as it doesn't have a 10,000 or 20,000 page analysis limit like the Moz Pro tool. In this way, you can get a lot of different analysis results by reducing a control process that would take hours manually to minutes. Considering that Google does not even share some of this data in its own service Search Console, we can say that SEO Spider does serious data mining.
I would like to talk a little bit about its usage. It is quite simple to start the process immediately after downloading and installing the program. You start the process by simply entering the URL of the website you want to research in the address bar and then pressing the "start" button. Once the process is complete, you will be presented with a list of different types of results and data indicators.
We can filter this data according to page structures and use it to see and analyze the results we want. If we give an example about the contents of the categories; Under the category called "Internal", we can easily see the links within the site and which pages have links between them.
In addition, as you can easily see under the "status code" column, we can check the status of the linked page, whether it is problematic or not, or whether it is redirected in the way we want.
The "External" tab is no different. In general, it talks about off-site links and their condition. In addition, the full link, the content (image, text, video, etc.) and the redirection patterns (status code) are specified.
Other important analysis information that Screaming Frog provides to its users are "Page Titles" and "Meta Descriptions". By examining these analysis results, we can easily detect double page titles and missing / missing meta description problems. In addition to these; h1, h2, etc. You can easily access a lot of data, from the use of headers to the analysis of visual files.
You can also analyze pages and applications such as AMP and Structured Data with this tool, while you can expand your analysis by establishing API connections with many tools such as Google Search Console, Google Analytics and PageSpeed Insights.
As I mentioned at the beginning of my article, it is very important to use the tools simultaneously because you can reach a problem that you cannot reach through this program with another tool.
BrowSEO
Although Screaming Frog and Moz Pro allow us to analyze our websites for everything from server errors to double page header issues, BrowSEO is an indispensable tool to fully analyze the site and better understand how Googlebots see our website "from the user's perspective".
BrowSEO is positioned differently from other programs with its very simple use and analysis results. Although the information it provides is much more limited than other tools, it reveals exactly what kind of structure your website has. I would like to enlighten the subject by talking about its immediate use.
When you type the address of your site directly into the address bar, you get a basic HTML view of your homepage and some analysis reports on the right. BrowSEO provides visual and numerical information on what kind of link system you have, especially for large-scale websites, by indicating on-site, off-site and nofollow links in different colors (yellow for on-site links and red for off-site links). Under the "Head" heading, it includes basic information about the website such as title, meta descriptions, infrastructure system, etc.
Thanks to this service, you can also work on preventing some major problems, as you can literally see your website through Google's eyes.
For example, you can see that the content that you attach great importance to being on your website, but that you have created with structures such as flash, AJAX, etc., is not perceived by bots and you can take measures accordingly.
Other outstanding features of BrowSEO are that it provides information about all the headings on the page, how it will appear in search engines in a possible search, and the so-called "Cloaking-Attempt", as you can see in the example above, which tests whether the pages appear the same way to Google's bots and organic visitors.
This system, which provides visual information in general, is served completely free of charge in a web-based manner. It is possible to obtain the outputs of all these analyzes from the "Download entire session" section.
Google Search Console
We can easily say that the most basic way to understand Google's on-site evaluation algorithms is to get information from Google. Therefore, using Search Console actively and effectively is a must for long-term and lasting on-site SEO success.
The best thing about this tool is that it is absolutely "rookie-friendly". Google simplifies things considerably by making adjustments in a way that is suitable for an ordinary user who has no knowledge or experience on the subject. Another feature that distinguishes Search Console from other services is that it is free. In addition to this, Google makes its service as usable as possible by offering its possibilities for mobile users to the fullest.
Undoubtedly one of the biggest advantages of Search Console is to see your website through Google's eyes. Although BrowSEO has the same situation, it does not provide as detailed information as Search Console.
It's also possible to see which keywords are directly caught in Googlebot analytics. If the keywords you are targeting are not being detected by Google, you have the opportunity to adjust your situation on the subject by developing content strategies accordingly.
One of the most advantageous points of Search Console is that it visibly reveals crawl errors with historical changes. Google notifies the website owner of the results that come directly from its bots in this way and offers the "Mark as fixed" option for users to be notified in case of correction.
In addition, by using the "Fetch as Google" tool, you can see the indexing of the main and sub-pages of your site through Google's eyes. Depending on these outputs organized in the form of a tree structure, you can see the parts that may cause negative SEO performance and take measures to prevent them.
As a result; All 4 tools I mentioned above have different and really effective services. I recommend using all of these tools simultaneously to perform a comprehensive and high-quality on-site SEO analysis and achieve the most accurate results. Although it may seem time-consuming and time-consuming, after a few practices, I recommend that you use the trial versions of the paid tools and the direct services of the free ones to have more detailed information about these tools, which I believe will get your work done in a quality way.
Bonus
DeepCrawl
DeepCrawl tool, like the Screaming Frog tool, simulates search engine spiders and crawls websites and creates issues as a result of this crawl and presents us with technical errors that need to be corrected on our site; duplicate meta tags, pages with weak content.
If you prefer to use the DeepCrawl tool; I strongly recommend you to take a look at the DeepCrawl SEO Tool Guide content prepared by Metehan Urhan from our team.
