How to Fix 404 Not Found Errors on Your Website
How to deal with 404 not found errors on your website
404 not found errors are common across the internet. Many times you stumble upon a link and when you click on it, you are taken to a 404 error page that says the content does not exist and you should instead try to use the search feature. If you are trying to manage the 404 errors on your website, you need not be too concerned either. There are some methods to manage these 404 errors on your website.
It is common across the web that old content gets deleted and new content is published. If there are links on the web somewhere that refer to the deleted link, those links will return 404. However, Google will crawl these links whether it finds them on your website or another source on the web. It reports these urls in the Google Search Console in Page Indexing -> Not found (404).
There are some pages that you might have intentionally deleted from your website and therefore, they are bound to be 404. However, sometimes a typo or a mistyped link url on another website can also lead to a 404 page. If you are seeing a large number of 404 errors reported in the Google search console, you might want to review them and fix as many as possible.
According to Google 404 errors do not have any impact on SEO or rankings and therefore you must not be worried about 404 errors on your website.
However, the question still remains how to deal with these 404 errors.
There are a few methods that you can adopt to deal with these 404 errors and a site audit might help you get rid of some of these links from your search console.
Fix 404 errors originating from your own website
404 errors might be happening because there are a large number of such links referenced inside the posts on your blog. When Google crawls your website, it crawls those urls also and then reports them in the page indexing report as 404 errors.
As a first step to get rid of 404 errors that might be originating from your own website, you must get your website crawled to find all those broken links that return a 404 status.
To fix 404 errors by fixing the broken links resulting in 404s on your website, you can use the free broken link checker by brokenlinkcheck.com. The free broken link checker can crawl 3000 pages and detect broken links. If you have a larger website, or you need to crawl more than 3000 urls, you can get a quote for the commercial version of the tool.
The tool will crawl and show all the broken links as well as the source pages on which these links were detected. If you ever changed a url slug for a category, deleted some posts or tags that were earlier referenced inside the content, chances are these broken links may be accounting for a large part of the 404 errors in your Google Search Console.
You can remove the urls from individual pages or just use search and replace to replace those links with something new. If you are using Wordpress, you can easily search and replace all occurrences of those broken links using the search and replace plugin.
However, Wordpress also offers a broken link checker plugin that can be used to detect and fix all those broken links on your website. Getting rid of those broken links on your website will offer a dual advantage.
First of all, you will be able to reduce the 404 errors on your website. Second, broken links hurt SEO and crawl budget. By fixing broken links on your website, you will be able to improve SEO and search rankings. It might also boost the traffic on your website. After fixing the broken links, wait for a few days and then if you check again, the fixed urls would have stopped showing in the 404 list in the search console.
Some 404 errors originating from your own blog might be caused by deleted posts, pages, categories or tags or a change in url structure and so on. Sometimes, we delete content that has grown old and useless. However, if such urls are still referenced inside the website, Google will keep crawling them and reporting them in the search console. So, you can remove the mention of such urls on your website or if there is fresh useful content to replace the old content, you can just safely redirect such deleted urls causing 404 errors.
Dealing with 404 errors with an external source
While it is easier to deal with the 404 errors that are originating from your own blog/website, it is not as easy to fix the 404 urls originating from an external source.
For example, one of my blogs hosted on Wordpress was reporting a large number of 404s which were mostly partial or broken urls. I reviewed some urls reported in the search console and found that they were originating mostly from an external pdf sharing homework site. In some cases, there was a typo, and in some only half the url was reported since the user might have made a mistake in the document he had uploaded to the pdf sharing site. Now, since I want to retain these backlinks, I must redirect them.
Inspect 404 urls to know their origin
It depends whether you want to keep those links or want to get rid of them. Do you think the source site is good and can be kept for links? If the external source quoting the url is a reputed one, the best option is to redirect the broken link to the functioning url on your website. You can create redirects inside the .htaccess file or use the Redirection plugin for Wordpress.
It is easy to decide whether you want to redirect a url or ignore it. If the url is linked from a reputed website, you might redirect it to a functioning url and otherwise not. To find out the source where Google found the 404 url, you need to go to the Google Search console and check out the page indexing section.
Open the list of 404 urls and click on any url and then inspect the url. Google will show you the source url where it found the 404 link in the report.
If you think that some spam website is creating such links, you can ignore them instead of redirecting them or you can 410 redirect them.
If possible you can also contact the website owner and request them to fix the typo in the url. However, it is not possible in all cases and therefore the safest thing to do to retain the SEO juice is to redirect them to the valid url.
In several cases, many of these 404 errors are gone by themselves with time since the external source where these urls are linked from might remove those links or fix them.
What to do when your website is bombarded by 404 errors and broken links from a single external domain?
It may happen that a single domain might end up creating hundred or rather thousands of 404s. Here is a clever remedy to fix such external broken links and 404s.
How to redirect 404 pages?
Redirecting 404 pages for Wordpress users is very easy. Just make a list of 404 urls that you want to redirect and the target urls and set 301 redirects using either the redirection plugin or the .htaccess file if you are using Apache server.
Install the Redirection plugin on your wordpress website and enter individual urls and target urls to redirect them properly. However, if you do not want to use a plugin, you can instead use the .htaccess file for creating redirects.
For example, you have a 404 url link:
https://example.com/broken-404-link/ and you want to redirect it to https://example.com/new-url/
You can set this redirect in the .htaccess file in the following manner:
Redirect 301 /broken-404-link/ https://example.com/new-url/
If you have deleted an old category and want to redirect all the post urls in that category to a new url:
RedirectMatch 301 /old-category/(.*) /new-category/$1
In case you do not want to redirect a 404 url, you can either ignore it or just 410 redirect it:
Redirect 410 /old-broken-url/
Redirect 410 /old-category/
A few things to be careful about:
If you are experiencing a surge in the number of 404 errors in the Google search console, there is no reason to worry. You can fix them and possibly get rid of a large number of annoying 404s in the search console by fixing the broken links on your own site.
First of all, you are not going to see any significant impact on your SEO or rankings caused by 404 errors. However, it might indicate that there is something wrong with your website.
Hacked websites can also experience a sharp rise in the number of 404 errors. If you see a sudden surge in such urls that have never existed (not broken urls or typos), it could indicate that your website has been compromised.
For example, you do not have pdfs in your website but you see a sudden increase in 404 urls ending in .pdf inside the search console. It could mean that your site has been compromised and somebody injected those urls there.
To get rid of such urls, you must let google crawl them and not block them using the robots.txt file.
Do not ever block 404 urls with robots.txt since that might cause these urls to get indexed. You may come across indexed though blocked by robots.txt inside your search console. So, if you want to get rid of malicious urls giving a 404 status, there is no need to block their crawling. If google does not find any reference to such urls on your website or external sources, it will stop crawling them and they will automatically drop from the list.
Do not use the url removal tool inside the search console to get rid of 404 not found urls. The url removal tool works only in the case of the urls that are inside Google’s index and not for 404 or urls marked as no index or not indexed for some other reason. Use the url removal tool only to get rid of urls that are not listed in the sitemap but are somehow indexed and which must not be there.
Fix 404 errors by reviewing the 404 urls in the search console. Inspect these urls and find out the source from which they are originating. If they are originating from a malicious source, you can disavow the source.
A large number of 404 urls might be originating from your own content. You may have linked to these urls in the past but they are not there any more. Google is still crawling those urls on the pages from which they were linked. To get rid of these urls, use a broken link checker which will help you find all urls on your website that are 404.
You can do the following things with 404 urls:
- Redirect them to a valid url if you think these urls can help you gain traffic or backlinks.
- Ignore 404 urls
- 410 Redirect them to show they have been intentionally and permanently removed.
- Remove such broken urls from your own website
- Restore deleted pages, categories, tags etc or 410 redirect them
Note: 410 redirects can be helpful to get such 404 urls which are not returning to be removed from your search console data faster.
Additional Resources:
Google webmasters Guidelines for 404 Not Found Errors
Suggested Reading
How to Analyze and Recover from a Drop in Website Traffic
How to Use Cloudfront CDN with Custom CDN Domain in Wordpress