Site Slow or Unreachable (High Number of 404's)

Overview

This article discusses the potential issues and fixes for slow websites returning a high number of 404 status codes for images and other missing assets.

Identifying the Problem

Have you noticed one or more of your sites becoming slow or possibly unreachable? The issue might be related to a large amount of 404 (Not Found) response codes. To determine if this is the cause of your performance issues, follow these steps:

(If you have multiple nodes, you will need to check each node using the following steps)

1. SSH to your VPS 
2. Tail the gateway log file:
   

   tail -1000000 /mnt/log/nginx/gateway.access.log | awk '$NF > 2 && $10 == "404" {print $6$8}' | sort | uniq -c | sort -nr | head

If you are seeing an abundance of output from image assets, then you might be a candidate for the fix outlined here. Large amounts of 404 responses will take a lot of CPU cycles on high traffic sites. Apply the fix below to see if it corrects your problem.

Potential Fixes

Cheaper 404 Images

If you have verified that the problem is a large number of 404 (Not Found) response codes for image assets, add the following lines to the .htaccess file for each of the impacted sites.

(If you have multiple nodes, you will need to apply the changes on each node that hosts the sites with the 404 response codes)

# Pagely Admin edit -- cheaper 404 for images
# To not crash the server due to overloading
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule \.(ico|gif|jpg|jpeg|png|swf|flv|pdf|map|mov|svg|mp4|mp3|zip|eot|ttf|woff|woff2|ttc|otf)$ - [nocase,redirect=404,last]
</IfModule>

Note: You might need to clear your cache after applying this change. To do so, follow the steps found in the How do I purge all the cached pages/assets for my site article.

For additional assistance, please do not hesitate to contact Pagely Support!