Media Library Latency, Lag, CRON?

When I add images to my Media Library they aren't available in the Media Library for about an hour. It seems like there is a queue or some sort of CRON that fires. The image appears in the media library as a transparent image for the hour.

How can I get the image to load when I upload?

The images were working previously but I'm now uploading to S3 with a CRON that fires every minute. Maybe that relates to the issue.

Edit Panel which shows the image correctly:

Media Library which shows break:

I have a CRON that runs a sync with the S3 bucket. I then have a proxy rewrite that serves my local dir from the S3.

so for example ls -lah /var/www/html/example/wp/wp-content/uploads/2018/12/ gives me:

-rw-r--r-- 1 apache apache 5.1K Jan 18 23:32 example-150x150.jpg
-rw-r--r-- 1 apache apache  11K Jan 18 23:32 example-272x182.jpg
-rw-r--r-- 1 apache apache  12K Jan 18 23:32 example-280x280.jpg
-rw-r--r-- 1 apache apache  11K Jan 18 23:32 example-300x169.jpg
-rw-r--r-- 1 apache apache  34K Jan 18 23:32 example-660x370.jpg
-rw-r--r-- 1 apache apache  41K Jan 18 23:32 example-768x432.jpg
-rw-r--r-- 1 apache apache  48K Jan 18 23:32 example-800x542.jpg
-rw-r--r-- 1 apache apache 170K Jan 18 23:32 example.jpg

then:

.jpg
.jpg
.jpg
.jpg
.jpg
.jpg
.jpg
.jpg

all load for me (one minute after the upload).

When I add images to my Media Library they aren't available in the Media Library for about an hour. It seems like there is a queue or some sort of CRON that fires. The image appears in the media library as a transparent image for the hour.

How can I get the image to load when I upload?

The images were working previously but I'm now uploading to S3 with a CRON that fires every minute. Maybe that relates to the issue.

Edit Panel which shows the image correctly:

Media Library which shows break:

I have a CRON that runs a sync with the S3 bucket. I then have a proxy rewrite that serves my local dir from the S3.

so for example ls -lah /var/www/html/example/wp/wp-content/uploads/2018/12/ gives me:

-rw-r--r-- 1 apache apache 5.1K Jan 18 23:32 example-150x150.jpg
-rw-r--r-- 1 apache apache  11K Jan 18 23:32 example-272x182.jpg
-rw-r--r-- 1 apache apache  12K Jan 18 23:32 example-280x280.jpg
-rw-r--r-- 1 apache apache  11K Jan 18 23:32 example-300x169.jpg
-rw-r--r-- 1 apache apache  34K Jan 18 23:32 example-660x370.jpg
-rw-r--r-- 1 apache apache  41K Jan 18 23:32 example-768x432.jpg
-rw-r--r-- 1 apache apache  48K Jan 18 23:32 example-800x542.jpg
-rw-r--r-- 1 apache apache 170K Jan 18 23:32 example.jpg

then:

http://example/wp/wp-content/uploads/2018/12/example-150x150.jpg
http://example/wp/wp-content/uploads/2018/12/example-272x182.jpg
http://example/wp/wp-content/uploads/2018/12/example-280x280.jpg
http://example/wp/wp-content/uploads/2018/12/example-300x169.jpg
http://example/wp/wp-content/uploads/2018/12/example-660x370.jpg
http://example/wp/wp-content/uploads/2018/12/example-768x432.jpg
http://example/wp/wp-content/uploads/2018/12/example-800x542.jpg
http://example/wp/wp-content/uploads/2018/12/example.jpg

all load for me (one minute after the upload).

• media-library
• wp-cron

Share Improve this question edited Jan 18, 2019 at 23:42 user3783243 asked Jan 18, 2019 at 22:32 user3783243user3783243 10333 bronze badges 7

• This is very unusual, in my experience, images are available instantly once upload completes. That you've changed things around to use S3 definitely looks like it might be responsible, are you using a plugin? There's no code in your question, and no debugging information such as URLs that work vs don't work, etc. Keep in mind that if this is a 3rd party plugin that you're having issues with then you need to take it up with their support, 3rd party plugins themes and services are offtopic here – Tom J Nowell ♦ Commented Jan 18, 2019 at 23:18
• Can you provide an example of a URL that works and a URL that doesn't? If you've set up a system cron to sync to S3, then no amount of WordPress know how will be of help as it isn't a WordPress problem but a general Linux problem. I don't know how long it takes for a file to become accessible once it's uploaded to S3, but I do know it has nothing to do with WordPress. Are you sure this is the best place to ask? WP doesn't do anything cron related to images on upload, it just uploads and it's instantly available, there's no delayed anything – Tom J Nowell ♦ Commented Jan 18, 2019 at 23:32
• I very much doubt WP has anything to do with this and that the problem lays entirely within your S3 uploading mechanism. My guess is that when you upload a file, it doesn't show because it's on the filesystem but not in S3, and your proxy redirects to a non-existant file which gives a 404. It shows in the image editor because it's being served by PHP from the local filesystem. No WP help will solve this, you need Amazon S3 knowledge, and this isn't the place for it – Tom J Nowell ♦ Commented Jan 18, 2019 at 23:34
• 1 If they all work then what's your question about? A full image is not the same URL and file as a medium, or a thumbnail, etc etc. Clearly there's a broken image in your question, right click on it and grab its URL. Additionally, wouldn't these show up in the browser dev console? – Tom J Nowell ♦ Commented Jan 18, 2019 at 23:40
• 1 For example, there's a screenshot of a broken thumbnail image in the media library, what's its URL? Does it 404? – Tom J Nowell ♦ Commented Jan 18, 2019 at 23:43

 |  Show 2 more comments

1 Answer 1

Sorted by: Reset to default 1

When I add images to my Media Library they aren't available in the Media Library for about an hour. It seems like there is a queue or some sort of CRON that fires.

WordPress handles uploads immediately, there is no latency, queues, or cron unless you introduce one yourself.

The image appears in the media library as a transparent image for the hour.

The images were working previously but I'm now uploading to S3 with a CRON that fires every minute. Maybe that relates to the issue.

This is the reason why it doesn't work, 100%. At a minimum there will be a minute of latency as a result of what you've done, assuming that the files are served instantly from S3, but that may not be the case. There may be latency on S3, but that's unconnected to WordPress.

So, lets pick that part:

1. S3 can take time for files to become available, a quick google search shows people who uploaded things but they took 24 hours to become available. You'll need to configure your S3 buckets appropriately, but that's not a WordPress question or problem.
2. Your setup doesn't improve your performance much, at best it might shave a little bit of bandwidth usage, but your server still gets hit for every request for an uploaded image
3. There are a tonne of plugins for this, and they rewrite the attachment URLs to go straight to S3, avoiding the need for a proxy, and even do the uploading on upload, so no need for a cron job either