[Feature] Upload from URL: Replace "1280" with "raw" in Tumblr links automatically

In category: Site Bug Reports & Feature Requests

Requested feature overview description.

If a user tries to upload an image directly from Tumblr, replace "_1280." with "_raw." automatically and load that image instead.

This assumes that every image has a "raw" link.

May also apply to "_500.", "_540." or others.

Why would it be useful?

Prevent some users from uploading lower quality images on accident.

What part(s) of the site page(s) are affected?

Post upload


Lance_Armstrong said:
Requested feature overview description.

If a user tries to upload an image directly from Tumblr, replace "_1280." with "_raw." automatically and load that image instead.

This assumes that every image has a "raw" link.

May also apply to "_500.", "_540." or others.

Why would it be useful?

Prevent some users from uploading lower quality images on accident.

What part(s) of the site page(s) are affected?

Post upload

Should have a way to determine if a raw link leads to something, just in case, using the given link if not.

Otherwise, sounds like a good idea.


kamimatsu said:
Should have a way to determine if a raw link leads to something, just in case, using the given link if not.

Otherwise, sounds like a good idea.

I assume that every single Tumblr direct link has a raw equivalent. If that's not true we'll know within hours since anybody who tries to create a bot to do this will find images with no raw links.


Lance_Armstrong said:
I assume that every single Tumblr direct link has a raw equivalent. If that's not true we'll know within hours since anybody who tries to create a bot to do this will find images with no raw links.

Oh, right, apparently the raw files for posts uploaded before December 13th, 2012 are inaccessible. (forum #235288)


JAKXXX3 said:
Oh, right, apparently the raw files for posts uploaded before December 13th, 2012 are inaccessible. (forum #235288)

That takes the feature request from "easy" to "harder".

e621 could still query the raw and then switch back to the 1280 link if the raw isn't found.

Genjar
Former Staff
5 months ago
2011 annoyed antennae arthropod biped black_markings blue_eyes clear_membrane clothed clothing crossed_arms cute duo feral front_view green_body human insect insect_wings lifting lol_comments male mammal markings moth nisimawari pellucid_hawk_moth portrait quadruped shirt shorts simple_background solo_focus spread_wings standing three-quarter_portrait three-quarter_view traditional_media_(artwork) watercolor_(artwork) white_background wings

Rating: Safe
Score: 253
User: Genjar
Date: May 29, 2013

JAKXXX3 said:
Oh, right, apparently the raw files for posts uploaded before December 13th, 2012 are inaccessible. (forum #235288)

I've got the error for newer posts too. Seems to have something to do with the size. Haven't had much time to test it.


Strong +1, checks can be done to see whether the raw version exists or not. This will feature will save people's time reuploading images and also the server space used by the old deleted images.


Heads up:

Raw file appears washed out in comparison to the original.

Raw:

Original: (source)

Can somebody with more technical know-how explain this before we start mass replacing images?


Knotty_Curls said:
Heads up:

Raw file appears washed out in comparison to the original.

Raw:

Original: (source)

Can somebody with more technical know-how explain this before we start mass replacing images?

People already started mass-replacing images...

Y'see, I knew that something like this was bound to happen, so I hesitated before "claiming my stake" in this mess.


Knotty_Curls said:
Heads up:

Raw file appears washed out in comparison to the original.

Raw:

Original: (source)

Can somebody with more technical know-how explain this before we start mass replacing images?

I haven't fully looked into it yet, so don't take this as the correct answer, but I'd imagine that the original one really does look more "washed out"

It's a common practice in photography to resize larger images to be smaller to give the illusion that the photos are sharper and higher quality, where they may appear quite blurry and lacking in detail at 25 megapixels. It looks like something similar happened here, albeit unintentionally.

When I get time, I will upload some images to Tumblr and compare the raw download to my original version and see if anything is being changed. If anybody else with the knowledge wants to do this before me, go ahead. A faster answer will be much better.

forum #235352

leomole
Privileged
5 months ago
2014 anthro bottle bottomless cat chair clothed clothing computer desk feline fur girly headphones hoodie inside jailbird leaning leaning_back legwear liam looking_at_viewer looking_back male mammal reclining sitting smile socks solo tan_fur

Rating: Safe
Score: 117
User: Onii-chan
Date: May 09, 2014

Knotty_Curls said:
Raw file appears washed out in comparison to the original.

I compared most of my updates manually and didn't notice this but it's very obvious in your example. Good catch.

Why can't things ever be simple?

Strongbird
Contributor
5 months ago

Siral_Exan said:
People already started mass-replacing images...

Y'see, I knew that something like this was bound to happen, so I hesitated before "claiming my stake" in this mess.

It's just a simple matter of checking images side-by-side before uploading, then. From what I've so far, none of the stuff I've uploaded to replace older posts looks 'washed out' like that. Johannes gave a pretty plausible explanation why it could just be affecting that particular image and ones that use a similar technique.

Genjar
Former Staff
5 months ago
2011 annoyed antennae arthropod biped black_markings blue_eyes clear_membrane clothed clothing crossed_arms cute duo feral front_view green_body human insect insect_wings lifting lol_comments male mammal markings moth nisimawari pellucid_hawk_moth portrait quadruped shirt shorts simple_background solo_focus spread_wings standing three-quarter_portrait three-quarter_view traditional_media_(artwork) watercolor_(artwork) white_background wings

Rating: Safe
Score: 253
User: Genjar
Date: May 29, 2013

Knotty_Curls said:
Raw file appears washed out in comparison to the original.

Must be some quirk in how large images are scaled down. Because it doesn't seem washed out if downloaded.


Knotty_Curls said:
Heads up:

Raw file appears washed out in comparison to the original.

Raw:

Original: (source)

Can somebody with more technical know-how explain this before we start mass replacing images?

The resized version had the gamma and color profile removed from it by Tumblr. The original has a gamma profile and color profile and the browser is using that to try and correct the colors to how they were intended to be displayed.

This is really common when transforming PNG files. Even e6 doesn't get it right. It's why you see some really bright/dark thumbnails for a PNG file every so often.


Strongbird said:
It's just a simple matter of checking images side-by-side before uploading, then. From what I've so far, none of the stuff I've uploaded to replace older posts looks 'washed out' like that. Johannes gave a pretty plausible explanation why it could just be affecting that particular image and ones that use a similar technique.

Users can do that in present tense, but in past tense there are already troves of gems (posts) cherry picked and reuploaded, some replaced by a janitor who did the cherry picking.

Now it is shown that this may not have been the best thing, and I'm sitting here thinking "no shit" while pondering a way to aid in fixing this mess as a user, if needed.

In hindsight, it may not have been the best idea to just announce this to the "world", showing this to a staff member for approval would have been better IMO.


Just a reminder, tumblr isn't only site with this stuff and I did already request this feature couple months ago.

forum #227631


I'm not seeing any washed-out effect between the two images birdmod linked.
When zoomed in all the way the jpg compression is just slightly more noticeable in the _1280 version.
Browser: palemoon 27.3 win7x64
Edit: it's annoying when mods promptly delete the images being used as examples.

Additional important things:
A bunch people are going to have their upload limits massacred if replacement gets scripted.
The order:score modifier is going to get eff'd up pretty bad as well when they're all reset to zero.
Less important, but we should really be preserving comments from one image to the other. The only reason not to do this is e.g, when someone points out a mistakes and the artist fixes it and posts a new version.


Ijerk said:
I'm not seeing any washed-out effect between the two image birdmod linked.
When zoomed in all the way the jpg compression is just slightly more noticeable in the _1280 version.
Browser: palemoon 27.3 win7x64
Edit: it's annoying when mods promptly delete the images being used as examples.

Additional important things:
A bunch people are going to have their upload limits massacred if replacement get scripted.
The order:score modifier is going to get eff'd up pretty bad as well when they're all reset to zero.
Less important, but we should really be preserving comments from one image to the other. The only reason not to do this is e.g, when someone points out a mistakes and the artist fixes it and posts a new version.

One thing to say about that: these new uploads were replacements for old posts. If they are reverted, the old posts get undeleted, and their (the old post specifically) score is kept along with any favorites the new posts had.


Siral_Exan said:
One thing to say about that: these new uploads were replacements for old posts. If they are reverted, the old posts get undeleted, and their (the old post specifically) score is kept along with any favorites the new posts had.

It might be a good idea to exclude these from the upload count calculation. We had the same issue before the not_furry grandfather clause was made, but because that isn't applicable here, the same damage is going to be done, and the option for the upload formula is still relevant.

Genjar
Former Staff
5 months ago
2011 annoyed antennae arthropod biped black_markings blue_eyes clear_membrane clothed clothing crossed_arms cute duo feral front_view green_body human insect insect_wings lifting lol_comments male mammal markings moth nisimawari pellucid_hawk_moth portrait quadruped shirt shorts simple_background solo_focus spread_wings standing three-quarter_portrait three-quarter_view traditional_media_(artwork) watercolor_(artwork) white_background wings

Rating: Safe
Score: 253
User: Genjar
Date: May 29, 2013

Ijerk said:
Additional important things:
A bunch people are going to have their upload limits massacred if replacement gets scripted.

Maybe so. There are a lot of pony posts from couple of users; replacing those would considerably hurt their upload limit. But since this is a special case, I'm sure they could contact the administration and ask for it to be reset.


Genjar said:
Maybe so. There are a lot of pony posts from couple of users; replacing those would considerably hurt their upload limit. But since this is a special case, I'm sure they could contact the administration and ask for it to be reset.

That's what happened to me (and likely others) when BBMBBF requested DNP status. My limit went down to 2 posts until an admin brought it up to 12.


KiraNoot said:
The resized version had the gamma and color profile removed from it by Tumblr. The original has a gamma profile and color profile and the browser is using that to try and correct the colors to how they were intended to be displayed.

This is really common when transforming PNG files. Even e6 doesn't get it right. It's why you see some really bright/dark thumbnails for a PNG file every so often.

If gamma/color profile is stripped away, does that mean there is a case to be made for uploading even the raw versions that have identical dimensions to the 1280 versions?

Actually, Mario says 1:1 MD5 match for the smaller raw images.

Strikerman said:
That's what happened to me (and likely others) when BBMBBF requested DNP status. My limit went down to 2 posts until an admin brought it up to 12.

I expect there will be a Great Tumblr Amnesty.

Genjar
Former Staff
5 months ago
2011 annoyed antennae arthropod biped black_markings blue_eyes clear_membrane clothed clothing crossed_arms cute duo feral front_view green_body human insect insect_wings lifting lol_comments male mammal markings moth nisimawari pellucid_hawk_moth portrait quadruped shirt shorts simple_background solo_focus spread_wings standing three-quarter_portrait three-quarter_view traditional_media_(artwork) watercolor_(artwork) white_background wings

Rating: Safe
Score: 253
User: Genjar
Date: May 29, 2013

Can we even be sure that this'll last? Nobody seems to have known about this trick until recently. Maybe Tumblr never intended this to be used by the public, and will patch it to be impossible to use soon..

In any case, here's a related suggestion:
How about automatically replacing the outdated broken subdomains in the sources? Such as the old 41.media.tumblr.com into 68.media.tumblr.com?
Edit: Never mind. That was before Tumblr changed the system.


Was there a conclusive answer to this?

Knotty_Curls said:
Heads up:

Raw file appears washed out in comparison to the original.

Raw:

Original: (source)

Can somebody with more technical know-how explain this before we start mass replacing images?

KiraNoot said:
The resized version had the gamma and color profile removed from it by Tumblr. The original has a gamma profile and color profile and the browser is using that to try and correct the colors to how they were intended to be displayed.

This is really common when transforming PNG files. Even e6 doesn't get it right. It's why you see some really bright/dark thumbnails for a PNG file every so often.


Cormy1 said:
Was there a conclusive answer to this?

I think we were waiting for the situation to stabilize, and for more information about this issue. Having to use media.tumblr.com urls instead of **.media.tumblr.com urls is one such development that has complicated matters.

A stopgap solution is to use this script which automatically redirects your tumblr image urls to the raw version.


I'm pretty much sitting on this issue because it has changed a lot. Once the pattern is stable for a month or so it should be easier to deal with. If I implement a pattern and then it stops working, direct URL uploads are broken for that domain, and nobody wins. Making emergency deployments for issues like that is very uhhhg.


KiraNoot said:
I'm pretty much sitting on this issue because it has changed a lot. Once the pattern is stable for a month or so it should be easier to deal with. If I implement a pattern and then it stops working, direct URL uploads are broken for that domain, and nobody wins. Making emergency deployments for issues like that is very uhhhg.

Have you given a though about this for other sites? As e.g. inkbunny has been unchanged for decades and with twitter this would prevent unwanted uploads, including JPG files being resaved as PNG by twitter itself.


Mario69 said:
e.g. inkbunny has been unchanged for decades

My sources tell me it was launched in 2010 ;)