API Upload error

In category: e621 Tools and Applications

Edit:
On error 423 (duplicate), there is no id of post that it is duplicate of as far as I can see.
Any way to get id of original?


I see a few problems here. First, you should fill in your user agent details so they are not the placeholder values.

The second problem is to ensure that you have an API key enabled, and use that for password hash. (You seem to have fixed this given the edit to your post)

The third problem is that you're submitting content as JSON, but the server wants multipart/form-data content, but it will RETURN JSON data to you.

The fourth problem is that you are submitting a string for the file parameter when you only want a source url to upload from. Do not submit a file parameter if you are not using it. You will receive 500 errors if you submit a string to the file field.

The next part is more complicated and I don't have enough experience to offer you good advice. .NET seems to have too many different HTTP request layers and they all seem to be pretty terrible and complicated to use, or come with a lot of caveats. Perhaps using something like RestSharp would make it easier, although the documentation for RestSharp seems quite lacking.

StackOverflow seems to be filled with what I would consider bad examples of how to do this, especially because they encourage you to hand build and submit multipart/form-data content type data, which is bad practice and very fragile.

If anyone has any more .NET C# or VB.NET experience it would be nice to hear you chime in on best practices for this.


First two are fine.

Hmm, are there any working examples? I could probably make something if I could see it.

-----------------------------3137622934070
Content-Disposition: form-data; name="authenticity_token"
Don't post your authenticity_token --Chaser
-----------------------------3137622934070
Content-Disposition: form-data; name="post[file]"; filename=""
Content-Type: application/octet-stream
-----------------------------3137622934070
Content-Disposition: form-data; name="post[upload_url]"
https://d.facdn.net/art/dreamandnightmare/1518368910/1518368910.dreamandnightmare_wet_cat.jpg
-----------------------------3137622934070
Content-Disposition: form-data; name="post[tags]"
anthro anus balls barefoot bathroom bathtub biceps butt close-up dream_and_nightmare feline fur green_eyes humanoid_penis inside looking_at_viewer looking_back male mammal manly muscular nude penis presenting presenting_anus presenting_hindquarters sharp_teeth smile solo spread_legs spreading teeth thick_penis thick_thighs tiger triceps water wet wet_fur whiskers 
-----------------------------3137622934070
Content-Disposition: form-data; name="post[source]"
-----------------------------3137622934070
Content-Disposition: form-data; name="post[description]"
-----------------------------3137622934070
Content-Disposition: form-data; name="post[parent_id]"
-----------------------------3137622934070
Content-Disposition: form-data; name="post[rating]"
explicit
-----------------------------3137622934070--

When inspecting request with Mozilla, it looks like that.


The fourth problem is that you are submitting a string for the file parameter when you only want a source url to upload from.

If I understand KiraNoot correctly, this:

Content-Disposition: form-data; name="post[file]"; filename=""
Content-Type: application/octet-stream

should not be present in any form; you shouldn't be sending it.
Which is different from sending a key with an empty value.

For the sake of cutting the size of your code down, I'll also quote the API docs:

There are only four mandatory fields: you need to supply the tags, and you need to supply the file, either through a multipart form or through a source URL. A source, even if blank, and a rating are also required.

So you don't need the empty description or parent_id either.

Some of the other stuff -- like tags -- looks correct, so have you partially overcome the sending-JSON problem?


I'm working on it, the post from above is what happens when you upload normally over website.


m8, don't post your authenticity token, I've edited it out but next time you might not be so lucky that a admin sees it first. :v


No luck so far either.

Removed

savageorange said:
If I understand KiraNoot correctly, this:
> file stuff
should not be present in any form; you shouldn't be sending it.
Which is different from sending a key with an empty value.

It's okay to send file with a blank body as long as there is no filename and their contents is 0 bytes long, because the internal script filters it back out and treats it as not properly set. However, it's easier to simply omit the field if it isn't being used. The workaround for empty files is for browsers, but it works even for scripts.

DelurC, you're kind of following the bad advice on StackOverflow and building your own multipart data formatter, which is error prone and not at all easy to debug. I, again, am not a .NET expert, but there has to be a better way of doing it than by hand.

If you're still getting 403, it's because you are no longer submitting your login and api key with the request. That works for a browser because it has a login session, but in a script you don't have one, so they must be present in the body of the request like the other fields.


KiraNoot said:
DelurC, you're kind of following the bad advice on StackOverflow and building your own multipart data formatter, which is error prone and not at all easy to debug.

you're talking to a dead man

This user was banned by Ratte
Reason:

Compromised.

Expires: Never


Wow. Ok, yeah, seems fair enough. Difficult to positively demonstrate that no-one viewed the thread in the critical time frame (when the auth token was unedited); At minimum one person (me) did see the token. I guess they're expected to contact an admin to reset their password and reenable their account?


Yeah, if he messages us and fixes his stuff we'll unban him.


KiraNoot said:
If you're still getting 403, it's because you are no longer submitting your login and api key with the request. That works for a browser because it has a login session, but in a script you don't have one, so they must be present in the body of the request like the other fields.

I'm getting a 500.

        Removed

There are some differences but nothing wildly different, other than that time

3137622934070
vs
636540617142575159


I'm getting 403 now, progress!!!

Removed

I think authentication is the problem, how do I authenticate?

The help page just says that it may be required, it doesn't say where it wants it.


What you are doing looks roughly right (I've successfully authorized before, it basically is just the login and password_hash) , but the fact that you are constructing it manually may be neglecting some detail of protocol.

Honestly in your position I would probably pick up CURL, and use that to do an upload (eg curl https://e621.net/post/create.json -d "post[upload_uri]=https://d.facdn.net/art/dreamandnightmare/1518368910/1518368910.dreamandnightmare_wet_cat.jpg" -d "post[tags]=anthro anus balls barefoot bathroom bathtub biceps butt close-up dream_and_nightmare feline fur green_eyes humanoid_penis inside looking_at_viewer looking_back male mammal manly muscular nude penis presenting presenting_anus presenting_hindquarters sharp_teeth smile solo spread_legs spreading teeth thick_penis thick_thighs tiger triceps water wet wet_fur whiskers" -d "post[source]=" -d "post[rating]=explicit" -d "login=myusername" -d "password_hash=myhash" -o response.json) . Then I could analyze the differences between the request CURL makes and the request the program makes.

(note: I am not sure off the top of my head whether sending login and pwhash via '-d' will work with https; -u myusername:mypassword might be required instead. If this is the case, it might be that your program is falling over for a similar reason.)

But that's getting into a different can of worms (commandline usage) which you might not want to.

BTW: while I was writing the above, another problem you may potentially be encountering occurred to me: you may need to URL-encode your upload_url.

Jazz
Privileged
3 months ago
anthro canine fantastic_mr_fox fox kristofferson male mammal marlfox solo unknown_artist

Rating: Safe
Score: 28
User: Lxs371
Date: January 18, 2010

Huh? :V

No idea what the problem could be, don't really have any .NET experience. I've just dug up some old code and uploaded a couple of pics via the API, supplying only these fields:

  • post[upload_url]
  • post[tags]
  • post[rating]
  • login
  • password_hash

and a user agent, everything worked fine. Double-check your login and password_hash fields, maybe after you've regenerated your API key you didn't update it in your program?

Also, since you're forming the request by hands, you might want to check if the request is not malformed too. Maybe post it to httpbin, and dump the output here:

  • Temporary change your API key to some word
  • Change the endpoint to http://www.httpbin.org/post
  • Run the program, dump the server response as plain text, edit out your IP address in it, and post it here.

Maybe you're missing Content Length or Host header, but I'm not sure if those are mandatory and/or if Net.HttpWebRequest fills them out for you.

My output from httpbin looked something like this:

{
  "args": {}, 
  "data": "", 
  "files": {}, 
  "form": {
    "login": "Jazz", 
    "password_hash": "secret", 
    "post[rating]": "safe", 
    "post[tags]": "j_axer cat solo clothed ", 
    "post[upload_url]": "https://d.facdn.net/art/jayaxer/1495847859/1495847859.jayaxer_skie_final.jpg"
  }, 
  "headers": {
    "Accept": "*/*", 
    "Connection": "close", 
    "Content-Length": "631", 
    "Content-Type": "multipart/form-data; boundary=------------------------0b636ef4766f545e", 
    "Host": "www.httpbin.org", 
    "User-Agent": "curl-7.49.0/cpr (contact Jazz on e621 if I fucked up; https://e621.net/user/show/1555)"
  }, 
  "json": null, 
  "origin": "----", 
  "url": "http://www.httpbin.org/post"
}

Alright thanks, I checked the data I was sending and there was a missing newline.
Now it works.

It was probably working most of the time.All those hours because of that one character...

Here is working code if anyone needs>

   Dim myHttpWebRequest As Net.HttpWebRequest = DirectCast(Net.WebRequest.Create("https://e621.net/post/create.json"), Net.HttpWebRequest)
        myHttpWebRequest.UserAgent = "Test Project/1.0 (by ???)"
        myHttpWebRequest.Method = "POST"
        Dim Boundary As String = "---------------------------" & DateTime.Now.Ticks.ToString
        myHttpWebRequest.ContentType = "multipart/form-data; boundary=" & Boundary
        Dim Upload_URL As String = ""
        Dim Tags As String = ""
        Dim Sources As String = ""
        Dim Description As String = ""
        Dim Rating As String = "e"
        Dim DataArray(1, 6) As String
        DataArray(0, 0) = "post[upload_url]"
        DataArray(1, 0) = Upload_URL
        DataArray(0, 1) = "post[tags]"
        DataArray(1, 1) = Tags
        DataArray(0, 2) = "post[source]"
        DataArray(1, 2) = Sources
        DataArray(0, 3) = "post[description]"
        DataArray(1, 3) = Description
        DataArray(0, 4) = "post[rating]"
        DataArray(1, 4) = Rating
        DataArray(0, 5) = "login"
        DataArray(1, 5) = UserName
        DataArray(0, 6) = "password_hash"
        DataArray(1, 6) = Hash
        Dim TempStream As IO.MemoryStream = New IO.MemoryStream
        For y = 0 To DataArray.GetLength(1) - 1
            Dim PostPart As Byte()
            If Not DataArray(1, y) = "" Then
                PostPart = Encoding.UTF8.GetBytes("--" & Boundary & vbCrLf& & "Content-Disposition: form-data; name=""" & DataArray(0, y) & """" & vbCrLf& & vbCrLf & DataArray(1, y))
                TempStream.Write(PostPart, 0, PostPart.Length)
                If y = DataArray.GetLength(1) - 1 Then
                    PostPart = Encoding.UTF8.GetBytes(vbCrLf& & "--" & Boundary & "--" & vbCrLf&)
                    TempStream.Write(PostPart, 0, PostPart.Length)
                Else
                    PostPart = Encoding.UTF8.GetBytes(vbCrLf&)
                    TempStream.Write(PostPart, 0, PostPart.Length)
                End If
            End If
        Next
        TempStream.Position = 0
        TempStream.CopyTo(myHttpWebRequest.GetRequestStream())
        TempStream.Close()
        Dim myHttpWebResponse As Net.HttpWebResponse
        Try
            myHttpWebResponse = myHttpWebRequest.GetResponse()
            Dim Reader = New IO.StreamReader(myHttpWebResponse.GetResponseStream())
            Dim Response As String = Reader.ReadToEnd()
            myHttpWebResponse.Close()
        Catch ex As Exception
            'whatever
        End Try