its been like that for like 10 minutes and counting. is it happening to anyone else too? or maybe only me...
When I try to add a post to a set it tells me "database query failed"

edit: also getting it to happen more often then ever lately

sjdhtbfiishr said:
its been like that for like 10 minutes and counting. is it happening to anyone else too? or maybe only me...
When I try to add a post to a set it tells me "database query failed"

edit: also getting it to happen more often then ever lately

topic #25734

Had the same just now. 2 post gave error: but no other info, another worked fine going into a different set.
Not sure if it was posts under niki3d or my set.

Post sets get increasingly unstable as the number of posts in them increases.
The current system has to save the full list of IDs to the database every time you add or remove a post.
If there are a lot of posts... well, the request may time out.

I had been aware of the issue for a while, and I am looking at potential solutions.
Nothing concrete yet, though.

cinder said:
Post sets get increasingly unstable as the number of posts in them increases.
The current system has to save the full list of IDs to the database every time you add or remove a post.
If there are a lot of posts... well, the request may time out.

I had been aware of the issue for a while, and I am looking at potential solutions.
Nothing concrete yet, though.

Are pools handled the same way?

All i get is "error: " nothing included :/

nin10dope said:
Are pools handled the same way?

Yes, but it's less of an issue, since pools are limited to only 1k posts.
With sets, issues can start to manifest as early as 2-3k, and become commonplace when nearing the limit of 10k.

cinder said:
Yes, but it's less of an issue, since pools are limited to only 1k posts.
With sets, issues can start to manifest as early as 2-3k, and become commonplace when nearing the limit of 10k.

If sets were to be made for just the individuals who make them, would implementing them to be clientside like blacklists be viable? Probably not as simply, but it could put the burden of the list of id's onto the client instead. I suppose not unlike separate Favorite lists
Just a few half baked ideas I thought up since I asked about the pools

nin10dope said:
If sets were to be made for just the individuals who make them, would implementing them to be clientside like blacklists be viable? Probably not as simply, but it could put the burden of the list of id's onto the client instead. I suppose not unlike separate Favorite lists
Just a few half baked ideas I thought up since I asked about the pools

users' blacklists are still hosted serverside.

I think another, admittedly risky, idea would be to simply not bring up/refer to the entire string of all ids in the set whenever someone clicks the +set button. Of course it sounds like that would have no prevention from multiple instances of the same post being added (among anything else that would go wrong)

dba_afish said:
users' blacklists are still hosted serverside.

They are stored server side, but no processing is done there

nin10dope said:
I think another, admittedly risky, idea would be to simply not bring up/refer to the entire string of all ids in the set whenever someone clicks the +set button. Of course it sounds like that would have no prevention from multiple instances of the same post being added (among anything else that would go wrong)

This likely would not fare any better, array columns are hell to work with
Why sets & pools use array columns rather than a plain string like posts do for favorites is a mystery to the world, they'd likely be just fine if they used a string instead, but it really is not easy to convert to that

donovan_dmc said:
They are stored server side, but no processing is done there

This likely would not fare any better, array columns are hell to work with
Why sets & pools use array columns rather than a plain string like posts do for favorites is a mystery to the world, they'd likely be just fine if they used a string instead, but it really is not easy to convert to that

I see, I was under the assumption that post ids in a set were stored as a string in the same way tags are stored for a post

cinder said:
Yes, but it's less of an issue, since pools are limited to only 1k posts.
With sets, issues can start to manifest as early as 2-3k, and become commonplace when nearing the limit of 10k.

Strange the set only had 177 posts when giving me the error.
Funny how I started using sets more because of the favorite limit only to find that sets also don't work when filled up.

pixelpile said:
Strange the set only had 177 posts when giving me the error.
Funny how I started using sets more because of the favorite limit only to find that sets also don't work when filled up.

Eh if one set gets too big you can just make another set as a part two

nin10dope said:
Eh if one set gets too big you can just make another set as a part two

If you can only manage some 200 posts per set you're going to run out of sets after ~15,000 posts

donovan_dmc said:
If you can only manage some 200 posts per set you're going to run out of sets after ~15,000 posts

This plus the 80k favs I could easily blow past if I wasn't already trying to avoid the favorite limit.

pixelpile said:
This plus the 80k favs I could easily blow past if I wasn't already trying to avoid the favorite limit.

Damn son, if you keep piling on those favs, you might just want another account to divide them in half if your organize them by tag

donovan_dmc said:
If you can only manage some 200 posts per set you're going to run out of sets after ~15,000 posts

I have way less and I also always get the Error:
Are there any other problems with sets that could give this error?

The only other related problem that I know could feasible give rise to this is that posts store a redundant list of pools & sets they're a part of (to facilitate the next/previous buttons & other things), so a post that's part of a lot of pools and/or sets could time out for the same base reason. I doubt this is the case, as again, it'll only cause problems if the post is in hundreds of sets + pools, and I doubt that's the case that frequently. More likely is that the server is under load & takes longer to process your request than normal, decreasing the chances it'll finish in time.
If the posts' updates are taking time to finish, I'd imagine separating the set & pool stores would help delay this - not because there's enough pools to interfere, but because using the same field for both means we need to add a set:/pool: prefix to differentiate between them, and that's extra data to handle - but I don't know enough about the database to be confident in that.
Regardless, the root problem is how sets are handled internally. Breaking sets down into a segmented collection of collections might stave off problems, but that's still a colossal pain & still might not work. I'd also suggest we look into using hashsets (the data structure, not the site feature) instead of arrays to potentially ease processing server-side. At the root of it, we're probably gonna have to either add stricter set limits like we did with favorites, or rework how we store sets & update the db.