Posted under e621 Tools and Applications
Hi! Wanted to share that I made a script that analyzes a user's public favorites data to generate an overview of their most liked tags, in the style of something like Spotify Wrapped. Here's an example of what that looks like:
The project's source code, alongside instructions of how to generate one of your own, can be found in the link below :3
Updated
I do have a few comments:
On a side note, you also might be able to request a full dump of the favorites db table? I know previous developers were willing to do it, I'm not sure about the current developer
donovan_dmc said:
Pretty neat toolI do have a few comments:
- Your requirements.txt seems to be missing some dependencies, you should try running it in a virtualenv or somewhere with nothing preinstalled beyond python, I had to install requests and matplotlib before it would run
- This could likely be made to work without an api key for users who have their favorites public and may have concerns about putting their api key into a random program
- Your instructions on the repo are likely too technical for the average user, I'd have trouble believing they know what clone means on github or how to clone a repository, downloading a zip of the repository would probably be the best for them
On a side note, you also might be able to request a full dump of the favorites db table? I know previous developers were willing to do it, I'm not sure about the current developer
Thanks for trying it out! I updated the repository based on your feedback
About the full dump of the favorites... that would be really cool for generating a more detailed and less biased average user profile! Do you know who I should contact about requesting this? (and do you know what kind of file size we're talking about?)
Updated
mormonara said:
About the full dump of the favorites... that would be really cool for generating a more detailed and less biased average user profile! Do you know who I should contact about requesting this? (and do you know what kind of file size we're talking about?)
The actual sysadmin that would do the dump would be Dari, but I'm not sure if he would check his dmails or even respond
Cinder is our developer, but he probably has more pressing matters to deal with
I'm not sure how good your chances are but you can still try
The favorites table is nearly 1.1 billion rows, with a presumed csv header of post_id,user_id and each column being 1-5 digits I'd put it at anywhere between 8 and 12 gigabytes
I like countershade_fur 8.7x more than the average user. Truly interesting.
Jokes aside, this is neat. I don't use my favorites so I had to make a quick change to look at my upvotes instead, which was easy as just using the posts.json endpoint instead.
https://i.imgur.com/re7vCay.png
my constant favoriting of my own sona has been laid bare
This is really cool! Here's mine: https://imgur.com/a/KfaqRzz
Unfortunately, mine isn't the most accurate since I've had to unfav 160k posts and move them to sets to make room for new favs (it's weird, don't question it, I have my system in my mind xD fav+upvote, move 10k to a new set when nearly out of favs, mass unfav those in the set) but regardless it is cool to see the output for what's currently in my favs!
However, I don't think the "% of your favs have this tag" is accurate. Some examples, the program says 4.4% of my favs have lay_the_dragon, I searched for lay_the_dragon fav:jerdude002 and got 165 results, and with 66682 favs, that comes out to 0.247%. non-mammal_breasts is 11.8% according to the program, search on my favs finds 2326 with that tag, percentage actually comes out to 3.488%, and prehistoric_species is 12.3% according to the program, search finds 1372 favs with that tag, actual percentage comes out to 2.057%.
On a funny note, the "your likes in one post" for me actually falls in my blacklist xP but I did already have it fav'd anyway
jerdude002 said:
However, I don't think the "% of your favs have this tag" is accurate. Some examples, the program says 4.4% of my favs have lay_the_dragon, I searched for lay_the_dragon fav:jerdude002 and got 165 results, and with 66682 favs, that comes out to 0.247%. non-mammal_breasts is 11.8% according to the program, search on my favs finds 2326 with that tag, percentage actually comes out to 3.488%, and prehistoric_species is 12.3% according to the program, search finds 1372 favs with that tag, actual percentage comes out to 2.057%.
It only fetches the most recent 3200 favorites (10 pages) by default, you can increase tbe number of pages but it will take longer (1 page per 320 favorites)
donovan_dmc said:
It only fetches the most recent 3200 favorites (10 pages) by default, you can increase tbe number of pages but it will take longer (1 page per 320 favorites)
Ahh, thanks! I figured there was some limit (at least by default) xP I adjusted it, this is less surprising of a result: https://imgur.com/a/aQKE9V4
I went and redid this with all of my favorites and upvotes
Unsurprising to me they aren't that different since I upvote all of my favorites
Also another note to mormonara, your shell script is not executable
I believe you need to use git update-index --chmod=+x generate.sh to have git actually keep the executable bit
This is really nicely done. Results for my account, apparently I really like prick_ears and whisker_spots.
Also, generate.sh looks for python3, but recommend.sh looks for python and so didn't work out of the box for me at least.
mahrai_ziller said:
This is really nicely done. Results for my account, apparently I really like prick_ears and whisker_spots.Also, generate.sh looks for python3, but recommend.sh looks for python and so didn't work out of the box for me at least.
Oh, thanks! Just updated the repository to make this consistent. How did you like the recommendation system? Haven't really gotten any feedback on it yet
donovan_dmc said:
I went and redid this with all of my favorites and upvotes
Unsurprising to me they aren't that different since I upvote all of my favoritesAlso another note to mormonara, your shell script is not executable
I believe you need to use git update-index --chmod=+x generate.sh to have git actually keep the executable bit
I thought the user always had to do that themselves before running... I had no idea....
"Could not find pip in the virtual environment. Something went wrong"
Python 3.10.0 installed
