A while ago I posted about an 8ch monitoring tool I made. I dunno if anyone is using it but I just rolled out a new version: git.teknik.io
Packaging is a bit better now, hopefully should solve some issues people were having last time. I had to push some changes back but on the hand I added some basic support for Tor.
Other urls found in this thread:
How basic support are we talking about here?
Care to share more about your vision of this project, user?
When you ask it to open threads in your browser, if your browser uses Tor, the normal 8ch.net link won't work (you can't post, and cloudflare blocks you sometime). So instead you can ask for the onion version of the same thread. It will also open the captcha page to streamline replying.
Refreshing threads through lizard is still done using the 8ch link. You can do "torify lizard r" in which case it should start loading the 8ch URL through Tor. Problem is if cloudflare realizes you're on Tor, lizard will see the CF page and shit itself. I'm planning a "refresh with onion link" feature for the next version.
Anything specific, or just generally?
It's a convenience tool to mitigate the slowness of 8ch. A lot of the threads I like, will take days or weeks to get a reply. But often that reply will be worth it, except that I'll forget to check the thread and by the time I see that reply it has been a while and now the original guy has forgotten about the thread. But with this I get notified pretty quick about new activity, so it has helped me keep interesting conversations going on very slow boards.
It's nice to imagine that one day this will be good enough to put in the sticky or something and recommend to all 8ch users, and everyone will use it and it will help all the small, niche but really cool boards we have be more successful. I doubt that will ever happen, but it was a fun learning experience writing this and I figure maybe some user would find it useful as well.
I'd like to add the ability to watch individual posts (maybe you asked something in the tech support thread and only really care about replies to you) and whole boards (so you are promptly notified about new threads). Should be easy enough but it would require substantially reorganizing the database model so I'll need to wait until I have a whole free day or two to code it.
If you want this to appeal to the majority of the site to improve slow boards, you might want to make/ask someone else to make a GUI that regular anons can work with easily.
Also, Winblows support would mean more users can be involved, OS X support would be nice but not terribly needed.
Haven't looked over the code, but I assume that would need an entire rewrite, so nevar evar.
Good work though, user. I've been looking for a tool like this for a while. I'll give it a whirl later today.
I actually had made a GUI client before, it's called Lizard also and it's somewhere on bitbucket.
Ended sucking up far too much time to deal with irrelevant GUI crap, wasn't really that convenient, and nobody used it to my knowledge. When I posted on tech to get feedback all I got was people complaining about >winblows and no feedback at all.
I see your point, but I'm afraid windows support will at least wait until the CLI is very polished and ready for plug and play use. If I have people clamoring for Win support I'd consider doing it sooner.
Doesn't pip run on that? It should work in that case.
In fact I bet it would work on Winblows as well, but you'd have to figure out how to get the path to work right and somehow deal with the godawful Windows CLI.
Not quite. I could just make a barebones wrapper around the CLI, that would be hardly any work at all.
Probably a Py GUI would be easy also, just add a form with a table and replace print() calls with GUI updates.
That's good to hear, thanks! Let me know if you have problems with it, it works on my OS but I haven't tested on different systems much. You can find me here, on the issue tracker, or email [email protected]/* */ Save the stack trace if you get any errors.
Keep up the good work, user.
Are you getting an error from pip install?
It's Python 3 code, so yes you can't run it with Py2. On my system pip defaults to pip3, but pip3 install should work too. Might require Python 3.6 but shouldn't.
You may want to mention more loudly in your readme that 3+ is required, for the more miopic amongst us. I see it is mentioned in your readme file, but it's easily missed.
Ah, thanks for pointing that out. I thought it was already clear but now I realize I forgot to review the readme after a big reorganization.
My pleasure. If you have some free time, please stop by walkran.com:7777. I would like to talk to you about integrating lizard into our world.
I think lizard would be right at home here.
fucking kekked
Is it OK to study this software and write GPL-ed software with same features (in another programming language) or would that be violation of license?
If you're talking about recreating the features then publishing your cloned software under the GPL, then that's okay.
Reminder that only because original license is MIT (very permissive)
But in the real world there are two licenses: free for everyone to copy and lie about it when asked (also called "open source"), and strictly forbidden to plagiarize (also known as "closed source").
How does this tool work for posts that are deleted by mods after lizard saves the thread?
Depending on how frequently you do the refresh, you will retain the local copy and be able to read wrongbad posts the mods don't want you to see.
Every time you call refresh on a thread, a local copy of its JSON (which is then parsed to check new replies etc.) and HTML (so that you can easily view in browser) is saved. This is never deleted.
For instance I sometimes put threads on it to read later. A few days later, the whole thread has gone 404, but since I have been calling the refresh periodically I can just go into the lizard_data dir and open the latest HTML. I can also look at earlier copies in case some replies were deleted.
To be honest I find that mod-deleted posts are not very interesting on 8ch. I care more about seeing a long thread on a board like pol, that I know I won't have time to read for a few days, at which point it will have been long purged. So I haven't really developed the anti mod deletion feature. The data is there and you can manually look at the HTMLs and easily spot them, especially if you schedule lizard r to run regularly (say every 5 minutes instead of every few days like I do) you will only miss a deleted post if the mod deleted it less than 5 min after it being posted.
If you are interested in using this as an un-delete tool, I'd be happy to develop those features if you help me with designing them (since you have the use case and I don't).
I like the idea, but I have a few suggestions:
First, some imo necessary additions:
1) List the thread title
2) Add a command to open a selected thread or group of threads
3) Have the add command accept multiple urls
Next, some things that might be nice, but might fall out of the scope of this program/be unreasonable to implement:
1) A command to output new replies directly in the console instead of opening a web browser
2) A search/filter function for listing threads
3) An interactive mode that continues running after being called, automatically updates, and accepts commands internally
4) Settings for downloading only text, text+thumbnails, etc...
5) Ways to alter the output format, i.e changing the order of "thread url"/"last seen"/etc, the maximum length, selective display, etc...
6) Some sort of config file
7) Track image replies vs text-only replies
Your (You) detection also doesn't work for me. I'm not sure how you implemented it, or if it can even be fixed, but so far it's reacted to several random people who got replied to.
Oh, also the "lizard create" command doesn't work, it's actually "lizard c". The error message is correct, but the default command list is wrong.
Thanks user, the feedback helps a lot!
Would love to, but where? Right now it's just narrow enough to fit on one half of the screen.
My shell lets me click the URL in the lizard l output. And there's lizard oa if you don't want to click all of them. oa does update the "last seen" status.
You still think it's necessary to have more open commands?
I'll think about this. Hopefully I can come up with a good syntax.
Yeah, I've been thinking of this also. But that would be the first step into full-fledged client territory rather than monitoring tool, which is why I've been hesitating.
What criteria do you think make sense? (I currently copy and paste it into a text editor/spreadsheet and filter that way)
I've deliberately made it stateless so that scripting is idiot proof, but I also feel like it would be a convenient feature. The concern is that it will make development confusing (since now every function can be done in two ways). Maybe I can hold off a bit until all the major features are complete and then start working on the interactive.
This was a wontfix for me, since it seemed more logical to just set a cronjob to call lizard r periodically. Thoughts on this?
What's your reasoning for this? I feel like text mode would be very infrequently used and "I changed my mind I want the images too now" would be a common issue.
I think this goes into the search/filter, but how about I just dump a csv and your own software can deal with the filtering?
Actually a "list only threads with activity" feature has been priority for me recently.
This is gonna involve a lot of learning for me, since I have no clue about such things, but definitely mandatory for v1.0.
My two confusions:
1) What syntax? Just dumping a flat JSON dictionary is easy but is that
2) What options do you think need to be configurable?
Definitely will implement this soon. Although you can kind of tell from lizard r output.
Right now it works by looking at the anchored reply associated with the thread, and tries to scan for them. So it's not really "You" detection, more like "replies to this post" detection.
It's pretty buggy, which is why I'm considering tracking replies, threads and board separately. If you can reproduce it on /test/ that would be really helpful.
Sounds like a bug, I'll look into it. Used to be c, but I changed to create since it's such a rare command.
Replace the url with just board/number, or maybe ditch the number too and just number the threads 01, 02, etc
i.e. instead of:
>8ch.net
>01 Holla Forums lizard, the 8ch monitor v0.3
There's also, at least for me, a lot of space between the url/last seen/etc... that could be thinned down. You don't need to fit the entire title, just as many characters as possible. Even a few words is better than a pile of random numbers.
That does work, but not if you replace the url with something shorter to fit in a thread title. lizard o seems pretty simple.
I was thinking commandline switches + file of defaults. Set things like the command to call when you open a thread in browser i.e.: browser=firefox, browser=chromium, an option for absolute instead of relative times, widths of the output fields so you can adjust the overall size of the output(I have a really huge screen so I could fit a lot more information, and some people have really small screens and might want to condense things or leave certain output fields out entirely).
It would work better as another program. But to make it work, you'd need a setting or a command to output verbose lines containing a lot more data to be piped to grep/etc. Like a command that dumps the whole OP text, url, number of replies/images, date started, etc... so I can filter/sort the results with a pipe.
I have all the threads pinned with the thread url, and every post is a detected as a (You). This is the only thread where I actually got a reply.
As an aside, this program is officially the best image scraper for Holla Forums I know of.
It's somewhat annoying that every time I post I have to update and reopen the thread to get rid of the reply marker. A way to mark a thread as read without opening it or some other way of dealing with this situation would be appreciated.
Also, I officially have no idea what the (you) detection is doing anymore. It seems totally random whether a thread gets a Y or a !.
seems like that day is today on the entire site