Project B.L.A.Z.E. BEGIN

If you were around back in March-April, you may have encountered a small thread named "Blazechan". That project was also by me, and was written in PHP and had lots of Fredrick-tier code.

I decided to rewrite Blazechan in Django for it to supersede the currently good but performance-wise lacking Infinity Next. I hope it will eventually replace it in Nextchan.

I will release the code once it reaches a usable state. Right now, only the index and board index is ready, pics related.

Please share suggestions/thoughts.

Other urls found in this thread:

gitgud.io/m712/blazechan
gitgud.io/m712/blazechan/blob/d1c429970be1d7710fb5c8a52b4a61a75fb4c21d/blazechan/settings.py#L22
github.com/jazzband/django-configurations
docs.djangoproject.com/en/1.10/topics/signals/
docs.djangoproject.com/en/1.10/topics/signals
channels.readthedocs.io/en/latest/
github.com/jimfunk/django-postgresql-netfields
stackoverflow.com/questions/2744632/change-text-factory-in-django-sqlite/28794677#28794677
github.com/alvra/django-spotnet
github.com/majestrate/nntpchan
nsot.readthedocs.io/en/latest/_modules/nsot/fields.html
endchan.xyz
fuckoff.kys
docs.djangoproject.com/en/1.10/topics/db/managers/
docs.djangoproject.com/en/1.10/topics/serialization/
code.djangoproject.com/wiki/CookBookSplitModelsToFile
laravel.com/docs/5.1/eloquent#query-scopes
github.com/tc77/django-extended-attachments
github.com/ahupp/python-magic
blaze.nextchan.org
sirdarckcat.blogspot.com/2016/12/how-to-bypass-csp-nonces-with-dom-xss.html
docs.djangoproject.com/en/1.10/topics/migrations/#custom-deconstruct-method
docs.djangoproject.com/en/1.10/_modules/django/core/validators/#URLValidator
nextchan.org/
gitgud.io/sun/PaintGo
github.com/Soreil/mnemonics/blob/master/mnemonics_out
twitter.com/NSFWRedditGif

Good job!

Is your website going to be a botnet? I hope not.

why even bother, django will have to be cached as well, you'll have to write yet another migration tool from infinity next. seems like a waste of time.

Thanks. And I don't think so, no.


Django is a better established web framework and python is generally faster than php. I can confirm this, it reached 369/r/s/c on my machine in the board index page.

369 request per second?
How have you benchmarked it?
Have you found bottleneck?

The bottleneck is the CPU at this point, I assume. I benchmarked it by running 10000 requests via curl to the page and then dividing the resulting time by 10000. It took approx. 30 seconds to process 10k requests on a single core, I am sure nginx etc. can do even better.

We need to write a chan engine in Assembly.

how established a web framework is doesn't really matter.
if you just want to waste time rewriting yet another imageboard, go ahead.
but don't pretend that you're spending your time wisely.
if you really wanted to do this, you'd be using floens python imageboard (flask) or atsonex's django imageboard to start with.
don't pretend that what's probably a few static requests means that issues are unsolvable with the codebase you just want to ignore. you just don't have the motivation to fix them with your small amount of resources, and writing more spaghetti sounds like an appealing way to waste time.

You should never assume things about performance. Find tools that Django devs use and measure things.
Also, curl probably sent requests serially, waiting for each request to finish before sending another. You should try benchmarking with ab (apache benchmark) or wrk2 or something and send multiple requests in parallel.

Just re right it all in opencl or better yet assembly for your gpu. Pump everything out via the hdmi Ethernet channel.

it probably be full of bloat and will be slow

why not just come to 32chan, its minimalistic and needs some more users tbh

Does it have Holla Forums-like board?

no but one can create it

and there is 16384 bit RSA encryption for HTTPS

Please make it easily compatible with SQLite, TinyIBs SQLite support is broken and I want to run it on my ARM board

There was some chink imageboard written in C but remember, if you go the low-level route you also need to write your own server stack specifically for the engine and that comes with a lot of security issues so it might not be worth it

JOSH CUCKED AGAIN

Can you just publish the code now? That way there's something to comment on. It doesn't matter that it's not in a usable state, it will take a while after it's technically usable to be actually useful too.

I've just kind of given up on imageboards
Holla Forums is filled with fucking retards, Nextchan is slow as fuck no matter how much I try and contribute. I was one step away from samefagging.
Parley, the suckless autist and the brazilian are a bunch of morons.
Odili is cool, but I don't really want to deal with the autists of Endchan combined with how slow it is.

4chan would probably not be too bad were it not for the google captcha and the faggot mods
Actually, the mods of 4chan aren't too bad compared to what this shithole has spawned, something that no one wants to talk about.
Imageboards are dead.

And by slow I mean user count, not response times.
The tor hidden service is really fast, especially.

This is why we laughed you off every other site, ricer.

qq moar

wow, nice gif from giphy bro, epic.
it truly portrays my inner sorrow that this place is packed with subhuman degenerates who don't even bat an eye when an obtrusive ad network is introduced, or when pigfucker senior tries to play celebrity and spouts bullshit for an hour about how he's not a kike who needs to be thrown into the oven.
i'm glad you've found your home where you can shitpost, and pretend that you're not a mentally retarded subhuman who would be better off hanging from the rope of your parents bathroom.

Holla Forums is still good for a few laughs. It's certainly not the oasis of sanity it used to be, but all imageboards are crap today.

4chan is crap for exactly the same reasons Holla Forums is crap. Both have cuck mods who use the pretext of "I was totally joking guize!" to ban the shit out of people for making their backside hurt. I suppose it is better in one sense: no bots or spamming. Even then, giving up your ability to speak your mind just because you meet a bot once and a while is pretty fucking retarded tbh.

Imageboards are dead though. Completely agree on that. Certainly they are dead in the sense they are filled with the exact kind of people outsiders were shunned by in the first place. And as we all know outsiders made chans what they are so without them chans are mostly bad social media platforms. Maybe that's the point of flooding these places with these kinds of people? We may never know.

The fact 8ch is shit doesn't matter chans are dead.
Maybe if you weren't such a fucking sperg, you would be able to use slower chans instead of requiring constant stimulation 24/7.

there's a difference between slow and buzzard pickings
the only way those other imageboards have a chance is if this shithole explodes
and I so very much hope it does

See, the problem with every codebase is that they're not mine. :^) Actually, the real problem is since almost all features are well established and are across multiple files, bugs and bloat are too.

You go start and I'll help! :D

Thanks for the advice, will do tonight when I am home.

Daily reminder that Lynxchan had slower build time than Infinity Next beta, both without caching.

Why?

If you insist. It'll be up on gitgud.io/m712/blazechan in a few hours.

that's what grep is for tbh fam
then you open all of them in your text editor, read it all, and then document it all because josh was too lazy to do so

I wish Chans would go the decentralised route - like Bitmessage, but so that anyone can host a node and allow normies to post through it on their browser.

My thought was that the nodes themselves could set the PoW difficulty required for them to propagate a message. This would allow each one to have differing sensitivities when dealing with spam.

Another thought was more expensive "PoW as Identity" to prevent too much spam/shitposting coming from one identity. Over a certain threshold (bytes/time), the PoW for posting becomes impractically high for that identity.

For web-front nodes, the PoW would have to be done client-side in browser using WebASM HAHAHAHAHAHAHA!

The future is p2p, whether it's web-based or not. The chan format (ephemeral threads around a topic, post numbers) is the best for casual discussion, but we can retain that and make improvements to the parts of the UX that are shitcanned by mods, spammers, and USG right now.

...

looks shitty fam, nextchan is better.

And takes 47ms to BEGIN processing a request.

It was uploaded just now, go check it.

You're using the "python" command (both in the README and in the shebangs), which means python2 on some systems and python3 on others. That is bad.

I said "Install and activate a virtual environment", which sets up both python2 and python3, and Django supports both Python 2 and 3.

I'm not sure I get this. Are you limiting yourself to writing code that's compatible with both versions?

Not really, I'm writing python3-compatible code where I can. I'll change the README in my next sommits I suppose.

Also, here's a dank screenshot of Blazechan running on my Galaxy S3.

gitgud.io/m712/blazechan/blob/d1c429970be1d7710fb5c8a52b4a61a75fb4c21d/blazechan/settings.py#L22

Please learn to read.

you are right. blazechan will never be used in production.

xD

You should not release it like this, because people usually do not read and they will run it with that insecure key that you provided.

You mean 47 ms to laravel boot?
that's really not too bad.

I work with Django professionally and who gives a shit? They'll also have to disable debug.
What I would do is use django-configurations where values like that are populated via environment variables.
Makes settings.py managable past a certain threshold as well.
I know it's saved me a lot of time when I switched to it. I like to label snippets in the settings file with the relevant project that it's referencing, or the version of Django that the snippet requires.
github.com/jazzband/django-configurations

New features in the chunk of commits I'll push today:
- Catalog.
- Bunch of changes to the UI.
- Bump ordering on index.
- Fixed frontpage recent posts.
- Overboard (hopefully).
Also, fun fact, currently one post takes ~30ms on a shitty 1.2Ghz ARM CPU (my tablet, which I use as my main dev environment. It has an X server with an arch chroot on it :^) It has 12 load average on idle with no programs running, lel).

Couple of things.
Use libmagic to correct file extensions/mimes. There's bound to be a wrapper for it in python already written.
It's what josh was trying to do, but without overly masturbating to OOP abstractions like "detectives."
Why bleaching parsed content? Do you not trust the markdown parser?
Try to mark as many contexts as safe in the templates as you can.
Use ugettext, and unicode, as much as possible. Not only does this help translation, but also speed within the context of the framework, just like marking as safe.
I had a couple more things when I was looking at your code earlier but I seem to have forgotten, had to tend to a disk that was completely full.
Probably just that you should use when your settings file starts getting unwieldly. It will.
Honestly take as much as possible from Next as you can. There's a lot right in there as concepts, because Fred and Josh talked about much of it, but it suffers the same fate as any PHP vomit that a single guy wrote. Not bad, just unwieldly classes, no documentation, and the project manager was a cripple who didn't give a fuck to keep him in line in any of these contexts.

Oh, and you probably want to start reading about how django does low level cache keys, Vary headers, etc. since you're not falling for the page cache meme.
I'd say that you shouldn't worry about caching entire pages on the application level. Better having accelerators handle based on Vary headers and cookies or whatever, since you're kind of following in the footsteps with a dynamic focus.
Django as a framework should be very good about this sort of thing, making it very easy for accelerators, low level cache keys, and more.

In addition, dispatch signals are really handy. Use them, even if blocking becomes a problem in some contexts, because that can be fixed easily.

Thanks for the advice, fam.

That's like, way down the line. Images are second class atm, the first class being completing posting and panel.
Yeah, sleuth has a shitload of useless bloat.
Not bleaching parsed content, bleaching the unparsed content. It goes like parser.convert(bleach.clean(self.body, tags=[]))
Why?
Thanks for ugettext. I'm looking into it.

Yeah, I'm not going to Josh it up and say "oh we can just cache later xD".
I'm planning on using Redis for application level caching of places, and Varnish in front of the application to cache pages. I'm not going to give a session cookie to everyone like IN does. Also, fuck file caching (like this site does).

Don't know what it is, looking into it.

Jesus fucking christ that took long to get it through. I fucking hate Kikeflare.

Oh good I'm not the only one who has to reload every page 15 times to post

Speed, because of django's security features
docs.djangoproject.com/en/1.10/topics/signals/

Pushed catalog, overboard, translation and style changes to master.


Should I do something like this: sacrifice posting speed by bleaching fields where html can be injected, and mark everything that could come from a user safe after doing it?

Yeah, that's what I was getting at. Something like that. I think someone might also provide helper fields that are bleached automatically with django. I've used the module before, but not with django.

> docs.djangoproject.com/en/1.10/topics/signals
So it's like Laravel's Event/Listener thing? This can be useful for invalidating application/document cache. Is there another Event system or is this the only one?

I can just do sanit = lambda x: bleach.clean(x, tags=[])self.name = sanit(self.name)self.subject = sanit(self.subject)etc.
to clean it up in the model's save method (overriding it ofc). The tags=[] is needed because by default bleach lets things like pass.

Alright, I'm hitting the hay now. Tomorrow, I'll work on posting, overcatalog and the permission system.

That's the one baked into the framework, but there's plenty of asynchronous event, combining django events with celery, redis queueing, etc.

Jesus, so much shit to read up on. Adding it to the list.

Django Channels can allow you to add real time chat. channels.readthedocs.io/en/latest/

Should we meguca it up and add realtime shitposting?

Yes. Hyperchan, real time autism now!

WE

WUZ

KANGZ

Incoming screenshots

Sheeeit, nevermind these screenshots, they're old. These are the latest

Aaaand part 2.

What front end libraries are you using? Jquery? Bootstrap?

looks like no javascript so far and templates/css inspired if not copied from next

Yeah, it's a lot to read, just do it one step at a time. You don't really need it, just block with the signal dispatcher for now.
What you really need to focus on is probably a dependency on postgres with github.com/jimfunk/django-postgresql-netfields as well
The best way to do it right, Josh did this right.

I'm thinking of this:
Instead of binary encoding IPs and/or just putting them in the DB as strings I can bcrypt them with the secret key in settings.py as the salt. That way if your database somehow leaks you can still be sure that the IPs are safe. This also keeps SQLite as a database option since it'll just be a CharField.

I'm using the unsemantic CSS library for the grid system. No JS is written yet.

Next did the CSS right IMO.

You can have a varchar with the generic IP address field. The problem is that, as the README states, the ORM uses an inefficient cast to do this, to accomodate all databases.
I don't understand, you were just bragging about how posting was so fast, and you want to add a considerable amount of time to check if a user is banned before posting.

Added overcatalog, rolled out my own custom user based on AbstractBaseUser + manager based on UserBaseManager as per the docs. Working on class based views to fix the spaghetti that is frontend/views.py. Will probably implement basic posting either today or tomorrow.

Just considering options fam. It may be worth sacrificing SQLite support in favor of PostgreSQL + binary IPs for speed, idk.

I'd do the bcrypt thing if you were going to share a ban list with others, but honestly it seems pointless when you can just prune IP addresses on a schedule. Needless to say, can't do that with active bans, or active reports. Also, what about ranges?

And, yeah, probably. It's always something you can migrate and drop the dependency, or vice versa.

also if you're actually serious about this I have next week off. How do you feel about Josh's data-widget system?
I'd volunteer to remove all the jquery cruft.

I found it sort of limiting, but I think if I sat down, did that, and also allowed modules to be overridden/modified like ccd0 suggested it could be palatable.
Meanwhile I have like five branches uncomitted on Next because I jump between five things. Big overhaul of the javascript to make it CSP friendly just sitting there.

Also, maybe a way for a module to just pass an object and have the options pane build it, instead of being limited to textbox|checkbox|whatever.

So, either write your own field that returns 'inet' for postgres, or a varbinary field for everyone else, with packed binary values. Or go the postgres specific route.

You're still going to have to monkeypatch sqlite3's DatabaseWrapper in the field, though.
stackoverflow.com/questions/2744632/change-text-factory-in-django-sqlite/28794677#28794677

You dare speak my name, user?

Okay, so here's an idea user (probably was you, saw it on nextchan/next/) shared:
We could do a specific model which has an id and an IP, and posts will have a foreign key to this model. That way, we can still delete multiple posts after the IP is pruned. Two concerns though:
A) Should we group the IP's? If so, what range?
B) Wouldn't posting on the same IP generate dupicate records after a while? Could this be mitigated by only pruning the IP from the database if there no posts from that IP/range (see above) in X days?

Yes, I am very serious about this.
That'd be much appreciated. Basically the only things to do in the JS department are
1. Get rid of jQuery
2. Port the classnames/selectors to ours
Other than that, it can be easily used.

Josh's widget settings menu shit is retarded. I should just be able to pass an HTML tree like
[ { "element": "input", "type": "checkbox", "class": "widget-content-sfw-checkbox", /* children: [{...}] */ }, ...]
and it should do its magic.

Database agnosticism is important but speed is more important. I'll probably patch with inet support.

fuck

Trying to bypass cloudflare

when will there be a way to test it out?

Basic posting is halfway done, but I have to halt my work now because of circumstances beyond my control.


You can clone it and test it locally, but there's still at least a few weeks of work before it is actually usable.

I like the way you think, user. I too would like to see this implemented, as spam prevention is (in my opinion) a top priority to make imageboards better and smaller ones more viable.

OP can this be the frontend to NNTPchan that we need right now? Shitty less functional UIs are part of what is holding back new chans. Also a problem is shitty attentionwhore mods, decentralization/federation is needed badly.

Replace backend with something that connects to NNTP and binds the necessary data to models and you have a frontend.

relevant:
github.com/alvra/django-spotnet
it's buried in there, but it's what we need already written. probably needs refactored.

I think that part is already handled though?
github.com/majestrate/nntpchan

The part that's covered in that project is interfacing with an NNTP server. Most of it is already written. I dunno how non-standard Jeff's shit is, though.

Ah, my mistake. Let's see what he thinks, I like where this is going.

So I searched and we're not the first to have to do this.
nsot.readthedocs.io/en/latest/_modules/nsot/fields.html
As I thought, you need to monkeypatch sqlite to handle text as bytes for this field.

why not a cidr field so you don't have to do _start and _end for bans.

i.e. /32 for single ip

That's not really true. For most difficult or tedious stuff you can use libraries and shit.

I've been doing a little web test thing in C and the only thing for which I can't find satisfactory library is templating, which shouldn't even that big of an issue, when I have printf, so all that remains is escaping and removal of illegal characters.
Otherwise there's libfcgi (~50k *.so) for the server stuff, libgmime for form parsing (~500k *.so - this is the ugliest thing, but that's to be expected if it's supposed to support the (pretty bloated) formal standard and the in-practice standard) and for storage DBs/redis have decent libraries.

And mod tools probably don't take enough resources to warrant writing them in C, unless you need to save space or avoid muh harmful shit.

why? are there no alternatives? (I know Perl and Rails are not good alternatives, but something else maybe?)
I'm no webdev but I've heard the worst things about PHP.

Did you even look at the repo? Do you know what Django is? Why are you talking without reading the OP?


I'm probably going to go for Postgres with the inet and cidr fields. Sorry if you wanted SQLite support.


I'm primarily going for the centralized imageboard route, but as I said in it's simple to make Blazechan adopt other types of imageboard so long as there's a backend for it.

Basic posting and overcatalog is done. I'm currently looking at image uploads. I expect a public test in January.
Commit coming in 4-5 hours.

Pushed posting, overcatalog and a number of fixes to master.

Are you planning to add some sort of API for getting threads, posts, etc. in JSON format?

It's already there.
/{board}/thread.json
/{board}/thread/{id}.json
I pushed it today.

inb4 RIIR

FAGMONKEY LET ME POST

/{board}/catalog.json ?

No. thread.json pretty much returns board info and threads, and you can pass it ?page=x to get more pages. I don't see why you'd bother. I could add a ?page=all to return all threads but it'd be expensive.

I'd do it how Josh did it, past the max pages a board specifies, if it's pushed off within that range, make a thread unbumpable/locked (or only allow something analagous to ghost posting?) on a schedule. have catalog.json return the first through the last page specified by board owner. Then, if a host wants to be their own archive, they can.

Of course you haven't added board settings so just an idea.

Can't we have it early? Nextchan is just one big giant HTTP 500 clusterfuck right now and Holla Forums itself is nearly impossible to post on since cuntmonkey dumped these half assed "anti-spam" measures on it.

endchan.xyz

I want a chan written by someone competent, reputable and not populated by mentally ill flat-earthers and remnants of pedo sites.

...

Go away cat faggot.

that's why endchan forked off lynxchan

fuckoff.kys

I post on Endchan more than anywhere else. Its the best chan next to Wizardchan.

You do understand I need to implement bans, panel and media before I can even make it public right? There's a shitton to do before it's even usable.

bump

bump

There will not be updates for a few days because i got sick. :^( sorry folks.

The question is which fucking CPU architecture?

m68k obviously.

Somehow I got volunteered to a project for a non-profit and I can't turn it down without feeling like an asshole. Still with you bb, just can't port Josh's javascript until around mid January.
I'd recommend I do it anyways, because I fixed a lot of it.
When you get global/board settings implemented, serialize them to JSON and expose them to the templates that way.
Would be templated like this:
{ "meme": "1", "board": "a", "board_settings": { "memesMax": "5", "memesMin": "1", "memeTextMaxLength": 2048, "memeTextMinLength": 420, },}
Or maybe we should make it a dynamic, heavily cached endpoint instead of templating it in the html. That is another request, every time, for JS users tho.
However you want to do it, I'll make it work.

Thanks for your help dude. I can't work on Blazechan right now either because my study is 5-6 Celsius and if I try to code there I can't stop sneezing and my hands get stiff as fuck.
I can bind the JSON API to a model method instead of a view, like Board.as_json(_with_threads for threads?) and use it in board index/thread views.
I'd prefer it in the templating system (it's the easier way to do it), but if you have a better solution for caching I'd be glad to hear it.

PS: can I do something like Laravel's scopeWhereXYZ with Django? Is there a way to make a JSON response straight from a Model instance? I'm currently doing it in a hacky way.

I found some answers.
Adding scopes is done through custom managers: docs.djangoproject.com/en/1.10/topics/db/managers/
JSON responses through serialization: docs.djangoproject.com/en/1.10/topics/serialization/
Also I'm likely going to split the models file to prevent it from growing to a gargantuan size using code.djangoproject.com/wiki/CookBookSplitModelsToFile .

Pushed a couple things to master, including basic caching.

You mean query scopes?
laravel.com/docs/5.1/eloquent#query-scopes
The equivalent would be
docs.djangoproject.com/en/1.10/topics/db/managers/

Fug, didn't read the next post.
Yeah, looks good.

Just preload it with http2 push and it's practically free. If someone's using a non-h2 client they're probably going to request json themselves manually anyway so it doesn't matter.

That's sort of interesting, to have a push manifest for static resources, but I don't think it'd going to be the sort of magic bullet to solve the problem of global+board settings that the javascript needs to make everything just werk.
It's a dynamic thing. Actually, it'd be two requests, if the site settings were one endpoint and board-specific settings were another.
Maybe construct a view that takes care of populating these contexts in the template, so it's nice and DRY, but http2 push probably not going to work for this.

ANd by that apparently in django it's a template tag, or a context processor. Hit cache, build it and cache it if it doesn't exist, and use the event system to invalidate and recache.
Extremely naive, but when and if it's done right you'll be thankful.
Rather than putting all that logic in the templates themselves.

I worked on the panel today, which is frankly a lot easier than many other parts. I also separated models.py and frontend/views.py into multiple files. I'll be able to push something meaningful to master either tomorrow or Sat.

Made a basic panel with authentication and pushed it to master.

bump

This project is great inspiration on how you should do attatchments internally:
github.com/tc77/django-extended-attachments
Although I'd just use github.com/ahupp/python-magic for detecting mimetypes

Blazechan is now live as a beta test.
blaze.nextchan.org
inb4 >imageboard without images, i just wanted to have a test instance i can fuck around with.

I think I broke something

Made can_post_in_board static. I'm still working on it, it's not suitable for shitposting just yet.

m7
m8
pls

Fug, will try to do a quick-n-dirty fix

I hope it's fixed, I don't have time atm.

Nice, another shitty engine that will never be used.
Nail in the coffin. We already have lynxchan that hasn't been adopted by any real site, why bother making inferior garbage based on inferior garbage?

yeah works

board.asm32.info
Just modify it from bb to chan.

Have you considered rewriting it in Rust?

...

Lynxchan is made by a retarded brazillian who doesn't even know how to use proper templating languages, with overrides.
Protip: the way Lynx implemented this makes it slower than Infinity Next (dynamic templating, on the fly, with a huge PHP web framework).
It's just hidden from benchmarks on the face because it's cached to MongoDB. Lynx admitted himself that his shit templates slower than Next.

Also, Next has actually good ideas.
The permissions system, options for dynamic content.
Lynx just took the embodiment of vichan and strapped MongoDB and Node to its back.
But hey, it's okay. VROOM VROOM, NIGGA, V8 GOES FAST.

MongoDB is a terrible mess of shortcuts and data races to squeeze more speed out of an unbalanced rocket-propelled unicycle. Sure, it's faster for most things (Postres beats Mongo in some benchmarks), but at what cost? Really, I'm waiting for this document store fad to die. It's effectively claiming that dynamic typing is better than static for anything that is not a trivial prototype.

Is the retard storing image blobs in the database?

Of course not.

You can't put raw binary into JSON so he does the responsible and secure thing and encodes them as data: URIs first

I hope you're joking.

Let's call it that because I don't want to do research myself to find out if those fucking base64 rumors are true

Lynx admitted to store images in MongoDB because of "sharding".

Oh God.

I mean, I don't think storing images in multiple servers is a bad idea. But using a database for storing images in multiple servers, that's what I disagree with. He mentioned "having to move multiple GBs of furry porn if you need to remove a server", but you do the same if you use database sharding and are going to remove a server.

The problem I have with images in the DB is performance. For mostly any CRUD applications not consisting of 2 million layers of Laravel abstractions or some shit the slowest component will always be the database. Why the fuck would you put even more load on your slowest component? Modern file systems are incredibly fast. Keep your images on a separate server or use a dedicated network FS, if you have to.

Yes, he is. He's either base64ing them or using GridFS, which is chunked, afaik. I forget which off-hand.

And, there is something to be said about that sort of architecture, but it's not like it's something NFS or something like Ceph can't deliver. The I/O bottleneck that Hotwheels complained about, which required him to "move furry porn" between servers, did not have anything to do with file uploads.
It was a consequence of recacheing.

He actually reached and I/O bottleneck on serving files? Top kek

No, he reached an I/O bottleneck on writing. With vichan/infinity with over one thousand boards recaching using vichan's originally naive implementation, it's writing a shitload of index pages/catalogs/threads to network attatched storage. It's recaching immediately instead of recaching lazily, or "semi-lazily", we might say.

Of course, it'd all depend on traffic.
4chan's compromise is caching a page to redis. Holla Forums is cached to redis on a schedule, while other threads/boards are recached on post.
I'm not exactly sure how they do 4chan Pass being backed by redis as a document cache. It's probably some sort of ESI capable HTTP cache, like ledge.

Seems CSP nonces are not just a dead end because of caching, but because they're useless when CSS3 attr selectors come into the mix:
sirdarckcat.blogspot.com/2016/12/how-to-bypass-csp-nonces-with-dom-xss.html
Never thought about that. Neat hack.

fucks sake.

Nextchan back up, Blazechan beta back up.

What do you want to use them for?

Preventing XSS?

I understand the policies, but what are the nonces for? AFAIK you can not load scripts from other domains (or explicitly allowed domains through CSP) dynamically with JS.

We were going to use them for inline JS a while back, then I took a long look at the javascript and realized we didn't need nonces for it. We still need an unsafe-inline CSS policy though. Until the aformentioned CSS3 attr's are implemented in browsers.
Because some features, like the dynamic board favicon and a few other things I can't recall, require inline CSS.

And, to explain: you use nonces for inline JS/CSS.
I'm guessing the front end hipsters are complaining about CSP being shit because they rely on a bunch of dependencies that fuck with the DOM in ways that it doesn't like, and are used to inline JS.
For what we'll need, it shouldn't be a problem. Except for the serveral use cases (board icons, poster ID colors rendered server-side) that require CSS3 attr selectors.

revive

How would you store and read post links in an SQL database?

infinity next is kill
project blaze is kill
OP is kill

Nice meme

You just format links with the parser.
And if you want backlinks to work without javascript you do it how Josh did it.

I am very much alive. Thanks for the bump.

Check out backend/models/Cite.py, I pushed some commits yesterday.

django is cool though

Some guy in /dpt/ is already doing that.

That doesn't appear to be an existing board here.

On 4chan/g/dpt

bump

I am alive, but busy as shit with work. I finished some parts of the control panel but not enough for a commit yet. I can change board settings from the panel now. Will commit today or tomorrow at most.

Finally pushed commits after around 20 days. Sorry for being absent, I had a lot of IRL business going on.

Was just keeping thread alive.
People are busy, shit happens.

Also, you might consider making a @deconstructable validator for Filetypes (your magic implementation looks p good), Quotas (max size divided among all files being uploaded), and Filesizes etc that's validated in the model (and serialized in a migration) instead of doing validation in the form itself.
docs.djangoproject.com/en/1.10/topics/migrations/#custom-deconstruct-method
docs.djangoproject.com/en/1.10/_modules/django/core/validators/#URLValidator
Just an idea if you get the time, didn't look at how you're doing it right now.

Currently I am enforcing file size limits via nginx because I haven't implemented quotas etc.
I did it in the form because I didn't want to litter models with processing, models are data after all. One thing that did come to mind is that I could move the thumbnailing to the PostAttachment model. But the rest is validation, and I'd like to keep it within the form. Thanks for the links.

That's what validators.py is for, it's data that needs to be validated, you specify the validators in the model field and the logic is in validators.py.
Django's imagefield works this way, as does the urlfield, etc.

Also, I would put thumbnailing in a helpers.py, and do that from your model.
Something like:

class ThumbnailHelper(FieldFile) def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) self.sizes = self.field.sizes for size in self.sizes: name = 'url_%sx%s' % size values = self.get_thumbnail_url(size) setattr(self, name, value) def_generate_image_thumb(self, image, t_width, t_height, crop=True) pass def _generate_video_thumb(self, video, t_width, t_height, crop=True, frames=500) pass

Something like that, anyway.

Oh, and you'd override save:
def save(self, name, content, save=True) super().save(name,content,save) ... magicbytes determines filetype for size in self.sizes width, height = size if magic = image data = self._generate_image_thumb(content, width, height) elif magic = video data = self.generate_video_thumb(content, width, height) elif magic = document ...
Or instead of the el/if you can use a method dispatcher, but probably not in this scenario because you don't want to thumb if it's not a category of content you can't thumbnail in the first place. Either way.

I've got two.

1) As alternative to CAPTCHA, allow proof of work: Every page served to visitors is associated with a hash, and there is a JS (which can be easily turned off) that will automatically start cracking that hash when the page loads (or when user clicks a button if you prefer). Valid hash solution must be submitted along with a reply. BO's can decide how many digits of the hash to use to control difficulty, or they can disable it completely.

2) Optional: Every time a thread is created, you are shown the last thread on page 0, with a prompt saying "This thread will die for your post, are you sure?"

lel
I already thought something about this. This is possible to do. I'm going to make captchas pluggable so this can just be another captcha backend.

Sorry for no updates, folks, stuck at work.

bery gud

Better than the current implementation in a bloated web framework for a dynamic language with a shit implementation fam

Yes, I guess breathing helium is better as a hobby than shoving a running chainsaw into your torso.

Initial speed doesn't matter, only caching matters.
Problem was the other framework that Next was written on was built from the ground up to hate document caching. It's all or nothing and it was like trying to ram a peg through a hole.
It was possible but not just pretty, and Josh relying on sessions for the captcha was the nail in the coffin. To decouple that just to be stuck with large, underdocumented classes (although it's easy to find your way around the project structure) that require a lot of grepping just to figure out where the tentacles of the beast are.
Django is nice this way because although you can organize it however you want there's pretty much a slot to fit everything. And the only time django will send a vary header is when you're checking the session explicitly, and not on every fucking page. Which means it's easier to cache off the bat.

Although, on second thought, this isn't really irredemable flaw.
If Jim and his shitty outsourcing firm wanted to fix it, they could have. I thought of intercepting via the validator to enter the captcha for no-js, and progressively enhance it. There was some docs I read that suggested this was possible, but I never bothered to formally suggest it or implement it myself.

See, there's a problem. Your templating is so slow you try to catch it at that level.

Here's some tips frendo:

1) Pick a real language. Best option nowadays for dynamic websites is probably Go, since the ecosystem is great. Other options are .NET Core languages, Crystal and Nim. All tackle a good balance between developer and machine efficiency.

Forget about dynamic languages, even well-JITted ones. But very specially forget about any language without good concurrency+parallelism solutions available. Your problem is *inherently* concurrent.

2) Cache live thread DATA (not documents) at the application level (or an in-memory kv DB).

The most expensive thing (unless you're royally fucking up) will be database access, not templating. If you do this, you can template non-thread dynamic shit easily (like banners).

3) Sessions are not expensive unless you're fucking up. Do them in a memory database if you're already using one to cache thread data, if not you can just use your relational DB in the same instance but in a different DB (or whatever will make the DB aggressively cache it, oh that's another thing, learn how your RDBMS does caching).

Everyone uses document caches, friendo.
Golang sucks dick, deal with it, it's a shit language made for Pajeets on an the degree assembly line.
And? Database caching and invalidation is easy. I'm not sure why you're giving me these tips, fucktard, as if I'm 15 years old.
No one said sessions were expensive, people said that Next's underlying framework uses session in a way that made document caching very hard.
The problem with Next was not database caching, it was well optimized and Josh even fixed bugs with Postgres on the framework level. It was always the fact that it didn't do document caching, static file caching at one point, and the fact that there was captchas being generated on every new session, on page load.

I don't appreciate the talking down, fucktard. Instead, I suggest you go slobber on Rob's dick some more, faggot.

Somehow I suspect that the reason 8ch is slow isn't the language. Typically languages are constant multiples of each other in performance, and the difference isn't that great. Compiled Python is pretty fast as well.

I'd sooner go with the simple explanation, namely that the administrators we've had or have are incompetent and lazy, and since they don't see coding as something that'll translate into more users or shekels, they don't bother. Of course that's the difference between being a bottom feeder-fag and accomplishment, being able to look past the immediate gain/loss to comprehend a grand vision. Oh well, maybe one day we'll get an imageboard admin that isn't a faggot.


Should have said "last page" not page 0. Oh well, cheers user. I like Python but don't know shit about Django or webapps, wish I could contribute.

8ch isn't slow, it's serving from a memory store. It was slow, because the document caching was fucking retarded and it was writing over NFS with all sorts of locking problems and assorted bullshit that is necessary thanks to tinyboard's architecture.
There were multiple "solutions" to the problem (if you call duct tape a solution), fixing the PHP to write the documents or some sort of lazy caching (fixed by czaks), writing to and serving out of redis (odilitime) and writing to a distributed document store written in haskell (codemonkey.)
It could probably serve pretty fast dynamically, if you wanted to completely rewrite it, but go ahead and look at the horrors in Tinyboard+Vichan and pretend it's actually worth saving.
It's only sometimes slow now because a server fell over or a piece of duct tape somehow got loose.
Yes

Are you planning to start a chan (website) after you get chan software in workable state?

Why not both? The way I do it is have 3 fields on each node - data, JSON and HTML. When fresh JSON is requested, the thread data and JSON are cached, but no HTML is generated. Same for HTML. If both JSON and HTML are cached on the same node, the data field can be freed.

you say that until you see the prices advertisers will offer for data

I am the admin of nextchan.org/ and I can tell you that I am straight. ;^)

I hope it will replace Infinity Next on Nextchan but I am not sure whether I should change the site name or not. Thoughts?

Status update: Currently working on panel and a lot of things are done.
- Site settings
- Role editing
- Password changing
- Board creation (25% done)
- Account registration
Interjection, when you see this, have you worked on the Next JS?

Blazechan. Make the theme red

I went to nextchan thing that you linked, and I went to Holla Forums board. It seems dead.
Are you planning to work on attracting more people after you finish B.L.A.Z.E.?

Christ you sound like a total ass. What the fuck does it matter if it's a waste of time in your opinion, maybe he just wants to do it for the hell of it. You know some people have interest they like to fiddle with while you only sit in your mothers basement being "cool". Kill yourself faggot.

has he hurt your feefees? better scurry off back to reddit you retard nigger.

He brings up a fair point but he's being too harsh. A lot of things in life are a waste, but they can still be rewarding if you enjoy them or end up learning something. Also just because a program has been written, doesn't mean it can't be written better. And honestly sometimes the initial barrier of understanding the codebase plus the technical debt of the original designer's poor decisions makes it so that it's less work to just rewrite.

Bump, pushed site settings, more coming this weekend.

New things. The board list in the panel looks normal now and I've managed to make the settings page not look like utter shit. And, uh, posting errors.

How do you version this? Wish you had before/after pictures, more exciting since I guess nobody is actually running this yet.

i see nothing wrong with writing something from scratch. its great those old code bases are there for reference

Can't work on it today because real life is a bitch. I completed site settings (many don't control anything yet but the setting fields in the database are done). I'll try to find time to work on post actions till Sunday.


It's currently incomplete so no versioning/tagging yet. When it is ready to be used as a complete imageboard I will start tagging from 1.0.

And to answer your question, before it was structured like pic #2 it was just , which was complete shit. I stole the current design from Infinity Next.

Nice tabs

ABANDON SHIP. THIS GUY IS AN IDIOT ALERT!

What do you want me to do? What is a sound tagging scheme?

Good online communities are dying in general. Too many averagenerates on the nets today and for some reason figuratively no community tries to filter who's allowed to enter and who must leave. The only good communities I use nowadays are niche forums dedicated to certain topics (e.g., func_msgboard).
I guess back in the days there was a natural filter removing all the cancer, that is "the Internet is too complicated" shit.
If you want to make a somewhat healthy community you should somehow "control" its userbase. But I have no idea what would be the perfect way. Things like "make it too complicated for normalfags to use" and forced Introduce Yourself posts might work, but... Yeah.
In the end of the day, I hate all the shills saying that image-boards are dead. I mean, yeah, they're dying. But that's fucking natural. We had BBSs, text-boards, forums, image-boards, then we got Reddit (that initially was trying to improve the formula). It's an evolution. Someone just have to come up with a new successor.

Please make sure that you do not have bullshit like this in the production version.

Untagged elements slapped around like a hotwheels-tier pajeet, this is a fucking massive pain in the ass to style with custom CSS or interact with through userscripts.

Don't worry about it, I'll strap some classes to those.

Bump, I need to vote something up.
I am about to implement banning actions and post deletion, and I had an idea I could do. Basically, I could make an Author model that has an ID, IP address, hash of the address and the date of the last post that was made from that IP; then I could make a foreign key to this model in place of post IPs. Should I do this or should I just add the author IPs to posts like Infinity Next does?

Pros:
- Can use moderator actions after the IP address is purged. Of course you won't be able to _ban_ the IP but you can still delete board-wide and see the history. I _could_ make banning possible by IP hash though.
- Can make it so the site instantly purges any IPs it collects and still has moderator actions.
Cons:
- This effectively ties your posts together. This might be a big con, that's why I am asking.

What do you say folks?

That's not really a con. A way to easily check what sort of post someone is making is absolutely necessary, both to deal with site trolls, and spammers. It may not be something you'd necessary want to let certain mods or janitors see, but the admin should be able to.

Abstracting things away from IP addresses is generally a smart thing to do, because IP's change all the time. Doing so gives lots more future feature flexibility. And letting mods see IP's is generally a bad idea anyways. Time has shown again and again to never trust a mod, because eventually one of them will try to get at you or your users.


If I can offer a suggestion of my own...
I've been thinking about spam/cp deterrent, in the context of someone who might use this software for a small community and wants to minimize the amount of work involved dealing with that shit. I'd like to see a system that you can turn on where a potential poster clicks a "give me an access password" button, and has to use that randomly generated password that you save and use to make posts.
-To get a password you'd fill out a captcha or "I'm a human" tests after clicking the button. User then saves it somehow. It's on them to figure out how. Automagic use of cookies should be optional.
-The generated password could be both optional (so no further captcha needed) or required (to post text and/or images.)
-An arbitrary time delay after generation before being able to post text.
-An arbitrary time delay after generation before being able to post images.
-The ability to revoke and block password use by IP/abstractidentityhash
-Password expires if not used for a arbitrary amount of time.
-Ability to revoke passwords made within a certain date range
This seems like an easy way to help tighten down a low volume chan from the constant storm of bots and shills, without the whole username/email/password thing that full on forums require. I'd love to hear how any of you other guys would deal with this.


By the way, if you ever get to the point where you want to be able to integrate potential oekaki programs, have a look at gitgud.io/sun/PaintGo

Semantic versioning.

Personally, I don't think there is much gain to all this abstraction. Just keep the actual IPs for an X amount of time. If you store the IPs natively as INET in Postgres, range searches and bans also become trivial.
Now to keep mods from seeing IPs, you can simply present them cryptographically secure hashes of the IP in a human-readable format like these here github.com/Soreil/mnemonics/blob/master/mnemonics_out
The con is that this ties all operations to posts, as to not expose the IPs. For example, to ban a poster, you ban a post. The server then retrieves the IP of the post (if it was not posted more than X amount of time ago) and bans it.

how bout jsut start by not servicing mobile devices?
thetangledweb.net profiler character profiles does not load on chrome or firefox so I have to use phony to make it look like my firefox is a desktop

One of the reasons I wanted to do the Author model abstraction was to enable imageboard admins to not store IPs on their Blazechan instance at all and use hashes/mnemonics (as you've just shown me) to ban people. Also, I think relying on Post objects is bad juju.
Gotta read up on that.

I am not really sure whether this is a reliable method, but I will make the captcha modular, so you'll be able to drop in something else and add the necessary functions and it will work. I'm thinking of porting the brennan captcha to python and using imagemagick instead of quadratic fire tire meme sine, and making a recaptcha plugin, I don't know whether this should be made, but it's to show an example.

hey bud
what's your plans for mobile interactivity?

If you're sure you'll never need range bans, then yeah, that will work fine.

How so?


Simpler captchas can be cracked by bots and there are solved captcha marketing services out there. Providing the most sophisticated implementation as a default is reasonable.

Bump, pushed some commits yesterday. Post actions is 30% done.

Guys, we need a logo, and a Blaze-tan. Pls help.

Author model is 50% done. Have a dank koding screenshot.
Also bumping so thread doesn't get killed by the new /g/ tier threads

Behold!

Not enough cd.. cd .. cd, try again

...

here's something to get you started

Icon in this case could represent Holla Forums's infinity symbol as a legacy/source, a stylized B, 2 classic imageboard clover-style things, and of course looking like firey blazes

Images are still fucky. Hope it won't rely too much on JS.

Here, an AIO pack

That shouldn't happen. Let me check it.

Perfect.

babbys first webdev
did bootstrap and lazyframewerks amaze you that much?

stop using PHP

Nothing on the site is using javascript, you fucking retard. The goal is progressive enhancement. No javascript is even written or copied from the old project, afaik.


samefag

Fuck off, retard. Go gargle gook outsourcing semen somewhere else.

I agree with the point of this though, if you remove/reduce the padding and shadows the interface would be more efficient and functional (especially on small screens).

Probably. I was just replying to the shitposting retard because he needed to be put in his place.
If we're throwing around stylesheet recommendations without doing anything because we're lazy fucks mine would be to use flexbox instead of hacks for the attatchments.
Because it's not like anyone actually cares about IE10 or old Safari.

Also, speaking of reducing use of javashit, libsass's custom functions are really neat. The python libsass implements them, I just used them for a friend's website for multiple, dynamic image slideshows with zero javascript.

@each $page, $images in $slideshow-map { @include slideshow($page, $images)}

Where $slideshow-map is populated with a libsass custom function that returns a list of tuples from the database Feels good.

How big is the current code base?

...

It's a meme you dip

[email protected]/* */ ~/blazechan % find -name '*.py' -exec cat {} \; -or -name '*.po' -exec cat {} \; | wc -l2749[email protected]/* */ ~/blazechan % find -name '*.py' -or -name '*.po' | wc -l50

Development is currently stopped because I'm fighting with real life issues and cannot allocate time. Pls donate $12K if you want me to continue. :^)))))

Hope it gets better user, don't forget to get back to it once the issues are resolved.

I next step with chans is making a full blown programme. Not one that needs to be accessed through browsers.

Have fun compiling to 72 different platforms and making sure they all work. Then come up with a way to make it accessible with one URI navigation. I'll wait.

The only platform it needs to work on is desktop Gentoo