Flat File vs. Relational

What type of database do you prefer and what type of projects are they for?

I've used MySQL only very briefly. More often than not, I'll use a simple flat file set up. I usually don't make anything more demanding than templates and single-user software. I never felt going through the trouble of relational databases was warranted in my case.

forgot pic of my db

That's why sqlite exists.

I was thinking of creating a DB of all my media files as a nas instead of using a file system.
Does anybody knowledgeable here know which would be better?
Seagate 8TB archival HDDs are dirt cheap for their size, so I was thinking of buying 4 and putting them in a Raid1.

What uses do databases serve outside of Networking exactly?

I don't think there is any database that you can actually mount, so a file system would obviously be better.


Concurrency

Well I would of course have a file system, but have the database handle getting files and such. Don't databases keep them all in one big file? That's what I thought SQLite does anyways.

Well, they split the file at some point, but generally yes.
What I meant is accessing the files. If you can't mount it you need another program to interact with the database, which would then be pretty much like a ftp client.

sqlite for anything that doesn't generate a loadavg >= 1.0. postgres for everything larger.

SQLite for small one-user projects here and there. It's got limited concurrency, so I might mess around with busy timeouts and see what happens.

Otherwise MySQL is my go-to but I want to switch to MariaDB.

this.
/thread

I just realized if I wanted to use PHP 7 I only have a choice between PDO and MySQLi? Am I fucked or what.

If it's a small database which won't grow, such as a configuration file, then flat file, otherwise just use PostgreSQL. It all depends on the scale of your program.

what were you using before?
i haven't had much a of a look at dbs

Any good books to into databases?

You had one job.

Don't

No, sqlite is a nightmare of bad design and should only be used if there was no way to use postgresql.

Try something like CouchDB or some other document oriented, schema-less, noSQL databases. They are pretty based, some even provide ACID-transactions.

But actually, you are a pleb because you never learnt to into relational database design, I won't hate you for that, but relational databases are always superior for large systems if you know what you are doing.

Is this just another complaint about the type coercion? What's so hard to understand about "everything is a string except primary keys"?

No, it's fucking garbage if you use it to share data. Every client has to be very carefully written to not corrupt the database. Using foreign keys and cascading delete? Sucks if a client that didn't enable it on every fucking connection touches your database. Expect it will automatically handle concurrency for you like any other database? Nope, every client must enable that or shit will just fail. The option to do that isn't even present until relatively recent versions. Think it will sanely handle concurrency to keep things fast and avoid starvation? Fucking nope, it polls under the hood at like 100ms so at high concurrency not only can you expect random MASSIVE bursts of slowness (it literally sleeps the full interval if it loses) but if the competition is on timers you might have a connection that keeps losing the race and never gets serviced and hits the timeout. There are a million things I've had to work around to get this piece of shit to work for us, it is the worst "database" I've ever had to use and I've used garbage from the '90s. The sloppy type system is just one more disaster on the pile.

Oh and I forgot my favorite, a WAL database client crash while it had a connection open? Enjoy the INFINITELY GROWING FILE even if you had limits on the WAL size.

I don't think you understand what "loadavg >= 1.0" implies

I don't think you understand that IT FUCKING SLEEPS THE WHOLE INTERVAL IF IT LOSES THE RACE. You can be stuck sitting there with your dick in your hand with no idea why your code doesn't seem to be getting anything done if you use poorly designed software like this somewhere in your stack, and timing synchronization problems are very hard to debug.

Example: Have a light load program that does a 1ms transaction every clock%100ms. Have a heavy load (100% CPU) program that does a transaction every clock%1ms and takes 1ms of CPU. If the light load program is scheduled first, the heavy load program loses the race for the lock and waits 100ms. At that time, the light load program has just been scheduled again and the heavy load program loses again. Worst case you'd get 1% CPU utilization and mysterious timeout errors. What's your loadavg there, champ?

I'm not sure if they've changed this since I ran into it but I wound up writing my own locking using flock to avoid this mess entirely. You might as well since sqlite's locking is a joke and nothing like what you'd expect from a database anyway. flock is a huge improvement in every way... The whole database is full of Josh-tier understanding of systems.

You sound like you've been caught doing exactly the sort of retarded concurrent shit with sqlite I've been saying not to this entire time.

Which for sqlite turned out to be defined as ANY concurrent shit. I didn't expect it to be that broken.

There's little point to using a database if it won't be accessed concurrently. 99% of the complexity of the thing is just wasted and you're stuck with all the bloat and inflexibility.

I use CSV you fucking niggers

SQL niggers eat a dick ok?

Depends what it's being used for.
If I need a real RDB server, postgres.
Mongodb if I need something more dynamic and hierarchical.
If I need something simple for an application where a full rewrite with each run is good enough, I use JSON.
For higher requirement applications (or as an easy application file format), sqlite or Berkeley db.

I use SQLite for anything that doesn't need multiple users accessing it.
I've been using MySQL elsewhere, but I'm thinking of trying out postgres.

I use postgres for essentially everything.

If I actually had enough data for it to matter, I would use HDFS/Hadoop. But I don't, so I don't.

If I'm just messing around and testing stuff out, I am probably using Ruby and will just use to_yaml for anything I need to store.

Usually MySQL
Sometimes MongoDB (Muh ORM)