Reconstruction

I'm looking for RPG with side quests or otherwise non-main quests that involve building/rebuilding/populating/repopulating a town or something similar (rebuilding gathering crew for a ship can work too)

Not a sim game or a strategy game or 4X or anything where the central theme and main goal is construction and city management, mind you.

Every time I've seen this in an RPG it helped with the overall worldbuilding and was a pretty damn good indicator of a game with good worldbuilding and (enjoyable) lore in general.

Look into skies of arcadia
You won't see it for a while, but you eventually get your own island
I recommend not instantly jumping to max the thing out, though, for plot reasons

Bravely Default. You rebuild and repopulate Nurende in a side mode that gives you acess to weapons.

Breath of Fire, starting with 2, has this, and Suikoden you have your own castle even if all you have to do there is getting more recruits to make it bigger. It is the entire point of Soul Blazer and later in the serie, Terranigma, and it is quite involved in the latter to get the entire world rebuilt.

You are perfectly describing these.

A hidden quest in breath of the wild involves the creation of a new town.

You have probably already played this, but just in case you haven't:
Dark Cloud. It's a dungeon delving third person action RPG about recovering parts of a town (Well, more like 3 towns, but that's not the point) from a dungeon and then rebuilding the town. You can even talk to the people of the town and they'll tell you something that they want (For example one character wants to be woken up in the morning by the sun, so you have to point their bedroom window towards the rising sun).

Shit, you beat me to it.

Dragon Quest Builders, but it's more an adventure game with minecraft overtones.

Wild Arms, along with it's PS2 remake Wild Arms: ACF, allow you to rebuild Adlehyde Castle Town after a particular event happens there early on in the game. While on that note, that place was honestly kind of impressive in the PS1 version, since every building could be entered (with the roofs/upper floors fading to transparent while inside, reducing the need to load into/out of buildings), as opposed to how in some games buildings are just there to make the town seem larger than you could actually explore. Since you posted Tales of Symphonia, Tales of Vesperia also has something similar, but it's more of setting up a town for the first time in a rather inhospitable area (Vesperia's world in general being dangerous to the point most towns have to have a barrier generator to keep monsters out), than rebuilding if memory serves (I'll admit it's been some years since I last played it). I want to say the Dark Cloud games, or at least one of the two, have that, but I haven't played them myself just yet.

Speaking of ability to enter buildings in towns, I think that's an element mostly lost in modern JRPGs: with the push for full 1-to-1 size 3D worlds, as most towns don't have many actual buildings you can enter on the screen (there might be plenty in the background to provide a sense of scale though), and even vendors wind up having open-air stalls at times as opposed to their own full on four-walls-and-a-ceiling stores. Towns taking multiple screens too, but maybe only being one or two screens in actual explorable depth. Also the loss of true overworlds, but that's another, albeit related, gripe.

The mini game in BD was like some gay shit out of a browser game and not all what op is looking for.