Make the character able to carry every weapon and make weapons actually unique and interesting, not another machine gun and shotgun. Rivet guns, soldering irons, harpoons, anything that looks like an engineering tool, not a weapon.
If you have to explain it with story, make his jacket have a riple in space inside. Every weapon is stored in some small room somewhere unimportant and he reaches inside to get it. You only get that ability after meeting Elizabeth.
Make this a part of the plot as well. Columbia doesn't have guns for civilians as they abhor violence at first so the Resistance has to make do with tools for weapons, while civies turn to Vigors instead. Law enforcement has guns but they explode after a while of picking them up.
Elizabeth is a witch and Vigors are magic. No science here, that's what they are. You don't have to explain shit and you can get away with a lot more this way. You can even present Vigors needing a special chemical that costs a lot and therefore only the upper classes get to use it, making for a good story telling element as well as a gameplay element. You'll find Salts in the richer areas and ammo for your guns in the poorer areas.
Make the world completely open, not linear as fuck. You can go wheverer you want from the start provided you can find the way there and this includes getting documents to go legally or shooting your way through very tough fights. Part of the game is now about figuring out where you need to go based on your current needs both for resources and for the story. Law Enforcement has a number of platoons it sets around the world looking for you and Elizabeth. The more shit you do, the more platoons show up. They recognize you on the spot and they are moving around Colombia. You gotta keep track of them to avoid being found and keep moving. If you shoot them down, more and tougher platoons will show up and the city will evolve into a violent landscape.
Elizabeth is a fucking trainwreck when you meet her. She doesn't talk, she's incredibly scared of everything and getting her out of her tower is quite a chalenge as her awesome magical powers are devastating as shit. Your mission from the start is to save her and get her out of Colombia, but you are stranded in there when you arrive. Getting out will take carving a new way out instead. From helping the higher classes or the resistance or simply doing your own thing, you will choose who to buddy up with in order to get down to earth.
Elizabeth is the original source of Vigors and the technology that keeps Columbia in the air. Daughter of some powerfull witch, she's expected to learn and harness her powers so she can provide the city with more fabulous shit when she's older and recharge the machine that keeps Columbia in the sky. She does not know it, but she has to die for that process to happen, like her mother did. She's kept in the tower for the protection of everyone that fears the witch but because they will need her one day.
Songbird is a creation of her father, a bird made to capture her mother and now attuned to Elizabeth. The father is dead but the bird still keeps his mission of finding and protecting Elizabeth and making sure she stays in her tower. He goes after you and keeps searching at night when you free her and has the ability to travel in time innately to follow you or look for you in other times.
There are 3 time periods in Colombia, you start in the Present. At anytime you can talk with Elizabeth in specific points of the world to travel back or forward in time to the other 2 time periods: Past and Future. Everything you do in the Past and Present can affect the next time periods and activating or doing something to affect the future is key here to solve many puzzles but you'll also often have to bring something from the next time period into the past to solve it.
As an example, you could try to start a revolution with the lower classes in order for them to seize control of Colombia and so you could lower the city amidst the chaos. To do so, you'd stockpile weapons in the Present to be used in the Future but perform some action in the past that triggers racial tension and the creation of the resistance, leading to a Present where people are actively fighting the upper classes. Perform well here and in the Future, the revolution will have won and you can exit Columbia.