Since poison damage is inflicted once the substance it's inside the victim's body, it doesn't make sense that a piece of cloth, no matter the weight, turns it less lethal. Still, I like the idea of giving fabrics a bit more love. It's very recurring to find builds with Armor Sloping or even Ballistics feats, I have made one with Skinner, yet I have not had the occasion of seeing one with Clothier. Many other, say, defensive or crafting options are always preferable.
You made me think of an answer to further develop the additional leather component that you propose (which inverts the current crafting design of leather armor), that is if it's considered appropiate. Let's take your idea of using one fabric as base for the robe - black, antithermic, heavier ones like kevlar or blast. In case of using the secondary component, they would still offer less protection than leather armor o tactical vests. So only glass cannons or agile builds would take them as you said. I get the point that this further sacrifices resistance to projectiles to get that elemental damage reduction and the lowest armor penalty (kevlar alone being less than when with a ballistic plate in the chest, unless you manage to craft something like the Tyrys fibers of Aegis). So, as for having an additional cloth, there's the possibility of a relatively light Biohazard - NBC piece of rubber. At the moment, Heartbreaker serpent skin or Biohazard Vests are the only sources of biological protection for crafting. I think it should be acceptable to give the fabric option. After all, those must be some of the materials used for the unique armors. This considering how improbable is to see the player using the Biohazard suit in combat, when a sea serpent, mutant dog or Chemical Assaul Unit armor is chosen first for its specialized bio and/or acid defenses. If the suggestion is for Infusion, it can't make Depot A obsolete, haha.
Speaking of new types, gas masks can be a compromise of mechanics and tailoring as lighter helmets with the option of respirators. But maybe I'm biased, because I'm fascinated with them. Nevertheless, they can get the dodge penalties of googles. Their bonus being protection and even night vision (looking at you, Grey Army Spec Ops), but not perception or other effects so that lens stay relevant.
And, it ocurred to me some potential for Biology and further customization for the padding, be it for the robe crafting suggestion or the other armor types. Bones. The more hellish the creature, the more the skill required. Can you imagine looking even slightly similar to Bænkräster? The Iron Heads motifs were skulls. We already use carapaces of insects and crustaceans for shields. Give an alternative to the concept behind the Lawgiver Helmet to enhance the Yell feat and intimidation in general. Not that we didn't have it (mutant dogs), but it can fit with the post apocalyptic theme... although we get that North Underrail is more civilized than South, lore wise. Still, I found it worthy of mention. It can even offer some small mechanical resistance depending of the creature, so if you want to stay light you only look for certain ones to get the intimidation bonus, or if you can spare the armor penalty, add (literally) the bulk of your trophies.