If it wasn't a problem for you why even ask? I have regularly gotten past silent isle on iron man runs before with all different builds, just silently kill the rathounds and then sneak past the bugs with some stealth gear. Also you can drop caltrops all over the left side of the island - the bugs tend not to walk over them so they won't randomly get aggroed or aggro your way off the island. It was pretty depressing to survive that fight at level 8 one time only to have Capn Broderick eat it while fighting the bugs and not have a way off the island.
Generally I don't like toning down the hardest difficulty - it's not there so you can easily win. The only thing I dislike about dominating is the economy squeeze - feels just a little too harsh. maybe 66% on normal and 33% on dominating...