Brown bags used to mean one tan tote that went with precisely nothing else in your wardrobe and got dragged out every spring whether it matched or not. That has changed properly in the last few years. There is more range now, more depth of colour, and brown has quietly taken over from black as the bag people reach for first, which I think is the right direction. It photographs softer, it wears in rather than wears out, and it does not show every mark the way black fake leather does under bright light.
The easiest shades to live with are the ones that sit between colours rather than trying to be a feature. A mid tan, a warm chestnut, a chocolate with a bit of red in it rather than a flat grey brown, these all behave themselves against navy, olive, cream, grey and most denim without you having to think about it. Where brown gets difficult is next to black. The two together can look like an outfit assembled in the dark rather than on purpose, so if you are wearing a black coat or black trousers, a very dark brown bag often reads as a mistake rather than a choice, and you are better off going properly black or properly tan and committing. Brown also struggles against cool toned brights, a sharp cobalt or a true red can make a brown bag look dull rather than grounding, so save those outfits for a block colour bag instead.
The honest limit is that brown is a daytime, autumn and winter colour at heart. In high summer, against pale linen or white, it can look heavier than the outfit needs, and a lot of women end up reaching for straw or a pale neutral instead for exactly that reason, which is fair. It also does nothing for an evening look that wants a bit of shine or drama, brown rarely reads as dressed up no matter how good the leather is, so for anything after dark I would still go elsewhere.
If you own none, start with a mid tan in a simple, structured shape rather than anything slouchy or with heavy hardware. That shade works from September through to April, goes under a coat without an argument, and does not date the way a very trend led colour or finish will in a couple of seasons. The darker, almost burgundy browns are lovely but they are a second bag, bought once you already know you reach for brown often enough to want a different mood from it. On its own, a brown bag will not do the job a black one does for evening, and it is worth being honest about that before you buy.