A lot of states have a minimum square inch requirement for rifle seasons. Usually, just a hat is not going to cut it.
As mentioned, I can't imagine the weather being so predictable a sweatshirt is part of the hunting kit. But I am out West.
I hunt primarily in Wyoming, Colorado, and CA. Surprisingly, the Colorado rifle can be spooky as more hunters take long cross-canyon shots in that 600-700+ range. While a few, no doubt have taken the time to train and use the correct equipment to drop an animal in their tracks. But many are unpracticed, using SFP scopes with BCD reticles that are NOT tunned to the elevation, load or compensated for partial zoom trying to make these long shots. It is scary and sad to hear shot after shot of misses, etc. The last thing you want is to be unseen anywhere near these yahoos.
Due to Covid and more people wanting to get away, the last two public seasons have been crazy busy, and all kinds of stuff like this. It sucks hearing two or three guys perform mag dumps and reloads. My son and I counted 24 shots in one case last year. We later ran into the hunters and they unashamed, recanted their story. Last season as we were breaking down an Elk, another elk walking completely "skylined" on the ridge above us, but only about 50 yards up the hill from us, with maybe an elevation on 30'; a hunter from 650 yards away shot at it, completely missing it. So not only was his choice poor to shoot at a skyline animal, but with crappy skills, he choose to shoot directly over our heads; I am sure we were in his scope's FOV.
The point of my rant, regardless if orange is required or not, in any gun season, I want to be seen, especially where people are taking long shots. I just have seen to much scary crap.