We all fuss a little too much with what we wear, if we care about it. A reader reminded me recently of a  wonderful passage in Fitzgerald’s Tender is the Night that illustrates quite how much this fiddling can undermine our ideas of elegance and poise: 

They were at Voisins waiting for Nicole, six of them, Rosemary, the Norths, Dick Diver and two young French musicians. They were looking over the other patrons of the restaurant to see if they had repose — Dick said no American men had any repose, except himself, and they were seeking an example to confront him with. Things looked black for them — not a man had come into the restaurant for ten minutes without raising his hand to his face.
“We ought never to have given up waxed mustaches,” said Abe. “Nevertheless Dick isn’t the ONLY man with repose —”
“Oh, yes, I am.”
“— but he may be the only sober man with repose.”
A well-dressed American had come in with two women who swooped and fluttered unselfconsciously around a table. Suddenly, he perceived that he was being watched — whereupon his hand rose spasmodically and arranged a phantom bulge in his necktie. In another unseated party a man endlessly patted his shaven cheek with his palm, and his companion mechanically raised and lowered the stub of a cold cigar. The luckier ones fingered eyeglasses and facial hair, the unequipped stroked blank mouths, or even pulled desperately at the lobes of their ears.
A well-known general came in, and Abe, counting on the man’s first year at West Point — that year during which no cadet can resign and from which none ever recovers — made a bet with Dick of five dollars.
His hands hanging naturally at his sides, the general waited to be seated. Once his arms swung suddenly backward like a jumper’s and Dick said, “Ah!” supposing he had lost control, but the general recovered and they breathed again — the agony was nearly over, the garçon was pulling out his chair . 
With a touch of fury the conqueror shot up his hand and scratched his gray immaculate head.
“You see,” said Dick smugly, “I’m the only one.”