The most common and most likely answer to this crossword clue is the 6 letter word SLACKS
I've seen this crossword clue in The Sunday Telegraph
S
L
A
C
K
S
More clues leading to the result SLACKS
Buddy explains!

The straight part
We know that "casual attire" is the straight/definition part of the clue. The straight part is often a synonym or definition, in this case casual attire = slacks.The cryptic/wordplay part
In cryptic crossword there's often an wordplay/cryptic part as well.This part involves a more complex hint, such as an anagram, homophone, hidden word, etc.
Sadly, We don't have an explanation for this specific crossword clue yet.
What does SLACKS mean?
SLACKS noun- The part of anything that hangs loose, having no strain upon it.
- A tidal marsh or shallow that periodically fills and drains.
- To slacken.
- To mitigate; to reduce the strength of.
- To lose cohesion or solidity by a chemical combination with water; to slake.
- A temporary speed restriction where track maintenance or engineering work is being carried out at a particular place.
- A valley, or small, shallow dell.
- Semi-formal trousers that are less formal than those part of a suit but suitable for wearing in most offices and therefore nowadays no longer considered casual trousers. (Takes a plural verb even when referring to a single pair; may be referred to as a pair of slacks)