The most common and most likely answer to this crossword clue is the 5 letter word ADOBE
A
D
O
B
E
More clues leading to the result ADOBE
Buddy explains!

The straight part
We know that "building material" is the straight/definition part of the clue. The straight part is often a synonym or definition, in this case building material = adobe.The cryptic/wordplay part
In cryptic crossword there's often an wordplay/cryptic part as well.This part involves a more complex hint, such as an anagram, homophone, hidden word, etc.
Sadly, We don't have an explanation for this specific crossword clue yet.
What does ADOBE mean?
ADOBE noun- An unburnt brick dried in the sun.
- The earth from which such bricks are made.
- A house made of adobe brick.