The Chinese build walls around the entire city, not just castles. Castles are smaller defense strongholds, only protect the noblemen and their servants etc., but the wall around cities protect the entire city of people.
About the reason, maybe partly because in history, noblemen in the west always treated themselves as different species, whilst in China, we tend to believe in a famous slogan dated about 2000 years ago, that’s King, Lord, Generals and ministers are not birthright, and famous sayings from Emperor is that, Interests of People come first, then the interests of state, then the Emperor.
The Chinese build stone castles, but it was not normal and most of the stones were recycled. China’s military strength is the number of soldiers but not the height of the castles, it’s the bigger difference from western ideas .