Concrete damage condition rating based on visual assessment is qualitative and subjective and depends on the investigator’s experience, knowledge, and skills, which results in varied assessments from each investigator. To resolve this problem, a condition rating scale of visual-based concrete damage assessment has been developed in order to show a measured condition and rate the concrete damage conditions of buildings. The scale can describe buildings that are in very good to very bad condition; a very bad rating means that the building cannot be used. This assessment also calculates the building structure element’s critical rating. The condition rating scale is determined based on the concrete’s damage condition and the concrete’s compressive strength requirements. Concrete damage-condition rating values are determined based on expert appraiser feedback that was obtained through a questionnaire, which was processed using the fuzzy logic approach. The condition rating scale usage of visual-based concrete damage assessment will show the building’s structural element damage condition rating and the building damage condition rating as a whole, as well as the actions that need to be taken. Assessment results can also show a priority scale to determine the next actions to be taken by the decision-maker.