Inspecting rooftops for property damage requires accurately identifying visible roof issues such as missing shingles, hail impact, and ponding water. However, manually reviewing roof inspection images can be time-consuming, inconsistent, and difficult to scale across many properties. Being able to automatically classify rooftop damage and generate a structured roof damage summary can help insurance teams, roofing contractors, and property managers review roof conditions more efficiently.

The Classify task on the Abilities tab can analyze rooftop imagery and assign one of several roof condition labels based on the visible damage type. In addition, the Describe task can generate a structured roof damage report that identifies the roof type, whether damage is visible, the type and location of the damage, severity, visual evidence, and recommended next steps.

For example, a roof image should be classified as No_Damage if the roof appears intact with no clear missing shingles, hail impact marks, or standing water. A roof image should be classified as Missing_Shingles if shingles are visibly missing, lifted, torn, displaced, or if exposed underlayment is visible. A roof image should be classified as Hail_Impact if there are visible circular impact marks, bruising, pitting, or granule loss patterns across the shingles. A roof image should be classified as Ponding if standing water or clear puddles are visible on a flat or low-slope roof.

We will need to separate true roof damage from confusing cases such as normal shingle texture, shadows, dirt, leaves, vents, roof stains, mild weathering, glare, or damp roof surfaces without actual pooled water.

Our expected inputs are images of rooftops, and the expected output will be a single classification label: No_Damage, Missing_Shingles, Hail_Impact, or Ponding. The Describe ability will then output a structured JSON roof damage report.

UI Tutorial

Step 1: Create a Classify Ability

Go to the Abilities tab and select the button Create Ability.

Fill out basic information about the ability such as its name and the description of the task itself. Since we are classifying a rooftop image, select the Task Type as Classify.

Name: roof-damage
Description: Classify rooftop images by roof damage type

Step 2: Classification Task Configuration

To configure the task, select a dataset for the specific task. If you have already uploaded your rooftop images in a dataset, simply select the name of your dataset.

However, if you haven't already done so, select New Dataset, upload your rooftop images, and create the labels No_Damage, Missing_Shingles, Hail_Impact, and Ponding.

When labeling the dataset, use No_Damage for roofs that appear intact, Missing_Shingles for roofs with missing, torn, lifted, or displaced shingles, Hail_Impact for roofs with visible hail bruising, dents, circular impact marks, pitting, or granule loss, and Ponding for flat or low-slope roofs with visible standing water.

Step 3: Classification Configuration

Our next step is to configure the prompt, select the model, and set the image size. For this use case, we recommend using the below prompt and settings for highest accuracy and best results.

Prompt:

Determine the primary roof condition in the image and assign exactly one label: ['No_Damage', 'Missing_Shingles', 'Hail_Impact', 'Ponding'].

Choose 'No_Damage' only if the image's main focus is a roof that appears intact, with no clearly visible missing shingles, hail impact marks, or standing water. Normal roof texture, mild dirt, leaves, roof vents, shadows, age discoloration, or lighting variation should not be classified as damage.

Choose 'Missing_Shingles' only if the image's main focus shows a shingle roof with one or more clearly missing, lifted, torn, or displaced shingles. This includes exposed underlayment, exposed decking, bare roof patches, uneven shingle rows, or visible gaps where shingles should be.

Choose 'Hail_Impact' only if the image's main focus shows visible hail-related surface damage. This includes circular impact marks, dents, bruised shingles, granule loss spots, pitting, or repeated small round damage patterns across the roof surface. Do not choose this label for normal shingle texture, dirt, shadows, glare, or general color variation.

Choose 'Ponding' only if the image's main focus shows standing water accumulated on a flat or low-slope roof. This includes visible puddles, reflective water pools, water collected around drains, or large wet areas where water has not drained. Do not choose this label for a roof that is merely damp without obvious pooled water.

If multiple roof conditions are visible, choose the most visually dominant roof damage type. If no clear damage is visible, choose 'No_Damage'.

Return only the single best-fitting label.

We chose Max New Tokens to be 10 since outputting one classification label will not require many new tokens. In addition, we use an image size of 640x640 to preserve enough detail to distinguish missing shingles, hail impact marks, ponding water, and normal roof surfaces. Since this is an image classification task rather than a video event detection task, FPS is not applicable.

Step 4: Run Classification Evaluation

To check how well the prompt does against the dataset, our next step is to run the evaluation. If needed, review the examples in your dataset to ensure all necessary images are correctly labeled and can be used in the evaluation.

Step 5: Check Classification Evaluation

All evaluations can be reviewed in the Abilities tab by clicking the dropdown arrow next to the associated ability-alias. Evaluations can take around 15-20 minutes to complete based on the size of the dataset.

In addition to the performance, recall, and precision percentages on the Abilities tab, you can see a visualization of what the model predicted by revisiting the dataset. Click on the three dots and select "Go to reference dataset".

Select one of the images in the dataset and click on the review button.

After running the evaluation, you can see what the model labeled as No_Damage, Missing_Shingles, Hail_Impact, or Ponding and compare it to your source of truth. With this, you can improve your prompts and thus improve your accuracy.

Step 6: Create a Describe Ability

For the Describe ability, use the same model and image size settings, but allow more output tokens because the model needs to return a structured JSON object.

Max New Tokens: 250
Scaled to: Medium – 640x640

Prompt:

You are a roof damage assessment assistant. Analyze the rooftop image and describe any visible roof damage.
Return only valid JSON. Do not include markdown or extra commentary.
Use this exact JSON structure:
{
"roof_type": null,
"damage_detected": null,
"damage_type": null,
"damage_location": null,
"severity": null,
"visual_evidence": null,
"recommended_next_step": null
}
Instructions:
- "roof_type" should describe the visible roof material or roof style, such as asphalt shingle roof, tile roof, metal roof, flat membrane roof, or unknown.
- "damage_detected" should be true if clear roof damage is visible, otherwise false.
- "damage_type" should be one of: "missing_shingles", "hail_impact", "ponding", "no_visible_damage", or "unknown".
- "damage_location" should describe where the damage appears in the image, such as center roof section, lower left slope, around roof vent, near drain, or unknown.
- "severity" should be one of: "none", "minor", "moderate", "severe", or "unknown".
- "visual_evidence" should briefly explain the visible clues supporting the assessment.
- "recommended_next_step" should briefly state what a property owner, inspector, or roofing professional should do next.
Do not guess damage that is not clearly visible. If the image does not clearly show a roof, set "damage_type" to "unknown".

Step 8: Test the Describe Ability

After creating the Describe ability, test it on several rooftop images from each class. The expected output should be a structured JSON object that summarizes the visible roof type, whether damage is present, the damage type, damage location, severity, evidence, and recommended next step.

For example, if the image shows missing shingles in the center of an asphalt shingle roof, the output should identify the roof type as an asphalt shingle roof, set damage_detected to true, set damage_type to missing_shingles, describe the location of the missing shingles, estimate the severity, and recommend professional inspection or repair.

Text Output:

{
"roof_type": "asphalt shingle roof",
"damage_detected": true,
"damage_type": "missing_shingles",
"damage_location": "center roof section",
"severity": "moderate",
"visual_evidence": "Several shingles are missing or severely damaged in the center of the roof, exposing the underlying underlayment and wood decking. The exposed area shows signs of weathering and potential water intrusion.",
"recommended_next_step": "Schedule a professional roof inspection to assess the extent of damage and determine if repairs or replacement are needed."
}

Tips for Accuracy

1. Explicit "Negative" Case

Telling the model exactly what not to look for is just as important as telling it what to look for. This is because if you only define damage types, the model may classify normal roof texture, dirt, leaves, vents, discoloration, or shadows as roof damage.

In our prompt, the explicit negative case is No_Damage. This label covers roofs that appear intact and do not show clear missing shingles, hail impact marks, or ponding water.

2. Define "Edge Cases"

The key to high accuracy is a deep understanding of the acceptance criteria for each roof damage type. In a roof inspection context, the line between normal weathering and actual damage can be thin.

3. Use Synthetic Images

Synthetic images from Nano Banana 2 can help create examples of normal roofs, missing shingles, hail impact, and ponding water. However, the images should still look like realistic roof inspection photos. Include variations in roof type, angle, lighting, severity, and camera distance so the model does not overfit to one visual style.

For No_Damage examples, include both intact shingle roofs and intact flat roofs. For Missing_Shingles examples, make sure exposed underlayment or clear gaps are visible. For Hail_Impact examples, show repeated circular impact marks or granule loss without missing shingles. For Ponding examples, show clear standing water on a flat or low-slope roof.