---
title: "Text-prompted Segment Anything on aerial imagery: Grounding DINO and SAM 2.1 in Python"
description: "Segment buildings, trees and water from text prompts with Grounding DINO and SAM 2.1 on a CPU, calibrated on one district and tested on another."
url: https://www.technicalgis.com/tutorials/text-prompt-segment-anything-grounding-dino-sam2/
author: Sean Blaney (https://www.technicalgis.com/about/#sean-blaney)
publisher: TechnicalGIS (https://www.technicalgis.com/about/)
published: 2026-11-20
updated: 2026-11-20
language: en-GB
---

# Text-prompted Segment Anything on aerial imagery: Grounding DINO and SAM 2.1 in Python

> Segment buildings, trees and water from text prompts with Grounding DINO and SAM 2.1 on a CPU, calibrated on one district and tested on another.

- Level: Advanced
- Tested with: Python 3.12
- Source code: [Download scripts](https://www.technicalgis.com/code/text-prompt-segment-anything-grounding-dino-sam2/)

## Key takeaways

1.  Grounding DINO turns a text prompt into boxes, and SAM 2.1 turns each box into an outline. Both are Apache-2.0 and run on a CPU: about 4 seconds per prompt per tile, 4 seconds to encode a 1024-pixel tile and 32 ms per box.
2.  Settings chosen on Kanaleneiland, Utrecht, held up on Parkwijk, 4 km away: building pixels F1 0.70 (IoU 0.54), the same as the best Parkwijk could reach with its own settings. Tree crowns scored F1 0.66.
3.  The words matter: 'a tree.' beat 'tree.', and cutting the image into 512-pixel tiles found more small objects. SAM's outlines lifted building IoU from 0.38 for the boxes alone to 0.54.
4.  It finds whole roofs, not addresses: of 440 Parkwijk buildings under 150 m², mostly terraced houses and sheds, it matched 7. Water scored F1 0.40, because flat roofs and a paved square were called water.

## Before you start

-   Segment Anything on satellite and aerial imagery in Python: a samgeo tutorial, which introduces SAM, image embeddings, box prompts and IoU
-   Comfortable with NumPy, rasterio and GeoPandas
-   Python 3.10+ with the CPU build of PyTorch, transformers, rasterio, geopandas and matplotlib (see requirements.txt)
-   About 3 GB of free disk space for PyTorch and the two models (1.7 GB together), and an internet connection; no GPU, account or token needed

**Short answer:** to segment aerial imagery from a text prompt in Python, load Grounding DINO (`IDEA-Research/grounding-dino-base`) and SAM 2.1 (`facebook/sam2.1-hiera-large`) with Hugging Face `transformers`. Cut the image into tiles, ask Grounding DINO for boxes matching a prompt such as `"house."`, and pass each box to SAM 2.1, which returns the object’s outline. Convert the masks to polygons with `rasterio.features.shapes`, using each tile’s geotransform. Then choose the prompt wording, tile size and score threshold on one area with good reference data, and check them on another before you trust the results.

Segment Anything (SAM) outlines objects very well, but it does not know what they are. In our [samgeo tutorial](https://www.technicalgis.com/tutorials/segment-anything-satellite-imagery/), we told it where to look with points and boxes. In practice, you often have neither: you want “every building” or “every tree” in an image. A text prompt fills that gap. An open-vocabulary detector, Grounding DINO, reads the prompt and draws boxes; SAM turns each box into a precise outline. Esri packages the same two models in ArcGIS Pro as **Text SAM**.

The hard part is not running the models; it is knowing how far to trust them. This tutorial works on two 512 × 512 m districts of Utrecht in the Netherlands, where the national topographic register, the BGT, records every building, every stretch of water and every council tree. We choose the settings on one district and measure them on the other.

You will learn how to:

1.  download **25 cm aerial photography** and matching **reference data** from the Dutch national services, PDOK and the BGT;
2.  run **Grounding DINO** on overlapping tiles so every object is counted once;
3.  turn every box into a **SAM 2.1** mask, encoding each tile once, and **georeference** the masks as polygons;
4.  **calibrate** the prompt wording, tile size and score threshold on one district;
5.  **test** them on a second district with pixel, object and point-based scores, and find where they fail.

All outputs below are real results from [`text_prompt_sam.py`](https://www.technicalgis.com/code/text-prompt-segment-anything-grounding-dino-sam2/), run on 20 November 2026 with Python 3.12, PyTorch 2.15.0 (CPU) and transformers 5.23.0. The machine was a desktop AMD Ryzen 9 7950X, limited to 8 threads, with no GPU. The first run takes a little over an hour, mostly in Grounding DINO, because it tries six prompts at two tile sizes on both sites. It caches the boxes and masks in `data/`, so later runs take a minute or two.

## Install the models for CPU

Install the CPU build of PyTorch first, then everything else:

```bash
python -m venv .venv-samtext
.venv-samtext\Scripts\activate          # on macOS or Linux: source .venv-samtext/bin/activate
pip install torch==2.15.0 torchvision==0.29.2 --index-url https://download.pytorch.org/whl/cpu
pip install -r requirements.txt        # transformers, rasterio, geopandas, matplotlib, ...
```

Both models come from the Hugging Face Hub the first time you load them: 0.9 GB for Grounding DINO base and 0.9 GB for SAM 2.1 Hiera Large. Neither needs an account or token. Recent versions of `transformers` include both architectures, so there is no need to install the original research repositories, which need a C++ compiler for Grounding DINO.

## Step 1: two districts of 25 cm aerial photography

PDOK publishes a new national aerial survey every year. The 2025 summer survey, at 25 cm, is available by WMS under CC BY 4.0. We request 2048 × 2048 pixels for each site, which is 512 × 512 m, and save them as GeoTIFFs in the Dutch national grid, RD New (EPSG:28992):

`text_prompt_sam.py`

```python
SITES = {
    "kanaleneiland": dict(name="Kanaleneiland", lonlat=(5.0990, 52.0720), role="calibration"),
    "parkwijk": dict(name="Parkwijk, Leidsche Rijn", lonlat=(5.0590, 52.0985), role="test"),
}
RES, PX = 0.25, 2048                   # 2048 x 2048 pixels of 25 cm = 512 x 512 m
WMS = "https://service.pdok.nl/hwh/luchtfotorgb/wms/v1_0"
LAYER = "2025_ortho25"                 # summer 2025, 25 cm; "Actueel_ortho25" always points at the latest year
...
params = dict(service="WMS", version="1.3.0", request="GetMap", layers=LAYER, styles="", crs=CRS,
              bbox=",".join(f"{v:.2f}" for v in s["bounds"]), width=PX, height=PX, format="image/png")
r = requests.get(WMS, params=params, timeout=300)
with MemoryFile(r.content) as mem, mem.open() as img:        # PDOK returns a JPEG whatever you ask for
    rgb = img.read()[:3]
```

**Kanaleneiland** is a 1950s and 1960s district of flats in long blocks, with tree-lined streets, a canal and a park lake. We use it to choose the settings. **Parkwijk**, in Leidsche Rijn, was built after 2000. It has terraced houses, apartment blocks, a railway, canals and car parks. It is used only to test the settings.

`Output — the sites`

```text
=== aerial photography (PDOK Luchtfoto 2025 ortho 25 cm) ===
Kanaleneiland            calibration centre RD 135240, 453790 | 2048 x 2048 px at 0.25 m = 512 x 512 m
Parkwijk, Leidsche Rijn  test        centre RD 132510, 456750 | 2048 x 2048 px at 0.25 m = 512 x 512 m
```

## Step 2: the reference, from the BGT

The BGT is the Dutch large-scale topographic register. Councils, water boards and Rijkswaterstaat maintain it to about 20 cm, and it is open under CC0. PDOK serves it through an OGC API Features endpoint, one collection per feature type. We fetch four: buildings (`pand`), water (`waterdeel`), point features including trees (`vegetatieobject_punt`) and road surfaces (`wegdeel`).

`text_prompt_sam.py`

```python
BGT = "https://api.pdok.nl/lv/bgt/ogc/v1/collections/{}/items"
RD_URI = "http://www.opengis.net/def/crs/EPSG/0/28992"

def bgt(collection: str, bounds, path: Path) -> gpd.GeoDataFrame:
    """Current features of one BGT collection in a box, from PDOK's OGC API Features (paged, cached)."""
    if not path.exists():
        feats, url = [], BGT.format(collection)
        params = {"bbox": ",".join(map(str, bounds)), "bbox-crs": RD_URI, "crs": RD_URI, "limit": 1000, "f": "json"}
        while url:
            r = requests.get(url, params=params, timeout=300)
            page = r.json()
            feats += page["features"]
            url, params = next((l["href"] for l in page["links"] if l["rel"] == "next"), None), None
        gpd.GeoDataFrame.from_features(feats, crs=CRS).to_file(path, driver="GPKG")
    g = gpd.read_file(path)
    return g[g["eind_registratie"].isna()]          # the API also returns every earlier version of each feature
```

The last line matters. The API returns the history of each feature as well as its current state: for Kanaleneiland, the water request returned seven records, only one of them current. A feature whose `eind_registratie` (end of registration) is empty is the current version.

We keep existing buildings (`status` “bestaand”), water at or above ground level, and points whose `plus_type` is “boom” (tree). The BGT records each terraced house as its own building, or `pand`. From the air, a terrace is one roof, so we also merge touching buildings into roof **blocks**:

`text_prompt_sam.py`

```python
pand = pand[pand["status"].eq("bestaand")]                       # existing buildings, not planned ones
water = water[water["relatieve_hoogteligging"].ge(0)]             # nothing in culverts or tunnels
trees = trees[trees["plus_type"].eq("boom") & trees.within(site_box)]
pand = pand.assign(geometry=pand.intersection(site_box)).explode(index_parts=False)
pand = pand[pand.area >= 5].reset_index(drop=True)
blocks = gpd.GeoDataFrame(geometry=list(shapely.get_parts(pand.buffer(0.05).union_all())), crs=CRS)
blocks["geometry"] = blocks.buffer(-0.05)
```

`Output — the reference`

```text
=== reference: BGT (Basisregistratie Grootschalige Topografie) ===
Kanaleneiland            buildings (pand)  318 | roof blocks  144 | water 1.70 ha | registered trees  887 | road and parking surface  8.50 ha
Parkwijk, Leidsche Rijn  buildings (pand)  464 | roof blocks  175 | water 2.18 ha | registered trees  533 | road and parking surface  9.57 ha
```

![Two square aerial photographs side by side. Left, Kanaleneiland: long blocks of flats outlined in orange, a canal and a lake outlined in blue, and hundreds of green dots for trees along the streets and in a park. Right, Parkwijk: terraced houses and apartment blocks outlined in orange, a railway crossing the middle, canals in blue and green tree dots along the streets.](https://www.technicalgis.com/_astro/01-sites-bgt-reference.Ct1qhdGf_Z2amC1r.webp)

_The two sites and the BGT reference: buildings (orange), water (blue) and trees registered by the city of Utrecht (green dots)._

The trees are those Utrecht manages: street, park and school trees. Trees in private gardens are not in the register. That makes the tree reference complete for recall in public space, but a crown in a garden counts as a false positive. Keep that in mind when you read the tree precision.

## Step 3: load Grounding DINO and SAM 2.1

`text_prompt_sam.py`

```python
GD_ID, SAM_ID = "IDEA-Research/grounding-dino-base", "facebook/sam2.1-hiera-large"
gd_proc = AutoProcessor.from_pretrained(GD_ID)
gd = AutoModelForZeroShotObjectDetection.from_pretrained(GD_ID).eval()
sam_proc = Sam2Processor.from_pretrained(SAM_ID)
sam = Sam2Model.from_pretrained(SAM_ID).eval()
```

`Output — the models`

```text
=== models ===
IDEA-Research/grounding-dino-base: 232 M parameters | facebook/sam2.1-hiera-large: 217 M parameters | loaded in 1.8 s (both Apache-2.0)
```

The 1.8 seconds is from the cache. The first run spends a few minutes downloading.

## Step 4: boxes from text with Grounding DINO

Grounding DINO’s prompt is a list of short phrases, each ending in a full stop: `"house."` or `"car. bus."`. It returns boxes with a score between 0 and 1 for how well each one matches. On aerial imagery, the scores are low. On Kanaleneiland the highest “house.” box scored 0.60, and nine boxes in ten scored below 0.30, which is below the 0.35–0.4 defaults suggested for everyday photographs. So we keep everything above 0.10 and choose the threshold later from the data.

Two further choices matter, and neither has an obvious answer, so we try both:

-   **Wording.** We try three phrasings for buildings (`"building."`, `"house."`, `"roof."`) and three for trees (`"tree."`, `"a tree."`, `"tree crown."`).
-   **Tile size.** Grounding DINO resizes every image so its short side is 800 pixels. A 1024-pixel tile is shrunk, and a car becomes about 14 pixels long. A 512-pixel tile is enlarged, and the same car becomes about 28 pixels long. Smaller tiles mean more calls but bigger objects.

The tiles overlap by half. Each tile keeps only the boxes whose centre falls in its central core, so an object on a tile edge is counted once, by the tile that sees it whole:

`text_prompt_sam.py`

```python
CANDIDATES = {"building": ["building.", "house.", "roof."], "tree": ["tree.", "a tree.", "tree crown."],
              "water": ["water."], "car": ["car."]}
DET_TILES = (1024, 512)
FLOOR = 0.10                         # keep every box above this; thresholds are chosen afterwards from the scores
MAX_BOX_FRAC = 0.5                   # a box covering half a tile is "the whole scene", never one object

def core(o: int, t: int) -> tuple[int, int]:
    lo = 0 if o == 0 else o + t // 4
    hi = PX if o + t == PX else o + t - t // 4
    return lo, hi

def detect(img: Image.Image, prompt: str):
    inputs = gd_proc(images=img, text=prompt, return_tensors="pt")
    with torch.inference_mode():
        out = gd(**inputs)
    res = gd_proc.post_process_grounded_object_detection(
        out, inputs.input_ids, threshold=FLOOR, text_threshold=FLOOR, target_sizes=[img.size[::-1]])[0]
    boxes, scores = res["boxes"], res["scores"]
    area = (boxes[:, 2] - boxes[:, 0]) * (boxes[:, 3] - boxes[:, 1])
    keep = area < MAX_BOX_FRAC * img.size[0] * img.size[1]
    boxes, scores = boxes[keep], scores[keep]
    keep = nms(boxes, scores, 0.5)                                    # one box per object
    return boxes[keep], scores[keep]
```

We send one class per call rather than `"building. tree. water. car."` in one prompt. Grounding DINO can take several phrases at once, but they then compete for the same boxes. One class per call keeps each class’s scores separate, so each class can have its own threshold.

## Step 5: an outline for every box with SAM 2.1

SAM 2.1’s image encoder is the slow part, so we encode each 1024-pixel tile once and then decode all its boxes in batches of 64. Each mask is turned into polygons with the tile’s own geotransform, so it lands in the right place in RD New:

`text_prompt_sam.py`

```python
def segment(embeddings, boxes: list, batch: int = 64):
    masks, quality = [], []
    for i in range(0, len(boxes), batch):
        inputs = sam_proc(input_boxes=[boxes[i:i + batch]], original_sizes=[[TILE, TILE]], return_tensors="pt")
        with torch.inference_mode():
            out = sam(image_embeddings=embeddings, input_boxes=inputs["input_boxes"], multimask_output=False)
        masks.append(sam_proc.post_process_masks(out.pred_masks, [[TILE, TILE]])[0][:, 0].numpy().astype(bool))
        quality.append(out.iou_scores[0, :, 0].numpy())
    return np.concatenate(masks), np.concatenate(quality)

def to_polygon(mask: np.ndarray, transform, min_part_m2: float = 0.5):
    parts = [shape(g) for g, v in rasterio.features.shapes(mask.astype("uint8"), mask=mask, transform=transform)
             if v == 1]
    parts = [p for p in parts if p.area >= min_part_m2]
    return shapely.MultiPolygon(parts).simplify(RES / 2) if parts else None
...
t_tf = from_origin(s["bounds"][0] + ox * RES, s["bounds"][3] - oy * RES, RES, RES)   # this tile's geotransform
with torch.inference_mode():
    emb = sam.get_image_embeddings(sam_proc(images=tile, return_tensors="pt")["pixel_values"])
```

`multimask_output=False` asks SAM for one mask per box. With a point, SAM’s three candidate masks help resolve ambiguity, such as a window or the whole house. A box already says how big the object is.

`Output — timings for the full first run`

```text
                                     count   total_s  seconds_each
step
Grounding DINO, 1024-px tile           144   583.458         4.052
Grounding DINO, 512-px tile            784  2981.202         3.803
SAM 2.1 image encoder, 1024-px tile     18    77.068         4.282
SAM 2.1 mask decoder, per box        14302   456.565         0.032
```

Grounding DINO took 4 seconds per prompt per tile, whatever the tile size. SAM 2.1 took 4.3 seconds to encode a tile and 32 milliseconds per box. With the four settings chosen below, one 512 × 512 m site takes about 8 minutes on 8 CPU threads, mostly in Grounding DINO on 512-pixel tiles.

![Three panels of the same 160 m square of Kanaleneiland. Left: many dotted white boxes and a few solid orange boxes over blocks of flats. Middle: the blocks of flats inside the solid boxes filled in orange. Right: orange outlines of those blocks closely following cyan BGT building outlines, while a cluster of small houses in the middle has cyan outlines but no orange.](https://www.technicalgis.com/_astro/00-hero-text-prompt-pipeline.BL0TRYA7_Z2vDSTw.webp)

_The pipeline on a corner of Kanaleneiland: Grounding DINO boxes for 'house.' (left), the SAM 2.1 mask in each box that passed the threshold (middle) and the polygons against the BGT (right)._

Figure 2 shows both the strength and the weakness. Where Grounding DINO’s box is right, SAM 2.1’s outline follows the roof edge almost exactly. The low-rise houses in the centre had boxes, shown dotted, but their scores were below the threshold, so SAM never saw them.

## Step 6: choose the settings on Kanaleneiland

Each class needs its own way of scoring:

-   **Buildings:** pixel F1 against the BGT buildings, meaning how much of the building area is right. We also match polygons one to one at IoU ≥ 0.5, against single `pand` and against roof blocks.
-   **Trees:** the BGT has points, not crowns. A crown counts as correct if it holds a registered tree, and a registered tree counts as found if it falls inside a crown.
-   **Water:** pixel F1 against the BGT water.
-   **Cars:** there is no register of parked cars, so we check where the detections are. The share that lie on a BGT road or parking surface measures precision.

`text_prompt_sam.py`

```python
def score(cls: str, pred: gpd.GeoDataFrame, s) -> dict:
    ref, pred = s["ref"], clip(pred, s)
    if cls == "building":            # headline: building area right, pixel by pixel; then object by object
        _, used_p, _ = match_objects(pred, ref["building"])
        _, used_pb, _ = match_objects(pred, ref["block"])
        return dict(pixel_scores(pred.geometry, ref["building"].geometry, s), objects=len(pred),
                    **{f"obj_{k}": v for k, v in prf(int(used_p.sum()), len(pred), len(ref["building"])).items()},
                    block_f1=prf(int(used_pb.sum()), len(pred), len(ref["block"]))["f1"])
    if cls == "tree":                # a crown is right if it holds a registered tree; a tree is found if covered
        hit = gpd.sjoin(ref["tree"], pred[["geometry"]], predicate="within")
        p, r = hit["index_right"].nunique() / len(pred), hit.index.nunique() / len(ref["tree"])
        return dict(precision=p, recall=r, f1=2 * p * r / (p + r) if p + r else 0.0, objects=len(pred))
    if cls == "water":
        return dict(pixel_scores(pred.geometry, ref["water"].geometry, s), objects=len(pred))
    road = ref["road"].geometry.iloc[0].buffer(1.0)
    return dict(objects=len(pred), on_road=float(pred.representative_point().within(road).mean()))
```

We score every wording at both tile sizes and at thresholds from 0.10 to 0.50 in steps of 0.025, and keep the best F1 for each class on Kanaleneiland only:

`Output — calibration on Kanaleneiland`

```text
best F1 on Kanaleneiland for each wording and tile size:
     cls      prompt  det_tile  thr  objects  precision  recall   f1
building   building.       512 0.20       88       0.64    0.86 0.74
building   building.      1024 0.18       77       0.60    0.92 0.73
building      house.       512 0.22       99       0.67    0.85 0.75
building      house.      1024 0.22       76       0.64    0.84 0.73
building       roof.       512 0.15      156       0.55    0.87 0.67
building       roof.      1024 0.15       93       0.54    0.87 0.66
    tree     a tree.       512 0.10      427       0.73    0.71 0.72
    tree     a tree.      1024 0.10      262       0.81    0.68 0.74
    tree tree crown.       512 0.10      461       0.66    0.71 0.68
    tree tree crown.      1024 0.10      143       0.80    0.55 0.65
    tree       tree.       512 0.12     1076       0.51    0.80 0.62
    tree       tree.      1024 0.18      232       0.71    0.74 0.72
   water      water.       512 0.30        7       0.75    0.56 0.64
   water      water.      1024 0.15        9       0.71    0.61 0.66
```

![Two horizontal bar charts. Buildings: 'house.' and 'building.' reach pixel F1 of about 0.73 to 0.75 at both tile sizes, while 'roof.' reaches about 0.67. Trees: 'a tree.' on 1024-pixel tiles scores highest, about 0.74; 'tree.' on 512-pixel tiles scores lowest, about 0.62.](https://www.technicalgis.com/_astro/04-prompt-wording-tile-size.BkMC6MMU_Z1h3NLF.webp)

_Best F1 on Kanaleneiland for each wording and tile size._

Some findings are worth knowing before you write your own prompts:

-   `"roof."` sounds right for imagery seen from above, but it was the worst building prompt, with the lowest precision of the three (55%).
-   `"tree."` on 512-pixel tiles returned 1,076 crowns, only half of which held a registered tree; `"a tree."` returned 427 and scored higher. On 1024-pixel tiles, `"a tree."` had the best F1 of all, 0.74.
-   The best thresholds were between 0.10 and 0.225. Grounding DINO’s scores for aerial imagery are low and differ by wording, so set the threshold by measuring rather than by habit.

For cars, the more we accept, the more land off the road:

`Output — cars on Kanaleneiland`

```text
         objects        on_road
det_tile    512    1024    512   1024
thr
0.10       506.0  197.0    0.79  0.82
0.15       243.0   72.0    0.88  0.89
0.20       126.0   26.0    0.90  0.85
0.25        66.0   10.0    0.89  1.00
0.30        28.0    3.0    0.96  1.00
```

We choose the setting that finds the most cars while keeping at least 95% on a road or parking surface. That is `"car."` on 512-pixel tiles at 0.30, which kept just 28 cars. That is a precise but very incomplete count: at 0.10, the same tiles gave 506 boxes, 79% of them on a road or parking surface. The 512-pixel tiles found 2.6 times as many cars as the 1024-pixel tiles at 0.10, which confirms that small objects need small tiles.

`Output — the chosen settings`

```text
chosen: building: 'house.' on 512-px tiles, score >= 0.225; tree: 'a tree.' on 1024-px tiles, score >= 0.1; water: 'water.' on 1024-px tiles, score >= 0.15; car: 'car.' on 512-px tiles, score >= 0.3
```

## Step 7: test on Parkwijk

Now we apply exactly the same settings to Parkwijk and compare with the best each site could have reached with its own settings:

`Output — accuracy with the Kanaleneiland settings`

```text
Kanaleneiland (calibration)
  building    99 polygons | pixels: precision 67%, recall 85%, IoU 0.60, F1 0.75 | objects (IoU >= 0.5): F1 0.12 against single BGT pand, 0.28 against roof blocks
  tree       262 crowns   | 81% hold a registered tree; 68% of 887 registered trees are inside a crown; F1 0.74
  water        9 polygons | pixels: precision 71%, recall 61%, IoU 0.49, F1 0.66
  car         28 cars     | 96% on a BGT road or parking surface
  best F1 this site could have reached with its own settings: building 0.75, tree 0.74, water 0.66

Parkwijk, Leidsche Rijn (test)
  building   103 polygons | pixels: precision 59%, recall 86%, IoU 0.54, F1 0.70 | objects (IoU >= 0.5): F1 0.08 against single BGT pand, 0.35 against roof blocks
  tree       242 crowns   | 65% hold a registered tree; 67% of 533 registered trees are inside a crown; F1 0.66
  water       11 polygons | pixels: precision 42%, recall 38%, IoU 0.25, F1 0.40
  car         33 cars     | 94% on a BGT road or parking surface
  best F1 this site could have reached with its own settings: building 0.70, tree 0.68, water 0.49
```

![The two districts with every polygon from the four chosen prompts: blocks of flats and apartment buildings in orange, street trees and park woodland in green, the lake and canals in blue, and a few cars in yellow. In Parkwijk, a paved square beside the railway is also shaded blue.](https://www.technicalgis.com/_astro/03-four-prompts-both-sites.O4_6sMSS_Z1vzNIL.webp)

_All four prompts with the Kanaleneiland settings, on both sites._

**Buildings transferred well.** On Parkwijk, 86% of the building area was found, with a pixel F1 of 0.70 and IoU of 0.54. That equals the best Parkwijk could have reached with any wording, tile size or threshold, so tuning on Kanaleneiland lost nothing. **Trees** dropped from 0.74 to 0.66. Parkwijk was built after 2000, so its street trees are younger, with smaller crowns. **Water** fell from 0.66 to 0.40.

![Three line charts of F1 against the Grounding DINO score threshold from 0.1 to 0.5, with Kanaleneiland as a solid line and Parkwijk dashed. Buildings: both peak near 0.225, at about 0.75 and 0.70. Trees: both are highest at 0.1 and fall steeply, reaching zero at 0.35. Water: Kanaleneiland peaks at 0.66 near 0.15, while Parkwijk peaks at about 0.45.](https://www.technicalgis.com/_astro/05-threshold-sweep.CJA69Ef3_ZQ2OLb.webp)

_F1 against the score threshold for the chosen wording and tile size. The grey line is the threshold chosen on Kanaleneiland._

Figure 5 shows why the building threshold transferred: both curves peak at the same place. Tree F1 fell to 0.50 on Kanaleneiland and 0.25 on Parkwijk by a threshold of 0.2. At 0.35, a common default for everyday photographs, no trees were found at all.

### What SAM adds

Are SAM’s outlines worth the extra step, or would Grounding DINO’s boxes do? We score the boxes themselves as building outlines:

`Output — boxes alone against SAM 2.1 masks`

```text
Kanaleneiland            boxes: pixel IoU 0.33, F1 0.50, object F1 0.04 | SAM 2.1 masks: pixel IoU 0.60, F1 0.75, object F1 0.12
Parkwijk, Leidsche Rijn  boxes: pixel IoU 0.38, F1 0.55, object F1 0.04 | SAM 2.1 masks: pixel IoU 0.54, F1 0.70, object F1 0.08
```

SAM lifted the building IoU from 0.33 to 0.60 on Kanaleneiland and from 0.38 to 0.54 on Parkwijk. Most of Utrecht’s buildings are not aligned with the image grid, so a box includes a lot of street and garden around each one.

### Where it fails

The object scores tell a different story from the pixel scores: F1 0.08 against single buildings and 0.35 against roof blocks. On Parkwijk, the chosen prompt found most of the building _area_ but few individual _buildings_:

`Output — Parkwijk buildings found, by footprint size`

```text
             buildings  recall
size
under 20 m²        200    0.00
20-50 m²            46    0.04
50-150 m²          194    0.03
150-500 m²          12    0.58
over 500 m²         12    0.67
median IoU of the matched buildings: 0.67
```

![Parkwijk with polygons coloured by match: large apartment blocks and a triangular building in green, whole terraces of houses in orange with many thin red outlines of individual houses inside them, and small red squares of garden sheds that have no polygon at all.](https://www.technicalgis.com/_astro/06-parkwijk-building-errors.B6ddSCGK_Z1Cf29V.webp)

_Parkwijk building errors: polygons that match one BGT building at IoU ≥ 0.5 (green), polygons with no match (orange) and BGT buildings not matched (red outlines)._

Figure 6 shows two failure modes:

-   **Terraces.** Grounding DINO draws one box around a row of eight houses, and SAM outlines the whole row. The area is right, but none of the eight single houses matches. This is a property of the image, not a bug: from above, a terrace is one roof.
-   **Sheds and garages.** Of 464 BGT buildings in Parkwijk, 200 are smaller than 20 m². None was found. At 25 cm a 10 m² shed is about 12 × 12 pixels, below what Grounding DINO detects reliably on a 512-pixel tile.

Across the 440 Parkwijk buildings under 150 m², it matched 7. If you need address-level footprints, start from boxes drawn around existing footprints, as in our [samgeo tutorial](https://www.technicalgis.com/tutorials/segment-anything-satellite-imagery/), or split SAM’s roof outlines by cadastral parcels.

Water failed in a different way. On Parkwijk, 1.12 ha of the 1.95 ha marked as water was not water: three flat roofs (0.42 ha) and the paved square in front of the station, which you can see shaded blue in Figure 3. On Kanaleneiland, the false water was dark shadow beside blocks of flats, a shaded road and a pale rectangle of hard surface by the park. Much of the narrow, shaded canal network was missed. Grounding DINO knows what “water” looks like in a photograph of a lake, not in a vertical view of a canal between trees. For water, a spectral index on near-infrared imagery or the BGT itself is more reliable.

Cars kept their precision on Parkwijk, 94% on a road or parking surface, but 33 cars is a small share of those parked there. For counting cars, use a detector trained on aerial images of vehicles.

## Step 8: save the polygons

The chosen polygons go into one GeoPackage, one layer per site and class, with Grounding DINO’s box score, SAM’s own quality estimate (`sam_iou`) and the box itself:

`text_prompt_sam.py`

```python
for key in SITES:
    for cls, g in final[key].items():
        if len(g):
            g.drop(columns="box").to_file(gpkg, layer=f"{key}_{cls}", driver="GPKG")
```

`Output — the GeoPackage`

```text
wrote 787 polygons in 8 layers to text_prompt_sam.gpkg (EPSG:28992)
```

Open `text_prompt_sam.gpkg` in QGIS over the PDOK Luchtfoto WMS to inspect the results. Style by `score` to see which polygons Grounding DINO was unsure about. `metrics.json` holds every score above.

## Limitations and how to improve them

-   **Scores are not probabilities.** A score of 0.22 for “house.” means nothing outside this wording, tile size and imagery. Calibrate on an area with good reference data, test on another, and keep the threshold with the prompt.
-   **Two sites, one city.** Both sites are Dutch suburbs photographed on the same summer survey. Expect different results for city centres, rural areas, winter imagery or other countries’ building styles. Calibrate again on a sample of your own area.
-   **CPU speed.** About 8 minutes per 0.26 km² for four classes means about half an hour per km². A GPU makes this practical for whole towns; on a CPU, ask for one class at a time and only on the tile size it needs.
-   **Larger tiles for big objects.** Large flats and the lake were found on both tile sizes; small houses, cars and young trees needed 512 pixels. A town-wide run could use 1024-pixel tiles for large classes and 512 for small ones.
-   **No cross-tile merging.** Each object is assigned to the tile whose core holds its box centre, but a very large building that crosses several tiles can still be split. Merging overlapping polygons of the same class after the run fixes most cases.
-   **Reference data has its own rules.** The BGT draws buildings at the walls, while the camera sees the roof with its overhang and lean. That limits IoU even for a perfect outline of the roof; the matched buildings had a median IoU of 0.67.

## Frequently asked questions

### Can Segment Anything segment from a text prompt?

The original SAM and SAM 2 cannot: they accept points, boxes and masks. The usual solution is to add an open-vocabulary detector such as Grounding DINO, which turns the text into boxes for SAM. This is what samgeo’s LangSAM and ArcGIS Pro’s Text SAM do. SAM 3, released by Meta in November 2025, accepts short text prompts directly, but its weights are gated and it needs a GPU in practice.

### What is the open-source equivalent of ArcGIS Pro’s Text SAM?

Text SAM is Grounding DINO followed by SAM, packaged as an ArcGIS deep learning package (DLPK). In Python, you can build the same chain with Hugging Face `transformers` as in this tutorial, or use samgeo’s `LangSAM` class. In QGIS, the GeoAI, SamGeo and GeoOSAM plugins offer text prompts.

### What prompt should I use for buildings in aerial imagery?

On our Utrecht test, `"house."` scored highest (pixel F1 0.75), with `"building."` close behind (0.74). `"roof."` was clearly worse (0.67). The best wording depends on the area and imagery, so test two or three wordings against a reference before running a large area.

### How accurate is Grounding DINO and SAM on aerial imagery?

On a district of Utrecht not used for tuning, with 25 cm imagery, it found 86% of the building area with a pixel F1 of 0.70 and IoU of 0.54. It found 67% of the registered street and park trees inside a crown, with crown F1 0.66, and water scored F1 0.40. It matched only 7 of 440 individual buildings under 150 m², because it outlines whole terraces and misses small sheds.

### Can I run Grounding DINO and SAM 2 without a GPU?

Yes. On 8 CPU threads, Grounding DINO base took about 4 seconds per prompt per tile, SAM 2.1 Hiera Large took 4.3 seconds to encode a 1024-pixel tile and 32 ms per box. Four classes on a 512 × 512 m site took about 8 minutes. That is fine for testing and small areas; for whole towns, use a GPU.

### Why are Grounding DINO’s scores so low on satellite and aerial imagery?

Grounding DINO was trained on everyday photographs, where a “house” is seen from the side. From above, it is less sure, and correct boxes scored 0.2–0.35 rather than 0.5 or more. That is why we kept every box above 0.10 and chose each threshold from a reference.

## Next steps

-   Start with points and boxes, automatic segmentation and samgeo in [Segment Anything on satellite and aerial imagery in Python](https://www.technicalgis.com/tutorials/segment-anything-satellite-imagery/).
-   Read our overview of [Segment Anything for GIS](https://www.technicalgis.com/articles/segment-anything-for-gis/): SAM, SAM 2 and SAM 3, the tools in Python, ArcGIS Pro and QGIS, and how to check the results.
-   Get building footprints and heights from lidar instead of imagery in [Extract building footprints and heights from LiDAR in Python](https://www.technicalgis.com/tutorials/building-footprints-lidar-python/).
-   Train a model on your own labels rather than prompting a foundation model in [Train a U-Net for satellite image segmentation in PyTorch](https://www.technicalgis.com/tutorials/train-unet-satellite-segmentation-pytorch/).
-   Turn other rasters into clean polygons in [Raster to vector in Python](https://www.technicalgis.com/tutorials/raster-to-vector-python/).

## Data and licences

-   **Aerial photography**: Luchtfoto 2025 Ortho 25 cm RGB, Beeldmateriaal Nederland, served by [PDOK](https://www.pdok.nl/introductie/-/article/pdok-luchtfoto-rgb-open-) under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/). Accessed 20 November 2026.
-   **Reference data**: Basisregistratie Grootschalige Topografie (BGT), the BGT source holders and Kadaster, via the [PDOK BGT OGC API](https://api.pdok.nl/lv/bgt/ogc/v1), under [CC0 1.0](https://creativecommons.org/publicdomain/zero/1.0/). Accessed 20 November 2026.
-   **Grounding DINO** (`IDEA-Research/grounding-dino-base`): IDEA Research, [Apache-2.0 licence](https://huggingface.co/IDEA-Research/grounding-dino-base). Liu et al. (2023), _Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection_, arXiv:2303.05499.
-   **SAM 2.1** (`facebook/sam2.1-hiera-large`): Meta AI, [Apache-2.0 licence](https://github.com/facebookresearch/sam2/blob/main/LICENSE). Ravi et al. (2024), _SAM 2: Segment Anything in Images and Videos_, arXiv:2408.00714.
-   Code: [`text_prompt_sam.py`](https://www.technicalgis.com/code/text-prompt-segment-anything-grounding-dino-sam2/), tested with Python 3.12.0, PyTorch 2.15.0 (CPU), transformers 5.23.0, rasterio 1.5.2, geopandas 1.2.0 and shapely 2.2.0.

## Key terms

- **Grounding DINO**: An open-vocabulary object detector from IDEA Research (2023, Apache-2.0). Given an image and a text prompt, it returns boxes for the objects the text describes, each with a score.
- **SAM 2 and SAM 2.1**: Meta's second Segment Anything Model (2024, Apache-2.0), with a faster Hiera image encoder and a memory that tracks objects through video. SAM 2.1 is the improved checkpoint released in September 2024.
- **Text SAM**: Esri's name for a deep learning package that chains Grounding DINO and SAM in ArcGIS Pro. This tutorial builds the same chain in Python, with SAM 2.1 in place of the original SAM.
- **BGT**: Basisregistratie Grootschalige Topografie: the Dutch national large-scale topographic register, with every building, road, water body and, where councils record them, every tree, at about 20 cm accuracy. Open data under CC0.
- **Pixel F1 and IoU**: Pixel scores compare the area marked as an object with the area in the reference. F1 is the harmonic mean of precision and recall; IoU is the shared area over the combined area. F1 = 2 IoU / (1 + IoU).
- **Calibration and test sites**: Settings are chosen on one site and only measured on another. A score on the site used for tuning is optimistic; the test site shows what to expect elsewhere.

## Data for this tutorial

-   [PDOK (Publieke Dienstverlening Op de Kaart)](https://www.technicalgis.com/data-sources/netherlands/#src-pdok) · Luchtfoto 2025 25 cm aerial photography by WMS
-   [BGT (Kadaster)](https://www.technicalgis.com/data-sources/netherlands/#src-kadaster-bgt) · buildings, water, trees and roads used as the reference, by OGC API Features

Licences and formats for each source are listed in the [GIS data sources catalogue](https://www.technicalgis.com/data-sources/).

---

Source: https://www.technicalgis.com/tutorials/text-prompt-segment-anything-grounding-dino-sam2/
