=== versions === python 3.12.0 | lightgbm 4.7.0 | scikit-learn 1.9.1 | numpy 2.5.4 | pandas 3.0.6 | odc-stac 0.5.3 | rasterio 1.5.2 === Step 1: samples and the shared spatial split === 9,000 samples: 6,544 train in 63 blocks (5 folds), 2,456 test in 27 blocks === Step 2: Sentinel-2, Sentinel-1 and terrain features === 62 Sentinel-2 features (4 seasons x 10 bands + 5 indices, plus 2 NDVI change features) 13 Sentinel-1 features (4 seasons x VV, VH, VH/VV in dB, plus the VH seasonal range) 2 terrain features; elevation 115 to 635 m, slope median 7.3 deg sample feature table: 9,000 x 77; share of missing values 0.69% (mostly winter Sentinel-2: LightGBM handles NaN natively) median of selected features by class (training samples only): bog heath brack semin impro broad conif water built ndre_summer 0.45 0.49 0.51 0.55 0.58 0.65 0.61 0.08 0.34 ndmi_spring 0.07 0.06 -0.01 0.24 0.32 0.13 0.34 0.02 0.05 vh_winter -18.15 -16.99 -15.39 -18.47 -18.17 -12.54 -13.45 -22.92 -13.39 ratio_summer -5.89 -4.92 -6.05 -5.10 -5.11 -5.88 -5.08 -4.42 -6.17 vv_spring -11.37 -11.74 -11.31 -12.72 -12.29 -7.70 -8.27 -18.83 -7.26 elevation 504.73 413.70 328.72 360.96 261.89 244.12 324.08 211.00 188.84 slope 3.75 6.99 15.87 8.24 7.13 12.99 14.22 0.00 4.17 === Step 3: class weights and LightGBM tuning with blocked 5-fold cross-validation === class weights: bog 0.98, heath 1.08, brack 1.02, semin 1.15, impro 1.01, broad 1.06, conif 0.83, water 0.84, built 1.14 S2 (62 features): 8 settings tuned in 51 s best: {'num_leaves': 15, 'min_data_in_leaf': 40, 'feature_fraction': 0.7}, 90 trees; blocked-CV log loss 0.4978, OA 0.831 +/- 0.020 S2+S1 (75 features): 8 settings tuned in 50 s best: {'num_leaves': 15, 'min_data_in_leaf': 40, 'feature_fraction': 0.3}, 90 trees; blocked-CV log loss 0.4393, OA 0.848 +/- 0.015 S2+S1+terrain (77 features): 8 settings tuned in 52 s num_leaves min_data_in_leaf feature_fraction cv_logloss n_trees 15 40 0.3 0.3931 98 15 10 0.3 0.3951 95 63 40 0.3 0.3958 92 15 40 0.7 0.4026 86 63 40 0.7 0.4033 85 15 10 0.7 0.4069 86 63 10 0.3 0.4079 83 63 10 0.7 0.4214 70 best: {'num_leaves': 15, 'min_data_in_leaf': 40, 'feature_fraction': 0.3}, 98 trees; blocked-CV log loss 0.3931, OA 0.867 +/- 0.022 same model, random 5-fold CV of the training samples: OA 0.894 (blocked 5-fold: 0.867) === Step 4: test-block accuracy, ablation and comparison === fit times: S2 0.5 s, S2+S1 0.4 s, S2+S1+terrain 0.4 s, Random forest 2.1 s Part 1 map sampled at the test pixels; unclassified there: 5 (scored as errors) test blocks: n = 2,456 samples in 27 blocks; 95% CI from 2,000 block-bootstrap resamples n_feat oa ci_lo ci_hi macro_f1 kappa f1_bog f1_heath f1_semin f1_impro bog<>heath semin<>impro model Part 1: spectral rules 8 0.600 0.527 0.664 0.631 0.547 0.504 0.601 0.327 0.541 86 256 Random forest (S2+S1+terrain) 77 0.831 0.758 0.883 0.840 0.808 0.837 0.849 0.606 0.714 58 192 LightGBM: S2 62 0.790 0.718 0.848 0.805 0.761 0.787 0.807 0.529 0.628 73 238 LightGBM: S2+S1 75 0.801 0.724 0.862 0.815 0.774 0.811 0.821 0.509 0.633 69 250 LightGBM: S2+S1+terrain 77 0.853 0.801 0.892 0.862 0.833 0.862 0.841 0.703 0.753 59 155 paired block-bootstrap differences in overall accuracy (95% CI; share of resamples <= 0): LightGBM: S2+S1 - LightGBM: S2 +0.011 (-0.001 to +0.026; 0.043) LightGBM: S2+S1+terrain - LightGBM: S2+S1 +0.052 (+0.022 to +0.092; 0.000) LightGBM: S2+S1+terrain - LightGBM: S2 +0.064 (+0.035 to +0.101; 0.000) LightGBM: S2+S1+terrain - Random forest (S2+S1+terrain) +0.023 (+0.003 to +0.048; 0.009) LightGBM: S2 - Part 1: spectral rules +0.190 (+0.159 to +0.220; 0.000) LightGBM: S2+S1+terrain - Part 1: spectral rules +0.254 (+0.209 to +0.305; 0.000) confusion matrix, LightGBM S2+S1+terrain (rows = Living England, columns = model): bog heath brack semin impro broad conif water built ref bog 224 24 2 7 0 0 0 0 0 ref heath 35 269 11 7 2 0 0 0 2 ref bracken 2 17 250 8 0 6 0 0 2 ref seminat_grass 2 0 10 226 131 1 0 0 0 ref improved_grass 0 0 1 24 254 1 0 0 2 ref broadleaf 0 0 4 0 4 282 19 0 7 ref conifer 0 4 0 0 0 7 108 0 1 ref water 0 0 0 1 0 0 1 132 4 ref built_bare 0 0 1 0 2 7 1 0 351 n_test producers_acc users_acc f1 f1_part1 Blanket bog 257 0.872 0.852 0.862 0.504 Heath 326 0.825 0.857 0.841 0.601 Bracken 285 0.877 0.896 0.887 0.649 Semi-natural grassland 370 0.611 0.828 0.703 0.327 Improved grassland 282 0.901 0.646 0.753 0.541 Broadleaf woodland 316 0.892 0.928 0.910 0.660 Conifer woodland 120 0.900 0.837 0.867 0.805 Water 138 0.957 1.000 0.978 0.891 Built-up and bare 362 0.970 0.951 0.960 0.697 overall accuracy 0.853 (block-bootstrap 95% CI 0.801 to 0.892); macro F1 0.862; kappa 0.833 === Step 5: feature importance (gain, TreeSHAP, grouped permutation) === share of total gain and of total mean |SHAP| by feature group: gain shap S2 red-edge bands 0.257 0.202 S2 SWIR 0.127 0.116 S2 visible 0.099 0.105 S1 VH 0.102 0.104 Red-edge indices 0.094 0.098 S1 VV 0.059 0.085 Elevation 0.068 0.080 NDWI 0.043 0.045 NDVI 0.038 0.045 S2 NIR 0.056 0.036 NDMI 0.027 0.033 Slope 0.017 0.026 S1 VH/VV ratio 0.013 0.026 top 12 features by mean |SHAP| (summed over classes): elevation 2.395 vh_spring 1.109 B06_spring 0.861 vv_summer 0.834 B11_summer 0.830 slope 0.789 vh_winter 0.784 B11_spring 0.775 vv_spring 0.764 B12_summer 0.673 B8A_spring 0.639 cire_autumn 0.619 top 8 features separating bog from heath (mean |SHAP(bog) - SHAP(heath)| on bog and heath test samples): elevation 0.830 B11_summer 0.247 cire_autumn 0.243 slope 0.225 B03_spring 0.208 B04_spring 0.205 cire_spring 0.193 ratio_spring 0.184 grouped permutation importance on the test blocks (drop in macro F1, 10 repeats): drop_macro_f1 sd n_features S2 red-edge bands 0.098 0.005 16 Elevation 0.058 0.007 1 S2 SWIR 0.049 0.002 8 S1 VH 0.023 0.004 5 S2 visible 0.019 0.003 12 Red-edge indices 0.018 0.002 8 S1 VV 0.012 0.003 4 NDWI 0.010 0.002 4 S2 NIR 0.007 0.001 4 Slope 0.004 0.001 1 NDMI 0.004 0.001 4 NDVI 0.001 0.003 6 S1 VH/VV ratio 0.001 0.003 4 === Step 6: wall-to-wall habitat map === predicted 5,625,000 pixels in 10 s; unclassified (no clear spring or summer view): 0.00% lightgbm_km2 part1_km2 living_england_km2 Blanket bog 86.2 131.0 90.5 Heath 122.9 109.9 151.5 Bracken 52.2 64.1 41.6 Semi-natural grassland 99.0 63.5 96.1 Improved grassland 97.1 101.9 90.0 Broadleaf woodland 44.0 35.4 39.7 Conifer woodland 16.6 17.3 17.2 Water 8.8 6.3 8.4 Built-up and bare 35.7 32.9 25.7 wall-to-wall agreement with Living England, LightGBM: test blocks 0.687, whole area 0.690 wall-to-wall agreement with Living England, Part 1 rules: test blocks 0.483, whole area 0.490 pixels with top-class probability 0.0-0.5: 9.0% of the map, agreement 0.342 pixels with top-class probability 0.5-0.8: 31.1% of the map, agreement 0.530 pixels with top-class probability 0.8-1.0: 59.9% of the map, agreement 0.826 saved 7 figures, habitat_lightgbm_2023.tif, the confidence GeoTIFF and metrics.json to output/ total runtime 3.0 min (features, tuning three feature sets, random forest, importance, map and figures)