AVSS2026 – UAV-Based Crop–Weed Field Monitoring with EMA-Guided Multi-Head Segmentation and Look-Twice Refinement

Minahil Fatima, Musaib Ahmed, Fezan Rasool, Murtaza Taj

Abstract:

Accurate plant phenotyping is critical for precision agriculture, supporting robust weed management, crop monitoring, and sustainable yield optimization. However, crop–weed semantic segmentation in field environments remains challenging due to severe class imbalance, overlapping plant structures, ambiguous boundaries, and the small, camouflaged nature of weeds.

We propose a hybrid deep learning framework for crop–weed segmentation on the PhenoBench dataset that integrates handcrafted vegetation priors with adaptive pseudo-label learning and boundary-aware refinement. The method extends a ResNet101-based architecture using a six-channel input comprising RGB, Contrast-Enhanced Index (CEI), Excess Red (ExR), and Sobel edge cues to enhance vegetation discrimination. To address weed sparsity and structural ambiguity, the framework incorporates weed-specific copy-paste augmentation, ENet-based class reweighting, and composite supervision combining Cross-Entropy, Dice, and Focal Loss.

Additionally, dual foreground–background auxiliary branches with orthogonality constraints are introduced, along with edge-aware supervision for improved boundary delineation. An Exponential Moving Average (EMA) teacher model enables Adaptive Pseudo-Mask (APM) refinement, inspired by dynamic pseudo-labeling. A Look-Twice refinement mechanism further enhances localization of small weed instances.

Experiments on PhenoBench demonstrate significant improvements over standard segmentation pipelines, achieving 88.33\% mIoU, 70.94\% weed IoU, and 82.84\% foreground mIoU.

Text Reference:

 M. Fatima, M. Ahmed, F. Rasool, and Murtaza Taj, "UAV-Based Crop--Weed Field Monitoring with EMA-Guided Multi-Head Segmentation and Look-Twice Refinement," in Proc. of the IEEE Int. Conf. on Advanced Visual and Signal-Based Systems (AVSS), 2026

Bibtex Reference:

@inproceedings{localizationlensMICCAI2025,
  author={M. Fatima, M. Ahmed, F. Rasool, and Murtaza Taj},
  title={UAV-Based Crop--Weed Field Monitoring with EMA-Guided Multi-Head Segmentation and Look-Twice Refinement},
  booktitle={Proc. of the IEEE Int. Conf. on Advanced Visual and Signal-Based Systems (AVSS)},
  year={2026},
}