3D Instance Segmentation/ IEEE TCSVT 2026

LaSSM Efficient Semantic-Spatial Query Decoding via Local Aggregation and State Space Models for 3D Instance Segmentation

Lei Yao¹, Yi Wang¹, Yawen Cui¹, Moyun Liu², Lap-Pui Chau¹

¹The Hong Kong Polytechnic University · ²Huazhong University of Science and Technology

TL;DR

We pick queries from superpoints using both semantic and spatial cues, then refine them with a coordinate-guided state space model instead of attention — first place on ScanNet++ V2 at one third of the FLOPs of the previous best.

ScanNet++ V2 Test 32.4mAP Ranked 1st on the public leaderboard

Over previous best +2.5mAP against SGIFormer (29.9)

Compute ⅓FLOPs 4.78G vs 13.51G

Challenge 1st place CVPR 2025 ScanNet++ challenge

Query distribution comparison: input point cloud, FPS sampling, semantic selection, and LaSSM. LaSSM covers regions that FPS misses. — **Query distribution.** Farthest point sampling scatters queries over empty floor (red box), while purely semantic selection collapses onto dominant surfaces (blue box). LaSSM combines both cues, covering the scene comprehensively and converging faster.

Abstract

Query-based 3D instance segmentation is held back on two fronts: sparse point clouds make query initialization difficult, and query decoders lean on computationally intensive attention. LaSSM addresses both, prioritising simplicity and efficiency. A hierarchical semantic-spatial query initializer derives queries from superpoints using both semantic cues and spatial distribution, covering the scene comprehensively and converging faster.

A coordinate-guided SSM decoder then refines them: local aggregation keeps each query within a geometrically coherent region, and a spatial dual-path SSM block captures dependencies across the query set through the associated coordinates. LaSSM ranks first on the ScanNet++ V2 leaderboard, beating the previous best by 2.5% mAP with only 1/3 FLOPs, and stays competitive on ScanNet, ScanNet200, S3DIS and ScanNet++ V1 at lower cost.

3D Instance Segmentation State Space Models Query Decoding Superpoints Efficiency

Method

LaSSM architecture: a backbone extracts point features, superpoint pooling forms features and coordinates, a hierarchical semantic-spatial initializer selects queries, and a coordinate-guided SSM decoder refines them into instance predictions. — **Overall architecture.** Point features are pooled into superpoints, from which the initializer draws a query set using both semantic activation and spatial coverage. The coordinate-guided SSM decoder then refines those queries into instance masks — without any global attention over the scene.

Component 01

Hierarchical Semantic-Spatial Query Initializer

Queries are drawn from superpoints by jointly weighing a semantic activation score and spatial distribution, instead of relying on farthest point sampling or semantics alone. This resolves the query initialization dilemma: the scene is covered comprehensively, and training converges faster.

Component 02

Coordinate-guided SSM Query Decoder

A local aggregation scheme confines each query to a geometrically coherent neighbourhood, and a spatial dual-path SSM block propagates dependencies across the query set using the associated coordinates — linear-time refinement in place of quadratic attention.

Results

Accuracy versus computation

SCANNET++ V2 · AP₅₀ vs FLOPs

LaSSM (ours) Previous best (SGIFormer) OneFormer3D runs out of memory on ScanNet++ V2

ScanNet++ V2 *— hidden test split, with validation AP₅₀ for reference*
Method	Val AP₅₀	Test mAP	Test AP₅₀	Test AP₂₅
PointGroup	14.7	9.1	15.2	21.0
SoftGroup	–	14.2	23.4	29.3
PTv3-PG	–	25.9	38.7	44.6
OneFormer3D	41.1	28.2	43.3	51.7
SPFormer	42.1	29.0	43.5	51.5
SGIFormer	41.1	29.9	45.7	54.4
LaSSM (ours)	43.5	32.4	48.0	54.8

Efficiency *— same backbone, measured under identical settings*
Method	ScanNet V2 AP₅₀	FLOPs	ScanNet++ V2 AP₅₀	FLOPs	Params	Time
OneFormer3D	76.3	4.082G	41.1	OOM	–	–
SGIFormer	78.4	4.456G	41.1	13.513G	15.96M	355.86ms
LaSSM (ours)	78.1	3.711G	43.5	4.777G	18.38M	379.94ms

Released checkpoints *— all trained on 4 GPUs*
Benchmark	mAP	AP₅₀	AP₂₅	Resources
ScanNet++ V2 Val	29.1	43.5	51.6	config log model
ScanNet Val	58.4	78.1	86.1	config log model
ScanNet200 Val	29.3	39.2	44.5	config log model

ScanNet++ V2 hidden test leaderboard showing LaSSM at the first position. — **ScanNet++ V2 leaderboard.** First place on the hidden test set (accessed 21 May 2025), and first place at the CVPR 2025 ScanNet++ challenge.

Qualitative Results

Qualitative instance segmentation results on the ScanNet V2 validation set compared with ground truth. — **ScanNet V2.** Instance predictions against ground truth on the validation set.

Qualitative instance segmentation results on large ScanNet++ V2 scenes. — **ScanNet++ V2.** Large, cluttered scenes with long-tail categories — where the efficiency of the decoder matters most.

Citation

@article{yao2025lassm,
  title={LaSSM: Efficient Semantic-Spatial Query Decoding via Local
         Aggregation and State Space Models for 3D Instance Segmentation},
  author={Yao, Lei and Wang, Yi and Cui, Yawen and Liu, Moyun and Chau, Lap-Pui},
  journal={IEEE Transactions on Circuits and Systems for Video Technology},
  year={2026},
}