hoi4d-leaderboard

Overview

In addition, we provide a benchmark suite that covers different aspects of the 4D understanding of human-object interaction. To ensure a fair evaluation of these tasks, we follow common best practices and use the results of the server-side evaluation test set.

We run three challenges that are developed based on the HOI4D dataset. Please see the corresponding homepage for specific descriptions of each challenge. Here, we will only provide short task descriptions and leaderboards.

Important: We do not approve accounts with email addresses from free email providers, such as gmail.com, qq.com, web.de, etc. Only university or company email addresses can be accessed. If you need to use a free email account, please contact us.

4D Semantic Segmentation

Task

In the semantic segmentation of 4D point clouds, we want to infer the semantic label of each 3D point. Therefore, the input of all evaluated methods is a list of coordinates of 3D points. E-ach method should then output a label for each point of the scan.

Metric

We use mean Jaccard or so-called intersection-over-union (mIoU) over all classes, i.e.,

where TPc , FPc, and FNc correspond to the number of true positive, false positive, and false negative predictions for class c, and C is the number of classes.

Leaderboard

Approach	Institution	mloU
Enhanced Point TransformerV2	Chinese University of Hong Kong(Shenzhen)	48.0
PPTr	IIIS, Tsinghua University	41.0
P4Transformer	National University of Singapore	40.1
To evaluate your results, please send the pred.npy file to liuyzchina@gmail.com or yunzeliu77@163.com.

submit

4D Action Segmentation

Task

In this task, we need to give each frame of the point cloud in the point cloud video an action category label. The task’s input is a point cloud video and the output is the action described in each frame of this video.

Task

The following three metrics are reported: framewise accuracy (Acc), segmental edit distance, as well as segmental F1 scores at the overlapping thresholds of 10%, 25%, and 50%. Overlapping thresholds are determined by the IoU ratio.

Leaderboard

The following leaderboard contains only published approaches, where we at least can provide an arXiv link.

Approach	Institution	Acc	Edit	Details
test123	test123	0.19133781763826607	6.979134269346354	F1@10 3.675145924911724 F1@25 0.05764934784175254 F1@50 0.0
test111	test111	0.19133781763826607	6.979134269346354	F1@10 3.675145924911724 F1@25 0.05764934784175254 F1@50 0.0
P4Tr	personal	0.7152914798206278	71.30564963523261	F1@10 72.63883799180464 F1@25 67.88153379751473 F1@50 56.27477762601193
PPTr	personal	0.7850149476831091	80.40452258471214	F1@10 81.90115136197697 F1@25 78.46110643077787 F1@50 69.34147711317044
test	test	0.31198056801195817	34.140717851121174	F1@10 28.143416600129278 F1@25 18.949849815596366 F1@50 13.208623246264404
test	test	0.31198056801195817	34.140717851121174	F1@10 28.143416600129278 F1@25 18.949849815596366 F1@50 13.208623246264404
XD-Transformer	ailab	0.852219730941704	91.17519481582283	F1@10 92.49361957664016 F1@25 90.96231797027474 F1@50 84.88214982735325
XD_Transformer	111	0.8322496263079223	90.62816999024744	F1@10 91.47117520804309 F1@25 89.61102534171782 F1@50 82.95364687276424
X4D-SceneFormer	No Disclosure	0.8322496263079223	90.62816999024744	F1@10 91.47117520804309 F1@25 89.61102534171782 F1@50 82.95364687276424
XD-Transformer	SH ailab	0.8522496263079222	91.38732622245608	F1@10 92.64838806869537 F1@25 91.04398915335945 F1@50 85.42482675504671
alisa_31	ZJU	0.8491405082212257	88.19816637961415	F1@10 90.2764467379285 F1@25 88.66936970143753 F1@50 82.83818650939921
alisa_30	ZJU	0.8492077727952168	87.75859590370912	F1@10 89.76314438659529 F1@25 88.23788223216252 F1@50 82.67947495783531
test_c_tk7	test	0.8092974588938715	76.67340242967721	F1@10 80.03180598350065 F1@25 77.97766954908393 F1@50 71.0399893980055
test_c_tk5	test	0.8061733931240658	73.95311308346066	F1@10 77.48916358966126 F1@25 75.23519023920372 F1@50 68.06871086851821
Sat_Merge_v3	SH ailab	0.8404783258594918	91.05041810266165	F1@10 92.64717169391358 F1@25 91.05840816639802 F1@50 85.3020185739504
SAT_Merge_v2	SH ailab	0.8404783258594918	91.05041810266165	F1@10 92.64717169391358 F1@25 91.05840816639802 F1@50 85.3020185739504
SAT_Merge_v2	SH ailab	0.8404783258594918	91.05041810266165	F1@10 92.64717169391358 F1@25 91.05840816639802 F1@50 85.3020185739504
SAT_Merge_v1	SH ailab	0.8405979073243647	91.0779665729014	F1@10 92.52978918423464 F1@25 90.88756492514514 F1@50 84.86862205927285
SAT_MERGE	SH ailab	0.8404783258594918	91.05041810266165	F1@10 92.64717169391358 F1@25 91.05840816639802 F1@50 85.3020185739504
cos_version29	ailab	0.8405979073243647	91.0819170320088	F1@10 92.52090253121062 F1@25 90.87160691787884 F1@50 84.84709655251403
cos_version28	ailab	0.8405904334828102	91.07390934462893	F1@10 92.5173703901657 F1@25 90.8681377414675 F1@50 84.84385737191724
alisa_29	ZJU	0.8508221225710015	88.26772105034811	F1@10 90.11514549534635 F1@25 88.53327447301623 F1@50 83.00040466468013
cos_version26	ailab	0.8406053811659193	91.05057053069882	F1@10 92.50438897794061 F1@25 90.85565987329211 F1@50 84.84085184337073
Multi-Conv-Res_final	Dalian University of Technology	0.8290059790732437	87.41627070424364	F1@10 88.96161200176498 F1@25 86.5568465950875 F1@50 79.34990439770556
0607	SJTU	0.8196038863976084	90.09866119757815	F1@10 91.27258746635908 F1@25 89.23490965013457 F1@50 81.93771626297578
cos_version24	ailab	0.8405680119581465	90.96080366526259	F1@10 92.4561403508772 F1@25 90.79328756674295 F1@50 84.79023646071701
Multi-Conv-Res11	Dalian University of Technology	0.8303811659192825	87.46054470065347	F1@10 89.11409198113208 F1@25 86.74823113207547 F1@50 79.77594339622642
Multi-Conv-Res0	Dalian University of Technology	0.8289536621823618	87.3393173540375	F1@10 88.88398277955625 F1@25 86.63207859587152 F1@50 79.58199948485853
2306061442	SJTU	0.4501420029895366	74.95797297667686	F1@10 59.99293369450006 F1@25 51.41914968790484 F1@50 32.30086758528638
test	SJTU	0.19133781763826607	6.979134269346354	F1@10 3.675145924911724 F1@25 0.05764934784175254 F1@50 0.0
Multi-Conv-Res10	Dalian University of Technology	0.8289536621823618	87.3393173540375	F1@10 88.88398277955625 F1@25 86.63207859587152 F1@50 79.58199948485853
cos_version23	ailab	0.8405306427503737	90.91913658434308	F1@10 92.41915802318488 F1@25 90.75655887736424 F1@50 84.77730323367908
cos_version23	ailab	0.8405306427503737	90.91913658434308	F1@10 92.41915802318488 F1@25 90.75655887736424 F1@50 84.77730323367908
2306061429	SJTU	0.7542750373692078	80.93867545414864	F1@10 82.50054549421777 F1@25 79.38759182486 F1@50 69.14684704342135
alisa_29	ZJU	0.851831091180867	87.97478083010287	F1@10 89.9556272690601 F1@25 88.37141075947046 F1@50 82.86332465436944
test	SJTU	0.7542750373692078	80.93867545414864	F1@10 82.50054549421777 F1@25 79.38759182486 F1@50 69.14684704342135
Multi-Conv-Res10	Dalian University of Technology	0.8289536621823618	87.3393173540375	F1@10 88.88398277955625 F1@25 86.63207859587152 F1@50 79.58199948485853
cos_version21	ailab	0.8376905829596413	91.11819971137608	F1@10 92.34404176670095 F1@25 90.8044662932053 F1@50 84.53183948782439
test	SJTU	0.4511733931240658	74.95797297667686	F1@10 59.19993718839556 F1@25 50.31994661013622 F1@50 31.429356573626972
cos_version21	ailab	0.8350896860986547	91.07177872421552	F1@10 92.18482601254992 F1@25 90.47347404449516 F1@50 84.04639665335615
alisa_28	ZJU	0.8504783258594918	88.22079506901783	F1@10 90.16507959851465 F1@25 88.5473730651862 F1@50 82.95893231368798
Multi-Conv-Res9	Dalian University of Technology	0.828034379671151	87.26804806477168	F1@10 88.98626505136797 F1@25 86.5559524247892 F1@50 79.03671244982877
alisa_27	ZJU	0.8511210762331839	88.07147602447434	F1@10 89.9083241657499 F1@25 88.32416574990832 F1@50 82.76494316098277
alisa_26	ZJU	0.8509865470852018	88.23664347531256	F1@10 90.05143277002205 F1@25 88.47171197648788 F1@50 82.92432035268186
Cos_version20	SH ailab	0.8291928251121077	91.03114864122064	F1@10 92.08289054197662 F1@25 90.07894337331106 F1@50 83.30803097009259
cos_version19	ailab	0.8405829596412556	90.93773806749792	F1@10 92.42499332850443 F1@25 90.75521329724371 F1@50 84.76230414395182
cos_version18	ailab	0.8350896860986547	91.07177872421552	F1@10 92.18482601254992 F1@25 90.47347404449516 F1@50 84.04639665335615
Cos_version17	ailab	0.8302989536621823	89.76397027640938	F1@10 91.09824840161917 F1@25 89.17640827753188 F1@50 82.7753187303749
cos_version16	ailab	0.8267189835575486	89.04176941701768	F1@10 90.63517298811418 F1@25 88.76523582405936 F1@50 81.7472935119994
panda	ZJU	0.851136023916293	88.07147602447434	F1@10 89.9083241657499 F1@25 88.32416574990832 F1@50 82.76494316098277

submit

Category-Level Object and Part Pose Tracking

Task

In this task, the input is a point cloud video, and given the pose of the object in the first frame, we track this object and give the pose of the object in every frame thereafter. Note that we are referring to the category-level object poses.

Task

The following metrics are used: 5°5cm: percentage of estimates with orientation error <5°and translation error <5cm. Rerr: mean orientation error in degrees. Terr: mean translation error in centimeters.

Leaderboard

The following leaderboard contains only published approaches, where we at least can provide an arXiv link.

Approach	Paper	Code	Institution	Details

submit