hoi4d-leaderboard

Overview

In addition, we provide a benchmark suite that covers different aspects of the 4D understanding of human-object interaction. To ensure a fair evaluation of these tasks, we follow common best practices and use the results of the server-side evaluation test set.

We run three challenges that are developed based on the HOI4D dataset. Please see the corresponding homepage for specific descriptions of each challenge. Here, we will only provide short task descriptions and leaderboards.

Important: We do not approve accounts with email addresses from free email providers, such as gmail.com, qq.com, web.de, etc. Only university or company email addresses can be accessed. If you need to use a free email account, please contact us.

4D Semantic Segmentation

Task

In the semantic segmentation of 4D point clouds, we want to infer the semantic label of each 3D point. Therefore, the input of all evaluated methods is a list of coordinates of 3D points. E-ach method should then output a label for each point of the scan.

Metric

We use mean Jaccard or so-called intersection-over-union (mIoU) over all classes, i.e.,

where TPc , FPc, and FNc correspond to the number of true positive, false positive, and false negative predictions for class c, and C is the number of classes.

Leaderboard

Approach	Institution	mloU
Enhanced Point TransformerV2	Chinese University of Hong Kong(Shenzhen)	48.0
PPTr	IIIS, Tsinghua University	41.0
P4Transformer	National University of Singapore	40.1
To evaluate your results, please send the pred.npy file to liuyzchina@gmail.com or yunzeliu77@163.com.

submit

4D Action Segmentation

Task

In this task, we need to give each frame of the point cloud in the point cloud video an action category label. The task’s input is a point cloud video and the output is the action described in each frame of this video.

Task

The following three metrics are reported: framewise accuracy (Acc), segmental edit distance, as well as segmental F1 scores at the overlapping thresholds of 10%, 25%, and 50%. Overlapping thresholds are determined by the IoU ratio.

Leaderboard

The following leaderboard contains only published approaches, where we at least can provide an arXiv link.

Approach	Institution	Acc	Edit	Details
alisa_24	ZJU	0.8524140508221226	87.82051441704616	F1@10 89.7966647162083 F1@25 88.31187829139849 F1@50 82.95055588063195
alisa_25	ZJU	0.852406576980568	87.81265092940069	F1@10 89.80323312120547 F1@25 88.31833808792334 F1@50 82.9566235096189
XD-Transformer	SH ailab	0.8522496263079222	91.38732622245608	F1@10 92.64838806869537 F1@25 91.04398915335945 F1@50 85.42482675504671
XD-Transformer	ailab	0.852219730941704	91.17519481582283	F1@10 92.49361957664016 F1@25 90.96231797027474 F1@50 84.88214982735325
alisa_29	ZJU	0.851831091180867	87.97478083010287	F1@10 89.9556272690601 F1@25 88.37141075947046 F1@50 82.86332465436944
HexFormer	PKU	0.8518086696562033	88.94718917874394	F1@10 90.09525911214095 F1@25 88.52844900511236 F1@50 82.996800176542
alisa_14	ZJU	0.851136023916293	88.07147602447434	F1@10 89.9083241657499 F1@25 88.32416574990832 F1@50 82.76494316098277
panda	ZJU	0.851136023916293	88.07147602447434	F1@10 89.9083241657499 F1@25 88.32416574990832 F1@50 82.76494316098277
alisa_27	ZJU	0.8511210762331839	88.07147602447434	F1@10 89.9083241657499 F1@25 88.32416574990832 F1@50 82.76494316098277
alisa_26	ZJU	0.8509865470852018	88.23664347531256	F1@10 90.05143277002205 F1@25 88.47171197648788 F1@50 82.92432035268186
alisa_7	ZJU	0.8509043348281017	87.93221777671194	F1@10 89.97029594044518 F1@25 88.5107631376288 F1@50 82.87065899006197
alisa_9	ZJU	0.8509043348281017	87.93221777671194	F1@10 89.97029594044518 F1@25 88.5107631376288 F1@50 82.87065899006197
alisa_29	ZJU	0.8508221225710015	88.26772105034811	F1@10 90.11514549534635 F1@25 88.53327447301623 F1@50 83.00040466468013
alisa_28	ZJU	0.8504783258594918	88.22079506901783	F1@10 90.16507959851465 F1@25 88.5473730651862 F1@50 82.95893231368798
alisa_30	ZJU	0.8492077727952168	87.75859590370912	F1@10 89.76314438659529 F1@25 88.23788223216252 F1@50 82.67947495783531
alisa_31	ZJU	0.8491405082212257	88.19816637961415	F1@10 90.2764467379285 F1@25 88.66936970143753 F1@50 82.83818650939921
alisa_21	ZJU	0.8480269058295964	88.20078933704173	F1@10 90.078136517765 F1@25 88.47117794486216 F1@50 82.78785198289843
alisa_19	ZJU	0.8469506726457399	87.36048862585304	F1@10 89.39997806135507 F1@25 87.78383121869173 F1@50 82.04321913049837
alisa_22	ZJU	0.8468385650224215	86.99802585040757	F1@10 89.19214770488215 F1@25 87.54287382325037 F1@50 81.6390571407721
alisa_23	ZJU	0.8467713004484305	87.15078206249964	F1@10 89.34024987214144 F1@25 87.63790458098926 F1@50 81.72718638123769
alisa_16	ZJU	0.8465097159940209	88.22798408864078	F1@10 90.1513473606497 F1@25 88.62310815799187 F1@50 82.59874492432631
alisa_17	ZJU	0.8462331838565023	86.93902979158919	F1@10 89.07269083621789 F1@25 87.4767717252687 F1@50 81.5886318090727
alisa_20	ZJU	0.8461061285500747	87.62795308884864	F1@10 89.61067526944791 F1@25 87.91700271280885 F1@50 82.02214238580541
alisa_18	ZJU	0.8460762331838565	86.88416127905313	F1@10 88.95846226655503 F1@25 87.3056900506025 F1@50 81.3571662601478
alisa_15	ZJU	0.8460463378176383	85.61009813281167	F1@10 88.04637430690573 F1@25 86.43335493627133 F1@50 80.77338518038454
alisa_11	ZJU	0.8444095665171898	86.712727305296	F1@10 88.83154642922044 F1@25 87.21788115573324 F1@50 81.36652734871889
alisa_12	ZJU	0.8442899850523169	86.92284630931188	F1@10 88.90992499817956 F1@25 87.30794436758174 F1@50 81.48984198645599
alisa_13	ZJU	0.8437817638266069	86.837199868755	F1@10 88.91073248871415 F1@25 87.2724625018203 F1@50 81.36740934906072
Multi-Conv-Res7	Dalian University of Technology	0.8436920777279522	86.56866819963777	F1@10 88.9527690852194 F1@25 86.99512408121679 F1@50 80.70009460737937
alisa_10	ZJU	0.8410762331838565	84.30390405384944	F1@10 86.89223770258742 F1@25 85.02274665908445 F1@50 79.30054023315327
cos_version26	ailab	0.8406053811659193	91.05057053069882	F1@10 92.50438897794061 F1@25 90.85565987329211 F1@50 84.84085184337073
cos_version29	ailab	0.8405979073243647	91.0819170320088	F1@10 92.52090253121062 F1@25 90.87160691787884 F1@50 84.84709655251403
SAT_Merge_v1	SH ailab	0.8405979073243647	91.0779665729014	F1@10 92.52978918423464 F1@25 90.88756492514514 F1@50 84.86862205927285
cos_version28	ailab	0.8405904334828102	91.07390934462893	F1@10 92.5173703901657 F1@25 90.8681377414675 F1@50 84.84385737191724
cos_version19	ailab	0.8405829596412556	90.93773806749792	F1@10 92.42499332850443 F1@25 90.75521329724371 F1@50 84.76230414395182
cos_version24	ailab	0.8405680119581465	90.96080366526259	F1@10 92.4561403508772 F1@25 90.79328756674295 F1@50 84.79023646071701
Multi-Conv-Res5	Dalian University of Technology	0.8405530642750374	85.57560849851167	F1@10 88.17794468115837 F1@25 86.28583808767243 F1@50 79.55513829710408
cos_version23	ailab	0.8405306427503737	90.91913658434308	F1@10 92.41915802318488 F1@25 90.75655887736424 F1@50 84.77730323367908
cos_version23	ailab	0.8405306427503737	90.91913658434308	F1@10 92.41915802318488 F1@25 90.75655887736424 F1@50 84.77730323367908
SAT_MERGE	SH ailab	0.8404783258594918	91.05041810266165	F1@10 92.64717169391358 F1@25 91.05840816639802 F1@50 85.3020185739504
SAT_Merge_v2	SH ailab	0.8404783258594918	91.05041810266165	F1@10 92.64717169391358 F1@25 91.05840816639802 F1@50 85.3020185739504
SAT_Merge_v2	SH ailab	0.8404783258594918	91.05041810266165	F1@10 92.64717169391358 F1@25 91.05840816639802 F1@50 85.3020185739504
Sat_Merge_v3	SH ailab	0.8404783258594918	91.05041810266165	F1@10 92.64717169391358 F1@25 91.05840816639802 F1@50 85.3020185739504
Multi-Conv-Res6	Dalian University of Technology	0.8385201793721974	85.79122676098416	F1@10 88.20976491862568 F1@25 86.25678119349006 F1@50 79.81916817359856
cos_version21	ailab	0.8376905829596413	91.11819971137608	F1@10 92.34404176670095 F1@25 90.8044662932053 F1@50 84.53183948782439
alisa_8	ZJU	0.8372720478325859	80.6485018098239	F1@10 83.35894546323806 F1@25 81.41242359047911 F1@50 74.99914626233651
Multi-Conv-Res8	Dalian University of Technology	0.8368385650224215	88.65632600109427	F1@10 90.21763351407562 F1@25 88.2418480279284 F1@50 81.44544306618138
cos_version18	ailab	0.8350896860986547	91.07177872421552	F1@10 92.18482601254992 F1@25 90.47347404449516 F1@50 84.04639665335615
cos_version21	ailab	0.8350896860986547	91.07177872421552	F1@10 92.18482601254992 F1@25 90.47347404449516 F1@50 84.04639665335615
X4D-SceneFormer	No Disclosure	0.8322496263079223	90.62816999024744	F1@10 91.47117520804309 F1@25 89.61102534171782 F1@50 82.95364687276424

submit

Category-Level Object and Part Pose Tracking

Task

In this task, the input is a point cloud video, and given the pose of the object in the first frame, we track this object and give the pose of the object in every frame thereafter. Note that we are referring to the category-level object poses.

Task

The following metrics are used: 5°5cm: percentage of estimates with orientation error <5°and translation error <5cm. Rerr: mean orientation error in degrees. Terr: mean translation error in centimeters.

Leaderboard

The following leaderboard contains only published approaches, where we at least can provide an arXiv link.

Approach	Paper	Code	Institution	Details

submit