Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fix](JDK17) Start Fe with JDK17 #30485

Merged
merged 1 commit into from Jan 29, 2024
Merged

Conversation

BePPPower
Copy link
Contributor

@BePPPower BePPPower commented Jan 29, 2024

Proposed changes

Issue Number: #30484

problem:
gson will use Java's reflection mechanism to generate a default Adapter, but JDK17 is prohibited from visiting such an access.

solution:
gson has provided solutions since 2.9.1, which can bypass this problem: Add support for reflection access filter by Marcono1234 · Pull Request #1905 · google/gson

We need to upgrade the gson version and use this solution

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@BePPPower
Copy link
Contributor Author

run buildall

@BePPPower BePPPower mentioned this pull request Jan 29, 2024
7 tasks
@doris-robot
Copy link

TPC-H: Total hot run time: 37894 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8d40209e3607a77c7a4e92784093368749f3de86, data reload: false

------ Round 1 ----------------------------------
q1	17896	5301	5039	5039
q2	2670	142	135	135
q3	11223	945	934	934
q4	4958	728	737	728
q5	7851	3037	3003	3003
q6	190	119	120	119
q7	1150	727	716	716
q8	9316	2015	2034	2015
q9	7236	6316	6295	6295
q10	8133	2414	2441	2414
q11	417	201	205	201
q12	762	277	293	277
q13	18005	3294	3273	3273
q14	285	244	236	236
q15	514	499	488	488
q16	454	417	413	413
q17	1005	509	480	480
q18	6969	6038	6281	6038
q19	1548	1410	1380	1380
q20	635	331	350	331
q21	6684	3128	3080	3080
q22	818	306	299	299
Total cold run time: 108719 ms
Total hot run time: 37894 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5122	5026	5061	5026
q2	340	245	227	227
q3	3008	2653	2721	2653
q4	1816	1563	1580	1563
q5	5466	5192	5156	5156
q6	207	115	115	115
q7	2104	1839	1842	1839
q8	3094	3197	3182	3182
q9	8602	8514	13554	8514
q10	5487	3446	3366	3366
q11	553	456	470	456
q12	743	577	604	577
q13	16969	3083	3100	3083
q14	276	270	266	266
q15	534	492	490	490
q16	525	455	471	455
q17	2033	1652	1785	1652
q18	8976	14062	9056	9056
q19	26469	1571	1517	1517
q20	4582	1904	1909	1904
q21	16114	4794	4768	4768
q22	1413	457	474	457
Total cold run time: 114433 ms
Total hot run time: 56322 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173702 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8d40209e3607a77c7a4e92784093368749f3de86, data reload: false

query1	951	326	321	321
query2	6563	2077	1932	1932
query3	6734	202	195	195
query4	35777	22153	22150	22150
query5	4454	348	412	348
query6	246	160	159	159
query7	4605	267	268	267
query8	241	173	170	170
query9	8557	2269	2253	2253
query10	414	208	207	207
query11	17863	15440	15466	15440
query12	125	69	62	62
query13	1685	369	382	369
query14	9192	6541	6582	6541
query15	216	178	184	178
query16	5699	251	251	251
query17	953	479	459	459
query18	1798	257	245	245
query19	178	131	130	130
query20	79	77	60	60
query21	205	123	128	123
query22	4985	4847	4806	4806
query23	31103	30365	30316	30316
query24	11626	2764	2762	2762
query25	591	315	307	307
query26	1833	137	140	137
query27	3189	281	278	278
query28	7756	1815	1810	1810
query29	1821	613	596	596
query30	278	135	141	135
query31	889	704	714	704
query32	72	52	48	48
query33	696	210	210	210
query34	1179	464	474	464
query35	856	748	748	748
query36	1297	1197	1195	1195
query37	91	61	56	56
query38	3283	3205	3165	3165
query39	1306	1255	1235	1235
query40	340	85	82	82
query41	37	35	34	34
query42	86	95	78	78
query43	528	504	490	490
query44	1055	678	687	678
query45	189	179	174	174
query46	1057	644	636	636
query47	1594	1523	1430	1430
query48	374	305	307	305
query49	1213	275	278	275
query50	658	306	303	303
query51	5335	5160	5183	5160
query52	92	77	77	77
query53	338	262	270	262
query54	266	184	199	184
query55	84	77	74	74
query56	178	162	162	162
query57	987	935	905	905
query58	190	151	158	151
query59	2951	2616	2606	2606
query60	208	186	172	172
query61	88	92	85	85
query62	653	348	356	348
query63	302	254	268	254
query64	5880	1765	1714	1714
query65	3273	3225	3236	3225
query66	1250	323	300	300
query67	14380	14524	14570	14524
query68	4004	489	481	481
query69	430	295	299	295
query70	1459	1484	1472	1472
query71	276	206	208	206
query72	3800	2867	2806	2806
query73	674	305	313	305
query74	6654	6245	6229	6229
query75	2802	2283	2311	2283
query76	2611	1040	1081	1040
query77	356	234	225	225
query78	9075	8877	8501	8501
query79	2396	491	486	486
query80	1935	320	321	320
query81	506	195	197	195
query82	702	83	82	82
query83	237	120	113	113
query84	286	79	78	78
query85	2150	336	331	331
query86	508	423	371	371
query87	3467	3311	3270	3270
query88	3762	2150	2142	2142
query89	428	361	364	361
query90	1946	184	183	183
query91	165	119	118	118
query92	55	45	43	43
query93	2211	415	428	415
query94	1382	155	154	154
query95	491	456	450	450
query96	612	315	306	306
query97	4231	4121	4087	4087
query98	220	191	181	181
query99	1046	660	653	653
Total cold run time: 283849 ms
Total hot run time: 173702 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.12 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8d40209e3607a77c7a4e92784093368749f3de86, data reload: false

query1	0.04	0.02	0.03
query2	0.06	0.03	0.02
query3	0.23	0.06	0.06
query4	1.68	0.09	0.10
query5	0.53	0.51	0.51
query6	1.20	0.62	0.64
query7	0.02	0.01	0.01
query8	0.04	0.02	0.02
query9	0.54	0.51	0.50
query10	0.55	0.55	0.57
query11	0.12	0.08	0.10
query12	0.10	0.09	0.09
query13	0.59	0.60	0.60
query14	0.78	0.78	0.81
query15	0.78	0.79	0.77
query16	0.38	0.39	0.39
query17	0.96	1.01	1.04
query18	0.22	0.28	0.26
query19	1.95	1.77	1.79
query20	0.01	0.01	0.01
query21	15.43	0.54	0.55
query22	2.45	2.72	1.59
query23	17.38	0.75	0.79
query24	2.43	1.16	1.37
query25	0.35	0.22	0.20
query26	0.54	0.14	0.13
query27	0.05	0.05	0.05
query28	10.71	0.85	0.84
query29	12.55	3.16	3.19
query30	0.64	0.60	0.58
query31	2.79	0.34	0.35
query32	3.38	0.47	0.48
query33	3.26	3.19	3.21
query34	15.83	4.35	4.27
query35	4.33	4.45	4.34
query36	1.10	1.07	1.07
query37	0.06	0.05	0.05
query38	0.04	0.03	0.03
query39	0.02	0.02	0.02
query40	0.16	0.13	0.13
query41	0.07	0.02	0.02
query42	0.02	0.01	0.01
query43	0.03	0.02	0.02
Total cold run time: 104.4 s
Total hot run time: 31.12 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 8d40209e3607a77c7a4e92784093368749f3de86 with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       13.4 seconds inserted 10000000 Rows, about 746K ops/s

Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 29, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

Copy link
Contributor

@kaka11chen kaka11chen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@morningman morningman merged commit 9d2780f into apache:master Jan 29, 2024
30 of 32 checks passed
yiguolei pushed a commit that referenced this pull request Jan 30, 2024
Issue Number: close #30484

problem:
gson will use Java's reflection mechanism to generate a default Adapter, but JDK17 is prohibited from visiting such an access.

solution:
gson has provided solutions since 2.9.1, which can bypass this problem: Add support for reflection access filter by Marcono1234 · Pull Request #1905 · google/gson

We need to upgrade the gson version and use this solution
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants