Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[load](cloud) Add FE config for streamload redirect policy: streamload_redirect_policy #35031

Merged

Conversation

gavinchou
Copy link
Collaborator

No description provided.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels May 18, 2024
Copy link
Contributor

PR approved by anyone and no changes requested.

@dataroaring
Copy link
Contributor

run buildall

@doris-robot
Copy link

TPC-DS: Total hot run time: 188837 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8ff9938559110442ebd6ae9cb132b67fb8a9d876, data reload: false

query1	936	396	363	363
query2	6296	2482	2388	2388
query3	6647	215	220	215
query4	23479	21970	21995	21970
query5	3763	430	413	413
query6	265	189	177	177
query7	4541	291	296	291
query8	240	185	194	185
query9	8553	2432	2462	2432
query10	408	244	246	244
query11	15292	14808	14764	14764
query12	120	96	91	91
query13	1597	362	363	362
query14	10386	7801	7744	7744
query15	258	166	173	166
query16	8096	261	263	261
query17	1683	591	560	560
query18	2077	272	273	272
query19	193	155	150	150
query20	90	83	86	83
query21	195	129	122	122
query22	5178	4904	4856	4856
query23	34307	33512	33436	33436
query24	10876	2943	2935	2935
query25	618	407	394	394
query26	1127	159	155	155
query27	2897	320	317	317
query28	7554	2021	2005	2005
query29	879	629	624	624
query30	275	178	171	171
query31	958	743	743	743
query32	97	51	52	51
query33	743	249	248	248
query34	1026	485	472	472
query35	795	664	664	664
query36	1075	942	938	938
query37	134	70	70	70
query38	2901	2799	2770	2770
query39	1631	1576	1591	1576
query40	198	128	127	127
query41	46	44	46	44
query42	103	104	103	103
query43	591	532	532	532
query44	1172	724	736	724
query45	266	259	255	255
query46	1082	728	768	728
query47	1973	1897	1871	1871
query48	386	302	295	295
query49	905	407	395	395
query50	756	396	379	379
query51	6956	6731	6836	6731
query52	107	89	92	89
query53	352	292	278	278
query54	850	427	430	427
query55	78	74	77	74
query56	247	225	228	225
query57	1234	1153	1170	1153
query58	226	204	214	204
query59	3385	3173	3251	3173
query60	273	243	245	243
query61	108	100	101	100
query62	673	463	473	463
query63	311	285	284	284
query64	8862	7410	7401	7401
query65	3124	3107	3063	3063
query66	908	330	333	330
query67	15585	15112	15005	15005
query68	10063	545	537	537
query69	602	313	304	304
query70	1378	1044	1111	1044
query71	524	267	265	265
query72	8667	2559	2418	2418
query73	1580	319	314	314
query74	6676	6153	6089	6089
query75	5505	2639	2631	2631
query76	5468	1025	947	947
query77	670	267	260	260
query78	10925	10187	10174	10174
query79	8748	522	521	521
query80	1257	436	422	422
query81	511	243	247	243
query82	235	91	94	91
query83	194	163	163	163
query84	264	85	81	81
query85	969	274	262	262
query86	351	299	317	299
query87	3259	3110	3057	3057
query88	4731	2318	2321	2318
query89	516	398	378	378
query90	2121	189	189	189
query91	125	96	97	96
query92	57	47	49	47
query93	6053	503	491	491
query94	1307	179	180	179
query95	382	299	292	292
query96	602	266	269	266
query97	3154	2960	3017	2960
query98	236	226	218	218
query99	1150	910	907	907
Total cold run time: 307223 ms
Total hot run time: 188837 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.7 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8ff9938559110442ebd6ae9cb132b67fb8a9d876, data reload: false

query1	0.04	0.03	0.04
query2	0.07	0.04	0.04
query3	0.23	0.05	0.05
query4	1.66	0.09	0.10
query5	0.50	0.49	0.54
query6	1.13	0.72	0.72
query7	0.02	0.01	0.01
query8	0.06	0.04	0.04
query9	0.54	0.49	0.48
query10	0.53	0.54	0.54
query11	0.15	0.12	0.12
query12	0.14	0.12	0.12
query13	0.60	0.59	0.60
query14	0.77	0.79	0.78
query15	0.83	0.82	0.80
query16	0.36	0.37	0.37
query17	1.02	1.02	1.02
query18	0.20	0.24	0.25
query19	1.80	1.65	1.73
query20	0.02	0.01	0.01
query21	15.66	0.70	0.67
query22	4.29	7.18	2.06
query23	18.31	1.39	1.25
query24	1.30	0.41	0.25
query25	0.15	0.08	0.07
query26	0.26	0.16	0.17
query27	0.08	0.07	0.07
query28	13.30	1.01	1.00
query29	13.75	3.29	3.29
query30	0.23	0.05	0.06
query31	2.86	0.41	0.40
query32	3.26	0.47	0.46
query33	2.79	2.84	2.86
query34	17.09	4.40	4.43
query35	4.47	4.52	4.62
query36	0.64	0.45	0.49
query37	0.18	0.16	0.14
query38	0.17	0.15	0.14
query39	0.05	0.03	0.04
query40	0.16	0.14	0.14
query41	0.08	0.05	0.05
query42	0.05	0.04	0.05
query43	0.04	0.03	0.03
Total cold run time: 109.84 s
Total hot run time: 30.7 s

@shuke987
Copy link
Collaborator

run buildall

@doris-robot
Copy link

TPC-DS: Total hot run time: 183155 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8ff9938559110442ebd6ae9cb132b67fb8a9d876, data reload: false

query1	930	389	369	369
query2	6318	2479	2328	2328
query3	6646	211	206	206
query4	24218	21808	22134	21808
query5	3581	424	448	424
query6	271	182	175	175
query7	4597	299	294	294
query8	241	204	192	192
query9	8495	2509	2458	2458
query10	426	241	250	241
query11	15742	14936	14735	14735
query12	114	82	79	79
query13	1569	357	347	347
query14	9541	8380	8444	8380
query15	242	168	172	168
query16	8107	248	259	248
query17	1713	552	535	535
query18	2043	290	275	275
query19	296	147	143	143
query20	87	82	84	82
query21	196	125	123	123
query22	5324	4940	4963	4940
query23	34341	33572	33457	33457
query24	10937	2925	2842	2842
query25	606	369	373	369
query26	1113	155	149	149
query27	2898	307	311	307
query28	7429	2041	2028	2028
query29	851	601	600	600
query30	273	173	173	173
query31	961	743	753	743
query32	85	49	54	49
query33	744	240	244	240
query34	999	458	472	458
query35	823	672	651	651
query36	1087	909	918	909
query37	131	68	66	66
query38	2940	2844	2775	2775
query39	1600	1539	1571	1539
query40	197	123	123	123
query41	46	44	43	43
query42	102	95	94	94
query43	599	585	533	533
query44	1173	715	739	715
query45	303	254	250	250
query46	1094	730	713	713
query47	1990	1875	1876	1875
query48	376	291	287	287
query49	891	388	384	384
query50	746	372	383	372
query51	6882	6843	6703	6703
query52	103	91	87	87
query53	360	285	295	285
query54	856	416	408	408
query55	71	70	70	70
query56	237	219	209	209
query57	1206	1153	1126	1126
query58	210	196	192	192
query59	3442	3312	3168	3168
query60	255	220	233	220
query61	87	84	114	84
query62	662	467	483	467
query63	305	282	291	282
query64	8750	2212	1700	1700
query65	3152	3091	3106	3091
query66	808	325	320	320
query67	15429	14829	15165	14829
query68	6306	527	535	527
query69	548	313	305	305
query70	1226	1090	1126	1090
query71	520	269	268	268
query72	7951	2554	2401	2401
query73	805	313	312	312
query74	6778	6206	6497	6206
query75	4467	2694	2685	2685
query76	4465	1005	935	935
query77	663	265	264	264
query78	10626	10222	10072	10072
query79	2356	503	516	503
query80	1300	442	433	433
query81	504	246	234	234
query82	788	96	93	93
query83	189	157	169	157
query84	266	86	81	81
query85	1329	270	262	262
query86	450	308	312	308
query87	3311	3191	3171	3171
query88	4238	2352	2301	2301
query89	469	392	373	373
query90	1988	181	179	179
query91	120	96	98	96
query92	57	49	46	46
query93	3486	485	485	485
query94	1123	175	172	172
query95	383	288	289	288
query96	580	270	265	265
query97	3163	3005	2981	2981
query98	241	213	208	208
query99	1245	921	919	919
Total cold run time: 290788 ms
Total hot run time: 183155 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.15 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8ff9938559110442ebd6ae9cb132b67fb8a9d876, data reload: false

query1	0.03	0.03	0.03
query2	0.08	0.03	0.04
query3	0.23	0.05	0.05
query4	1.68	0.07	0.06
query5	0.48	0.49	0.48
query6	1.17	0.72	0.73
query7	0.02	0.02	0.01
query8	0.04	0.04	0.04
query9	0.54	0.48	0.47
query10	0.54	0.54	0.53
query11	0.15	0.11	0.12
query12	0.14	0.12	0.12
query13	0.58	0.59	0.61
query14	0.78	0.79	0.76
query15	0.82	0.80	0.81
query16	0.38	0.38	0.36
query17	1.02	1.04	0.95
query18	0.24	0.24	0.21
query19	1.89	1.82	1.65
query20	0.02	0.01	0.00
query21	16.02	0.68	0.67
query22	3.84	7.48	2.65
query23	18.26	1.33	1.24
query24	1.41	0.33	0.20
query25	0.13	0.08	0.08
query26	0.26	0.16	0.17
query27	0.08	0.08	0.08
query28	13.50	1.03	0.99
query29	13.27	3.38	3.36
query30	0.24	0.06	0.05
query31	2.87	0.38	0.38
query32	3.28	0.47	0.47
query33	2.90	2.94	2.93
query34	17.22	4.43	4.38
query35	4.50	4.48	4.56
query36	0.66	0.48	0.46
query37	0.18	0.15	0.16
query38	0.15	0.14	0.15
query39	0.04	0.04	0.04
query40	0.16	0.14	0.14
query41	0.08	0.04	0.04
query42	0.05	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.97 s
Total hot run time: 31.15 s

@dataroaring dataroaring merged commit bb407ee into apache:master May 22, 2024
27 of 29 checks passed
M1saka2003 pushed a commit to M1saka2003/doris that referenced this pull request May 24, 2024
dataroaring pushed a commit that referenced this pull request May 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.x reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants