Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make Unit Test Great! #7969

Open
HuSharp opened this issue Mar 22, 2024 · 0 comments
Open

Make Unit Test Great! #7969

HuSharp opened this issue Mar 22, 2024 · 0 comments
Labels
type/development The issue belongs to a development tasks

Comments

@HuSharp
Copy link
Member

HuSharp commented Mar 22, 2024 •
edited

Development Task

  • The unstable CI has been a hinder to our daily development. Sometimes we have to run it again and again, and again ... to merge a PR.
  • When run make basic-test in local machine, there will be a lot of strange problems, as well as running extremely slow :(

Need 5:44.15 total for Intel(R) Xeon(R) Gold 6240 CPU @ 2.60GH, 144 core 72 CPU
but local mac(m1pro 16G) needs > 15min

We use a wrong way to parallel. Some test cases are not side effect free, for example:

  • the enable and disable of the failpoint
  • modification of the configuration
  • change of the global variable in a single process.
    etc...

What's the solution?

Add back parallel by run a single test case in a isolated OS process #7952

If the test cases with side effect run parallelly with others, it could cause some test fail unexpectedly.
When they're in a single OS process, make them parallel is not a big deal.

print function time

when using ./pd-ut run --junitfile xxx, can get junitfile as XML file to get every function cost time.

Furthermore

Related
@HuSharp HuSharp added the type/development The issue belongs to a development tasks label Mar 22, 2024
This was referenced Mar 22, 2024
Merged
Merged
Closed
@HuSharp HuSharp mentioned this issue Mar 29, 2024
Merged
ti-chi-bot bot pushed a commit that referenced this issue Mar 29, 2024
@HuSharp
tests/member: change leader priority check ticker to reduce test time (…
a44b208 
…#8001)

ref #7969

Signed-off-by: husharp <jinhao.hu@pingcap.com>
This was referenced Apr 2, 2024
Merged
Closed
@HuSharp HuSharp mentioned this issue Apr 9, 2024
Merged
ti-chi-bot bot pushed a commit that referenced this issue Apr 9, 2024
@HuSharp
tests/tso: make TestPrimaryPriorityChange stable (#8045)
3475fe5 
ref #7969

Signed-off-by: husharp <jinhao.hu@pingcap.com>
ti-chi-bot bot added a commit that referenced this issue Apr 10, 2024
@HuSharp @ti-chi-bot
*: make unit test great (#7952)
f0eb74b 
ref #7969

Signed-off-by: husharp <jinhao.hu@pingcap.com>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
ti-chi-bot bot pushed a commit that referenced this issue Apr 11, 2024
@HuSharp
tests/api: refine TestPreparingProgress to make test stable (#8014)
39f5712 
ref #7969

Signed-off-by: husharp <jinhao.hu@pingcap.com>
@HuSharp HuSharp mentioned this issue Apr 15, 2024
Merged
ti-chi-bot bot added a commit that referenced this issue Apr 15, 2024
@HuSharp @ti-chi-bot
cluster: make TestStoreOverloadedWithReplace stable (#8068)
a109f9c 
ref #7969

Signed-off-by: husharp <jinhao.hu@pingcap.com>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
This was referenced Apr 16, 2024
Merged
Closed
ti-chi-bot bot added a commit that referenced this issue Apr 16, 2024
@HuSharp @ti-chi-bot
tools/pd-ut: accelarate build by compile-without-link (#8069)
22543a9 
ref #7969

Signed-off-by: husharp <jinhao.hu@pingcap.com>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
@HuSharp HuSharp mentioned this issue Apr 19, 2024
Merged
ti-chi-bot bot pushed a commit that referenced this issue Apr 19, 2024
@HuSharp
tests/etcd: remove mod to make test stable (#8102)
32bf572 
ref #7969, close #8091

Signed-off-by: husharp <jinhao.hu@pingcap.com>
nolouch pushed a commit to nolouch/pd that referenced this issue Apr 29, 2024
@HuSharp @nolouch
tests/etcd: remove mod to make test stable (tikv#8102)
da16533 
ref tikv#7969, close tikv#8091

Signed-off-by: husharp <jinhao.hu@pingcap.com>
This was referenced May 11, 2024
Merged
Merged
ti-chi-bot bot pushed a commit that referenced this issue May 13, 2024
@HuSharp
pkg/ratelimit: make TestConcurrencyLimiterAcquire stable (#8169)
5d2b773 
ref #7969

Signed-off-by: husharp <jinhao.hu@pingcap.com>
ti-chi-bot bot added a commit that referenced this issue May 13, 2024
@HuSharp @ti-chi-bot
tests/operators: pause all checkers to make test stable (#8166)
5f8c38b 
ref #7969

Signed-off-by: husharp <jinhao.hu@pingcap.com>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
This was referenced May 14, 2024
Merged
Merged
Merged
ti-chi-bot bot pushed a commit that referenced this issue May 14, 2024
@HuSharp
test/server: replace default 30s to 1s to reduce test cost time (#8171)
ba7e63a 
ref #7969

Signed-off-by: husharp <jinhao.hu@pingcap.com>
@HuSharp HuSharp mentioned this issue May 15, 2024
Merged
ti-chi-bot bot added a commit that referenced this issue May 15, 2024
@HuSharp @ti-chi-bot
tests/server/api: make TestPreparingProgress stable (#8177)
91d4fad 
ref #7969

Signed-off-by: husharp <jinhao.hu@pingcap.com>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
@HuSharp HuSharp mentioned this issue May 17, 2024
Merged
ti-chi-bot bot pushed a commit that referenced this issue May 17, 2024
@HuSharp @nolouch
*: support generate the test coverage file and using it in ci (#7991)
4477285 
ref #7969

Signed-off-by: husharp <jinhao.hu@pingcap.com>

Co-authored-by: ShuNing <nolouch@gmail.com>
This was referenced May 20, 2024
Merged
Merged
ti-chi-bot bot pushed a commit that referenced this issue May 20, 2024
@HuSharp
tests/client: unify http client in tests (#8199)
3e672b2 
ref #7969

Signed-off-by: husharp <jinhao.hu@pingcap.com>
ti-chi-bot bot pushed a commit that referenced this issue May 21, 2024
@HuSharp
tools/ut: replace integration test in ci (#8200)
2fabb74 
ref #7969

Signed-off-by: husharp <jinhao.hu@pingcap.com>
@HuSharp HuSharp mentioned this issue May 21, 2024
Merged
ti-chi-bot bot added a commit that referenced this issue May 22, 2024
@HuSharp @ti-chi-bot
tools/ut: add alloc server for pd-ut (#8203)
d7c7a41 
ref #7969

Signed-off-by: husharp <jinhao.hu@pingcap.com>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
ti-chi-bot bot pushed a commit that referenced this issue May 23, 2024
@HuSharp
tools/ut: add a parallel parameter (#8186)
0f3e1f9 
ref #7969

add parallel parameter

Signed-off-by: husharp <jinhao.hu@pingcap.com>
okJiang pushed a commit to okJiang/pd that referenced this issue May 23, 2024
@HuSharp @okJiang
tools/ut: add a parallel parameter (tikv#8186)
3c25cf9 
ref tikv#7969

add parallel parameter

Signed-off-by: husharp <jinhao.hu@pingcap.com>
Signed-off-by: okJiang <819421878@qq.com>
okJiang pushed a commit to okJiang/pd that referenced this issue May 23, 2024
@HuSharp @okJiang
tools/ut: add a parallel parameter (tikv#8186)
9f60979 
ref tikv#7969

add parallel parameter

Signed-off-by: husharp <jinhao.hu@pingcap.com>
Signed-off-by: okJiang <819421878@qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type/development The issue belongs to a development tasks
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@HuSharp