Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Facing slowness while running JSCPD #591

Open
Sudharshan45 opened this issue Jun 7, 2023 · 5 comments
Open

Facing slowness while running JSCPD #591

Sudharshan45 opened this issue Jun 7, 2023 · 5 comments

Comments

@Sudharshan45
Copy link

Sudharshan45 commented Jun 7, 2023

Hi kucherenko,
I am having 55,000 lines file in my repo. if we run JSCPD for that file takes more than 10 minutes. Most of the time consumes by Token generation Method name for reference (this.tokenizer.generateMaps(id, text, format, this.options); ) could please suggest solution for reduce time. Thanks in advance :)

Screenshot 2023-06-06 at 10 00 09 PM Screenshot 2023-06-06 at 10 00 56 PM
@kucherenko
Copy link
Owner

Hi @Sudharshan45
Sometimes tokenization process is slow, and I have no clear answer on how to fix it, usually, it requires more investigation of the sources.

@benjaminaudet
Copy link

benjaminaudet commented Aug 11, 2023

it would be very interesting to have a bit of your hardware config @Sudharshan45 just to fast check if it's not hardware related.
Personally I didn't faced any issues running it on a 110k lines of codes. It was a bit long but clearly far away from 10min.
My PC is a Latitude 5520. 16gigs i5

Oh, and out of curiosity, you're not on a hybrid development environment using Windows with WSL2?
If it's the case, assure using jscpd in the same filesystem as where you're repo is located.

@KineticTheory
Copy link

I'm having a similar pain-point. jscpd is taking about 15 minutes to parse through 100k LOC. I'm running in GitHub's super-linter image.

@Sudharshan45
Copy link
Author

Hi @benjaminaudet, I am using MAC M1 16gb and running JSCPD for single js file containing 55k lines of code. I fixed that issue by slicing single file into multiple files

@benjaminaudet
Copy link

Hi @benjaminaudet, I am using MAC M1 16gb and running JSCPD for single js file containing 55k lines of code. I fixed that issue by slicing single file into multiple files

Just realize you were doing it on one file of 55k LOC... I mean, seems logic to me, your PC was bottlenecking at some point.
Next time, make it a bit clearer that you're environment is so particular.
Happy you found a solution, best wishes to you. Issue resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants