Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

the lock of lpush #703

Closed
cogbee opened this issue Jan 14, 2016 · 6 comments
Closed

the lock of lpush #703

cogbee opened this issue Jan 14, 2016 · 6 comments

Comments

@cogbee
Copy link

cogbee commented Jan 14, 2016

when I use lpush,accidentally the program stop here(use log to locate).I am sure that redis is working perfectly becasue the other machines is working well.
the more information is here:

[Switching to thread 29 (Thread 0x7fba5d86a700 (LWP 14668))]#0 0x000000376620d720 in sem_wait () from /lib64/libpthread.so.0
(gdb) bt
#0 0x000000376620d720 in sem_wait () from /lib64/libpthread.so.0
#1 0x00000000004ddc18 in PyThread_acquire_lock (lock=0x7fba541e2fa0, waitflag=) at Python/thread_pthread.h:324
#2 0x00000000004e2232 in lock_PyThread_acquire_lock (self=0x2e41df0, args=) at ./Modules/threadmodule.c:52
#3 0x00000000004a8b40 in call_function (f=, throwflag=) at Python/ceval.c:4021
#4 PyEval_EvalFrameEx (f=, throwflag=) at Python/ceval.c:2666
#5 0x00000000004aa627 in PyEval_EvalCodeEx (co=0x7fba63977330, globals=, locals=, args=, argcount=1, kws=0x7fba54001320, kwcount=0,

defs=0x7fba632ad0a8, defcount=1, closure=0x0) at Python/ceval.c:3253

#6 0x00000000004a87d4 in fast_function (f=, throwflag=) at Python/ceval.c:4117
#7 call_function (f=, throwflag=) at Python/ceval.c:4042
#8 PyEval_EvalFrameEx (f=, throwflag=) at Python/ceval.c:2666
#9 0x00000000004aa627 in PyEval_EvalCodeEx (co=0x7fba6245f1b0, globals=, locals=, args=, argcount=1, kws=0x7fba540010f8, kwcount=0,

defs=0x7fba6245aec0, defcount=2, closure=0x0) at Python/ceval.c:3253

#10 0x00000000004a87d4 in fast_function (f=, throwflag=) at Python/ceval.c:4117
#11 call_function (f=, throwflag=) at Python/ceval.c:4042
#12 PyEval_EvalFrameEx (f=, throwflag=) at Python/ceval.c:2666
#13 0x00000000004aa627 in PyEval_EvalCodeEx (co=0x2bea7b0, globals=, locals=, args=, argcount=5, kws=0x7fba69913068, kwcount=0, defs=0x2bf25b8,

defcount=3, closure=0x0) at Python/ceval.c:3253

#14 0x000000000050db78 in function_call (func=0x2bf0c80, arg=0x2e29a70, kw=0x2bd4360) at Objects/funcobject.c:526
#15 0x00000000004197c7 in PyObject_Call (func=0x2bf0c80, arg=, kw=) at Objects/abstract.c:2529
#16 0x00000000004a748a in ext_do_call (f=, throwflag=) at Python/ceval.c:4334
#17 PyEval_EvalFrameEx (f=, throwflag=) at Python/ceval.c:2705
#18 0x00000000004a9ab5 in fast_function (f=, throwflag=) at Python/ceval.c:4107
#19 call_function (f=, throwflag=) at Python/ceval.c:4042
#20 PyEval_EvalFrameEx (f=, throwflag=) at Python/ceval.c:2666
#21 0x00000000004a9ab5 in fast_function (f=, throwflag=) at Python/ceval.c:4107
#22 call_function (f=, throwflag=) at Python/ceval.c:4042
#23 PyEval_EvalFrameEx (f=, throwflag=) at Python/ceval.c:2666
#24 0x00000000004aa627 in PyEval_EvalCodeEx (co=0x7fba6397f330, globals=, locals=, args=, argcount=1, kws=0x0, kwcount=0, defs=0x0, defcount=0,

closure=0x0) at Python/ceval.c:3253

#25 0x000000000050da7e in function_call (func=0x7fba632c41b8, arg=0x2e2b110, kw=0x0) at Objects/funcobject.c:526
#26 0x00000000004197c7 in PyObject_Call (func=0x7fba632c41b8, arg=, kw=) at Objects/abstract.c:2529
#27 0x00000000004223ef in instancemethod_call (func=0x7fba632c41b8, arg=0x2e2b110, kw=0x0) at Objects/classobject.c:2602
#28 0x00000000004197c7 in PyObject_Call (func=0x252ef00, arg=, kw=) at Objects/abstract.c:2529
#29 0x00000000004a2413 in PyEval_CallObjectWithKeywords (func=0x252ef00, arg=0x7fba69913050, kw=) at Python/ceval.c:3890
#30 0x00000000004e26f2 in t_bootstrap (boot_raw=0x2c86190) at ./Modules/threadmodule.c:614
#31 0x0000003766207851 in start_thread () from /lib64/libpthread.so.0
#32 0x0000003765ee767d in clone () from /lib64/libc.so.6

so is there any suggestion and help? is the problem of Lock?

@andymccurdy
Copy link
Contributor

Do you know where in the redis-py code this is? There's not a lot to go on from the C stack trace.

Looks like your program is multithreaded. Can you describe how you're using redis-py w/ threads?

Also, what version of redis-py are you using?

@cogbee
Copy link
Author

cogbee commented Jan 15, 2016

the version is redis-py-2.10.3
I am sure the problem is the command : lpush
usage example:
r = redis.Redis(host=Constants.REDIS_HOST_NAME, port=Constants.REDIS_PORT_NORMAL, password=Constants.REDIS_PASSWORD)
r.lpush(_,_*)

we have a few threads all running this code.
Is there any help?
thank you.

@andymccurdy
Copy link
Contributor

There's nothing about the LPUSH command specifically that would be causing a locking issue. My guess is that something about the way you're running multiple threads is causing issues. I'm happy to help diagnose if you can either share your code or put together a stripped down example that causes the locking issue.

@cogbee
Copy link
Author

cogbee commented Jan 18, 2016

the way I solve this problem is to try catch the lpush. when the program run at the point of lpush,maybe the data is too large, the thread is crash.and my program stop here then.
I will to negotiate with the redis to see whether they have the limit.
thank you for your help any way.

@andymccurdy
Copy link
Contributor

If you have relatively large date, I'd suggest upgrading to the latest version of redis-py. There have been some improvements dealing with larger amounts of data.

@Grokzen
Copy link

Grokzen commented Dec 27, 2018

@andymccurdy This issue seems to be resolved and could be closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants