fscache: handle trailing slash in exists_case prefix #16772
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
for more information, see https://pre-commit.ci
This comment has been minimized.
This comment has been minimized.
|
The diff looks good I think. Not sure what to do about the failing test though. Looks like that whole testfile was not really written for Windows, and for this test the difference between |
mypy/fscache.py
Outdated
| @@ -230,6 +230,7 @@ def exists_case(self, path: str, prefix: str) -> bool: | |||
| if path in self.exists_case_cache: | |||
| return self.exists_case_cache[path] | |||
| head, tail = os.path.split(path) | |||
| prefix = prefix.rstrip(os.sep) | |||
| if not head.startswith(prefix) or not tail: | |||
There was a problem hiding this comment.
Hm, sort of pre-existing, but I guess stripping is a little problematic for startswith, e.g. path=/foobar/baz, prefix=/foo. I guess it just means there ends up being another listdir
There was a problem hiding this comment.
Ah yes, didn't think about that before. That's indeed pre-existing, but changed the condition a bit to catch that as well.
This comment has been minimized.
This comment has been minimized.
for more information, see https://pre-commit.ci
|
Diff from mypy_primer, showing the effect of this PR on open source code: manticore (https://github.com/trailofbits/manticore)
- mypy: can't read file '/tmp/mypy_primer/projects/_manticore_venv/lib/python3.10/site-packages//google': No such file or directory
+ tests/auto_generators/make_dump.py:204: error: Name "CS_ARCH_X86" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:205: error: Name "CS_MODE_32" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:205: error: Name "CS_MODE_64" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:206: error: Name "Cs" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:251: error: Name "B" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:252: error: Name "B" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:256: error: Name "X86_OP_MEM" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:271: error: Name "X86_OP_IMM" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:273: error: Name "X86_OP_REG" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:313: error: Name "X86_REG_AH" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:313: error: Name "X86_REG_AX" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:314: error: Name "X86_REG_AL" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:314: error: Name "X86_REG_AX" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:315: error: Name "X86_REG_AX" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:315: error: Name "X86_REG_EAX" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:316: error: Name "X86_REG_EAX" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:316: error: Name "X86_REG_RAX" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:317: error: Name "X86_REG_RAX" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:317: error: Name "X86_REG_INVALID" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:318: error: Name "X86_REG_BH" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:318: error: Name "X86_REG_BX" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:319: error: Name "X86_REG_BL" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:319: error: Name "X86_REG_BX" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:320: error: Name "X86_REG_BX" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:320: error: Name "X86_REG_EBX" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:321: error: Name "X86_REG_EBX" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:321: error: Name "X86_REG_RBX" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:322: error: Name "X86_REG_RBX" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:322: error: Name "X86_REG_INVALID" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:323: error: Name "X86_REG_CH" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:323: error: Name "X86_REG_CX" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:324: error: Name "X86_REG_CL" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:324: error: Name "X86_REG_CX" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:325: error: Name "X86_REG_CX" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:325: error: Name "X86_REG_ECX" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:326: error: Name "X86_REG_ECX" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:326: error: Name "X86_REG_RCX" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:327: error: Name "X86_REG_RCX" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:327: error: Name "X86_REG_INVALID" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:328: error: Name "X86_REG_DH" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:328: error: Name "X86_REG_DX" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:329: error: Name "X86_REG_DL" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:329: error: Name "X86_REG_DX" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:330: error: Name "X86_REG_DX" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:330: error: Name "X86_REG_EDX" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:331: error: Name "X86_REG_EDX" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:331: error: Name "X86_REG_RDX" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:332: error: Name "X86_REG_RDX" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:332: error: Name "X86_REG_INVALID" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:333: error: Name "X86_REG_DIL" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:333: error: Name "X86_REG_EDI" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:334: error: Name "X86_REG_DI" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:334: error: Name "X86_REG_EDI" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:335: error: Name "X86_REG_EDI" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:335: error: Name "X86_REG_RDI" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:336: error: Name "X86_REG_RDI" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:336: error: Name "X86_REG_INVALID" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:337: error: Name "X86_REG_SIL" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:337: error: Name "X86_REG_ESI" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:338: error: Name "X86_REG_SI" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:338: error: Name "X86_REG_ESI" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:339: error: Name "X86_REG_ESI" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:339: error: Name "X86_REG_RSI" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:340: error: Name "X86_REG_RSI" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:340: error: Name "X86_REG_INVALID" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:370: error: Name "X86_OP_IMM" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:372: error: Name "X86_OP_REG" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:375: error: Name "X86_OP_MEM" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:390: error: Name "xrange" is not defined [name-defined]
+ tests/auto_generators/make_dump.py:422: error: Incompatible types in assignment (expression has type "str", variable has type "int") [assignment]
+ tests/auto_generators/make_dump.py:423: error: Invalid index type "int" for "Dict[str, Any]"; expected type "str" [index]
+ tests/wasm/json2mc.py:63: error: Need type annotation for "modules" (hint: "modules: List[<type>] = ...") [var-annotated]
+ tests/wasm/json2mc.py:103: error: No overload variant of "__getitem__" of "list" matches argument type "None" [call-overload]
+ tests/wasm/json2mc.py:103: note: Possible overload variants:
+ tests/wasm/json2mc.py:103: note: def __getitem__(self, SupportsIndex, /) -> Any
+ tests/wasm/json2mc.py:103: note: def __getitem__(self, slice, /) -> List[Any]
+ tests/wasm/json2mc.py:159: error: Invalid index type "Optional[int]" for "List[Module]"; expected type "SupportsIndex" [index]
+ tests/wasm/json2mc.py:161: error: Invalid index type "Optional[int]" for "List[Module]"; expected type "SupportsIndex" [index]
+ manticore/utils/log.py:14: error: Unused "type: ignore" comment [unused-ignore]
+ manticore/utils/log.py:176: error: Unused "type: ignore" comment [unused-ignore]
+ tests/auto_generators/make_tests.py:18: error: Need type annotation for "op_count" (hint: "op_count: Dict[<type>, <type>] = ...") [var-annotated]
+ manticore/utils/config.py:42: error: Incompatible default for argument "description" (default has type "None", argument has type "str") [assignment]
+ manticore/utils/config.py:42: note: PEP 484 prohibits implicit Optional. Accordingly, mypy has changed its default to no_implicit_optional=True
+ manticore/utils/config.py:42: note: Use https://github.com/hauntsaninja/no_implicit_optional to automatically upgrade your codebase
+ manticore/utils/config.py:95: error: Incompatible default for argument "description" (default has type "None", argument has type "str") [assignment]
+ manticore/utils/config.py:95: note: PEP 484 prohibits implicit Optional. Accordingly, mypy has changed its default to no_implicit_optional=True
+ manticore/utils/config.py:95: note: Use https://github.com/hauntsaninja/no_implicit_optional to automatically upgrade your codebase
+ manticore/utils/config.py:118: error: Incompatible default for argument "description" (default has type "None", argument has type "str") [assignment]
+ manticore/utils/config.py:118: note: PEP 484 prohibits implicit Optional. Accordingly, mypy has changed its default to no_implicit_optional=True
+ manticore/utils/config.py:118: note: Use https://github.com/hauntsaninja/no_implicit_optional to automatically upgrade your codebase
+ manticore/core/smtlib/solver.py:346: error: Incompatible default for argument "init" (default has type "None", argument has type "Sequence[str]") [assignment]
+ manticore/core/smtlib/solver.py:346: note: PEP 484 prohibits implicit Optional. Accordingly, mypy has changed its default to no_implicit_optional=True
... (truncated 97 lines) ...
streamlit (https://github.com/streamlit/streamlit)
- mypy: can't read file '/tmp/mypy_primer/projects/_streamlit_venv/lib/python3.10/site-packages//google': No such file or directory
+ lib/tests/streamlit/pyspark_mocks.py:18:1: error: Cannot find implementation or library stub for module named "pyspark.sql.dataframe" [import-not-found]
+ lib/tests/streamlit/pyspark_mocks.py:19:1: error: Cannot find implementation or library stub for module named "pyspark.sql.session" [import-not-found]
+ lib/tests/streamlit/pyspark_mocks.py:20:1: error: Cannot find implementation or library stub for module named "pyspark.sql.types" [import-not-found]
+ lib/tests/streamlit/external/langchain/capturing_callback_handler.py:26:1: error: Cannot find implementation or library stub for module named "langchain.callbacks.base" [import-not-found]
+ lib/streamlit/runtime/stats.py:20:1: error: Cannot find implementation or library stub for module named "streamlit.proto.openmetrics_data_model_pb2" [import-not-found]
+ lib/tests/streamlit/proto_compatibility_test.py:17:1: error: Cannot find implementation or library stub for module named "parameterized" [import-not-found]
+ lib/tests/streamlit/proto_compatibility_test.py:19:1: error: Cannot find implementation or library stub for module named "streamlit.proto.Alert_pb2" [import-not-found]
+ lib/tests/streamlit/proto_compatibility_test.py:20:1: error: Cannot find implementation or library stub for module named "streamlit.proto.AppPage_pb2" [import-not-found]
+ lib/tests/streamlit/proto_compatibility_test.py:21:1: error: Cannot find implementation or library stub for module named "streamlit.proto.Common_pb2" [import-not-found]
+ lib/tests/streamlit/proto_compatibility_test.py:22:1: error: Cannot find implementation or library stub for module named "streamlit.proto.Exception_pb2" [import-not-found]
+ lib/tests/streamlit/proto_compatibility_test.py:23:1: error: Cannot find implementation or library stub for module named "streamlit.proto.NewSession_pb2" [import-not-found]
+ lib/tests/streamlit/proto_compatibility_test.py:34:1: error: Cannot find implementation or library stub for module named "streamlit.proto.ParentMessage_pb2" [import-not-found]
+ lib/tests/streamlit/proto_compatibility_test.py:35:1: error: Cannot find implementation or library stub for module named "streamlit.proto.SessionStatus_pb2" [import-not-found]
+ lib/tests/streamlit/elements/lib/dicttools_test.py:18:1: error: Cannot find implementation or library stub for module named "parameterized" [import-not-found]
+ lib/streamlit/elements/empty.py:17:1: error: Cannot find implementation or library stub for module named "streamlit.proto.Empty_pb2" [import-not-found]
+ lib/streamlit/version.py:19:1: error: Cannot find implementation or library stub for module named "importlib_metadata" [import-not-found]
+ lib/streamlit/runtime/uploaded_file_manager.py:21:1: error: Cannot find implementation or library stub for module named "streamlit.proto.Common_pb2" [import-not-found]
+ lib/streamlit/runtime/forward_msg_queue.py:18:1: error: Cannot find implementation or library stub for module named "streamlit.proto.Delta_pb2" [import-not-found]
+ lib/streamlit/runtime/forward_msg_queue.py:19:1: error: Cannot find implementation or library stub for module named "streamlit.proto.ForwardMsg_pb2" [import-not-found]
+ lib/streamlit/runtime/forward_msg_queue.py: note: In function "_is_composable_message":
+ lib/streamlit/runtime/forward_msg_queue.py:107:5: error: Returning Any from function declared to return "bool" [no-any-return]
+ lib/streamlit/watcher/event_based_path_watcher.py:44:1: error: Cannot find implementation or library stub for module named "watchdog.observers.api" [import-not-found]
+ lib/streamlit/runtime/state/query_params.py:21:1: error: Cannot find implementation or library stub for module named "streamlit.proto.ForwardMsg_pb2" [import-not-found]
+ lib/streamlit/elements/progress.py:20:1: error: Cannot find implementation or library stub for module named "streamlit.proto.Progress_pb2" [import-not-found]
+ lib/streamlit/runtime/metrics_util.py:31:1: error: Cannot find implementation or library stub for module named "streamlit.proto.ForwardMsg_pb2" [import-not-found]
+ lib/streamlit/runtime/metrics_util.py:32:1: error: Cannot find implementation or library stub for module named "streamlit.proto.PageProfile_pb2" [import-not-found]
+ lib/streamlit/runtime/forward_msg_cache.py:21:1: error: Cannot find implementation or library stub for module named "streamlit.proto.ForwardMsg_pb2" [import-not-found]
+ lib/streamlit/runtime/forward_msg_cache.py: note: In function "populate_hash_if_needed":
+ lib/streamlit/runtime/forward_msg_cache.py:62:5: error: Returning Any from function declared to return "str" [no-any-return]
+ lib/streamlit/runtime/runtime_util.py:21:1: error: Cannot find implementation or library stub for module named "streamlit.proto.ForwardMsg_pb2" [import-not-found]
+ lib/streamlit/runtime/runtime_util.py: note: In function "is_cacheable_msg":
+ lib/streamlit/runtime/runtime_util.py:60:5: error: Returning Any from function declared to return "bool" [no-any-return]
+ lib/streamlit/runtime/runtime_util.py: note: In function "serialize_forward_msg":
+ lib/streamlit/runtime/runtime_util.py:80:5: error: Returning Any from function declared to return "bytes" [no-any-return]
+ lib/streamlit/elements/text.py:17:1: error: Cannot find implementation or library stub for module named "streamlit.proto.Text_pb2" [import-not-found]
+ lib/streamlit/elements/snow.py:17:1: error: Cannot find implementation or library stub for module named "streamlit.proto.Snow_pb2" [import-not-found]
+ lib/streamlit/elements/layouts.py:20:1: error: Cannot find implementation or library stub for module named "streamlit.proto.Block_pb2" [import-not-found]
+ lib/streamlit/elements/iframe.py:17:1: error: Cannot find implementation or library stub for module named "streamlit.proto.IFrame_pb2" [import-not-found]
+ lib/streamlit/elements/deck_gl_json_chart.py:22:1: error: Cannot find implementation or library stub for module named "streamlit.proto.DeckGlJsonChart_pb2" [import-not-found]
+ lib/streamlit/elements/deck_gl_json_chart.py: note: In function "_get_pydeck_tooltip":
+ lib/streamlit/elements/deck_gl_json_chart.py:150:9: error: Returning Any from function declared to return "Optional[Dict[str, str]]" [no-any-return]
+ lib/streamlit/elements/bokeh_chart.py:24:1: error: Cannot find implementation or library stub for module named "streamlit.proto.BokehChart_pb2" [import-not-found]
+ lib/streamlit/elements/balloons.py:17:1: error: Cannot find implementation or library stub for module named "streamlit.proto.Balloons_pb2" [import-not-found]
+ lib/streamlit/elements/alert.py:17:1: error: Cannot find implementation or library stub for module named "streamlit.proto.Alert_pb2" [import-not-found]
+ lib/streamlit/type_util.py:241:41: error: Implicit generic "Any". Use "typing.Dict" and specify generic parameters [type-arg]
+ lib/streamlit/type_util.py:241:47: error: Implicit generic "Any". Use "typing.List" and specify generic parameters [type-arg]
+ lib/streamlit/type_util.py:241:53: error: Implicit generic "Any". Use "typing.Set" and specify generic parameters [type-arg]
+ lib/streamlit/error_util.py:24:1: error: Cannot find implementation or library stub for module named "rich" [import-not-found]
+ lib/streamlit/error_util.py:38: error: Unused "type: ignore" comment [unused-ignore]
+ lib/streamlit/error_util.py:41:1: error: Cannot find implementation or library stub for module named "rich.console" [import-not-found]
+ lib/streamlit/elements/pyplot.py:26:1: error: Cannot find implementation or library stub for module named "streamlit.proto.Image_pb2" [import-not-found]
+ lib/streamlit/elements/plotly_chart.py:26:1: error: Cannot find implementation or library stub for module named "streamlit.proto.PlotlyChart_pb2" [import-not-found]
+ lib/streamlit/elements/media.py:24:1: error: Cannot find implementation or library stub for module named "streamlit.proto.Audio_pb2" [import-not-found]
+ lib/streamlit/elements/media.py:25:1: error: Cannot find implementation or library stub for module named "streamlit.proto.Video_pb2" [import-not-found]
+ lib/streamlit/elements/map.py:40:1: error: Cannot find implementation or library stub for module named "streamlit.proto.DeckGlJsonChart_pb2" [import-not-found]
+ lib/streamlit/elements/image.py:37:1: error: Cannot find implementation or library stub for module named "streamlit.proto.Image_pb2" [import-not-found]
+ lib/streamlit/elements/image.py:44:1: error: Cannot find implementation or library stub for module named "numpy.typing" [import-not-found]
+ lib/streamlit/elements/graphviz_chart.py:25:1: error: Cannot find implementation or library stub for module named "streamlit.proto.GraphVizChart_pb2" [import-not-found]
+ lib/streamlit/elements/exception.py:30:1: error: Cannot find implementation or library stub for module named "streamlit.proto.Exception_pb2" [import-not-found]
+ lib/streamlit/elements/lib/pandas_styler_utils.py:21:1: error: Cannot find implementation or library stub for module named "streamlit.proto.Arrow_pb2" [import-not-found]
+ lib/streamlit/runtime/state/common.py:39:1: error: Cannot find implementation or library stub for module named "streamlit.proto.Arrow_pb2" [import-not-found]
+ lib/streamlit/runtime/state/common.py:40:1: error: Cannot find implementation or library stub for module named "streamlit.proto.Button_pb2" [import-not-found]
+ lib/streamlit/runtime/state/common.py:41:1: error: Cannot find implementation or library stub for module named "streamlit.proto.CameraInput_pb2" [import-not-found]
+ lib/streamlit/runtime/state/common.py:42:1: error: Cannot find implementation or library stub for module named "streamlit.proto.ChatInput_pb2" [import-not-found]
... (truncated 1139 lines) ...
mypy-protobuf (https://github.com/dropbox/mypy-protobuf)
- mypy: can't read file '/tmp/mypy_primer/projects/_mypy-protobuf_venv/lib/python3.10/site-packages//google': No such file or directory
+ mypy_protobuf/extensions_pb2.pyi:6: error: Cannot find implementation or library stub for module named "google" [import-not-found]
+ mypy_protobuf/extensions_pb2.pyi:6: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
+ mypy_protobuf/extensions_pb2.pyi:20: error: Class cannot subclass "Message" (has type "Any") [misc]
+ mypy_protobuf/main.py:21: error: Cannot find implementation or library stub for module named "google" [import-not-found]
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The current
exists_cacheimplementation does not handle trailing separators on the givenprefixcorrectly. More precisely, it decides thatfoo/is not a prefix offoo. This results in unexpectedly returningTruefor directories that don't exist and that are logically under the given prefix.This also indirectly resolves #16767 because the linked issue in typeshed causes mypy to end up with a trailing separator in one of its directory existence checks. The linked issue should still be fixed, but this lets mypy handle the case gracefully, rather than giving a confusing error about not being able to read a directory.