forked from ipython/ipython
-
Notifications
You must be signed in to change notification settings - Fork 2
/
completer.py
3322 lines (2693 loc) · 114 KB
/
completer.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
"""Completion for IPython.
This module started as fork of the rlcompleter module in the Python standard
library. The original enhancements made to rlcompleter have been sent
upstream and were accepted as of Python 2.3,
This module now support a wide variety of completion mechanism both available
for normal classic Python code, as well as completer for IPython specific
Syntax like magics.
Latex and Unicode completion
============================
IPython and compatible frontends not only can complete your code, but can help
you to input a wide range of characters. In particular we allow you to insert
a unicode character using the tab completion mechanism.
Forward latex/unicode completion
--------------------------------
Forward completion allows you to easily type a unicode character using its latex
name, or unicode long description. To do so type a backslash follow by the
relevant name and press tab:
Using latex completion:
.. code::
\\alpha<tab>
α
or using unicode completion:
.. code::
\\GREEK SMALL LETTER ALPHA<tab>
α
Only valid Python identifiers will complete. Combining characters (like arrow or
dots) are also available, unlike latex they need to be put after the their
counterpart that is to say, ``F\\\\vec<tab>`` is correct, not ``\\\\vec<tab>F``.
Some browsers are known to display combining characters incorrectly.
Backward latex completion
-------------------------
It is sometime challenging to know how to type a character, if you are using
IPython, or any compatible frontend you can prepend backslash to the character
and press :kbd:`Tab` to expand it to its latex form.
.. code::
\\α<tab>
\\alpha
Both forward and backward completions can be deactivated by setting the
:std:configtrait:`Completer.backslash_combining_completions` option to
``False``.
Experimental
============
Starting with IPython 6.0, this module can make use of the Jedi library to
generate completions both using static analysis of the code, and dynamically
inspecting multiple namespaces. Jedi is an autocompletion and static analysis
for Python. The APIs attached to this new mechanism is unstable and will
raise unless use in an :any:`provisionalcompleter` context manager.
You will find that the following are experimental:
- :any:`provisionalcompleter`
- :any:`IPCompleter.completions`
- :any:`Completion`
- :any:`rectify_completions`
.. note::
better name for :any:`rectify_completions` ?
We welcome any feedback on these new API, and we also encourage you to try this
module in debug mode (start IPython with ``--Completer.debug=True``) in order
to have extra logging information if :any:`jedi` is crashing, or if current
IPython completer pending deprecations are returning results not yet handled
by :any:`jedi`
Using Jedi for tab completion allow snippets like the following to work without
having to execute any code:
>>> myvar = ['hello', 42]
... myvar[1].bi<tab>
Tab completion will be able to infer that ``myvar[1]`` is a real number without
executing almost any code unlike the deprecated :any:`IPCompleter.greedy`
option.
Be sure to update :any:`jedi` to the latest stable version or to try the
current development version to get better completions.
Matchers
========
All completions routines are implemented using unified *Matchers* API.
The matchers API is provisional and subject to change without notice.
The built-in matchers include:
- :any:`IPCompleter.dict_key_matcher`: dictionary key completions,
- :any:`IPCompleter.magic_matcher`: completions for magics,
- :any:`IPCompleter.unicode_name_matcher`,
:any:`IPCompleter.fwd_unicode_matcher`
and :any:`IPCompleter.latex_name_matcher`: see `Forward latex/unicode completion`_,
- :any:`back_unicode_name_matcher` and :any:`back_latex_name_matcher`: see `Backward latex completion`_,
- :any:`IPCompleter.file_matcher`: paths to files and directories,
- :any:`IPCompleter.python_func_kw_matcher` - function keywords,
- :any:`IPCompleter.python_matches` - globals and attributes (v1 API),
- ``IPCompleter.jedi_matcher`` - static analysis with Jedi,
- :any:`IPCompleter.custom_completer_matcher` - pluggable completer with a default
implementation in :any:`InteractiveShell` which uses IPython hooks system
(`complete_command`) with string dispatch (including regular expressions).
Differently to other matchers, ``custom_completer_matcher`` will not suppress
Jedi results to match behaviour in earlier IPython versions.
Custom matchers can be added by appending to ``IPCompleter.custom_matchers`` list.
Matcher API
-----------
Simplifying some details, the ``Matcher`` interface can described as
.. code-block::
MatcherAPIv1 = Callable[[str], list[str]]
MatcherAPIv2 = Callable[[CompletionContext], SimpleMatcherResult]
Matcher = MatcherAPIv1 | MatcherAPIv2
The ``MatcherAPIv1`` reflects the matcher API as available prior to IPython 8.6.0
and remains supported as a simplest way for generating completions. This is also
currently the only API supported by the IPython hooks system `complete_command`.
To distinguish between matcher versions ``matcher_api_version`` attribute is used.
More precisely, the API allows to omit ``matcher_api_version`` for v1 Matchers,
and requires a literal ``2`` for v2 Matchers.
Once the API stabilises future versions may relax the requirement for specifying
``matcher_api_version`` by switching to :any:`functools.singledispatch`, therefore
please do not rely on the presence of ``matcher_api_version`` for any purposes.
Suppression of competing matchers
---------------------------------
By default results from all matchers are combined, in the order determined by
their priority. Matchers can request to suppress results from subsequent
matchers by setting ``suppress`` to ``True`` in the ``MatcherResult``.
When multiple matchers simultaneously request surpression, the results from of
the matcher with higher priority will be returned.
Sometimes it is desirable to suppress most but not all other matchers;
this can be achieved by adding a list of identifiers of matchers which
should not be suppressed to ``MatcherResult`` under ``do_not_suppress`` key.
The suppression behaviour can is user-configurable via
:std:configtrait:`IPCompleter.suppress_competing_matchers`.
"""
# Copyright (c) IPython Development Team.
# Distributed under the terms of the Modified BSD License.
#
# Some of this code originated from rlcompleter in the Python standard library
# Copyright (C) 2001 Python Software Foundation, www.python.org
from __future__ import annotations
import builtins as builtin_mod
import enum
import glob
import inspect
import itertools
import keyword
import os
import re
import string
import sys
import tokenize
import time
import unicodedata
import uuid
import warnings
from ast import literal_eval
from collections import defaultdict
from contextlib import contextmanager
from dataclasses import dataclass
from functools import cached_property, partial
from types import SimpleNamespace
from typing import (
Iterable,
Iterator,
List,
Tuple,
Union,
Any,
Sequence,
Dict,
Optional,
TYPE_CHECKING,
Set,
Sized,
TypeVar,
Literal,
)
from IPython.core.guarded_eval import guarded_eval, EvaluationContext
from IPython.core.error import TryNext
from IPython.core.inputtransformer2 import ESC_MAGIC
from IPython.core.latex_symbols import latex_symbols, reverse_latex_symbol
from IPython.core.oinspect import InspectColors
from IPython.testing.skipdoctest import skip_doctest
from IPython.utils import generics
from IPython.utils.decorators import sphinx_options
from IPython.utils.dir2 import dir2, get_real_method
from IPython.utils.docs import GENERATING_DOCUMENTATION
from IPython.utils.path import ensure_dir_exists
from IPython.utils.process import arg_split
from traitlets import (
Bool,
Enum,
Int,
List as ListTrait,
Unicode,
Dict as DictTrait,
Union as UnionTrait,
observe,
)
from traitlets.config.configurable import Configurable
import __main__
# skip module docstests
__skip_doctest__ = True
try:
import jedi
jedi.settings.case_insensitive_completion = False
import jedi.api.helpers
import jedi.api.classes
JEDI_INSTALLED = True
except ImportError:
JEDI_INSTALLED = False
if TYPE_CHECKING or GENERATING_DOCUMENTATION and sys.version_info >= (3, 11):
from typing import cast
from typing_extensions import TypedDict, NotRequired, Protocol, TypeAlias, TypeGuard
else:
from typing import Generic
def cast(type_, obj):
"""Workaround for `TypeError: MatcherAPIv2() takes no arguments`"""
return obj
# do not require on runtime
NotRequired = Tuple # requires Python >=3.11
TypedDict = Dict # by extension of `NotRequired` requires 3.11 too
Protocol = object # requires Python >=3.8
TypeAlias = Any # requires Python >=3.10
TypeGuard = Generic # requires Python >=3.10
if GENERATING_DOCUMENTATION:
from typing import TypedDict
# -----------------------------------------------------------------------------
# Globals
#-----------------------------------------------------------------------------
# ranges where we have most of the valid unicode names. We could be more finer
# grained but is it worth it for performance While unicode have character in the
# range 0, 0x110000, we seem to have name for about 10% of those. (131808 as I
# write this). With below range we cover them all, with a density of ~67%
# biggest next gap we consider only adds up about 1% density and there are 600
# gaps that would need hard coding.
_UNICODE_RANGES = [(32, 0x323B0), (0xE0001, 0xE01F0)]
# Public API
__all__ = ["Completer", "IPCompleter"]
if sys.platform == 'win32':
PROTECTABLES = ' '
else:
PROTECTABLES = ' ()[]{}?=\\|;:\'#*"^&'
# Protect against returning an enormous number of completions which the frontend
# may have trouble processing.
MATCHES_LIMIT = 500
# Completion type reported when no type can be inferred.
_UNKNOWN_TYPE = "<unknown>"
# sentinel value to signal lack of a match
not_found = object()
class ProvisionalCompleterWarning(FutureWarning):
"""
Exception raise by an experimental feature in this module.
Wrap code in :any:`provisionalcompleter` context manager if you
are certain you want to use an unstable feature.
"""
pass
warnings.filterwarnings('error', category=ProvisionalCompleterWarning)
@skip_doctest
@contextmanager
def provisionalcompleter(action='ignore'):
"""
This context manager has to be used in any place where unstable completer
behavior and API may be called.
>>> with provisionalcompleter():
... completer.do_experimental_things() # works
>>> completer.do_experimental_things() # raises.
.. note::
Unstable
By using this context manager you agree that the API in use may change
without warning, and that you won't complain if they do so.
You also understand that, if the API is not to your liking, you should report
a bug to explain your use case upstream.
We'll be happy to get your feedback, feature requests, and improvements on
any of the unstable APIs!
"""
with warnings.catch_warnings():
warnings.filterwarnings(action, category=ProvisionalCompleterWarning)
yield
def has_open_quotes(s):
"""Return whether a string has open quotes.
This simply counts whether the number of quote characters of either type in
the string is odd.
Returns
-------
If there is an open quote, the quote character is returned. Else, return
False.
"""
# We check " first, then ', so complex cases with nested quotes will get
# the " to take precedence.
if s.count('"') % 2:
return '"'
elif s.count("'") % 2:
return "'"
else:
return False
def protect_filename(s, protectables=PROTECTABLES):
"""Escape a string to protect certain characters."""
if set(s) & set(protectables):
if sys.platform == "win32":
return '"' + s + '"'
else:
return "".join(("\\" + c if c in protectables else c) for c in s)
else:
return s
def expand_user(path:str) -> Tuple[str, bool, str]:
"""Expand ``~``-style usernames in strings.
This is similar to :func:`os.path.expanduser`, but it computes and returns
extra information that will be useful if the input was being used in
computing completions, and you wish to return the completions with the
original '~' instead of its expanded value.
Parameters
----------
path : str
String to be expanded. If no ~ is present, the output is the same as the
input.
Returns
-------
newpath : str
Result of ~ expansion in the input path.
tilde_expand : bool
Whether any expansion was performed or not.
tilde_val : str
The value that ~ was replaced with.
"""
# Default values
tilde_expand = False
tilde_val = ''
newpath = path
if path.startswith('~'):
tilde_expand = True
rest = len(path)-1
newpath = os.path.expanduser(path)
if rest:
tilde_val = newpath[:-rest]
else:
tilde_val = newpath
return newpath, tilde_expand, tilde_val
def compress_user(path:str, tilde_expand:bool, tilde_val:str) -> str:
"""Does the opposite of expand_user, with its outputs.
"""
if tilde_expand:
return path.replace(tilde_val, '~')
else:
return path
def completions_sorting_key(word):
"""key for sorting completions
This does several things:
- Demote any completions starting with underscores to the end
- Insert any %magic and %%cellmagic completions in the alphabetical order
by their name
"""
prio1, prio2 = 0, 0
if word.startswith('__'):
prio1 = 2
elif word.startswith('_'):
prio1 = 1
if word.endswith('='):
prio1 = -1
if word.startswith('%%'):
# If there's another % in there, this is something else, so leave it alone
if not "%" in word[2:]:
word = word[2:]
prio2 = 2
elif word.startswith('%'):
if not "%" in word[1:]:
word = word[1:]
prio2 = 1
return prio1, word, prio2
class _FakeJediCompletion:
"""
This is a workaround to communicate to the UI that Jedi has crashed and to
report a bug. Will be used only id :any:`IPCompleter.debug` is set to true.
Added in IPython 6.0 so should likely be removed for 7.0
"""
def __init__(self, name):
self.name = name
self.complete = name
self.type = 'crashed'
self.name_with_symbols = name
self.signature = ""
self._origin = "fake"
self.text = "crashed"
def __repr__(self):
return '<Fake completion object jedi has crashed>'
_JediCompletionLike = Union[jedi.api.Completion, _FakeJediCompletion]
class Completion:
"""
Completion object used and returned by IPython completers.
.. warning::
Unstable
This function is unstable, API may change without warning.
It will also raise unless use in proper context manager.
This act as a middle ground :any:`Completion` object between the
:any:`jedi.api.classes.Completion` object and the Prompt Toolkit completion
object. While Jedi need a lot of information about evaluator and how the
code should be ran/inspected, PromptToolkit (and other frontend) mostly
need user facing information.
- Which range should be replaced replaced by what.
- Some metadata (like completion type), or meta information to displayed to
the use user.
For debugging purpose we can also store the origin of the completion (``jedi``,
``IPython.python_matches``, ``IPython.magics_matches``...).
"""
__slots__ = ['start', 'end', 'text', 'type', 'signature', '_origin']
def __init__(
self,
start: int,
end: int,
text: str,
*,
type: Optional[str] = None,
_origin="",
signature="",
) -> None:
warnings.warn(
"``Completion`` is a provisional API (as of IPython 6.0). "
"It may change without warnings. "
"Use in corresponding context manager.",
category=ProvisionalCompleterWarning,
stacklevel=2,
)
self.start = start
self.end = end
self.text = text
self.type = type
self.signature = signature
self._origin = _origin
def __repr__(self):
return '<Completion start=%s end=%s text=%r type=%r, signature=%r,>' % \
(self.start, self.end, self.text, self.type or '?', self.signature or '?')
def __eq__(self, other) -> bool:
"""
Equality and hash do not hash the type (as some completer may not be
able to infer the type), but are use to (partially) de-duplicate
completion.
Completely de-duplicating completion is a bit tricker that just
comparing as it depends on surrounding text, which Completions are not
aware of.
"""
return self.start == other.start and \
self.end == other.end and \
self.text == other.text
def __hash__(self):
return hash((self.start, self.end, self.text))
class SimpleCompletion:
"""Completion item to be included in the dictionary returned by new-style Matcher (API v2).
.. warning::
Provisional
This class is used to describe the currently supported attributes of
simple completion items, and any additional implementation details
should not be relied on. Additional attributes may be included in
future versions, and meaning of text disambiguated from the current
dual meaning of "text to insert" and "text to used as a label".
"""
__slots__ = ["text", "type"]
def __init__(self, text: str, *, type: Optional[str] = None):
self.text = text
self.type = type
def __repr__(self):
return f"<SimpleCompletion text={self.text!r} type={self.type!r}>"
class _MatcherResultBase(TypedDict):
"""Definition of dictionary to be returned by new-style Matcher (API v2)."""
#: Suffix of the provided ``CompletionContext.token``, if not given defaults to full token.
matched_fragment: NotRequired[str]
#: Whether to suppress results from all other matchers (True), some
#: matchers (set of identifiers) or none (False); default is False.
suppress: NotRequired[Union[bool, Set[str]]]
#: Identifiers of matchers which should NOT be suppressed when this matcher
#: requests to suppress all other matchers; defaults to an empty set.
do_not_suppress: NotRequired[Set[str]]
#: Are completions already ordered and should be left as-is? default is False.
ordered: NotRequired[bool]
@sphinx_options(show_inherited_members=True, exclude_inherited_from=["dict"])
class SimpleMatcherResult(_MatcherResultBase, TypedDict):
"""Result of new-style completion matcher."""
# note: TypedDict is added again to the inheritance chain
# in order to get __orig_bases__ for documentation
#: List of candidate completions
completions: Sequence[SimpleCompletion] | Iterator[SimpleCompletion]
class _JediMatcherResult(_MatcherResultBase):
"""Matching result returned by Jedi (will be processed differently)"""
#: list of candidate completions
completions: Iterator[_JediCompletionLike]
AnyMatcherCompletion = Union[_JediCompletionLike, SimpleCompletion]
AnyCompletion = TypeVar("AnyCompletion", AnyMatcherCompletion, Completion)
@dataclass
class CompletionContext:
"""Completion context provided as an argument to matchers in the Matcher API v2."""
# rationale: many legacy matchers relied on completer state (`self.text_until_cursor`)
# which was not explicitly visible as an argument of the matcher, making any refactor
# prone to errors; by explicitly passing `cursor_position` we can decouple the matchers
# from the completer, and make substituting them in sub-classes easier.
#: Relevant fragment of code directly preceding the cursor.
#: The extraction of token is implemented via splitter heuristic
#: (following readline behaviour for legacy reasons), which is user configurable
#: (by switching the greedy mode).
token: str
#: The full available content of the editor or buffer
full_text: str
#: Cursor position in the line (the same for ``full_text`` and ``text``).
cursor_position: int
#: Cursor line in ``full_text``.
cursor_line: int
#: The maximum number of completions that will be used downstream.
#: Matchers can use this information to abort early.
#: The built-in Jedi matcher is currently excepted from this limit.
# If not given, return all possible completions.
limit: Optional[int]
@cached_property
def text_until_cursor(self) -> str:
return self.line_with_cursor[: self.cursor_position]
@cached_property
def line_with_cursor(self) -> str:
return self.full_text.split("\n")[self.cursor_line]
#: Matcher results for API v2.
MatcherResult = Union[SimpleMatcherResult, _JediMatcherResult]
class _MatcherAPIv1Base(Protocol):
def __call__(self, text: str) -> List[str]:
"""Call signature."""
...
#: Used to construct the default matcher identifier
__qualname__: str
class _MatcherAPIv1Total(_MatcherAPIv1Base, Protocol):
#: API version
matcher_api_version: Optional[Literal[1]]
def __call__(self, text: str) -> List[str]:
"""Call signature."""
...
#: Protocol describing Matcher API v1.
MatcherAPIv1: TypeAlias = Union[_MatcherAPIv1Base, _MatcherAPIv1Total]
class MatcherAPIv2(Protocol):
"""Protocol describing Matcher API v2."""
#: API version
matcher_api_version: Literal[2] = 2
def __call__(self, context: CompletionContext) -> MatcherResult:
"""Call signature."""
...
#: Used to construct the default matcher identifier
__qualname__: str
Matcher: TypeAlias = Union[MatcherAPIv1, MatcherAPIv2]
def _is_matcher_v1(matcher: Matcher) -> TypeGuard[MatcherAPIv1]:
api_version = _get_matcher_api_version(matcher)
return api_version == 1
def _is_matcher_v2(matcher: Matcher) -> TypeGuard[MatcherAPIv2]:
api_version = _get_matcher_api_version(matcher)
return api_version == 2
def _is_sizable(value: Any) -> TypeGuard[Sized]:
"""Determines whether objects is sizable"""
return hasattr(value, "__len__")
def _is_iterator(value: Any) -> TypeGuard[Iterator]:
"""Determines whether objects is sizable"""
return hasattr(value, "__next__")
def has_any_completions(result: MatcherResult) -> bool:
"""Check if any result includes any completions."""
completions = result["completions"]
if _is_sizable(completions):
return len(completions) != 0
if _is_iterator(completions):
try:
old_iterator = completions
first = next(old_iterator)
result["completions"] = cast(
Iterator[SimpleCompletion],
itertools.chain([first], old_iterator),
)
return True
except StopIteration:
return False
raise ValueError(
"Completions returned by matcher need to be an Iterator or a Sizable"
)
def completion_matcher(
*,
priority: Optional[float] = None,
identifier: Optional[str] = None,
api_version: int = 1,
):
"""Adds attributes describing the matcher.
Parameters
----------
priority : Optional[float]
The priority of the matcher, determines the order of execution of matchers.
Higher priority means that the matcher will be executed first. Defaults to 0.
identifier : Optional[str]
identifier of the matcher allowing users to modify the behaviour via traitlets,
and also used to for debugging (will be passed as ``origin`` with the completions).
Defaults to matcher function's ``__qualname__`` (for example,
``IPCompleter.file_matcher`` for the built-in matched defined
as a ``file_matcher`` method of the ``IPCompleter`` class).
api_version: Optional[int]
version of the Matcher API used by this matcher.
Currently supported values are 1 and 2.
Defaults to 1.
"""
def wrapper(func: Matcher):
func.matcher_priority = priority or 0 # type: ignore
func.matcher_identifier = identifier or func.__qualname__ # type: ignore
func.matcher_api_version = api_version # type: ignore
if TYPE_CHECKING:
if api_version == 1:
func = cast(MatcherAPIv1, func)
elif api_version == 2:
func = cast(MatcherAPIv2, func)
return func
return wrapper
def _get_matcher_priority(matcher: Matcher):
return getattr(matcher, "matcher_priority", 0)
def _get_matcher_id(matcher: Matcher):
return getattr(matcher, "matcher_identifier", matcher.__qualname__)
def _get_matcher_api_version(matcher):
return getattr(matcher, "matcher_api_version", 1)
context_matcher = partial(completion_matcher, api_version=2)
_IC = Iterable[Completion]
def _deduplicate_completions(text: str, completions: _IC)-> _IC:
"""
Deduplicate a set of completions.
.. warning::
Unstable
This function is unstable, API may change without warning.
Parameters
----------
text : str
text that should be completed.
completions : Iterator[Completion]
iterator over the completions to deduplicate
Yields
------
`Completions` objects
Completions coming from multiple sources, may be different but end up having
the same effect when applied to ``text``. If this is the case, this will
consider completions as equal and only emit the first encountered.
Not folded in `completions()` yet for debugging purpose, and to detect when
the IPython completer does return things that Jedi does not, but should be
at some point.
"""
completions = list(completions)
if not completions:
return
new_start = min(c.start for c in completions)
new_end = max(c.end for c in completions)
seen = set()
for c in completions:
new_text = text[new_start:c.start] + c.text + text[c.end:new_end]
if new_text not in seen:
yield c
seen.add(new_text)
def rectify_completions(text: str, completions: _IC, *, _debug: bool = False) -> _IC:
"""
Rectify a set of completions to all have the same ``start`` and ``end``
.. warning::
Unstable
This function is unstable, API may change without warning.
It will also raise unless use in proper context manager.
Parameters
----------
text : str
text that should be completed.
completions : Iterator[Completion]
iterator over the completions to rectify
_debug : bool
Log failed completion
Notes
-----
:any:`jedi.api.classes.Completion` s returned by Jedi may not have the same start and end, though
the Jupyter Protocol requires them to behave like so. This will readjust
the completion to have the same ``start`` and ``end`` by padding both
extremities with surrounding text.
During stabilisation should support a ``_debug`` option to log which
completion are return by the IPython completer and not found in Jedi in
order to make upstream bug report.
"""
warnings.warn("`rectify_completions` is a provisional API (as of IPython 6.0). "
"It may change without warnings. "
"Use in corresponding context manager.",
category=ProvisionalCompleterWarning, stacklevel=2)
completions = list(completions)
if not completions:
return
starts = (c.start for c in completions)
ends = (c.end for c in completions)
new_start = min(starts)
new_end = max(ends)
seen_jedi = set()
seen_python_matches = set()
for c in completions:
new_text = text[new_start:c.start] + c.text + text[c.end:new_end]
if c._origin == 'jedi':
seen_jedi.add(new_text)
elif c._origin == 'IPCompleter.python_matches':
seen_python_matches.add(new_text)
yield Completion(new_start, new_end, new_text, type=c.type, _origin=c._origin, signature=c.signature)
diff = seen_python_matches.difference(seen_jedi)
if diff and _debug:
print('IPython.python matches have extras:', diff)
if sys.platform == 'win32':
DELIMS = ' \t\n`!@#$^&*()=+[{]}|;\'",<>?'
else:
DELIMS = ' \t\n`!@#$^&*()=+[{]}\\|;:\'",<>?'
GREEDY_DELIMS = ' =\r\n'
class CompletionSplitter(object):
"""An object to split an input line in a manner similar to readline.
By having our own implementation, we can expose readline-like completion in
a uniform manner to all frontends. This object only needs to be given the
line of text to be split and the cursor position on said line, and it
returns the 'word' to be completed on at the cursor after splitting the
entire line.
What characters are used as splitting delimiters can be controlled by
setting the ``delims`` attribute (this is a property that internally
automatically builds the necessary regular expression)"""
# Private interface
# A string of delimiter characters. The default value makes sense for
# IPython's most typical usage patterns.
_delims = DELIMS
# The expression (a normal string) to be compiled into a regular expression
# for actual splitting. We store it as an attribute mostly for ease of
# debugging, since this type of code can be so tricky to debug.
_delim_expr = None
# The regular expression that does the actual splitting
_delim_re = None
def __init__(self, delims=None):
delims = CompletionSplitter._delims if delims is None else delims
self.delims = delims
@property
def delims(self):
"""Return the string of delimiter characters."""
return self._delims
@delims.setter
def delims(self, delims):
"""Set the delimiters for line splitting."""
expr = '[' + ''.join('\\'+ c for c in delims) + ']'
self._delim_re = re.compile(expr)
self._delims = delims
self._delim_expr = expr
def split_line(self, line, cursor_pos=None):
"""Split a line of text with a cursor at the given position.
"""
l = line if cursor_pos is None else line[:cursor_pos]
return self._delim_re.split(l)[-1]
class Completer(Configurable):
greedy = Bool(
False,
help="""Activate greedy completion.
.. deprecated:: 8.8
Use :std:configtrait:`Completer.evaluation` and :std:configtrait:`Completer.auto_close_dict_keys` instead.
When enabled in IPython 8.8 or newer, changes configuration as follows:
- ``Completer.evaluation = 'unsafe'``
- ``Completer.auto_close_dict_keys = True``
""",
).tag(config=True)
evaluation = Enum(
("forbidden", "minimal", "limited", "unsafe", "dangerous"),
default_value="limited",
help="""Policy for code evaluation under completion.
Successive options allow to enable more eager evaluation for better
completion suggestions, including for nested dictionaries, nested lists,
or even results of function calls.
Setting ``unsafe`` or higher can lead to evaluation of arbitrary user
code on :kbd:`Tab` with potentially unwanted or dangerous side effects.
Allowed values are:
- ``forbidden``: no evaluation of code is permitted,
- ``minimal``: evaluation of literals and access to built-in namespace;
no item/attribute evaluationm no access to locals/globals,