Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make TelegramObject immutable #3249

Merged
merged 105 commits into from Dec 15, 2022
Merged
Show file tree
Hide file tree
Changes from 96 commits
Commits
Show all changes
105 commits
Select commit Hold shift + click to select a range
3c0fd67
Get a very rough start on TelegramObject(api_kwargs)
Bibo-Joshi Sep 8, 2022
790c53e
More work on api_kwargs
Bibo-Joshi Sep 9, 2022
1cc0f2e
More work on api_kwargs
Bibo-Joshi Sep 9, 2022
54d999f
Only copy data if necessary
Bibo-Joshi Sep 9, 2022
702b008
pre-commit updates
Bibo-Joshi Sep 9, 2022
b4f7efe
add _apply_api_kwargs
Bibo-Joshi Sep 9, 2022
912aa0e
Get started on tests
Bibo-Joshi Sep 9, 2022
55a7c94
More tests
Bibo-Joshi Sep 10, 2022
674dd4c
Handle deprecated field in stickerset
Bibo-Joshi Sep 10, 2022
e8884e7
Finish on existing tests
Bibo-Joshi Sep 11, 2022
07a3001
include api_kwargs in to_dict
Bibo-Joshi Sep 11, 2022
0e0da1a
API kwargs on unpickling
Bibo-Joshi Sep 11, 2022
3789e59
deepsource
Bibo-Joshi Sep 11, 2022
16be53c
more deepsource
Bibo-Joshi Sep 11, 2022
f0f9a3f
fix another failing test
Bibo-Joshi Sep 11, 2022
f29da8d
Documentation of TelegramObject
Bibo-Joshi Sep 11, 2022
0789f21
More Documentation
Bibo-Joshi Sep 11, 2022
23f6afe
Very rough first start on making TO immutable
Bibo-Joshi Sep 18, 2022
bf81325
Call `_freeze()` in all the `__init__` s
Bibo-Joshi Sep 18, 2022
76a5aba
add `__delattr__` and add basic test
Bibo-Joshi Sep 18, 2022
92c1d87
Unfreeze after `super().__init__` where necessary
Bibo-Joshi Sep 18, 2022
43cd729
Get started on adapting tests
Bibo-Joshi Sep 20, 2022
bc9d879
Get a few more tests running
Bibo-Joshi Sep 20, 2022
b317709
Merge branch 'master' into TO-overhaul
Bibo-Joshi Sep 25, 2022
21a75aa
Get startet on review
Bibo-Joshi Sep 25, 2022
ea3f13d
More review & dropping some unneeded `_parse_data` calls
Bibo-Joshi Sep 25, 2022
3ff5cae
more review
Bibo-Joshi Sep 25, 2022
17bd0da
Fix signature caching for subclasses
Bibo-Joshi Sep 25, 2022
d5ac429
doc fix
Bibo-Joshi Sep 25, 2022
1896b85
pre-commit
Bibo-Joshi Sep 25, 2022
059c9a0
coverage
Bibo-Joshi Sep 25, 2022
74dd8b6
update test_official
Bibo-Joshi Sep 25, 2022
8b8b386
Tone down on versionchanged
Bibo-Joshi Sep 25, 2022
0016e0b
Add tests for empty `api_kwargs`
Bibo-Joshi Sep 25, 2022
5c99ed2
bug fix
Bibo-Joshi Sep 25, 2022
5175cc7
More test adaptions
Bibo-Joshi Sep 25, 2022
3f81766
make `api_kwargs` a kw-only argument, pass to super in a few places a…
Bibo-Joshi Sep 29, 2022
cf6475b
tippity tappity, fixing tests
Bibo-Joshi Sep 29, 2022
7031807
Merge branch 'master' into TO-overhaul
Bibo-Joshi Oct 7, 2022
d9e511b
Merge branch 'TO-overhaul' into TO-immutable
Bibo-Joshi Oct 7, 2022
484e228
Merge branch 'master' into TO-immutable
Bibo-Joshi Oct 7, 2022
b5b2cf0
Make api_kwargs read-only and fix some merge errors
Bibo-Joshi Oct 7, 2022
14611d7
Merge branch 'master' into TO-immutable
Bibo-Joshi Oct 31, 2022
d0194e3
Try adapting to #3311
Bibo-Joshi Oct 31, 2022
7d44114
Remove an unnecessary unfreeze
Bibo-Joshi Oct 31, 2022
1464c80
Merge branch 'master' into TO-immutable
Bibo-Joshi Nov 3, 2022
ba73b3d
First draft of a script that converts all list-style class attributes…
Bibo-Joshi Nov 3, 2022
8745329
Sequence, not Collection
Bibo-Joshi Nov 7, 2022
0ea335f
Add versionchanged directives through transition script
Bibo-Joshi Nov 7, 2022
7ca6b94
try handling files with multiple classes
Bibo-Joshi Nov 7, 2022
eca7129
Just rerun the script if a file contains multiple classes
Bibo-Joshi Nov 8, 2022
b3ee612
Convert Union annotations
Bibo-Joshi Nov 8, 2022
27941cd
Improve handling of optionals in init
Bibo-Joshi Nov 8, 2022
a3bee06
More script tweaking
Bibo-Joshi Nov 8, 2022
852e4ec
Apply transition script
Bibo-Joshi Nov 8, 2022
a7cb547
make pre-commit happier
Bibo-Joshi Nov 9, 2022
a7ff921
Get a number of tests working
Bibo-Joshi Nov 11, 2022
ba5f301
more tests working
Bibo-Joshi Nov 11, 2022
e3569ba
more tests
Bibo-Joshi Nov 11, 2022
8291560
Merge branch 'master' into TO-immutable
Bibo-Joshi Nov 11, 2022
28486d3
*more tests!*
Bibo-Joshi Nov 11, 2022
874003b
Merge branch 'master' into TO-immutable
Bibo-Joshi Nov 11, 2022
e6cea13
Merge branch 'master' into TO-immutable
Bibo-Joshi Nov 13, 2022
ccf8fe0
fix a few more tests
Bibo-Joshi Nov 14, 2022
7e8b08e
more tests
Bibo-Joshi Nov 14, 2022
3c43dea
Merge branch 'master' into TO-immutable
Bibo-Joshi Nov 15, 2022
0acdae3
Make de_list return tuples
Bibo-Joshi Nov 15, 2022
3688ff8
Start with the hashes
Bibo-Joshi Nov 15, 2022
9a17d48
handle nested lists
Bibo-Joshi Nov 18, 2022
3cdb718
Fix docstrings for attributes
Bibo-Joshi Nov 18, 2022
46d4433
doc fixes
Bibo-Joshi Nov 18, 2022
408d7eb
silence DS on transition script
Bibo-Joshi Nov 18, 2022
58d0996
fix a test that hid in timeout errors so far …
Bibo-Joshi Nov 18, 2022
8eaa2ca
Merge branch 'master' into TO-immutable
Bibo-Joshi Nov 25, 2022
ee9e47e
fix a few tests
Bibo-Joshi Nov 25, 2022
dbccb6d
fix pickle-related tests
Bibo-Joshi Nov 25, 2022
373d179
add CSI comment
Bibo-Joshi Nov 25, 2022
8b767ad
a few doc fixes
Bibo-Joshi Nov 27, 2022
a260699
add a context manager to unfreeze TO
Bibo-Joshi Nov 27, 2022
53d91c7
use the new context manager
Bibo-Joshi Nov 27, 2022
11b32c5
try fixing tests
Bibo-Joshi Nov 28, 2022
07640da
Merge branch 'master' into TO-immutable
Bibo-Joshi Nov 30, 2022
81b8abd
Get started on review
Bibo-Joshi Nov 30, 2022
5800635
A bit more review
Bibo-Joshi Dec 1, 2022
2fafd40
squecne -> sequence
Bibo-Joshi Dec 1, 2022
23123f5
a few more docstring transitions
Bibo-Joshi Dec 1, 2022
109f355
A few "List" -> "Tuple"
Bibo-Joshi Dec 1, 2022
93e5af8
Rename every mention of "list" in return docstrings to tuple - it's j…
Bibo-Joshi Dec 1, 2022
687f572
add versionchanged to TO
Bibo-Joshi Dec 1, 2022
662ebc8
fix two tests
Bibo-Joshi Dec 1, 2022
6783256
Tuple type hints: arbitrary length
Bibo-Joshi Dec 2, 2022
d150dab
Filter out Nones in de_list and adapt some type hinting
Bibo-Joshi Dec 2, 2022
0ddd394
spelling bee
Bibo-Joshi Dec 2, 2022
78d5bfa
misc review
Bibo-Joshi Dec 2, 2022
c09311d
much tuple
Bibo-Joshi Dec 2, 2022
5c9bf57
Merge branch 'master' into TO-immutable
Bibo-Joshi Dec 2, 2022
3d2125f
Only filter out None
Bibo-Joshi Dec 2, 2022
6301afd
revert a docstring change
Bibo-Joshi Dec 2, 2022
bbe7dc6
Fix a signature in `ExtBot`
Bibo-Joshi Dec 2, 2022
24e9f0e
Drop empty sequences in `to_dict`
Bibo-Joshi Dec 2, 2022
fafe1d2
Remove transition script
Bibo-Joshi Dec 4, 2022
d5903f2
DS fixes
Bibo-Joshi Dec 6, 2022
971fa9c
Merge branch 'master' into TO-immutable
Bibo-Joshi Dec 6, 2022
c291f58
Merge branch 'master' into TO-immutable
Bibo-Joshi Dec 13, 2022
ac5634c
adapt to merge conflicts
Bibo-Joshi Dec 13, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/source/telegram.telegramobject.rst
Expand Up @@ -4,4 +4,4 @@ telegram.TelegramObject
.. autoclass:: telegram.TelegramObject
:members:
:show-inheritance:
:special-members: __repr__, __getitem__, __eq__, __hash__, __setstate__, __getstate__, __deepcopy__
:special-members: __repr__, __getitem__, __eq__, __hash__, __setstate__, __getstate__, __deepcopy__, __setattr__, __delattr__
8 changes: 7 additions & 1 deletion docs/substitutions/global.rst
Expand Up @@ -40,4 +40,10 @@

.. |disable_notification| replace:: Sends the message silently. Users will receive a notification with no sound.

.. |reply_to_msg_id| replace:: If the message is a reply, ID of the original message.
.. |reply_to_msg_id| replace:: If the message is a reply, ID of the original message.

.. |sequenceclassargs| replace:: Accepts any :class:`collections.abc.Sequence` as input instead of just a list. The input is converted to a tuple.

.. |tupleclassattrs| replace:: This attribute is now an immutable tuple.

.. |alwaystuple| replace:: This attribute is now always a tuple, that may be empty.
179 changes: 179 additions & 0 deletions immutabilize.py
@@ -0,0 +1,179 @@
# Loops through all TG classes and converts list-type attributes to tuples.
import inspect
import pickle
import re
from pathlib import Path

import isort

import telegram

changed_files = set()
processed_classes = set()

try:
processed_classes = pickle.load(open("processed_classes.pickle", "rb"))
except Exception:
print("Could not load pickle file")
processed_classes = set()

unprocessed_classes = set()

# loop through all classes in the `telegram` module
classes = inspect.getmembers(telegram, inspect.isclass)
for name, cls in classes:
if cls in processed_classes:
continue

print("Processing class", name)
# first adjust the __init__ of the class
params = inspect.signature(cls.__init__).parameters
params_to_change = {}
for param in params.values():
if "List" in str(param.annotation):
print(" Converting list-type parameter", param.name, "to Sequence")
params_to_change[param.name] = param.default
print(" ", param.name, "default value:", repr(param.default))

if not params_to_change:
continue

class_source_file = Path(inspect.getfile(cls))
if class_source_file not in changed_files:
changed_files.add(class_source_file)
processed_classes.add(cls)
else:
unprocessed_classes.add(cls)
continue

_, class_start_line = inspect.getsourcelines(cls)
class_source = inspect.getsource(cls)
class_source_lines = class_source.splitlines()
class_length = len(class_source_lines)

init_source_lines, init_start_line = inspect.getsourcelines(cls.__init__)
init_length = len(init_source_lines)
init_source = inspect.getsource(cls.__init__)

args_start_line = -1
attributes_start_line = -1
# Search "Args:" block in the docstring
for i, line in enumerate(class_source_lines):
if line.strip().startswith("Args:"):
args_start_line = i
if line.strip().startswith("Attributes:"):
attributes_start_line = i
if class_start_line + i == init_start_line:
break

# In the "Args:" block replace "List[" by "Sequence["
for i in range(args_start_line + 1, attributes_start_line):
whitespaces = -1
for param in params_to_change:
if f"{param} (List[" not in class_source_lines[i]:
continue

class_source_lines[i] = class_source_lines[i].replace(
f"{param} (List[", f"{param} (Sequence["
)
whitespaces = re.match(r" +", class_source_lines[i]).end() + 4
j = i + 1
lemontree210 marked this conversation as resolved.
Show resolved Hide resolved
while class_source_lines[j] and (
re.match(r"\s+", class_source_lines[j]).end() >= whitespaces
):
j = j + 1

class_source_lines[j - 1] += (
f"\n\n{whitespaces * ' '}.. versionchanged:: 20.0\n{whitespaces * ' '}"
" |sequenceclassargs|"
)
if class_source_lines[j]:
class_source_lines[j - 1] += "\n"

# In the "Attributes:" block replace "List[" by "Sequence["
for i in range(attributes_start_line + 1, class_length):
whitespaces = -1
for param in params_to_change:
if f"{param} (List[" not in class_source_lines[i]:
continue

class_source_lines[i] = class_source_lines[i].replace(
f"{param} (List[", f"{param} (Tuple["
)
whitespaces = re.match(r" +", class_source_lines[i]).end() + 4
j = i + 1
while class_source_lines[j] and (
re.match(r"\s+", class_source_lines[j]).end() >= whitespaces
):
j = j + 1

class_source_lines[j - 1] += (
f"\n\n{whitespaces * ' '}.. versionchanged:: 20.0\n{whitespaces * ' '}"
" |tupleclassattrs|"
)
if class_source_lines[j]:
class_source_lines[j - 1] += "\n"

# Adjust type annotations in the __init__ and converts to tuples before assigning to
# attributes
for param, default_value in params_to_change.items():
init_source = init_source.replace(param + ": List", param + ": Sequence")
init_source = re.sub(
rf"{param}: Union\[List\[(\w+)\], Tuple\[\w+, \.\.\.\]\]",
rf"{param}: Sequence[\1]",
init_source,
)
init_source = re.sub(
rf"{param}: Union\[Tuple\[\w+, \.\.\.\], List\[(\w+)\]\]",
rf"{param}: Sequence[\1]",
init_source,
)
if default_value is None:
init_source = re.sub(
rf"self\.{param} = (\S*)\n",
rf"self.{param} = tuple(\1) if \1 else None\n",
init_source,
)
else:
init_source = re.sub(
rf"self\.{param} = (\S*)\n", rf"self.{param} = tuple(\1)\n", init_source
)
init_source = re.sub(
rf"self\.{param} = (.*) or \[\]\n",
rf"self.{param} = tuple(\1) if \1 else ()\n",
init_source,
)

file_contents = class_source_file.read_text(encoding="utf-8")
file_contents = file_contents.splitlines()

# Insert new __init__
file_contents[
init_start_line - 1 : init_start_line + init_length - 1
] = init_source.splitlines()

# Insert new docstring
file_contents[class_start_line - 1 : init_start_line - 2] = class_source_lines[
: init_start_line - class_start_line - 1
]

i = 0
while file_contents[i].startswith("#"):
i += 1
file_contents[i] += "\nfrom typing import Sequence"

file_contents = "\n".join(file_contents) + "\n"

# Sort imports
file_contents = isort.code(file_contents)

class_source_file.write_text(file_contents, encoding="utf-8")

if unprocessed_classes:
print(
"Rerun the script to finish the conversion. I can't handle files that contain multiple "
"classes. The following classes were not processed:"
f"{', '.join([cls.__name__ for cls in unprocessed_classes])}"
)

pickle.dump(processed_classes, open("processed_classes.pickle", "wb"))
1 change: 1 addition & 0 deletions setup.cfg
Expand Up @@ -21,6 +21,7 @@ disable = duplicate-code,too-many-arguments,too-many-public-methods,too-few-publ
missing-class-docstring,too-many-locals,too-many-lines,too-many-branches,
too-many-statements
enable=useless-suppression ; Warns about unused pylint ignores
exclude-protected=_unfrozen

[tool:pytest]
testpaths = tests
Expand Down