Matt Wozniski 32a0faba43 gh-119517: Fixes for pasting in pyrepl (#120253) ... * Remove pyrepl's optimization for self-insert This will be replaced by a less specialized optimization. * Use line-buffering when pyrepl echoes pastes Previously echoing was totally suppressed until the entire command had been pasted and the terminal ended paste mode, but this gives the user no feedback to indicate that an operation is in progress. Drawing something to the screen once per line strikes a balance between perceived responsiveness and performance. * Remove dead code from pyrepl `msg_at_bottom` is always true. * Speed up pyrepl's screen rendering computation The Reader in pyrepl doesn't hold a complete representation of the screen area being drawn as persistent state. Instead, it recomputes it, on each keypress. This is fast enough for a few hundred bytes, but incredibly slow as the input buffer grows into the kilobytes (likely because of pasting). Rather than making some expensive and expansive changes to the repl's internal representation of the screen, add some caching: remember some data from one refresh to the next about what was drawn to the screen and, if we don't find anything that has invalidated the results that were computed last time around, reuse them. To keep this caching as simple as possible, all we'll do is look for lines in the buffer that were above the cursor the last time we were asked to update the screen, and that are still above the cursor now. We assume that nothing can affect a line that comes before both the old and new cursor location without us being informed. Based on this assumption, we can reuse old lines, which drastically speeds up the overwhelmingly common case where the user is typing near the end of the buffer. * Speed up pyrepl prompt drawing Cache the `can_colorize()` call rather than repeatedly recomputing it. This call looks up an environment variable, and is called once per character typed at the REPL. The environment variable lookup shows up as a hot spot when profiling, and we don't expect this to change while the REPL is running. * Speed up pasting multiple lines into the REPL Previously, we were checking whether the command should be accepted each time a line break was encountered, but that's not the expected behavior. In bracketed paste mode, we expect everything pasted to be part of a single block of code, and encountering a newline shouldn't behave like a user pressing <Enter> to execute a command. The user should always have a chance to review the pasted command before running it. * Use a read buffer for input in pyrepl Previously we were reading one byte at a time, which causes much slower IO than necessary. Instead, read in chunks, processing previously read data before asking for more. * Optimize finding width of a single character `wlen` finds the width of a multi-character string by adding up the width of each character, and then subtracting the width of any escape sequences. It's often called for single character strings, however, which can't possibly contain escape sequences. Optimize for that case. * Optimize disp_str for ASCII characters Since every ASCII character is known to display as single width, we can avoid not only the Unicode data lookup in `disp_str` but also the one hidden in `str_width` for them. * Speed up cursor movements in long pyrepl commands When the current pyrepl command buffer contains many lines, scrolling up becomes slow. We have optimizations in place to reuse lines above the cursor position from one refresh to the next, but don't currently try to reuse lines below the cursor position in the same way, so we wind up with quadratic behavior where all lines of the buffer below the cursor are recomputed each time the cursor moves up another line. Optimize this by only computing one screen's worth of lines beyond the cursor position. Any lines beyond that can't possibly be shown by the console, and bounding this makes scrolling up have linear time complexity instead. --------- Signed-off-by: Matt Wozniski <mwozniski@bloomberg.net> Co-authored-by: Pablo Galindo <pablogsal@gmail.com>		2024-06-11 16:42:10 +00:00
..
__phello__
_pyrepl	gh-119517: Fixes for pasting in pyrepl (#120253)	2024-06-11 16:42:10 +00:00
asyncio	gh-113892: Add a extra check to ProactorEventLoop.sock_connect to ensure that the given socket is in non-blocking mode (#119519)	2024-06-01 09:05:19 -07:00
collections	gh-118932: ChainMap.__contains__ performance improvement (gh-118946)	2024-05-11 15:55:23 -05:00
concurrent	gh-120121: Add InvalidStateError to concurrent.futures.__all__ (#120123)	2024-06-08 21:41:19 +05:30
ctypes	gh-87969: Align docs and docstrings with implementation for ctypes' [w]string_at() (#25384)	2024-04-18 14:57:37 +00:00
curses	gh-60436: fix curses textbox backspace/del (#103783)	2023-04-26 22:54:07 +02:00
dbm	gh-100414: Make dbm.sqlite3 the preferred dbm backend (#115447)	2024-02-14 13:47:19 +00:00
email	Remove almost all unpaired backticks in docstrings (#119231)	2024-05-22 12:35:18 -04:00
encodings	gh-85287: Change codecs to raise precise UnicodeEncodeError and UnicodeDecodeError (#113674)	2024-03-17 04:58:42 +00:00
ensurepip	gh-114965: Updated bundled pip to 24.0 (gh-114966)	2024-02-03 17:45:09 +00:00
html	gh-100210: Correct the comment link for unescaping HTML (#100212)	2023-02-19 11:18:12 +01:00
http	Remove almost all unpaired backticks in docstrings (#119231)	2024-05-22 12:35:18 -04:00
idlelib	gh-119174: Fix high DPI causes turtledemo(turtle-graphics examples) windows blurry (#119175)	2024-05-20 23:32:00 -04:00
importlib	gh-106531: Apply changes from importlib_resources 6.3.2 (#117054)	2024-06-04 06:36:28 +00:00
json	gh-95382: Improve performance of json encoder with indent (GH-118105)	2024-05-06 11:04:39 +03:00
logging	gh-119819: Update logging configuration to support joinable multiproc… (GH-120090)	2024-06-05 07:25:47 +01:00
multiprocessing	gh-103134: Update multiprocessing.managers.ListProxy and DictProxy (GH-103133)	2024-05-20 14:28:36 +00:00
pathlib	GH-116380: Move pathlib-specific code from glob to pathlib._abc. (#120011)	2024-06-07 17:59:34 +01:00
pydoc_data	Python 3.13.0b1	2024-05-08 11:21:00 +02:00
re	GH-83162: Rename re.error for better clarity. (#101677)	2023-12-11 15:45:08 -05:00
site-packages
sqlite3	gh-118924: Remove sqlite3.version and sqlite3.version_info (#118925)	2024-05-10 20:42:34 +00:00
sysconfig	gh-116622: Android sysconfig updates (#118352)	2024-05-01 16:47:54 +00:00
test	gh-119180: PEP 649 compiler changes (#119361)	2024-06-11 13:06:49 +00:00
tkinter	gh-120211: Fix tkinter.ttk with Tcl/Tk 9.0 (GH-120213)	2024-06-07 10:49:07 +00:00
tomllib
turtledemo	gh-118673: Remove shebang and executable bits from stdlib modules. (#119658)	2024-05-29 12:43:19 -04:00
unittest	gh-65454: avoid triggering call to a PropertyMock in NonCallableMock.__setattr__ (#120019)	2024-06-11 05:42:49 +00:00
urllib	gh-118827: Remove Quoter from urllib.parse (#118828)	2024-06-03 10:50:29 +03:00
venv	gh-90329: Add _winapi.GetLongPathName and GetShortPathName and use in venv to reduce warnings (GH-117817)	2024-04-15 15:36:06 +01:00
wsgiref	Remove almost all unpaired backticks in docstrings (#119231)	2024-05-22 12:35:18 -04:00
xml	gh-119577: Adjust DeprecationWarning when testing element truth values in ElementTree (GH-119762)	2024-06-06 20:18:30 -07:00
xmlrpc	xmlrpc.client uses datetime.datetime.isoformat() (#105741)	2023-06-14 17:00:40 +02:00
zipfile	gh-119588: Implement zipfile.Path.is_symlink (zipp 3.19.0). (#119591)	2024-06-03 11:13:07 -04:00
zoneinfo	gh-106233: Fix stacklevel in zoneinfo.InvalidTZPathWarning (GH-106234)	2024-02-06 15:08:56 +02:00
__future__.py
__hello__.py
_aix_support.py
_android_support.py	gh-116622: Redirect stdout and stderr to system log when embedded in an Android app (#118063)	2024-04-30 16:00:31 +02:00
_collections_abc.py	gh-118803: Remove ByteString from typing and collections.abc (#118804)	2024-05-09 00:37:55 +01:00
_colorize.py	gh-117225: Move colorize functionality to own internal module (#118283)	2024-05-01 12:27:06 -06:00
_compat_pickle.py	gh-75552: Remove deprecated tkinter.tix module (GH-104902)	2023-05-27 12:34:19 -05:00
_compression.py
_ios_support.py	gh-119253: use ImportError in _ios_support (#119254)	2024-05-20 16:39:30 -04:00
_markupbase.py
_opcode_metadata.py	gh-119676: remove several pseudo instructions which are use only in codegen (#119677)	2024-05-28 19:05:38 +00:00
_osx_support.py	gh-102362: Fix macOS version number in result of sysconfig.get_platform() (GH-112942)	2023-12-18 18:51:58 -05:00
_py_abc.py
_pydatetime.py	gh-120268: Prohibit passing `None to _pydatetime.date.fromtimestamp` (#120269)	2024-06-08 16:45:57 -04:00
_pydecimal.py	gh-118164: str(10**10000) hangs if the C _decimal module is missing (#118503)	2024-05-04 18:22:33 -05:00
_pyio.py	gh-95782: Fix io.BufferedReader.tell() etc. being able to return offsets < 0 (GH-99709)	2024-02-17 11:16:06 +00:00
_pylong.py	gh-119057: Use better error messages for zero division (#119066)	2024-06-03 19:03:56 +03:00
_sitebuiltins.py
_strptime.py	GH-70647: Deprecate strptime day of month parsing without a year present to avoid leap-year bugs (GH-117107)	2024-04-03 14:19:49 +02:00
_threading_local.py
_weakrefset.py
abc.py	gh-87864: Use correct function definition syntax in the docs (#103312)	2023-04-11 16:50:25 +03:00
antigravity.py
argparse.py	gh-118805: Remove type, choices, metavar params of BooleanOptionalAction (#118806)	2024-05-09 11:46:45 +00:00
ast.py	gh-117865: Defer import of re in ast (#119546)	2024-05-28 11:04:08 -07:00
base64.py	gh-118673: Remove shebang and executable bits from stdlib modules. (#119658)	2024-05-29 12:43:19 -04:00
bdb.py	gh-58933: Make pdb return to caller frame correctly when f_trace is not set (#118979)	2024-05-13 13:38:21 +01:00
bisect.py	GH-102833: Mention the key function in the docstrings (GH-103009)	2023-03-25 02:19:20 -05:00
bz2.py	gh-115961: Add name and mode attributes for compressed file-like objects (GH-116036)	2024-04-21 11:46:39 +03:00
calendar.py	gh-112240: Add option to calendar module CLI to specify the weekday to start each week (GH-112241)	2024-01-29 16:58:21 +00:00
cmd.py	Remove almost all unpaired backticks in docstrings (#119231)	2024-05-22 12:35:18 -04:00
code.py	gh-118893: Evaluate all statements in the new REPL separately (#119318)	2024-05-21 23:16:56 +00:00
codecs.py	gh-66143: Allow copying and pickling of CodecInfo object (GH-109235)	2023-09-29 20:07:09 +03:00
codeop.py	gh-113744: Add a new IncompleteInputError exception to improve incomplete input detection in the codeop module (#113745)	2024-01-30 16:21:30 +00:00
colorsys.py	gh-106498: Revert incorrect colorsys.rgb_to_hls change (#106627)	2023-07-11 11:07:20 -04:00
compileall.py	gh-117205: Increase chunksize when compiling pyc in parallel (#117206)	2024-04-03 15:24:24 -07:00
configparser.py	Remove almost all unpaired backticks in docstrings (#119231)	2024-05-22 12:35:18 -04:00
contextlib.py	gh-103791: handle BaseExceptionGroup in contextlib.suppress() (#111910)	2023-11-10 13:32:36 +00:00
contextvars.py
copy.py	gh-114264: Optimize performance of copy.deepcopy by adding a fast path for atomic types (GH-114266)	2024-06-07 18:42:01 +03:00
copyreg.py
cProfile.py	gh-118673: Remove shebang and executable bits from stdlib modules. (#119658)	2024-05-29 12:43:19 -04:00
csv.py	gh-114628: Display csv.Error without context (#115005)	2024-02-04 20:57:54 -05:00
dataclasses.py	gh-118033: Fix __weakref__ not set for generic dataclasses (#118099)	2024-05-09 11:36:17 +03:00
datetime.py	gh-84976: Add back UTC to datetime.__all__ (#104920)	2023-05-25 11:18:56 -04:00
decimal.py	gh-71966: Move the module docstring from _pydecimal to decimal (GH-117919)	2024-04-17 10:18:24 +03:00
difflib.py	gh-115801: Only allow sequence of strings as input for difflib.unified_diff (GH-118333)	2024-06-10 14:06:18 +03:00
dis.py	gh-119180: Add LOAD_COMMON_CONSTANT opcode (#119321)	2024-05-22 00:46:39 +00:00
doctest.py	Remove almost all unpaired backticks in docstrings (#119231)	2024-05-22 12:35:18 -04:00
enum.py	gh-118650: Exclude _repr_* methods from Enum's _sunder_ reservation (GH-118651)	2024-05-07 12:35:51 +02:00
filecmp.py	gh-57141: Add dircmp shallow option (GH-109499)	2024-03-04 17:27:43 +00:00
fileinput.py	Use bool in fileinput.input() docstring and tests for the inplace argument (GH-111998)	2024-01-27 23:47:55 +02:00
fnmatch.py	GH-72904: Add glob.translate() function (#106703)	2023-11-13 17:15:56 +00:00
fractions.py	gh-119838: Treat Fraction as a real value in mixed arithmetic operations with complex (GH-119839)	2024-06-03 12:29:01 +03:00
ftplib.py	Remove almost all unpaired backticks in docstrings (#119231)	2024-05-22 12:35:18 -04:00
functools.py	GH-100242: bring functools.py partial implementation more in line with C code (GH-100244)	2024-04-17 15:34:46 +02:00
genericpath.py	gh-117114: Make os.path.isdevdrive available on all platforms (GH-117115)	2024-03-25 22:55:11 +00:00
getopt.py	Remove almost all unpaired backticks in docstrings (#119231)	2024-05-22 12:35:18 -04:00
getpass.py	gh-76912: Raise OSError from any failure in getpass.getuser() (#29739)	2023-11-27 10:05:55 -08:00
gettext.py	gh-88434: Emit deprecation warnings for non-integer numbers in gettext if translation not found (GH-110574)	2023-10-14 09:07:02 +03:00
glob.py	GH-116380: Move pathlib-specific code from glob to pathlib._abc. (#120011)	2024-06-07 17:59:34 +01:00
graphlib.py
gzip.py	gh-115961: Add name and mode attributes for compressed file-like objects (GH-116036)	2024-04-21 11:46:39 +03:00
hashlib.py	gh-99108: Refactor _sha256 & _sha512 into _sha2. (#101924)	2023-02-15 22:08:20 -08:00
heapq.py	gh-119721: Integrate documentation fixes into heapq module docstring. (gh-119722)	2024-05-29 11:39:34 -05:00
hmac.py	gh-112999: Replace the outdated "deprecated" directives with "versionchanged" (GH-113000)	2023-12-12 18:31:04 +02:00
imaplib.py	Remove almost all unpaired backticks in docstrings (#119231)	2024-05-22 12:35:18 -04:00
inspect.py	gh-119180: PEP 649 compiler changes (#119361)	2024-06-11 13:06:49 +00:00
io.py	gh-111356: io: Add missing documented objects to io.__all__ (#111370)	2023-11-10 16:18:52 +09:00
ipaddress.py	gh-120128: fix description of argument to ipaddress.collapse_addresses() (#120131)	2024-06-06 00:52:40 +03:00
keyword.py	gh-103763: Implement PEP 695 (#103764)	2023-05-15 20:36:23 -07:00
linecache.py	linecache: Fix docstring location (#117948)	2024-04-16 15:37:18 -07:00
locale.py	gh-91565: Replace bugs.python.org links with Devguide/GitHub ones (GH-91568)	2024-04-01 13:02:07 +00:00
lzma.py	gh-115961: Add name and mode attributes for compressed file-like objects (GH-116036)	2024-04-21 11:46:39 +03:00
mailbox.py	gh-117467: Add preserving of mailbox owner on flush (GH-117510)	2024-04-04 13:32:53 +03:00
mimetypes.py	Remove almost all unpaired backticks in docstrings (#119231)	2024-05-22 12:35:18 -04:00
modulefinder.py	gh-114099 - Add iOS framework loading machinery. (GH-116454)	2024-03-19 08:36:19 -04:00
netrc.py
ntpath.py	gh-118263: Add additional arguments to path_t (Argument Clinic type) in posixmodule (GH-118355)	2024-05-24 19:04:17 +01:00
nturl2path.py
numbers.py
opcode.py	gh-119180: Add LOAD_COMMON_CONSTANT opcode (#119321)	2024-05-22 00:46:39 +00:00
operator.py	gh-118285: Fix signatures of operator.{attrgetter,itemgetter,methodcaller} instances (GH-118316)	2024-04-29 19:30:48 +03:00
optparse.py
os.py	gh-120057: Add os.environ.refresh() method (#120059)	2024-06-10 16:34:17 +00:00
pdb.py	gh-118673: Remove shebang and executable bits from stdlib modules. (#119658)	2024-05-29 12:43:19 -04:00
pickle.py	gh-85567: Fix resouce warnings in pickle and pickletools CLIs (GH-113618)	2024-01-06 00:12:34 +02:00
pickletools.py	gh-115146: Fix typo in pickletools.py documentation (GH-115148)	2024-02-08 10:12:58 +02:00
pkgutil.py	gh-97850: Deprecate find_loader and get_loader in pkgutil (GH-98520)	2023-05-03 16:11:54 -07:00
platform.py	gh-118673: Remove shebang and executable bits from stdlib modules. (#119658)	2024-05-29 12:43:19 -04:00
plistlib.py	gh-111803: Support loading more deeply nested lists in binary plist format (GH-114024)	2024-01-13 15:26:55 +02:00
poplib.py	gh-106200: Remove unused imports (#106201)	2023-06-28 11:55:41 +00:00
posixpath.py	pathlib ABCs: remove duplicate realpath() implementation. (#119178)	2024-06-05 18:54:50 +01:00
pprint.py	[pprint]: Add docstring about PrettyPrinter.underscore_numbers parameter (#112963)	2023-12-13 12:04:17 +00:00
profile.py	gh-118673: Remove shebang and executable bits from stdlib modules. (#119658)	2024-05-29 12:43:19 -04:00
pstats.py	GH-83162: Rename re.error for better clarity. (#101677)	2023-12-11 15:45:08 -05:00
pty.py	gh-118824: Remove deprecated master_open and slave_open from pty (#118826)	2024-05-28 16:42:35 +03:00
py_compile.py
pyclbr.py
pydoc.py	bpo-37755: Use configured output in pydoc instead of pager (GH-15105)	2024-06-08 09:19:13 +00:00
queue.py	gh-117531: Unblock getters after non-immediate queue shutdown (#117532)	2024-04-10 08:01:42 -07:00
quopri.py	gh-118673: Remove shebang and executable bits from stdlib modules. (#119658)	2024-05-29 12:43:19 -04:00
random.py	gh-118131: Command-line interface for the random module (#118132)	2024-05-05 06:30:03 +00:00
reprlib.py	gh-109818: reprlib.recursive_repr copies __type_params__ (#109819)	2023-09-27 19:26:42 -07:00
rlcompleter.py	gh-113978: Ignore warnings on text completion inside REPL (#113979)	2024-05-21 18:28:21 +02:00
runpy.py	gh-99437: runpy: decode path-like objects before setting globals	2024-01-15 16:58:50 +00:00
sched.py
secrets.py
selectors.py	gh-110038: KqueueSelector must count all read/write events (#110039)	2023-09-28 17:25:10 +00:00
shelve.py	gh-107089: Improve Shelf.clear method performance (gh-107090)	2023-07-29 09:08:11 +09:00
shlex.py
shutil.py	GH-89727: Fix shutil.rmtree() recursion error on deep trees (#119808)	2024-06-01 19:49:12 +01:00
signal.py	gh-112559: Avoid unnecessary conversion attempts to enum_klass in signal.py (#113040)	2023-12-23 17:07:52 -08:00
site.py	GH-119496: accept UTF-8 BOM in .pth files (GH-119503)	2024-05-24 14:29:19 +00:00
smtplib.py	gh-118673: Remove shebang and executable bits from stdlib modules. (#119658)	2024-05-29 12:43:19 -04:00
socket.py	gh-110383: Document socket.makefile() accepts combined modes (#119150)	2024-05-21 16:23:50 +00:00
socketserver.py	gh-103673: Add missing ForkingUnixStreamServer and ForkingUnixDatagramServer socketservers (#103674)	2023-04-24 22:35:52 +00:00
sre_compile.py
sre_constants.py
sre_parse.py
ssl.py	gh-107361: strengthen default SSL context flags (#112389)	2024-03-06 13:44:58 -08:00
stat.py	gh-113666: Adding missing UF_ and SF_ flags to module 'stat' (#113667)	2024-01-15 12:22:43 +01:00
statistics.py	Refactor (mostly rearrange) the statistics module (gh-119930)	2024-06-01 22:07:46 -05:00
string.py
stringprep.py
struct.py
subprocess.py	GH-118844: Fix build failures when combining --disable-gil with --enable-experimental-jit (GH-118935)	2024-05-11 16:19:31 -07:00
symtable.py	gh-119180: PEP 649 compiler changes (#119361)	2024-06-11 13:06:49 +00:00
tabnanny.py	gh-118673: Remove shebang and executable bits from stdlib modules. (#119658)	2024-05-29 12:43:19 -04:00
tarfile.py	gh-118673: Remove shebang and executable bits from stdlib modules. (#119658)	2024-05-29 12:43:19 -04:00
tempfile.py	gh-59616: Support os.chmod(follow_symlinks=True) and os.lchmod() on Windows (GH-113049)	2023-12-14 13:28:37 +02:00
textwrap.py	gh-107369: optimize textwrap.indent() (#107374)	2023-07-29 06:37:23 +00:00
this.py
threading.py	gh-114271: Fix race in Thread.join() (#114839)	2024-03-16 13:56:30 +01:00
timeit.py	gh-118673: Remove shebang and executable bits from stdlib modules. (#119658)	2024-05-29 12:43:19 -04:00
token.py	gh-107015: Remove async_hacks from the tokenizer (#107018)	2023-07-26 16:34:15 +01:00
tokenize.py	gh-115154: Fix untokenize handling of unicode named literals (#115171)	2024-02-19 14:54:10 +00:00
trace.py	gh-118673: Remove shebang and executable bits from stdlib modules. (#119658)	2024-05-29 12:43:19 -04:00
traceback.py	gh-99180: Make StackSummary.should_show_carets private (#119554)	2024-05-25 17:08:32 +00:00
tracemalloc.py
tty.py	gh-114328: tty cbreak mode should not alter ICRNL (#114335)	2024-01-21 15:25:52 -08:00
turtle.py	no-issue: Capitalise 'PhotoImage' (gh-108958)	2023-09-25 21:37:40 +09:00
types.py	gh-109599: Add types.CapsuleType (#109600)	2023-09-25 19:50:39 +02:00
typing.py	gh-119180: PEP 649 compiler changes (#119361)	2024-06-11 13:06:49 +00:00
uuid.py	gh-113308: Remove some internal parts of uuid module (#115934)	2024-03-14 13:01:41 +03:00
warnings.py	gh-117535: Change unknown filename of warnings from sys to <sys> (#118018)	2024-04-18 20:50:09 -07:00
wave.py	gh-105096: Deprecate wave getmarkers() method (#105098)	2023-05-31 12:09:41 +00:00
weakref.py
webbrowser.py	gh-118673: Remove shebang and executable bits from stdlib modules. (#119658)	2024-05-29 12:43:19 -04:00
zipapp.py	gh-104527: zippapp will now avoid appending an archive to itself. (gh-106076)	2023-06-26 10:09:08 +01:00
zipimport.py	Remove references to private symbols from zipimport module docstring (GH-119015)	2024-05-15 11:21:52 -05:00