cpython/Lib
Gregory P. Smith 133bf0927e
[3.11] gh-103861: Fix Zip64 extensions not being properly applied in some cases (GH-103863) (#104534)
Fix Zip64 extensions not being properly applied in some cases:

Fixes an issue where adding a small file to a `ZipFile`
object while forcing zip64 extensions causes an extra Zip64 record to be
added to the zip, but doesn't update the `min_version` or file sizes in
the primary central directory header.

Also fixed an edge case in checking if zip64 extensions are required:

This fixes an issue where if data requiring zip64 extensions was added
to an unseekable stream without specifying `force_zip64=True`, zip64
extensions would not be used and a RuntimeError would not be raised when
closing the file (even though the size would be known at that point).
This would result in successfully writing corrupt zip files.

Deciding if zip64 extensions are required outside of the `FileHeader`
function means that both `FileHeader` and `_ZipWriteFile` will always be
in sync. Previously, the `FileHeader` function could enable zip64
extensions without propagating that decision to the `_ZipWriteFile`
class, which would then not correctly write the data descriptor record
or check for errors on close.

If anyone is actually using `ZipInfo.FileHeader` as a public API without
explicitly passing True or False in for zip64, their own code may still be
susceptible to that kind of bug unless they make a similar change to
where the zip64 decision happens.

Fixes GH-103861

---------

.
(cherry picked from commit 798bcaa1eb)

Co-authored-by: Carey Metcalfe <carey@cmetcalfe.ca>
2023-05-17 01:07:10 -07:00
..
__phello__ bpo-45020: Add more test cases for frozen modules. (gh-28664) 2021-09-30 18:38:52 -06:00
asyncio [3.11] gh-98539: Make _SSLTransportProtocol.abort() safe to call when closed (GH-104474) (#104485) 2023-05-14 21:24:07 +00:00
collections bpo-26579: Add object.__getstate__(). (GH-2821) 2022-04-06 20:00:14 +03:00
concurrent gh-94440: Fix issue of ProcessPoolExecutor shutdown hanging (GH-94468) 2023-03-15 22:09:20 -07:00
ctypes [3.11] gh-99952: fix refcount issues in ctypes.Structure from_param() result (#101339) 2023-01-26 01:02:37 -08:00
curses
dbm bpo-40563: Support pathlike objects on dbm/shelve (GH-21849) 2021-09-10 15:26:16 +03:00
distutils [3.11] gh-95853: Add script to automate WASM build (GH-95828, GH-95985, GH-96045, GH-96389, GH-96744) (GH-96749) 2022-09-13 12:06:39 +02:00
email gh-101021: Document binary parameters as bytes (GH-101024) 2023-01-14 12:22:08 -08:00
encodings [3.11] gh-98433: Fix quadratic time idna decoding. (GH-99092) (#99222) 2022-11-07 18:57:10 -08:00
ensurepip [3.11] gh-103872: Update bundled pip version to 23.1.2 (gh-103874) 2023-04-26 10:50:34 +00:00
html [3.11] gh-100210: Correct the comment link for unescaping HTML (GH-100212) (#102044) 2023-02-21 07:52:02 +08:00
http [3.11] gh-103204: http.server - Enforce that HTTP version numbers must consist only of digits (GH-103205) (#104438) 2023-05-12 20:54:12 +00:00
idlelib [3.11] GH-71383: IDLE - Document testing subsets of modules (GH-104463) (#104464) 2023-05-14 04:18:30 +00:00
importlib [3.11] GH-102700: allow built-in modules to be submodules (GH-103162) (GH-103322) 2023-04-06 16:06:02 -07:00
json gh-96959: Update HTTP links which are redirected to HTTPS (GH-96961) 2022-09-25 01:20:45 -07:00
lib2to3 gh-90473: Misc test fixes for WASI (GH-93218) 2022-05-25 07:24:32 -07:00
logging [3.11] gh-99811: Use correct variable to search for time in format string (GH-99812) (GH-99851) 2022-11-28 19:16:49 +00:00
msilib gh-91217: deprecate msilib (GH-91515) 2022-04-14 12:50:11 -07:00
multiprocessing [3.11] bpo-40882: Fix a memory leak in SharedMemory on Windows (GH-20684) (#99973) 2022-12-05 13:38:25 +01:00
pydoc_data Python 3.11.3 2023-04-04 23:22:17 +01:00
re gh-91404: Revert "bpo-23689: re module, fix memory leak when a match is terminated by a signal or allocation failure (GH-32283) (GH-93882) 2022-06-17 01:43:56 -07:00
site-packages
sqlite3 gh-79009: sqlite3.iterdump now correctly handles tables with autoincrement (GH-9621) 2022-06-19 16:25:13 -07:00
test [3.11] gh-103861: Fix Zip64 extensions not being properly applied in some cases (GH-103863) (#104534) 2023-05-17 01:07:10 -07:00
tkinter [3.11] gh-104494: Update certain Tkinter pack/place tests for Tk 8.7 errors (GH-104495) (#104569) 2023-05-16 20:03:51 -04:00
tomllib bpo-40059: Add tomllib (PEP-680) (GH-31498) 2022-03-08 09:26:13 +01:00
turtledemo [3.11] Rewrite the turtledemo makeGraphFrame method (GH-104224) (#104238) 2023-05-06 15:31:28 +00:00
unittest gh-103329: Add regression test for PropertyMock with side effect (GH-103358) 2023-04-08 00:01:20 -07:00
urllib [3.11] gh-103848: Adds checks to ensure that bracketed hosts found by urlsplit are of IPv6 or IPvFuture format (GH-103849) (#104349) 2023-05-10 06:35:24 +00:00
venv [3.11] gh-98741: Remove useless check_home usage from is_python_build usage (GH-98743) (GH-98746) 2022-10-26 23:35:54 +01:00
wsgiref gh-96959: Update HTTP links which are redirected to HTTPS (GH-96961) 2022-09-25 01:20:45 -07:00
xml gh-96175: add missing self._localName assignment in xml.dom.minidom.Attr (GH-96176) 2022-08-23 10:18:19 -07:00
xmlrpc bpo-47126: Update to canonical PEP URLs specified by PEP 676 (GH-32124) 2022-03-30 12:00:27 +01:00
zoneinfo [3.11] Fix typos in documentation and comments (GH-102374) (#102376) 2023-03-02 16:00:06 +00:00
__future__.py gh-93626: Set the release for __future__.annotations to None (GH-93628) (GH-94553) 2022-07-05 11:16:32 +02:00
__hello__.py bpo-47084: Clear Unicode cached representations on finalization (GH-32032) 2022-03-22 13:53:51 +01:00
_aix_support.py
_bootsubprocess.py
_collections_abc.py Minor code nit: Move an unrelated statement out of a try clause in Sequence.index (GH-32330) 2022-04-06 13:03:36 -05:00
_compat_pickle.py bpo-46565: del loop vars that are leaking into module namespaces (GH-30993) 2022-02-03 11:20:08 +02:00
_compression.py
_markupbase.py
_osx_support.py [codemod] Fix non-matching bracket pairs (GH-28473) 2021-09-22 01:09:00 +02:00
_py_abc.py
_pydecimal.py gh-91291: Accept attributes as keyword arguments in decimal.localcontext (#32242) 2022-04-21 21:27:15 -07:00
_pyio.py gh-93099: Fix _pyio to use locale module properly (gh-93136) 2022-05-23 18:03:37 -07:00
_sitebuiltins.py
_strptime.py
_threading_local.py
_weakrefset.py bpo-26579: Add object.__getstate__(). (GH-2821) 2022-04-06 20:00:14 +03:00
abc.py gh-87864: Use correct function definition syntax in the docs (GH-103312) 2023-04-11 07:19:33 -07:00
aifc.py gh-47061: Deprecate chunk (GH-91419) 2022-04-11 15:02:41 -07:00
antigravity.py
argparse.py [3.11] gh-101640: Make argparse _print_message catch any write error (GH-101802) (#104250) 2023-05-06 23:17:27 +00:00
ast.py gh-99341: Cover type ignore nodes when incrementing line numbers (GH-99422) 2022-11-22 03:03:26 -08:00
asynchat.py bpo-47061: use warnings._deprecated() with asynchat, asyncore, and smtpd (GH-32350) 2022-04-06 11:22:39 -07:00
asyncore.py bpo-47061: use warnings._deprecated() with asynchat, asyncore, and smtpd (GH-32350) 2022-04-06 11:22:39 -07:00
base64.py bpo-35970: Add help flag to base64 module (GH-28774) 2021-10-06 18:38:43 -07:00
bdb.py [3.11] gh-101517: fix line number propagation in code generated for except* (#103550) (#103816) 2023-04-25 18:08:54 +01:00
bisect.py
bz2.py bpo-45475: Revert __iter__ optimization for GzipFile, BZ2File, and LZMAFile. (GH-29016) 2021-10-19 11:51:48 +09:00
calendar.py bpo-46659: Enhance LocaleTextCalendar for C locale (GH-31214) 2022-02-24 14:29:08 +01:00
cgi.py bpo-47061: deprecate cgi and cgitb (GH-32410) 2022-04-08 17:15:35 -07:00
cgitb.py bpo-47061: deprecate cgi and cgitb (GH-32410) 2022-04-08 17:15:35 -07:00
chunk.py gh-47061: Deprecate chunk (GH-91419) 2022-04-11 15:02:41 -07:00
cmd.py gh-67248: cmd: Sort miscellaneous help topics (#92254) 2022-05-03 21:36:52 -06:00
code.py
codecs.py gh-51511: Note that codecs.open()'s encoding parameter affects automatic conversion to binary mode (GH-94370) 2022-10-21 16:27:41 -07:00
codeop.py gh-96052: codeop: fix handling compiler warnings in incomplete input (GH-96132) 2022-09-25 01:21:00 -07:00
colorsys.py Improve consistency of colorsys.rgb_to_hsv (GH-27277) 2021-07-23 09:59:30 -03:00
compileall.py bpo-40447: accept all path-like objects in compileall.compile_file (GH-19883) 2022-12-23 12:43:13 -08:00
configparser.py [3.11] gh-100520: Fix rst markup in configparser docstrings (GH-100524) (#100533) 2022-12-26 21:13:11 +05:30
contextlib.py gh-95882: fix regression in the traceback of exceptions propagated from inside a contextlib context manager (GH-95883) 2023-01-03 08:18:45 -08:00
contextvars.py
copy.py gh-90494: Reject 6th element of the __reduce__() tuple (GH-93609) (GH-93631) 2022-06-10 16:00:19 +02:00
copyreg.py bpo-26579: Add object.__getstate__(). (GH-2821) 2022-04-06 20:00:14 +03:00
cProfile.py gh-103935: Use io.open_code() when executing code in trace and profile modules (GH-103947) 2023-04-27 21:23:46 +00:00
crypt.py gh-95231: Disable md5 & crypt modules if FIPS is enabled (GH-94742) 2022-08-15 08:37:51 -07:00
csv.py bpo-43625: Enhance csv sniffer has_headers() to be more accurate (GH-26939) 2021-07-30 19:10:37 +02:00
dataclasses.py [3.11] gh-104035: Do not ignore user-defined __{get,set}state__ in slotted frozen dataclasses (GH-104041) (#104044) 2023-05-01 10:02:41 -06:00
datetime.py [3.11] gh-98378: Add small format string example to strftime comments (GH-98379) (#98395) 2022-10-22 13:00:26 +01:00
decimal.py
difflib.py Correct method name typo (#91970) 2022-04-27 15:28:56 -06:00
dis.py bpo-40222: Mark exception table function in the dis module as private (GH-95961) 2022-08-14 08:08:04 -07:00
doctest.py gh-99433: Fix doctest failure on types.MethodWrapperType (GH-99434) 2022-12-30 04:18:54 -08:00
enum.py [3.11] gh-103479: [Enum] require __new__ to be considered a data type (GH-103495) (GH-103514) 2023-04-13 12:04:06 -07:00
filecmp.py bpo-42958: Improve description of shallow= in filecmp.cmp docs (GH-27166) 2021-08-04 21:39:45 +02:00
fileinput.py [3.11] gh-101961 fileinput.hookcompressed should not set the encoding value for the binary mode (gh-102068) (#102098) 2023-02-21 12:39:16 +09:00
fnmatch.py gh-89973: Fix re.error in the fnmatch module. (GH-93072) 2022-06-05 02:39:11 -07:00
fractions.py bpo-44547: Make Fractions objects instances of typing.SupportsInt (GH-27851) 2021-10-22 00:09:47 +02:00
ftplib.py
functools.py gh-89828: Do not relay the __class__ attribute in GenericAlias (GH-93754) 2022-06-18 07:41:25 -07:00
genericpath.py
getopt.py
getpass.py
gettext.py
glob.py bpo-37578: glob.glob -- added include_hidden parameter (GH-30153) 2021-12-18 06:23:34 -08:00
graphlib.py bpo-45359: Support TopologicalSorter type subscript (GH-28714) 2021-12-08 20:52:57 +02:00
gzip.py gh-90839: Forward gzip.compress() compresslevel to zlib (gh-31215) 2022-04-12 22:46:40 +09:00
hashlib.py bpo-45150: Add hashlib.file_digest() for efficient file hashing (GH-31930) 2022-03-22 02:37:00 -07:00
heapq.py Update: usage doc for heappushpop (GH-91451) 2022-04-17 23:12:33 -05:00
hmac.py
imaplib.py
imghdr.py gh-91217: deprecate imghdr (#91461) 2022-04-13 10:47:41 -07:00
imp.py bpo-45019: Do some cleanup related to frozen modules. (gh-28319) 2021-09-13 16:18:37 -06:00
inspect.py [3.11] gh-75367: Fix data descriptor detection in inspect.getattr_static (GH-104517) (#104557) 2023-05-16 17:57:34 +00:00
io.py [3.11] io: Add missing f (#96701) 2022-09-09 10:28:26 +01:00
ipaddress.py gh-82836: fix private network check (GH-97733) 2022-11-29 14:12:57 -08:00
keyword.py
linecache.py gh-92336: linecache.getline should not raise exceptions on decoding errors (GH-94410) 2022-06-30 02:59:33 -07:00
locale.py gh-90817: Deprecate explicitly locale.resetlocale() (GH-93196) 2022-05-25 13:29:58 -07:00
lzma.py bpo-45475: Revert __iter__ optimization for GzipFile, BZ2File, and LZMAFile. (GH-29016) 2021-10-19 11:51:48 +09:00
mailbox.py
mailcap.py [3.11] gh-68966: Make mailcap refuse to match unsafe filenames/types/params (GH-91993) (GH-93458) 2022-06-03 08:25:58 -07:00
mimetypes.py bpo-45639: Add webp and avif image formats to mimetypes (#29259) 2022-05-03 15:17:57 -06:00
modulefinder.py bpo-45017: move opcode-related logic from modulefinder to dis (GH-28246) 2021-09-09 14:04:12 +01:00
netrc.py bpo-28806: Continue work: improve the netrc library (GH-26330) 2021-11-17 11:07:54 +02:00
nntplib.py gh-91217: deprecate nntplib (GH-91543) 2022-04-15 12:32:56 -07:00
ntpath.py [3.11] GH-88013: Fix TypeError raised by ntpath.realpath in some cases (GH-102813, GH-103343) 2023-04-07 21:38:56 +01:00
nturl2path.py
numbers.py [3.11] gh-68163: Correct conversion of Rational instances to float (GH-25619) (#96556) 2022-09-08 12:03:53 +01:00
opcode.py [3.11] GH-93516: Backport GH-93769 (GH-94231) 2022-06-28 16:30:22 +01:00
operator.py bpo-44019: Add operator.call() to __all__ for the operator module (GH-29110) 2021-10-21 19:05:36 +09:00
optparse.py
os.py gh-57179: Add note on symlinks for os.walk (GH-94799) 2022-10-07 20:00:04 -07:00
pathlib.py [3.11] GH-87695: Fix OSError from pathlib.Path.glob() (GH-104292) (GH-104362) 2023-05-10 23:19:49 +00:00
pdb.py [3.11] gh-103225: Fixed zero lineno issue for pdb (#103265) (#104262) 2023-05-07 12:21:57 +01:00
pickle.py gh-90494: Reject 6th element of the __reduce__() tuple (GH-93609) (GH-93631) 2022-06-10 16:00:19 +02:00
pickletools.py
pipes.py gh-91217: deprecate-pipes (GH-91779) 2022-04-21 19:28:34 -07:00
pkgutil.py [3.11] gh-68654: Clarify subdirectories used by pkgutil.extend_path (GH-103701) 2023-04-23 02:57:10 -06:00
platform.py gh-100750: pass encoding kwarg in lib/platform.py (GH-100751) 2023-01-20 21:07:56 -08:00
plistlib.py gh-101992: update pstlib module documentation (GH-102133) 2023-03-04 22:36:00 -08:00
poplib.py
posixpath.py gh-96192: fix os.ismount() to use a path that is str or bytes (GH-96194) 2022-11-30 15:31:49 -08:00
pprint.py bpo-45557: Fix underscore_numbers in pprint.pprint(). (GH-29129) 2021-10-21 16:42:55 -04:00
profile.py gh-103935: Use io.open_code() when executing code in trace and profile modules (GH-103947) 2023-04-27 21:23:46 +00:00
pstats.py [3.11] Fix type annotation of pstats.FunctionProfile.ncalls (GH-96741) (#96835) 2022-10-05 14:57:11 -07:00
pty.py bpo-26228: [doc] Adapt PTY documentation updates from GH-4167 (GH-27754) 2021-08-13 12:57:07 +02:00
py_compile.py bpo-45428: Fix reading filenames from stdin in py_compile (GH-28848) 2021-10-15 12:38:55 +03:00
pyclbr.py
pydoc.py [3.11] gh-98374: Suppress ImportError for invalid query for help() co… (gh-98472) 2022-10-20 11:43:21 +09:00
queue.py gh-90879: Fix missing parameter for put_nowait() (GH-91514) 2022-04-14 17:23:57 +09:00
quopri.py
random.py [3.11] GH-101097: Switch from standard interval notation to greater or less than signs for random.random()'s documentation (GH-101119) (GH-101246) 2023-01-22 13:45:55 -08:00
reprlib.py bpo-39549: reprlib.Repr uses a “fillvalue” attribute (GH-18343) 2021-09-22 15:45:58 -05:00
rlcompleter.py gh-92345: Import rlcompleter before sys.path is extended (#92346) 2022-05-05 21:24:16 +02:00
runpy.py bpo-26792: Improve docstrings of runpy module run_functions (#30729) 2022-04-29 12:22:46 -06:00
sched.py
secrets.py bpo-47126: Update to canonical PEP URLs specified by PEP 676 (GH-32124) 2022-03-30 12:00:27 +01:00
selectors.py bpo-46583: remove unused sys.version_info check from selectors (GH-31023) 2022-02-02 10:15:02 +02:00
shelve.py
shlex.py
shutil.py [3.11] gh-102950: Implement PEP 706 – Filter for tarfile.extractall (GH-102953) (GH-103832) 2023-04-28 17:41:09 +02:00
signal.py bpo-27718: Fix help for the signal module (GH-30063) 2021-12-13 11:21:55 +02:00
site.py gh-90473: disable user site packages on WASI/Emscripten (GH-93633) 2022-06-09 09:12:51 -07:00
smtpd.py bpo-47061: use warnings._deprecated() with asynchat, asyncore, and smtpd (GH-32350) 2022-04-06 11:22:39 -07:00
smtplib.py bpo-43124: Fix smtplib multiple CRLF injection (GH-25987) 2021-08-29 16:10:50 +02:00
sndhdr.py gh-85525: Indicate supported sound header formats (GH-21575) 2022-10-15 08:09:20 -07:00
socket.py Fix broken link to MSDN (GH-102355) 2023-03-13 06:42:30 -07:00
socketserver.py bpo-40280: Disable AF_UNIX, AF_PACKET, SO_REUSE* on Emscripten (#31829) 2022-03-11 23:25:14 +01:00
sre_compile.py bpo-47152: Convert the re module into a package (GH-32177) 2022-04-02 11:35:13 +03:00
sre_constants.py bpo-47152: Convert the re module into a package (GH-32177) 2022-04-02 11:35:13 +03:00
sre_parse.py bpo-47152: Convert the re module into a package (GH-32177) 2022-04-02 11:35:13 +03:00
ssl.py bpo-46604: fix function name in ssl module docstring (#31064) 2022-05-03 09:56:24 -06:00
stat.py
statistics.py [3.11] GH-99155: Fix NormalDist pickle with 0 and 1 protocols (GH-99156). (GH-99188) 2022-11-07 01:41:08 -08:00
string.py bpo-46307: Add string.Template.get_identifiers() method (GH-30493) 2022-01-11 11:15:42 -08:00
stringprep.py
struct.py
subprocess.py [3.11] gh-87474: Fix file descriptor leaks in subprocess.Popen (GH-96351) (#104563) 2023-05-17 00:48:10 -07:00
sunau.py gh-91217: deprecate sunau (GH-91866) 2022-04-25 16:26:43 -07:00
symtable.py Change list to view object (GH-93661) 2022-06-11 04:20:52 -07:00
sysconfig.py [3.11] gh-92897: Ensure venv --copies respects source build property of the creating interpreter (GH-92899) (GH-94567) 2022-07-05 16:40:17 +01:00
tabnanny.py gh-97639: Remove tokenize.NL check from tabnanny (GH-97640) 2022-10-03 19:10:00 -07:00
tarfile.py [3.11] gh-102950: Implement PEP 706 – Filter for tarfile.extractall (GH-102953) (GH-103832) 2023-04-28 17:41:09 +02:00
telnetlib.py gh-91217: deprecate telnetlib (GH-91958) 2022-04-26 10:45:08 -07:00
tempfile.py gh-83499: Fix closing file descriptors in tempfile (GH-93874) 2022-06-26 01:38:06 -07:00
textwrap.py bpo-46544: Do not leak x and uspace in textwrap.TextWrapper (GH-30955) 2022-01-27 13:55:58 +02:00
this.py
threading.py fix threading.Event.isSet() docstring (GH-96297) 2022-08-26 22:32:03 -07:00
timeit.py
token.py
tokenize.py bpo-46565: del loop vars that are leaking into module namespaces (GH-30993) 2022-02-03 11:20:08 +02:00
trace.py gh-103935: Use io.open_code() when executing code in trace and profile modules (GH-103947) 2023-04-27 21:23:46 +00:00
traceback.py [3.11] gh-99103: Normalize specialized traceback anchors against the current line (#99423) 2022-11-21 22:16:12 +00:00
tracemalloc.py
tty.py
turtle.py gh-101100: Fix Sphinx warnings in turtle module (GH-102340) 2023-03-13 04:33:38 -07:00
types.py gh-89828: Do not relay the __class__ attribute in GenericAlias (GH-93754) 2022-06-18 07:41:25 -07:00
typing.py [3.11] gh-104328: Fix typo in `typing.Generic` multiple inheritance error message (GH-104335) (#104338) 2023-05-09 19:42:57 +00:00
uu.py [3.11] gh-99889: Fix directory traversal security flaw in uu.decode() (GH-104096) (#104329) 2023-05-09 09:46:25 -07:00
uuid.py gh-98415: Fix uuid.getnode() ifconfig implementation (GH-98423) 2022-11-02 12:12:31 -07:00
warnings.py gh-91230: Concise catch_warnings with simplefilter (#91435) 2022-04-23 17:55:22 -07:00
wave.py gh-47061: Deprecate chunk (GH-91419) 2022-04-11 15:02:41 -07:00
weakref.py Remove unnecessary registration of weakref.WeakSet to _collections_abc.Set (GH-32211) 2022-03-31 09:11:35 -05:00
webbrowser.py bpo-43137: Revert "webbrowser: Don't run gvfs-open on GNOME" (GH-30417) 2022-01-05 11:53:23 +00:00
xdrlib.py gh-91217: deprecate xdrlib (GH-92066) 2022-04-29 18:22:10 -07:00
zipapp.py
zipfile.py [3.11] gh-103861: Fix Zip64 extensions not being properly applied in some cases (GH-103863) (#104534) 2023-05-17 01:07:10 -07:00
zipimport.py GH-87235: Make sure "python /dev/fd/9 9</path/to/script.py" works on macOS (GH-99768) 2022-11-27 03:19:02 -08:00