Commit graph

23122 commits

Author SHA1 Message Date
Victor Stinner
bb33ccfc5a bpo-30908: Fix dangling thread in test_os.TestSendfile (#2680) (#2844)
tearDown() now clears explicitly the self.server variable to make
sure that the thread is completely cleared when tearDownClass()
checks if all threads have been cleaned up.

Fix the following warning:

$ ./python -m test --fail-env-changed -m test.test_os.TestSendfile.test_keywords -R 3:1 test_os
(...)
Warning -- threading_cleanup() failed to cleanup 0 threads after 3 sec (count: 0, dangling: 2)
(...)
Tests result: ENV CHANGED
(cherry picked from commit d1cc037d14)
2017-07-24 17:40:50 +02:00
Gregory P. Smith
8de48fe046 [3.6] Make test_shutil test_disk_usage not depend on the cwd fs (GH-2597) (#2820)
Make test_shutil test_disk_usage not depend on the current working directory's filesystem.
(cherry picked from commit 529746c905)
2017-07-22 23:00:39 -07:00
Victor Stinner
96ef06f397 bpo-30822: Exclude tzdata from regrtest --all (#2775) (#2777)
When running the test suite using --use=all / -u all, exclude tzdata
since it makes test_datetime too slow (15-20 min on some buildbots)
which then times out on some buildbots.

-u tzdata must now be enabled explicitly, -u tzdata or -u all,tzdata,
to run all test_datetime tests.

Fix also regrtest command line parser to allow passing -u
extralargefile to run test_zipfile64.

Travis CI: remove -tzdata. Replace -u all,-tzdata,-cpu with -u all,-cpu since tzdata is now excluded from -u all.
(cherry picked from commit 5b392bbaeb)
2017-07-20 17:08:48 +02:00
Ammar Askar
ae4dca7701 [3.6] bpo-30883: Use pythontest.net instead of debian.org in test_urllib2net (GH-2755) 2017-07-20 03:21:09 +03:00
Serhiy Storchaka
a819e5e1e6 [3.6] bpo-30936: Fix a reference leak in json when fail to sort keys. (GH-2712). (#2727)
(cherry picked from commit 49f6449ef4)
2017-07-16 07:48:08 +03:00
Serhiy Storchaka
28343e3392 [3.6] bpo-30876: Relative import from unloaded package now reimports the package (GH-2639) (#2676)
instead of failing with SystemError.

Relative import from non-package now fails with ImportError rather than
SystemError.
(cherry picked from commit 8a9cd20edc)
2017-07-16 07:44:25 +03:00
Serhiy Storchaka
7d0a995820 [3.6] bpo-30911: Add tests for bad boolean arguments for accelerated json (GH-2690) (#2692)
encoder and decoder.
(cherry picked from commit d3aaa2f)
2017-07-13 11:40:25 +03:00
terryjreedy
a9bf62cadf [3.6] bpo-30899: Add unittests, 100% coverage, for IDLE's two ConfigParser subclasses. (GH-2662) (#2685)
Patch by Louie Lu.
(cherry picked from commit 50c9435)
2017-07-12 14:42:37 -04:00
Serhiy Storchaka
ecfe4f678b [3.6] bpo-30879: os.listdir() and os.scandir() now emit bytes names when (GH-2634) (#2656)
called with bytes-like argument..
(cherry picked from commit 1180e5a518)
2017-07-11 07:16:11 +03:00
Nir Soffer
04f77d4677 [3.6] bpo-29854: Fix segfault in call_readline() (GH-728)
If history-length is set in .inputrc, and the history file is double the
history size (or more), history_get(N) returns NULL, and python
segfaults. Fix that by checking for NULL return value.

It seems that the root cause is incorrect handling of bigger history in
readline, but Python should not segfault even if readline returns
unexpected value.

This issue affects only GNU readline. When using libedit emulation
system history size option does not work.
2017-07-08 21:51:21 +03:00
Joel Hillacre
c60d2f5e86 bpo-30532: Fix whitespace folding in certain cases (#2591)
Leading whitespace was incorrectly dropped during folding of certain lines in the _header_value_parser's folding algorithm.  This makes the whitespace handling code consistent.
2017-07-06 17:28:22 -04:00
Serhiy Storchaka
03b0e8374b [3.6] bpo-30814: Fixed a race condition when import a submodule from a package. (GH-2580). (#2598)
(cherry picked from commit b4baacee1a)
2017-07-06 08:38:24 +03:00
Yury Selivanov
aaa4f99151 [3.6] bpo-30828: Fix out of bounds write in `asyncio.CFuture.remove_done_callback() (GH-2569) (#2590)
(cherry picked from commit 833a3b0d37)
2017-07-05 14:03:10 -04:00
Serhiy Storchaka
bebd2cfa5f [3.6] bpo-30441: Fix bug when modifying os.environ while iterating over it (GH-2409). (#2556)
(cherry picked from commit 8a8d28501f)
2017-07-04 07:55:32 +03:00
Victor Stinner
4132adb4b8 bpo-30764: Fix regrtest --fail-env-changed --forever (#2536) (#2539)
--forever now stops if a fail changes the environment.
(cherry picked from commit 5e87592fd1)
2017-07-03 12:50:36 +02:00
Nick Coghlan
bfdc6fdc0e [3.6] bpo-30597: Show expected input in custom 'print' error message. (GH-2531)
(cherry picked from commit 3a7f03584a)
2017-07-03 17:49:50 +10:00
Antoine Pitrou
3024c05290 [3.6] bpo-30703: Improve signal delivery (GH-2415) (#2527)
* [3.6] bpo-30703: Improve signal delivery (GH-2415)

* Improve signal delivery

Avoid using Py_AddPendingCall from signal handler, to avoid calling signal-unsafe functions.

* Remove unused function

* Improve comments

* Add stress test

* Adapt for --without-threads

* Add second stress test

* Add NEWS blurb

* Address comments @haypo.
(cherry picked from commit c08177a1cc)

* bpo-30796: Fix failures in signal delivery stress test (#2488)

* bpo-30796: Fix failures in signal delivery stress test

setitimer() can have a poor minimum resolution on some machines,
this would make the test reach its deadline (and a stray signal
could then kill a subsequent test).

* Make sure to clear the itimer after the test
2017-07-01 19:12:05 +02:00
Victor Stinner
a3ca94d050 [3.6] bpo-29512, bpo-30776: Backport regrtest enhancements from master to 3.6 (#2513)
* bpo-29512: Add test.bisect, bisect failing tests (#2452)

Add a new "python3 -m test.bisect" tool to bisect failing tests.

It can be used to find which test method(s) leak references, leak
files, etc.
(cherry picked from commit 84d9d14a1f)

* bpo-30776: regrtest: reduce memleak false positive (#2484)

Only report a leak if each run leaks at least one memory block.
(cherry picked from commit beeca6e1e5)
2017-06-30 17:31:16 +02:00
Victor Stinner
23caf8cfc6 bpo-30280: Cleanup threads in ayncio tests (#2501) (#2511)
* bpo-30280: asyncio now cleans up threads

asyncio base TestCase now uses threading_setup() and
threading_cleanup() of test.support to cleanup threads.

* asyncio: Fix TestBaseSelectorEventLoop cleanup

bpo-30280: TestBaseSelectorEventLoop of
test.test_asyncio.test_selector_events now correctly closes the event
loop: cleanup its executor to not leak threads.

Don't override the close() method of the event loop, only override
the_close_self_pipe() method.

(cherry picked from commit b903067462)
2017-06-30 17:20:33 +02:00
Victor Stinner
d0aac5da59 bpo-30812: Fix test_warnings, restore _showwarnmsg (#2504) (#2507)
bpo-26568, bpo-30812: Fix test_showwarnmsg_missing(): restore the
attribute after removing it.
(cherry picked from commit 7eebeb8fb8)
2017-06-30 13:12:12 +02:00
Antoine Pitrou
6f3cb059fd [3.6] bpo-30807: signal.setitimer() may disable the timer by mistake (GH-2493) (#2497)
* bpo-30807: signal.setitimer() may disable the timer by mistake

* Add NEWS blurb
(cherry picked from commit 729780a810)
2017-06-30 10:54:24 +02:00
Antoine Pitrou
38d6a40898 [3.6] Clear potential ref cycle between Process and Process target (GH-2470) (#2471)
* Clear potential ref cycle between Process and Process target

Besides Process.join() not being called, this was an indirect cause of bpo-30775.
The threading module already does this.

* Add issue reference.
(cherry picked from commit 79d37ae979)
2017-06-28 13:08:20 +02:00
Antoine Pitrou
e022aad73a [3.6] bpo-30775: Fix refleaks in test_multiprocessing (GH-2467) (#2468)
Forgetting to call Process.join() can keep some resources alive.
(cherry picked from commit a79f8faccf)
2017-06-28 11:49:38 +02:00
Serhiy Storchaka
0834905d9b [3.6] bpo-13617: Reject embedded null characters in wchar* strings. (GH-2302) (#2462)
Based on patch by Victor Stinner.

Add private C API function _PyUnicode_AsUnicode() which is similar to
PyUnicode_AsUnicode(), but checks for null characters..
(cherry picked from commit f7eae0adfc)
2017-06-28 09:27:35 +03:00
Victor Stinner
35d2ca2b94 [3.6] bpo-30523, bpo-30764, bpo-30776: Sync regrtest from master (#2441)
* bpo-30523: regrtest --list-cases --match (#2401)

* regrtest --list-cases now supports --match and --match-file options.
  Example: ./python -m test --list-cases -m FileTests test_os
* --list-cases now also sets support.verbose to False to prevent
  messages to stdout when loading test modules.
* Add support._match_test() private function.
(cherry picked from commit ace56d5836)

* bpo-30764: regrtest: add --fail-env-changed option (#2402)

* bpo-30764: regrtest: change exit code on failure

* Exit code 2 if failed tests ("bad")
* Exit code 3 if interrupted

* bpo-30764: regrtest: add --fail-env-changed option

If the option is set, mark a test as failed if it alters the
environment, for example if it creates a file without removing it.

(cherry picked from commit 63f54c6893)

* bpo-30776: reduce regrtest -R false positives (#2422)

* Change the regrtest --huntrleaks checker to decide if a test file
  leaks or not. Require that each run leaks at least 1 reference.
* Warmup runs are now completely ignored: ignored in the checker test
  and not used anymore to compute the sum.
* Add an unit test for a reference leak.

Example of reference differences previously considered a failure
(leak) and now considered as success (success, no leak):

    [3, 0, 0]
    [0, 1, 0]
    [8, -8, 1]
(cherry picked from commit 48b5c422ff)
2017-06-27 16:04:15 +02:00
Victor Stinner
9ad50d9459 bpo-30764: test_subprocess uses SuppressCrashReport (#2405) (#2410)
bpo-30764, bpo-29335: test_child_terminated_in_stopped_state() of
test_subprocess now uses support.SuppressCrashReport() to prevent the
creation of a core dump on FreeBSD.
(cherry picked from commit cdee3f14f7)
2017-06-27 00:00:33 +02:00
Serhiy Storchaka
9c2dc0c58a [3.6] bpo-30746: Prohibited the '=' character in environment variable names (GH-2382) (#2391)
in `os.putenv()` and `os.spawn*()`..
(cherry picked from commit 77703942c5)
2017-06-25 09:49:15 +03:00
Dong-hee Na
504b95047a [3.6] bpo-30616: Functional API of enum allows to create empty enums. (#2304) (#2324) 2017-06-24 09:12:20 -07:00
Serhiy Storchaka
e7135751b8 [3.6] bpo-30730: Prevent environment variables injection in subprocess on Windows. (GH-2325) (#2360)
Prevent passing other invalid environment variables and command arguments..
(cherry picked from commit d174d24a5d)
2017-06-23 20:17:38 +03:00
Victor Stinner
cea2174ab7 bpo-30604: Skip CoExtra tests if ctypes is missing (#2356) (#2358)
(cherry picked from commit a4b091e135)
2017-06-23 15:24:28 +02:00
Serhiy Storchaka
c1d5345679 [3.6] bpo-30727: Fix a race condition in test_threading. (GH-2334) (#2351)
(cherry picked from commit 32cb968)
2017-06-23 13:52:06 +03:00
Serhiy Storchaka
a1115e1a04 [3.6] bpo-29755: Fixed the lgettext() family of functions in the gettext module. (GH-2266) (#2297)
They now always return bytes.

Updated the gettext documentation..
(cherry picked from commit 26cb4657bc)
2017-06-20 18:06:35 +03:00
Victor Stinner
536c1f1246 bpo-30500: urllib: Simplify splithost by calling into urlparse. (#1849) (#2289)
The current regex based splitting produces a wrong result. For example::

  http://abc#@def

Web browsers parse that URL as ``http://abc/#@def``, that is, the host
is ``abc``, the path is ``/``, and the fragment is ``#@def``.
(cherry picked from commit 90e01e50ef)
2017-06-20 15:37:24 +02:00
Victor Stinner
acdf159b24 bpo-29887: test_normalization handles PermissionError (#1196) (#2275)
Skip test_normalization.test_main() if download raises a permission
error.
(cherry picked from commit d13d54748d)
2017-06-19 15:42:43 +02:00
Mariatta
51f40a81a4 [3.6] bpo-29887: Test normalization now fails if download fails (GH-905) (#2271)
* test_normalization fails if download fails

bpo-29887. The test is still skipped if "-u urlfetch" option is not
passed to regrtest (python3 -m test -u urlfetch test_normalization).

* Fix ResourceWarning in test_normalization

bpo-29887: Fix ResourceWarning in test_normalization if tests are
interrupted by CTRL+c.
(cherry picked from commit 722a3af092)
2017-06-19 13:22:54 +02:00
Mariatta
e9f4d8db5f [email] bpo-29478: Fix passing max_line_length=None from Compat32 policy (GH-595) (GH-2233)
If max_line_length=None is specified while using the Compat32 policy,
it is no longer ignored..
(cherry picked from commit b459f74826)
2017-06-16 07:18:58 -07:00
Serhiy Storchaka
2eca5b465f [3.6] bpo-30682: Removed a too-strict assertion that failed for certain f-strings. (GH-2232) (#2242)
This caused a segfault on eval("f'\\\n'") and eval("f'\\\r'") in debug build..
(cherry picked from commit 11e97f2f80)
2017-06-16 09:29:42 -04:00
Victor Stinner
a0ccc54e6d Synchronize libregrtest from master to 3.6 (#2244)
* bpo-30523: regrtest: Add --list-cases option (#2238)
* bpo-30284: Fix regrtest for out of tree build (#1481)
* bpo-30540: regrtest: add --matchfile option (#1909)
* bpo-30258: regrtest: Fix run_tests_multiprocess() (#1479)
* bpo-30263: regrtest: log system load (#1452)
2017-06-16 14:39:09 +02:00
Dong-hee Na
e45ea377b8 bpo-30149: Fix partialmethod without explicit self parameter (#1308) (#1662) 2017-06-15 17:41:57 +03:00
Serhiy Storchaka
cf58dfb44c [3.6] bpo-29931 fix __lt__ check in ipaddress.ip_interface for both v4 and v6. (GH-879) (#2217)
the original logic was just comparing the network address
but this is wrong because if the network address is equal then
we need to compare the ip address for breaking the tie

add more ip_interface comparison tests.
(cherry picked from commit 7bd8d3e794)
2017-06-15 17:16:38 +03:00
Serhiy Storchaka
523a243840 [3.6] bpo-30605: Fix compiling binary regexs with BytesWarnings enabled. (GH-2016) (#2214)
Running our unit tests with `-bb` enabled triggered this failure..
(cherry picked from commit 171b9a354e)
2017-06-15 16:55:22 +03:00
Victor Stinner
8a39af9457 bpo-30231: Remove skipped test_imaplib tests (#1419) (#2192)
The public cyrus.andrew.cmu.edu IMAP server (port 993) doesn't accept
TLS connection using our self-signed x509 certificate. Remove the two
tests which are already skipped.

Write a new test_certfile_arg_warn() unit test for the certfile
deprecation warning.
(cherry picked from commit b18563da88)
2017-06-14 22:43:01 +02:00
Victor Stinner
c2a506e40e [3.6] bpo-30649: test_os tolerates 50 ms delta for utime (#2156) (#2175)
* bpo-30649: test_os tolerates 50 ms delta for utime (#2156)

On Windows, tolerate a delta of 50 ms instead of 20 ms in
test_utime_current() and test_utime_current_old() of test_os.

On other platforms, reduce the delta from 20 ms to 10 ms.
(cherry picked from commit c94caca65c)

* bpo-30649: Revert utime delta in test_os (#2176)

PPC64 Fedora 3.x buildbot requires at least a delta of 14 ms: revert
the utime delta to 20 ms.

I tried 10 ms, but test_os failed on the PPC64 Fedora 3.x buildbot.
(cherry picked from commit 3402f72688)
2017-06-14 14:26:52 +02:00
Antoine Pitrou
6fd0345995 [3.6] bpo-24484: Avoid race condition in multiprocessing cleanup (GH-2159) (#2166)
* bpo-24484: Avoid race condition in multiprocessing cleanup

The finalizer registry can be mutated while inspected by multiprocessing
at process exit.

* Use test.support.start_threads()

* Add Misc/NEWS.
(cherry picked from commit 1eb6c0074d)
2017-06-13 17:51:26 +02:00
Nick Coghlan
92e9e35292 [3.6] bpo-29514: Check magic number for bugfix releases (#2157)
Add a test to check the current MAGIC_NUMBER against the
expected number for the release if the current release is
at candidate or final level. On test failure, describe to
the developer the procedure for changing the magic number.

This ensures that pre-merge CI will automatically pick up
on magic number changes in maintenance releases (and
explain why those are problematic), rather than relying on
all core developers to be aware of the implications of
such changes.
2017-06-13 20:32:46 +10:00
Dino Viehland
2997fec01e [3.6] bpo-30604: Move co_extra_freefuncs to interpreter state to avoid crashes in threads (#2015)
* Move co_extra_freefuncs to interpreter state to avoid crashes in
multi-threaded scenarios involving deletion of code objects

* Don't require that extra be zero initialized

* Build test list instead of defining empty test class

* Ensure extra is always assigned on success

* Keep the old fields in the thread state object, just don't use them
Add new linked list of code extra objects on a per-interpreter basis
  so that interpreter state size isn't changed

* Rename __PyCodeExtraState_Get and add comment about it going away in 3.7
Fix sort order of import's in test_code.py

* Remove an extraneous space

* Remove docstrings for comments

* Touch up formatting

* Fix casing of coextra local

* Fix casing of another variable

* Prefix PyCodeExtraState with __ to match C API for getting it

* Update NEWS file for bpo-30604
2017-06-12 21:46:35 -04:00
Serhiy Storchaka
d89dc844d2 [3.6] bpo-28994: Fixed errors handling in atexit._run_exitfuncs(). (GH-2034) (#2121)
The traceback no longer displayed for SystemExit raised in a callback registered by atexit..
(cherry picked from commit 3fd54d4a7e)
2017-06-12 09:02:13 +03:00
Yury Selivanov
83d30bd667 Revert "[3.6] bpo-29406: asyncio SSL contexts leak sockets after calling close with certain servers (GH-409) (#2062)" (#2112)
This reverts commit 6e14fd2a14.
2017-06-11 16:46:45 +02:00
Yury Selivanov
176f2ebdad bpo-30508: Don't log exceptions if Task/Future "cancel()" method was called. (#2109) 2017-06-11 14:00:14 +00:00
Mariatta
8d15b19be6 bpo-28556: Updates to typing module (GH-2076) (GH-2087)
This PR contains two updates to typing module:

- Support ContextManager on all versions (original PR by Jelle Zijlstra).
- Add generic AsyncContextManager..
(cherry picked from commit 29fda8db16)
2017-06-10 14:41:00 -07:00