| 
									
										
											  
											
												bpo-40503: PEP 615: Tests and implementation for zoneinfo (GH-19909)
This is the initial implementation of PEP 615, the zoneinfo module,
ported from the standalone reference implementation (see
https://www.python.org/dev/peps/pep-0615/#reference-implementation for a
link, which has a more detailed commit history).
This includes (hopefully) all functional elements described in the PEP,
but documentation is found in a separate PR. This includes:
1. A pure python implementation of the ZoneInfo class
2. A C accelerated implementation of the ZoneInfo class
3. Tests with 100% branch coverage for the Python code (though C code
   coverage is less than 100%).
4. A compile-time configuration option on Linux (though not on Windows)
Differences from the reference implementation:
- The module is arranged slightly differently: the accelerated module is
  `_zoneinfo` rather than `zoneinfo._czoneinfo`, which also necessitates
  some changes in the test support function. (Suggested by Victor
  Stinner and Steve Dower.)
- The tests are arranged slightly differently and do not include the
  property tests. The tests live at test/test_zoneinfo/test_zoneinfo.py
  rather than test/test_zoneinfo.py or test/test_zoneinfo/__init__.py
  because we may do some refactoring in the future that would likely
  require this separation anyway; we may:
        - include the property tests
        - automatically run all the tests against both pure Python and C,
          rather than manually constructing C and Python test classes (similar
          to the way this works with test_datetime.py, which generates C
          and Python test cases from datetimetester.py).
- This includes a compile-time configuration option on Linux (though not
  on Windows); added with much help from Thomas Wouters.
- Integration into the CPython build system is obviously different from
  building a standalone zoneinfo module wheel.
- This includes configuration to install the tzdata package as part of
  CI, though only on the coverage jobs. Introducing a PyPI dependency as
  part of the CI build was controversial, and this is seen as less of a
  major change, since the coverage jobs already depend on pip and PyPI.
Additional changes that were introduced as part of this PR, most / all of
which were backported to the reference implementation:
- Fixed reference and memory leaks
    With much debugging help from Pablo Galindo
- Added smoke tests ensuring that the C and Python modules are built
    The import machinery can be somewhat fragile, and the "seamlessly falls
    back to pure Python" nature of this module makes it so that a problem
    building the C extension or a failure to import the pure Python version
    might easily go unnoticed.
- Adjustments to zoneinfo.__dir__
    Suggested by Petr Viktorin.
- Slight refactorings as suggested by Steve Dower.
- Removed unnecessary if check on std_abbr
    Discovered this because of a missing line in branch coverage.
											
										 
											2020-05-16 04:20:06 -04:00
										 |  |  | import os | 
					
						
							|  |  |  | import sysconfig | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | def reset_tzpath(to=None): | 
					
						
							|  |  |  |     global TZPATH | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |     tzpaths = to | 
					
						
							|  |  |  |     if tzpaths is not None: | 
					
						
							|  |  |  |         if isinstance(tzpaths, (str, bytes)): | 
					
						
							|  |  |  |             raise TypeError( | 
					
						
							|  |  |  |                 f"tzpaths must be a list or tuple, " | 
					
						
							|  |  |  |                 + f"not {type(tzpaths)}: {tzpaths!r}" | 
					
						
							|  |  |  |             ) | 
					
						
							| 
									
										
										
										
											2020-05-29 09:34:30 -04:00
										 |  |  | 
 | 
					
						
							|  |  |  |         if not all(map(os.path.isabs, tzpaths)): | 
					
						
							| 
									
										
											  
											
												bpo-40503: PEP 615: Tests and implementation for zoneinfo (GH-19909)
This is the initial implementation of PEP 615, the zoneinfo module,
ported from the standalone reference implementation (see
https://www.python.org/dev/peps/pep-0615/#reference-implementation for a
link, which has a more detailed commit history).
This includes (hopefully) all functional elements described in the PEP,
but documentation is found in a separate PR. This includes:
1. A pure python implementation of the ZoneInfo class
2. A C accelerated implementation of the ZoneInfo class
3. Tests with 100% branch coverage for the Python code (though C code
   coverage is less than 100%).
4. A compile-time configuration option on Linux (though not on Windows)
Differences from the reference implementation:
- The module is arranged slightly differently: the accelerated module is
  `_zoneinfo` rather than `zoneinfo._czoneinfo`, which also necessitates
  some changes in the test support function. (Suggested by Victor
  Stinner and Steve Dower.)
- The tests are arranged slightly differently and do not include the
  property tests. The tests live at test/test_zoneinfo/test_zoneinfo.py
  rather than test/test_zoneinfo.py or test/test_zoneinfo/__init__.py
  because we may do some refactoring in the future that would likely
  require this separation anyway; we may:
        - include the property tests
        - automatically run all the tests against both pure Python and C,
          rather than manually constructing C and Python test classes (similar
          to the way this works with test_datetime.py, which generates C
          and Python test cases from datetimetester.py).
- This includes a compile-time configuration option on Linux (though not
  on Windows); added with much help from Thomas Wouters.
- Integration into the CPython build system is obviously different from
  building a standalone zoneinfo module wheel.
- This includes configuration to install the tzdata package as part of
  CI, though only on the coverage jobs. Introducing a PyPI dependency as
  part of the CI build was controversial, and this is seen as less of a
  major change, since the coverage jobs already depend on pip and PyPI.
Additional changes that were introduced as part of this PR, most / all of
which were backported to the reference implementation:
- Fixed reference and memory leaks
    With much debugging help from Pablo Galindo
- Added smoke tests ensuring that the C and Python modules are built
    The import machinery can be somewhat fragile, and the "seamlessly falls
    back to pure Python" nature of this module makes it so that a problem
    building the C extension or a failure to import the pure Python version
    might easily go unnoticed.
- Adjustments to zoneinfo.__dir__
    Suggested by Petr Viktorin.
- Slight refactorings as suggested by Steve Dower.
- Removed unnecessary if check on std_abbr
    Discovered this because of a missing line in branch coverage.
											
										 
											2020-05-16 04:20:06 -04:00
										 |  |  |             raise ValueError(_get_invalid_paths_message(tzpaths)) | 
					
						
							|  |  |  |         base_tzpath = tzpaths | 
					
						
							|  |  |  |     else: | 
					
						
							|  |  |  |         env_var = os.environ.get("PYTHONTZPATH", None) | 
					
						
							|  |  |  |         if env_var is not None: | 
					
						
							|  |  |  |             base_tzpath = _parse_python_tzpath(env_var) | 
					
						
							|  |  |  |         else: | 
					
						
							|  |  |  |             base_tzpath = _parse_python_tzpath( | 
					
						
							|  |  |  |                 sysconfig.get_config_var("TZPATH") | 
					
						
							|  |  |  |             ) | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |     TZPATH = tuple(base_tzpath) | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | def _parse_python_tzpath(env_var): | 
					
						
							|  |  |  |     if not env_var: | 
					
						
							|  |  |  |         return () | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |     raw_tzpath = env_var.split(os.pathsep) | 
					
						
							|  |  |  |     new_tzpath = tuple(filter(os.path.isabs, raw_tzpath)) | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |     # If anything has been filtered out, we will warn about it | 
					
						
							|  |  |  |     if len(new_tzpath) != len(raw_tzpath): | 
					
						
							|  |  |  |         import warnings | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |         msg = _get_invalid_paths_message(raw_tzpath) | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |         warnings.warn( | 
					
						
							| 
									
										
										
										
											2021-04-25 13:45:05 -04:00
										 |  |  |             "Invalid paths specified in PYTHONTZPATH environment variable. " | 
					
						
							| 
									
										
											  
											
												bpo-40503: PEP 615: Tests and implementation for zoneinfo (GH-19909)
This is the initial implementation of PEP 615, the zoneinfo module,
ported from the standalone reference implementation (see
https://www.python.org/dev/peps/pep-0615/#reference-implementation for a
link, which has a more detailed commit history).
This includes (hopefully) all functional elements described in the PEP,
but documentation is found in a separate PR. This includes:
1. A pure python implementation of the ZoneInfo class
2. A C accelerated implementation of the ZoneInfo class
3. Tests with 100% branch coverage for the Python code (though C code
   coverage is less than 100%).
4. A compile-time configuration option on Linux (though not on Windows)
Differences from the reference implementation:
- The module is arranged slightly differently: the accelerated module is
  `_zoneinfo` rather than `zoneinfo._czoneinfo`, which also necessitates
  some changes in the test support function. (Suggested by Victor
  Stinner and Steve Dower.)
- The tests are arranged slightly differently and do not include the
  property tests. The tests live at test/test_zoneinfo/test_zoneinfo.py
  rather than test/test_zoneinfo.py or test/test_zoneinfo/__init__.py
  because we may do some refactoring in the future that would likely
  require this separation anyway; we may:
        - include the property tests
        - automatically run all the tests against both pure Python and C,
          rather than manually constructing C and Python test classes (similar
          to the way this works with test_datetime.py, which generates C
          and Python test cases from datetimetester.py).
- This includes a compile-time configuration option on Linux (though not
  on Windows); added with much help from Thomas Wouters.
- Integration into the CPython build system is obviously different from
  building a standalone zoneinfo module wheel.
- This includes configuration to install the tzdata package as part of
  CI, though only on the coverage jobs. Introducing a PyPI dependency as
  part of the CI build was controversial, and this is seen as less of a
  major change, since the coverage jobs already depend on pip and PyPI.
Additional changes that were introduced as part of this PR, most / all of
which were backported to the reference implementation:
- Fixed reference and memory leaks
    With much debugging help from Pablo Galindo
- Added smoke tests ensuring that the C and Python modules are built
    The import machinery can be somewhat fragile, and the "seamlessly falls
    back to pure Python" nature of this module makes it so that a problem
    building the C extension or a failure to import the pure Python version
    might easily go unnoticed.
- Adjustments to zoneinfo.__dir__
    Suggested by Petr Viktorin.
- Slight refactorings as suggested by Steve Dower.
- Removed unnecessary if check on std_abbr
    Discovered this because of a missing line in branch coverage.
											
										 
											2020-05-16 04:20:06 -04:00
										 |  |  |             + msg, | 
					
						
							|  |  |  |             InvalidTZPathWarning, | 
					
						
							|  |  |  |         ) | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |     return new_tzpath | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | def _get_invalid_paths_message(tzpaths): | 
					
						
							|  |  |  |     invalid_paths = (path for path in tzpaths if not os.path.isabs(path)) | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |     prefix = "\n    " | 
					
						
							|  |  |  |     indented_str = prefix + prefix.join(invalid_paths) | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |     return ( | 
					
						
							|  |  |  |         "Paths should be absolute but found the following relative paths:" | 
					
						
							|  |  |  |         + indented_str | 
					
						
							|  |  |  |     ) | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | def find_tzfile(key): | 
					
						
							|  |  |  |     """Retrieve the path to a TZif file from a key.""" | 
					
						
							|  |  |  |     _validate_tzfile_path(key) | 
					
						
							|  |  |  |     for search_path in TZPATH: | 
					
						
							|  |  |  |         filepath = os.path.join(search_path, key) | 
					
						
							|  |  |  |         if os.path.isfile(filepath): | 
					
						
							|  |  |  |             return filepath | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |     return None | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | _TEST_PATH = os.path.normpath(os.path.join("_", "_"))[:-1] | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | def _validate_tzfile_path(path, _base=_TEST_PATH): | 
					
						
							|  |  |  |     if os.path.isabs(path): | 
					
						
							|  |  |  |         raise ValueError( | 
					
						
							|  |  |  |             f"ZoneInfo keys may not be absolute paths, got: {path}" | 
					
						
							|  |  |  |         ) | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |     # We only care about the kinds of path normalizations that would change the | 
					
						
							|  |  |  |     # length of the key - e.g. a/../b -> a/b, or a/b/ -> a/b. On Windows, | 
					
						
							|  |  |  |     # normpath will also change from a/b to a\b, but that would still preserve | 
					
						
							|  |  |  |     # the length. | 
					
						
							|  |  |  |     new_path = os.path.normpath(path) | 
					
						
							|  |  |  |     if len(new_path) != len(path): | 
					
						
							|  |  |  |         raise ValueError( | 
					
						
							|  |  |  |             f"ZoneInfo keys must be normalized relative paths, got: {path}" | 
					
						
							|  |  |  |         ) | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |     resolved = os.path.normpath(os.path.join(_base, new_path)) | 
					
						
							|  |  |  |     if not resolved.startswith(_base): | 
					
						
							|  |  |  |         raise ValueError( | 
					
						
							|  |  |  |             f"ZoneInfo keys must refer to subdirectories of TZPATH, got: {path}" | 
					
						
							|  |  |  |         ) | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | del _TEST_PATH | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2020-05-17 21:55:11 -04:00
										 |  |  | def available_timezones(): | 
					
						
							|  |  |  |     """Returns a set containing all available time zones.
 | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |     .. caution:: | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |         This may attempt to open a large number of files, since the best way to | 
					
						
							|  |  |  |         determine if a given file on the time zone search path is to open it | 
					
						
							|  |  |  |         and check for the "magic string" at the beginning. | 
					
						
							|  |  |  |     """
 | 
					
						
							|  |  |  |     from importlib import resources | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |     valid_zones = set() | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |     # Start with loading from the tzdata package if it exists: this has a | 
					
						
							|  |  |  |     # pre-assembled list of zones that only requires opening one file. | 
					
						
							|  |  |  |     try: | 
					
						
							|  |  |  |         with resources.open_text("tzdata", "zones") as f: | 
					
						
							|  |  |  |             for zone in f: | 
					
						
							|  |  |  |                 zone = zone.strip() | 
					
						
							|  |  |  |                 if zone: | 
					
						
							|  |  |  |                     valid_zones.add(zone) | 
					
						
							|  |  |  |     except (ImportError, FileNotFoundError): | 
					
						
							|  |  |  |         pass | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |     def valid_key(fpath): | 
					
						
							|  |  |  |         try: | 
					
						
							|  |  |  |             with open(fpath, "rb") as f: | 
					
						
							|  |  |  |                 return f.read(4) == b"TZif" | 
					
						
							|  |  |  |         except Exception:  # pragma: nocover | 
					
						
							|  |  |  |             return False | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |     for tz_root in TZPATH: | 
					
						
							|  |  |  |         if not os.path.exists(tz_root): | 
					
						
							|  |  |  |             continue | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |         for root, dirnames, files in os.walk(tz_root): | 
					
						
							|  |  |  |             if root == tz_root: | 
					
						
							|  |  |  |                 # right/ and posix/ are special directories and shouldn't be | 
					
						
							|  |  |  |                 # included in the output of available zones | 
					
						
							|  |  |  |                 if "right" in dirnames: | 
					
						
							|  |  |  |                     dirnames.remove("right") | 
					
						
							|  |  |  |                 if "posix" in dirnames: | 
					
						
							|  |  |  |                     dirnames.remove("posix") | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |             for file in files: | 
					
						
							|  |  |  |                 fpath = os.path.join(root, file) | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |                 key = os.path.relpath(fpath, start=tz_root) | 
					
						
							|  |  |  |                 if os.sep != "/":  # pragma: nocover | 
					
						
							|  |  |  |                     key = key.replace(os.sep, "/") | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |                 if not key or key in valid_zones: | 
					
						
							|  |  |  |                     continue | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |                 if valid_key(fpath): | 
					
						
							|  |  |  |                     valid_zones.add(key) | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |     if "posixrules" in valid_zones: | 
					
						
							|  |  |  |         # posixrules is a special symlink-only time zone where it exists, it | 
					
						
							|  |  |  |         # should not be included in the output | 
					
						
							|  |  |  |         valid_zones.remove("posixrules") | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |     return valid_zones | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
											  
											
												bpo-40503: PEP 615: Tests and implementation for zoneinfo (GH-19909)
This is the initial implementation of PEP 615, the zoneinfo module,
ported from the standalone reference implementation (see
https://www.python.org/dev/peps/pep-0615/#reference-implementation for a
link, which has a more detailed commit history).
This includes (hopefully) all functional elements described in the PEP,
but documentation is found in a separate PR. This includes:
1. A pure python implementation of the ZoneInfo class
2. A C accelerated implementation of the ZoneInfo class
3. Tests with 100% branch coverage for the Python code (though C code
   coverage is less than 100%).
4. A compile-time configuration option on Linux (though not on Windows)
Differences from the reference implementation:
- The module is arranged slightly differently: the accelerated module is
  `_zoneinfo` rather than `zoneinfo._czoneinfo`, which also necessitates
  some changes in the test support function. (Suggested by Victor
  Stinner and Steve Dower.)
- The tests are arranged slightly differently and do not include the
  property tests. The tests live at test/test_zoneinfo/test_zoneinfo.py
  rather than test/test_zoneinfo.py or test/test_zoneinfo/__init__.py
  because we may do some refactoring in the future that would likely
  require this separation anyway; we may:
        - include the property tests
        - automatically run all the tests against both pure Python and C,
          rather than manually constructing C and Python test classes (similar
          to the way this works with test_datetime.py, which generates C
          and Python test cases from datetimetester.py).
- This includes a compile-time configuration option on Linux (though not
  on Windows); added with much help from Thomas Wouters.
- Integration into the CPython build system is obviously different from
  building a standalone zoneinfo module wheel.
- This includes configuration to install the tzdata package as part of
  CI, though only on the coverage jobs. Introducing a PyPI dependency as
  part of the CI build was controversial, and this is seen as less of a
  major change, since the coverage jobs already depend on pip and PyPI.
Additional changes that were introduced as part of this PR, most / all of
which were backported to the reference implementation:
- Fixed reference and memory leaks
    With much debugging help from Pablo Galindo
- Added smoke tests ensuring that the C and Python modules are built
    The import machinery can be somewhat fragile, and the "seamlessly falls
    back to pure Python" nature of this module makes it so that a problem
    building the C extension or a failure to import the pure Python version
    might easily go unnoticed.
- Adjustments to zoneinfo.__dir__
    Suggested by Petr Viktorin.
- Slight refactorings as suggested by Steve Dower.
- Removed unnecessary if check on std_abbr
    Discovered this because of a missing line in branch coverage.
											
										 
											2020-05-16 04:20:06 -04:00
										 |  |  | class InvalidTZPathWarning(RuntimeWarning): | 
					
						
							|  |  |  |     """Warning raised if an invalid path is specified in PYTHONTZPATH.""" | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | TZPATH = () | 
					
						
							|  |  |  | reset_tzpath() |