cpython/Include/internal/pycore_llist.h

107 lines
2.4 KiB
C
Raw Normal View History

gh-108724: Add PyMutex and _PyParkingLot APIs (gh-109344) PyMutex is a one byte lock with fast, inlineable lock and unlock functions for the common uncontended case. The design is based on WebKit's WTF::Lock. PyMutex is built using the _PyParkingLot APIs, which provides a cross-platform futex-like API (based on WebKit's WTF::ParkingLot). This internal API will be used for building other synchronization primitives used to implement PEP 703, such as one-time initialization and events. This also includes tests and a mini benchmark in Tools/lockbench/lockbench.py to compare with the existing PyThread_type_lock. Uncontended acquisition + release: * Linux (x86-64): PyMutex: 11 ns, PyThread_type_lock: 44 ns * macOS (arm64): PyMutex: 13 ns, PyThread_type_lock: 18 ns * Windows (x86-64): PyMutex: 13 ns, PyThread_type_lock: 38 ns PR Overview: The primary purpose of this PR is to implement PyMutex, but there are a number of support pieces (described below). * PyMutex: A 1-byte lock that doesn't require memory allocation to initialize and is generally faster than the existing PyThread_type_lock. The API is internal only for now. * _PyParking_Lot: A futex-like API based on the API of the same name in WebKit. Used to implement PyMutex. * _PyRawMutex: A word sized lock used to implement _PyParking_Lot. * PyEvent: A one time event. This was used a bunch in the "nogil" fork and is useful for testing the PyMutex implementation, so I've included it as part of the PR. * pycore_llist.h: Defines common operations on doubly-linked list. Not strictly necessary (could do the list operations manually), but they come up frequently in the "nogil" fork. ( Similar to https://man.freebsd.org/cgi/man.cgi?queue) --------- Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
2023-09-19 11:54:29 -04:00
// A doubly-linked list that can be embedded in a struct.
//
// Usage:
// struct llist_node head = LLIST_INIT(head);
// typedef struct {
// ...
// struct llist_node node;
// ...
// } MyObj;
//
// llist_insert_tail(&head, &obj->node);
// llist_remove(&obj->node);
//
// struct llist_node *node;
// llist_for_each(node, &head) {
// MyObj *obj = llist_data(node, MyObj, node);
// ...
// }
//
#ifndef Py_INTERNAL_LLIST_H
#define Py_INTERNAL_LLIST_H
#include <stddef.h>
#ifdef __cplusplus
extern "C" {
#endif
#ifndef Py_BUILD_CORE
# error "Py_BUILD_CORE must be defined to include this header"
#endif
struct llist_node {
struct llist_node *next;
struct llist_node *prev;
};
// Get the struct containing a node.
#define llist_data(node, type, member) (_Py_CONTAINER_OF(node, type, member))
gh-108724: Add PyMutex and _PyParkingLot APIs (gh-109344) PyMutex is a one byte lock with fast, inlineable lock and unlock functions for the common uncontended case. The design is based on WebKit's WTF::Lock. PyMutex is built using the _PyParkingLot APIs, which provides a cross-platform futex-like API (based on WebKit's WTF::ParkingLot). This internal API will be used for building other synchronization primitives used to implement PEP 703, such as one-time initialization and events. This also includes tests and a mini benchmark in Tools/lockbench/lockbench.py to compare with the existing PyThread_type_lock. Uncontended acquisition + release: * Linux (x86-64): PyMutex: 11 ns, PyThread_type_lock: 44 ns * macOS (arm64): PyMutex: 13 ns, PyThread_type_lock: 18 ns * Windows (x86-64): PyMutex: 13 ns, PyThread_type_lock: 38 ns PR Overview: The primary purpose of this PR is to implement PyMutex, but there are a number of support pieces (described below). * PyMutex: A 1-byte lock that doesn't require memory allocation to initialize and is generally faster than the existing PyThread_type_lock. The API is internal only for now. * _PyParking_Lot: A futex-like API based on the API of the same name in WebKit. Used to implement PyMutex. * _PyRawMutex: A word sized lock used to implement _PyParking_Lot. * PyEvent: A one time event. This was used a bunch in the "nogil" fork and is useful for testing the PyMutex implementation, so I've included it as part of the PR. * pycore_llist.h: Defines common operations on doubly-linked list. Not strictly necessary (could do the list operations manually), but they come up frequently in the "nogil" fork. ( Similar to https://man.freebsd.org/cgi/man.cgi?queue) --------- Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
2023-09-19 11:54:29 -04:00
// Iterate over a list.
#define llist_for_each(node, head) \
for (node = (head)->next; node != (head); node = node->next)
// Iterate over a list, but allow removal of the current node.
#define llist_for_each_safe(node, head) \
for (struct llist_node *_next = (node = (head)->next, node->next); \
node != (head); node = _next, _next = node->next)
#define LLIST_INIT(head) { &head, &head }
static inline void
llist_init(struct llist_node *head)
{
head->next = head;
head->prev = head;
}
// Returns 1 if the list is empty, 0 otherwise.
static inline int
llist_empty(struct llist_node *head)
{
return head->next == head;
}
// Appends to the tail of the list.
static inline void
llist_insert_tail(struct llist_node *head, struct llist_node *node)
{
node->prev = head->prev;
node->next = head;
head->prev->next = node;
head->prev = node;
}
// Remove a node from the list.
static inline void
llist_remove(struct llist_node *node)
{
struct llist_node *prev = node->prev;
struct llist_node *next = node->next;
prev->next = next;
next->prev = prev;
node->prev = NULL;
node->next = NULL;
}
// Append all nodes from head2 onto head1. head2 is left empty.
static inline void
llist_concat(struct llist_node *head1, struct llist_node *head2)
{
if (!llist_empty(head2)) {
head1->prev->next = head2->next;
head2->next->prev = head1->prev;
head1->prev = head2->prev;
head2->prev->next = head1;
llist_init(head2);
}
}
#ifdef __cplusplus
}
#endif
#endif /* !Py_INTERNAL_LLIST_H */