-
-
Notifications
You must be signed in to change notification settings - Fork 34.2k
Open
Labels
3.14bugs and security fixesbugs and security fixes3.15new features, bugs and security fixesnew features, bugs and security fixestopic-free-threadingtype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
Description
This bug was originally found by a colleague. Memory usage (RSS) keeps increasing even though allocated blocks stays essentially constant. Self contained reproducer below:
import gc
import sys
import os
import concurrent.futures
def get_rss_mb():
with open("/proc/self/statm") as f:
return int(f.read().split()[1]) * os.sysconf("SC_PAGE_SIZE") / (1024 * 1024)
def work(_):
return [bytearray(64) for _ in range(4000)]
NUM_ITERATIONS = int(sys.argv[1]) if len(sys.argv) > 1 else 20
pool = concurrent.futures.ThreadPoolExecutor(max_workers=4)
for i in range(NUM_ITERATIONS):
futures = [pool.submit(work, j) for j in range(100)]
results = [f.result() for f in futures]
del futures, results # free worker-allocated objects on main thread
gc.collect()
gc.collect()
blocks = sys.getallocatedblocks()
print(f"iter {i:3d}: RSS={get_rss_mb():7.1f} MB {blocks=}")
pool.shutdown()
print(f"after shutdown: RSS={get_rss_mb():7.1f} MB")The problem is that we end up having orphaned mimalloc pages in the full queue, even though those pages have available blocks or are completely empty. They're "leaked" until the thread exits.
I think #145614 is also related.
I think there are at least two underlying bugs here:
_PyMem_mi_page_maybe_freeleaves empty pages in the full page queue. This bug can happen when a page goes from "full" to "all free"._mi_page_free_collectcalls_PyMem_mi_page_clear_qsbrifpage->local_free != NULL. This can get erroneously triggered even the by traversing heaps. We only want to clear the page QSBR counter when it becomes empty again.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
3.14bugs and security fixesbugs and security fixes3.15new features, bugs and security fixesnew features, bugs and security fixestopic-free-threadingtype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error