In 2.4.*, mlockall(MCL_FUTURE) is erroneously inherited by child processes across fork() and exec(): 1. across a fork(), the inherited memory is not locked, but any new memory allocations by the child will be VM_LOCKED. 2. across a subsequent exec(), *all* of the exec'd program's memory except for its stack pages will be VM_LOCKED. The problem is: 1. if MCL_FUTURE, mm->def_flags gets set to VM_LOCKED in do_mlockall(). 2. mm->def_flags is not cleared during subsequent forks and execs. 3 mm->def_flags, with the leftover VM_LOCKED flag set, is subsequently utilized in calc_vm_flags() when called by do_brk() to extend the address space of a forked process, and by do_mmap_pgoff() when building the non-stack address space of an exec'd process. The proposed patch puts the fix in mm_init(), which seems to be the most appropriate place since it's called by copy_mm(), and by mm_alloc() on behalf of exec_mmap(): # diff -u linux/kernel/fork.c linux-2.4.17/kernel/fork.c --- linux/kernel/fork.c Tue Jan 8 15:11:13 2002 +++ linux-2.4.17/kernel/fork.c Tue Jan 8 15:12:26 2002 @@ -219,6 +219,7 @@ init_rwsem(&mm->mmap_sem); mm->page_table_lock = SPIN_LOCK_UNLOCKED; mm->pgd = pgd_alloc(mm); + mm->def_flags = 0; if (mm->pgd) return mm; free_mm(mm); Note that it worked OK in 2.2 because mm->def_flags was explicitly cleared in mm_alloc(), which was called by both copy_mm() and exec_mmap(). But things were shuffled around a bit in 2.4, and it must have gotten lost in the translation... Dave Anderson ============================================================================== David Anderson anderson@mclinux.com Mission Critical Linux, Inc. http://www.mclinux.com 100 Foot of John St. Work: 978-606-0225 Lowell, MA 01852 Fax: 978-446-9470 ============================================================================== - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/