Reapply "Modules: Cache PCMs in memory and avoid a use-after-free"
This reverts commit r298185, effectively reapplying r298165, after fixing the new unit tests (PR32338). The memory buffer generator doesn't null-terminate the MemoryBuffer it creates; this version of the commit informs getMemBuffer about that to avoid the assert. Original commit message follows: ---- Clang's internal build system for implicit modules uses lock files to ensure that after a process writes a PCM it will read the same one back in (without contention from other -cc1 commands). Since PCMs are read from disk repeatedly while invalidating, building, and importing, the lock is not released quickly. Furthermore, the LockFileManager is not robust in every environment. Other -cc1 commands can stall until timeout (after about eight minutes). This commit changes the lock file from being necessary for correctness to a (possibly dubious) performance hack. The remaining benefit is to reduce duplicate work in competing -cc1 commands which depend on the same module. Follow-up commits will change the internal build system to continue after a timeout, and reduce the timeout. Perhaps we should reconsider blocking at all. This also fixes a use-after-free, when one part of a compilation validates a PCM and starts using it, and another tries to swap out the PCM for something new. The PCMCache is a new type called MemoryBufferCache, which saves memory buffers based on their filename. Its ownership is shared by the CompilerInstance and ModuleManager. - The ModuleManager stores PCMs there that it loads from disk, never touching the disk if the cache is hot. - When modules fail to validate, they're removed from the cache. - When a CompilerInstance is spawned to build a new module, each already-loaded PCM is assumed to be valid, and is frozen to avoid the use-after-free. - Any newly-built module is written directly to the cache to avoid the round-trip to the filesystem, making lock files unnecessary for correctness. Original patch by Manman Ren; most testcases by Adrian Prantl! git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@298278 91177308-0d34-0410-b5e6-96231b3b80d8
Showing
- include/clang/Basic/DiagnosticSerializationKinds.td 5 additions, 0 deletionsinclude/clang/Basic/DiagnosticSerializationKinds.td
- include/clang/Basic/MemoryBufferCache.h 80 additions, 0 deletionsinclude/clang/Basic/MemoryBufferCache.h
- include/clang/Frontend/ASTUnit.h 2 additions, 0 deletionsinclude/clang/Frontend/ASTUnit.h
- include/clang/Frontend/CompilerInstance.h 7 additions, 1 deletioninclude/clang/Frontend/CompilerInstance.h
- include/clang/Lex/Preprocessor.h 4 additions, 0 deletionsinclude/clang/Lex/Preprocessor.h
- include/clang/Serialization/ASTReader.h 3 additions, 0 deletionsinclude/clang/Serialization/ASTReader.h
- include/clang/Serialization/ASTWriter.h 5 additions, 0 deletionsinclude/clang/Serialization/ASTWriter.h
- include/clang/Serialization/Module.h 3 additions, 3 deletionsinclude/clang/Serialization/Module.h
- include/clang/Serialization/ModuleManager.h 7 additions, 1 deletioninclude/clang/Serialization/ModuleManager.h
- lib/Basic/CMakeLists.txt 1 addition, 0 deletionslib/Basic/CMakeLists.txt
- lib/Basic/MemoryBufferCache.cpp 48 additions, 0 deletionslib/Basic/MemoryBufferCache.cpp
- lib/Frontend/ASTUnit.cpp 10 additions, 4 deletionslib/Frontend/ASTUnit.cpp
- lib/Frontend/CompilerInstance.cpp 17 additions, 6 deletionslib/Frontend/CompilerInstance.cpp
- lib/Lex/Preprocessor.cpp 5 additions, 5 deletionslib/Lex/Preprocessor.cpp
- lib/Serialization/ASTReader.cpp 62 additions, 20 deletionslib/Serialization/ASTReader.cpp
- lib/Serialization/ASTWriter.cpp 10 additions, 2 deletionslib/Serialization/ASTWriter.cpp
- lib/Serialization/GeneratePCH.cpp 2 additions, 1 deletionlib/Serialization/GeneratePCH.cpp
- lib/Serialization/ModuleManager.cpp 20 additions, 7 deletionslib/Serialization/ModuleManager.cpp
- test/Modules/Inputs/system-out-of-date/X.h 1 addition, 0 deletionstest/Modules/Inputs/system-out-of-date/X.h
- test/Modules/Inputs/system-out-of-date/Y.h 1 addition, 0 deletionstest/Modules/Inputs/system-out-of-date/Y.h
Loading
Please register or sign in to comment