Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Misc]: CMake Clean-up / Refactor Tasks #9129

Open
6 tasks
LucasWilkinson opened this issue Oct 7, 2024 · 0 comments
Open
6 tasks

[Misc]: CMake Clean-up / Refactor Tasks #9129

LucasWilkinson opened this issue Oct 7, 2024 · 0 comments
Labels

Comments

@LucasWilkinson
Copy link
Contributor

LucasWilkinson commented Oct 7, 2024

In an effort to make the CMake more readable, stable and easy to use we have a few tasks we'd like to work on, creating a GitHub issue here to track that progress, some planned changes/investigations:

  • Have vllm-flash-attn use ExternalProject currently vllm-flash-attn uses the parent CMake scope which creates many footguns since it is in a separate repo, using ExternalProject will mean that the vllm-flash-attn will be run in a separate CMake scope/process
  • Warn that PTX builds are not currently supported (post [CI/Build] Per file CUDA Archs (improve wheel size and dev build times) #8845), currently if there is a +PTX in TORCH_CUDA_ARCH_LIST this will be ignored. We should warn when this is the case. Alternatively we can add support for PTX builds although this is generally not desirable since PTX increases the wheel size by quite a bit (PTX is larger than SASS), and we already build for all currently supported arches.
  • Rename define_gpu_extension_target, currently this is used for CPU extensions too so the name is now misleading
  • Potential build both C++ and CUDA extensions when building for CUDA and using torch dispatcher to dispatch between the two, [Kernel] Factor registrations #8424
  • Look into removing early returns in CMakeLists.txt (potentially move backends into its own files)
  • Add a CI test of local builds, i.e. pip install -e .
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant