Repository navigation
Optimize FP64 comparison functions on GPUs with slow FP64 performance - #12231
lennartvoelz wants to merge 12 commits into
Conversation
- adds __fp64_to_ordered_int helper that uses a selection form to convert the bit pattern to int64 with the same ordering as fp64 - adds double overloads for isgreater, isless, isgreaterequal, islessequal and islessgreater - uses _CCCL_DISPATCH_SLOW_FP64_TARGET to dispatch between "slow" and "fast" fp64 arithmetic targets
quite -> quiet in min_max.h
- adds __fp64_to_total_order_int helper that maps the bit pattern to int64 in IEEE totalOrder (-0 < +0) - adds branch-free __fp64_fmax_fmin_int helper that matches the hardware result bitwise (NaN handling, quiet bit) - uses _CCCL_DISPATCH_SLOW_FP64_TARGET in fmax and fmin to dispatch to the integer path for double
- tests for fmin/fmax and the comparison functions isless, islessequal, isgreater, isgreaterequal, islessgreater
…argets the generic fallback (x < y ? y : x) returns either zero for ±0, while the integer path on slow targets orders -0 < +0
miscco
left a comment
miscco
left a comment
There was a problem hiding this comment.
LGTM but I want @fbusato or @davebayer to weigh in too
| #include <cuda/std/__type_traits/is_extended_arithmetic.h> | ||
| #include <cuda/std/__type_traits/promote.h> | ||
| #include <cuda/std/cstdint> | ||
| #include <cuda/std/limits> |
There was a problem hiding this comment.
Nitpick: I believe we can just include the non-extended header here
There was a problem hiding this comment.
Isn't this already the non-extended include?
Or perhaps I don't really understand what it means...
| _CCCL_DISPATCH_SLOW_FP64_TARGET( | ||
| const bool __is_less_equal = ::cuda::std::__fp64_to_ordered_int(__x) <= ::cuda::std::__fp64_to_ordered_int(__y); | ||
| return !(::cuda::std::isnan(__x) || ::cuda::std::isnan(__y)) && __is_less_equal;) | ||
| if (::cuda::std::isnan(__x) || ::cuda::std::isnan(__y)) | ||
| { | ||
| return false; | ||
| } | ||
| return __x <= __y; | ||
| } |
There was a problem hiding this comment.
I am curious as to whether this is something we want to do always, what happens if we do
return !(::cuda::std::isnan(__x) || ::cuda::std::isnan(__y)) && __x <= __y;below?
There was a problem hiding this comment.
I had this thought too, but for the fast path targets (e.g. sm_80) the early return does not result in a branch
|
Ready for review |
|
Important Review skippedReview was skipped as selected files did not have any reviewable changes.
 |
| Layer / File(s) | Summary |
|---|---|
Double comparison predicates libcudacxx/include/cuda/std/__cmath/traits.h, libcudacxx/test/libcudacxx/std/numerics/c.math/fp_traits.pass.cpp, libcudacxx/benchmarks/bench/comparison/basic.cu |
Adds ordered-integer implementations for the five double comparison predicates on slow-FP64 targets. Tests cover signed zeros, NaNs, infinities, denormals, limits, and mixed arguments. Adds benchmarks for the predicates. |
Double min/max functions libcudacxx/include/cuda/std/__cmath/min_max.h, libcudacxx/test/libcudacxx/std/numerics/c.math/fp_min_max.pass.cpp, libcudacxx/benchmarks/bench/fmin_fmax/basic.cu |
Adds integer-based double selection and slow-FP64 dispatch for fmin and fmax. Tests cover signed zeros, NaNs, infinities, denormals, and mixed arguments. Adds benchmarks for both functions. |
Possibly related issues
NVIDIA/cccl issue 11894requests integer-path optimizations for several double-precision<cmath>functions on slow-FP64 GPUs. This PR implements paths for comparison predicates andfmin/fmax; the issue also names functions not changed here.
Suggested reviewers: fbusato, miscco, davebayer
Priority:
Change: Refactor
Merge Risk: 
6f186
-
No actionable issue remains; the new edge-case tests exercise the device paths, so this change is mergeable after normal checks.
Comment @coderabbitai help to get the list of available commands.
Description
Part of #11894
Comparison functions
isgreater,isless,isgreaterequal,islessequalandislessgreatercompile toDSETPfordouble, which is slow on GPUs with low FP64 throughput -> idea to convert to int64 representation with the same ordering as double for the comparison__fp64_to_ordered_intmaps the bits ofdoubletoint64_twith the same ordering:I chose a selection form:
key = x < 0 ? INT64_MIN - x : x -> -0 and +0 map to the same value
The function is local to
traits.h, as discussed in [FEA]: Optimize double-precision<cmath>functionalities with integer path #11894isnanfast pathBenchmarks:
No FP64 instructions are emitted by the compiler on slow FP64 targets.
fmax/fmin
__fp64_to_total_order_intmaps the bits of adoubletoint64_tin total order, i.e. -0 < +0. NaNs are placed beyond +-inf__fp64_fmax_fmin_intadds function template that provides branch free min/max paths for doubles with integer instructions. -0 < +0, single NaN ignored and two NaNs return first argument quietedBenchmarks:
fmax/fminalso affect downstream paths in CUB viacuda::std::minimum/maximum:The only downstream benchmark negatively affected is using fabs, which is the reason for the slowdown. I'll address this in a separate PR.
Added tests for comparisons and fmin/fmax that cover edge cases
Checklist