Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Sven Woop
|
98b2d5eeaf | ||
|
Sven Woop
|
7b9518fa99 | ||
|
EC TF Bind account
|
2f614dd107 | ||
|
Louis Feng
|
61d5413855 |
@@ -3,5 +3,3 @@ Win32
|
||||
x64
|
||||
*.suo
|
||||
*.ncb
|
||||
.*sw?
|
||||
.DS_Store
|
||||
|
||||
-460
@@ -1,460 +0,0 @@
|
||||
Version History
|
||||
---------------
|
||||
|
||||
### New Features in Embree 2.16.5
|
||||
- Bugfix in the robust triangle intersector that rarely caused NaNs.
|
||||
- Fixed bug in hybrid traversal kernel when BVH leaf was entered with no
|
||||
active rays. This rarely caused crashes when used with instancing.
|
||||
- Fixed bug introduced in Embree 2.16.2 which caused instancing not to
|
||||
work properly when a smaller than the native SIMD width was
|
||||
used in ray packet mode.
|
||||
- Fixed bug in the curve geometry intersector that caused rendering
|
||||
artefacts for Bézier curves with p0=p1 and/or p2=p3.
|
||||
- Fixed bug in the curve geometry intersector that caused hit results
|
||||
with NaNs to be reported.
|
||||
- Fixed masking bug that caused rare cracks in curve geometry.
|
||||
- Enabled support for SSE2 in precompiled binaries again.
|
||||
|
||||
### New Features in Embree 2.16.4
|
||||
- Bugfix in the ribbon intersector for hair primitives. Non-normalized
|
||||
rays caused wrong intersection distance to be reported.
|
||||
|
||||
### New Features in Embree 2.16.3
|
||||
- Increased accuracy for handling subdivision surfaces. This fixes
|
||||
cracks when using displacement mapping but reduces performance
|
||||
at irregular vertices.
|
||||
- Fixed a bug where subdivision geometry was not properly updated
|
||||
when modifying only the tesselation rate and vertex array.
|
||||
|
||||
### New Features in Embree 2.16.2
|
||||
- Fixed bug that caused NULL intersection context in intersection
|
||||
filter when instancing was used.
|
||||
- Fixed an issue where uv's where outside the triangle (or quad) for
|
||||
very small triangles (or quads). In robust mode we improved the uv
|
||||
calculation to avoid that issue, in fast mode we accept that
|
||||
inconsistency for better performance.
|
||||
- Changed UV encoding for non-quad subdivision patches to
|
||||
allow a subpatch UV range of `[-0.5,1.5[`. Using this new encoding
|
||||
one can use finite differences to calculate derivatives if required.
|
||||
Please adjust your code in case you rely on the old encoding.
|
||||
|
||||
### New Features in Embree 2.16.1
|
||||
- Workaround for compile issues with Visual Studio 2017
|
||||
- Fixed bug in subdiv code for static scenes when using tessellation
|
||||
levels larger than 50.
|
||||
- Fixed low performance when adding many geometries to a scene.
|
||||
- Fixed high memory consumption issue when using instances in
|
||||
dynamic scene (by disabling two level builder for user geometries
|
||||
and instances).
|
||||
|
||||
### New Features in Embree 2.16.0
|
||||
- Improved multi-segment motion blur support for scenes with
|
||||
different number of time steps per mesh.
|
||||
- New top level BVH builder that improves build times and BVH quality
|
||||
of two-level BVHs.
|
||||
- Added support to enable only a single ISA. Previously code was
|
||||
always compiled for SSE2.
|
||||
- Improved single ray tracing performance for incoherent rays on
|
||||
AVX-512 architectures by 5-10%.
|
||||
- Improved packet/hybrid ray tracing performance for incoherent rays
|
||||
on AVX-512 architectures by 10-30%.
|
||||
- Improved stream ray tracing performance for coherent rays in
|
||||
structure-of-pointers layout by 40-70%.
|
||||
- BVH builder for compact scenes of triangles and quads needs
|
||||
essentially no temporary memory anymore. This doubles the
|
||||
maximal scene size that can be rendered in compact mode.
|
||||
- Triangles no longer store the geometry normal in fast/default mode
|
||||
which reduces memory consumption by up to 20%.
|
||||
- Compact mode uses BVH4 now consistently which reduces memory
|
||||
consumption by up to 10%.
|
||||
- Reduced memory consumption for small scenes (of 10k-100k primitives)
|
||||
and dynamic scenes.
|
||||
- Improved performance of user geometries and instances through BVH8
|
||||
support.
|
||||
- The API supports now specifying the geometry ID of a geometry at
|
||||
construction time. This way matching the geometry ID used by
|
||||
Embree and the application is simplified.
|
||||
- Fixed a bug that would have caused a failure of the BVH builder
|
||||
for dynamic scenes when run on a machine with more then 1000 threads.
|
||||
- Fixed a bug that could have been triggered when reaching the maximal
|
||||
number of mappings under Linux (`vm.max_map_count`). This could have
|
||||
happened when creating a large number of small static scenes.
|
||||
- Added huge page support for Windows and MacOSX (experimental).
|
||||
- Added support for Visual Studio 2017.
|
||||
- Removed support for Visual Studio 2012.
|
||||
- Precompiled binaries now require a CPU supporting at least the
|
||||
SSE4.2 ISA.
|
||||
- We no longer provide precompiled binaries for 32 bit on Windows.
|
||||
- Under Windows one now has to use the platform toolset option in
|
||||
CMake to switch to Clang or the Intel® Compiler.
|
||||
- Fixed a bug for subdivision meshes when using the incoherent scene
|
||||
flag.
|
||||
- Fixed a bug in the line geometry intersection, that caused reporting
|
||||
an invalid line segment intersection with primID -1.
|
||||
- Buffer stride for vertex buffers of different time steps of triangle
|
||||
and quad meshes have to be identical now.
|
||||
- Fixed a bug in the curve geometry intersection code when passed a
|
||||
perfect cylinder.
|
||||
|
||||
### New Features in Embree 2.15.0
|
||||
|
||||
- Added `rtcCommitJoin` mode that allows thread to join a build
|
||||
operation. When using the internal tasking system this allows
|
||||
Embree to solely use the threads that called `rtcCommitJoin` to
|
||||
build the scene, while previously also normal worker threads
|
||||
participated in the build. You should no longer use `rtcCommit`
|
||||
to join a build.
|
||||
- Added `rtcDeviceSetErrorFunction2` API call, which sets an error
|
||||
callback function which additionally gets passed a user provided
|
||||
pointer (`rtcDeviceSetErrorFunction` is now deprecated).
|
||||
- Added `rtcDeviceSetMemoryMonitorFunction2` API call, which sets a
|
||||
memory monitor callback function which additionally get passed a
|
||||
user provided pointer. (`rtcDeviceSetMemoryMonitorFunction` is now
|
||||
deprecated).
|
||||
- Build performance for hair geometry improved by up to 2×.
|
||||
- Standard BVH build performance increased by 5%.
|
||||
- Added API extension to use internal Morton-code based builder, the
|
||||
standard binned-SAH builder, and the spatial split-based SAH builder.
|
||||
- Added support for BSpline hair and curves. Embree uses
|
||||
either the Bézier or BSpline basis internally, and converts other
|
||||
curves, which requires more memory during rendering. For reduced
|
||||
memory consumption set the `EMBREE_NATIVE_SPLINE_BASIS` to the basis
|
||||
your application uses (which is set to `BEZIER` by default).
|
||||
- Setting the number of threads through `tbb::taskscheduler_init`
|
||||
object on the application side is now working properly.
|
||||
- Windows and Linux releases are build using AVX-512 support.
|
||||
- Implemented hybrid traversal for hair and line segments for
|
||||
improved ray packet performance.
|
||||
- AVX-512 code compiles with Clang 4.0.0
|
||||
- Fixed crash when ray packets were disabled in CMake.
|
||||
|
||||
### New Features in Embree 2.14.0
|
||||
|
||||
- Added `ignore_config_files` option to init flags that allows the
|
||||
application to ignore Embree configuration files.
|
||||
- Face-varying interpolation is now supported for subdivision surfaces.
|
||||
- Up to 16 user vertex buffers are supported for vertex
|
||||
attribute interpolation.
|
||||
- Deprecated `rtcSetBoundaryMode` function, please use the new
|
||||
`rtcSetSubdivisionMode` function.
|
||||
- Added `RTC_SUBDIV_PIN_BOUNDARY` mode for handling boundaries of
|
||||
subdivision meshes.
|
||||
- Added `RTC_SUBDIV_PIN_ALL` mode to enforce linear interpolation
|
||||
for subdivision meshes.
|
||||
- Optimized object generation performance for dynamic scenes.
|
||||
- Reduced memory consumption when using lots of small dynamic objects.
|
||||
- Fixed bug for subdivision surfaces using low tessellation rates.
|
||||
- Hair geometry now uses a new ribbon intersector that intersects with
|
||||
ray-facing quads. The new intersector also returns the v-coordinate
|
||||
of the hair intersection, and fixes artefacts at junction points
|
||||
between segments, at the cost of a small performance hit.
|
||||
- Added `rtcSetBuffer2` function, that additionally gets the number of
|
||||
elements of a buffer. In dynamic scenes, this function allows to
|
||||
quickly change buffer sizes, making it possible to change the number
|
||||
of primitives of a mesh or the number of crease features for
|
||||
subdivision surfaces.
|
||||
- Added simple 'viewer_anim' tutorial for rendering key
|
||||
frame animations and 'buildbench' for measuring BVH (re-)build
|
||||
performance for static and dynamic scenes.
|
||||
- Added more AVX-512 optimizations for future architectures.
|
||||
|
||||
### New Features in Embree 2.13.0
|
||||
|
||||
- Improved performance for compact (but not robust) scenes.
|
||||
- Added robust mode for motion blurred triangles and quads.
|
||||
- Added fast dynamic mode for user geometries.
|
||||
- Up to 20% faster BVH build performance on the second generation
|
||||
Intel® Xeon Phi™ processor codenamed Knights Landing.
|
||||
- Improved quality of the spatial split builder.
|
||||
- Improved performance for coherent streams of ray packets (SOA
|
||||
layout), e.g. for fast primary visibility.
|
||||
- Various bug fixes in tessellation cache, quad-based spatial
|
||||
split builder, etc.
|
||||
|
||||
### New Features in Embree 2.12.0
|
||||
|
||||
- Added support for multi-segment motion blur for all primitive types.
|
||||
- API support for stream of pointers to single rays (`rtcIntersect1Mp`
|
||||
and `rtcOccluded1Mp`)
|
||||
- Improved BVH refitting performance for dynamic scenes.
|
||||
- Improved high-quality mode for quads (added spatial split builder
|
||||
for quads)
|
||||
- Faster dynamic scenes for triangle and quad-based meshes on AVX2
|
||||
enabled machines.
|
||||
- Performance and correctness bugfix in optimization for streams of
|
||||
coherent (single) rays.
|
||||
- Fixed large memory consumption (issue introduced in Embree v2.11.0).
|
||||
If you use Embree v2.11.0 please upgrade to Embree v2.12.0.
|
||||
- Reduced memory consumption for dynamic scenes containing small
|
||||
meshes.
|
||||
- Added support to start and affinitize TBB worker threads by passing
|
||||
"`start_threads=1,set_affinity=1`" to `rtcNewDevice`. These settings
|
||||
are recommended on systems with a high thread count.
|
||||
- `rtcInterpolate2` can now be called within a displacement shader.
|
||||
- Added initial support for Microsoft's Parallel Pattern Library (PPL)
|
||||
as tasking system alternative (for optimal performance TBB is
|
||||
highly recommended).
|
||||
- Updated to TBB 2017 which is released under the Apache v2.0 license.
|
||||
- Dropped support for Visual Studio 2012 Win32 compiler. Visual Studio
|
||||
2012 x64 is still supported.
|
||||
|
||||
### New Features in Embree 2.11.0
|
||||
|
||||
- Improved performance for streams of coherent (single) rays flagged
|
||||
with `RTC_INTERSECT_COHERENT`. For such coherent ray streams, e.g.
|
||||
primary rays, the performance typically improves by 1.3-2×.
|
||||
- New spatial split BVH builder for triangles, which is 2-6× faster
|
||||
than the previous version and more memory conservative.
|
||||
- Improved performance and scalability of all standard BVH builders on
|
||||
systems with large core counts.
|
||||
- Fixed `rtcGetBounds` for motion blur scenes.
|
||||
- Thread affinity is now on by default when running on the latest
|
||||
Intel® Xeon Phi™ processor.
|
||||
- Added AVX-512 support for future Intel® Xeon processors.
|
||||
|
||||
### New Features in Embree 2.10.0
|
||||
|
||||
- Added a new curve geometry which renders the sweep surface of a
|
||||
circle along a Bézier curve.
|
||||
- Intersection filters can update the `tfar` ray distance.
|
||||
- Geometry types can get disabled at compile time.
|
||||
- Modified and extended the ray stream API.
|
||||
- Added new callback mechanism for the ray stream API.
|
||||
- Improved ray stream performance (up to 5-10%).
|
||||
- Up to 20% faster morton builder on machines with large core counts.
|
||||
- Lots of optimizations for the second generation Intel® Xeon Phi™
|
||||
processor codenamed Knights Landing.
|
||||
- Added experimental support for compressed BVH nodes (reduces node
|
||||
size to 56-62% of uncompressed size). Compression introduces a
|
||||
typical performance overhead of ~10%.
|
||||
- Bugfix in backface culling mode. We do now properly cull the
|
||||
backfaces and not the frontfaces.
|
||||
- Feature freeze for the first generation Intel® Xeon Phi™ coprocessor
|
||||
codenamed Knights Corner. We will still maintain and add bug fixes
|
||||
to Embree v2.9.0, but Embree 2.10 and future versions will no longer
|
||||
support it.
|
||||
|
||||
### New Features in Embree 2.9.0
|
||||
|
||||
- Improved shadow ray performance (10-100% depending on the scene).
|
||||
- Added initial support for ray streams (10-30% higher performance
|
||||
depending on ray coherence in the stream).
|
||||
- Added support to calculate second order derivatives using the
|
||||
`rtcInterpolate2` function.
|
||||
- Changed the parametrization for triangular subdivision faces to
|
||||
the same scheme used for pentagons.
|
||||
- Added support to query the Embree configuration using the
|
||||
`rtcDeviceGetParameter` function.
|
||||
|
||||
### New Features in Embree 2.8.1
|
||||
|
||||
- Added support for setting per geometry tessellation rate (supported
|
||||
for subdivision and Bézier geometries).
|
||||
- Added support for motion blurred instances.
|
||||
|
||||
### New Features in Embree 2.8.0
|
||||
|
||||
- Added support for line segment geometry.
|
||||
- Added support for quad geometry (replaces triangle-pairs feature).
|
||||
- Added support for linear motion blur of user geometries.
|
||||
- Improved performance through AVX-512 optimizations.
|
||||
- Improved performance of lazy scene build (when using TBB 4.4 update
|
||||
2).
|
||||
- Improved performance through huge page support under linux.
|
||||
|
||||
### New Features in Embree 2.7.1
|
||||
|
||||
- Internal tasking system supports cancellation of build operations.
|
||||
- ISPC mode for robust and compact scenes got significantly faster
|
||||
(implemented hybrid traversal for bvh4.triangle4v and
|
||||
bvh4.triangle4i).
|
||||
- Hair rendering got faster as we fixed some issues with the SAH
|
||||
heuristic cost factors.
|
||||
- BVH8 got slight faster for single ray traversal (improved sorting
|
||||
when hitting more than 4 boxes).
|
||||
- BVH build performance got up to 30% faster on CPUs with high core
|
||||
counts (improved parallel partition code).
|
||||
- High quality build mode again working properly (spatial splits had
|
||||
been deactivated in v2.7.0 due to some bug).
|
||||
- Support for merging two adjacent triangles sharing a common edge
|
||||
into a triangle-pair primitive (can reduce memory consumption and
|
||||
BVH build times by up to 50% for mostly quad-based input meshes).
|
||||
- Internal cleanups (reduced number of traversal kernels by more
|
||||
templating).
|
||||
- Reduced stack size requirements of BVH builders.
|
||||
- Fixed crash for dynamic scenes, triggered by deleting all
|
||||
geometries from the scene.
|
||||
|
||||
### New Features in Embree 2.7.0
|
||||
|
||||
- Added device concept to Embree to allow different components of an
|
||||
application to use Embree without interfering with each other.
|
||||
- Fixed memory leak in twolevel builder used for dynamic scenes.
|
||||
- Fixed bug in tesselation cache that caused crashes for subdivision
|
||||
surfaces.
|
||||
- Fixed bug in internal task scheduler that caused deadlocks when
|
||||
using `rtcCommitThread`.
|
||||
- Improved hit-distance accuracy for thin triangles in robust mode.
|
||||
- Added support to disable ray packet support in cmake.
|
||||
|
||||
### New Features in Embree 2.6.2
|
||||
|
||||
- Fixed bug triggered by instantiating motion blur geometry.
|
||||
- Fixed bug in hit UV coordinates of static subdivision geometries.
|
||||
- Performance improvements when only changing tessellation levels for
|
||||
subdivision geometry per frame.
|
||||
- Added ray packet intersectors for subdivision geometry, resulting in
|
||||
improved performance for coherent rays.
|
||||
- Reduced virtual address space usage for static geometries.
|
||||
- Fixed some AVX2 code paths when compiling with GCC or Clang.
|
||||
- Bugfix for subdiv patches with non-matching winding order.
|
||||
- Bugfix in ISA detection of AVX-512.
|
||||
|
||||
### New Features in Embree 2.6.1
|
||||
|
||||
- Major performance improvements for ray tracing subdivision surfaces,
|
||||
e.g. up to 2× faster for scenes where only the tessellation levels
|
||||
are changing per frame, and up to 3× faster for scenes with lots of
|
||||
crease features
|
||||
- Initial support for architectures supporting the new 16-wide AVX-512
|
||||
ISA
|
||||
- Implemented intersection filter callback support for subdivision
|
||||
surfaces
|
||||
- Added `RTC_IGNORE_INVALID_RAYS` CMake option which makes the ray
|
||||
intersectors more robust against full tree traversal caused by
|
||||
invalid ray inputs (e.g. INF, NaN, etc)
|
||||
|
||||
### New Features in Embree 2.6.0
|
||||
|
||||
- Added `rtcInterpolate` function to interpolate per vertex
|
||||
attributes
|
||||
- Added `rtcSetBoundaryMode` function that can be used to select the
|
||||
boundary handling for subdivision surfaces
|
||||
- Fixed a traversal bug that caused rays with very small ray
|
||||
direction components to miss geometry
|
||||
- Performance improvements for the robust traversal mode
|
||||
- Fixed deadlock when calling `rtcCommit` from multiple
|
||||
threads on same scene
|
||||
|
||||
### New Features in Embree 2.5.1
|
||||
|
||||
- On dual socket workstations, the initial BVH build performance
|
||||
almost doubled through a better memory allocation scheme
|
||||
- Reduced memory usage for subdivision surface objects with crease
|
||||
features
|
||||
- `rtcCommit` performance is robust against unset "flush to zero" and
|
||||
"denormals are zero" flags. However, enabling these flags in your
|
||||
application is still recommended
|
||||
- Reduced memory usage for subdivision surfaces with borders and
|
||||
infinitely sharp creases
|
||||
- Lots of internal cleanups and bug fixes for both Intel® Xeon® and
|
||||
Intel® Xeon Phi™
|
||||
|
||||
### New Features in Embree 2.5.0
|
||||
|
||||
- Improved hierarchy build performance on both Intel Xeon and Intel
|
||||
Xeon Phi
|
||||
- Vastly improved tessellation cache for ray tracing subdivision
|
||||
surfaces
|
||||
- Added `rtcGetUserData` API call to query per geometry user pointer
|
||||
set through `rtcSetUserData`
|
||||
- Added support for memory monitor callback functions to track and
|
||||
limit memory consumption
|
||||
- Added support for progress monitor callback functions to track build
|
||||
progress and cancel long build operations
|
||||
- BVH builders can be used to build user defined hierarchies inside
|
||||
the application (see tutorial [BVH Builder])
|
||||
- Switched to TBB as default tasking system on Xeon to get even faster
|
||||
hierarchy build times and better integration for applications that
|
||||
also use TBB
|
||||
- `rtcCommit` can get called from multiple TBB threads to join the
|
||||
hierarchy build operations
|
||||
|
||||
### New Features in Embree 2.4
|
||||
|
||||
- Support for Catmull Clark subdivision surfaces (triangle/quad base
|
||||
primitives)
|
||||
- Support for vector displacements on Catmull Clark subdivision
|
||||
surfaces
|
||||
- Various bug fixes (e.g. 4-byte alignment of vertex buffers works)
|
||||
|
||||
### New Features in Embree 2.3.3
|
||||
|
||||
- BVH builders more robustly handle invalid input data (Intel Xeon
|
||||
processor family)
|
||||
- Motion blur support for hair geometry (Xeon)
|
||||
- Improved motion blur performance for triangle geometry (Xeon)
|
||||
- Improved robust ray tracing mode (Xeon)
|
||||
- Added `rtcCommitThread` API call for easier integration into
|
||||
existing tasking systems (Xeon and Intel Xeon Phi coprocessor)
|
||||
- Added support for recording and replaying all
|
||||
`rtcIntersect`/`rtcOccluded` calls (Xeon and Xeon Phi)
|
||||
|
||||
### New Features in Embree 2.3.2
|
||||
|
||||
- Improved mixed AABB/OBB-BVH for hair geometry (Xeon Phi)
|
||||
- Reduced amount of pre-allocated memory for BVH builders (Xeon Phi)
|
||||
- New 64 bit Morton code-based BVH builder (Xeon Phi)
|
||||
- (Enhanced) Morton code-based BVH builders use now tree rotations to
|
||||
improve BVH quality (Xeon Phi)
|
||||
- Bug fixes (Xeon and Xeon Phi)
|
||||
|
||||
### New Features in Embree 2.3.1
|
||||
|
||||
- High quality BVH mode improves spatial splits which result in up to
|
||||
30% performance improvement for some scenes (Xeon)
|
||||
- Compile time enabled intersection filter functions do not reduce
|
||||
performance if no intersection filter is used in the scene (Xeon and
|
||||
Xeon Phi)
|
||||
- Improved ray tracing performance for hair geometry by \>20% on Xeon
|
||||
Phi. BVH for hair geometry requires 20% less memory
|
||||
- BVH8 for AVX/AVX2 targets improves performance for single ray
|
||||
tracing on Haswell by up to 12% and by up to 5% for hybrid (Xeon)
|
||||
- Memory conservative BVH for Xeon Phi now uses BVH node quantization
|
||||
to lower memory footprint (requires half the memory footprint of the
|
||||
default BVH)
|
||||
|
||||
### New Features in Embree 2.3
|
||||
|
||||
- Support for ray tracing hair geometry (Xeon and Xeon Phi)
|
||||
- Catching errors through error callback function
|
||||
- Faster hybrid traversal (Xeon and Xeon Phi)
|
||||
- New memory conservative BVH for Xeon Phi
|
||||
- Faster Morton code-based builder on Xeon
|
||||
- Faster binned-SAH builder on Xeon Phi
|
||||
- Lots of code cleanups/simplifications/improvements (Xeon and Xeon
|
||||
Phi)
|
||||
|
||||
### New Features in Embree 2.2
|
||||
|
||||
- Support for motion blur on Xeon Phi
|
||||
- Support for intersection filter callback functions
|
||||
- Support for buffer sharing with the application
|
||||
- Lots of AVX2 optimizations, e.g. \~20% faster 8-wide hybrid
|
||||
traversal
|
||||
- Experimental support for 8-wide (AVX/AVX2) and 16-wide BVHs (Xeon
|
||||
Phi)
|
||||
|
||||
### New Features in Embree 2.1
|
||||
|
||||
- New future proof API with a strong focus on supporting dynamic
|
||||
scenes
|
||||
- Lots of optimizations for 8-wide AVX2 (Haswell architecture)
|
||||
- Automatic runtime code selection for SSE, AVX, and AVX2
|
||||
- Support for user-defined geometry
|
||||
- New and improved BVH builders:
|
||||
- Fast adaptive Morton code-based builder (without SAH-based
|
||||
top-level rebuild)
|
||||
- Both the SAH and Morton code-based builders got faster (Xeon
|
||||
Phi)
|
||||
- New variant of the SAH-based builder using triangle pre-splits
|
||||
(Xeon Phi)
|
||||
|
||||
### New Features in Embree 2.0
|
||||
|
||||
- Support for the Intel® Xeon Phi™ coprocessor platform
|
||||
- Support for high-performance "packet" kernels on SSE, AVX, and Xeon
|
||||
Phi
|
||||
- Integration with the Intel® SPMD Program Compiler (ISPC)
|
||||
- Instantiation and fast BVH reconstruction
|
||||
- Example photo-realistic rendering engine for both C++ and ISPC
|
||||
+177
-321
@@ -1,5 +1,5 @@
|
||||
## ======================================================================== ##
|
||||
## Copyright 2009-2017 Intel Corporation ##
|
||||
## Copyright 2009-2014 Intel Corporation ##
|
||||
## ##
|
||||
## Licensed under the Apache License, Version 2.0 (the "License"); ##
|
||||
## you may not use this file except in compliance with the License. ##
|
||||
@@ -14,365 +14,218 @@
|
||||
## limitations under the License. ##
|
||||
## ======================================================================== ##
|
||||
|
||||
SET(EMBREE_VERSION_MAJOR 2)
|
||||
SET(EMBREE_VERSION_MINOR 17)
|
||||
SET(EMBREE_VERSION_PATCH 0)
|
||||
SET(EMBREE_VERSION_NOTE "-alpha.0")
|
||||
SET(EMBREE_VERSION ${EMBREE_VERSION_MAJOR}.${EMBREE_VERSION_MINOR}.${EMBREE_VERSION_PATCH})
|
||||
MATH(EXPR EMBREE_VERSION_NUMBER "10000*${EMBREE_VERSION_MAJOR} + 100*${EMBREE_VERSION_MINOR} + ${EMBREE_VERSION_PATCH}")
|
||||
SET(CPACK_RPM_PACKAGE_RELEASE 1)
|
||||
PROJECT(embree)
|
||||
|
||||
PROJECT(embree${EMBREE_VERSION_MAJOR})
|
||||
CMAKE_MINIMUM_REQUIRED(VERSION 2.4)
|
||||
|
||||
CMAKE_MINIMUM_REQUIRED(VERSION 2.8.11)
|
||||
|
||||
# find git version
|
||||
IF(EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/.git)
|
||||
FIND_PACKAGE(Git)
|
||||
IF(GIT_FOUND)
|
||||
EXECUTE_PROCESS(
|
||||
COMMAND ${GIT_EXECUTABLE} rev-parse HEAD
|
||||
WORKING_DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}"
|
||||
OUTPUT_VARIABLE "EMBREE_HASH"
|
||||
ERROR_QUIET
|
||||
OUTPUT_STRIP_TRAILING_WHITESPACE)
|
||||
ELSE()
|
||||
SET(EMBREE_HASH 0)
|
||||
ENDIF()
|
||||
ENDIF()
|
||||
|
||||
CONFIGURE_FILE(
|
||||
"${PROJECT_SOURCE_DIR}/kernels/rtcore_version.h.in"
|
||||
"${PROJECT_SOURCE_DIR}/include/embree2/rtcore_version.h"
|
||||
)
|
||||
CONFIGURE_FILE(
|
||||
"${PROJECT_SOURCE_DIR}/kernels/hash.h.in"
|
||||
"${PROJECT_BINARY_DIR}/hash.h"
|
||||
)
|
||||
FILE(WRITE "${PROJECT_BINARY_DIR}/hash" "${EMBREE_HASH}")
|
||||
|
||||
IF(COMMAND cmake_policy)
|
||||
if (POLICY CMP0003)
|
||||
cmake_policy(SET CMP0003 NEW)
|
||||
endif()
|
||||
if (POLICY CMP0042)
|
||||
cmake_policy(SET CMP0042 OLD)
|
||||
endif()
|
||||
ENDIF(COMMAND cmake_policy)
|
||||
if(COMMAND cmake_policy)
|
||||
cmake_policy(SET CMP0003 NEW)
|
||||
endif(COMMAND cmake_policy)
|
||||
|
||||
MARK_AS_ADVANCED(CMAKE_BACKWARDS_COMPATIBILITY)
|
||||
MARK_AS_ADVANCED(CMAKE_INSTALL_PREFIX)
|
||||
MARK_AS_ADVANCED(EXECUTABLE_OUTPUT_PATH)
|
||||
MARK_AS_ADVANCED(LIBRARY_OUTPUT_PATH)
|
||||
MARK_AS_ADVANCED(CLEAR CMAKE_VERBOSE_MAKEFILE)
|
||||
|
||||
MARK_AS_ADVANCED(CMAKE_OSX_ARCHITECTURES)
|
||||
MARK_AS_ADVANCED(CMAKE_OSX_DEPLOYMENT_TARGET)
|
||||
MARK_AS_ADVANCED(CMAKE_OSX_SYSROOT)
|
||||
MARK_AS_ADVANCED(CLEAR CMAKE_CXX_COMPILER)
|
||||
MARK_AS_ADVANCED(GLUT_cocoa_LIBRARY)
|
||||
|
||||
SET(CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/common/cmake ${CMAKE_MODULE_PATH})
|
||||
|
||||
INCLUDE(test)
|
||||
|
||||
##############################################################
|
||||
# Embree configuration
|
||||
##############################################################
|
||||
|
||||
GET_PROPERTY(SHARED_LIBS_SUPPORTED GLOBAL PROPERTY TARGET_SUPPORTS_SHARED_LIBS)
|
||||
IF(SHARED_LIBS_SUPPORTED)
|
||||
OPTION(EMBREE_STATIC_LIB "Build Embree as a static library.")
|
||||
MARK_AS_ADVANCED(CLEAR EMBREE_STATIC_LIB)
|
||||
ELSE()
|
||||
SET(EMBREE_STATIC_LIB ON CACHE BOOL "Build Embree as a static library." FORCE)
|
||||
MARK_AS_ADVANCED(EMBREE_STATIC_LIB)
|
||||
ENDIF()
|
||||
IF (EMBREE_STATIC_LIB)
|
||||
SET(EMBREE_LIB_TYPE STATIC)
|
||||
ADD_DEFINITIONS(-DEMBREE_STATIC_LIB)
|
||||
ELSE()
|
||||
SET(EMBREE_LIB_TYPE SHARED)
|
||||
SET(RTCORE_RAY_MASK OFF CACHE BOOL "Enables ray mask support.")
|
||||
IF (RTCORE_RAY_MASK)
|
||||
ADD_DEFINITIONS(-D__USE_RAY_MASK__)
|
||||
ENDIF()
|
||||
|
||||
OPTION(EMBREE_ISPC_SUPPORT "Build Embree with support for ISPC applications." ON)
|
||||
OPTION(EMBREE_TUTORIALS "Enable to build Embree tutorials" ON)
|
||||
|
||||
##############################################################################
|
||||
# Configurations (add configurations also to common/cmake/embree-config-default.cmake)
|
||||
|
||||
OPTION(EMBREE_STAT_COUNTERS "Enables statistic counters.")
|
||||
OPTION(EMBREE_STACK_PROTECTOR "When enabled Embree compiles with stack protection against return address overrides." OFF)
|
||||
|
||||
OPTION(EMBREE_RAY_MASK "Enables ray mask support.")
|
||||
OPTION(EMBREE_BACKFACE_CULLING "Enables backface culling.")
|
||||
OPTION(EMBREE_INTERSECTION_FILTER "Enables intersection filter callback." ON)
|
||||
OPTION(EMBREE_INTERSECTION_FILTER_RESTORE "Restores previous hit when hit is filtered out." ON)
|
||||
OPTION(EMBREE_IGNORE_INVALID_RAYS "Ignores invalid rays." OFF) # FIXME: enable by default?
|
||||
|
||||
OPTION(EMBREE_GEOMETRY_TRIANGLES "Enables support for triangle geometries." ON)
|
||||
OPTION(EMBREE_GEOMETRY_QUADS "Enables support for quad geometries." ON)
|
||||
OPTION(EMBREE_GEOMETRY_LINES "Enables support for line geometries." ON)
|
||||
OPTION(EMBREE_GEOMETRY_HAIR "Enables support for hair geometries." ON)
|
||||
OPTION(EMBREE_GEOMETRY_SUBDIV "Enables support for subdiv geometries." ON)
|
||||
OPTION(EMBREE_GEOMETRY_USER "Enables support for user geometries." ON)
|
||||
OPTION(EMBREE_RAY_PACKETS "Enabled support for ray packets." ON)
|
||||
|
||||
SET(EMBREE_NATIVE_SPLINE_BASIS BEZIER CACHE STRING "Sets the basis for curves which Embree uses internally. Other types are converted and need more memory.")
|
||||
SET_PROPERTY(CACHE EMBREE_NATIVE_SPLINE_BASIS PROPERTY STRINGS BSPLINE BEZIER)
|
||||
IF (EMBREE_NATIVE_SPLINE_BASIS STREQUAL "BSPLINE")
|
||||
SET(EMBREE_NATIVE_CURVE_BSPLINE ON)
|
||||
ELSE()
|
||||
SET(EMBREE_NATIVE_CURVE_BSPLINE OFF)
|
||||
SET(RTCORE_STAT_COUNTERS OFF CACHE BOOL "Enables statistic counters.")
|
||||
IF (RTCORE_STAT_COUNTERS)
|
||||
ADD_DEFINITIONS(-D__USE_STAT_COUNTERS__)
|
||||
ENDIF()
|
||||
|
||||
SET(EMBREE_TASKING_SYSTEM "TBB" CACHE STRING "Selects tasking system")
|
||||
IF (WIN32)
|
||||
SET_PROPERTY(CACHE EMBREE_TASKING_SYSTEM PROPERTY STRINGS TBB INTERNAL PPL)
|
||||
ELSE()
|
||||
SET_PROPERTY(CACHE EMBREE_TASKING_SYSTEM PROPERTY STRINGS TBB INTERNAL)
|
||||
SET(RTCORE_BACKFACE_CULLING OFF CACHE BOOL "Enables backface culling.")
|
||||
IF (RTCORE_BACKFACE_CULLING)
|
||||
ADD_DEFINITIONS(-D__BACKFACE_CULLING__)
|
||||
ENDIF()
|
||||
|
||||
IF (EMBREE_TASKING_SYSTEM STREQUAL "TBB")
|
||||
SET(TASKING_TBB ON )
|
||||
SET(TASKING_INTERNAL OFF)
|
||||
SET(TASKING_PPL OFF )
|
||||
ADD_DEFINITIONS(-DTASKING_TBB)
|
||||
LIST(APPEND ISPC_DEFINITIONS -DTASKING_TBB)
|
||||
ELSEIF (EMBREE_TASKING_SYSTEM STREQUAL "PPL")
|
||||
SET(TASKING_PPL ON )
|
||||
SET(TASKING_TBB OFF )
|
||||
SET(TASKING_INTERNAL OFF)
|
||||
ADD_DEFINITIONS(-DTASKING_PPL)
|
||||
LIST(APPEND ISPC_DEFINITIONS -DTASKING_PPL)
|
||||
ELSE()
|
||||
SET(TASKING_INTERNAL ON )
|
||||
SET(TASKING_TBB OFF)
|
||||
SET(TASKING_PPL OFF )
|
||||
ADD_DEFINITIONS(-DTASKING_INTERNAL)
|
||||
LIST(APPEND ISPC_DEFINITIONS -DTASKING_INTERNAL)
|
||||
IF (XEON_PHI_ISA)
|
||||
# UNSET(RTCORE_INTERSECTION_FILTER CACHE)
|
||||
SET(RTCORE_INTERSECTION_FILTER OFF CACHE BOOL "Enables intersection filter callback.")
|
||||
ELSE ()
|
||||
SET(RTCORE_INTERSECTION_FILTER ON CACHE BOOL "Enables intersection filter callback.")
|
||||
ENDIF()
|
||||
|
||||
CONFIGURE_FILE(
|
||||
"${PROJECT_SOURCE_DIR}/kernels/config.h.in"
|
||||
"${PROJECT_BINARY_DIR}/config.h"
|
||||
)
|
||||
IF (RTCORE_INTERSECTION_FILTER)
|
||||
ADD_DEFINITIONS(-D__INTERSECTION_FILTER__)
|
||||
ENDIF()
|
||||
|
||||
SET(RTCORE_BUFFER_STRIDE ON CACHE BOOL "Enables buffer strides.")
|
||||
IF (RTCORE_BUFFER_STRIDE)
|
||||
ADD_DEFINITIONS(-D__BUFFER_STRIDE__)
|
||||
ENDIF()
|
||||
|
||||
SET(RTCORE_SPINLOCKS OFF CACHE BOOL "Use spinning locks.")
|
||||
|
||||
IF (RTCORE_SPINLOCKS)
|
||||
ADD_DEFINITIONS(-D__SPINLOCKS__)
|
||||
ENDIF()
|
||||
|
||||
SET(RTCORE_TASKLOGGER OFF CACHE BOOL "Allows creating scheduling diagram of tasks.")
|
||||
IF (RTCORE_TASKLOGGER)
|
||||
ADD_DEFINITIONS(-D__LOG_TASKS__)
|
||||
ENDIF()
|
||||
|
||||
SET(RTCORE_EXPORT_ALL_SYMBOLS OFF CACHE BOOL "Lets Embree shared library export all symbols.")
|
||||
IF (RTCORE_EXPORT_ALL_SYMBOLS)
|
||||
ADD_DEFINITIONS(-D__EXPORT_ALL_SYMBOLS__)
|
||||
ENDIF()
|
||||
|
||||
SET(RTCORE_ENABLE_RAYSTREAM_LOGGER OFF CACHE BOOL "Enable ray stream logger")
|
||||
IF (RTCORE_ENABLE_RAYSTREAM_LOGGER)
|
||||
ADD_DEFINITIONS(-D__ENABLE_RAYSTREAM_LOGGER__)
|
||||
ENDIF()
|
||||
|
||||
##############################################################
|
||||
# ISA configuration
|
||||
##############################################################
|
||||
|
||||
IF(CMAKE_CXX_COMPILER_WRAPPER STREQUAL "CrayPrgEnv")
|
||||
SET(EMBREE_MAX_ISA "NONE" CACHE STRING "Selects highest ISA to support.")
|
||||
ELSE()
|
||||
SET(EMBREE_MAX_ISA "AVX2" CACHE STRING "Selects highest ISA to support.")
|
||||
ENDIF()
|
||||
SET_PROPERTY(CACHE EMBREE_MAX_ISA PROPERTY STRINGS NONE SSE2 SSE4.2 AVX AVX2 AVX512KNL AVX512SKX)
|
||||
SET(XEON_ISA "AVX2" CACHE STRING "Select highest ISA to support (SSE2, SSE3, SSSE3, SSE4.1, SSE4.2, AVX, AVX-I, AVX2")
|
||||
SET(XEON_PHI_ISA OFF CACHE BOOL "Generate code path for Xeon Phi")
|
||||
|
||||
IF (EMBREE_MAX_ISA STREQUAL "NONE")
|
||||
OPTION(EMBREE_ISA_SSE2 "Enables SSE2 ISA.")
|
||||
OPTION(EMBREE_ISA_SSE42 "Enables SSE4.2 ISA.")
|
||||
OPTION(EMBREE_ISA_AVX "Enables AVX ISA.")
|
||||
OPTION(EMBREE_ISA_AVX2 "Enables AVX2 ISA.")
|
||||
OPTION(EMBREE_ISA_AVX512KNL "Enables AVX512 ISA for Knights Landing.")
|
||||
OPTION(EMBREE_ISA_AVX512SKX "Enables AVX512 ISA for Skylake.")
|
||||
IF(NOT (EMBREE_ISA_SSE2 OR EMBREE_ISA_SSE42 OR EMBREE_ISA_AVX OR
|
||||
EMBREE_ISA_AVX2 OR EMBREE_ISA_AVX512KNL OR EMBREE_ISA_AVX512SKX))
|
||||
MESSAGE(STATUS "Detecting default ISA:")
|
||||
INCLUDE(check_isa_default)
|
||||
CHECK_ISA_DEFAULT(EMBREE_ISA_DEFAULT)
|
||||
MESSAGE(STATUS "Detecting default ISA: ${EMBREE_ISA_DEFAULT}")
|
||||
SET(EMBREE_ISA_${EMBREE_ISA_DEFAULT} ON)
|
||||
ENDIF()
|
||||
ELSE()
|
||||
UNSET(EMBREE_ISA_SSE2 CACHE)
|
||||
UNSET(EMBREE_ISA_SSE42 CACHE)
|
||||
UNSET(EMBREE_ISA_AVX CACHE)
|
||||
UNSET(EMBREE_ISA_AVX2 CACHE)
|
||||
UNSET(EMBREE_ISA_AVX512KNL CACHE)
|
||||
UNSET(EMBREE_ISA_AVX512SKX CACHE)
|
||||
IF (XEON_ISA STREQUAL "SSE2")
|
||||
SET(ISA 1)
|
||||
ENDIF ()
|
||||
|
||||
IF(EMBREE_MAX_ISA STREQUAL "SSE2")
|
||||
SET(ISA 1)
|
||||
ELSEIF(EMBREE_MAX_ISA STREQUAL "SSE4.2")
|
||||
SET(ISA 2)
|
||||
ELSEIF(EMBREE_MAX_ISA STREQUAL "AVX")
|
||||
SET(ISA 3)
|
||||
ELSEIF(EMBREE_MAX_ISA STREQUAL "AVX2")
|
||||
SET(ISA 4)
|
||||
ELSEIF(EMBREE_MAX_ISA STREQUAL "AVX512KNL")
|
||||
SET(ISA 5)
|
||||
ELSEIF(EMBREE_MAX_ISA STREQUAL "AVX512SKX")
|
||||
SET(ISA 6)
|
||||
ELSE()
|
||||
MESSAGE(FATAL_ERROR "Unsupported ISA specified: " ${EMBREE_MAX_ISA})
|
||||
ENDIF()
|
||||
IF (XEON_ISA STREQUAL "SSE3")
|
||||
SET(ISA 2)
|
||||
ENDIF ()
|
||||
|
||||
SET(EMBREE_ISA_SSE2 OFF)
|
||||
SET(EMBREE_ISA_SSE42 OFF)
|
||||
SET(EMBREE_ISA_AVX OFF)
|
||||
SET(EMBREE_ISA_AVX2 OFF)
|
||||
SET(EMBREE_ISA_AVX512KNL OFF)
|
||||
SET(EMBREE_ISA_AVX512SKX OFF)
|
||||
IF (XEON_ISA STREQUAL "SSSE3")
|
||||
SET(ISA 3)
|
||||
ENDIF ()
|
||||
|
||||
IF (ISA GREATER 0)
|
||||
SET(EMBREE_ISA_SSE2 ON)
|
||||
ENDIF ()
|
||||
IF (ISA GREATER 1)
|
||||
SET(EMBREE_ISA_SSE42 ON)
|
||||
ENDIF ()
|
||||
IF (ISA GREATER 2)
|
||||
SET(EMBREE_ISA_AVX ON)
|
||||
ENDIF ()
|
||||
IF (ISA GREATER 3)
|
||||
SET(EMBREE_ISA_AVX2 ON)
|
||||
ENDIF ()
|
||||
IF (ISA GREATER 4)
|
||||
SET(EMBREE_ISA_AVX512KNL ON)
|
||||
ENDIF ()
|
||||
IF (ISA GREATER 5)
|
||||
SET(EMBREE_ISA_AVX512SKX ON)
|
||||
ENDIF ()
|
||||
ENDIF()
|
||||
IF (XEON_ISA STREQUAL "SSE4.1")
|
||||
SET(ISA 4)
|
||||
ENDIF ()
|
||||
|
||||
IF (XEON_ISA STREQUAL "SSE4.2")
|
||||
SET(ISA 5)
|
||||
ENDIF ()
|
||||
|
||||
IF (XEON_ISA STREQUAL "AVX")
|
||||
SET(ISA 6)
|
||||
ENDIF ()
|
||||
|
||||
IF (XEON_ISA STREQUAL "AVX-I")
|
||||
SET(ISA 7)
|
||||
ENDIF ()
|
||||
|
||||
IF (XEON_ISA STREQUAL "AVX2")
|
||||
SET(ISA 8)
|
||||
ENDIF ()
|
||||
|
||||
IF (XEON_ISA STREQUAL "AVX512")
|
||||
SET(ISA 9)
|
||||
ENDIF ()
|
||||
|
||||
SET(TARGET_SSE2 OFF)
|
||||
SET(TARGET_SSE3 OFF)
|
||||
SET(TARGET_SSSE3 OFF)
|
||||
SET(TARGET_SSE41 OFF)
|
||||
SET(TARGET_SSE42 OFF)
|
||||
SET(TARGET_AVX OFF)
|
||||
SET(TARGET_AVXI OFF)
|
||||
SET(TARGET_AVX2 OFF)
|
||||
SET(TARGET_AVX512 OFF)
|
||||
|
||||
IF (ISA GREATER 0)
|
||||
SET(TARGET_SSE2 ON)
|
||||
SET(ISPC_TARGETS "sse2")
|
||||
ENDIF ()
|
||||
|
||||
IF (ISA GREATER 1)
|
||||
SET(TARGET_SSE3 ON)
|
||||
ENDIF ()
|
||||
|
||||
IF (ISA GREATER 2)
|
||||
SET(TARGET_SSSE3 ON)
|
||||
ENDIF ()
|
||||
|
||||
IF (ISA GREATER 3)
|
||||
SET(TARGET_SSE41 ON)
|
||||
ENDIF ()
|
||||
|
||||
IF (ISA GREATER 4)
|
||||
SET(TARGET_SSE42 ON)
|
||||
SET(ISPC_TARGETS ${ISPC_TARGETS} "sse4")
|
||||
ENDIF ()
|
||||
|
||||
IF (ISA GREATER 5)
|
||||
SET(TARGET_AVX ON)
|
||||
SET(ISPC_TARGETS ${ISPC_TARGETS} "avx")
|
||||
ENDIF ()
|
||||
|
||||
IF (ISA GREATER 6)
|
||||
SET(TARGET_AVXI ON)
|
||||
ENDIF ()
|
||||
|
||||
IF (ISA GREATER 7)
|
||||
SET(TARGET_AVX2 ON)
|
||||
SET(ISPC_TARGETS ${ISPC_TARGETS} "avx2")
|
||||
ENDIF ()
|
||||
|
||||
IF (ISA GREATER 8)
|
||||
SET(TARGET_AVX512 ON)
|
||||
ENDIF ()
|
||||
|
||||
SET(ISPC_TARGETS "${ISPC_TARGETS}")
|
||||
STRING(REGEX REPLACE ";" "," ISPC_TARGETS "${ISPC_TARGETS}")
|
||||
|
||||
##############################################################
|
||||
# Compiler
|
||||
##############################################################
|
||||
|
||||
SET(CONFIGURATION_TYPES "Debug;Release;RelWithDebInfo")
|
||||
SET(COMPILER "GCC" CACHE STRING "Set to GCC, CLANG, or ICC")
|
||||
|
||||
IF (WIN32)
|
||||
IF (NOT DEFAULT_CMAKE_CONFIGURATION_TYPES_SET)
|
||||
SET(CMAKE_CONFIGURATION_TYPES "${CONFIGURATION_TYPES}" CACHE STRING "List of generated configurations." FORCE)
|
||||
SET(DEFAULT_CMAKE_CONFIGURATION_TYPES_SET ON CACHE INTERNAL "Default CMake configuration types set.")
|
||||
ENDIF()
|
||||
SET_PROPERTY(GLOBAL PROPERTY USE_FOLDERS ON)
|
||||
|
||||
IF (${CMAKE_GENERATOR_TOOLSET} MATCHES "^LLVM" )
|
||||
MESSAGE("CLANG detected")
|
||||
INCLUDE(clang)
|
||||
ELSEIF(${CMAKE_GENERATOR_TOOLSET} MATCHES "^Intel")
|
||||
MESSAGE("Intel Compiler detected")
|
||||
INCLUDE (icc)
|
||||
ELSE()
|
||||
IF (EMBREE_ISA_AVX512KNL OR EMBREE_ISA_AVX512SKX)
|
||||
MESSAGE(FATAL_ERROR "Microsoft Visual C++ Compiler does not support AVX512. Please use Intel Compiler or Clang.")
|
||||
ENDIF()
|
||||
INCLUDE (msvc)
|
||||
ENDIF()
|
||||
|
||||
ELSE (WIN32)
|
||||
IF(CMAKE_CXX_COMPILER_WRAPPER STREQUAL "CrayPrgEnv")
|
||||
INCLUDE(crayprgenv)
|
||||
ELSEIF (${CMAKE_CXX_COMPILER_ID} STREQUAL "GNU")
|
||||
INCLUDE (gcc)
|
||||
ELSEIF (${CMAKE_CXX_COMPILER_ID} STREQUAL "Clang")
|
||||
INCLUDE (clang)
|
||||
ELSEIF (${CMAKE_CXX_COMPILER_ID} STREQUAL "Intel")
|
||||
INCLUDE (icc)
|
||||
ELSE ()
|
||||
MESSAGE(FATAL_ERROR "Unsupported compiler: " ${CMAKE_CXX_COMPILER_ID})
|
||||
ENDIF ()
|
||||
|
||||
IF(NOT CMAKE_BUILD_TYPE)
|
||||
SET(CMAKE_BUILD_TYPE "Release" CACHE STRING "Specifies the build type." FORCE)
|
||||
SET_PROPERTY(CACHE CMAKE_BUILD_TYPE PROPERTY STRINGS ${CONFIGURATION_TYPES})
|
||||
ENDIF(NOT CMAKE_BUILD_TYPE)
|
||||
|
||||
OPTION(CMAKE_VERBOSE_MAKEFILE "Enables verbose mode.")
|
||||
MARK_AS_ADVANCED(CLEAR CMAKE_VERBOSE_MAKEFILE)
|
||||
|
||||
ENDIF (WIN32)
|
||||
|
||||
##############################################################
|
||||
# ISA configuration continued
|
||||
##############################################################
|
||||
|
||||
SET(SSE2 0)
|
||||
SET(SSE42 1)
|
||||
SET(AVX 2)
|
||||
SET(AVX2 3)
|
||||
SET(AVX512KNL 4)
|
||||
SET(AVX512SKX 5)
|
||||
|
||||
UNSET(FLAGS_LOWEST)
|
||||
|
||||
IF (EMBREE_ISA_SSE2)
|
||||
ADD_DEFINITIONS(-DEMBREE_TARGET_SSE2)
|
||||
LIST(APPEND ISPC_TARGETS "sse2")
|
||||
IF(NOT FLAGS_LOWEST)
|
||||
SET(ISA_LOWEST ${SSE2})
|
||||
SET(FLAGS_LOWEST ${FLAGS_SSE2})
|
||||
ENDIF()
|
||||
ENDIF()
|
||||
|
||||
IF (EMBREE_ISA_SSE42)
|
||||
ADD_DEFINITIONS(-DEMBREE_TARGET_SSE42)
|
||||
LIST(APPEND ISPC_TARGETS "sse4")
|
||||
IF(NOT FLAGS_LOWEST)
|
||||
SET(ISA_LOWEST ${SSE42})
|
||||
SET(FLAGS_LOWEST ${FLAGS_SSE42})
|
||||
ENDIF()
|
||||
IF (COMPILER STREQUAL "GCC")
|
||||
INCLUDE (gcc)
|
||||
ELSEIF (COMPILER STREQUAL "CLANG")
|
||||
INCLUDE (clang)
|
||||
ELSEIF (COMPILER STREQUAL "ICC")
|
||||
INCLUDE (icc)
|
||||
ELSE ()
|
||||
MESSAGE(FATAL_ERROR "Unknown compiler specified: " ${COMPILER})
|
||||
ENDIF ()
|
||||
|
||||
IF (EMBREE_ISA_AVX)
|
||||
ADD_DEFINITIONS(-DEMBREE_TARGET_AVX)
|
||||
LIST(APPEND ISPC_TARGETS "avx")
|
||||
IF(NOT FLAGS_LOWEST)
|
||||
SET(ISA_LOWEST ${AVX})
|
||||
SET(FLAGS_LOWEST ${FLAGS_AVX})
|
||||
ENDIF()
|
||||
ENDIF ()
|
||||
IF(NOT CMAKE_BUILD_TYPE)
|
||||
SET(CMAKE_BUILD_TYPE "Release" CACHE STRING "Choose the build type: Debug Release RelWithDebInfo MinSizeRel." FORCE)
|
||||
ENDIF(NOT CMAKE_BUILD_TYPE)
|
||||
|
||||
IF (EMBREE_ISA_AVX2)
|
||||
ADD_DEFINITIONS(-DEMBREE_TARGET_AVX2)
|
||||
LIST(APPEND ISPC_TARGETS "avx2")
|
||||
IF(NOT FLAGS_LOWEST)
|
||||
SET(ISA_LOWEST ${AVX2})
|
||||
SET(FLAGS_LOWEST ${FLAGS_AVX2})
|
||||
ENDIF()
|
||||
ENDIF ()
|
||||
|
||||
IF (EMBREE_ISA_AVX512KNL)
|
||||
ADD_DEFINITIONS(-DEMBREE_TARGET_AVX512KNL)
|
||||
LIST(APPEND ISPC_TARGETS "avx512knl-i32x16")
|
||||
IF(NOT FLAGS_LOWEST)
|
||||
SET(ISA_LOWEST ${AVX512KNL})
|
||||
SET(FLAGS_LOWEST ${FLAGS_AVX512KNL})
|
||||
ENDIF()
|
||||
ENDIF ()
|
||||
|
||||
IF (EMBREE_ISA_AVX512SKX)
|
||||
ADD_DEFINITIONS(-DEMBREE_TARGET_AVX512SKX)
|
||||
LIST(APPEND ISPC_TARGETS "avx512skx-i32x16")
|
||||
IF(NOT FLAGS_LOWEST)
|
||||
SET(ISA_LOWEST ${AVX512SKX})
|
||||
SET(FLAGS_LOWEST ${FLAGS_AVX512SKX})
|
||||
ENDIF()
|
||||
ENDIF ()
|
||||
SET(CMAKE_VERBOSE_MAKEFILE false CACHE BOOL "Enables verbose mode.")
|
||||
|
||||
INCLUDE (ispc)
|
||||
|
||||
##############################################################
|
||||
# Create Binary Packages (uses above config options)
|
||||
##############################################################
|
||||
include(package)
|
||||
|
||||
##############################################################
|
||||
# Search paths
|
||||
##############################################################
|
||||
INCLUDE_DIRECTORIES(${PROJECT_SOURCE_DIR}/include)
|
||||
INCLUDE_DIRECTORIES("${PROJECT_BINARY_DIR}")
|
||||
INCLUDE_DIRECTORIES_ISPC("${PROJECT_BINARY_DIR}")
|
||||
IF (TASKING_TBB)
|
||||
FIND_PACKAGE(TBB REQUIRED)
|
||||
INCLUDE_DIRECTORIES(${TBB_INCLUDE_DIRS})
|
||||
ENDIF()
|
||||
INCLUDE_DIRECTORIES(${PROJECT_SOURCE_DIR} ${PROJECT_SOURCE_DIR}/common)
|
||||
INCLUDE_DIRECTORIES(${PROJECT_SOURCE_DIR} ${PROJECT_SOURCE_DIR}/include)
|
||||
INCLUDE_DIRECTORIES_ISPC(${PROJECT_SOURCE_DIR})
|
||||
LINK_DIRECTORIES(${EXECUTABLE_OUTPUT_PATH})
|
||||
|
||||
##############################################################
|
||||
# Output paths
|
||||
##############################################################
|
||||
SET(CMAKE_RUNTIME_OUTPUT_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}")
|
||||
SET(CMAKE_ARCHIVE_OUTPUT_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}")
|
||||
SET(CMAKE_LIBRARY_OUTPUT_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}")
|
||||
|
||||
SET(CMAKE_RUNTIME_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}")
|
||||
SET(CMAKE_ARCHIVE_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}")
|
||||
SET(CMAKE_LIBRARY_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}")
|
||||
|
||||
##############################################################
|
||||
# Directories to compile
|
||||
@@ -380,26 +233,29 @@ SET(CMAKE_LIBRARY_OUTPUT_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}")
|
||||
|
||||
ADD_SUBDIRECTORY(common)
|
||||
ADD_SUBDIRECTORY(kernels)
|
||||
ADD_SUBDIRECTORY(tutorials)
|
||||
|
||||
IF (EMBREE_TUTORIALS)
|
||||
ADD_SUBDIRECTORY(tutorials)
|
||||
ENDIF()
|
||||
SET (__XEON__ 1)
|
||||
ADD_SUBDIRECTORY(tests)
|
||||
|
||||
IF (XEON_PHI_ISA)
|
||||
SET (__XEON__ 0)
|
||||
ADD_SUBDIRECTORY(tests tests_xeonphi)
|
||||
ENDIF ()
|
||||
|
||||
##############################################################
|
||||
# Uninstall
|
||||
# Install Headers
|
||||
##############################################################
|
||||
|
||||
configure_file(
|
||||
"${CMAKE_CURRENT_SOURCE_DIR}/common/cmake/uninstall.cmake.in"
|
||||
"${CMAKE_CURRENT_BINARY_DIR}/uninstall.cmake"
|
||||
IMMEDIATE @ONLY)
|
||||
INSTALL(DIRECTORY include/embree2 DESTINATION include FILES_MATCHING PATTERN "*.h")
|
||||
INSTALL(DIRECTORY include/embree2 DESTINATION include FILES_MATCHING PATTERN "*.isph")
|
||||
|
||||
add_custom_target(uninstall
|
||||
COMMAND ${CMAKE_COMMAND} -P ${CMAKE_CURRENT_BINARY_DIR}/uninstall.cmake)
|
||||
# ##################################################################
|
||||
# Create Documentation. Call via 'make doc'
|
||||
# ##################################################################
|
||||
|
||||
SET_PROPERTY(TARGET uninstall PROPERTY FOLDER CMakePredefinedTargets)
|
||||
ADD_CUSTOM_TARGET(doc
|
||||
make -C ${CMAKE_CURRENT_SOURCE_DIR}/doc
|
||||
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/doc
|
||||
COMMENT "Generating documentation" VERBATIM)
|
||||
|
||||
##############################################################
|
||||
# Has to be last
|
||||
##############################################################
|
||||
INCLUDE(CPack)
|
||||
|
||||
-202
@@ -1,202 +0,0 @@
|
||||
|
||||
Apache License
|
||||
Version 2.0, January 2004
|
||||
http://www.apache.org/licenses/
|
||||
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
|
||||
1. Definitions.
|
||||
|
||||
"License" shall mean the terms and conditions for use, reproduction,
|
||||
and distribution as defined by Sections 1 through 9 of this document.
|
||||
|
||||
"Licensor" shall mean the copyright owner or entity authorized by
|
||||
the copyright owner that is granting the License.
|
||||
|
||||
"Legal Entity" shall mean the union of the acting entity and all
|
||||
other entities that control, are controlled by, or are under common
|
||||
control with that entity. For the purposes of this definition,
|
||||
"control" means (i) the power, direct or indirect, to cause the
|
||||
direction or management of such entity, whether by contract or
|
||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
|
||||
"You" (or "Your") shall mean an individual or Legal Entity
|
||||
exercising permissions granted by this License.
|
||||
|
||||
"Source" form shall mean the preferred form for making modifications,
|
||||
including but not limited to software source code, documentation
|
||||
source, and configuration files.
|
||||
|
||||
"Object" form shall mean any form resulting from mechanical
|
||||
transformation or translation of a Source form, including but
|
||||
not limited to compiled object code, generated documentation,
|
||||
and conversions to other media types.
|
||||
|
||||
"Work" shall mean the work of authorship, whether in Source or
|
||||
Object form, made available under the License, as indicated by a
|
||||
copyright notice that is included in or attached to the work
|
||||
(an example is provided in the Appendix below).
|
||||
|
||||
"Derivative Works" shall mean any work, whether in Source or Object
|
||||
form, that is based on (or derived from) the Work and for which the
|
||||
editorial revisions, annotations, elaborations, or other modifications
|
||||
represent, as a whole, an original work of authorship. For the purposes
|
||||
of this License, Derivative Works shall not include works that remain
|
||||
separable from, or merely link (or bind by name) to the interfaces of,
|
||||
the Work and Derivative Works thereof.
|
||||
|
||||
"Contribution" shall mean any work of authorship, including
|
||||
the original version of the Work and any modifications or additions
|
||||
to that Work or Derivative Works thereof, that is intentionally
|
||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||
or by an individual or Legal Entity authorized to submit on behalf of
|
||||
the copyright owner. For the purposes of this definition, "submitted"
|
||||
means any form of electronic, verbal, or written communication sent
|
||||
to the Licensor or its representatives, including but not limited to
|
||||
communication on electronic mailing lists, source code control systems,
|
||||
and issue tracking systems that are managed by, or on behalf of, the
|
||||
Licensor for the purpose of discussing and improving the Work, but
|
||||
excluding communication that is conspicuously marked or otherwise
|
||||
designated in writing by the copyright owner as "Not a Contribution."
|
||||
|
||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||
on behalf of whom a Contribution has been received by Licensor and
|
||||
subsequently incorporated within the Work.
|
||||
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
copyright license to reproduce, prepare Derivative Works of,
|
||||
publicly display, publicly perform, sublicense, and distribute the
|
||||
Work and such Derivative Works in Source or Object form.
|
||||
|
||||
3. Grant of Patent License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
(except as stated in this section) patent license to make, have made,
|
||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||
where such license applies only to those patent claims licensable
|
||||
by such Contributor that are necessarily infringed by their
|
||||
Contribution(s) alone or by combination of their Contribution(s)
|
||||
with the Work to which such Contribution(s) was submitted. If You
|
||||
institute patent litigation against any entity (including a
|
||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||
or a Contribution incorporated within the Work constitutes direct
|
||||
or contributory patent infringement, then any patent licenses
|
||||
granted to You under this License for that Work shall terminate
|
||||
as of the date such litigation is filed.
|
||||
|
||||
4. Redistribution. You may reproduce and distribute copies of the
|
||||
Work or Derivative Works thereof in any medium, with or without
|
||||
modifications, and in Source or Object form, provided that You
|
||||
meet the following conditions:
|
||||
|
||||
(a) You must give any other recipients of the Work or
|
||||
Derivative Works a copy of this License; and
|
||||
|
||||
(b) You must cause any modified files to carry prominent notices
|
||||
stating that You changed the files; and
|
||||
|
||||
(c) You must retain, in the Source form of any Derivative Works
|
||||
that You distribute, all copyright, patent, trademark, and
|
||||
attribution notices from the Source form of the Work,
|
||||
excluding those notices that do not pertain to any part of
|
||||
the Derivative Works; and
|
||||
|
||||
(d) If the Work includes a "NOTICE" text file as part of its
|
||||
distribution, then any Derivative Works that You distribute must
|
||||
include a readable copy of the attribution notices contained
|
||||
within such NOTICE file, excluding those notices that do not
|
||||
pertain to any part of the Derivative Works, in at least one
|
||||
of the following places: within a NOTICE text file distributed
|
||||
as part of the Derivative Works; within the Source form or
|
||||
documentation, if provided along with the Derivative Works; or,
|
||||
within a display generated by the Derivative Works, if and
|
||||
wherever such third-party notices normally appear. The contents
|
||||
of the NOTICE file are for informational purposes only and
|
||||
do not modify the License. You may add Your own attribution
|
||||
notices within Derivative Works that You distribute, alongside
|
||||
or as an addendum to the NOTICE text from the Work, provided
|
||||
that such additional attribution notices cannot be construed
|
||||
as modifying the License.
|
||||
|
||||
You may add Your own copyright statement to Your modifications and
|
||||
may provide additional or different license terms and conditions
|
||||
for use, reproduction, or distribution of Your modifications, or
|
||||
for any such Derivative Works as a whole, provided Your use,
|
||||
reproduction, and distribution of the Work otherwise complies with
|
||||
the conditions stated in this License.
|
||||
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||
any Contribution intentionally submitted for inclusion in the Work
|
||||
by You to the Licensor shall be under the terms and conditions of
|
||||
this License, without any additional terms or conditions.
|
||||
Notwithstanding the above, nothing herein shall supersede or modify
|
||||
the terms of any separate license agreement you may have executed
|
||||
with Licensor regarding such Contributions.
|
||||
|
||||
6. Trademarks. This License does not grant permission to use the trade
|
||||
names, trademarks, service marks, or product names of the Licensor,
|
||||
except as required for reasonable and customary use in describing the
|
||||
origin of the Work and reproducing the content of the NOTICE file.
|
||||
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||
agreed to in writing, Licensor provides the Work (and each
|
||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
implied, including, without limitation, any warranties or conditions
|
||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||
appropriateness of using or redistributing the Work and assume any
|
||||
risks associated with Your exercise of permissions under this License.
|
||||
|
||||
8. Limitation of Liability. In no event and under no legal theory,
|
||||
whether in tort (including negligence), contract, or otherwise,
|
||||
unless required by applicable law (such as deliberate and grossly
|
||||
negligent acts) or agreed to in writing, shall any Contributor be
|
||||
liable to You for damages, including any direct, indirect, special,
|
||||
incidental, or consequential damages of any character arising as a
|
||||
result of this License or out of the use or inability to use the
|
||||
Work (including but not limited to damages for loss of goodwill,
|
||||
work stoppage, computer failure or malfunction, or any and all
|
||||
other commercial damages or losses), even if such Contributor
|
||||
has been advised of the possibility of such damages.
|
||||
|
||||
9. Accepting Warranty or Additional Liability. While redistributing
|
||||
the Work or Derivative Works thereof, You may choose to offer,
|
||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||
or other liability obligations and/or rights consistent with this
|
||||
License. However, in accepting such obligations, You may act only
|
||||
on Your own behalf and on Your sole responsibility, not on behalf
|
||||
of any other Contributor, and only if You agree to indemnify,
|
||||
defend, and hold each Contributor harmless for any liability
|
||||
incurred by, or claims asserted against, such Contributor by reason
|
||||
of your accepting any such warranty or additional liability.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
APPENDIX: How to apply the Apache License to your work.
|
||||
|
||||
To apply the Apache License to your work, attach the following
|
||||
boilerplate notice, with the fields enclosed by brackets "[]"
|
||||
replaced with your own identifying information. (Don't include
|
||||
the brackets!) The text should be enclosed in the appropriate
|
||||
comment syntax for the file format. We also recommend that a
|
||||
file or class name and description of purpose be included on the
|
||||
same "printed page" as the copyright notice for easier
|
||||
identification within third-party archives.
|
||||
|
||||
Copyright [yyyy] [name of copyright owner]
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
@@ -1,5 +1,5 @@
|
||||
## ======================================================================== ##
|
||||
## Copyright 2009-2017 Intel Corporation ##
|
||||
## Copyright 2009-2014 Intel Corporation ##
|
||||
## ##
|
||||
## Licensed under the Apache License, Version 2.0 (the "License"); ##
|
||||
## you may not use this file except in compliance with the License. ##
|
||||
@@ -14,9 +14,12 @@
|
||||
## limitations under the License. ##
|
||||
## ======================================================================== ##
|
||||
|
||||
SET(__XEON__ 1)
|
||||
ADD_SUBDIRECTORY(sys)
|
||||
ADD_SUBDIRECTORY(math)
|
||||
ADD_SUBDIRECTORY(simd)
|
||||
ADD_SUBDIRECTORY(lexers)
|
||||
ADD_SUBDIRECTORY(tasking)
|
||||
ADD_SUBDIRECTORY(algorithms)
|
||||
|
||||
IF (XEON_PHI_ISA)
|
||||
SET(__XEON__ 0)
|
||||
ADD_SUBDIRECTORY(sys sys_xeonphi)
|
||||
ADD_SUBDIRECTORY(simd simd_xeonphi)
|
||||
ENDIF ()
|
||||
|
||||
@@ -1,69 +0,0 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#include "parallel_filter.h"
|
||||
#include "../sys/regression.h"
|
||||
#include <map>
|
||||
|
||||
namespace embree
|
||||
{
|
||||
struct parallel_filter_regression_test : public RegressionTest
|
||||
{
|
||||
parallel_filter_regression_test(const char* name) : RegressionTest(name) {
|
||||
registerRegressionTest(this);
|
||||
}
|
||||
|
||||
bool run ()
|
||||
{
|
||||
bool passed = true;
|
||||
auto pred = [&]( uint32_t v ) { return (v & 0x3) == 0; };
|
||||
|
||||
for (size_t N=10; N<1000000; N=size_t(2.1*N))
|
||||
{
|
||||
size_t N0 = rand() % N;
|
||||
|
||||
/* initialize array with random numbers */
|
||||
std::vector<uint32_t> src(N);
|
||||
std::map<uint32_t,int> m;
|
||||
for (size_t i=0; i<N; i++) src[i] = rand();
|
||||
|
||||
/* count elements up */
|
||||
for (size_t i=N0; i<N; i++)
|
||||
if (pred(src[i]))
|
||||
m[src[i]] = 0;
|
||||
for (size_t i=N0; i<N; i++)
|
||||
if (pred(src[i]))
|
||||
m[src[i]]++;
|
||||
|
||||
/* filter array */
|
||||
//size_t M = sequential_filter(src.data(),N0,N,pred);
|
||||
size_t M = parallel_filter(src.data(),N0,N,size_t(1024),pred);
|
||||
|
||||
/* check if filtered data is correct */
|
||||
for (size_t i=N0; i<M; i++) {
|
||||
passed &= pred(src[i]);
|
||||
m[src[i]]--;
|
||||
}
|
||||
for (size_t i=N0; i<M; i++)
|
||||
passed &= (m[src[i]] == 0);
|
||||
}
|
||||
|
||||
return passed;
|
||||
}
|
||||
};
|
||||
|
||||
parallel_filter_regression_test parallel_filter_regression("parallel_filter_regression");
|
||||
}
|
||||
@@ -1,106 +0,0 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#pragma once
|
||||
|
||||
#include "parallel_for.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
template<typename Ty, typename Index, typename Predicate>
|
||||
inline Index sequential_filter( Ty* data, const Index first, const Index last, const Predicate& predicate)
|
||||
{
|
||||
Index j = first;
|
||||
for (Index i=first; i<last; i++)
|
||||
if (predicate(data[i]))
|
||||
data[j++] = data[i];
|
||||
|
||||
return j;
|
||||
}
|
||||
|
||||
template<typename Ty, typename Index, typename Predicate>
|
||||
inline Index parallel_filter( Ty* data, const Index begin, const Index end, const Index minStepSize, const Predicate& predicate)
|
||||
{
|
||||
/* sequential fallback */
|
||||
if (end-begin <= minStepSize)
|
||||
return sequential_filter(data,begin,end,predicate);
|
||||
|
||||
/* calculate number of tasks to use */
|
||||
enum { MAX_TASKS = 512 };
|
||||
const Index numThreads = TaskScheduler::threadCount();
|
||||
const Index numBlocks = (end-begin+minStepSize-1)/minStepSize;
|
||||
const Index taskCount = min(numThreads,numBlocks,(Index)MAX_TASKS);
|
||||
|
||||
/* filter blocks */
|
||||
Index nused[MAX_TASKS];
|
||||
Index nfree[MAX_TASKS];
|
||||
parallel_for(taskCount, [&](const Index taskIndex)
|
||||
{
|
||||
const Index i0 = begin+(taskIndex+0)*(end-begin)/taskCount;
|
||||
const Index i1 = begin+(taskIndex+1)*(end-begin)/taskCount;
|
||||
const Index i2 = sequential_filter(data,i0,i1,predicate);
|
||||
nused[taskIndex] = i2-i0;
|
||||
nfree[taskIndex] = i1-i2;
|
||||
});
|
||||
|
||||
/* calculate offsets */
|
||||
Index sused=0;
|
||||
Index sfree=0;
|
||||
Index pfree[MAX_TASKS];
|
||||
for (Index i=0; i<taskCount; i++)
|
||||
{
|
||||
sused+=nused[i];
|
||||
Index cfree = nfree[i]; pfree[i] = sfree; sfree+=cfree;
|
||||
}
|
||||
|
||||
/* return if we did not filter out any element */
|
||||
assert(sfree <= end-begin);
|
||||
assert(sused <= end-begin);
|
||||
if (sused == end-begin)
|
||||
return end;
|
||||
|
||||
/* otherwise we have to copy misplaced elements around */
|
||||
parallel_for(taskCount, [&](const Index taskIndex)
|
||||
{
|
||||
/* destination to write elements to */
|
||||
Index dst = begin+(taskIndex+0)*(end-begin)/taskCount+nused[taskIndex];
|
||||
Index dst_end = min(dst+nfree[taskIndex],begin+sused);
|
||||
if (dst_end <= dst) return;
|
||||
|
||||
/* range of misplaced elements to copy to destination */
|
||||
Index r0 = pfree[taskIndex];
|
||||
Index r1 = r0+dst_end-dst;
|
||||
|
||||
/* find range in misplaced elements in back to front order */
|
||||
Index k0=0;
|
||||
for (Index i=taskCount-1; i>0; i--)
|
||||
{
|
||||
if (k0 > r1) break;
|
||||
Index k1 = k0+nused[i];
|
||||
Index src = begin+(i+0)*(end-begin)/taskCount+nused[i];
|
||||
for (Index i=max(r0,k0); i<min(r1,k1); i++) {
|
||||
Index isrc = src-i+k0-1;
|
||||
assert(dst >= begin && dst < end);
|
||||
assert(isrc >= begin && isrc < end);
|
||||
data[dst++] = data[isrc];
|
||||
}
|
||||
k0 = k1;
|
||||
}
|
||||
});
|
||||
|
||||
return begin+sused;
|
||||
}
|
||||
}
|
||||
@@ -1,61 +0,0 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#include "parallel_for.h"
|
||||
#include "../sys/regression.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
struct parallel_for_regression_test : public RegressionTest
|
||||
{
|
||||
parallel_for_regression_test(const char* name) : RegressionTest(name) {
|
||||
registerRegressionTest(this);
|
||||
}
|
||||
|
||||
bool run ()
|
||||
{
|
||||
bool passed = true;
|
||||
|
||||
const size_t M = 10;
|
||||
for (size_t N=10; N<10000000; N=size_t(2.1*N))
|
||||
{
|
||||
/* sequentially calculate sum of squares */
|
||||
size_t sum0 = 0;
|
||||
for (size_t i=0; i<N; i++) {
|
||||
sum0 += i*i;
|
||||
}
|
||||
|
||||
/* parallel calculation of sum of squares */
|
||||
for (size_t m=0; m<M; m++)
|
||||
{
|
||||
std::atomic<size_t> sum1(0);
|
||||
parallel_for( size_t(0), size_t(N), size_t(1024), [&](const range<size_t>& r)
|
||||
{
|
||||
size_t s = 0;
|
||||
for (size_t i=r.begin(); i<r.end(); i++)
|
||||
s += i*i;
|
||||
sum1 += s;
|
||||
});
|
||||
passed = sum0 == sum1;
|
||||
}
|
||||
}
|
||||
|
||||
return passed;
|
||||
}
|
||||
};
|
||||
|
||||
parallel_for_regression_test parallel_for_regression("parallel_for_regression_test");
|
||||
}
|
||||
@@ -1,133 +0,0 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#pragma once
|
||||
|
||||
#include "../common/tasking/taskscheduler.h"
|
||||
#include "../sys/array.h"
|
||||
#include "../math/math.h"
|
||||
#include "../common/math/range.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
/* parallel_for without range */
|
||||
template<typename Index, typename Func>
|
||||
__forceinline void parallel_for( const Index N, const Func& func)
|
||||
{
|
||||
#if defined(TASKING_INTERNAL)
|
||||
if (N) {
|
||||
TaskScheduler::spawn(Index(0),N,Index(1),[&] (const range<Index>& r) {
|
||||
assert(r.size() == 1);
|
||||
func(r.begin());
|
||||
});
|
||||
if (!TaskScheduler::wait())
|
||||
throw std::runtime_error("task cancelled");
|
||||
}
|
||||
|
||||
#elif defined(TASKING_TBB)
|
||||
tbb::parallel_for(Index(0),N,Index(1),[&](Index i) {
|
||||
func(i);
|
||||
});
|
||||
if (tbb::task::self().is_cancelled())
|
||||
throw std::runtime_error("task cancelled");
|
||||
|
||||
#elif defined(TASKING_PPL)
|
||||
concurrency::parallel_for(Index(0),N,Index(1),[&](Index i) {
|
||||
func(i);
|
||||
});
|
||||
#else
|
||||
# error "no tasking system enabled"
|
||||
#endif
|
||||
}
|
||||
|
||||
/* parallel for with range and granulatity */
|
||||
template<typename Index, typename Func>
|
||||
__forceinline void parallel_for( const Index first, const Index last, const Index minStepSize, const Func& func)
|
||||
{
|
||||
assert(first <= last);
|
||||
#if defined(TASKING_INTERNAL)
|
||||
TaskScheduler::spawn(first,last,minStepSize,func);
|
||||
if (!TaskScheduler::wait())
|
||||
throw std::runtime_error("task cancelled");
|
||||
|
||||
#elif defined(TASKING_TBB)
|
||||
tbb::parallel_for(tbb::blocked_range<Index>(first,last,minStepSize),[&](const tbb::blocked_range<Index>& r) {
|
||||
func(range<Index>(r.begin(),r.end()));
|
||||
});
|
||||
if (tbb::task::self().is_cancelled())
|
||||
throw std::runtime_error("task cancelled");
|
||||
|
||||
#elif defined(TASKING_PPL)
|
||||
concurrency::parallel_for(first, last, Index(1) /*minStepSize*/, [&](Index i) {
|
||||
func(range<Index>(i,i+1));
|
||||
});
|
||||
|
||||
#else
|
||||
# error "no tasking system enabled"
|
||||
#endif
|
||||
}
|
||||
|
||||
/* parallel for with range */
|
||||
template<typename Index, typename Func>
|
||||
__forceinline void parallel_for( const Index first, const Index last, const Func& func)
|
||||
{
|
||||
assert(first <= last);
|
||||
parallel_for(first,last,(Index)1,func);
|
||||
}
|
||||
|
||||
#if defined(TASKING_TBB) && (TBB_INTERFACE_VERSION > 4001)
|
||||
|
||||
template<typename Index, typename Func>
|
||||
__forceinline void parallel_for_static( const Index N, const Func& func)
|
||||
{
|
||||
tbb::parallel_for(Index(0),N,Index(1),[&](Index i) {
|
||||
func(i);
|
||||
},tbb::simple_partitioner());
|
||||
if (tbb::task::self().is_cancelled())
|
||||
throw std::runtime_error("task cancelled");
|
||||
}
|
||||
|
||||
typedef tbb::affinity_partitioner affinity_partitioner;
|
||||
|
||||
template<typename Index, typename Func>
|
||||
__forceinline void parallel_for_affinity( const Index N, const Func& func, tbb::affinity_partitioner& ap)
|
||||
{
|
||||
tbb::parallel_for(Index(0),N,Index(1),[&](Index i) {
|
||||
func(i);
|
||||
},ap);
|
||||
if (tbb::task::self().is_cancelled())
|
||||
throw std::runtime_error("task cancelled");
|
||||
}
|
||||
|
||||
#else
|
||||
|
||||
template<typename Index, typename Func>
|
||||
__forceinline void parallel_for_static( const Index N, const Func& func)
|
||||
{
|
||||
parallel_for(N,func);
|
||||
}
|
||||
|
||||
struct affinity_partitioner {
|
||||
};
|
||||
|
||||
template<typename Index, typename Func>
|
||||
__forceinline void parallel_for_affinity( const Index N, const Func& func, affinity_partitioner& ap)
|
||||
{
|
||||
parallel_for(N,func);
|
||||
}
|
||||
|
||||
#endif
|
||||
}
|
||||
@@ -1,76 +0,0 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#include "parallel_for_for.h"
|
||||
#include "../sys/regression.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
struct parallel_for_for_regression_test : public RegressionTest
|
||||
{
|
||||
parallel_for_for_regression_test(const char* name) : RegressionTest(name) {
|
||||
registerRegressionTest(this);
|
||||
}
|
||||
|
||||
bool run ()
|
||||
{
|
||||
bool passed = true;
|
||||
|
||||
/* create vector with random numbers */
|
||||
size_t sum0 = 0;
|
||||
size_t K = 0;
|
||||
const size_t M = 1000;
|
||||
std::vector<std::vector<size_t>* > array2(M);
|
||||
for (size_t i=0; i<M; i++) {
|
||||
const size_t N = rand() % 1024;
|
||||
K+=N;
|
||||
array2[i] = new std::vector<size_t>(N);
|
||||
for (size_t j=0; j<N; j++)
|
||||
sum0 += (*array2[i])[j] = rand();
|
||||
}
|
||||
|
||||
/* array to test global index */
|
||||
std::vector<atomic<size_t>> verify_k(K);
|
||||
for (size_t i=0; i<K; i++) verify_k[i].store(0);
|
||||
|
||||
/* add all numbers using parallel_for_for */
|
||||
std::atomic<size_t> sum1(0);
|
||||
parallel_for_for( array2, size_t(1), [&](std::vector<size_t>* v, const range<size_t>& r, size_t k) -> size_t
|
||||
{
|
||||
size_t s = 0;
|
||||
for (size_t i=r.begin(); i<r.end(); i++) {
|
||||
s += (*v)[i];
|
||||
verify_k[k++]++;
|
||||
}
|
||||
sum1 += s;
|
||||
return sum1;
|
||||
});
|
||||
passed &= (sum0 == sum1);
|
||||
|
||||
/* check global index */
|
||||
for (size_t i=0; i<K; i++)
|
||||
passed &= (verify_k[i] == 1);
|
||||
|
||||
/* delete vectors again */
|
||||
for (size_t i=0; i<array2.size(); i++)
|
||||
delete array2[i];
|
||||
|
||||
return passed;
|
||||
}
|
||||
};
|
||||
|
||||
parallel_for_for_regression_test parallel_for_for_regression("parallel_for_for_regression_test");
|
||||
}
|
||||
@@ -1,162 +0,0 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#pragma once
|
||||
|
||||
#include "parallel_for.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
template<typename ArrayArray, typename Func>
|
||||
__forceinline void sequential_for_for( ArrayArray& array2, const size_t minStepSize, const Func& func )
|
||||
{
|
||||
size_t k=0;
|
||||
for (size_t i=0; i!=array2.size(); ++i) {
|
||||
const size_t N = array2[i]->size();
|
||||
if (N) func(array2[i],range<size_t>(0,N),k);
|
||||
k+=N;
|
||||
}
|
||||
}
|
||||
|
||||
class ParallelForForState
|
||||
{
|
||||
public:
|
||||
|
||||
enum { MAX_TASKS = 512 };
|
||||
|
||||
__forceinline ParallelForForState ()
|
||||
: taskCount(0) {}
|
||||
|
||||
template<typename ArrayArray>
|
||||
__forceinline ParallelForForState (ArrayArray& array2, const size_t minStepSize) {
|
||||
init(array2,minStepSize);
|
||||
}
|
||||
|
||||
template<typename ArrayArray>
|
||||
__forceinline void init ( ArrayArray& array2, const size_t minStepSize )
|
||||
{
|
||||
/* first calculate total number of elements */
|
||||
size_t N = 0;
|
||||
for (size_t i=0; i<array2.size(); i++) {
|
||||
N += array2[i] ? array2[i]->size() : 0;
|
||||
}
|
||||
this->N = N;
|
||||
|
||||
/* calculate number of tasks to use */
|
||||
const size_t numThreads = TaskScheduler::threadCount();
|
||||
const size_t numBlocks = (N+minStepSize-1)/minStepSize;
|
||||
taskCount = max(size_t(1),min(numThreads,numBlocks,size_t(ParallelForForState::MAX_TASKS)));
|
||||
|
||||
/* calculate start (i,j) for each task */
|
||||
size_t taskIndex = 0;
|
||||
i0[taskIndex] = 0;
|
||||
j0[taskIndex] = 0;
|
||||
size_t k0 = (++taskIndex)*N/taskCount;
|
||||
for (size_t i=0, k=0; taskIndex < taskCount; i++)
|
||||
{
|
||||
assert(i<array2.size());
|
||||
size_t j=0, M = array2[i] ? array2[i]->size() : 0;
|
||||
while (j<M && k+M-j >= k0 && taskIndex < taskCount) {
|
||||
assert(taskIndex<taskCount);
|
||||
i0[taskIndex] = i;
|
||||
j0[taskIndex] = j += k0-k;
|
||||
k=k0;
|
||||
k0 = (++taskIndex)*N/taskCount;
|
||||
}
|
||||
k+=M-j;
|
||||
}
|
||||
}
|
||||
|
||||
__forceinline size_t size() const {
|
||||
return N;
|
||||
}
|
||||
|
||||
public:
|
||||
size_t i0[MAX_TASKS];
|
||||
size_t j0[MAX_TASKS];
|
||||
size_t taskCount;
|
||||
size_t N;
|
||||
};
|
||||
|
||||
template<typename ArrayArray, typename Func>
|
||||
__forceinline void parallel_for_for( ArrayArray& array2, const size_t minStepSize, const Func& func )
|
||||
{
|
||||
ParallelForForState state(array2,minStepSize);
|
||||
|
||||
parallel_for(state.taskCount, [&](const size_t taskIndex)
|
||||
{
|
||||
/* calculate range */
|
||||
const size_t k0 = (taskIndex+0)*state.size()/state.taskCount;
|
||||
const size_t k1 = (taskIndex+1)*state.size()/state.taskCount;
|
||||
size_t i0 = state.i0[taskIndex];
|
||||
size_t j0 = state.j0[taskIndex];
|
||||
|
||||
/* iterate over arrays */
|
||||
size_t k=k0;
|
||||
for (size_t i=i0; k<k1; i++) {
|
||||
const size_t N = array2[i] ? array2[i]->size() : 0;
|
||||
const size_t r0 = j0, r1 = min(N,r0+k1-k);
|
||||
if (r1 > r0) func(array2[i],range<size_t>(r0,r1),k);
|
||||
k+=r1-r0; j0 = 0;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
template<typename ArrayArray, typename Func>
|
||||
__forceinline void parallel_for_for( ArrayArray& array2, const Func& func )
|
||||
{
|
||||
parallel_for_for(array2,1,func);
|
||||
}
|
||||
|
||||
template<typename ArrayArray, typename Value, typename Func, typename Reduction>
|
||||
__forceinline Value parallel_for_for_reduce( ArrayArray& array2, const size_t minStepSize, const Value& identity, const Func& func, const Reduction& reduction )
|
||||
{
|
||||
ParallelForForState state(array2,minStepSize);
|
||||
Value temp[ParallelForForState::MAX_TASKS];
|
||||
|
||||
for (size_t i=0; i<state.taskCount; i++)
|
||||
temp[i] = identity;
|
||||
|
||||
parallel_for(state.taskCount, [&](const size_t taskIndex)
|
||||
{
|
||||
/* calculate range */
|
||||
const size_t k0 = (taskIndex+0)*state.size()/state.taskCount;
|
||||
const size_t k1 = (taskIndex+1)*state.size()/state.taskCount;
|
||||
size_t i0 = state.i0[taskIndex];
|
||||
size_t j0 = state.j0[taskIndex];
|
||||
|
||||
/* iterate over arrays */
|
||||
size_t k=k0;
|
||||
for (size_t i=i0; k<k1; i++) {
|
||||
const size_t N = array2[i] ? array2[i]->size() : 0;
|
||||
const size_t r0 = j0, r1 = min(N,r0+k1-k);
|
||||
if (r1 > r0) temp[taskIndex] = reduction(temp[taskIndex],func(array2[i],range<size_t>(r0,r1),k));
|
||||
k+=r1-r0; j0 = 0;
|
||||
}
|
||||
});
|
||||
|
||||
Value ret = identity;
|
||||
for (size_t i=0; i<state.taskCount; i++)
|
||||
ret = reduction(ret,temp[i]);
|
||||
return ret;
|
||||
}
|
||||
|
||||
template<typename ArrayArray, typename Value, typename Func, typename Reduction>
|
||||
__forceinline Value parallel_for_for_reduce( ArrayArray& array2, const Value& identity, const Func& func, const Reduction& reduction)
|
||||
{
|
||||
return parallel_for_for_reduce(array2,1,identity,func,reduction);
|
||||
}
|
||||
}
|
||||
@@ -1,98 +0,0 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#include "parallel_for_for_prefix_sum.h"
|
||||
#include "../sys/regression.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
struct parallel_for_for_prefix_sum_regression_test : public RegressionTest
|
||||
{
|
||||
parallel_for_for_prefix_sum_regression_test(const char* name) : RegressionTest(name) {
|
||||
registerRegressionTest(this);
|
||||
}
|
||||
|
||||
bool run ()
|
||||
{
|
||||
bool passed = true;
|
||||
|
||||
/* create vector with random numbers */
|
||||
const size_t M = 10;
|
||||
std::vector<atomic<size_t>> flattened;
|
||||
typedef std::vector<std::vector<size_t>* > ArrayArray;
|
||||
ArrayArray array2(M);
|
||||
size_t K = 0;
|
||||
for (size_t i=0; i<M; i++) {
|
||||
const size_t N = rand() % 10;
|
||||
K += N;
|
||||
array2[i] = new std::vector<size_t>(N);
|
||||
for (size_t j=0; j<N; j++)
|
||||
(*array2[i])[j] = rand() % 10;
|
||||
}
|
||||
|
||||
/* array to test global index */
|
||||
std::vector<atomic<size_t>> verify_k(K);
|
||||
for (size_t i=0; i<K; i++) verify_k[i].store(0);
|
||||
|
||||
ParallelForForPrefixSumState<size_t> state(array2,size_t(1));
|
||||
|
||||
/* dry run only counts */
|
||||
size_t S = parallel_for_for_prefix_sum0( state, array2, size_t(0), [&](std::vector<size_t>* v, const range<size_t>& r, size_t k) -> size_t
|
||||
{
|
||||
size_t s = 0;
|
||||
for (size_t i=r.begin(); i<r.end(); i++) {
|
||||
s += (*v)[i];
|
||||
verify_k[k++]++;
|
||||
}
|
||||
return s;
|
||||
}, [](size_t v0, size_t v1) { return v0+v1; });
|
||||
|
||||
/* create properly sized output array */
|
||||
flattened.resize(S);
|
||||
for (auto& a : flattened) a.store(0);
|
||||
|
||||
/* now we actually fill the flattened array */
|
||||
parallel_for_for_prefix_sum1( state, array2, size_t(0), [&](std::vector<size_t>* v, const range<size_t>& r, size_t k, const size_t base) -> size_t
|
||||
{
|
||||
size_t s = 0;
|
||||
for (size_t i=r.begin(); i<r.end(); i++) {
|
||||
for (size_t j=0; j<(*v)[i]; j++) {
|
||||
flattened[base+s+j]++;
|
||||
}
|
||||
s += (*v)[i];
|
||||
verify_k[k++]++;
|
||||
}
|
||||
return s;
|
||||
}, [](size_t v0, size_t v1) { return v0+v1; });
|
||||
|
||||
/* check global index */
|
||||
for (size_t i=0; i<K; i++)
|
||||
passed &= (verify_k[i] == 2);
|
||||
|
||||
/* check if each element was assigned exactly once */
|
||||
for (size_t i=0; i<flattened.size(); i++)
|
||||
passed &= (flattened[i] == 1);
|
||||
|
||||
/* delete arrays again */
|
||||
for (size_t i=0; i<array2.size(); i++)
|
||||
delete array2[i];
|
||||
|
||||
return passed;
|
||||
}
|
||||
};
|
||||
|
||||
parallel_for_for_prefix_sum_regression_test parallel_for_for_prefix_sum_regression("parallel_for_for_prefix_sum_regression_test");
|
||||
}
|
||||
@@ -1,125 +0,0 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#pragma once
|
||||
|
||||
#include "parallel_for_for.h"
|
||||
#include "parallel_prefix_sum.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
template<typename Value>
|
||||
struct ParallelForForPrefixSumState : public ParallelForForState
|
||||
{
|
||||
__forceinline ParallelForForPrefixSumState () {}
|
||||
|
||||
template<typename ArrayArray>
|
||||
__forceinline ParallelForForPrefixSumState (ArrayArray& array2, const size_t minStepSize)
|
||||
: ParallelForForState(array2,minStepSize) {}
|
||||
|
||||
ParallelPrefixSumState<Value> prefix_state;
|
||||
};
|
||||
|
||||
template<typename ArrayArray, typename Index, typename Value, typename Func, typename Reduction>
|
||||
__forceinline Value parallel_for_for_prefix_sum0( ParallelForForPrefixSumState<Value>& state, ArrayArray& array2, Index minStepSize,
|
||||
const Value& identity, const Func& func, const Reduction& reduction)
|
||||
{
|
||||
/* calculate number of tasks to use */
|
||||
const size_t taskCount = state.taskCount;
|
||||
/* perform parallel prefix sum */
|
||||
parallel_for(taskCount, [&](const size_t taskIndex)
|
||||
{
|
||||
const size_t k0 = (taskIndex+0)*state.size()/taskCount;
|
||||
const size_t k1 = (taskIndex+1)*state.size()/taskCount;
|
||||
size_t i0 = state.i0[taskIndex];
|
||||
size_t j0 = state.j0[taskIndex];
|
||||
|
||||
/* iterate over arrays */
|
||||
size_t k=k0;
|
||||
Value N=identity;
|
||||
for (size_t i=i0; k<k1; i++) {
|
||||
const size_t size = array2[i] ? array2[i]->size() : 0;
|
||||
const size_t r0 = j0, r1 = min(size,r0+k1-k);
|
||||
if (r1 > r0) N = reduction(N, func(array2[i],range<size_t>(r0,r1),k));
|
||||
k+=r1-r0; j0 = 0;
|
||||
}
|
||||
state.prefix_state.counts[taskIndex] = N;
|
||||
});
|
||||
|
||||
/* calculate prefix sum */
|
||||
Value sum=identity;
|
||||
for (size_t i=0; i<taskCount; i++)
|
||||
{
|
||||
const Value c = state.prefix_state.counts[i];
|
||||
state.prefix_state.sums[i] = sum;
|
||||
sum=reduction(sum,c);
|
||||
}
|
||||
|
||||
return sum;
|
||||
}
|
||||
|
||||
template<typename ArrayArray, typename Index, typename Value, typename Func, typename Reduction>
|
||||
__forceinline Value parallel_for_for_prefix_sum1( ParallelForForPrefixSumState<Value>& state, ArrayArray& array2, Index minStepSize,
|
||||
const Value& identity, const Func& func, const Reduction& reduction)
|
||||
{
|
||||
/* calculate number of tasks to use */
|
||||
const size_t taskCount = state.taskCount;
|
||||
/* perform parallel prefix sum */
|
||||
parallel_for(taskCount, [&](const size_t taskIndex)
|
||||
{
|
||||
const size_t k0 = (taskIndex+0)*state.size()/taskCount;
|
||||
const size_t k1 = (taskIndex+1)*state.size()/taskCount;
|
||||
size_t i0 = state.i0[taskIndex];
|
||||
size_t j0 = state.j0[taskIndex];
|
||||
|
||||
/* iterate over arrays */
|
||||
size_t k=k0;
|
||||
Value N=identity;
|
||||
for (size_t i=i0; k<k1; i++) {
|
||||
const size_t size = array2[i] ? array2[i]->size() : 0;
|
||||
const size_t r0 = j0, r1 = min(size,r0+k1-k);
|
||||
if (r1 > r0) N = reduction(N, func(array2[i],range<size_t>(r0,r1),k,reduction(state.prefix_state.sums[taskIndex],N)));
|
||||
k+=r1-r0; j0 = 0;
|
||||
}
|
||||
state.prefix_state.counts[taskIndex] = N;
|
||||
});
|
||||
|
||||
/* calculate prefix sum */
|
||||
Value sum=identity;
|
||||
for (size_t i=0; i<taskCount; i++)
|
||||
{
|
||||
const Value c = state.prefix_state.counts[i];
|
||||
state.prefix_state.sums[i] = sum;
|
||||
sum=reduction(sum,c);
|
||||
}
|
||||
|
||||
return sum;
|
||||
}
|
||||
|
||||
template<typename ArrayArray, typename Value, typename Func, typename Reduction>
|
||||
__forceinline Value parallel_for_for_prefix_sum0( ParallelForForPrefixSumState<Value>& state, ArrayArray& array2,
|
||||
const Value& identity, const Func& func, const Reduction& reduction)
|
||||
{
|
||||
return parallel_for_for_prefix_sum0(state,array2,size_t(1),identity,func,reduction);
|
||||
}
|
||||
|
||||
template<typename ArrayArray, typename Value, typename Func, typename Reduction>
|
||||
__forceinline Value parallel_for_for_prefix_sum1( ParallelForForPrefixSumState<Value>& state, ArrayArray& array2,
|
||||
const Value& identity, const Func& func, const Reduction& reduction)
|
||||
{
|
||||
return parallel_for_for_prefix_sum1(state,array2,size_t(1),identity,func,reduction);
|
||||
}
|
||||
}
|
||||
@@ -1,60 +0,0 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#include "parallel_map.h"
|
||||
#include "../sys/regression.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
struct parallel_map_regression_test : public RegressionTest
|
||||
{
|
||||
parallel_map_regression_test(const char* name) : RegressionTest(name) {
|
||||
registerRegressionTest(this);
|
||||
}
|
||||
|
||||
bool run ()
|
||||
{
|
||||
bool passed = true;
|
||||
|
||||
/* create key/value vectors with random numbers */
|
||||
const size_t N = 10000;
|
||||
std::vector<uint32_t> keys(N);
|
||||
std::vector<uint32_t> vals(N);
|
||||
for (size_t i=0; i<N; i++) keys[i] = 2*unsigned(i)*647382649;
|
||||
for (size_t i=0; i<N; i++) std::swap(keys[i],keys[rand()%N]);
|
||||
for (size_t i=0; i<N; i++) vals[i] = 2*rand();
|
||||
|
||||
/* create map */
|
||||
parallel_map<uint32_t,uint32_t> map;
|
||||
map.init(keys,vals);
|
||||
|
||||
/* check that all keys are properly mapped */
|
||||
for (size_t i=0; i<N; i++) {
|
||||
const uint32_t* val = map.lookup(keys[i]);
|
||||
passed &= val && (*val == vals[i]);
|
||||
}
|
||||
|
||||
/* check that these keys are not in the map */
|
||||
for (size_t i=0; i<N; i++) {
|
||||
passed &= !map.lookup(keys[i]+1);
|
||||
}
|
||||
|
||||
return passed;
|
||||
}
|
||||
};
|
||||
|
||||
parallel_map_regression_test parallel_map_regression("parallel_map_regression_test");
|
||||
}
|
||||
@@ -1,98 +0,0 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#pragma once
|
||||
|
||||
#include "parallel_sort.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
/*! implementation of a key/value map with parallel construction */
|
||||
template<typename Key, typename Val>
|
||||
class parallel_map
|
||||
{
|
||||
/* key/value pair to build the map */
|
||||
struct KeyValue
|
||||
{
|
||||
__forceinline KeyValue () {}
|
||||
|
||||
__forceinline KeyValue (const Key key, const Val val)
|
||||
: key(key), val(val) {}
|
||||
|
||||
__forceinline operator Key() const {
|
||||
return key;
|
||||
}
|
||||
|
||||
public:
|
||||
Key key;
|
||||
Val val;
|
||||
};
|
||||
|
||||
public:
|
||||
|
||||
/*! parallel map constructors */
|
||||
parallel_map () {}
|
||||
|
||||
/*! construction from pair of vectors */
|
||||
template<typename KeyVector, typename ValVector>
|
||||
parallel_map (const KeyVector& keys, const ValVector& values) { init(keys,values); }
|
||||
|
||||
/*! initialized the parallel map from a vector with keys and values */
|
||||
template<typename KeyVector, typename ValVector>
|
||||
void init(const KeyVector& keys, const ValVector& values)
|
||||
{
|
||||
/* reserve sufficient space for all data */
|
||||
assert(keys.size() == values.size());
|
||||
vec.resize(keys.size());
|
||||
|
||||
/* generate key/value pairs */
|
||||
parallel_for( size_t(0), keys.size(), size_t(4*4096), [&](const range<size_t>& r) {
|
||||
for (size_t i=r.begin(); i<r.end(); i++)
|
||||
vec[i] = KeyValue((Key)keys[i],values[i]);
|
||||
});
|
||||
|
||||
/* perform parallel radix sort of the key/value pairs */
|
||||
std::vector<KeyValue> temp(keys.size());
|
||||
radix_sort<KeyValue,Key>(vec.data(),temp.data(),keys.size());
|
||||
}
|
||||
|
||||
/*! Returns a pointer to the value associated with the specified key. The pointer will be nullptr of the key is not contained in the map. */
|
||||
__forceinline const Val* lookup(const Key& key) const
|
||||
{
|
||||
typename std::vector<KeyValue>::const_iterator i = std::lower_bound(vec.begin(), vec.end(), key);
|
||||
if (i == vec.end()) return nullptr;
|
||||
if (i->key != key) return nullptr;
|
||||
return &i->val;
|
||||
}
|
||||
|
||||
/*! If the key is in the map, the function returns the value associated with the key, otherwise it returns the default value. */
|
||||
__forceinline Val lookup(const Key& key, const Val& def) const
|
||||
{
|
||||
typename std::vector<KeyValue>::const_iterator i = std::lower_bound(vec.begin(), vec.end(), key);
|
||||
if (i == vec.end()) return def;
|
||||
if (i->key != key) return def;
|
||||
return i->val;
|
||||
}
|
||||
|
||||
/*! clears all state */
|
||||
void clear() {
|
||||
vec.clear();
|
||||
}
|
||||
|
||||
private:
|
||||
std::vector<KeyValue> vec; //!< vector containing sorted elements
|
||||
};
|
||||
}
|
||||
@@ -1,66 +0,0 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#include "parallel_partition.h"
|
||||
#include "../sys/regression.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
struct parallel_partition_regression_test : public RegressionTest
|
||||
{
|
||||
parallel_partition_regression_test(const char* name) : RegressionTest(name) {
|
||||
registerRegressionTest(this);
|
||||
}
|
||||
|
||||
bool run ()
|
||||
{
|
||||
bool passed = true;
|
||||
|
||||
for (size_t i=0; i<100; i++)
|
||||
{
|
||||
/* create random permutation */
|
||||
size_t N = std::rand() % 1000000;
|
||||
std::vector<unsigned> array(N);
|
||||
for (unsigned i=0; i<N; i++) array[i] = i;
|
||||
for (auto& v : array) std::swap(v,array[std::rand()%array.size()]);
|
||||
size_t split = std::rand() % (N+1);
|
||||
|
||||
/* perform parallel partitioning */
|
||||
size_t left_sum = 0, right_sum = 0;
|
||||
size_t mid = parallel_partitioning(array.data(),0,array.size(),0,left_sum,right_sum,
|
||||
[&] ( size_t i ) { return i < split; },
|
||||
[] ( size_t& sum, unsigned v) { sum += v; },
|
||||
[] ( size_t& sum, size_t v) { sum += v; },
|
||||
128);
|
||||
|
||||
/*serial_partitioning(array.data(),0,array.size(),left_sum,right_sum,
|
||||
[&] ( size_t i ) { return i < split; },
|
||||
[] ( size_t& left_sum, int v) { left_sum += v; });*/
|
||||
|
||||
/* verify result */
|
||||
passed &= mid == split;
|
||||
passed &= left_sum == split*(split-1)/2;
|
||||
passed &= right_sum == N*(N-1)/2-left_sum;
|
||||
for (size_t i=0; i<split; i++) passed &= array[i] < split;
|
||||
for (size_t i=split; i<N; i++) passed &= array[i] >= split;
|
||||
}
|
||||
|
||||
return passed;
|
||||
}
|
||||
};
|
||||
|
||||
parallel_partition_regression_test parallel_partition_regression("parallel_partition_regression_test");
|
||||
}
|
||||
@@ -1,278 +0,0 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#pragma once
|
||||
|
||||
#include "parallel_for.h"
|
||||
#include "../common/math/range.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
/* serial partitioning */
|
||||
template<typename T, typename V, typename IsLeft, typename Reduction_T>
|
||||
__forceinline size_t serial_partitioning(T* array,
|
||||
const size_t begin,
|
||||
const size_t end,
|
||||
V& leftReduction,
|
||||
V& rightReduction,
|
||||
const IsLeft& is_left,
|
||||
const Reduction_T& reduction_t)
|
||||
{
|
||||
T* l = array + begin;
|
||||
T* r = array + end - 1;
|
||||
|
||||
while(1)
|
||||
{
|
||||
/* *l < pivot */
|
||||
while (likely(l <= r && is_left(*l) ))
|
||||
{
|
||||
//prefetchw(l+4); // FIXME: enable?
|
||||
reduction_t(leftReduction,*l);
|
||||
++l;
|
||||
}
|
||||
/* *r >= pivot) */
|
||||
while (likely(l <= r && !is_left(*r)))
|
||||
{
|
||||
//prefetchw(r-4); FIXME: enable?
|
||||
reduction_t(rightReduction,*r);
|
||||
--r;
|
||||
}
|
||||
if (r<l) break;
|
||||
|
||||
reduction_t(leftReduction ,*r);
|
||||
reduction_t(rightReduction,*l);
|
||||
xchg(*l,*r);
|
||||
l++; r--;
|
||||
}
|
||||
|
||||
return l - array;
|
||||
}
|
||||
|
||||
template<typename T, typename V, typename Vi, typename IsLeft, typename Reduction_T, typename Reduction_V>
|
||||
class __aligned(64) parallel_partition_task
|
||||
{
|
||||
ALIGNED_CLASS;
|
||||
private:
|
||||
|
||||
static const size_t MAX_TASKS = 512;
|
||||
|
||||
T* array;
|
||||
size_t N;
|
||||
const IsLeft& is_left;
|
||||
const Reduction_T& reduction_t;
|
||||
const Reduction_V& reduction_v;
|
||||
const Vi& identity;
|
||||
|
||||
size_t numTasks;
|
||||
__aligned(64) size_t counter_start[MAX_TASKS+1];
|
||||
__aligned(64) size_t counter_left[MAX_TASKS+1];
|
||||
__aligned(64) range<ssize_t> leftMisplacedRanges[MAX_TASKS];
|
||||
__aligned(64) range<ssize_t> rightMisplacedRanges[MAX_TASKS];
|
||||
__aligned(64) V leftReductions[MAX_TASKS];
|
||||
__aligned(64) V rightReductions[MAX_TASKS];
|
||||
|
||||
public:
|
||||
|
||||
__forceinline parallel_partition_task(T* array,
|
||||
const size_t N,
|
||||
const Vi& identity,
|
||||
const IsLeft& is_left,
|
||||
const Reduction_T& reduction_t,
|
||||
const Reduction_V& reduction_v,
|
||||
const size_t BLOCK_SIZE)
|
||||
|
||||
: array(array), N(N), is_left(is_left), reduction_t(reduction_t), reduction_v(reduction_v), identity(identity),
|
||||
numTasks(min((N+BLOCK_SIZE-1)/BLOCK_SIZE,min(TaskScheduler::threadCount(),MAX_TASKS))) {}
|
||||
|
||||
__forceinline const range<ssize_t>* findStartRange(size_t& index, const range<ssize_t>* const r, const size_t numRanges)
|
||||
{
|
||||
size_t i = 0;
|
||||
while(index >= (size_t)r[i].size())
|
||||
{
|
||||
assert(i < numRanges);
|
||||
index -= (size_t)r[i].size();
|
||||
i++;
|
||||
}
|
||||
return &r[i];
|
||||
}
|
||||
|
||||
__forceinline void swapItemsInMisplacedRanges(const size_t numLeftMisplacedRanges,
|
||||
const size_t numRightMisplacedRanges,
|
||||
const size_t startID,
|
||||
const size_t endID)
|
||||
{
|
||||
size_t leftLocalIndex = startID;
|
||||
size_t rightLocalIndex = startID;
|
||||
const range<ssize_t>* l_range = findStartRange(leftLocalIndex,leftMisplacedRanges,numLeftMisplacedRanges);
|
||||
const range<ssize_t>* r_range = findStartRange(rightLocalIndex,rightMisplacedRanges,numRightMisplacedRanges);
|
||||
|
||||
size_t l_left = l_range->size() - leftLocalIndex;
|
||||
size_t r_left = r_range->size() - rightLocalIndex;
|
||||
T *__restrict__ l = &array[l_range->begin() + leftLocalIndex];
|
||||
T *__restrict__ r = &array[r_range->begin() + rightLocalIndex];
|
||||
size_t size = endID - startID;
|
||||
size_t items = min(size,min(l_left,r_left));
|
||||
|
||||
while (size)
|
||||
{
|
||||
if (unlikely(l_left == 0))
|
||||
{
|
||||
l_range++;
|
||||
l_left = l_range->size();
|
||||
l = &array[l_range->begin()];
|
||||
items = min(size,min(l_left,r_left));
|
||||
}
|
||||
|
||||
if (unlikely(r_left == 0))
|
||||
{
|
||||
r_range++;
|
||||
r_left = r_range->size();
|
||||
r = &array[r_range->begin()];
|
||||
items = min(size,min(l_left,r_left));
|
||||
}
|
||||
|
||||
size -= items;
|
||||
l_left -= items;
|
||||
r_left -= items;
|
||||
|
||||
while(items) {
|
||||
items--;
|
||||
xchg(*l++,*r++);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
__forceinline size_t partition(V& leftReduction, V& rightReduction)
|
||||
{
|
||||
/* partition the individual ranges for each task */
|
||||
parallel_for(numTasks,[&] (const size_t taskID) {
|
||||
const size_t startID = (taskID+0)*N/numTasks;
|
||||
const size_t endID = (taskID+1)*N/numTasks;
|
||||
V local_left(identity);
|
||||
V local_right(identity);
|
||||
const size_t mid = serial_partitioning(array,startID,endID,local_left,local_right,is_left,reduction_t);
|
||||
counter_start[taskID] = startID;
|
||||
counter_left [taskID] = mid-startID;
|
||||
leftReductions[taskID] = local_left;
|
||||
rightReductions[taskID] = local_right;
|
||||
});
|
||||
counter_start[numTasks] = N;
|
||||
counter_left[numTasks] = 0;
|
||||
|
||||
/* finalize the reductions */
|
||||
for (size_t i=0; i<numTasks; i++) {
|
||||
reduction_v(leftReduction,leftReductions[i]);
|
||||
reduction_v(rightReduction,rightReductions[i]);
|
||||
}
|
||||
|
||||
/* calculate mid point for partitioning */
|
||||
size_t mid = counter_left[0];
|
||||
for (size_t i=1; i<numTasks; i++)
|
||||
mid += counter_left[i];
|
||||
const range<ssize_t> globalLeft (0,mid);
|
||||
const range<ssize_t> globalRight(mid,N);
|
||||
|
||||
/* calculate all left and right ranges that are on the wrong global side */
|
||||
size_t numMisplacedRangesLeft = 0;
|
||||
size_t numMisplacedRangesRight = 0;
|
||||
size_t numMisplacedItemsLeft = 0;
|
||||
size_t numMisplacedItemsRight = 0;
|
||||
|
||||
for (size_t i=0; i<numTasks; i++)
|
||||
{
|
||||
const range<ssize_t> left_range (counter_start[i], counter_start[i] + counter_left[i]);
|
||||
const range<ssize_t> right_range(counter_start[i] + counter_left[i], counter_start[i+1]);
|
||||
const range<ssize_t> left_misplaced = globalLeft. intersect(right_range);
|
||||
const range<ssize_t> right_misplaced = globalRight.intersect(left_range);
|
||||
|
||||
if (!left_misplaced.empty())
|
||||
{
|
||||
numMisplacedItemsLeft += left_misplaced.size();
|
||||
leftMisplacedRanges[numMisplacedRangesLeft++] = left_misplaced;
|
||||
}
|
||||
|
||||
if (!right_misplaced.empty())
|
||||
{
|
||||
numMisplacedItemsRight += right_misplaced.size();
|
||||
rightMisplacedRanges[numMisplacedRangesRight++] = right_misplaced;
|
||||
}
|
||||
}
|
||||
assert( numMisplacedItemsLeft == numMisplacedItemsRight );
|
||||
|
||||
/* if no items are misplaced we are done */
|
||||
if (numMisplacedItemsLeft == 0)
|
||||
return mid;
|
||||
|
||||
/* otherwise we copy the items to the right place in parallel */
|
||||
parallel_for(numTasks,[&] (const size_t taskID) {
|
||||
const size_t startID = (taskID+0)*numMisplacedItemsLeft/numTasks;
|
||||
const size_t endID = (taskID+1)*numMisplacedItemsLeft/numTasks;
|
||||
swapItemsInMisplacedRanges(numMisplacedRangesLeft,numMisplacedRangesRight,startID,endID);
|
||||
});
|
||||
|
||||
return mid;
|
||||
}
|
||||
};
|
||||
|
||||
template<typename T, typename V, typename Vi, typename IsLeft, typename Reduction_T, typename Reduction_V>
|
||||
__noinline size_t parallel_partitioning(T* array,
|
||||
const size_t begin,
|
||||
const size_t end,
|
||||
const Vi &identity,
|
||||
V &leftReduction,
|
||||
V &rightReduction,
|
||||
const IsLeft& is_left,
|
||||
const Reduction_T& reduction_t,
|
||||
const Reduction_V& reduction_v,
|
||||
size_t BLOCK_SIZE = 128)
|
||||
{
|
||||
/* fall back to single threaded partitioning for small N */
|
||||
if (unlikely(end-begin < BLOCK_SIZE))
|
||||
return serial_partitioning(array,begin,end,leftReduction,rightReduction,is_left,reduction_t);
|
||||
|
||||
/* otherwise use parallel code */
|
||||
else {
|
||||
typedef parallel_partition_task<T,V,Vi,IsLeft,Reduction_T,Reduction_V> partition_task;
|
||||
std::unique_ptr<partition_task> p(new partition_task(&array[begin],end-begin,identity,is_left,reduction_t,reduction_v,BLOCK_SIZE));
|
||||
return begin+p->partition(leftReduction,rightReduction);
|
||||
}
|
||||
}
|
||||
|
||||
template<typename T, typename V, typename Vi, typename IsLeft, typename Reduction_T, typename Reduction_V>
|
||||
__noinline size_t parallel_partitioning(T* array,
|
||||
const size_t begin,
|
||||
const size_t end,
|
||||
const Vi &identity,
|
||||
V &leftReduction,
|
||||
V &rightReduction,
|
||||
const IsLeft& is_left,
|
||||
const Reduction_T& reduction_t,
|
||||
const Reduction_V& reduction_v,
|
||||
size_t BLOCK_SIZE,
|
||||
size_t PARALLEL_THRESHOLD)
|
||||
{
|
||||
/* fall back to single threaded partitioning for small N */
|
||||
if (unlikely(end-begin < PARALLEL_THRESHOLD))
|
||||
return serial_partitioning(array,begin,end,leftReduction,rightReduction,is_left,reduction_t);
|
||||
|
||||
/* otherwise use parallel code */
|
||||
else {
|
||||
typedef parallel_partition_task<T,V,Vi,IsLeft,Reduction_T,Reduction_V> partition_task;
|
||||
std::unique_ptr<partition_task> p(new partition_task(&array[begin],end-begin,identity,is_left,reduction_t,reduction_v,BLOCK_SIZE));
|
||||
return begin+p->partition(leftReduction,rightReduction);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,61 +0,0 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#include "parallel_prefix_sum.h"
|
||||
#include "../sys/regression.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
struct parallel_prefix_sum_regression_test : public RegressionTest
|
||||
{
|
||||
parallel_prefix_sum_regression_test(const char* name) : RegressionTest(name) {
|
||||
registerRegressionTest(this);
|
||||
}
|
||||
|
||||
bool run ()
|
||||
{
|
||||
bool passed = true;
|
||||
const size_t M = 10;
|
||||
|
||||
for (size_t N=10; N<10000000; N=size_t(2.1*N))
|
||||
{
|
||||
/* initialize array with random numbers */
|
||||
uint32_t sum0 = 0;
|
||||
std::vector<uint32_t> src(N);
|
||||
for (size_t i=0; i<N; i++) {
|
||||
sum0 += src[i] = rand();
|
||||
}
|
||||
|
||||
/* calculate parallel prefix sum */
|
||||
std::vector<uint32_t> dst(N);
|
||||
for (auto& v : dst) v = 0;
|
||||
|
||||
for (size_t i=0; i<M; i++) {
|
||||
uint32_t sum1 = parallel_prefix_sum(src,dst,N,0,std::plus<uint32_t>());
|
||||
passed &= (sum0 == sum1);
|
||||
}
|
||||
|
||||
/* check if prefix sum is correct */
|
||||
for (size_t i=0, sum=0; i<N; sum+=src[i++])
|
||||
passed &= ((uint32_t)sum == dst[i]);
|
||||
}
|
||||
|
||||
return passed;
|
||||
}
|
||||
};
|
||||
|
||||
parallel_prefix_sum_regression_test parallel_prefix_sum_regression("parallel_prefix_sum_regression");
|
||||
}
|
||||
@@ -1,98 +0,0 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#pragma once
|
||||
|
||||
#include "parallel_for.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
template<typename Value>
|
||||
struct ParallelPrefixSumState
|
||||
{
|
||||
enum { MAX_TASKS = 512 };
|
||||
Value counts[MAX_TASKS];
|
||||
Value sums [MAX_TASKS];
|
||||
};
|
||||
|
||||
template<typename Index, typename Value, typename Func, typename Reduction>
|
||||
__forceinline Value parallel_prefix_sum( ParallelPrefixSumState<Value>& state, Index first, Index last, Index minStepSize, const Value& identity, const Func& func, const Reduction& reduction)
|
||||
{
|
||||
/* calculate number of tasks to use */
|
||||
const size_t numThreads = TaskScheduler::threadCount();
|
||||
const size_t numBlocks = (last-first+minStepSize-1)/minStepSize;
|
||||
const size_t taskCount = min(numThreads,numBlocks,size_t(ParallelPrefixSumState<Value>::MAX_TASKS));
|
||||
|
||||
/* perform parallel prefix sum */
|
||||
parallel_for(taskCount, [&](const size_t taskIndex)
|
||||
{
|
||||
const size_t i0 = first+(taskIndex+0)*(last-first)/taskCount;
|
||||
const size_t i1 = first+(taskIndex+1)*(last-first)/taskCount;
|
||||
state.counts[taskIndex] = func(range<size_t>(i0,i1),state.sums[taskIndex]);
|
||||
});
|
||||
|
||||
/* calculate prefix sum */
|
||||
Value sum=identity;
|
||||
for (size_t i=0; i<taskCount; i++)
|
||||
{
|
||||
const Value c = state.counts[i];
|
||||
state.sums[i] = sum;
|
||||
sum=reduction(sum,c);
|
||||
}
|
||||
|
||||
return sum;
|
||||
}
|
||||
|
||||
/*! parallel calculation of prefix sums */
|
||||
template<typename SrcArray, typename DstArray, typename Value, typename Add>
|
||||
__forceinline Value parallel_prefix_sum(const SrcArray& src, DstArray& dst, size_t N, const Value& identity, const Add& add, const size_t SINGLE_THREAD_THRESHOLD = 4096)
|
||||
{
|
||||
/* perform single threaded prefix operation for small N */
|
||||
if (N < SINGLE_THREAD_THRESHOLD)
|
||||
{
|
||||
Value sum=identity;
|
||||
for (size_t i=0; i<N; sum=add(sum,src[i++])) dst[i] = sum;
|
||||
return sum;
|
||||
}
|
||||
|
||||
/* perform parallel prefix operation for large N */
|
||||
else
|
||||
{
|
||||
ParallelPrefixSumState<Value> state;
|
||||
|
||||
/* initial run just sets up start values for subtasks */
|
||||
parallel_prefix_sum( state, size_t(0), size_t(N), size_t(1024), identity, [&](const range<size_t>& r, const Value& sum) -> Value {
|
||||
|
||||
Value s = identity;
|
||||
for (size_t i=r.begin(); i<r.end(); i++) s = add(s,src[i]);
|
||||
return s;
|
||||
|
||||
}, add);
|
||||
|
||||
/* final run calculates prefix sum */
|
||||
return parallel_prefix_sum( state, size_t(0), size_t(N), size_t(1024), identity, [&](const range<size_t>& r, const Value& sum) -> Value {
|
||||
|
||||
Value s = identity;
|
||||
for (size_t i=r.begin(); i<r.end(); i++) {
|
||||
dst[i] = add(sum,s);
|
||||
s = add(s,src[i]);
|
||||
}
|
||||
return s;
|
||||
|
||||
}, add);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,62 +0,0 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#include "parallel_reduce.h"
|
||||
#include "../sys/regression.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
struct parallel_reduce_regression_test : public RegressionTest
|
||||
{
|
||||
parallel_reduce_regression_test(const char* name) : RegressionTest(name) {
|
||||
registerRegressionTest(this);
|
||||
}
|
||||
|
||||
bool run ()
|
||||
{
|
||||
bool passed = true;
|
||||
|
||||
const size_t M = 10;
|
||||
for (size_t N=10; N<10000000; N=size_t(2.1*N))
|
||||
{
|
||||
/* sequentially calculate sum of squares */
|
||||
size_t sum0 = 0;
|
||||
for (size_t i=0; i<N; i++) {
|
||||
sum0 += i*i;
|
||||
}
|
||||
|
||||
/* parallel calculation of sum of squares */
|
||||
for (size_t m=0; m<M; m++)
|
||||
{
|
||||
size_t sum1 = parallel_reduce( size_t(0), size_t(N), size_t(1024), size_t(0), [&](const range<size_t>& r) -> size_t
|
||||
{
|
||||
size_t s = 0;
|
||||
for (size_t i=r.begin(); i<r.end(); i++)
|
||||
s += i*i;
|
||||
return s;
|
||||
},
|
||||
[](const size_t v0, const size_t v1) {
|
||||
return v0+v1;
|
||||
});
|
||||
passed = sum0 == sum1;
|
||||
}
|
||||
}
|
||||
return passed;
|
||||
}
|
||||
};
|
||||
|
||||
parallel_reduce_regression_test parallel_reduce_regression("parallel_reduce_regression_test");
|
||||
}
|
||||
@@ -1,149 +0,0 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#pragma once
|
||||
|
||||
#include "parallel_for.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
template<typename Index, typename Value, typename Func, typename Reduction>
|
||||
__forceinline Value sequential_reduce( const Index first, const Index last, const Value& identity, const Func& func, const Reduction& reduction )
|
||||
{
|
||||
return func(range<Index>(first,last));
|
||||
}
|
||||
|
||||
template<typename Index, typename Value, typename Func, typename Reduction>
|
||||
__forceinline Value sequential_reduce( const Index first, const Index last, const Index minStepSize, const Value& identity, const Func& func, const Reduction& reduction )
|
||||
{
|
||||
return func(range<Index>(first,last));
|
||||
}
|
||||
|
||||
template<typename Index, typename Value, typename Func, typename Reduction>
|
||||
__noinline Value parallel_reduce_internal( Index taskCount, const Index first, const Index last, const Index minStepSize, const Value& identity, const Func& func, const Reduction& reduction )
|
||||
{
|
||||
const Index maxTasks = 512;
|
||||
const Index threadCount = (Index) TaskScheduler::threadCount();
|
||||
taskCount = min(taskCount,threadCount,maxTasks);
|
||||
|
||||
/* parallel invokation of all tasks */
|
||||
dynamic_large_stack_array(Value,values,taskCount,4096); // consumes at most 4096 bytes on the stack
|
||||
parallel_for(taskCount, [&](const Index taskIndex) {
|
||||
const Index k0 = first+(taskIndex+0)*(last-first)/taskCount;
|
||||
const Index k1 = first+(taskIndex+1)*(last-first)/taskCount;
|
||||
values[taskIndex] = func(range<Index>(k0,k1));
|
||||
});
|
||||
|
||||
/* perform reduction over all tasks */
|
||||
Value v = identity;
|
||||
for (Index i=0; i<taskCount; i++) v = reduction(v,values[i]);
|
||||
return v;
|
||||
}
|
||||
|
||||
template<typename Index, typename Value, typename Func, typename Reduction>
|
||||
__forceinline Value parallel_reduce( const Index first, const Index last, const Index minStepSize, const Value& identity, const Func& func, const Reduction& reduction )
|
||||
{
|
||||
#if defined(TASKING_INTERNAL)
|
||||
|
||||
/* fast path for small number of iterations */
|
||||
Index taskCount = (last-first+minStepSize-1)/minStepSize;
|
||||
if (likely(taskCount == 1)) {
|
||||
return func(range<Index>(first,last));
|
||||
}
|
||||
return parallel_reduce_internal(taskCount,first,last,minStepSize,identity,func,reduction);
|
||||
|
||||
#elif defined(TASKING_TBB)
|
||||
const Value v = tbb::parallel_reduce(tbb::blocked_range<Index>(first,last,minStepSize),identity,
|
||||
[&](const tbb::blocked_range<Index>& r, const Value& start) { return reduction(start,func(range<Index>(r.begin(),r.end()))); },
|
||||
reduction);
|
||||
if (tbb::task::self().is_cancelled())
|
||||
throw std::runtime_error("task cancelled");
|
||||
return v;
|
||||
#else // TASKING_PPL
|
||||
struct AlignedValue
|
||||
{
|
||||
char storage[__alignof(Value)+sizeof(Value)];
|
||||
static uintptr_t alignUp(uintptr_t p, size_t a) { return p + (~(p - 1) % a); };
|
||||
Value* getValuePtr() { return reinterpret_cast<Value*>(alignUp(uintptr_t(storage), __alignof(Value))); }
|
||||
const Value* getValuePtr() const { return reinterpret_cast<Value*>(alignUp(uintptr_t(storage), __alignof(Value))); }
|
||||
AlignedValue(const Value& v) { new(getValuePtr()) Value(v); }
|
||||
AlignedValue(const AlignedValue& v) { new(getValuePtr()) Value(*v.getValuePtr()); }
|
||||
AlignedValue(const AlignedValue&& v) { new(getValuePtr()) Value(*v.getValuePtr()); };
|
||||
AlignedValue& operator = (const AlignedValue& v) { *getValuePtr() = *v.getValuePtr(); return *this; };
|
||||
AlignedValue& operator = (const AlignedValue&& v) { *getValuePtr() = *v.getValuePtr(); return *this; };
|
||||
operator Value() const { return *getValuePtr(); }
|
||||
};
|
||||
|
||||
struct Iterator_Index
|
||||
{
|
||||
Index v;
|
||||
typedef std::forward_iterator_tag iterator_category;
|
||||
typedef AlignedValue value_type;
|
||||
typedef Index difference_type;
|
||||
typedef Index distance_type;
|
||||
typedef AlignedValue* pointer;
|
||||
typedef AlignedValue& reference;
|
||||
__forceinline Iterator_Index() {}
|
||||
__forceinline Iterator_Index(Index v) : v(v) {}
|
||||
__forceinline bool operator== (Iterator_Index other) { return v == other.v; }
|
||||
__forceinline bool operator!= (Iterator_Index other) { return v != other.v; }
|
||||
__forceinline Iterator_Index operator++() { return Iterator_Index(++v); }
|
||||
__forceinline Iterator_Index operator++(int) { return Iterator_Index(v++); }
|
||||
};
|
||||
|
||||
auto range_reduction = [&](Iterator_Index begin, Iterator_Index end, const AlignedValue& start) {
|
||||
assert(begin.v < end.v);
|
||||
return reduction(start, func(range<Index>(begin.v, end.v)));
|
||||
};
|
||||
const Value v = concurrency::parallel_reduce(Iterator_Index(first), Iterator_Index(last), AlignedValue(identity), range_reduction, reduction);
|
||||
return v;
|
||||
#endif
|
||||
}
|
||||
|
||||
template<typename Index, typename Value, typename Func, typename Reduction>
|
||||
__forceinline Value parallel_reduce( const Index first, const Index last, const Index minStepSize, const Index parallel_threshold, const Value& identity, const Func& func, const Reduction& reduction )
|
||||
{
|
||||
if (likely(last-first < parallel_threshold)) {
|
||||
return func(range<Index>(first,last));
|
||||
} else {
|
||||
return parallel_reduce(first,last,minStepSize,identity,func,reduction);
|
||||
}
|
||||
}
|
||||
|
||||
template<typename Index, typename Value, typename Func, typename Reduction>
|
||||
__forceinline Value parallel_reduce( const range<Index> range, const Index minStepSize, const Index parallel_threshold, const Value& identity, const Func& func, const Reduction& reduction )
|
||||
{
|
||||
return parallel_reduce(range.begin(),range.end(),minStepSize,parallel_threshold,identity,func,reduction);
|
||||
}
|
||||
|
||||
template<typename Index, typename Value, typename Func, typename Reduction>
|
||||
__forceinline Value parallel_reduce( const Index first, const Index last, const Value& identity, const Func& func, const Reduction& reduction )
|
||||
{
|
||||
auto funcr = [&] ( const range<Index> r ) {
|
||||
Value v = identity;
|
||||
for (Index i=r.begin(); i<r.end(); i++)
|
||||
v = reduction(v,func(i));
|
||||
return v;
|
||||
};
|
||||
return parallel_reduce(first,last,Index(1),identity,funcr,reduction);
|
||||
}
|
||||
|
||||
template<typename Index, typename Value, typename Func, typename Reduction>
|
||||
__forceinline Value parallel_reduce( const range<Index> range, const Value& identity, const Func& func, const Reduction& reduction )
|
||||
{
|
||||
return parallel_reduce(range.begin(),range.end(),Index(1),identity,func,reduction);
|
||||
}
|
||||
}
|
||||
@@ -1,65 +0,0 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#pragma once
|
||||
|
||||
#include "parallel_sort.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
/* implementation of a set of values with parallel construction */
|
||||
template<typename T>
|
||||
class parallel_set
|
||||
{
|
||||
public:
|
||||
|
||||
/*! default constructor for the parallel set */
|
||||
parallel_set () {}
|
||||
|
||||
/*! construction from vector */
|
||||
template<typename Vector>
|
||||
parallel_set (const Vector& in) { init(in); }
|
||||
|
||||
/*! initialized the parallel set from a vector */
|
||||
template<typename Vector>
|
||||
void init(const Vector& in)
|
||||
{
|
||||
/* copy data to internal vector */
|
||||
vec.resize(in.size());
|
||||
parallel_for( size_t(0), in.size(), size_t(4*4096), [&](const range<size_t>& r) {
|
||||
for (size_t i=r.begin(); i<r.end(); i++)
|
||||
vec[i] = in[i];
|
||||
});
|
||||
|
||||
/* sort the data */
|
||||
std::vector<T> temp(in.size());
|
||||
radix_sort<T>(vec.data(),temp.data(),vec.size());
|
||||
}
|
||||
|
||||
/*! tests if some element is in the set */
|
||||
__forceinline bool lookup(const T& elt) const {
|
||||
return std::binary_search(vec.begin(), vec.end(), elt);
|
||||
}
|
||||
|
||||
/*! clears all state */
|
||||
void clear() {
|
||||
vec.clear();
|
||||
}
|
||||
|
||||
private:
|
||||
std::vector<T> vec; //!< vector containing sorted elements
|
||||
};
|
||||
}
|
||||
@@ -1,63 +0,0 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#include "parallel_sort.h"
|
||||
#include "../sys/regression.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
template<typename Key>
|
||||
struct RadixSortRegressionTest : public RegressionTest
|
||||
{
|
||||
RadixSortRegressionTest(const char* name) : RegressionTest(name) {
|
||||
registerRegressionTest(this);
|
||||
}
|
||||
|
||||
bool run ()
|
||||
{
|
||||
bool passed = true;
|
||||
const size_t M = 10;
|
||||
|
||||
for (size_t N=10; N<1000000; N=size_t(2.1*N))
|
||||
{
|
||||
std::vector<Key> src(N); memset(src.data(),0,N*sizeof(Key));
|
||||
std::vector<Key> tmp(N); memset(tmp.data(),0,N*sizeof(Key));
|
||||
for (size_t i=0; i<N; i++) src[i] = uint64_t(rand())*uint64_t(rand());
|
||||
|
||||
/* calculate checksum */
|
||||
Key sum0 = 0; for (size_t i=0; i<N; i++) sum0 += src[i];
|
||||
|
||||
/* sort numbers */
|
||||
for (size_t i=0; i<M; i++) {
|
||||
radix_sort<Key>(src.data(),tmp.data(),N);
|
||||
}
|
||||
|
||||
/* calculate checksum */
|
||||
Key sum1 = 0; for (size_t i=0; i<N; i++) sum1 += src[i];
|
||||
if (sum0 != sum1) passed = false;
|
||||
|
||||
/* check if numbers are sorted */
|
||||
for (size_t i=1; i<N; i++)
|
||||
passed &= src[i-1] <= src[i];
|
||||
}
|
||||
|
||||
return passed;
|
||||
}
|
||||
};
|
||||
|
||||
RadixSortRegressionTest<uint32_t> test_u32("RadixSortRegressionTestU32");
|
||||
RadixSortRegressionTest<uint64_t> test_u64("RadixSortRegressionTestU64");
|
||||
}
|
||||
@@ -1,456 +0,0 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#pragma once
|
||||
|
||||
#include "../simd/simd.h"
|
||||
#include "parallel_for.h"
|
||||
#include <algorithm>
|
||||
|
||||
namespace embree
|
||||
{
|
||||
template<class T>
|
||||
__forceinline void insertionsort_ascending(T *__restrict__ array, const size_t length)
|
||||
{
|
||||
for(size_t i = 1;i<length;++i)
|
||||
{
|
||||
T v = array[i];
|
||||
size_t j = i;
|
||||
while(j > 0 && v < array[j-1])
|
||||
{
|
||||
array[j] = array[j-1];
|
||||
--j;
|
||||
}
|
||||
array[j] = v;
|
||||
}
|
||||
}
|
||||
|
||||
template<class T>
|
||||
__forceinline void insertionsort_decending(T *__restrict__ array, const size_t length)
|
||||
{
|
||||
for(size_t i = 1;i<length;++i)
|
||||
{
|
||||
T v = array[i];
|
||||
size_t j = i;
|
||||
while(j > 0 && v > array[j-1])
|
||||
{
|
||||
array[j] = array[j-1];
|
||||
--j;
|
||||
}
|
||||
array[j] = v;
|
||||
}
|
||||
}
|
||||
|
||||
template<class T>
|
||||
void quicksort_ascending(T *__restrict__ t,
|
||||
const ssize_t begin,
|
||||
const ssize_t end)
|
||||
{
|
||||
if (likely(begin < end))
|
||||
{
|
||||
const T pivotvalue = t[begin];
|
||||
ssize_t left = begin - 1;
|
||||
ssize_t right = end + 1;
|
||||
|
||||
while(1)
|
||||
{
|
||||
while (t[--right] > pivotvalue);
|
||||
while (t[++left] < pivotvalue);
|
||||
|
||||
if (left >= right) break;
|
||||
|
||||
const T temp = t[right];
|
||||
t[right] = t[left];
|
||||
t[left] = temp;
|
||||
}
|
||||
|
||||
const int pivot = right;
|
||||
quicksort_ascending(t, begin, pivot);
|
||||
quicksort_ascending(t, pivot + 1, end);
|
||||
}
|
||||
}
|
||||
|
||||
template<class T>
|
||||
void quicksort_decending(T *__restrict__ t,
|
||||
const ssize_t begin,
|
||||
const ssize_t end)
|
||||
{
|
||||
if (likely(begin < end))
|
||||
{
|
||||
const T pivotvalue = t[begin];
|
||||
ssize_t left = begin - 1;
|
||||
ssize_t right = end + 1;
|
||||
|
||||
while(1)
|
||||
{
|
||||
while (t[--right] < pivotvalue);
|
||||
while (t[++left] > pivotvalue);
|
||||
|
||||
if (left >= right) break;
|
||||
|
||||
const T temp = t[right];
|
||||
t[right] = t[left];
|
||||
t[left] = temp;
|
||||
}
|
||||
|
||||
const int pivot = right;
|
||||
quicksort_decending(t, begin, pivot);
|
||||
quicksort_decending(t, pivot + 1, end);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
template<class T, ssize_t THRESHOLD>
|
||||
void quicksort_insertionsort_ascending(T *__restrict__ t,
|
||||
const ssize_t begin,
|
||||
const ssize_t end)
|
||||
{
|
||||
if (likely(begin < end))
|
||||
{
|
||||
const ssize_t size = end-begin+1;
|
||||
if (likely(size <= THRESHOLD))
|
||||
{
|
||||
insertionsort_ascending<T>(&t[begin],size);
|
||||
}
|
||||
else
|
||||
{
|
||||
const T pivotvalue = t[begin];
|
||||
ssize_t left = begin - 1;
|
||||
ssize_t right = end + 1;
|
||||
|
||||
while(1)
|
||||
{
|
||||
while (t[--right] > pivotvalue);
|
||||
while (t[++left] < pivotvalue);
|
||||
|
||||
if (left >= right) break;
|
||||
|
||||
const T temp = t[right];
|
||||
t[right] = t[left];
|
||||
t[left] = temp;
|
||||
}
|
||||
|
||||
const ssize_t pivot = right;
|
||||
quicksort_insertionsort_ascending<T,THRESHOLD>(t, begin, pivot);
|
||||
quicksort_insertionsort_ascending<T,THRESHOLD>(t, pivot + 1, end);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
template<class T, ssize_t THRESHOLD>
|
||||
void quicksort_insertionsort_decending(T *__restrict__ t,
|
||||
const ssize_t begin,
|
||||
const ssize_t end)
|
||||
{
|
||||
if (likely(begin < end))
|
||||
{
|
||||
const ssize_t size = end-begin+1;
|
||||
if (likely(size <= THRESHOLD))
|
||||
{
|
||||
insertionsort_decending<T>(&t[begin],size);
|
||||
}
|
||||
else
|
||||
{
|
||||
|
||||
const T pivotvalue = t[begin];
|
||||
ssize_t left = begin - 1;
|
||||
ssize_t right = end + 1;
|
||||
|
||||
while(1)
|
||||
{
|
||||
while (t[--right] < pivotvalue);
|
||||
while (t[++left] > pivotvalue);
|
||||
|
||||
if (left >= right) break;
|
||||
|
||||
const T temp = t[right];
|
||||
t[right] = t[left];
|
||||
t[left] = temp;
|
||||
}
|
||||
|
||||
const ssize_t pivot = right;
|
||||
quicksort_insertionsort_decending<T,THRESHOLD>(t, begin, pivot);
|
||||
quicksort_insertionsort_decending<T,THRESHOLD>(t, pivot + 1, end);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
template<typename T>
|
||||
static void radixsort32(T* const morton, const size_t num, const unsigned int shift = 3*8)
|
||||
{
|
||||
static const unsigned int BITS = 8;
|
||||
static const unsigned int BUCKETS = (1 << BITS);
|
||||
static const unsigned int CMP_SORT_THRESHOLD = 16;
|
||||
|
||||
__aligned(64) unsigned int count[BUCKETS];
|
||||
|
||||
/* clear buckets */
|
||||
for (size_t i=0;i<BUCKETS;i++) count[i] = 0;
|
||||
|
||||
/* count buckets */
|
||||
#if defined(__INTEL_COMPILER)
|
||||
#pragma nounroll
|
||||
#endif
|
||||
for (size_t i=0;i<num;i++)
|
||||
count[(unsigned(morton[i]) >> shift) & (BUCKETS-1)]++;
|
||||
|
||||
/* prefix sums */
|
||||
__aligned(64) unsigned int head[BUCKETS];
|
||||
__aligned(64) unsigned int tail[BUCKETS];
|
||||
|
||||
head[0] = 0;
|
||||
for (size_t i=1; i<BUCKETS; i++)
|
||||
head[i] = head[i-1] + count[i-1];
|
||||
|
||||
for (size_t i=0; i<BUCKETS-1; i++)
|
||||
tail[i] = head[i+1];
|
||||
|
||||
tail[BUCKETS-1] = head[BUCKETS-1] + count[BUCKETS-1];
|
||||
|
||||
assert(tail[BUCKETS-1] == head[BUCKETS-1] + count[BUCKETS-1]);
|
||||
assert(tail[BUCKETS-1] == num);
|
||||
|
||||
/* in-place swap */
|
||||
for (size_t i=0;i<BUCKETS;i++)
|
||||
{
|
||||
/* process bucket */
|
||||
while(head[i] < tail[i])
|
||||
{
|
||||
T v = morton[head[i]];
|
||||
while(1)
|
||||
{
|
||||
const size_t b = (unsigned(v) >> shift) & (BUCKETS-1);
|
||||
if (b == i) break;
|
||||
std::swap(v,morton[head[b]++]);
|
||||
}
|
||||
assert((unsigned(v) >> shift & (BUCKETS-1)) == i);
|
||||
morton[head[i]++] = v;
|
||||
}
|
||||
}
|
||||
if (shift == 0) return;
|
||||
|
||||
size_t offset = 0;
|
||||
for (size_t i=0;i<BUCKETS;i++)
|
||||
if (count[i])
|
||||
{
|
||||
|
||||
for (size_t j=offset;j<offset+count[i]-1;j++)
|
||||
assert(((unsigned(morton[j]) >> shift) & (BUCKETS-1)) == i);
|
||||
|
||||
if (unlikely(count[i] < CMP_SORT_THRESHOLD))
|
||||
insertionsort_ascending(morton + offset, count[i]);
|
||||
else
|
||||
radixsort32(morton + offset, count[i], shift-BITS);
|
||||
|
||||
for (size_t j=offset;j<offset+count[i]-1;j++)
|
||||
assert(morton[j] <= morton[j+1]);
|
||||
|
||||
offset += count[i];
|
||||
}
|
||||
}
|
||||
|
||||
template<typename Ty, typename Key>
|
||||
class ParallelRadixSort
|
||||
{
|
||||
static const size_t MAX_TASKS = 512;
|
||||
static const size_t BITS = 8;
|
||||
static const size_t BUCKETS = (1 << BITS);
|
||||
typedef unsigned int TyRadixCount[BUCKETS];
|
||||
|
||||
template<typename T>
|
||||
static bool compare(const T& v0, const T& v1) {
|
||||
return (Key)v0 < (Key)v1;
|
||||
}
|
||||
|
||||
public:
|
||||
ParallelRadixSort (Ty* const src, Ty* const tmp, const size_t N, const size_t blockSize)
|
||||
: radixCount(nullptr), src(src), tmp(tmp), N(N)
|
||||
{
|
||||
assert(blockSize > 0);
|
||||
|
||||
/* perform single threaded sort for small N */
|
||||
if (N<=blockSize) // handles also special case of 0!
|
||||
{
|
||||
/* do inplace sort inside destination array */
|
||||
std::sort(src,src+N,compare<Ty>);
|
||||
}
|
||||
|
||||
/* perform parallel sort for large N */
|
||||
else
|
||||
{
|
||||
const size_t numThreads = min((N+blockSize-1)/blockSize,TaskScheduler::threadCount(),size_t(MAX_TASKS));
|
||||
tbbRadixSort(numThreads);
|
||||
}
|
||||
}
|
||||
|
||||
private:
|
||||
|
||||
void tbbRadixIteration0(const Key shift,
|
||||
const Ty* __restrict const src,
|
||||
Ty* __restrict const dst,
|
||||
const size_t threadIndex, const size_t threadCount)
|
||||
{
|
||||
const size_t startID = (threadIndex+0)*N/threadCount;
|
||||
const size_t endID = (threadIndex+1)*N/threadCount;
|
||||
|
||||
/* mask to extract some number of bits */
|
||||
const Key mask = BUCKETS-1;
|
||||
|
||||
/* count how many items go into the buckets */
|
||||
for (size_t i=0; i<BUCKETS; i++)
|
||||
radixCount[threadIndex][i] = 0;
|
||||
|
||||
/* iterate over src array and count buckets */
|
||||
unsigned int * __restrict const count = radixCount[threadIndex];
|
||||
#if defined(__INTEL_COMPILER)
|
||||
#pragma nounroll
|
||||
#endif
|
||||
for (size_t i=startID; i<endID; i++) {
|
||||
#if defined(__X86_64__)
|
||||
const size_t index = ((size_t)(Key)src[i] >> (size_t)shift) & (size_t)mask;
|
||||
#else
|
||||
const Key index = ((Key)src[i] >> shift) & mask;
|
||||
#endif
|
||||
count[index]++;
|
||||
}
|
||||
}
|
||||
|
||||
void tbbRadixIteration1(const Key shift,
|
||||
const Ty* __restrict const src,
|
||||
Ty* __restrict const dst,
|
||||
const size_t threadIndex, const size_t threadCount)
|
||||
{
|
||||
const size_t startID = (threadIndex+0)*N/threadCount;
|
||||
const size_t endID = (threadIndex+1)*N/threadCount;
|
||||
|
||||
/* mask to extract some number of bits */
|
||||
const Key mask = BUCKETS-1;
|
||||
|
||||
/* calculate total number of items for each bucket */
|
||||
__aligned(64) unsigned int total[BUCKETS];
|
||||
/*
|
||||
for (size_t i=0; i<BUCKETS; i++)
|
||||
total[i] = 0;
|
||||
*/
|
||||
for (size_t i=0; i<BUCKETS; i+=VSIZEX)
|
||||
vintx::store(&total[i], zero);
|
||||
|
||||
for (size_t i=0; i<threadCount; i++)
|
||||
{
|
||||
/*
|
||||
for (size_t j=0; j<BUCKETS; j++)
|
||||
total[j] += radixCount[i][j];
|
||||
*/
|
||||
for (size_t j=0; j<BUCKETS; j+=VSIZEX)
|
||||
vintx::store(&total[j], vintx::load(&total[j]) + vintx::load(&radixCount[i][j]));
|
||||
}
|
||||
|
||||
/* calculate start offset of each bucket */
|
||||
__aligned(64) unsigned int offset[BUCKETS];
|
||||
offset[0] = 0;
|
||||
for (size_t i=1; i<BUCKETS; i++)
|
||||
offset[i] = offset[i-1] + total[i-1];
|
||||
|
||||
/* calculate start offset of each bucket for this thread */
|
||||
for (size_t i=0; i<threadIndex; i++)
|
||||
{
|
||||
/*
|
||||
for (size_t j=0; j<BUCKETS; j++)
|
||||
offset[j] += radixCount[i][j];
|
||||
*/
|
||||
for (size_t j=0; j<BUCKETS; j+=VSIZEX)
|
||||
vintx::store(&offset[j], vintx::load(&offset[j]) + vintx::load(&radixCount[i][j]));
|
||||
}
|
||||
|
||||
/* copy items into their buckets */
|
||||
#if defined(__INTEL_COMPILER)
|
||||
#pragma nounroll
|
||||
#endif
|
||||
for (size_t i=startID; i<endID; i++) {
|
||||
const Ty elt = src[i];
|
||||
#if defined(__X86_64__)
|
||||
const size_t index = ((size_t)(Key)src[i] >> (size_t)shift) & (size_t)mask;
|
||||
#else
|
||||
const size_t index = ((Key)src[i] >> shift) & mask;
|
||||
#endif
|
||||
dst[offset[index]++] = elt;
|
||||
}
|
||||
}
|
||||
|
||||
void tbbRadixIteration(const Key shift, const bool last,
|
||||
const Ty* __restrict src, Ty* __restrict dst,
|
||||
const size_t numTasks)
|
||||
{
|
||||
affinity_partitioner ap;
|
||||
parallel_for_affinity(numTasks,[&] (size_t taskIndex) { tbbRadixIteration0(shift,src,dst,taskIndex,numTasks); },ap);
|
||||
parallel_for_affinity(numTasks,[&] (size_t taskIndex) { tbbRadixIteration1(shift,src,dst,taskIndex,numTasks); },ap);
|
||||
}
|
||||
|
||||
void tbbRadixSort(const size_t numTasks)
|
||||
{
|
||||
radixCount = (TyRadixCount*) alignedMalloc(MAX_TASKS*sizeof(TyRadixCount));
|
||||
|
||||
if (sizeof(Key) == sizeof(uint32_t)) {
|
||||
tbbRadixIteration(0*BITS,0,src,tmp,numTasks);
|
||||
tbbRadixIteration(1*BITS,0,tmp,src,numTasks);
|
||||
tbbRadixIteration(2*BITS,0,src,tmp,numTasks);
|
||||
tbbRadixIteration(3*BITS,1,tmp,src,numTasks);
|
||||
}
|
||||
else if (sizeof(Key) == sizeof(uint64_t))
|
||||
{
|
||||
tbbRadixIteration(0*BITS,0,src,tmp,numTasks);
|
||||
tbbRadixIteration(1*BITS,0,tmp,src,numTasks);
|
||||
tbbRadixIteration(2*BITS,0,src,tmp,numTasks);
|
||||
tbbRadixIteration(3*BITS,0,tmp,src,numTasks);
|
||||
tbbRadixIteration(4*BITS,0,src,tmp,numTasks);
|
||||
tbbRadixIteration(5*BITS,0,tmp,src,numTasks);
|
||||
tbbRadixIteration(6*BITS,0,src,tmp,numTasks);
|
||||
tbbRadixIteration(7*BITS,1,tmp,src,numTasks);
|
||||
}
|
||||
alignedFree(radixCount);
|
||||
radixCount = nullptr;
|
||||
}
|
||||
|
||||
private:
|
||||
TyRadixCount* radixCount;
|
||||
Ty* const src;
|
||||
Ty* const tmp;
|
||||
const size_t N;
|
||||
};
|
||||
|
||||
template<typename Ty>
|
||||
void radix_sort(Ty* const src, Ty* const tmp, const size_t N, const size_t blockSize = 8192)
|
||||
{
|
||||
ParallelRadixSort<Ty,Ty>(src,tmp,N,blockSize);
|
||||
}
|
||||
|
||||
template<typename Ty, typename Key>
|
||||
void radix_sort(Ty* const src, Ty* const tmp, const size_t N, const size_t blockSize = 8192)
|
||||
{
|
||||
ParallelRadixSort<Ty,Key>(src,tmp,N,blockSize);
|
||||
}
|
||||
|
||||
template<typename Ty>
|
||||
void radix_sort_u32(Ty* const src, Ty* const tmp, const size_t N, const size_t blockSize = 8192) {
|
||||
radix_sort<Ty,uint32_t>(src,tmp,N,blockSize);
|
||||
}
|
||||
|
||||
template<typename Ty>
|
||||
void radix_sort_u64(Ty* const src, Ty* const tmp, const size_t N, const size_t blockSize = 8192) {
|
||||
radix_sort<Ty,uint64_t>(src,tmp,N,blockSize);
|
||||
}
|
||||
}
|
||||
@@ -1,5 +1,5 @@
|
||||
## ======================================================================== ##
|
||||
## Copyright 2009-2017 Intel Corporation ##
|
||||
## Copyright 2009-2014 Intel Corporation ##
|
||||
## ##
|
||||
## Licensed under the Apache License, Version 2.0 (the "License"); ##
|
||||
## you may not use this file except in compliance with the License. ##
|
||||
@@ -14,20 +14,23 @@
|
||||
## limitations under the License. ##
|
||||
## ======================================================================== ##
|
||||
|
||||
set(CTEST_PROJECT_NAME "Embree")
|
||||
set(CTEST_NIGHTLY_START_TIME "22:00:00 UTC")
|
||||
set(TEST_MODELS_HASH 01ada7ec1ace15ddba73aaab3f53ac6aac6a0309)
|
||||
FIND_PATH(COI_INCLUDE_PATH source/COIProcess_source.h /opt/intel/mic/coi/include /usr/include/intel-coi)
|
||||
FIND_PATH(COI_HOST_LIBRARY_DIR libcoi_host.so /opt/intel/mic/coi/host-linux-release/lib/)
|
||||
FIND_PATH(COI_DEV_LIBRARY_DIR libcoi_device.so /opt/mpss/*/sysroots/k1om-mpss-linux/usr/lib64/)
|
||||
|
||||
IF (NOT CTEST_DROP_SITE)
|
||||
set(CTEST_DROP_METHOD "http")
|
||||
set(CTEST_DROP_SITE "cdash")
|
||||
set(CTEST_DROP_LOCATION "/CDash/submit.php?project=Embree")
|
||||
set(CTEST_DROP_SITE_CDASH TRUE)
|
||||
endif()
|
||||
IF (COI_INCLUDE_PATH AND COI_HOST_LIBRARY_DIR AND COI_DEV_LIBRARY_DIR)
|
||||
SET(COI_FOUND TRUE)
|
||||
SET(COI_INCLUDE_PATHS ${COI_INCLUDE_PATH})
|
||||
SET(COI_HOST_LIBRARIES ${COI_HOST_LIBRARY_DIR}/libcoi_host.so)
|
||||
SET(COI_DEV_LIBRARIES ${COI_DEV_LIBRARY_DIR}/libcoi_device.so)
|
||||
ELSE ()
|
||||
MESSAGE(FATAL_ERROR "COI installation not found")
|
||||
ENDIF ()
|
||||
|
||||
list (APPEND CTEST_CUSTOM_WARNING_EXCEPTION "warning #1478") # deprecated function used
|
||||
#list (APPEND CTEST_CUSTOM_WARNING_EXCEPTION "warning #10237") # -lcilkrts linked in dynamically, static library not available")
|
||||
list (APPEND CTEST_CUSTOM_WARNING_EXCEPTION "-Wextern-initializer")
|
||||
MARK_AS_ADVANCED(COI_INCLUDE_PATH)
|
||||
MARK_AS_ADVANCED(COI_HOST_LIBRARY_DIR)
|
||||
MARK_AS_ADVANCED(COI_DEV_LIBRARY_DIR)
|
||||
|
||||
SET(CTEST_CUSTOM_MAXIMUM_PASSED_TEST_OUTPUT_SIZE 200000)
|
||||
SET(CTEST_CUSTOM_MAXIMUM_FAILED_TEST_OUTPUT_SIZE 800000)
|
||||
MARK_AS_ADVANCED(COI_INCLUDE_PATHS)
|
||||
MARK_AS_ADVANCED(COI_HOST_LIBRARIES)
|
||||
MARK_AS_ADVANCED(COI_DEV_LIBRARIES)
|
||||
@@ -1,24 +0,0 @@
|
||||
## ======================================================================== ##
|
||||
## Copyright 2009-2017 Intel Corporation ##
|
||||
## ##
|
||||
## Licensed under the Apache License, Version 2.0 (the "License"); ##
|
||||
## you may not use this file except in compliance with the License. ##
|
||||
## You may obtain a copy of the License at ##
|
||||
## ##
|
||||
## http://www.apache.org/licenses/LICENSE-2.0 ##
|
||||
## ##
|
||||
## Unless required by applicable law or agreed to in writing, software ##
|
||||
## distributed under the License is distributed on an "AS IS" BASIS, ##
|
||||
## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ##
|
||||
## See the License for the specific language governing permissions and ##
|
||||
## limitations under the License. ##
|
||||
## ======================================================================== ##
|
||||
|
||||
FIND_PATH( PNG_INCLUDE_DIR NAMES png.h )
|
||||
FIND_LIBRARY( PNG_LIBRARIES NAMES png )
|
||||
|
||||
INCLUDE(FindPackageHandleStandardArgs)
|
||||
FIND_PACKAGE_HANDLE_STANDARD_ARGS(PNG DEFAULT_MSG PNG_INCLUDE_DIR PNG_LIBRARIES)
|
||||
|
||||
MARK_AS_ADVANCED(PNG_INCLUDE_DIR)
|
||||
MARK_AS_ADVANCED(PNG_LIBRARIES)
|
||||
@@ -1,128 +0,0 @@
|
||||
## ======================================================================== ##
|
||||
## Copyright 2009-2017 Intel Corporation ##
|
||||
## ##
|
||||
## Licensed under the Apache License, Version 2.0 (the "License"); ##
|
||||
## you may not use this file except in compliance with the License. ##
|
||||
## You may obtain a copy of the License at ##
|
||||
## ##
|
||||
## http://www.apache.org/licenses/LICENSE-2.0 ##
|
||||
## ##
|
||||
## Unless required by applicable law or agreed to in writing, software ##
|
||||
## distributed under the License is distributed on an "AS IS" BASIS, ##
|
||||
## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ##
|
||||
## See the License for the specific language governing permissions and ##
|
||||
## limitations under the License. ##
|
||||
## ======================================================================== ##
|
||||
|
||||
IF (WIN32)
|
||||
FIND_PATH(EMBREE_TBB_ROOT include/tbb/tbb.h
|
||||
DOC "Root of TBB installation"
|
||||
PATHS ${PROJECT_SOURCE_DIR}/tbb "C:/Program Files (x86)/Intel/Composer XE/tbb"
|
||||
NO_DEFAULT_PATH
|
||||
)
|
||||
FIND_PATH(EMBREE_TBB_ROOT include/tbb/tbb.h)
|
||||
|
||||
IF (CMAKE_SIZEOF_VOID_P EQUAL 8)
|
||||
SET(TBB_ARCH intel64)
|
||||
ELSE()
|
||||
SET(TBB_ARCH ia32)
|
||||
ENDIF()
|
||||
|
||||
IF (MSVC10)
|
||||
SET(TBB_VCVER vc10)
|
||||
ELSEIF (MSVC11)
|
||||
SET(TBB_VCVER vc11)
|
||||
ELSE()
|
||||
SET(TBB_VCVER vc12)
|
||||
ENDIF()
|
||||
|
||||
SET(TBB_LIBDIR ${EMBREE_TBB_ROOT}/lib/${TBB_ARCH}/${TBB_VCVER})
|
||||
SET(TBB_BINDIR ${EMBREE_TBB_ROOT}/bin/${TBB_ARCH}/${TBB_VCVER})
|
||||
|
||||
IF (EMBREE_TBB_ROOT STREQUAL "")
|
||||
FIND_PATH(TBB_INCLUDE_DIR tbb/task_scheduler_init.h)
|
||||
FIND_LIBRARY(TBB_LIBRARY tbb)
|
||||
FIND_LIBRARY(TBB_LIBRARY_MALLOC tbbmalloc)
|
||||
ELSE()
|
||||
SET(TBB_INCLUDE_DIR TBB_INCLUDE_DIR-NOTFOUND)
|
||||
SET(TBB_LIBRARY TBB_LIBRARY-NOTFOUND)
|
||||
SET(TBB_LIBRARY_MALLOC TBB_LIBRARY_MALLOC-NOTFOUND)
|
||||
FIND_PATH(TBB_INCLUDE_DIR tbb/task_scheduler_init.h PATHS ${EMBREE_TBB_ROOT}/include NO_DEFAULT_PATH)
|
||||
FIND_LIBRARY(TBB_LIBRARY tbb PATHS ${TBB_LIBDIR} NO_DEFAULT_PATH)
|
||||
FIND_LIBRARY(TBB_LIBRARY_MALLOC tbbmalloc PATHS ${TBB_LIBDIR} NO_DEFAULT_PATH)
|
||||
ENDIF()
|
||||
|
||||
ELSE ()
|
||||
|
||||
FIND_PATH(EMBREE_TBB_ROOT include/tbb/tbb.h
|
||||
DOC "Root of TBB installation"
|
||||
PATHS ${PROJECT_SOURCE_DIR}/tbb /opt/intel/tbb
|
||||
NO_DEFAULT_PATH
|
||||
)
|
||||
FIND_PATH(EMBREE_TBB_ROOT include/tbb/tbb.h)
|
||||
|
||||
IF (EMBREE_TBB_ROOT STREQUAL "")
|
||||
FIND_PATH(TBB_INCLUDE_DIR tbb/task_scheduler_init.h)
|
||||
FIND_LIBRARY(TBB_LIBRARY tbb)
|
||||
FIND_LIBRARY(TBB_LIBRARY_MALLOC tbbmalloc)
|
||||
|
||||
ELSEIF (EXISTS ${EMBREE_TBB_ROOT}/src)
|
||||
OPTION(EMBREE_TBB_STATIC_LIB "Build TBB as a static library (building TBB as a static library is NOT recommended)")
|
||||
if (EMBREE_TBB_STATIC_LIB)
|
||||
include(${EMBREE_TBB_ROOT}/cmake/TBBBuild.cmake)
|
||||
tbb_build(TBB_ROOT ${EMBREE_TBB_ROOT} CONFIG_DIR TBB_DIR MAKE_ARGS extra_inc=big_iron.inc)
|
||||
SET(TBB_INCLUDE_DIR ${EMBREE_TBB_ROOT}/include)
|
||||
SET(TBB_LIBRARY ${PROJECT_BINARY_DIR}/tbb_cmake_build/tbb_cmake_build_subdir_release/libtbb.a)
|
||||
SET(TBB_LIBRARY_MALLOC ${PROJECT_BINARY_DIR}/tbb_cmake_build/tbb_cmake_build_subdir_release/libtbbmalloc.a)
|
||||
else()
|
||||
include(${EMBREE_TBB_ROOT}/cmake/TBBBuild.cmake)
|
||||
tbb_build(TBB_ROOT ${EMBREE_TBB_ROOT} CONFIG_DIR TBB_DIR)
|
||||
SET(TBB_INCLUDE_DIR ${EMBREE_TBB_ROOT}/include)
|
||||
SET(TBB_LIBRARY ${PROJECT_BINARY_DIR}/tbb_cmake_build/tbb_cmake_build_subdir_release/libtbb.so.2)
|
||||
SET(TBB_LIBRARY_MALLOC ${PROJECT_BINARY_DIR}/tbb_cmake_build/tbb_cmake_build_subdir_release/libtbbmalloc.so.2)
|
||||
endif()
|
||||
|
||||
ELSE()
|
||||
SET(TBB_INCLUDE_DIR TBB_INCLUDE_DIR-NOTFOUND)
|
||||
SET(TBB_LIBRARY TBB_LIBRARY-NOTFOUND)
|
||||
SET(TBB_LIBRARY_MALLOC TBB_LIBRARY_MALLOC-NOTFOUND)
|
||||
IF (APPLE)
|
||||
FIND_PATH(TBB_INCLUDE_DIR tbb/task_scheduler_init.h PATHS ${EMBREE_TBB_ROOT}/include NO_DEFAULT_PATH)
|
||||
FIND_LIBRARY(TBB_LIBRARY tbb PATHS ${EMBREE_TBB_ROOT}/lib NO_DEFAULT_PATH)
|
||||
FIND_LIBRARY(TBB_LIBRARY_MALLOC tbbmalloc PATHS ${EMBREE_TBB_ROOT}/lib NO_DEFAULT_PATH)
|
||||
ELSE()
|
||||
FIND_PATH(TBB_INCLUDE_DIR tbb/task_scheduler_init.h PATHS ${EMBREE_TBB_ROOT}/include NO_DEFAULT_PATH)
|
||||
FIND_LIBRARY(TBB_LIBRARY tbb PATHS ${EMBREE_TBB_ROOT}/lib/intel64/gcc4.4 ${EMBREE_TBB_ROOT}/lib ${EMBREE_TBB_ROOT}/lib64 /usr/libx86_64-linux-gnu/ NO_DEFAULT_PATH)
|
||||
FIND_LIBRARY(TBB_LIBRARY_MALLOC tbbmalloc PATHS ${EMBREE_TBB_ROOT}/lib/intel64/gcc4.4 ${EMBREE_TBB_ROOT}/lib ${EMBREE_TBB_ROOT}/lib64 /usr/libx86_64-linux-gnu/ NO_DEFAULT_PATH)
|
||||
ENDIF()
|
||||
ENDIF()
|
||||
|
||||
ENDIF()
|
||||
|
||||
INCLUDE(FindPackageHandleStandardArgs)
|
||||
FIND_PACKAGE_HANDLE_STANDARD_ARGS(TBB DEFAULT_MSG TBB_INCLUDE_DIR TBB_LIBRARY TBB_LIBRARY_MALLOC)
|
||||
|
||||
IF (TBB_FOUND)
|
||||
SET(TBB_INCLUDE_DIRS ${TBB_INCLUDE_DIR})
|
||||
SET(TBB_LIBRARIES ${TBB_LIBRARY} ${TBB_LIBRARY_MALLOC})
|
||||
ENDIF()
|
||||
|
||||
MARK_AS_ADVANCED(TBB_INCLUDE_DIR)
|
||||
MARK_AS_ADVANCED(TBB_LIBRARY)
|
||||
MARK_AS_ADVANCED(TBB_LIBRARY_MALLOC)
|
||||
|
||||
##############################################################
|
||||
# Install TBB
|
||||
##############################################################
|
||||
|
||||
IF (WIN32)
|
||||
INSTALL(PROGRAMS ${TBB_BINDIR}/tbb.dll ${TBB_BINDIR}/tbbmalloc.dll DESTINATION ${CMAKE_INSTALL_BINDIR} COMPONENT examples)
|
||||
INSTALL(PROGRAMS ${TBB_BINDIR}/tbb.dll ${TBB_BINDIR}/tbbmalloc.dll DESTINATION ${CMAKE_INSTALL_LIBDIR} COMPONENT lib)
|
||||
INSTALL(PROGRAMS ${TBB_LIBDIR}/tbb.lib ${TBB_LIBDIR}/tbbmalloc.lib DESTINATION ${CMAKE_INSTALL_LIBDIR} COMPONENT lib)
|
||||
ELSEIF (EMBREE_ZIP_MODE)
|
||||
IF (APPLE)
|
||||
INSTALL(PROGRAMS ${EMBREE_TBB_ROOT}/lib/libtbb.dylib ${EMBREE_TBB_ROOT}/lib/libtbbmalloc.dylib DESTINATION ${CMAKE_INSTALL_LIBDIR} COMPONENT lib)
|
||||
ELSE()
|
||||
INSTALL(PROGRAMS ${EMBREE_TBB_ROOT}/lib/intel64/gcc4.4/libtbb.so.2 ${EMBREE_TBB_ROOT}/lib/intel64/gcc4.4/libtbbmalloc.so.2 DESTINATION ${CMAKE_INSTALL_LIBDIR} COMPONENT lib)
|
||||
ENDIF()
|
||||
ENDIF()
|
||||
@@ -1,5 +1,5 @@
|
||||
## ======================================================================== ##
|
||||
## Copyright 2009-2017 Intel Corporation ##
|
||||
## Copyright 2009-2014 Intel Corporation ##
|
||||
## ##
|
||||
## Licensed under the Apache License, Version 2.0 (the "License"); ##
|
||||
## you may not use this file except in compliance with the License. ##
|
||||
@@ -14,19 +14,21 @@
|
||||
## limitations under the License. ##
|
||||
## ======================================================================== ##
|
||||
|
||||
ADD_LIBRARY(algorithms OBJECT
|
||||
parallel_for.cpp
|
||||
parallel_reduce.cpp
|
||||
parallel_prefix_sum.cpp
|
||||
parallel_for_for.cpp
|
||||
parallel_for_for_prefix_sum.cpp
|
||||
parallel_partition.cpp
|
||||
parallel_sort.cpp
|
||||
parallel_set.cpp
|
||||
parallel_map.cpp
|
||||
parallel_filter.cpp
|
||||
)
|
||||
FIND_PATH(EMBREE_INCLUDE_PATH embree2/rtcore.h
|
||||
/usr/include
|
||||
/usr/local/include
|
||||
/opt/local/include)
|
||||
|
||||
SET_PROPERTY(TARGET algorithms PROPERTY FOLDER common)
|
||||
FIND_LIBRARY(EMBREE_LIBRARY NAMES embree PATHS
|
||||
/usr/lib
|
||||
/usr/local/lib
|
||||
/opt/local/lib)
|
||||
|
||||
|
||||
FIND_LIBRARY(EMBREE_LIBRARY_MIC NAMES embree_xeonphi PATHS
|
||||
/usr/lib
|
||||
/usr/local/lib
|
||||
/opt/local/lib)
|
||||
|
||||
IF (EMBREE_INCLUDE_PATH AND EMBREE_LIBRARY)
|
||||
SET(EMBREE_FOUND TRUE)
|
||||
ENDIF ()
|
||||
@@ -1,31 +0,0 @@
|
||||
## ======================================================================== ##
|
||||
## Copyright 2009-2017 Intel Corporation ##
|
||||
## ##
|
||||
## Licensed under the Apache License, Version 2.0 (the "License"); ##
|
||||
## you may not use this file except in compliance with the License. ##
|
||||
## You may obtain a copy of the License at ##
|
||||
## ##
|
||||
## http://www.apache.org/licenses/LICENSE-2.0 ##
|
||||
## ##
|
||||
## Unless required by applicable law or agreed to in writing, software ##
|
||||
## distributed under the License is distributed on an "AS IS" BASIS, ##
|
||||
## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ##
|
||||
## See the License for the specific language governing permissions and ##
|
||||
## limitations under the License. ##
|
||||
## ======================================================================== ##
|
||||
|
||||
IF (WIN32 OR APPLE)
|
||||
return()
|
||||
ENDIF()
|
||||
|
||||
execute_process(COMMAND objdump -C -t ${file} OUTPUT_VARIABLE output)
|
||||
string(REPLACE "\n" ";" output ${output})
|
||||
|
||||
foreach (line ${output})
|
||||
if ("${line}" MATCHES "O .bss")
|
||||
if (NOT "${line}" MATCHES "std::__ioinit" AND # this is caused by iostream initialization and is likely also ok
|
||||
NOT "${line}" MATCHES "\\(\\)::") # this matches a static inside a function which is fine
|
||||
message(WARNING "\nProblematic global variable in non-SSE code:\n" ${line})
|
||||
endif()
|
||||
endif()
|
||||
endforeach()
|
||||
@@ -1,40 +0,0 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2017 Kitware, Inc. //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#if \
|
||||
defined(__AVX512F__) && defined(__AVX512CD__) && \
|
||||
defined(__AVX512DQ__) && defined(__AVX512BW__) && defined(__AVX512VL__)
|
||||
char const *info_isa = "ISA" ":" "AVX512SKX";
|
||||
#elif \
|
||||
defined(__AVX512F__) && defined(__AVX512CD__) && \
|
||||
defined(__AVX512ER__) && defined(__AVX512PF__)
|
||||
char const *info_isa = "ISA" ":" "AVX512KNL";
|
||||
#elif defined(__AVX2__)
|
||||
char const *info_isa = "ISA" ":" "AVX2";
|
||||
#elif defined(__AVX__)
|
||||
char const *info_isa = "ISA" ":" "AVX";
|
||||
#elif defined(__SSE4_2__)
|
||||
char const *info_isa = "ISA" ":" "SSE42";
|
||||
#else // defined(__SSE2__)
|
||||
char const *info_isa = "ISA" ":" "SSE2";
|
||||
#endif
|
||||
|
||||
int main(int argc, char **argv)
|
||||
{
|
||||
int require = 0;
|
||||
require += info_isa[argc];
|
||||
return require;
|
||||
}
|
||||
@@ -1,33 +0,0 @@
|
||||
## ======================================================================== ##
|
||||
## Copyright 2017 Kitware, Inc. ##
|
||||
## ##
|
||||
## Licensed under the Apache License, Version 2.0 (the "License"); ##
|
||||
## you may not use this file except in compliance with the License. ##
|
||||
## You may obtain a copy of the License at ##
|
||||
## ##
|
||||
## http://www.apache.org/licenses/LICENSE-2.0 ##
|
||||
## ##
|
||||
## Unless required by applicable law or agreed to in writing, software ##
|
||||
## distributed under the License is distributed on an "AS IS" BASIS, ##
|
||||
## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ##
|
||||
## See the License for the specific language governing permissions and ##
|
||||
## limitations under the License. ##
|
||||
## ======================================================================== ##
|
||||
|
||||
SET(CHECK_ISA_DIR ${CMAKE_CURRENT_LIST_DIR})
|
||||
FUNCTION(CHECK_ISA_DEFAULT OUTVAR)
|
||||
SET(ISA_DEFAULT_BIN ${CMAKE_BINARY_DIR}${CMAKE_FILES_DIRECTORY}/check_isa_default.bin)
|
||||
SET(SRC ${CHECK_ISA_DIR}/check_isa.cpp)
|
||||
TRY_COMPILE(ISA_DEFAULT_COMPILE
|
||||
${CMAKE_BINARY_DIR}
|
||||
${SRC}
|
||||
COPY_FILE ${ISA_DEFAULT_BIN}
|
||||
)
|
||||
IF(NOT ISA_DEFAULT_COMPILE)
|
||||
SET(ISA_DEFAULT "SSE2")
|
||||
RETURN()
|
||||
ENDIF()
|
||||
FILE(STRINGS ${ISA_DEFAULT_BIN} ISA_DEFAULT REGEX "^ISA:")
|
||||
STRING(REPLACE "ISA:" "" ISA_DEFAULT "${ISA_DEFAULT}")
|
||||
SET(${OUTVAR} ${ISA_DEFAULT} PARENT_SCOPE)
|
||||
ENDFUNCTION()
|
||||
@@ -1,37 +0,0 @@
|
||||
## ======================================================================== ##
|
||||
## Copyright 2009-2017 Intel Corporation ##
|
||||
## ##
|
||||
## Licensed under the Apache License, Version 2.0 (the "License"); ##
|
||||
## you may not use this file except in compliance with the License. ##
|
||||
## You may obtain a copy of the License at ##
|
||||
## ##
|
||||
## http://www.apache.org/licenses/LICENSE-2.0 ##
|
||||
## ##
|
||||
## Unless required by applicable law or agreed to in writing, software ##
|
||||
## distributed under the License is distributed on an "AS IS" BASIS, ##
|
||||
## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ##
|
||||
## See the License for the specific language governing permissions and ##
|
||||
## limitations under the License. ##
|
||||
## ======================================================================== ##
|
||||
|
||||
IF (WIN32 OR APPLE)
|
||||
return()
|
||||
ENDIF()
|
||||
|
||||
execute_process(COMMAND objdump -d ${file} OUTPUT_VARIABLE output)
|
||||
string(REPLACE "\n" ";" output ${output})
|
||||
|
||||
SET(lastfunc "")
|
||||
foreach (line ${output})
|
||||
if ("${line}" MATCHES "^.*<([^>]*)>:$")
|
||||
SET(lastfunc ${CMAKE_MATCH_1})
|
||||
endif()
|
||||
if ("${line}" MATCHES ".*sub[ ]+[$]([^,]*),%rsp.*")
|
||||
set(bytes ${CMAKE_MATCH_1})
|
||||
if ("${bytes}" GREATER "4096")
|
||||
if ("${lastfunc}" MATCHES ".*recurse.*")
|
||||
message(WARNING "Large stack space requirement: ${lastfunc} size: ${bytes}")
|
||||
endif()
|
||||
endif()
|
||||
endif()
|
||||
endforeach()
|
||||
+15
-96
@@ -1,5 +1,5 @@
|
||||
## ======================================================================== ##
|
||||
## Copyright 2009-2017 Intel Corporation ##
|
||||
## Copyright 2009-2014 Intel Corporation ##
|
||||
## ##
|
||||
## Licensed under the Apache License, Version 2.0 (the "License"); ##
|
||||
## you may not use this file except in compliance with the License. ##
|
||||
@@ -20,101 +20,20 @@ SET(FLAGS_SSSE3 "-mssse3")
|
||||
SET(FLAGS_SSE41 "-msse4.1")
|
||||
SET(FLAGS_SSE42 "-msse4.2")
|
||||
SET(FLAGS_AVX "-mavx")
|
||||
SET(FLAGS_AVX2 "-mf16c -mavx2 -mfma -mlzcnt -mbmi -mbmi2")
|
||||
SET(FLAGS_AVX512KNL "-march=knl")
|
||||
SET(FLAGS_AVX512SKX "-march=skx")
|
||||
SET(FLAGS_AVX2 "-mavx2 -mfma -mlzcnt -mbmi -mbmi2")
|
||||
|
||||
IF (WIN32)
|
||||
|
||||
SET(COMMON_CXX_FLAGS "")
|
||||
SET(COMMON_CXX_FLAGS "${COMMON_CXX_FLAGS} /EHsc") # catch C++ exceptions only and extern "C" functions never throw a C++ exception
|
||||
# SET(COMMON_CXX_FLAGS "${COMMON_CXX_FLAGS} /MP") # compile source files in parallel
|
||||
SET(COMMON_CXX_FLAGS "${COMMON_CXX_FLAGS} /GR") # enable runtime type information (on by default)
|
||||
SET(COMMON_CXX_FLAGS "${COMMON_CXX_FLAGS} -Xclang -fcxx-exceptions") # enable C++ exceptions in Clang
|
||||
IF (EMBREE_STACK_PROTECTOR)
|
||||
SET(COMMON_CXX_FLAGS "${COMMON_CXX_FLAGS} /GS") # protects against return address overrides
|
||||
ELSE()
|
||||
SET(COMMON_CXX_FLAGS "${COMMON_CXX_FLAGS} /GS-") # do not protect against return address overrides
|
||||
ENDIF()
|
||||
|
||||
SET(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} ${COMMON_CXX_FLAGS}")
|
||||
SET(CMAKE_EXE_LINKER_FLAGS_DEBUG "${CMAKE_EXE_LINKER_FLAGS_DEBUG} /DEBUG") # generate debug information
|
||||
SET(CMAKE_SHARED_LINKER_FLAGS_DEBUG "${CMAKE_SHARED_LINKER_FLAGS_DEBUG} /DEBUG") # generate debug information
|
||||
|
||||
SET(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} ${COMMON_CXX_FLAGS}")
|
||||
SET(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} /Ox") # enable full optimizations
|
||||
SET(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} /Oi") # inline intrinsic functions
|
||||
SET(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} /Gy") # package individual functions
|
||||
SET(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -fno-vectorize") # disable auto vectorization
|
||||
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} ${COMMON_CXX_FLAGS}")
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} /Ox") # enable full optimizations
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} /Oi") # inline intrinsic functions
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} /Gy") # package individual functions
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} -fno-vectorize") # disable auto vectorization
|
||||
SET(CMAKE_EXE_LINKER_FLAGS_RELWITHDEBINFO "${CMAKE_EXE_LINKER_FLAGS_RELWITHDEBINFO} /DEBUG") # generate debug information
|
||||
SET(CMAKE_SHARED_LINKER_FLAGS_RELWITHDEBINFO "${CMAKE_SHARED_LINKER_FLAGS_RELWITHDEBINFO} /DEBUG") # generate debug information
|
||||
|
||||
SET(SECURE_LINKER_FLAGS "")
|
||||
SET(SECURE_LINKER_FLAGS "${SECURE_LINKER_FLAGS} /NXCompat") # compatible with data execution prevention (on by default)
|
||||
SET(SECURE_LINKER_FLAGS "${SECURE_LINKER_FLAGS} /DynamicBase") # random rebase of executable at load time
|
||||
IF (CMAKE_SIZEOF_VOID_P EQUAL 4)
|
||||
SET(SECURE_LINKER_FLAGS "${SECURE_LINKER_FLAGS} /SafeSEH") # invoke known exception handlers (Win32 only, x64 exception handlers are safe by design)
|
||||
ENDIF()
|
||||
SET(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${SECURE_LINKER_FLAGS}")
|
||||
SET(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} ${SECURE_LINKER_FLAGS}")
|
||||
|
||||
INCLUDE(msvc_post)
|
||||
|
||||
ELSE()
|
||||
|
||||
OPTION(EMBREE_IGNORE_CMAKE_CXX_FLAGS "When enabled Embree ignores default CMAKE_CXX_FLAGS." ON)
|
||||
IF (EMBREE_IGNORE_CMAKE_CXX_FLAGS)
|
||||
SET(CMAKE_CXX_FLAGS "")
|
||||
ENDIF()
|
||||
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wall") # enables most warnings
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wformat -Wformat-security") # enables string format vulnerability warnings
|
||||
IF (NOT APPLE)
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fPIE") # enables support for more secure position independent execution
|
||||
ENDIF()
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fPIC") # generate position independent code suitable for shared libraries
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11") # enables C++11 features
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fvisibility=hidden") # makes all symbols hidden by default
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fvisibility-inlines-hidden") # makes all inline symbols hidden by default
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fno-strict-aliasing") # disables strict aliasing rules
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -D_FORTIFY_SOURCE=2") # perform extra security checks for some standard library calls
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fno-tree-vectorize") # disable auto vectorizer
|
||||
IF (EMBREE_STACK_PROTECTOR)
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fstack-protector") # protects against return address overrides
|
||||
ENDIF()
|
||||
|
||||
SET(CMAKE_CXX_FLAGS_DEBUG "")
|
||||
SET(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -DDEBUG") # enable assertions
|
||||
SET(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -DTBB_USE_DEBUG") # configure TBB in debug mode
|
||||
SET(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -g") # generate debug information
|
||||
SET(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -O0") # disable optimizations
|
||||
|
||||
SET(CMAKE_CXX_FLAGS_RELEASE "")
|
||||
SET(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -DNDEBUG") # disable assertions
|
||||
SET(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -O3") # enable full optimizations
|
||||
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "")
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} -DDEBUG") # enable assertions
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} -DTBB_USE_DEBUG") # configure TBB in debug mode
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} -g") # generate debug information
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} -O3") # enable full optimizations
|
||||
|
||||
IF (APPLE)
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -mmacosx-version-min=10.7") # makes sure code runs on older MacOSX versions
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -stdlib=libc++") # link against libc++ which supports C++11 features
|
||||
ELSE(APPLE)
|
||||
SET(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl,--no-undefined") # issues link error for undefined symbols in shared library
|
||||
SET(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -z noexecstack") # we do not need an executable stack
|
||||
SET(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -z relro -z now") # re-arranges data sections to increase security
|
||||
SET(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -z noexecstack") # we do not need an executable stack
|
||||
SET(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -z relro -z now") # re-arranges data sections to increase security
|
||||
SET(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -pie") # enables position independent execution for executable
|
||||
ENDIF(APPLE)
|
||||
SET(CMAKE_CXX_COMPILER "clang++")
|
||||
SET(CMAKE_C_COMPILER "clang")
|
||||
SET(CMAKE_CXX_FLAGS "-fPIC")
|
||||
SET(CMAKE_CXX_FLAGS_DEBUG "-DDEBUG -g -O2")
|
||||
SET(CMAKE_CXX_FLAGS_RELEASE "-DNDEBUG -O3 -Wstrict-aliasing=0 ")
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "-DNDEBUG -g -O3 -Wstrict-aliasing=0 ")
|
||||
SET(CMAKE_EXE_LINKER_FLAGS "")
|
||||
|
||||
IF (NOT RTCORE_EXPORT_ALL_SYMBOLS)
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fvisibility-inlines-hidden -fvisibility=hidden")
|
||||
ENDIF()
|
||||
|
||||
IF (APPLE)
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -mmacosx-version-min=10.7")
|
||||
ENDIF (APPLE)
|
||||
|
||||
@@ -1,22 +0,0 @@
|
||||
## ======================================================================== ##
|
||||
## Copyright 2017 Kitware, Inc. ##
|
||||
## ##
|
||||
## Licensed under the Apache License, Version 2.0 (the "License"); ##
|
||||
## you may not use this file except in compliance with the License. ##
|
||||
## You may obtain a copy of the License at ##
|
||||
## ##
|
||||
## http://www.apache.org/licenses/LICENSE-2.0 ##
|
||||
## ##
|
||||
## Unless required by applicable law or agreed to in writing, software ##
|
||||
## distributed under the License is distributed on an "AS IS" BASIS, ##
|
||||
## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ##
|
||||
## See the License for the specific language governing permissions and ##
|
||||
## limitations under the License. ##
|
||||
## ======================================================================== ##
|
||||
|
||||
SET(FLAGS_SSE2 "-target-cpu=xeon")
|
||||
SET(FLAGS_SSE42 "-target-cpu=interlagos")
|
||||
SET(FLAGS_AVX "-target-cpu=sandybridge")
|
||||
SET(FLAGS_AVX2 "-target-cpu=haswell")
|
||||
SET(FLAGS_AVX512KNL "-target-cpu=mic-knl")
|
||||
SET(FLAGS_AVX512SKX "-target-cpu=x86-skylake")
|
||||
@@ -1,5 +1,5 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
@@ -14,4 +14,5 @@
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#define RTCORE_HASH "@EMBREE_HASH@"
|
||||
int main() { return 0; }
|
||||
|
||||
@@ -1,23 +0,0 @@
|
||||
## ======================================================================== ##
|
||||
## Copyright 2009-2017 Intel Corporation ##
|
||||
## ##
|
||||
## Licensed under the Apache License, Version 2.0 (the "License"); ##
|
||||
## you may not use this file except in compliance with the License. ##
|
||||
## You may obtain a copy of the License at ##
|
||||
## ##
|
||||
## http://www.apache.org/licenses/LICENSE-2.0 ##
|
||||
## ##
|
||||
## Unless required by applicable law or agreed to in writing, software ##
|
||||
## distributed under the License is distributed on an "AS IS" BASIS, ##
|
||||
## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ##
|
||||
## See the License for the specific language governing permissions and ##
|
||||
## limitations under the License. ##
|
||||
## ======================================================================== ##
|
||||
|
||||
# use default install config
|
||||
INCLUDE(${CMAKE_CURRENT_LIST_DIR}/embree-config-install.cmake)
|
||||
|
||||
# and override path variables to match for build directory
|
||||
SET(EMBREE_INCLUDE_DIRS @PROJECT_SOURCE_DIR@/include)
|
||||
SET(EMBREE_LIBRARY @PROJECT_BINARY_DIR@/@EMBREE_LIBRARY_NAME@)
|
||||
SET(EMBREE_LIBRARIES ${EMBREE_LIBRARY})
|
||||
@@ -1,59 +0,0 @@
|
||||
## ======================================================================== ##
|
||||
## Copyright 2009-2017 Intel Corporation ##
|
||||
## ##
|
||||
## Licensed under the Apache License, Version 2.0 (the "License"); ##
|
||||
## you may not use this file except in compliance with the License. ##
|
||||
## You may obtain a copy of the License at ##
|
||||
## ##
|
||||
## http://www.apache.org/licenses/LICENSE-2.0 ##
|
||||
## ##
|
||||
## Unless required by applicable law or agreed to in writing, software ##
|
||||
## distributed under the License is distributed on an "AS IS" BASIS, ##
|
||||
## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ##
|
||||
## See the License for the specific language governing permissions and ##
|
||||
## limitations under the License. ##
|
||||
## ======================================================================== ##
|
||||
|
||||
SET(EMBREE_ROOT_DIR ${CMAKE_CURRENT_LIST_DIR}/@EMBREE_RELATIV_ROOT_DIR@)
|
||||
GET_FILENAME_COMPONENT(EMBREE_ROOT_DIR "${EMBREE_ROOT_DIR}" ABSOLUTE)
|
||||
|
||||
SET(EMBREE_INCLUDE_DIRS ${EMBREE_ROOT_DIR}/@CMAKE_INSTALL_INCLUDEDIR@)
|
||||
SET(EMBREE_LIBRARY ${EMBREE_ROOT_DIR}/@CMAKE_INSTALL_LIBDIR@/@EMBREE_LIBRARY_NAME@)
|
||||
SET(EMBREE_LIBRARIES ${EMBREE_LIBRARY})
|
||||
|
||||
MARK_AS_ADVANCED(embree_DIR)
|
||||
|
||||
SET(EMBREE_VERSION @EMBREE_VERSION@)
|
||||
SET(EMBREE_VERSION_MAJOR @EMBREE_VERSION_MAJOR@)
|
||||
SET(EMBREE_VERSION_MINOR @EMBREE_VERSION_MINOR@)
|
||||
SET(EMBREE_VERSION_PATCH @EMBREE_VERSION_PATCH@)
|
||||
SET(EMBREE_VERSION_NOTE "@EMBREE_VERSION_NOTE@")
|
||||
|
||||
SET(EMBREE_MAX_ISA @EMBREE_MAX_ISA@)
|
||||
SET(EMBREE_ISA_SSE2 @EMBREE_ISA_SSE2@)
|
||||
SET(EMBREE_ISA_SSE42 @EMBREE_ISA_SSE42@)
|
||||
SET(EMBREE_ISA_AVX @EMBREE_ISA_AVX@)
|
||||
SET(EMBREE_ISA_AVX2 @EMBREE_ISA_AVX2@)
|
||||
SET(EMBREE_ISA_AVX512KNL @EMBREE_ISA_AVX512KNL@)
|
||||
SET(EMBREE_ISA_AVX512SKX @EMBREE_ISA_AVX512SKX@)
|
||||
|
||||
SET(EMBREE_BUILD_TYPE @CMAKE_BUILD_TYPE@)
|
||||
SET(EMBREE_ISPC_SUPPORT @EMBREE_ISPC_SUPPORT@)
|
||||
SET(EMBREE_STATIC_LIB @EMBREE_STATIC_LIB@)
|
||||
SET(EMBREE_TUTORIALS @EMBREE_TUTORIALS@)
|
||||
|
||||
SET(EMBREE_RAY_MASK @EMBREE_RAY_MASK@)
|
||||
SET(EMBREE_STAT_COUNTERS @EMBREE_STAT_COUNTERS@)
|
||||
SET(EMBREE_BACKFACE_CULLING @EMBREE_BACKFACE_CULLING@)
|
||||
SET(EMBREE_INTERSECTION_FILTER @EMBREE_INTERSECTION_FILTER@)
|
||||
SET(EMBREE_INTERSECTION_FILTER_RESTORE @EMBREE_INTERSECTION_FILTER_RESTORE@)
|
||||
SET(EMBREE_IGNORE_INVALID_RAYS @EMBREE_IGNORE_INVALID_RAYS@)
|
||||
SET(EMBREE_TASKING_SYSTEM @EMBREE_TASKING_SYSTEM@)
|
||||
|
||||
SET(EMBREE_GEOMETRY_TRIANGLES @EMBREE_GEOMETRY_TRIANGLES@)
|
||||
SET(EMBREE_GEOMETRY_QUADS @EMBREE_GEOMETRY_QUADS@)
|
||||
SET(EMBREE_GEOMETRY_LINES @EMBREE_GEOMETRY_LINES@)
|
||||
SET(EMBREE_GEOMETRY_HAIR @EMBREE_GEOMETRY_HAIR@)
|
||||
SET(EMBREE_GEOMETRY_SUBDIV @EMBREE_GEOMETRY_SUBDIV@)
|
||||
SET(EMBREE_GEOMETRY_USER @EMBREE_GEOMETRY_USER@)
|
||||
SET(EMBREE_RAY_PACKETS @EMBREE_RAY_PACKETS@)
|
||||
+17
-50
@@ -1,5 +1,5 @@
|
||||
## ======================================================================== ##
|
||||
## Copyright 2009-2017 Intel Corporation ##
|
||||
## Copyright 2009-2014 Intel Corporation ##
|
||||
## ##
|
||||
## Licensed under the Apache License, Version 2.0 (the "License"); ##
|
||||
## you may not use this file except in compliance with the License. ##
|
||||
@@ -15,58 +15,25 @@
|
||||
## ======================================================================== ##
|
||||
|
||||
SET(FLAGS_SSE2 "-msse2")
|
||||
SET(FLAGS_SSE3 "-msse3")
|
||||
SET(FLAGS_SSSE3 "-mssse3")
|
||||
SET(FLAGS_SSE41 "-msse4.1")
|
||||
SET(FLAGS_SSE42 "-msse4.2")
|
||||
SET(FLAGS_AVX "-mavx")
|
||||
SET(FLAGS_AVX2 "-mf16c -mavx2 -mfma -mlzcnt -mbmi -mbmi2")
|
||||
SET(FLAGS_AVX512KNL "-mavx512f -mavx512pf -mavx512er -mavx512cd -mf16c -mavx2 -mfma -mlzcnt -mbmi -mbmi2")
|
||||
SET(FLAGS_AVX512SKX "-mavx512f -mavx512dq -mavx512cd -mavx512bw -mavx512vl -mf16c -mavx2 -mfma -mlzcnt -mbmi -mbmi2")
|
||||
SET(FLAGS_AVX2 "-mf16c -mavx2 -mfma -mlzcnt -mabm -mbmi -mbmi2")
|
||||
|
||||
OPTION(EMBREE_IGNORE_CMAKE_CXX_FLAGS "When enabled Embree ignores default CMAKE_CXX_FLAGS." ON)
|
||||
IF (EMBREE_IGNORE_CMAKE_CXX_FLAGS)
|
||||
SET(CMAKE_CXX_FLAGS "")
|
||||
SET(CMAKE_CXX_COMPILER "g++")
|
||||
SET(CMAKE_C_COMPILER "gcc")
|
||||
SET(CMAKE_CXX_FLAGS "-fPIC")
|
||||
SET(CMAKE_CXX_FLAGS_DEBUG "-DDEBUG -g -O2")
|
||||
SET(CMAKE_CXX_FLAGS_RELEASE "-DNDEBUG -O3 -Wstrict-aliasing=0 ")
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "-DNDEBUG -g -O3 -Wstrict-aliasing=0 ")
|
||||
SET(CMAKE_EXE_LINKER_FLAGS "")
|
||||
|
||||
IF (NOT RTCORE_EXPORT_ALL_SYMBOLS)
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fvisibility-inlines-hidden -fvisibility=hidden")
|
||||
ENDIF()
|
||||
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wall") # enables most warnings
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wformat -Wformat-security") # enables string format vulnerability warnings
|
||||
IF (NOT APPLE)
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fPIE") # enables support for more secure position independent execution
|
||||
ENDIF()
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fPIC") # generate position independent code suitable for shared libraries
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11") # enables C++11 features
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fvisibility=hidden") # makes all symbols hidden by default
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fvisibility-inlines-hidden") # makes all inline symbols hidden by default
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fno-strict-aliasing") # disables strict aliasing rules
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -D_FORTIFY_SOURCE=2") # perform extra security checks for some standard library calls
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fno-tree-vectorize") # disable auto vectorizer
|
||||
IF (EMBREE_STACK_PROTECTOR)
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fstack-protector") # protects against return address overrides
|
||||
ENDIF()
|
||||
|
||||
SET(CMAKE_CXX_FLAGS_DEBUG "")
|
||||
SET(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -DDEBUG") # enable assertions
|
||||
SET(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -DTBB_USE_DEBUG") # configure TBB in debug mode
|
||||
SET(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -g") # generate debug information
|
||||
SET(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -O0") # disable optimizations
|
||||
|
||||
SET(CMAKE_CXX_FLAGS_RELEASE "")
|
||||
SET(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -DNDEBUG") # disable assertions
|
||||
SET(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -O3") # enable full optimizations
|
||||
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "")
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} -DDEBUG") # enable assertions
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} -DTBB_USE_DEBUG") # configure TBB in debug mode
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} -g") # generate debug information
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} -O3") # enable full optimizations
|
||||
|
||||
IF (APPLE)
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -mmacosx-version-min=10.7") # makes sure code runs on older MacOSX versions
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -stdlib=libc++") # link against libc++ which supports C++11 features
|
||||
ELSE(APPLE)
|
||||
SET(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl,--no-undefined") # issues link error for undefined symbols in shared library
|
||||
SET(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -z noexecstack") # we do not need an executable stack
|
||||
SET(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -z relro -z now") # re-arranges data sections to increase security
|
||||
SET(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -z noexecstack") # we do not need an executable stack
|
||||
SET(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -z relro -z now") # re-arranges data sections to increase security
|
||||
SET(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -pie") # enables position independent execution for executable
|
||||
ENDIF(APPLE)
|
||||
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -mmacosx-version-min=10.7")
|
||||
ENDIF (APPLE)
|
||||
|
||||
+23
-134
@@ -1,5 +1,5 @@
|
||||
## ======================================================================== ##
|
||||
## Copyright 2009-2017 Intel Corporation ##
|
||||
## Copyright 2009-2014 Intel Corporation ##
|
||||
## ##
|
||||
## Licensed under the Apache License, Version 2.0 (the "License"); ##
|
||||
## you may not use this file except in compliance with the License. ##
|
||||
@@ -14,139 +14,28 @@
|
||||
## limitations under the License. ##
|
||||
## ======================================================================== ##
|
||||
|
||||
IF (WIN32)
|
||||
SET(FLAGS_SSE2 "-xsse2")
|
||||
SET(FLAGS_SSE3 "-xsse3")
|
||||
SET(FLAGS_SSSE3 "-xssse3")
|
||||
SET(FLAGS_SSE41 "-xsse4.1")
|
||||
SET(FLAGS_SSE42 "-xsse4.2")
|
||||
SET(FLAGS_AVX "-xAVX")
|
||||
SET(FLAGS_AVX2 "-xCORE-AVX2")
|
||||
SET(FLAGS_AVX512 "-xMIC-AVX512")
|
||||
|
||||
SET(FLAGS_SSE2 "/QxSSE2")
|
||||
SET(FLAGS_SSE42 "/QxSSE4.2")
|
||||
SET(FLAGS_AVX "/arch:AVX")
|
||||
SET(FLAGS_AVX2 "/QxCORE-AVX2")
|
||||
SET(FLAGS_AVX512KNL "/QxMIC-AVX512")
|
||||
SET(FLAGS_AVX512SKX "/QxCORE-AVX512")
|
||||
|
||||
SET(COMMON_CXX_FLAGS "")
|
||||
SET(COMMON_CXX_FLAGS "${COMMON_CXX_FLAGS} /EHsc") # catch C++ exceptions only and extern "C" functions never throw a C++ exception
|
||||
SET(COMMON_CXX_FLAGS "${COMMON_CXX_FLAGS} /MP") # compile source files in parallel
|
||||
SET(COMMON_CXX_FLAGS "${COMMON_CXX_FLAGS} /GR") # enable runtime type information (on by default)
|
||||
SET(COMMON_CXX_FLAGS "${COMMON_CXX_FLAGS} /Qvec-") # disable auto vectorizer
|
||||
IF (EMBREE_STACK_PROTECTOR)
|
||||
SET(COMMON_CXX_FLAGS "${COMMON_CXX_FLAGS} /GS") # protects against return address overrides
|
||||
ELSE()
|
||||
SET(COMMON_CXX_FLAGS "${COMMON_CXX_FLAGS} /GS-") # do not protect against return address overrides
|
||||
ENDIF()
|
||||
|
||||
SET(COMMON_CXX_FLAGS "${COMMON_CXX_FLAGS} /Qdiag-disable:11074 ") # remark #11074: Inlining inhibited by limit max-size
|
||||
SET(COMMON_CXX_FLAGS "${COMMON_CXX_FLAGS} /Qdiag-disable:11075 ") # remark #11075: To get full report use -Qopt-report:4 -Qopt-report-phase ipo
|
||||
|
||||
SET(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} ${COMMON_CXX_FLAGS}")
|
||||
SET(CMAKE_EXE_LINKER_FLAGS_DEBUG "${CMAKE_EXE_LINKER_FLAGS_DEBUG} /DEBUG") # generate debug information
|
||||
SET(CMAKE_SHARED_LINKER_FLAGS_DEBUG "${CMAKE_SHARED_LINKER_FLAGS_DEBUG} /DEBUG") # generate debug information
|
||||
|
||||
SET(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} ${COMMON_CXX_FLAGS}")
|
||||
SET(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} /Ox") # enable full optimizations
|
||||
SET(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} /Oi") # inline intrinsic functions
|
||||
SET(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} /Gy") # package individual functions
|
||||
SET(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} /Qinline-max-total-size-") # no size limit when performing inlining
|
||||
SET(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} /Qinline-factor=200") # increase default inline factors by 2x
|
||||
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} ${COMMON_CXX_FLAGS}")
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} /Ox") # enable full optimizations
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} /Oi") # inline intrinsic functions
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} /Gy") # package individual functions
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} /Qinline-max-total-size-") # no size limit when performing inlining
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} /Qinline-factor=200") # increase default inline factors by 2x
|
||||
SET(CMAKE_EXE_LINKER_FLAGS_RELWITHDEBINFO "${CMAKE_EXE_LINKER_FLAGS_RELWITHDEBINFO} /DEBUG") # generate debug information
|
||||
SET(CMAKE_SHARED_LINKER_FLAGS_RELWITHDEBINFO "${CMAKE_SHARED_LINKER_FLAGS_RELWITHDEBINFO} /DEBUG") # generate debug information
|
||||
|
||||
SET(SECURE_LINKER_FLAGS "")
|
||||
SET(SECURE_LINKER_FLAGS "${SECURE_LINKER_FLAGS} /NXCompat") # compatible with data execution prevention (on by default)
|
||||
SET(SECURE_LINKER_FLAGS "${SECURE_LINKER_FLAGS} /DynamicBase") # random rebase of executable at load time
|
||||
IF (CMAKE_SIZEOF_VOID_P EQUAL 4)
|
||||
SET(SECURE_LINKER_FLAGS "${SECURE_LINKER_FLAGS} /SafeSEH") # invoke known exception handlers (Win32 only, x64 exception handlers are safe by design)
|
||||
ENDIF()
|
||||
SET(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${SECURE_LINKER_FLAGS}")
|
||||
SET(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} ${SECURE_LINKER_FLAGS}")
|
||||
|
||||
SET(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} /IGNORE:4217") # locally defined symbol XXX imported in function YYY (happens as the ISPC API layer uses exported library functions)
|
||||
SET(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} /IGNORE:4049") # warning LNK4049: locally defined symbol _rtcOccluded1M imported
|
||||
|
||||
INCLUDE(msvc_post)
|
||||
|
||||
ELSE()
|
||||
|
||||
IF (APPLE)
|
||||
SET(FLAGS_SSE2 "-xssse3") # on MacOSX ICC does not support SSE2
|
||||
ELSE()
|
||||
SET(FLAGS_SSE2 "-xsse2")
|
||||
ENDIF()
|
||||
SET(FLAGS_SSE42 "-xsse4.2")
|
||||
SET(FLAGS_AVX "-xAVX")
|
||||
SET(FLAGS_AVX2 "-xCORE-AVX2")
|
||||
SET(FLAGS_AVX512KNL "-xMIC-AVX512")
|
||||
SET(FLAGS_AVX512SKX "-xCORE-AVX512")
|
||||
|
||||
OPTION(EMBREE_IGNORE_CMAKE_CXX_FLAGS "When enabled Embree ignores default CMAKE_CXX_FLAGS." ON)
|
||||
IF (EMBREE_IGNORE_CMAKE_CXX_FLAGS)
|
||||
SET(CMAKE_CXX_FLAGS "")
|
||||
ENDIF()
|
||||
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wall") # enables most warnings
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wformat -Wformat-security") # enables string format vulnerability warnings
|
||||
IF (NOT APPLE)
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fPIE") # enables support for more secure position independent execution
|
||||
ENDIF()
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fPIC") # generate position independent code suitable for shared libraries
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11") # enables C++11 features
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fvisibility=hidden") # makes all symbols hidden by default
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fvisibility-inlines-hidden") # makes all inline symbols hidden by default
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -no-ansi-alias") # disables strict aliasing rules
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -D_FORTIFY_SOURCE=2") # perform extra security checks for some standard library calls
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -no-vec") # disable auto vectorizer
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fasm-blocks") # enable assembly blocks
|
||||
IF (EMBREE_STACK_PROTECTOR)
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fstack-protector") # protects against return address overrides
|
||||
ENDIF()
|
||||
IF(NOT CMAKE_CXX_COMPILER_WRAPPER STREQUAL "CrayPrgEnv")
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -static-intel") # links intel runtime statically
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -no-intel-extensions") # disables some intel extensions which cause symbols to be exported
|
||||
ENDIF()
|
||||
|
||||
SET(CMAKE_CXX_FLAGS_DEBUG "")
|
||||
SET(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -DDEBUG") # enables assertions
|
||||
SET(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -DTBB_USE_DEBUG") # configures TBB in debug mode
|
||||
SET(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -g") # generate debug information
|
||||
SET(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -O0") # disable all optimizations
|
||||
|
||||
SET(CMAKE_CXX_FLAGS_RELEASE "")
|
||||
SET(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -DNDEBUG") # disable assertions
|
||||
SET(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -O3") # full optimizations
|
||||
SET(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -restrict") # enable restrict keyword
|
||||
SET(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -no-inline-max-total-size") # no size limit when performing inlining
|
||||
SET(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -no-inline-max-per-compile") # no maximal number of inlinings per compilation unit
|
||||
SET(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -inline-factor=200") # increase default inline factors limits by 2x
|
||||
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "")
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} -DDEBUG") # enables assertions
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} -DTBB_USE_DEBUG") # configures TBB in debug mode
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} -g") # generate debug information
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} -O3") # full optimizations
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} -restrict") # enable restrict keyword
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} -no-inline-max-total-size") # no size limit when performing inlining
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} -no-inline-max-per-compile") # no maximal number of inlinings per compilation unit
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} -inline-factor=200") # increase default inline factors by 2x
|
||||
|
||||
#SET(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -g -debug inline-debug-info")
|
||||
#SET(CMAKE_EXE_LINKER_FLAGS "-g")
|
||||
|
||||
IF (APPLE)
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -mmacosx-version-min=10.7") # makes sure code runs on older MacOSX versions
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -stdlib=libc++") # link against libc++ which supports C++11 features
|
||||
ELSE(APPLE)
|
||||
SET(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl,--no-undefined") # issues link error for undefined symbols in shared library
|
||||
SET(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -z noexecstack") # we do not need an executable stack
|
||||
SET(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -z relro -z now") # re-arranges data sections to increase security
|
||||
SET(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -z noexecstack") # we do not need an executable stack
|
||||
SET(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -z relro -z now") # re-arranges data sections to increase security
|
||||
SET(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -pie") # enables position independent execution for executable
|
||||
ENDIF(APPLE)
|
||||
SET(CMAKE_CXX_COMPILER "icpc")
|
||||
SET(CMAKE_C_COMPILER "icc")
|
||||
SET(CMAKE_CXX_FLAGS "-Wall -fPIC -static-intel -fasm-blocks")
|
||||
SET(CMAKE_CXX_FLAGS_DEBUG "-DDEBUG -g -O2 ")
|
||||
SET(CMAKE_CXX_FLAGS_RELEASE "-DNDEBUG -O3 -no-ansi-alias -restrict ")
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "-DNDEBUG -g -O3 -no-ansi-alias -restrict ")
|
||||
SET(CMAKE_EXE_LINKER_FLAGS "")
|
||||
|
||||
IF (NOT RTCORE_EXPORT_ALL_SYMBOLS)
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fvisibility-inlines-hidden -fvisibility=hidden")
|
||||
ENDIF()
|
||||
|
||||
IF (APPLE)
|
||||
SET (CMAKE_SHARED_LINKER_FLAGS ${CMAKE_SHARED_LINKER_FLAGS_INIT} -dynamiclib)
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -mmacosx-version-min=10.7")
|
||||
ENDIF (APPLE)
|
||||
|
||||
@@ -0,0 +1,33 @@
|
||||
## ======================================================================== ##
|
||||
## Copyright 2009-2014 Intel Corporation ##
|
||||
## ##
|
||||
## Licensed under the Apache License, Version 2.0 (the "License"); ##
|
||||
## you may not use this file except in compliance with the License. ##
|
||||
## You may obtain a copy of the License at ##
|
||||
## ##
|
||||
## http://www.apache.org/licenses/LICENSE-2.0 ##
|
||||
## ##
|
||||
## Unless required by applicable law or agreed to in writing, software ##
|
||||
## distributed under the License is distributed on an "AS IS" BASIS, ##
|
||||
## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ##
|
||||
## See the License for the specific language governing permissions and ##
|
||||
## limitations under the License. ##
|
||||
## ======================================================================== ##
|
||||
|
||||
SET (CMAKE_CXX_COMPILER icpc)
|
||||
SET (CMAKE_CXX_FLAGS "-mmic -restrict -Wall -fasm-blocks -fPIC")
|
||||
SET (CMAKE_CXX_FLAGS_NOOPT "-g -O0 -DDEBUG")
|
||||
SET (CMAKE_CXX_FLAGS_DEBUG "-g -w1 -O2 -DDEBUG ")
|
||||
|
||||
#--no-exceptions --no-rtti
|
||||
# -ansi-alias # ISPC not working with ansi-alias flag
|
||||
|
||||
SET (CMAKE_CXX_FLAGS_RELEASE "-O3 -DNDEBUG -fp-model fast -fimf-precision=low -fasm-blocks -no-inline-max-total-size -inline-factor=200 -fPIC -fma -restrict -no-prec-div -no-prec-sqrt -mGLOB_default_function_attrs=\"use_vec_for_imul=on;use_fast_math=on;gather_scatter_loop_jknzd=on;gather_scatter_loop_unroll=2;use_gather_scatter_hint=on;c_sch_nop_insertion=on;c_avoid_movz_and_movs=off;c_avoid_integer_ciscization=on;avoid_long_vector_ints=on;avoid_loads_with_extend=on;smart_mem_conflicts=on\" -mP2OPT_hlo_prefetch=F ")
|
||||
|
||||
SET (CMAKE_CXX_FLAGS_RELWITHDEBINFO "-g -O3 -DNDEBUG -fp-model fast -fimf-precision=low -fasm-blocks -no-inline-max-total-size -inline-factor=200 -fPIC -fma -restrict -no-prec-div -no-prec-sqrt -mGLOB_default_function_attrs=\"use_vec_for_imul=on;use_fast_math=on;gather_scatter_loop_jknzd=on;gather_scatter_loop_unroll=2;use_gather_scatter_hint=on;c_sch_nop_insertion=on;c_avoid_movz_and_movs=off;c_avoid_integer_ciscization=on;avoid_long_vector_ints=on;avoid_loads_with_extend=on;smart_mem_conflicts=on\" -mP2OPT_hlo_prefetch=F ")
|
||||
|
||||
SET(CMAKE_LINKER icpc)
|
||||
SET(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -static-intel")
|
||||
|
||||
#SET(CMAKE_CXX_LINK_EXECUTABLE "<CMAKE_CXX_COMPILER> -mmic -static-intel -rdynamic -fPIC -L${CMAKE_BINARY_DIR} <LINK_FLAGS> -o <TARGET> ${CMAKE_GNULD_IMAGE_VERSION} <OBJECTS> <LINK_LIBRARIES>")
|
||||
#SET(CMAKE_CXX_CREATE_SHARED_LIBRARY "<CMAKE_CXX_COMPILER> -mmic -static-intel <LANGUAGE_COMPILE_FLAGS> <CMAKE_SHARED_MODULE_CXX_FLAGS> <LINK_FLAGS> <CMAKE_SHARED_MODULE_CREATE_CXX_FLAGS> -o <TARGET> ${CMAKE_GNULD_IMAGE_VERSION} <OBJECTS> <LINK_LIBRARIES> -zmuldefs")
|
||||
+73
-127
@@ -1,5 +1,5 @@
|
||||
## ======================================================================== ##
|
||||
## Copyright 2009-2017 Intel Corporation ##
|
||||
## Copyright 2009-2014 Intel Corporation ##
|
||||
## ##
|
||||
## Licensed under the Apache License, Version 2.0 (the "License"); ##
|
||||
## you may not use this file except in compliance with the License. ##
|
||||
@@ -14,109 +14,73 @@
|
||||
## limitations under the License. ##
|
||||
## ======================================================================== ##
|
||||
|
||||
# ##################################################################
|
||||
# add macro INCLUDE_DIRECTORIES_ISPC() that allows to specify search
|
||||
# paths for ISPC sources
|
||||
# ##################################################################
|
||||
SET(ISPC_INCLUDE_DIR "")
|
||||
SET (ISPC_INCLUDE_DIR "")
|
||||
MACRO (INCLUDE_DIRECTORIES_ISPC)
|
||||
SET(ISPC_INCLUDE_DIR ${ISPC_INCLUDE_DIR} ${ARGN})
|
||||
SET (ISPC_INCLUDE_DIR ${ISPC_INCLUDE_DIR} ${ARGN})
|
||||
ENDMACRO ()
|
||||
|
||||
IF (EMBREE_ISPC_SUPPORT)
|
||||
execute_process(COMMAND which ispc OUTPUT_VARIABLE ISPC_EXECUTABLE)
|
||||
#IF (EXISTS "/home/benthin/Projects/ispc/ispc")
|
||||
#"${ISPC_EXECUTABLE}")
|
||||
#message(${ISPC_EXECUTABLE})
|
||||
execute_process(COMMAND dirname ${ISPC_EXECUTABLE} OUTPUT_VARIABLE ISPC_DIR_TEMP)
|
||||
#ENDIF()
|
||||
STRING(REGEX REPLACE "\n" "" ISPC_DIR "${ISPC_DIR_TEMP}")
|
||||
|
||||
# ISPC versions to look for, in decending order (newest first)
|
||||
SET(ISPC_VERSION_WORKING "1.9.1" "1.9.0" "1.8.3" "1.8.2")
|
||||
LIST(GET ISPC_VERSION_WORKING -1 ISPC_VERSION_REQUIRED)
|
||||
|
||||
IF (NOT EMBREE_ISPC_EXECUTABLE)
|
||||
# try sibling folder as hint for path of ISPC
|
||||
IF (APPLE)
|
||||
SET(ISPC_DIR_SUFFIX "osx")
|
||||
ELSEIF(WIN32)
|
||||
SET(ISPC_DIR_SUFFIX "windows")
|
||||
IF (MSVC14)
|
||||
LIST(APPEND ISPC_DIR_SUFFIX "windows-vs2015")
|
||||
ELSE()
|
||||
LIST(APPEND ISPC_DIR_SUFFIX "windows-vs2013")
|
||||
ENDIF()
|
||||
ELSE()
|
||||
SET(ISPC_DIR_SUFFIX "linux")
|
||||
ENDIF()
|
||||
FOREACH(ver ${ISPC_VERSION_WORKING})
|
||||
FOREACH(suffix ${ISPC_DIR_SUFFIX})
|
||||
LIST(APPEND ISPC_DIR_HINT ${PROJECT_SOURCE_DIR}/../ispc-v${ver}-${suffix})
|
||||
ENDFOREACH()
|
||||
ENDFOREACH()
|
||||
|
||||
FIND_PROGRAM(EMBREE_ISPC_EXECUTABLE ispc PATHS ${ISPC_DIR_HINT} DOC "Path to the ISPC executable.")
|
||||
IF (NOT EMBREE_ISPC_EXECUTABLE)
|
||||
MESSAGE(FATAL_ERROR "Intel SPMD Compiler (ISPC) not found. Disable EMBREE_ISPC_SUPPORT or install ISPC.")
|
||||
ELSE()
|
||||
MESSAGE(STATUS "Found Intel SPMD Compiler (ISPC): ${EMBREE_ISPC_EXECUTABLE}")
|
||||
ENDIF()
|
||||
ENDIF()
|
||||
|
||||
# check ISPC version
|
||||
EXECUTE_PROCESS(COMMAND ${EMBREE_ISPC_EXECUTABLE} --version OUTPUT_VARIABLE ISPC_OUTPUT)
|
||||
STRING(REGEX MATCH " ([0-9]+[.][0-9]+[.][0-9]+)(dev|knl)? " DUMMY "${ISPC_OUTPUT}")
|
||||
SET(ISPC_VERSION ${CMAKE_MATCH_1})
|
||||
|
||||
IF (ISPC_VERSION VERSION_LESS ISPC_VERSION_REQUIRED)
|
||||
MESSAGE(FATAL_ERROR "Need at least version ${ISPC_VERSION_REQUIRED} of Intel SPMD Compiler (ISPC).")
|
||||
ENDIF()
|
||||
|
||||
GET_FILENAME_COMPONENT(ISPC_DIR ${EMBREE_ISPC_EXECUTABLE} PATH)
|
||||
|
||||
SET(EMBREE_ISPC_ADDRESSING 32 CACHE INT "32vs64 bit addressing in ispc")
|
||||
SET(EMBREE_ISPC_ADDRESSING 32 CACHE INT
|
||||
"32vs64 bit addressing in ispc")
|
||||
MARK_AS_ADVANCED(EMBREE_ISPC_ADDRESSING)
|
||||
|
||||
MACRO (ISPC_COMPILE)
|
||||
SET(ISPC_ADDITIONAL_ARGS "")
|
||||
|
||||
SET(ISPC_TARGET_EXT ${CMAKE_CXX_OUTPUT_EXTENSION})
|
||||
STRING(REPLACE ";" "," ISPC_TARGET_ARGS "${ISPC_TARGETS}")
|
||||
|
||||
IF (CMAKE_SIZEOF_VOID_P EQUAL 8)
|
||||
SET(ISPC_ARCHITECTURE "x86-64")
|
||||
MACRO (ispc_compile targets)
|
||||
IF (__XEON__)
|
||||
SET (ISPC_TARGET_EXT o)
|
||||
ELSE()
|
||||
SET(ISPC_ARCHITECTURE "x86")
|
||||
SET (ISPC_TARGET_EXT cpp)
|
||||
SET (ISPC_TARGET_ALIGNED_MEMORY --opt=force-aligned-memory)
|
||||
ENDIF()
|
||||
|
||||
IF (CMAKE_OSX_ARCHITECTURES STREQUAL "i386")
|
||||
SET(ISPC_ARCHITECTURE "x86")
|
||||
ELSE()
|
||||
SET(ISPC_ARCHITECTURE "x86-64")
|
||||
ENDIF()
|
||||
|
||||
SET(ISPC_TARGET_DIR ${CMAKE_CURRENT_BINARY_DIR})
|
||||
INCLUDE_DIRECTORIES(${CMAKE_CURRENT_SOURCE_DIR} ${ISPC_TARGET_DIR})
|
||||
|
||||
IF(ISPC_INCLUDE_DIR)
|
||||
STRING(REPLACE ";" ";-I;" ISPC_INCLUDE_DIR_PARMS "${ISPC_INCLUDE_DIR}")
|
||||
SET(ISPC_INCLUDE_DIR_PARMS "-I" ${ISPC_INCLUDE_DIR_PARMS})
|
||||
STRING(REGEX REPLACE ";" ";-I;" ISPC_INCLUDE_DIR_PARMS "${ISPC_INCLUDE_DIR}")
|
||||
SET(ISPC_INCLUDE_DIR_PARMS "-I" ${ISPC_INCLUDE_DIR_PARMS})
|
||||
ENDIF()
|
||||
|
||||
IF (WIN32 OR "${CMAKE_BUILD_TYPE}" STREQUAL "Release")
|
||||
SET(ISPC_OPT_FLAGS -O3)
|
||||
IF (__XEON__)
|
||||
STRING(REGEX REPLACE "," ";" target_list "${targets}")
|
||||
SET(ISPC_TARGETS ${targets})
|
||||
ELSE()
|
||||
SET(ISPC_OPT_FLAGS -O2 -g)
|
||||
ENDIF()
|
||||
|
||||
IF (WIN32)
|
||||
SET(ISPC_ADDITIONAL_ARGS ${ISPC_ADDITIONAL_ARGS} --dllexport)
|
||||
ELSE()
|
||||
SET(ISPC_ADDITIONAL_ARGS ${ISPC_ADDITIONAL_ARGS} --pic)
|
||||
SET(ISPC_TARGETS generic-16 --emit-c++ -D__XEON_PHI__ --c++-include-file=${ISPC_DIR}/examples/intrinsics/knc.h)
|
||||
ENDIF()
|
||||
|
||||
SET(ISPC_OBJECTS "")
|
||||
|
||||
FOREACH(src ${ARGN})
|
||||
|
||||
GET_FILENAME_COMPONENT(fname ${src} NAME_WE)
|
||||
GET_FILENAME_COMPONENT(dir ${src} PATH)
|
||||
|
||||
SET(outdir "${ISPC_TARGET_DIR}/${dir}")
|
||||
SET(input ${CMAKE_CURRENT_SOURCE_DIR}/${src})
|
||||
IF("${dir}" STREQUAL "")
|
||||
SET(outdir ${ISPC_TARGET_DIR})
|
||||
ELSE("${dir}" STREQUAL "")
|
||||
SET(outdir ${ISPC_TARGET_DIR}/${dir})
|
||||
ENDIF("${dir}" STREQUAL "")
|
||||
SET(outdirh ${ISPC_TARGET_DIR})
|
||||
|
||||
SET(deps "")
|
||||
IF (EXISTS ${outdir}/${fname}.dev.idep)
|
||||
FILE(READ ${outdir}/${fname}.dev.idep contents)
|
||||
STRING(REPLACE " " ";" contents "${contents}")
|
||||
STRING(REPLACE ";" "\\\\;" contents "${contents}")
|
||||
STRING(REPLACE "\n" ";" contents "${contents}")
|
||||
STRING(REGEX REPLACE " " ";" contents "${contents}")
|
||||
STRING(REGEX REPLACE ";" "\\\\;" contents "${contents}")
|
||||
STRING(REGEX REPLACE "\n" ";" contents "${contents}")
|
||||
FOREACH(dep ${contents})
|
||||
IF (EXISTS ${dep})
|
||||
SET(deps ${deps} ${dep})
|
||||
@@ -124,49 +88,47 @@ MACRO (ISPC_COMPILE)
|
||||
ENDFOREACH(dep ${contents})
|
||||
ENDIF ()
|
||||
|
||||
SET(results "${outdir}/${fname}.dev${ISPC_TARGET_EXT}")
|
||||
SET(results "${outdir}/${fname}.dev.${ISPC_TARGET_EXT}")
|
||||
|
||||
# if we have multiple targets add additional object files
|
||||
LIST(LENGTH ISPC_TARGETS NUM_TARGETS)
|
||||
IF (NUM_TARGETS GREATER 1)
|
||||
FOREACH(target ${ISPC_TARGETS})
|
||||
# in v1.9.0 ISPC changed the ISA suffix of avx512knl-i32x16 to just 'avx512knl'
|
||||
IF (${target} STREQUAL "avx512knl-i32x16" AND NOT ISPC_VERSION VERSION_LESS "1.9.0")
|
||||
SET(target "avx512knl")
|
||||
ELSEIF (${target} STREQUAL "avx512skx-i32x16")
|
||||
SET(target "avx512skx")
|
||||
ENDIF()
|
||||
SET(results ${results} "${outdir}/${fname}.dev_${target}${ISPC_TARGET_EXT}")
|
||||
ENDFOREACH()
|
||||
IF (__XEON__)
|
||||
IF (${targets} MATCHES ".*,.*")
|
||||
FOREACH(target ${target_list})
|
||||
SET(results ${results} "${outdir}/${fname}.dev_${target}.${ISPC_TARGET_EXT}")
|
||||
ENDFOREACH()
|
||||
ENDIF()
|
||||
ENDIF()
|
||||
|
||||
|
||||
ADD_CUSTOM_COMMAND(
|
||||
OUTPUT ${results} ${ISPC_TARGET_DIR}/${fname}_ispc.h
|
||||
COMMAND ${CMAKE_COMMAND} -E make_directory ${outdir}
|
||||
COMMAND ${EMBREE_ISPC_EXECUTABLE}
|
||||
-I ${CMAKE_CURRENT_SOURCE_DIR}
|
||||
OUTPUT ${results} ${outdirh}/${fname}_ispc.h
|
||||
COMMAND mkdir -p ${outdir} \; ispc
|
||||
-I ${CMAKE_CURRENT_SOURCE_DIR}
|
||||
${ISPC_INCLUDE_DIR_PARMS}
|
||||
${ISPC_DEFINITIONS}
|
||||
--arch=${ISPC_ARCHITECTURE}
|
||||
--addressing=${EMBREE_ISPC_ADDRESSING}
|
||||
${ISPC_OPT_FLAGS}
|
||||
--target=${ISPC_TARGET_ARGS}
|
||||
--addressing=${EMBREE_ISPC_ADDRESSING}
|
||||
--pic
|
||||
-O3
|
||||
--target=${ISPC_TARGETS}
|
||||
--woff
|
||||
# --wno-perf
|
||||
--opt=fast-math
|
||||
${ISPC_ADDITIONAL_ARGS}
|
||||
-h ${ISPC_TARGET_DIR}/${fname}_ispc.h
|
||||
-MMM ${outdir}/${fname}.dev.idep
|
||||
-o ${outdir}/${fname}.dev${ISPC_TARGET_EXT}
|
||||
${input}
|
||||
DEPENDS ${input} ${deps}
|
||||
COMMENT "Building ISPC object ${outdir}/${fname}.dev${ISPC_TARGET_EXT}"
|
||||
)
|
||||
${ISPC_TARGET_ALIGNED_MEMORY}
|
||||
# --opt=force-aligned-memory
|
||||
-h ${outdirh}/${fname}_ispc.h
|
||||
-MMM ${outdir}/${fname}.dev.idep
|
||||
-o ${outdir}/${fname}.dev.${ISPC_TARGET_EXT}
|
||||
${CMAKE_CURRENT_SOURCE_DIR}/${src}
|
||||
\;
|
||||
DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/${src}
|
||||
${deps})
|
||||
|
||||
SET(ISPC_OBJECTS ${ISPC_OBJECTS} ${results})
|
||||
|
||||
ENDFOREACH()
|
||||
|
||||
ENDMACRO()
|
||||
|
||||
MACRO (ADD_EMBREE_ISPC_EXECUTABLE name)
|
||||
MACRO (add_ispc_executable name)
|
||||
SET(ISPC_SOURCES "")
|
||||
SET(OTHER_SOURCES "")
|
||||
FOREACH(src ${ARGN})
|
||||
@@ -177,11 +139,12 @@ MACRO (ADD_EMBREE_ISPC_EXECUTABLE name)
|
||||
SET(OTHER_SOURCES ${OTHER_SOURCES} ${src})
|
||||
ENDIF ()
|
||||
ENDFOREACH()
|
||||
ISPC_COMPILE(${ISPC_SOURCES})
|
||||
ISPC_COMPILE (${ISPC_TARGETS} ${ISPC_SOURCES})
|
||||
ADD_EXECUTABLE(${name} ${ISPC_OBJECTS} ${OTHER_SOURCES})
|
||||
# SET_TARGET_PROPERTIES(${name} PROPERTIES LINKER_LANGUAGE C)
|
||||
ENDMACRO()
|
||||
|
||||
MACRO (ADD_ISPC_LIBRARY name type)
|
||||
MACRO (add_ispc_library name type)
|
||||
SET(ISPC_SOURCES "")
|
||||
SET(OTHER_SOURCES "")
|
||||
FOREACH(src ${ARGN})
|
||||
@@ -192,24 +155,7 @@ MACRO (ADD_ISPC_LIBRARY name type)
|
||||
SET(OTHER_SOURCES ${OTHER_SOURCES} ${src})
|
||||
ENDIF ()
|
||||
ENDFOREACH()
|
||||
ISPC_COMPILE(${ISPC_SOURCES})
|
||||
ISPC_COMPILE (${ISPC_TARGETS} ${ISPC_SOURCES})
|
||||
ADD_LIBRARY(${name} ${type} ${ISPC_OBJECTS} ${OTHER_SOURCES})
|
||||
# SET_TARGET_PROPERTIES(${name} PROPERTIES LINKER_LANGUAGE C)
|
||||
ENDMACRO()
|
||||
|
||||
ELSE (EMBREE_ISPC_SUPPORT)
|
||||
|
||||
MACRO (ADD_ISPC_LIBRARY name type)
|
||||
SET(ISPC_SOURCES "")
|
||||
SET(OTHER_SOURCES "")
|
||||
FOREACH(src ${ARGN})
|
||||
GET_FILENAME_COMPONENT(ext ${src} EXT)
|
||||
IF (ext STREQUAL ".ispc")
|
||||
SET(ISPC_SOURCES ${ISPC_SOURCES} ${src})
|
||||
ELSE ()
|
||||
SET(OTHER_SOURCES ${OTHER_SOURCES} ${src})
|
||||
ENDIF ()
|
||||
ENDFOREACH()
|
||||
ADD_LIBRARY(${name} ${type} ${OTHER_SOURCES})
|
||||
ENDMACRO()
|
||||
|
||||
ENDIF (EMBREE_ISPC_SUPPORT)
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
|
||||
<PropertyGroup
|
||||
Condition="'$(ISPCBeforeTargets)' == '' and '$(ISPCAfterTargets)' == '' and '$(ConfigurationType)' != 'Makefile'">
|
||||
<ISPCBeforeTargets>Midl</ISPCBeforeTargets>
|
||||
<ISPCAfterTargets>CustomBuild</ISPCAfterTargets>
|
||||
</PropertyGroup>
|
||||
<PropertyGroup>
|
||||
<ISPCDependsOn
|
||||
Condition="'$(ConfigurationType)' != 'Makefile'">_SelectedFiles;$(ISPCDependsOn)</ISPCDependsOn>
|
||||
</PropertyGroup>
|
||||
<ItemDefinitionGroup>
|
||||
<ISPC>
|
||||
<GenerateDebugInformation>False</GenerateDebugInformation>
|
||||
<Assertions>true</Assertions>
|
||||
<Aligned>False</Aligned>
|
||||
<PIC>False</PIC>
|
||||
<WarningsAsErrors>False</WarningsAsErrors>
|
||||
<NoWarnings>False</NoWarnings>
|
||||
<ObjectFileName>$(IntDir)%(Filename).ispc.obj</ObjectFileName>
|
||||
<HeaderFile>$(IntDir)%(Filename)_ispc.h</HeaderFile>
|
||||
<Architecture>1</Architecture>
|
||||
<Processor>1</Processor>
|
||||
<TargetISA>0</TargetISA>
|
||||
<WarningLevel>1</WarningLevel>
|
||||
<MathLibrary>0</MathLibrary>
|
||||
<Opt>1</Opt>
|
||||
<CommandLineTemplate>ispc.exe [AllOptions] [AdditionalOptions] [Inputs]</CommandLineTemplate>
|
||||
<Outputs Condition="'$(Configuration)'=='Debug'">$(IntDir)%(Filename).ispc.obj;$(IntDir)%(Filename).ispc_sse2.obj;$(IntDir)%(Filename).ispc_sse4.obj</Outputs>
|
||||
<Outputs Condition="'$(Configuration)'=='Release'">$(IntDir)%(Filename).ispc.obj;$(IntDir)%(Filename).ispc_sse2.obj;$(IntDir)%(Filename).ispc_sse4.obj</Outputs>
|
||||
<Outputs Condition="'$(Configuration)'=='ReleaseAVX'">$(IntDir)%(Filename).ispc.obj;$(IntDir)%(Filename).ispc_sse2.obj;$(IntDir)%(Filename).ispc_sse4.obj;$(IntDir)%(Filename).ispc_avx.obj</Outputs>
|
||||
<Outputs Condition="'$(Configuration)'=='ReleaseAVX2'">$(IntDir)%(Filename).ispc.obj;$(IntDir)%(Filename).ispc_sse2.obj;$(IntDir)%(Filename).ispc_sse4.obj;$(IntDir)%(Filename).ispc_avx.obj;;$(IntDir)%(Filename).ispc_avx2.obj</Outputs>
|
||||
<Outputs Condition="'$(MSBuildProjectName)'=='embree'">$(IntDir)%(Filename).ispc.obj;$(IntDir)%(Filename).ispc_sse2.obj;$(IntDir)%(Filename).ispc_sse4.obj;$(IntDir)%(Filename).ispc_avx.obj;$(IntDir)%(Filename).ispc_avx2.obj</Outputs>
|
||||
<ExecutionDescription>Compiling %(Filename)%(Extension) ...</ExecutionDescription>
|
||||
</ISPC>
|
||||
</ItemDefinitionGroup>
|
||||
</Project>
|
||||
@@ -0,0 +1,225 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<VisualStudioToolFile
|
||||
Name="ISPC"
|
||||
Version="8,00"
|
||||
>
|
||||
<Rules>
|
||||
<CustomBuildRule
|
||||
Name="ISPC"
|
||||
DisplayName="Intel SPMD Programm Compiler (ISPC)"
|
||||
CommandLine="ispc.exe [AllOptions] [AdditionalOptions] [Inputs]"
|
||||
Outputs="$(IntDir)\$(InputName).ispc*.obj"
|
||||
FileExtensions="*.ispc"
|
||||
ExecutionDescription="Compiling $(InputFileName) ..."
|
||||
>
|
||||
<Properties>
|
||||
<EnumProperty
|
||||
Name="Architecture"
|
||||
DisplayName="Target Architecture"
|
||||
Description="Select Target Architecture."
|
||||
DefaultValue="1"
|
||||
>
|
||||
<Values>
|
||||
<EnumValue
|
||||
Value="0"
|
||||
Switch="--arch=x86"
|
||||
DisplayName="x86"
|
||||
/>
|
||||
<EnumValue
|
||||
Value="1"
|
||||
Switch="--arch=x86-64"
|
||||
DisplayName="x86-64"
|
||||
/>
|
||||
</Values>
|
||||
</EnumProperty>
|
||||
<EnumProperty
|
||||
Name="TargetISA"
|
||||
DisplayName="Target ISA"
|
||||
Description="Select Target ISA."
|
||||
>
|
||||
<Values>
|
||||
<EnumValue
|
||||
Value="0"
|
||||
Switch="--target=sse2"
|
||||
DisplayName="SSE2"
|
||||
/>
|
||||
<EnumValue
|
||||
Value="1"
|
||||
Switch="--target=sse4"
|
||||
DisplayName="SSE4"
|
||||
/>
|
||||
<EnumValue
|
||||
Value="2"
|
||||
Switch="--target=avx"
|
||||
DisplayName="AVX"
|
||||
/>
|
||||
<EnumValue
|
||||
Value="3"
|
||||
Switch="--target=avx1.1"
|
||||
DisplayName="AVX-I"
|
||||
/>
|
||||
<EnumValue
|
||||
Value="4"
|
||||
Switch="--target=AVX2"
|
||||
DisplayName="AVX2"
|
||||
/>
|
||||
<EnumValue
|
||||
Value="5"
|
||||
Switch="--target=sse2,sse4"
|
||||
DisplayName="SSE2,SSE4"
|
||||
/>
|
||||
<EnumValue
|
||||
Value="6"
|
||||
Switch="--target=sse2,sse4,avx"
|
||||
DisplayName="SSE2,SSE4,AVX"
|
||||
/>
|
||||
<EnumValue
|
||||
Value="7"
|
||||
Switch="--target=sse2,sse4,avx,avx2"
|
||||
DisplayName="SSE2,SSE4,AVX,AVX2"
|
||||
/>
|
||||
</Values>
|
||||
</EnumProperty>
|
||||
<StringProperty
|
||||
Name="ObjectFileName"
|
||||
DisplayName="Object File Name"
|
||||
PropertyPageName="Output"
|
||||
Description="Specifies the name of the output object file."
|
||||
Switch="-o "[value]""
|
||||
DefaultValue="$(IntDir)\$(InputName).ispc.obj"
|
||||
/>
|
||||
<StringProperty
|
||||
Name="PreprocessorDefinitions"
|
||||
DisplayName="Preprocessor Definitions"
|
||||
Description="Defines a text macro with the given name."
|
||||
Switch="-D"[value]""
|
||||
Delimited="true"
|
||||
Inheritable="true"
|
||||
/>
|
||||
<StringProperty
|
||||
Name="IncludePaths"
|
||||
DisplayName="Include Paths"
|
||||
Description="Sets path for include file. "
|
||||
Switch="-I "[value]""
|
||||
Delimited="true"
|
||||
Inheritable="true"
|
||||
/>
|
||||
<EnumProperty
|
||||
Name="WarningLevel"
|
||||
DisplayName="Warning Level"
|
||||
Description="Sets the warning level."
|
||||
DefaultValue="1"
|
||||
>
|
||||
<Values>
|
||||
<EnumValue
|
||||
Value="0"
|
||||
Switch="--woff"
|
||||
DisplayName="Warning Level 0"
|
||||
/>
|
||||
<EnumValue
|
||||
Value="1"
|
||||
Switch="--wno-perf"
|
||||
DisplayName="Warning Level 1"
|
||||
/>
|
||||
<EnumValue
|
||||
Value="2"
|
||||
DisplayName="Warning Level 2"
|
||||
/>
|
||||
</Values>
|
||||
</EnumProperty>
|
||||
<BooleanProperty
|
||||
Name="GenerateDebugInformation"
|
||||
DisplayName="Generate Debug Information"
|
||||
Description="Generates Debug Information. (-g)"
|
||||
Switch="-g"
|
||||
/>
|
||||
<EnumProperty
|
||||
Name="MathLibrary"
|
||||
DisplayName="Math Library"
|
||||
Description="Select Math Library"
|
||||
>
|
||||
<Values>
|
||||
<EnumValue
|
||||
Value="0"
|
||||
Switch="--math-lib=default"
|
||||
DisplayName="ISPC Internal"
|
||||
/>
|
||||
<EnumValue
|
||||
Value="1"
|
||||
Switch="--math-lib=fast"
|
||||
DisplayName="Fast"
|
||||
/>
|
||||
<EnumValue
|
||||
Value="2"
|
||||
Switch="--math-lib=svml"
|
||||
DisplayName="SVML"
|
||||
/>
|
||||
<EnumValue
|
||||
Value="3"
|
||||
Switch="--math-lib=system"
|
||||
DisplayName="System"
|
||||
/>
|
||||
</Values>
|
||||
</EnumProperty>
|
||||
<EnumProperty
|
||||
Name="Opt"
|
||||
DisplayName="Optimization Level"
|
||||
Description="Optimization Level"
|
||||
DefaultValue="1"
|
||||
>
|
||||
<Values>
|
||||
<EnumValue
|
||||
Value="0"
|
||||
Switch="-O0"
|
||||
DisplayName="O0"
|
||||
/>
|
||||
<EnumValue
|
||||
Value="1"
|
||||
Switch="-O1"
|
||||
DisplayName="O1"
|
||||
/>
|
||||
</Values>
|
||||
</EnumProperty>
|
||||
<BooleanProperty
|
||||
Name="Assertions"
|
||||
DisplayName="Assertions"
|
||||
Description="Disables Assertions"
|
||||
Switch="--opt=disable-assertions"
|
||||
DefaultValue="true"
|
||||
/>
|
||||
<BooleanProperty
|
||||
Name="Aligned"
|
||||
DisplayName="Force Aligned Loads"
|
||||
Description="Forces Aligned Loads."
|
||||
Switch="--opt=force-aligned-memory"
|
||||
/>
|
||||
<BooleanProperty
|
||||
Name="PIC"
|
||||
DisplayName="Position Independent Code"
|
||||
Description="Generate Position Independent Code"
|
||||
Switch="--pic"
|
||||
/>
|
||||
<BooleanProperty
|
||||
Name="WarningsAsErrors"
|
||||
DisplayName="Treat Warnings as Errors"
|
||||
Description="Treat Warnings as Errors."
|
||||
Switch="--werror"
|
||||
/>
|
||||
<StringProperty
|
||||
Name="HeaderFile"
|
||||
DisplayName="Header File Name"
|
||||
PropertyPageName="Output"
|
||||
Description="Interface Header File."
|
||||
Switch="-h "[value]""
|
||||
DefaultValue="$(IntDir)\$(InputName)_ispc.h"
|
||||
/>
|
||||
<BooleanProperty
|
||||
Name="NoWarnings"
|
||||
DisplayName="Disable Warnings"
|
||||
Description="Disable Warnings"
|
||||
Switch="--woff"
|
||||
/>
|
||||
</Properties>
|
||||
</CustomBuildRule>
|
||||
</Rules>
|
||||
</VisualStudioToolFile>
|
||||
@@ -0,0 +1,96 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
|
||||
<ItemGroup>
|
||||
<PropertyPageSchema
|
||||
Include="$(MSBuildThisFileDirectory)$(MSBuildThisFileName).xml" />
|
||||
<AvailableItemName
|
||||
Include="ISPC">
|
||||
<Targets>_ISPC</Targets>
|
||||
</AvailableItemName>
|
||||
</ItemGroup>
|
||||
<UsingTask
|
||||
TaskName="ISPC"
|
||||
TaskFactory="XamlTaskFactory"
|
||||
AssemblyName="Microsoft.Build.Tasks.v4.0">
|
||||
<Task>$(MSBuildThisFileDirectory)$(MSBuildThisFileName).xml</Task>
|
||||
</UsingTask>
|
||||
<Target
|
||||
Name="_ISPC"
|
||||
BeforeTargets="$(ISPCBeforeTargets)"
|
||||
AfterTargets="$(ISPCAfterTargets)"
|
||||
Condition="'@(ISPC)' != ''"
|
||||
DependsOnTargets="$(ISPCDependsOn);ComputeISPCOutput"
|
||||
Outputs="%(ISPC.Outputs)"
|
||||
Inputs="%(ISPC.Identity);%(ISPC.AdditionalDependencies);$(MSBuildProjectFile)">
|
||||
<ItemGroup
|
||||
Condition="'@(SelectedFiles)' != ''">
|
||||
<ISPC
|
||||
Remove="@(ISPC)"
|
||||
Condition="'%(Identity)' != '@(SelectedFiles)'" />
|
||||
</ItemGroup>
|
||||
<ItemGroup>
|
||||
<ISPC_tlog
|
||||
Include="%(ISPC.Outputs)"
|
||||
Condition="'%(ISPC.Outputs)' != '' and '%(ISPC.ExcludedFromBuild)' != 'true'">
|
||||
<Source>@(ISPC, '|')</Source>
|
||||
</ISPC_tlog>
|
||||
</ItemGroup>
|
||||
<Message
|
||||
Importance="High"
|
||||
Text="%(ISPC.ExecutionDescription)" />
|
||||
<WriteLinesToFile
|
||||
Condition="'@(ISPC_tlog)' != '' and '%(ISPC_tlog.ExcludedFromBuild)' != 'true'"
|
||||
File="$(IntDir)$(ProjectName).write.1.tlog"
|
||||
Lines="^%(ISPC_tlog.Source);@(ISPC_tlog->'%(Fullpath)')" />
|
||||
<ISPC
|
||||
Condition="'@(ISPC)' != '' and '%(ISPC.ExcludedFromBuild)' != 'true'"
|
||||
CommandLineTemplate="%(ISPC.CommandLineTemplate)"
|
||||
GenerateDebugInformation="%(ISPC.GenerateDebugInformation)"
|
||||
Assertions="%(ISPC.Assertions)"
|
||||
Aligned="%(ISPC.Aligned)"
|
||||
PIC="%(ISPC.PIC)"
|
||||
WarningsAsErrors="%(ISPC.WarningsAsErrors)"
|
||||
NoWarnings="%(ISPC.NoWarnings)"
|
||||
ObjectFileName="%(ISPC.ObjectFileName)"
|
||||
PreprocessorDefinitions="%(ISPC.PreprocessorDefinitions)"
|
||||
IncludePaths="%(ISPC.IncludePaths)"
|
||||
HeaderFile="%(ISPC.HeaderFile)"
|
||||
Architecture="%(ISPC.Architecture)"
|
||||
TargetISA="%(ISPC.TargetISA)"
|
||||
WarningLevel="%(ISPC.WarningLevel)"
|
||||
MathLibrary="%(ISPC.MathLibrary)"
|
||||
Opt="%(ISPC.Opt)"
|
||||
AdditionalOptions="%(ISPC.AdditionalOptions)"
|
||||
Inputs="%(ISPC.Identity)" />
|
||||
</Target>
|
||||
<PropertyGroup>
|
||||
<ComputeLinkInputsTargets>
|
||||
$(ComputeLinkInputsTargets);
|
||||
ComputeISPCOutput;
|
||||
</ComputeLinkInputsTargets>
|
||||
<ComputeLibInputsTargets>
|
||||
$(ComputeLibInputsTargets);
|
||||
ComputeISPCOutput;
|
||||
</ComputeLibInputsTargets>
|
||||
</PropertyGroup>
|
||||
<Target
|
||||
Name="ComputeISPCOutput"
|
||||
Condition="'@(ISPC)' != ''">
|
||||
<ItemGroup>
|
||||
<ISPCDirsToMake
|
||||
Condition="'@(ISPC)' != '' and '%(ISPC.ExcludedFromBuild)' != 'true'"
|
||||
Include="%(ISPC.Outputs)" />
|
||||
<Link
|
||||
Include="%(ISPCDirsToMake.Identity)"
|
||||
Condition="'%(Extension)'=='.obj' or '%(Extension)'=='.res' or '%(Extension)'=='.rsc' or '%(Extension)'=='.lib'" />
|
||||
<Lib
|
||||
Include="%(ISPCDirsToMake.Identity)"
|
||||
Condition="'%(Extension)'=='.obj' or '%(Extension)'=='.res' or '%(Extension)'=='.rsc' or '%(Extension)'=='.lib'" />
|
||||
<ImpLib
|
||||
Include="%(ISPCDirsToMake.Identity)"
|
||||
Condition="'%(Extension)'=='.obj' or '%(Extension)'=='.res' or '%(Extension)'=='.rsc' or '%(Extension)'=='.lib'" />
|
||||
</ItemGroup>
|
||||
<MakeDir
|
||||
Directories="@(ISPCDirsToMake->'%(RootDir)%(Directory)')" />
|
||||
</Target>
|
||||
</Project>
|
||||
@@ -0,0 +1,296 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<ProjectSchemaDefinitions xmlns="clr-namespace:Microsoft.Build.Framework.XamlTypes;assembly=Microsoft.Build.Framework" xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" xmlns:sys="clr-namespace:System;assembly=mscorlib" xmlns:transformCallback="Microsoft.Cpp.Dev10.ConvertPropertyCallback">
|
||||
<Rule
|
||||
Name="ISPC"
|
||||
PageTemplate="tool"
|
||||
DisplayName="Intel SPMD Programm Compiler (ISPC)"
|
||||
Order="200">
|
||||
<Rule.DataSource>
|
||||
<DataSource
|
||||
Persistence="ProjectFile"
|
||||
ItemType="ISPC" />
|
||||
</Rule.DataSource>
|
||||
<Rule.Categories>
|
||||
<Category
|
||||
Name="General">
|
||||
<Category.DisplayName>
|
||||
<sys:String>General</sys:String>
|
||||
</Category.DisplayName>
|
||||
</Category>
|
||||
<Category
|
||||
Name="Command Line"
|
||||
Subtype="CommandLine">
|
||||
<Category.DisplayName>
|
||||
<sys:String>Command Line</sys:String>
|
||||
</Category.DisplayName>
|
||||
</Category>
|
||||
</Rule.Categories>
|
||||
<StringListProperty
|
||||
Name="Inputs"
|
||||
Category="Command Line"
|
||||
IsRequired="true"
|
||||
Switch=" ">
|
||||
<StringListProperty.DataSource>
|
||||
<DataSource
|
||||
Persistence="ProjectFile"
|
||||
ItemType="ISPC"
|
||||
SourceType="Item" />
|
||||
</StringListProperty.DataSource>
|
||||
</StringListProperty>
|
||||
<BoolProperty
|
||||
Name="GenerateDebugInformation"
|
||||
HelpContext="0"
|
||||
DisplayName="Generate Debug Information"
|
||||
Description="Generates Debug Information. (-g)"
|
||||
Switch="-g" />
|
||||
<BoolProperty
|
||||
Name="Assertions"
|
||||
HelpContext="0"
|
||||
DisplayName="Assertions"
|
||||
Description="Disables Assertions"
|
||||
Switch="--opt=disable-assertions" />
|
||||
<BoolProperty
|
||||
Name="Aligned"
|
||||
HelpContext="0"
|
||||
DisplayName="Force Aligned Loads"
|
||||
Description="Forces Aligned Loads."
|
||||
Switch="--opt=force-aligned-memory" />
|
||||
<BoolProperty
|
||||
Name="PIC"
|
||||
HelpContext="0"
|
||||
DisplayName="Position Independent Code"
|
||||
Description="Generate Position Independent Code"
|
||||
Switch="--pic" />
|
||||
<BoolProperty
|
||||
Name="WarningsAsErrors"
|
||||
HelpContext="0"
|
||||
DisplayName="Treat Warnings as Errors"
|
||||
Description="Treat Warnings as Errors."
|
||||
Switch="--werror" />
|
||||
<BoolProperty
|
||||
Name="NoWarnings"
|
||||
HelpContext="0"
|
||||
DisplayName="Disable Warnings"
|
||||
Description="Disable Warnings"
|
||||
Switch="--woff" />
|
||||
<StringProperty
|
||||
Name="ObjectFileName"
|
||||
Category="Output"
|
||||
HelpContext="0"
|
||||
DisplayName="Object File Name"
|
||||
Description="Specifies the name of the output object file."
|
||||
Switch="-o "[value]"" />
|
||||
<StringListProperty
|
||||
Name="PreprocessorDefinitions"
|
||||
HelpContext="0"
|
||||
DisplayName="Preprocessor Definitions"
|
||||
Description="Defines a text macro with the given name."
|
||||
Separator=";"
|
||||
Switch="-D"[value]"" />
|
||||
<StringListProperty
|
||||
Name="IncludePaths"
|
||||
HelpContext="0"
|
||||
DisplayName="Include Paths"
|
||||
Description="Sets path for include file. "
|
||||
Separator=";"
|
||||
Switch="-I "[value]"" />
|
||||
<StringProperty
|
||||
Name="HeaderFile"
|
||||
Category="Output"
|
||||
HelpContext="0"
|
||||
DisplayName="Header File Name"
|
||||
Description="Interface Header File."
|
||||
Switch="-h "[value]"" />
|
||||
<EnumProperty
|
||||
Name="Architecture"
|
||||
HelpContext="0"
|
||||
DisplayName="Target Architecture"
|
||||
Description="Select Target Architecture.">
|
||||
<EnumValue
|
||||
Name="0"
|
||||
DisplayName="x86"
|
||||
Switch="--arch=x86" />
|
||||
<EnumValue
|
||||
Name="1"
|
||||
DisplayName="x86-64"
|
||||
Switch="--arch=x86-64" />
|
||||
</EnumProperty>
|
||||
<EnumProperty
|
||||
Name="TargetISA"
|
||||
HelpContext="0"
|
||||
DisplayName="Target ISA"
|
||||
Description="Select Target ISA.">
|
||||
<EnumValue
|
||||
Name="0"
|
||||
DisplayName="SSE2"
|
||||
Switch="--target=sse2" />
|
||||
<EnumValue
|
||||
Name="1"
|
||||
DisplayName="SSE4"
|
||||
Switch="--target=sse4" />
|
||||
<EnumValue
|
||||
Name="2"
|
||||
DisplayName="AVX"
|
||||
Switch="--target=avx" />
|
||||
<EnumValue
|
||||
Name="3"
|
||||
DisplayName="AVX-I"
|
||||
Switch="--target=avx1.1" />
|
||||
<EnumValue
|
||||
Name="4"
|
||||
DisplayName="AVX2"
|
||||
Switch="--target=AVX2" />
|
||||
<EnumValue
|
||||
Name="5"
|
||||
DisplayName="SSE2,SSE4"
|
||||
Switch="--target=sse2,sse4" />
|
||||
<EnumValue
|
||||
Name="6"
|
||||
DisplayName="SSE2,SSE4,AVX"
|
||||
Switch="--target=sse2,sse4,avx" />
|
||||
<EnumValue
|
||||
Name="7"
|
||||
DisplayName="SSE2,SSE4,AVX,AVX2"
|
||||
Switch="--target=sse2,sse4,avx,avx2" />
|
||||
</EnumProperty>
|
||||
<EnumProperty
|
||||
Name="WarningLevel"
|
||||
HelpContext="0"
|
||||
DisplayName="Warning Level"
|
||||
Description="Sets the warning level.">
|
||||
<EnumValue
|
||||
Name="0"
|
||||
DisplayName="Warning Level 0"
|
||||
Switch="--woff" />
|
||||
<EnumValue
|
||||
Name="1"
|
||||
DisplayName="Warning Level 1"
|
||||
Switch="--wno-perf" />
|
||||
<EnumValue
|
||||
Name="2"
|
||||
DisplayName="Warning Level 2" />
|
||||
</EnumProperty>
|
||||
<EnumProperty
|
||||
Name="MathLibrary"
|
||||
HelpContext="0"
|
||||
DisplayName="Math Library"
|
||||
Description="Select Math Library">
|
||||
<EnumValue
|
||||
Name="0"
|
||||
DisplayName="ISPC Internal"
|
||||
Switch="--math-lib=default" />
|
||||
<EnumValue
|
||||
Name="1"
|
||||
DisplayName="Fast"
|
||||
Switch="--math-lib=fast" />
|
||||
<EnumValue
|
||||
Name="2"
|
||||
DisplayName="SVML"
|
||||
Switch="--math-lib=svml" />
|
||||
<EnumValue
|
||||
Name="3"
|
||||
DisplayName="System"
|
||||
Switch="--math-lib=system" />
|
||||
</EnumProperty>
|
||||
<EnumProperty
|
||||
Name="Opt"
|
||||
HelpContext="0"
|
||||
DisplayName="Optimization Level"
|
||||
Description="Optimization Level">
|
||||
<EnumValue
|
||||
Name="0"
|
||||
DisplayName="O0"
|
||||
Switch="-O0" />
|
||||
<EnumValue
|
||||
Name="1"
|
||||
DisplayName="O1"
|
||||
Switch="-O1" />
|
||||
</EnumProperty>
|
||||
<StringProperty
|
||||
Name="CommandLineTemplate"
|
||||
DisplayName="Command Line"
|
||||
Visible="False"
|
||||
IncludeInCommandLine="False" />
|
||||
<DynamicEnumProperty
|
||||
Name="ISPCBeforeTargets"
|
||||
Category="General"
|
||||
EnumProvider="Targets"
|
||||
IncludeInCommandLine="False">
|
||||
<DynamicEnumProperty.DisplayName>
|
||||
<sys:String>Execute Before</sys:String>
|
||||
</DynamicEnumProperty.DisplayName>
|
||||
<DynamicEnumProperty.Description>
|
||||
<sys:String>Specifies the targets for the build customization to run before.</sys:String>
|
||||
</DynamicEnumProperty.Description>
|
||||
<DynamicEnumProperty.ProviderSettings>
|
||||
<NameValuePair
|
||||
Name="Exclude"
|
||||
Value="^ISPCBeforeTargets|^Compute" />
|
||||
</DynamicEnumProperty.ProviderSettings>
|
||||
<DynamicEnumProperty.DataSource>
|
||||
<DataSource
|
||||
Persistence="ProjectFile"
|
||||
HasConfigurationCondition="true" />
|
||||
</DynamicEnumProperty.DataSource>
|
||||
</DynamicEnumProperty>
|
||||
<DynamicEnumProperty
|
||||
Name="ISPCAfterTargets"
|
||||
Category="General"
|
||||
EnumProvider="Targets"
|
||||
IncludeInCommandLine="False">
|
||||
<DynamicEnumProperty.DisplayName>
|
||||
<sys:String>Execute After</sys:String>
|
||||
</DynamicEnumProperty.DisplayName>
|
||||
<DynamicEnumProperty.Description>
|
||||
<sys:String>Specifies the targets for the build customization to run after.</sys:String>
|
||||
</DynamicEnumProperty.Description>
|
||||
<DynamicEnumProperty.ProviderSettings>
|
||||
<NameValuePair
|
||||
Name="Exclude"
|
||||
Value="^ISPCAfterTargets|^Compute" />
|
||||
</DynamicEnumProperty.ProviderSettings>
|
||||
<DynamicEnumProperty.DataSource>
|
||||
<DataSource
|
||||
Persistence="ProjectFile"
|
||||
ItemType=""
|
||||
HasConfigurationCondition="true" />
|
||||
</DynamicEnumProperty.DataSource>
|
||||
</DynamicEnumProperty>
|
||||
<StringListProperty
|
||||
Name="Outputs"
|
||||
DisplayName="Outputs"
|
||||
Visible="False"
|
||||
IncludeInCommandLine="False" />
|
||||
<StringProperty
|
||||
Name="ExecutionDescription"
|
||||
DisplayName="Execution Description"
|
||||
Visible="False"
|
||||
IncludeInCommandLine="False" />
|
||||
<StringListProperty
|
||||
Name="AdditionalDependencies"
|
||||
DisplayName="Additional Dependencies"
|
||||
IncludeInCommandLine="False"
|
||||
Visible="false" />
|
||||
<StringProperty
|
||||
Subtype="AdditionalOptions"
|
||||
Name="AdditionalOptions"
|
||||
Category="Command Line">
|
||||
<StringProperty.DisplayName>
|
||||
<sys:String>Additional Options</sys:String>
|
||||
</StringProperty.DisplayName>
|
||||
<StringProperty.Description>
|
||||
<sys:String>Additional Options</sys:String>
|
||||
</StringProperty.Description>
|
||||
</StringProperty>
|
||||
</Rule>
|
||||
<ItemType
|
||||
Name="ISPC"
|
||||
DisplayName="Intel SPMD Programm Compiler (ISPC)" />
|
||||
<FileExtension
|
||||
Name="*.ispc"
|
||||
ContentType="ISPC" />
|
||||
<ContentType
|
||||
Name="ISPC"
|
||||
DisplayName="Intel SPMD Programm Compiler (ISPC)"
|
||||
ItemType="ISPC" />
|
||||
</ProjectSchemaDefinitions>
|
||||
@@ -1,62 +0,0 @@
|
||||
## ======================================================================== ##
|
||||
## Copyright 2009-2017 Intel Corporation ##
|
||||
## ##
|
||||
## Licensed under the Apache License, Version 2.0 (the "License"); ##
|
||||
## you may not use this file except in compliance with the License. ##
|
||||
## You may obtain a copy of the License at ##
|
||||
## ##
|
||||
## http://www.apache.org/licenses/LICENSE-2.0 ##
|
||||
## ##
|
||||
## Unless required by applicable law or agreed to in writing, software ##
|
||||
## distributed under the License is distributed on an "AS IS" BASIS, ##
|
||||
## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ##
|
||||
## See the License for the specific language governing permissions and ##
|
||||
## limitations under the License. ##
|
||||
## ======================================================================== ##
|
||||
|
||||
SET(FLAGS_SSE2 "/D__SSE__ /D__SSE2__")
|
||||
SET(FLAGS_SSE42 "${FLAGS_SSE2} /D__SSE3__ /D__SSSE3__ /D__SSE4_1__ /D__SSE4_2__")
|
||||
SET(FLAGS_AVX "${FLAGS_SSE42} /arch:AVX")
|
||||
SET(FLAGS_AVX2 "${FLAGS_SSE42} /arch:AVX2")
|
||||
|
||||
SET(COMMON_CXX_FLAGS "")
|
||||
SET(COMMON_CXX_FLAGS "${COMMON_CXX_FLAGS} /EHsc") # catch C++ exceptions only and extern "C" functions never throw a C++ exception
|
||||
SET(COMMON_CXX_FLAGS "${COMMON_CXX_FLAGS} /MP") # compile source files in parallel
|
||||
SET(COMMON_CXX_FLAGS "${COMMON_CXX_FLAGS} /GR") # enable runtime type information (on by default)
|
||||
IF (EMBREE_STACK_PROTECTOR)
|
||||
SET(COMMON_CXX_FLAGS "${COMMON_CXX_FLAGS} /GS") # protects against return address overrides
|
||||
ELSE()
|
||||
SET(COMMON_CXX_FLAGS "${COMMON_CXX_FLAGS} /GS-") # do not protect against return address overrides
|
||||
ENDIF()
|
||||
SET(COMMON_CXX_FLAGS "${COMMON_CXX_FLAGS} ${FLAGS_SSE2}")
|
||||
|
||||
SET(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} ${COMMON_CXX_FLAGS}")
|
||||
SET(CMAKE_EXE_LINKER_FLAGS_DEBUG "${CMAKE_EXE_LINKER_FLAGS_DEBUG} /DEBUG") # generate debug information
|
||||
SET(CMAKE_SHARED_LINKER_FLAGS_DEBUG "${CMAKE_SHARED_LINKER_FLAGS_DEBUG} /DEBUG") # generate debug information
|
||||
|
||||
SET(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} ${COMMON_CXX_FLAGS}")
|
||||
SET(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} /Ox") # enable full optimizations
|
||||
SET(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} /Oi") # inline intrinsic functions
|
||||
SET(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} /Gy") # package individual functions
|
||||
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} ${COMMON_CXX_FLAGS}")
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} /Ox") # enable full optimizations
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} /Oi") # inline intrinsic functions
|
||||
SET(CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} /Gy") # package individual functions
|
||||
SET(CMAKE_EXE_LINKER_FLAGS_RELWITHDEBINFO "${CMAKE_EXE_LINKER_FLAGS_RELWITHDEBINFO} /DEBUG") # generate debug information
|
||||
SET(CMAKE_SHARED_LINKER_FLAGS_RELWITHDEBINFO "${CMAKE_SHARED_LINKER_FLAGS_RELWITHDEBINFO} /DEBUG") # generate debug information
|
||||
|
||||
SET(SECURE_LINKER_FLAGS "")
|
||||
SET(SECURE_LINKER_FLAGS "${SECURE_LINKER_FLAGS} /NXCompat") # compatible with data execution prevention (on by default)
|
||||
SET(SECURE_LINKER_FLAGS "${SECURE_LINKER_FLAGS} /DynamicBase") # random rebase of executable at load time
|
||||
IF (CMAKE_SIZEOF_VOID_P EQUAL 4)
|
||||
SET(SECURE_LINKER_FLAGS "${SECURE_LINKER_FLAGS} /SafeSEH") # invoke known exception handlers (Win32 only, x64 exception handlers are safe by design)
|
||||
ENDIF()
|
||||
SET(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${SECURE_LINKER_FLAGS}")
|
||||
SET(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} ${SECURE_LINKER_FLAGS}")
|
||||
|
||||
SET(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} /IGNORE:4217") # locally defined symbol XXX imported in function YYY (happens as the ISPC API layer uses exported library functions)
|
||||
SET(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} /IGNORE:4049") # warning LNK4049: locally defined symbol _rtcOccluded1M imported
|
||||
|
||||
INCLUDE(msvc_post)
|
||||
|
||||
@@ -1,26 +0,0 @@
|
||||
## ======================================================================== ##
|
||||
## Copyright 2009-2017 Intel Corporation ##
|
||||
## ##
|
||||
## Licensed under the Apache License, Version 2.0 (the "License"); ##
|
||||
## you may not use this file except in compliance with the License. ##
|
||||
## You may obtain a copy of the License at ##
|
||||
## ##
|
||||
## http://www.apache.org/licenses/LICENSE-2.0 ##
|
||||
## ##
|
||||
## Unless required by applicable law or agreed to in writing, software ##
|
||||
## distributed under the License is distributed on an "AS IS" BASIS, ##
|
||||
## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ##
|
||||
## See the License for the specific language governing permissions and ##
|
||||
## limitations under the License. ##
|
||||
## ======================================================================== ##
|
||||
|
||||
# optionally use static runtime library
|
||||
OPTION(EMBREE_STATIC_RUNTIME "Use the static version of the C/C++ runtime library." ON)
|
||||
IF (EMBREE_STATIC_RUNTIME)
|
||||
STRING(REPLACE "/MDd" "/MTd" CMAKE_CXX_FLAGS_DEBUG ${CMAKE_CXX_FLAGS_DEBUG})
|
||||
STRING(REPLACE "/MD" "/MT" CMAKE_CXX_FLAGS_RELWITHDEBINFO ${CMAKE_CXX_FLAGS_RELWITHDEBINFO})
|
||||
STRING(REPLACE "/MD" "/MT" CMAKE_CXX_FLAGS_RELEASE ${CMAKE_CXX_FLAGS_RELEASE})
|
||||
ENDIF()
|
||||
|
||||
# remove define NDEBUG and instead set define DEBUG for config RelWithDebInfo
|
||||
STRING(REPLACE "NDEBUG" "DEBUG" CMAKE_CXX_FLAGS_RELWITHDEBINFO ${CMAKE_CXX_FLAGS_RELWITHDEBINFO})
|
||||
@@ -1,266 +0,0 @@
|
||||
## ======================================================================== ##
|
||||
## Copyright 2009-2017 Intel Corporation ##
|
||||
## ##
|
||||
## Licensed under the Apache License, Version 2.0 (the "License"); ##
|
||||
## you may not use this file except in compliance with the License. ##
|
||||
## You may obtain a copy of the License at ##
|
||||
## ##
|
||||
## http://www.apache.org/licenses/LICENSE-2.0 ##
|
||||
## ##
|
||||
## Unless required by applicable law or agreed to in writing, software ##
|
||||
## distributed under the License is distributed on an "AS IS" BASIS, ##
|
||||
## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ##
|
||||
## See the License for the specific language governing permissions and ##
|
||||
## limitations under the License. ##
|
||||
## ======================================================================== ##
|
||||
|
||||
INCLUDE(GNUInstallDirs)
|
||||
|
||||
if (NOT DEFINED EMBREE_ZIP_MODE)
|
||||
set(EMBREE_ZIP_MODE OFF)
|
||||
endif()
|
||||
|
||||
IF (NOT EMBREE_ZIP_MODE AND NOT WIN32 AND NOT APPLE)
|
||||
SET(CMAKE_INSTALL_BINDIR ${CMAKE_INSTALL_BINDIR}/embree${EMBREE_VERSION_MAJOR})
|
||||
SET(CMAKE_INSTALL_FULL_BINDIR ${CMAKE_INSTALL_FULL_BINDIR}/embree${EMBREE_VERSION_MAJOR})
|
||||
ENDIF()
|
||||
|
||||
# use full absolute path as install name
|
||||
IF (NOT EMBREE_ZIP_MODE)
|
||||
SET(CMAKE_INSTALL_NAME_DIR ${CMAKE_INSTALL_FULL_LIBDIR})
|
||||
ELSE()
|
||||
IF(APPLE)
|
||||
SET(CMAKE_INSTALL_RPATH "@loader_path/../lib")
|
||||
ELSE()
|
||||
SET(CMAKE_INSTALL_RPATH "$ORIGIN/../lib")
|
||||
ENDIF()
|
||||
ENDIF()
|
||||
|
||||
##############################################################
|
||||
# Install Headers
|
||||
##############################################################
|
||||
INSTALL(DIRECTORY include/embree2 DESTINATION ${CMAKE_INSTALL_INCLUDEDIR} COMPONENT devel)
|
||||
|
||||
##############################################################
|
||||
# Install Models
|
||||
##############################################################
|
||||
INSTALL(DIRECTORY tutorials/models DESTINATION "${CMAKE_INSTALL_BINDIR}" COMPONENT examples)
|
||||
|
||||
##############################################################
|
||||
# Install Documentation
|
||||
##############################################################
|
||||
|
||||
INSTALL(FILES ${PROJECT_SOURCE_DIR}/LICENSE.txt DESTINATION ${CMAKE_INSTALL_DOCDIR} COMPONENT lib)
|
||||
INSTALL(FILES ${PROJECT_SOURCE_DIR}/CHANGELOG.md DESTINATION ${CMAKE_INSTALL_DOCDIR} COMPONENT lib)
|
||||
INSTALL(FILES ${PROJECT_SOURCE_DIR}/README.md DESTINATION ${CMAKE_INSTALL_DOCDIR} COMPONENT lib)
|
||||
INSTALL(FILES ${PROJECT_SOURCE_DIR}/readme.pdf DESTINATION ${CMAKE_INSTALL_DOCDIR} COMPONENT lib)
|
||||
|
||||
SET(CPACK_NSIS_MENU_LINKS ${CPACK_NSIS_MENU_LINKS} "${CMAKE_INSTALL_DOCDIR}/LICENSE.txt" "LICENSE")
|
||||
SET(CPACK_NSIS_MENU_LINKS ${CPACK_NSIS_MENU_LINKS} "${CMAKE_INSTALL_DOCDIR}/CHANGELOG.txt" "CHANGELOG")
|
||||
SET(CPACK_NSIS_MENU_LINKS ${CPACK_NSIS_MENU_LINKS} "${CMAKE_INSTALL_DOCDIR}/README.md" "README.md")
|
||||
SET(CPACK_NSIS_MENU_LINKS ${CPACK_NSIS_MENU_LINKS} "${CMAKE_INSTALL_DOCDIR}/readme.pdf" "readme.pdf")
|
||||
|
||||
##############################################################
|
||||
# Install scripts to set embree paths
|
||||
##############################################################
|
||||
|
||||
IF (EMBREE_ZIP_MODE)
|
||||
IF (WIN32)
|
||||
ELSEIF(APPLE)
|
||||
CONFIGURE_FILE(${PROJECT_SOURCE_DIR}/scripts/install_macosx/embree-vars.sh embree-vars.sh @ONLY)
|
||||
CONFIGURE_FILE(${PROJECT_SOURCE_DIR}/scripts/install_macosx/embree-vars.csh embree-vars.csh @ONLY)
|
||||
INSTALL(FILES ${PROJECT_BINARY_DIR}/embree-vars.sh DESTINATION "." COMPONENT lib)
|
||||
INSTALL(FILES ${PROJECT_BINARY_DIR}/embree-vars.csh DESTINATION "." COMPONENT lib)
|
||||
ELSE()
|
||||
CONFIGURE_FILE(${PROJECT_SOURCE_DIR}/scripts/install_linux/embree-vars.sh embree-vars.sh @ONLY)
|
||||
CONFIGURE_FILE(${PROJECT_SOURCE_DIR}/scripts/install_linux/embree-vars.csh embree-vars.csh @ONLY)
|
||||
INSTALL(FILES ${PROJECT_BINARY_DIR}/embree-vars.sh DESTINATION "." COMPONENT lib)
|
||||
INSTALL(FILES ${PROJECT_BINARY_DIR}/embree-vars.csh DESTINATION "." COMPONENT lib)
|
||||
ENDIF()
|
||||
ENDIF()
|
||||
|
||||
##############################################################
|
||||
# Install Embree CMake Configuration
|
||||
##############################################################
|
||||
|
||||
IF (NOT EMBREE_ZIP_MODE)
|
||||
SET(EMBREE_CONFIG_VERSION ${EMBREE_VERSION})
|
||||
ELSE()
|
||||
SET(EMBREE_CONFIG_VERSION ${EMBREE_VERSION_MAJOR})
|
||||
ENDIF()
|
||||
|
||||
IF (APPLE AND NOT EMBREE_ZIP_MODE)
|
||||
CONFIGURE_FILE(scripts/install_macosx/uninstall.command uninstall.command @ONLY)
|
||||
INSTALL(PROGRAMS "${PROJECT_BINARY_DIR}/uninstall.command" DESTINATION ${CMAKE_INSTALL_BINDIR}/.. COMPONENT lib)
|
||||
ENDIF()
|
||||
|
||||
# why does this have to be so complicated...
|
||||
IF (WIN32)
|
||||
SET(EMBREE_LIBRARY_NAME ${CMAKE_IMPORT_LIBRARY_PREFIX}embree${CMAKE_IMPORT_LIBRARY_SUFFIX})
|
||||
ELSEIF (APPLE)
|
||||
SET(EMBREE_LIBRARY_NAME ${CMAKE_SHARED_LIBRARY_PREFIX}embree.${EMBREE_CONFIG_VERSION}${CMAKE_SHARED_LIBRARY_SUFFIX})
|
||||
ELSE()
|
||||
SET(EMBREE_LIBRARY_NAME ${CMAKE_SHARED_LIBRARY_PREFIX}embree${CMAKE_SHARED_LIBRARY_SUFFIX}.${EMBREE_CONFIG_VERSION})
|
||||
ENDIF()
|
||||
|
||||
IF (WIN32 OR EMBREE_ZIP_MODE)
|
||||
# for local "installs" and on Windows we want the cmake config files placed
|
||||
# in the install root, such that users can point the CMake variable
|
||||
# embree_DIR just to the install folder
|
||||
SET(EMBREE_CMAKECONFIG_DIR ".")
|
||||
SET(EMBREE_RELATIV_ROOT_DIR ".")
|
||||
ELSE()
|
||||
SET(EMBREE_CMAKECONFIG_DIR "${CMAKE_INSTALL_LIBDIR}/cmake/embree-${EMBREE_VERSION}")
|
||||
SET(EMBREE_RELATIV_ROOT_DIR "../../..")
|
||||
ENDIF()
|
||||
CONFIGURE_FILE(common/cmake/embree-config.cmake embree-config-install.cmake @ONLY)
|
||||
CONFIGURE_FILE(common/cmake/embree-config-version.cmake embree-config-version.cmake @ONLY)
|
||||
# create a config file for the build directory
|
||||
CONFIGURE_FILE(common/cmake/embree-config-builddir.cmake embree-config.cmake @ONLY)
|
||||
|
||||
INSTALL(FILES "${PROJECT_BINARY_DIR}/embree-config-install.cmake" DESTINATION ${EMBREE_CMAKECONFIG_DIR} RENAME "embree-config.cmake" COMPONENT devel)
|
||||
INSTALL(FILES "${PROJECT_BINARY_DIR}/embree-config-version.cmake" DESTINATION ${EMBREE_CMAKECONFIG_DIR} COMPONENT devel)
|
||||
|
||||
##############################################################
|
||||
# CPack specific stuff
|
||||
##############################################################
|
||||
|
||||
SET(CPACK_PACKAGE_NAME "Embree")
|
||||
SET(CPACK_PACKAGE_FILE_NAME "embree-${EMBREE_VERSION}")
|
||||
#SET(CPACK_PACKAGE_ICON ${PROJECT_SOURCE_DIR}/embree-doc/images/icon.png)
|
||||
#SET(CPACK_PACKAGE_RELOCATABLE TRUE)
|
||||
SET(CPACK_STRIP_FILES TRUE)
|
||||
|
||||
SET(CPACK_PACKAGE_VERSION_MAJOR ${EMBREE_VERSION_MAJOR})
|
||||
SET(CPACK_PACKAGE_VERSION_MINOR ${EMBREE_VERSION_MINOR})
|
||||
SET(CPACK_PACKAGE_VERSION_PATCH ${EMBREE_VERSION_PATCH})
|
||||
SET(CPACK_PACKAGE_DESCRIPTION_SUMMARY "Embree: High Performance Ray Tracing Kernels")
|
||||
SET(CPACK_PACKAGE_VENDOR "Intel Corporation")
|
||||
SET(CPACK_PACKAGE_CONTACT embree_support@intel.com)
|
||||
|
||||
SET(CPACK_COMPONENT_LIB_DISPLAY_NAME "Library")
|
||||
SET(CPACK_COMPONENT_LIB_DESCRIPTION "The Embree library including documentation.")
|
||||
|
||||
SET(CPACK_COMPONENT_DEVEL_DISPLAY_NAME "Development")
|
||||
SET(CPACK_COMPONENT_DEVEL_DESCRIPTION "Header Files for C and ISPC required to develop applications with Embree.")
|
||||
|
||||
SET(CPACK_COMPONENT_EXAMPLES_DISPLAY_NAME "Examples")
|
||||
SET(CPACK_COMPONENT_EXAMPLES_DESCRIPTION "Tutorials demonstrating how to use Embree.")
|
||||
|
||||
# dependencies between components
|
||||
SET(CPACK_COMPONENT_DEVEL_DEPENDS lib)
|
||||
SET(CPACK_COMPONENT_EXAMPLES_DEPENDS lib)
|
||||
SET(CPACK_COMPONENT_LIB_REQUIRED ON) # always install the libs
|
||||
|
||||
# point to readme and license files
|
||||
SET(CPACK_RESOURCE_FILE_README ${PROJECT_SOURCE_DIR}/README.md)
|
||||
SET(CPACK_RESOURCE_FILE_LICENSE ${PROJECT_SOURCE_DIR}/LICENSE.txt)
|
||||
|
||||
# Windows specific settings
|
||||
IF(WIN32)
|
||||
|
||||
IF (CMAKE_SIZEOF_VOID_P EQUAL 8)
|
||||
SET(ARCH x64)
|
||||
SET(PROGRAMFILES "\$PROGRAMFILES64")
|
||||
SET(CPACK_PACKAGE_NAME "${CPACK_PACKAGE_NAME} x64")
|
||||
ELSE()
|
||||
SET(ARCH win32)
|
||||
SET(PROGRAMFILES "\$PROGRAMFILES")
|
||||
SET(CPACK_PACKAGE_NAME "${CPACK_PACKAGE_NAME} Win32")
|
||||
ENDIF()
|
||||
|
||||
# NSIS specific settings
|
||||
IF (NOT EMBREE_ZIP_MODE)
|
||||
SET(CPACK_GENERATOR NSIS)
|
||||
SET(CPACK_PACKAGE_FILE_NAME "${CPACK_PACKAGE_FILE_NAME}.${ARCH}")
|
||||
SET(CPACK_COMPONENTS_ALL lib devel examples)
|
||||
SET(CPACK_NSIS_INSTALL_ROOT "${PROGRAMFILES}\\\\Intel")
|
||||
SET(CPACK_PACKAGE_INSTALL_DIRECTORY "Embree v${EMBREE_VERSION} ${ARCH}")
|
||||
SET(CPACK_NSIS_DISPLAY_NAME "Embree v${EMBREE_VERSION} ${ARCH}")
|
||||
SET(CPACK_NSIS_PACKAGE_NAME "Embree v${EMBREE_VERSION} ${ARCH}")
|
||||
SET(CPACK_NSIS_URL_INFO_ABOUT http://embree.github.io/)
|
||||
#SET(CPACK_NSIS_HELP_LINK http://embree.github.io/downloads.html#windows)
|
||||
SET(CPACK_NSIS_MUI_ICON ${PROJECT_SOURCE_DIR}/scripts/install_windows/icon32.ico)
|
||||
SET(CPACK_NSIS_CONTACT ${CPACK_PACKAGE_CONTACT})
|
||||
#SET(CPACK_NSIS_EXTRA_PREINSTALL_COMMANDS ${CPACK_NSIS_EXTRA_PREINSTALL_COMMANDS} "\n CreateDirectory \\\"$SMPROGRAMS\\\\$STARTMENU_FOLDER\\\"")
|
||||
#SET(CPACK_NSIS_EXTRA_PREINSTALL_COMMANDS ${CPACK_NSIS_EXTRA_PREINSTALL_COMMANDS} "\n CreateDirectory \\\"$SMPROGRAMS\\\\$STARTMENU_FOLDER\\\\documentation\\\"")
|
||||
#SET(CPACK_NSIS_EXTRA_PREINSTALL_COMMANDS ${CPACK_NSIS_EXTRA_PREINSTALL_COMMANDS} "\n CreateDirectory \\\"$SMPROGRAMS\\\\$STARTMENU_FOLDER\\\\tutorials\\\" \n")
|
||||
#SET(CPACK_NSIS_EXTRA_UNINSTALL_COMMANDS ${CPACK_NSIS_EXTRA_UNINSTALL_COMMANDS} "\n ${UNINSTALL_LIST}\n RMDir \\\"$SMPROGRAMS\\\\$STARTMENU_FOLDER\\\\documentation\\\"")
|
||||
#SET(CPACK_NSIS_EXTRA_UNINSTALL_COMMANDS ${CPACK_NSIS_EXTRA_UNINSTALL_COMMANDS} "\n ${UNINSTALL_LIST}\n RMDir \\\"$SMPROGRAMS\\\\$STARTMENU_FOLDER\\\\tutorials\\\"")
|
||||
#SET(CPACK_NSIS_EXTRA_UNINSTALL_COMMANDS ${CPACK_NSIS_EXTRA_UNINSTALL_COMMANDS} "\n ${UNINSTALL_LIST}\n RMDir \\\"$SMPROGRAMS\\\\$STARTMENU_FOLDER\\\"\n")
|
||||
IF (EMBREE_TESTING_PACKAGE)
|
||||
ADD_TEST(NAME "BuildPackage" WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR} COMMAND ${PROJECT_SOURCE_DIR}/scripts/package_win.bat ${CPACK_PACKAGE_FILE_NAME}.exe ${CMAKE_BUILD_TYPE})
|
||||
ENDIF()
|
||||
ELSE()
|
||||
SET(CPACK_GENERATOR ZIP)
|
||||
SET(CPACK_PACKAGE_FILE_NAME "${CPACK_PACKAGE_FILE_NAME}.${ARCH}.windows")
|
||||
SET(CPACK_MONOLITHIC_INSTALL 1)
|
||||
IF (EMBREE_TESTING_PACKAGE)
|
||||
ADD_TEST(NAME "BuildPackage" WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR} COMMAND ${PROJECT_SOURCE_DIR}/scripts/package_win.bat ${CPACK_PACKAGE_FILE_NAME}.zip ${CMAKE_BUILD_TYPE})
|
||||
ENDIF()
|
||||
ENDIF()
|
||||
|
||||
# MacOSX specific settings
|
||||
ELSEIF(APPLE)
|
||||
|
||||
CONFIGURE_FILE(README.md README.txt)
|
||||
SET(CPACK_RESOURCE_FILE_README ${PROJECT_BINARY_DIR}/README.txt)
|
||||
|
||||
IF (NOT EMBREE_ZIP_MODE)
|
||||
SET(CPACK_PACKAGING_INSTALL_PREFIX ${CMAKE_INSTALL_PREFIX})
|
||||
SET(CPACK_GENERATOR PackageMaker)
|
||||
SET(CPACK_PACKAGE_FILE_NAME "${CPACK_PACKAGE_FILE_NAME}.x86_64")
|
||||
#SET(CPACK_COMPONENTS_ALL lib devel examples)
|
||||
SET(CPACK_MONOLITHIC_INSTALL 1)
|
||||
SET(CPACK_PACKAGE_NAME embree-${EMBREE_VERSION})
|
||||
SET(CPACK_PACKAGE_VENDOR "intel") # creates short name com.intel.embree2.xxx in pkgutil
|
||||
SET(CPACK_OSX_PACKAGE_VERSION 10.7)
|
||||
IF (EMBREE_TESTING_PACKAGE)
|
||||
ADD_TEST(NAME "BuildPackage" WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR} COMMAND ${PROJECT_SOURCE_DIR}/scripts/package_macosx.sh ${CMAKE_BUILD_TYPE} ${CPACK_PACKAGE_FILE_NAME}.dmg)
|
||||
ENDIF()
|
||||
ELSE()
|
||||
SET(CPACK_GENERATOR TGZ)
|
||||
SET(CPACK_PACKAGE_FILE_NAME "${CPACK_PACKAGE_FILE_NAME}.x86_64.macosx")
|
||||
SET(CPACK_MONOLITHIC_INSTALL 1)
|
||||
IF (EMBREE_TESTING_PACKAGE)
|
||||
ADD_TEST(NAME "BuildPackage" WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR} COMMAND ${PROJECT_SOURCE_DIR}/scripts/package_macosx.sh ${CMAKE_BUILD_TYPE} ${CPACK_PACKAGE_FILE_NAME}.tgz)
|
||||
ENDIF()
|
||||
ENDIF()
|
||||
|
||||
# Linux specific settings
|
||||
ELSE()
|
||||
|
||||
|
||||
IF (NOT EMBREE_ZIP_MODE)
|
||||
|
||||
SET(CPACK_GENERATOR RPM)
|
||||
SET(CPACK_COMPONENTS_ALL devel lib examples)
|
||||
SET(CPACK_RPM_COMPONENT_INSTALL ON)
|
||||
SET(CPACK_RPM_FILE_NAME RPM-DEFAULT)
|
||||
SET(CPACK_RPM_devel_PACKAGE_ARCHITECTURE noarch)
|
||||
SET(CPACK_RPM_PACKAGE_LICENSE "ASL 2.0") # Apache Software License, Version 2.0
|
||||
SET(CPACK_RPM_PACKAGE_GROUP "Development/Libraries")
|
||||
SET(CPACK_RPM_CHANGELOG_FILE ${CMAKE_BINARY_DIR}/rpm_changelog.txt) # ChangeLog of the RPM
|
||||
STRING(TIMESTAMP CHANGELOG_DATE "%a %b %d %Y")
|
||||
SET(RPM_CHANGELOG "* ${CHANGELOG_DATE} Johannes Günther <johannes.guenther@intel.com> - ${EMBREE_VERSION}-${CPACK_RPM_PACKAGE_RELEASE}\n- First package")
|
||||
FILE(WRITE ${CPACK_RPM_CHANGELOG_FILE} ${RPM_CHANGELOG})
|
||||
SET(CPACK_RPM_PACKAGE_URL http://embree.github.io/)
|
||||
SET(CPACK_RPM_DEFAULT_DIR_PERMISSIONS OWNER_READ OWNER_WRITE OWNER_EXECUTE GROUP_READ GROUP_EXECUTE WORLD_READ WORLD_EXECUTE) # otherwise cmake directory has wrong 775 permissions
|
||||
|
||||
# post install and uninstall scripts
|
||||
SET(CPACK_RPM_lib_POST_INSTALL_SCRIPT_FILE ${PROJECT_SOURCE_DIR}/common/cmake/rpm_ldconfig.sh)
|
||||
SET(CPACK_RPM_lib_POST_UNINSTALL_SCRIPT_FILE ${PROJECT_SOURCE_DIR}/common/cmake/rpm_ldconfig.sh)
|
||||
IF (EMBREE_TESTING_PACKAGE)
|
||||
ADD_TEST(NAME "BuildPackage" WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR} COMMAND ${PROJECT_SOURCE_DIR}/scripts/package_linux.sh ${EMBREE_ZIP_MODE} ${EMBREE_VERSION})
|
||||
ENDIF()
|
||||
ELSE()
|
||||
|
||||
SET(CPACK_GENERATOR TGZ)
|
||||
SET(CPACK_PACKAGE_FILE_NAME "${CPACK_PACKAGE_FILE_NAME}.x86_64.linux")
|
||||
SET(CPACK_MONOLITHIC_INSTALL 1)
|
||||
IF (EMBREE_TESTING_PACKAGE)
|
||||
ADD_TEST(NAME "BuildPackage" WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR} COMMAND ${PROJECT_SOURCE_DIR}/scripts/package_linux.sh ${EMBREE_ZIP_MODE} ${EMBREE_VERSION})
|
||||
ENDIF()
|
||||
ENDIF()
|
||||
|
||||
ENDIF()
|
||||
@@ -1 +0,0 @@
|
||||
/sbin/ldconfig
|
||||
@@ -1,148 +0,0 @@
|
||||
## ======================================================================== ##
|
||||
## Copyright 2009-2017 Intel Corporation ##
|
||||
## ##
|
||||
## Licensed under the Apache License, Version 2.0 (the "License"); ##
|
||||
## you may not use this file except in compliance with the License. ##
|
||||
## You may obtain a copy of the License at ##
|
||||
## ##
|
||||
## http://www.apache.org/licenses/LICENSE-2.0 ##
|
||||
## ##
|
||||
## Unless required by applicable law or agreed to in writing, software ##
|
||||
## distributed under the License is distributed on an "AS IS" BASIS, ##
|
||||
## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ##
|
||||
## See the License for the specific language governing permissions and ##
|
||||
## limitations under the License. ##
|
||||
## ======================================================================== ##
|
||||
|
||||
IF(NOT BUILD_TESTING)
|
||||
SET(BUILD_TESTING OFF CACHE BOOL "Build the testing tree.")
|
||||
ENDIF()
|
||||
|
||||
INCLUDE(CTest)
|
||||
|
||||
IF (BUILD_TESTING)
|
||||
|
||||
CMAKE_HOST_SYSTEM_INFORMATION(RESULT memory QUERY TOTAL_PHYSICAL_MEMORY)
|
||||
|
||||
SET(EMBREE_TESTING_MODEL_DIR "${PROJECT_SOURCE_DIR}/models" CACHE PATH "Path to the folder containing the Embree models for regression testing.")
|
||||
SET(EMBREE_TESTING_INTENSIVE OFF CACHE BOOL "Turns intensive testing on.")
|
||||
SET(EMBREE_TESTING_MEMCHECK OFF CACHE BOOL "Turns on memory checking for some tests.")
|
||||
SET(EMBREE_TESTING_BENCHMARK OFF CACHE BOOL "Turns benchmarking on.")
|
||||
SET(EMBREE_TESTING_BENCHMARK_DATABASE "${PROJECT_BINARY_DIR}" CACHE PATH "Path to database for benchmarking.")
|
||||
SET(EMBREE_TESTING_PACKAGE OFF CACHE BOOL "Packages release as test.")
|
||||
SET(EMBREE_TESTING_KLOCWORK OFF CACHE BOOL "Runs Kocwork as test.")
|
||||
|
||||
SET(EMBREE_TESTING_SDE OFF CACHE STRING "Uses SDE to run tests for specified CPU.")
|
||||
SET_PROPERTY(CACHE EMBREE_TESTING_SDE PROPERTY STRINGS OFF pnr nhm wsm snb ivb hsw bdw knl skl skx cnl)
|
||||
|
||||
IF( NOT EXISTS "${EMBREE_TESTING_MODEL_DIR}/embree-models-subdiv.txt"
|
||||
OR NOT EXISTS "${EMBREE_TESTING_MODEL_DIR}/embree-models-small-win32.txt"
|
||||
OR NOT EXISTS "${EMBREE_TESTING_MODEL_DIR}/embree-models-small-x64.txt"
|
||||
OR NOT EXISTS "${EMBREE_TESTING_MODEL_DIR}/embree-models-large.txt")
|
||||
MESSAGE(FATAL_ERROR "Invalid Embree testing model repository. Either disable BUILD_TESTING or properly set EMBREE_TESTING_MODEL_DIR.")
|
||||
ENDIF()
|
||||
|
||||
FILE(READ "${EMBREE_TESTING_MODEL_DIR}/embree-models-subdiv.txt" models_subdiv)
|
||||
STRING(REGEX REPLACE "\n" ";" models_subdiv "${models_subdiv}")
|
||||
|
||||
FILE(READ "${EMBREE_TESTING_MODEL_DIR}/embree-models-small-win32.txt" models_small_win32)
|
||||
STRING(REGEX REPLACE "\n" ";" models_small_win32 "${models_small_win32}")
|
||||
|
||||
FILE(READ "${EMBREE_TESTING_MODEL_DIR}/embree-models-small-x64.txt" models_small_x64)
|
||||
STRING(REGEX REPLACE "\n" ";" models_small_x64 "${models_small_x64}")
|
||||
|
||||
FILE(READ "${EMBREE_TESTING_MODEL_DIR}/embree-models-large.txt" models_large)
|
||||
STRING(REGEX REPLACE "\n" ";" models_large "${models_large}")
|
||||
|
||||
IF (EMBREE_TESTING_INTENSIVE)
|
||||
SET(models ${models_small_x64})
|
||||
IF (${memory} GREATER 10000)
|
||||
SET(models ${models} ${models_large})
|
||||
ENDIF()
|
||||
ELSE()
|
||||
SET(models ${models_small_win32})
|
||||
ENDIF()
|
||||
|
||||
IF (WIN32)
|
||||
SET(MY_PROJECT_BINARY_DIR "${CMAKE_CURRENT_BINARY_DIR}/${CMAKE_BUILD_TYPE}")
|
||||
ELSE()
|
||||
SET(MY_PROJECT_BINARY_DIR "${CMAKE_CURRENT_BINARY_DIR}")
|
||||
ENDIF()
|
||||
|
||||
MACRO (ADD_EMBREE_MODEL_TEST name reference executable args model)
|
||||
|
||||
ADD_TEST(NAME ${name}
|
||||
WORKING_DIRECTORY ${MY_PROJECT_BINARY_DIR}
|
||||
COMMAND python ${PROJECT_SOURCE_DIR}/scripts/invoke_test.py
|
||||
--name ${name}
|
||||
--modeldir ${EMBREE_TESTING_MODEL_DIR}
|
||||
--reference ${reference}
|
||||
--model ${model}
|
||||
--sde ${EMBREE_TESTING_SDE}
|
||||
--execute ${MY_PROJECT_BINARY_DIR}/${executable} ${args})
|
||||
|
||||
IF (EMBREE_ISPC_SUPPORT AND EMBREE_RAY_PACKETS)
|
||||
ADD_TEST(NAME ${name}_ispc
|
||||
WORKING_DIRECTORY ${MY_PROJECT_BINARY_DIR}
|
||||
COMMAND python ${PROJECT_SOURCE_DIR}/scripts/invoke_test.py
|
||||
--name ${name}_ispc
|
||||
--modeldir ${EMBREE_TESTING_MODEL_DIR}
|
||||
--reference ${reference}
|
||||
--model ${model}
|
||||
--sde ${EMBREE_TESTING_SDE}
|
||||
--execute ${MY_PROJECT_BINARY_DIR}/${executable}_ispc ${args})
|
||||
ENDIF()
|
||||
ENDMACRO()
|
||||
|
||||
MACRO (ADD_EMBREE_MODELS_TEST name reference executable)
|
||||
FOREACH (model ${models})
|
||||
STRING(REGEX REPLACE "/" "_" modelname "${model}")
|
||||
STRING(REGEX REPLACE ".ecs" "" modelname "${modelname}")
|
||||
ADD_EMBREE_MODEL_TEST(${name}_${modelname} ${reference}_${modelname} ${executable} "${ARGN}" ${model})
|
||||
ENDFOREACH()
|
||||
ENDMACRO()
|
||||
|
||||
MACRO (ADD_EMBREE_SUBDIV_MODELS_TEST name reference executable)
|
||||
FOREACH (model ${models_subdiv})
|
||||
STRING(REGEX REPLACE "/" "_" modelname "${model}")
|
||||
STRING(REGEX REPLACE ".ecs" "" modelname "${modelname}")
|
||||
ADD_EMBREE_MODEL_TEST(${name}_${modelname} ${reference}_${modelname} ${executable} "${ARGN}" ${model})
|
||||
ENDFOREACH()
|
||||
ENDMACRO()
|
||||
|
||||
IF (EMBREE_TESTING_KLOCWORK)
|
||||
ADD_TEST(NAME Klocwork WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR} COMMAND ${PROJECT_SOURCE_DIR}/scripts/klocwork.sh)
|
||||
SET_TESTS_PROPERTIES(Klocwork PROPERTIES TIMEOUT 2400)
|
||||
ENDIF()
|
||||
|
||||
ELSE()
|
||||
|
||||
MACRO (ADD_EMBREE_MODEL_TEST name reference executable args model)
|
||||
ENDMACRO()
|
||||
|
||||
MACRO (ADD_EMBREE_SUBDIV_MODELS_TEST name reference executable)
|
||||
ENDMACRO()
|
||||
|
||||
MACRO (ADD_EMBREE_MODELS_TEST name)
|
||||
ENDMACRO()
|
||||
|
||||
ENDIF()
|
||||
|
||||
MACRO (ADD_EMBREE_TEST name)
|
||||
ADD_EMBREE_MODEL_TEST(${name} ${name} ${name} "" "default")
|
||||
ENDMACRO()
|
||||
|
||||
MACRO (ADD_EMBREE_TEST2 name exe args)
|
||||
ADD_EMBREE_MODEL_TEST(${name} ${exe} ${exe} "${args}" "default")
|
||||
ENDMACRO()
|
||||
|
||||
IF (EMBREE_TESTING_MEMCHECK)
|
||||
find_program( EMBREE_MEMORYCHECK_COMMAND valgrind )
|
||||
set( EMBREE_MEMORYCHECK_COMMAND_OPTIONS "--trace-children=yes --leak-check=full --show-leak-kinds=definite --errors-for-leak-kinds=definite --error-exitcode=1" )
|
||||
|
||||
FUNCTION(ADD_MEMCHECK_TEST name binary)
|
||||
set(memcheck_command "${EMBREE_MEMORYCHECK_COMMAND} ${EMBREE_MEMORYCHECK_COMMAND_OPTIONS}")
|
||||
separate_arguments(memcheck_command)
|
||||
add_test(NAME ${name} COMMAND ${memcheck_command} ${MY_PROJECT_BINARY_DIR}/${binary} ${ARGN})
|
||||
ENDFUNCTION()
|
||||
ENDIF()
|
||||
@@ -1,38 +0,0 @@
|
||||
## ======================================================================== ##
|
||||
## Copyright 2009-2017 Intel Corporation ##
|
||||
## ##
|
||||
## Licensed under the Apache License, Version 2.0 (the "License"); ##
|
||||
## you may not use this file except in compliance with the License. ##
|
||||
## You may obtain a copy of the License at ##
|
||||
## ##
|
||||
## http://www.apache.org/licenses/LICENSE-2.0 ##
|
||||
## ##
|
||||
## Unless required by applicable law or agreed to in writing, software ##
|
||||
## distributed under the License is distributed on an "AS IS" BASIS, ##
|
||||
## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ##
|
||||
## See the License for the specific language governing permissions and ##
|
||||
## limitations under the License. ##
|
||||
## ======================================================================== ##
|
||||
|
||||
# additional parameters (beyond the name) are treated as additional dependencies
|
||||
# if ADDITIONAL_LIBRARIES is set these will be included during linking
|
||||
|
||||
MACRO (ADD_TUTORIAL TUTORIAL_NAME)
|
||||
|
||||
ADD_EXECUTABLE(${TUTORIAL_NAME} ${TUTORIAL_NAME}.cpp ${TUTORIAL_NAME}_device.cpp ${ARGN})
|
||||
TARGET_LINK_LIBRARIES(${TUTORIAL_NAME} embree tutorial image tutorial_device noise ${ADDITIONAL_LIBRARIES})
|
||||
SET_PROPERTY(TARGET ${TUTORIAL_NAME} PROPERTY FOLDER tutorials/single)
|
||||
INSTALL(TARGETS ${TUTORIAL_NAME} DESTINATION ${CMAKE_INSTALL_BINDIR} COMPONENT examples)
|
||||
SET(CPACK_NSIS_MENU_LINKS ${CPACK_NSIS_MENU_LINKS} "${CMAKE_INSTALL_BINDIR}/${TUTORIAL_NAME}" "${TUTORIAL_NAME}")
|
||||
|
||||
IF (EMBREE_ISPC_SUPPORT AND EMBREE_RAY_PACKETS)
|
||||
ADD_EMBREE_ISPC_EXECUTABLE(${TUTORIAL_NAME}_ispc ${TUTORIAL_NAME}.cpp ${TUTORIAL_NAME}_device.ispc)
|
||||
TARGET_LINK_LIBRARIES(${TUTORIAL_NAME}_ispc embree tutorial image tutorial_device_ispc noise noise_ispc)
|
||||
SET_PROPERTY(TARGET ${TUTORIAL_NAME}_ispc PROPERTY FOLDER tutorials/ispc)
|
||||
INSTALL(TARGETS ${TUTORIAL_NAME}_ispc DESTINATION ${CMAKE_INSTALL_BINDIR} COMPONENT examples)
|
||||
SET(CPACK_NSIS_MENU_LINKS ${CPACK_NSIS_MENU_LINKS} "${CMAKE_INSTALL_BINDIR}/${TUTORIAL_NAME}_ispc" "${TUTORIAL_NAME}_ispc")
|
||||
ENDIF()
|
||||
|
||||
SET(CPACK_NSIS_MENU_LINKS ${CPACK_NSIS_MENU_LINKS} PARENT_SCOPE)
|
||||
|
||||
ENDMACRO ()
|
||||
@@ -1,37 +0,0 @@
|
||||
## ======================================================================== ##
|
||||
## Copyright 2009-2015 Intel Corporation ##
|
||||
## ##
|
||||
## Licensed under the Apache License, Version 2.0 (the "License"); ##
|
||||
## you may not use this file except in compliance with the License. ##
|
||||
## You may obtain a copy of the License at ##
|
||||
## ##
|
||||
## http://www.apache.org/licenses/LICENSE-2.0 ##
|
||||
## ##
|
||||
## Unless required by applicable law or agreed to in writing, software ##
|
||||
## distributed under the License is distributed on an "AS IS" BASIS, ##
|
||||
## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ##
|
||||
## See the License for the specific language governing permissions and ##
|
||||
## limitations under the License. ##
|
||||
## ======================================================================== ##
|
||||
|
||||
IF(NOT EXISTS "@CMAKE_CURRENT_BINARY_DIR@/install_manifest.txt")
|
||||
MESSAGE(FATAL_ERROR "Cannot find install manifest: @CMAKE_CURRENT_BINARY_DIR@/install_manifest.txt")
|
||||
ENDIF(NOT EXISTS "@CMAKE_CURRENT_BINARY_DIR@/install_manifest.txt")
|
||||
|
||||
FILE(READ "@CMAKE_CURRENT_BINARY_DIR@/install_manifest.txt" files)
|
||||
STRING(REGEX REPLACE "\n" ";" files "${files}")
|
||||
FOREACH(file ${files})
|
||||
MESSAGE(STATUS "Uninstalling $ENV{DESTDIR}${file}")
|
||||
IF(IS_SYMLINK "$ENV{DESTDIR}${file}" OR EXISTS "$ENV{DESTDIR}${file}")
|
||||
EXEC_PROGRAM(
|
||||
"@CMAKE_COMMAND@" ARGS "-E remove \"$ENV{DESTDIR}${file}\""
|
||||
OUTPUT_VARIABLE rm_out
|
||||
RETURN_VALUE rm_retval
|
||||
)
|
||||
IF(NOT "${rm_retval}" STREQUAL 0)
|
||||
MESSAGE(FATAL_ERROR "Problem when removing $ENV{DESTDIR}${file}")
|
||||
ENDIF(NOT "${rm_retval}" STREQUAL 0)
|
||||
ELSE(IS_SYMLINK "$ENV{DESTDIR}${file}" OR EXISTS "$ENV{DESTDIR}${file}")
|
||||
MESSAGE(STATUS "File $ENV{DESTDIR}${file} does not exist.")
|
||||
ENDIF(IS_SYMLINK "$ENV{DESTDIR}${file}" OR EXISTS "$ENV{DESTDIR}${file}")
|
||||
ENDFOREACH(file)
|
||||
@@ -1,18 +0,0 @@
|
||||
## ======================================================================== ##
|
||||
## Copyright 2009-2017 Intel Corporation ##
|
||||
## ##
|
||||
## Licensed under the Apache License, Version 2.0 (the "License"); ##
|
||||
## you may not use this file except in compliance with the License. ##
|
||||
## You may obtain a copy of the License at ##
|
||||
## ##
|
||||
## http://www.apache.org/licenses/LICENSE-2.0 ##
|
||||
## ##
|
||||
## Unless required by applicable law or agreed to in writing, software ##
|
||||
## distributed under the License is distributed on an "AS IS" BASIS, ##
|
||||
## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ##
|
||||
## See the License for the specific language governing permissions and ##
|
||||
## limitations under the License. ##
|
||||
## ======================================================================== ##
|
||||
|
||||
ADD_LIBRARY(math STATIC constants.cpp)
|
||||
SET_PROPERTY(TARGET math PROPERTY FOLDER common)
|
||||
+13
-36
@@ -1,5 +1,5 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
@@ -16,10 +16,9 @@
|
||||
|
||||
#pragma once
|
||||
|
||||
#include "linearspace2.h"
|
||||
#include "linearspace3.h"
|
||||
#include "quaternion.h"
|
||||
#include "bbox.h"
|
||||
#include "math/linearspace3.h"
|
||||
#include "math/quaternion.h"
|
||||
#include "math/bbox.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
@@ -48,7 +47,7 @@ namespace embree
|
||||
__forceinline AffineSpaceT( const VectorT& vx, const VectorT& vy, const VectorT& vz, const VectorT& p ) : l(vx,vy,vz), p(p) {}
|
||||
__forceinline AffineSpaceT( const L& l, const VectorT& p ) : l(l), p(p) {}
|
||||
|
||||
template<typename L1> __forceinline AffineSpaceT( const AffineSpaceT<L1>& s ) : l(s.l), p(s.p) {}
|
||||
template<typename L1> __forceinline explicit AffineSpaceT( const AffineSpaceT<L1>& s ) : l(s.l), p(s.p) {}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
// Constants
|
||||
@@ -58,21 +57,18 @@ namespace embree
|
||||
__forceinline AffineSpaceT( OneTy ) : l(one), p(zero) {}
|
||||
|
||||
/*! return matrix for scaling */
|
||||
static __forceinline AffineSpaceT scale(const VectorT& s) { return L::scale(s); }
|
||||
static __forceinline AffineSpaceT scale(const VectorT& s) { return AffineSpaceT(L::scale(s),zero); }
|
||||
|
||||
/*! return matrix for translation */
|
||||
static __forceinline AffineSpaceT translate(const VectorT& p) { return AffineSpaceT(one,p); }
|
||||
|
||||
/*! return matrix for rotation, only in 2D */
|
||||
static __forceinline AffineSpaceT rotate(const ScalarT& r) { return L::rotate(r); }
|
||||
/*! return matrix for rotation around arbitrary axis */
|
||||
static __forceinline AffineSpaceT rotate(const VectorT& u, const ScalarT& r) { return AffineSpaceT(L::rotate(u,r),zero); }
|
||||
|
||||
/*! return matrix for rotation around arbitrary point (2D) or axis (3D) */
|
||||
static __forceinline AffineSpaceT rotate(const VectorT& u, const ScalarT& r) { return L::rotate(u,r); }
|
||||
|
||||
/*! return matrix for rotation around arbitrary axis and point, only in 3D */
|
||||
/*! return matrix for rotation around arbitrary axis and point */
|
||||
static __forceinline AffineSpaceT rotate(const VectorT& p, const VectorT& u, const ScalarT& r) { return translate(+p) * rotate(u,r) * translate(-p); }
|
||||
|
||||
/*! return matrix for looking at given point, only in 3D */
|
||||
/*! return matrix for looking at given point */
|
||||
static __forceinline AffineSpaceT lookat(const VectorT& eye, const VectorT& point, const VectorT& up) {
|
||||
VectorT Z = normalize(point-eye);
|
||||
VectorT U = normalize(cross(up,Z));
|
||||
@@ -107,9 +103,9 @@ namespace embree
|
||||
template<typename L> __forceinline AffineSpaceT<L>& operator /=( AffineSpaceT<L>& a, const AffineSpaceT<L>& b ) { return a = a / b; }
|
||||
template<typename L> __forceinline AffineSpaceT<L>& operator /=( AffineSpaceT<L>& a, const ScalarT & b ) { return a = a / b; }
|
||||
|
||||
template<typename L> __forceinline VectorT xfmPoint (const AffineSpaceT<L>& m, const VectorT& p) { return madd(VectorT(p.x),m.l.vx,madd(VectorT(p.y),m.l.vy,madd(VectorT(p.z),m.l.vz,m.p))); }
|
||||
template<typename L> __forceinline VectorT xfmVector(const AffineSpaceT<L>& m, const VectorT& v) { return xfmVector(m.l,v); }
|
||||
template<typename L> __forceinline VectorT xfmNormal(const AffineSpaceT<L>& m, const VectorT& n) { return xfmNormal(m.l,n); }
|
||||
template<typename L> __forceinline const VectorT xfmPoint (const AffineSpaceT<L>& m, const VectorT& p) { return madd(VectorT(p.x),m.l.vx,madd(VectorT(p.y),m.l.vy,madd(VectorT(p.z),m.l.vz,m.p))); }
|
||||
template<typename L> __forceinline const VectorT xfmVector(const AffineSpaceT<L>& m, const VectorT& v) { return xfmVector(m.l,v); }
|
||||
template<typename L> __forceinline const VectorT xfmNormal(const AffineSpaceT<L>& m, const VectorT& n) { return xfmNormal(m.l,n); }
|
||||
|
||||
__forceinline const BBox<Vec3fa> xfmBounds(const AffineSpaceT<LinearSpace3<Vec3fa> >& m, const BBox<Vec3fa>& b)
|
||||
{
|
||||
@@ -132,20 +128,6 @@ namespace embree
|
||||
template<typename L> __forceinline bool operator ==( const AffineSpaceT<L>& a, const AffineSpaceT<L>& b ) { return a.l == b.l && a.p == b.p; }
|
||||
template<typename L> __forceinline bool operator !=( const AffineSpaceT<L>& a, const AffineSpaceT<L>& b ) { return a.l != b.l || a.p != b.p; }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Select
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
template<typename L> __forceinline AffineSpaceT<L> select ( const typename L::Vector::Scalar::Bool& s, const AffineSpaceT<L>& t, const AffineSpaceT<L>& f ) {
|
||||
return AffineSpaceT<L>(select(s,t.l,f.l),select(s,t.p,f.p));
|
||||
}
|
||||
|
||||
/*! blending */
|
||||
template<typename T>
|
||||
__forceinline AffineSpaceT<T> lerp(const AffineSpaceT<T>& l0, const AffineSpaceT<T>& l1, const float t) {
|
||||
return AffineSpaceT<T>(lerp(l0.l,l1.l,t),lerp(l0.p,l1.p,t));
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
// Output Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
@@ -158,15 +140,10 @@ namespace embree
|
||||
// Template Instantiations
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
typedef AffineSpaceT<LinearSpace2f> AffineSpace2f;
|
||||
typedef AffineSpaceT<LinearSpace3f> AffineSpace3f;
|
||||
typedef AffineSpaceT<LinearSpace3fa> AffineSpace3fa;
|
||||
typedef AffineSpaceT<Quaternion3f > OrthonormalSpace3f;
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/*! Template Specialization for 2D: return matrix for rotation around point (rotation around arbitrarty vector is not meaningful in 2D) */
|
||||
template<> __forceinline AffineSpace2f AffineSpace2f::rotate(const Vec2f& p, const float& r) { return translate(+p) * AffineSpace2f(LinearSpace2f::rotate(r)) * translate(-p); }
|
||||
|
||||
#undef VectorT
|
||||
#undef ScalarT
|
||||
}
|
||||
|
||||
+33
-180
@@ -1,5 +1,5 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
@@ -41,26 +41,24 @@ namespace embree
|
||||
/// Extending Bounds
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const BBox& extend(const BBox& other) { lower = min(lower,other.lower); upper = max(upper,other.upper); return *this; }
|
||||
__forceinline const BBox& extend(const T & other) { lower = min(lower,other ); upper = max(upper,other ); return *this; }
|
||||
__forceinline void extend(const BBox& other) { lower = min(lower,other.lower); upper = max(upper,other.upper); }
|
||||
__forceinline void extend(const T & other) { lower = min(lower,other ); upper = max(upper,other ); }
|
||||
|
||||
/*! tests if box is empty */
|
||||
__forceinline void extend_atomic(const BBox& other) {
|
||||
atomic_min_f32(&lower.x,other.lower.x);
|
||||
atomic_min_f32(&lower.y,other.lower.y);
|
||||
atomic_min_f32(&lower.z,other.lower.z);
|
||||
atomic_max_f32(&upper.x,other.upper.x);
|
||||
atomic_max_f32(&upper.y,other.upper.y);
|
||||
atomic_max_f32(&upper.z,other.upper.z);
|
||||
}
|
||||
|
||||
/*! computes the size of the box */
|
||||
__forceinline bool empty() const { for (int i=0; i<T::N; i++) if (lower[i] > upper[i]) return true; return false; }
|
||||
|
||||
/*! computes the size of the box */
|
||||
__forceinline T size() const { return upper - lower; }
|
||||
|
||||
/*! computes the center of the box */
|
||||
__forceinline T center() const { return 0.5f*(lower+upper); }
|
||||
|
||||
/*! computes twice the center of the box */
|
||||
__forceinline T center2() const { return lower+upper; }
|
||||
|
||||
/*! merges two boxes */
|
||||
__forceinline static const BBox merge (const BBox& a, const BBox& b) {
|
||||
return BBox(min(a.lower, b.lower), max(a.upper, b.upper));
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constants
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
@@ -73,32 +71,15 @@ namespace embree
|
||||
__forceinline BBox( PosInfTy ): lower(neg_inf), upper(pos_inf) {}
|
||||
};
|
||||
|
||||
template<> __forceinline bool BBox<float>::empty() const {
|
||||
return lower > upper;
|
||||
}
|
||||
|
||||
#if defined(__SSE__)
|
||||
template<> __forceinline bool BBox<Vec3fa>::empty() const {
|
||||
return !all(le_mask(lower,upper));
|
||||
}
|
||||
#endif
|
||||
|
||||
/*! tests if box is finite */
|
||||
__forceinline bool isvalid( const BBox<Vec3fa>& v ) {
|
||||
return all(gt_mask(v.lower,Vec3fa_t(-FLT_LARGE)) & lt_mask(v.upper,Vec3fa_t(+FLT_LARGE)));
|
||||
}
|
||||
|
||||
/*! tests if box has finite entries */
|
||||
__forceinline bool is_finite( const BBox<Vec3fa>& b) {
|
||||
return is_finite(b.lower) && is_finite(b.upper);
|
||||
}
|
||||
|
||||
/*! test if point contained in box */
|
||||
__forceinline bool inside ( const BBox<Vec3fa>& b, const Vec3fa& p ) { return all(ge_mask(p,b.lower) & le_mask(p,b.upper)); }
|
||||
|
||||
/*! computes the center of the box */
|
||||
template<typename T> __forceinline const T center (const BBox<T>& box) { return T(.5f)*(box.lower + box.upper); }
|
||||
template<typename T> __forceinline const T center2(const BBox<T>& box) { return box.lower + box.upper; }
|
||||
template<typename T> __forceinline const T center (const BBox<T>& box) { return T(0.5f)*center2(box); }
|
||||
|
||||
/*! computes the volume of a bounding box */
|
||||
__forceinline float volume ( const BBox<Vec3fa>& b ) { return reduce_mul(b.size()); }
|
||||
@@ -108,19 +89,10 @@ namespace embree
|
||||
__forceinline float volume( const BBox<Vec3f>& b ) { return reduce_mul(b.size()); }
|
||||
|
||||
/*! computes the surface area of a bounding box */
|
||||
template<typename T> __forceinline const T area( const BBox<Vec2<T> >& b ) { const Vec2<T> d = b.size(); return d.x*d.y; }
|
||||
|
||||
template<typename T> __forceinline const T halfArea( const BBox<Vec3<T> >& b ) { return halfArea(b.size()); }
|
||||
template<typename T> __forceinline const T area( const BBox<Vec3<T> >& b ) { return 2.0f*halfArea(b); }
|
||||
|
||||
__forceinline float halfArea( const BBox<Vec3fa>& b ) { return halfArea(b.size()); }
|
||||
__forceinline float area( const BBox<Vec3fa>& b ) { return 2.0f*halfArea(b); }
|
||||
|
||||
template<typename Vec> __forceinline float safeArea( const BBox<Vec>& b ) { if (b.empty()) return 0.0f; else return area(b); }
|
||||
|
||||
template<typename T> __forceinline float expectedApproxHalfArea(const BBox<T>& box) {
|
||||
return halfArea(box);
|
||||
}
|
||||
template<typename T> __forceinline const T area( const BBox<Vec2<T> >& b ) { const Vec2<T> d = size(b); return d.x*d.y; }
|
||||
__forceinline float area( const BBox<Vec3fa>& b ) { const Vec3fa d = b.size(); return 2.0f*(d.x*(d.y+d.z)+d.y*d.z); }
|
||||
__forceinline float safeArea( const BBox<Vec3fa>& b ) { if (b.empty()) return 0.0f; else return area(b); }
|
||||
__forceinline float halfArea( const BBox<Vec3fa>& b ) { const Vec3fa d = b.size(); return d.x*(d.y+d.z)+d.y*d.z; }
|
||||
|
||||
/*! merges bounding boxes and points */
|
||||
template<typename T> __forceinline const BBox<T> merge( const BBox<T>& a, const T& b ) { return BBox<T>(min(a.lower, b ), max(a.upper, b )); }
|
||||
@@ -143,167 +115,48 @@ namespace embree
|
||||
template<typename T> __forceinline BBox<T> operator *( const float& a, const BBox<T>& b ) { return BBox<T>(a*b.lower,a*b.upper); }
|
||||
template<typename T> __forceinline BBox<T> operator *( const T& a, const BBox<T>& b ) { return BBox<T>(a*b.lower,a*b.upper); }
|
||||
|
||||
/*! translations */
|
||||
template<typename T> __forceinline BBox<T> operator +( const BBox<T>& a, const BBox<T>& b ) { return BBox<T>(a.lower+b.lower,a.upper+b.upper); }
|
||||
template<typename T> __forceinline BBox<T> operator -( const BBox<T>& a, const BBox<T>& b ) { return BBox<T>(a.lower-b.lower,a.upper-b.upper); }
|
||||
template<typename T> __forceinline BBox<T> operator +( const BBox<T>& a, const T & b ) { return BBox<T>(a.lower+b ,a.upper+b ); }
|
||||
template<typename T> __forceinline BBox<T> operator -( const BBox<T>& a, const T & b ) { return BBox<T>(a.lower-b ,a.upper-b ); }
|
||||
|
||||
/*! extension */
|
||||
template<typename T> __forceinline BBox<T> enlarge(const BBox<T>& a, const T& b) { return BBox<T>(a.lower-b, a.upper+b); }
|
||||
template<typename T> __forceinline BBox<T> enlarge(const BBox<T>& a, const T& b) { return BBox<T>(a.lower - b, a.upper +b); }
|
||||
|
||||
/*! intersect bounding boxes */
|
||||
template<typename T> __forceinline const BBox<T> intersect( const BBox<T>& a, const BBox<T>& b ) { return BBox<T>(max(a.lower, b.lower), min(a.upper, b.upper)); }
|
||||
template<typename T> __forceinline const BBox<T> intersect( const BBox<T>& a, const BBox<T>& b, const BBox<T>& c ) { return intersect(a,intersect(b,c)); }
|
||||
template<typename T> __forceinline const BBox<T> intersect( const BBox<T>& a, const BBox<T>& b, const BBox<T>& c, const BBox<T>& d ) { return intersect(intersect(a,b),intersect(c,d)); }
|
||||
|
||||
/*! subtract bounds from each other */
|
||||
template<typename T> __forceinline void subtract(const BBox<T>& a, const BBox<T>& b, BBox<T>& c, BBox<T>& d)
|
||||
{
|
||||
c.lower = a.lower;
|
||||
c.upper = min(a.upper,b.lower);
|
||||
d.lower = max(a.lower,b.upper);
|
||||
d.upper = a.upper;
|
||||
}
|
||||
|
||||
/*! tests if bounding boxes (and points) are disjoint (empty intersection) */
|
||||
template<typename T> __inline bool disjoint( const BBox<T>& a, const BBox<T>& b ) { return intersect(a,b).empty(); }
|
||||
template<typename T> __inline bool disjoint( const BBox<T>& a, const T& b ) { return disjoint(a,BBox<T>(b)); }
|
||||
template<typename T> __inline bool disjoint( const T& a, const BBox<T>& b ) { return disjoint(BBox<T>(a),b); }
|
||||
template<typename T> __inline bool disjoint( const BBox<T>& a, const BBox<T>& b )
|
||||
{ const T d = min(a.upper, b.upper) - max(a.lower, b.lower); for ( size_t i = 0 ; i < T::N ; i++ ) if ( d[i] < typename T::Scalar(zero) ) return true; return false; }
|
||||
template<typename T> __inline bool disjoint( const BBox<T>& a, const T& b )
|
||||
{ const T d = min(a.upper, b) - max(a.lower, b); for ( size_t i = 0 ; i < T::N ; i++ ) if ( d[i] < typename T::Scalar(zero) ) return true; return false; }
|
||||
template<typename T> __inline bool disjoint( const T& a, const BBox<T>& b )
|
||||
{ const T d = min(a, b.upper) - max(a, b.lower); for ( size_t i = 0 ; i < T::N ; i++ ) if ( d[i] < typename T::Scalar(zero) ) return true; return false; }
|
||||
|
||||
/*! tests if bounding boxes (and points) are conjoint (non-empty intersection) */
|
||||
template<typename T> __inline bool conjoint( const BBox<T>& a, const BBox<T>& b ) { return !intersect(a,b).empty(); }
|
||||
template<typename T> __inline bool conjoint( const BBox<T>& a, const T& b ) { return conjoint(a,BBox<T>(b)); }
|
||||
template<typename T> __inline bool conjoint( const T& a, const BBox<T>& b ) { return conjoint(BBox<T>(a),b); }
|
||||
template<typename T> __inline bool conjoint( const BBox<T>& a, const BBox<T>& b )
|
||||
{ const T d = min(a.upper, b.upper) - max(a.lower, b.lower); for ( size_t i = 0 ; i < T::N ; i++ ) if ( d[i] < typename T::Scalar(zero) ) return false; return true; }
|
||||
template<typename T> __inline bool conjoint( const BBox<T>& a, const T& b )
|
||||
{ const T d = min(a.upper, b) - max(a.lower, b); for ( size_t i = 0 ; i < T::N ; i++ ) if ( d[i] < typename T::Scalar(zero) ) return false; return true; }
|
||||
template<typename T> __inline bool conjoint( const T& a, const BBox<T>& b )
|
||||
{ const T d = min(a, b.upper) - max(a, b.lower); for ( size_t i = 0 ; i < T::N ; i++ ) if ( d[i] < typename T::Scalar(zero) ) return false; return true; }
|
||||
|
||||
/*! subset relation */
|
||||
template<typename T> __inline bool subset( const BBox<T>& a, const BBox<T>& b )
|
||||
{
|
||||
for ( size_t i = 0; i < T::N; i++ ) if ( a.lower[i] < b.lower[i] ) return false;
|
||||
for ( size_t i = 0; i < T::N; i++ ) if ( a.upper[i] > b.upper[i] ) return false;
|
||||
for ( size_t i = 0 ; i < T::N ; i++ ) if ( a.lower[i] < b.lower[i] ) return false;
|
||||
for ( size_t i = 0 ; i < T::N ; i++ ) if ( a.upper[i] > b.upper[i] ) return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
/*! blending */
|
||||
template<typename T>
|
||||
__forceinline BBox<T> lerp(const BBox<T>& b0, const BBox<T>& b1, const float t) {
|
||||
return BBox<T>(lerp(b0.lower,b1.lower,t),lerp(b0.upper,b1.upper,t));
|
||||
}
|
||||
|
||||
/*! output operator */
|
||||
template<typename T> __forceinline std::ostream& operator<<(std::ostream& cout, const BBox<T>& box) {
|
||||
return cout << "[" << box.lower << "; " << box.upper << "]";
|
||||
}
|
||||
|
||||
/*! default template instantiations */
|
||||
typedef BBox<float> BBox1f;
|
||||
typedef BBox<Vec2f> BBox2f;
|
||||
typedef BBox<Vec3f> BBox3f;
|
||||
typedef BBox<Vec3fa> BBox3fa;
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// SSE / AVX / MIC specializations
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
#if defined __SSE__
|
||||
#include "../simd/sse.h"
|
||||
#endif
|
||||
|
||||
#if defined __AVX__
|
||||
#include "../simd/avx.h"
|
||||
#endif
|
||||
|
||||
#if defined(__AVX512F__)
|
||||
#include "../simd/avx512.h"
|
||||
#endif
|
||||
|
||||
namespace embree
|
||||
{
|
||||
template<int N>
|
||||
__forceinline BBox<Vec3<vfloat<N>>> transpose(const BBox3fa* bounds);
|
||||
|
||||
template<>
|
||||
__forceinline BBox<Vec3<vfloat4>> transpose<4>(const BBox3fa* bounds)
|
||||
{
|
||||
BBox<Vec3<vfloat4>> dest;
|
||||
|
||||
transpose((vfloat4&)bounds[0].lower,
|
||||
(vfloat4&)bounds[1].lower,
|
||||
(vfloat4&)bounds[2].lower,
|
||||
(vfloat4&)bounds[3].lower,
|
||||
dest.lower.x,
|
||||
dest.lower.y,
|
||||
dest.lower.z);
|
||||
|
||||
transpose((vfloat4&)bounds[0].upper,
|
||||
(vfloat4&)bounds[1].upper,
|
||||
(vfloat4&)bounds[2].upper,
|
||||
(vfloat4&)bounds[3].upper,
|
||||
dest.upper.x,
|
||||
dest.upper.y,
|
||||
dest.upper.z);
|
||||
|
||||
return dest;
|
||||
}
|
||||
|
||||
#if defined(__AVX__)
|
||||
template<>
|
||||
__forceinline BBox<Vec3<vfloat8>> transpose<8>(const BBox3fa* bounds)
|
||||
{
|
||||
BBox<Vec3<vfloat8>> dest;
|
||||
|
||||
transpose((vfloat4&)bounds[0].lower,
|
||||
(vfloat4&)bounds[1].lower,
|
||||
(vfloat4&)bounds[2].lower,
|
||||
(vfloat4&)bounds[3].lower,
|
||||
(vfloat4&)bounds[4].lower,
|
||||
(vfloat4&)bounds[5].lower,
|
||||
(vfloat4&)bounds[6].lower,
|
||||
(vfloat4&)bounds[7].lower,
|
||||
dest.lower.x,
|
||||
dest.lower.y,
|
||||
dest.lower.z);
|
||||
|
||||
transpose((vfloat4&)bounds[0].upper,
|
||||
(vfloat4&)bounds[1].upper,
|
||||
(vfloat4&)bounds[2].upper,
|
||||
(vfloat4&)bounds[3].upper,
|
||||
(vfloat4&)bounds[4].upper,
|
||||
(vfloat4&)bounds[5].upper,
|
||||
(vfloat4&)bounds[6].upper,
|
||||
(vfloat4&)bounds[7].upper,
|
||||
dest.upper.x,
|
||||
dest.upper.y,
|
||||
dest.upper.z);
|
||||
|
||||
return dest;
|
||||
}
|
||||
#endif
|
||||
|
||||
template<int N>
|
||||
__forceinline BBox3fa merge(const BBox3fa* bounds);
|
||||
|
||||
template<>
|
||||
__forceinline BBox3fa merge<4>(const BBox3fa* bounds)
|
||||
{
|
||||
const Vec3fa lower = min(min(bounds[0].lower,bounds[1].lower),
|
||||
min(bounds[2].lower,bounds[3].lower));
|
||||
const Vec3fa upper = max(max(bounds[0].upper,bounds[1].upper),
|
||||
max(bounds[2].upper,bounds[3].upper));
|
||||
return BBox3fa(lower,upper);
|
||||
}
|
||||
|
||||
#if defined(__AVX__)
|
||||
template<>
|
||||
__forceinline BBox3fa merge<8>(const BBox3fa* bounds)
|
||||
{
|
||||
const Vec3fa lower = min(min(min(bounds[0].lower,bounds[1].lower),min(bounds[2].lower,bounds[3].lower)),
|
||||
min(min(bounds[4].lower,bounds[5].lower),min(bounds[6].lower,bounds[7].lower)));
|
||||
const Vec3fa upper = max(max(max(bounds[0].upper,bounds[1].upper),max(bounds[2].upper,bounds[3].upper)),
|
||||
max(max(bounds[4].upper,bounds[5].upper),max(bounds[6].upper,bounds[7].upper)));
|
||||
return BBox3fa(lower,upper);
|
||||
}
|
||||
#endif
|
||||
}
|
||||
|
||||
|
||||
+4
-3
@@ -1,5 +1,5 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
@@ -16,7 +16,8 @@
|
||||
|
||||
#pragma once
|
||||
|
||||
#include "math.h"
|
||||
#include "sys/platform.h"
|
||||
#include "math/math.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
@@ -55,6 +56,6 @@ namespace embree
|
||||
}
|
||||
|
||||
/*! default template instantiations */
|
||||
typedef Col3<unsigned char> Col3uc;
|
||||
typedef Col3<unsigned char> Col3c;
|
||||
typedef Col3<float > Col3f;
|
||||
}
|
||||
|
||||
+4
-3
@@ -1,5 +1,5 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
@@ -16,7 +16,8 @@
|
||||
|
||||
#pragma once
|
||||
|
||||
#include "math.h"
|
||||
#include "sys/platform.h"
|
||||
#include "math/math.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
@@ -55,6 +56,6 @@ namespace embree
|
||||
}
|
||||
|
||||
/*! default template instantiations */
|
||||
typedef Col4<unsigned char> Col4uc;
|
||||
typedef Col4<unsigned char> Col4c;
|
||||
typedef Col4<float > Col4f;
|
||||
}
|
||||
|
||||
+24
-56
@@ -1,5 +1,5 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
@@ -16,12 +16,12 @@
|
||||
|
||||
#pragma once
|
||||
|
||||
#include "constants.h"
|
||||
#include "../sys/constants.h"
|
||||
#include "../simd/sse.h"
|
||||
|
||||
#include "col3.h"
|
||||
#include "col4.h"
|
||||
|
||||
#include "../simd/sse.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
@@ -45,16 +45,16 @@ namespace embree
|
||||
__forceinline explicit Color4 (const float v) : m128(_mm_set1_ps(v)) {}
|
||||
__forceinline Color4 (const float r, const float g, const float b, const float a) : m128(_mm_set_ps(a,b,g,r)) {}
|
||||
|
||||
__forceinline explicit Color4 ( const Col3uc& other ) { m128 = _mm_mul_ps(_mm_set_ps(255.0f,other.b,other.g,other.r),_mm_set1_ps(one_over_255)); }
|
||||
__forceinline explicit Color4 ( const Col3f& other ) { m128 = _mm_set_ps(1.0f,other.b,other.g,other.r); }
|
||||
__forceinline explicit Color4 ( const Col4uc& other ) { m128 = _mm_mul_ps(_mm_set_ps(other.a,other.b,other.g,other.r),_mm_set1_ps(one_over_255)); }
|
||||
__forceinline explicit Color4 ( const Col4f& other ) { m128 = _mm_set_ps(other.a,other.b,other.g,other.r); }
|
||||
__forceinline explicit Color4 ( const Col3c& other ) { m128 = _mm_mul_ps(_mm_set_ps(255.0f,other.b,other.g,other.r),_mm_set1_ps(one_over_255)); }
|
||||
__forceinline explicit Color4 ( const Col3f& other ) { m128 = _mm_set_ps(1.0f,other.b,other.g,other.r); }
|
||||
__forceinline explicit Color4 ( const Col4c& other ) { m128 = _mm_mul_ps(_mm_set_ps(other.a,other.b,other.g,other.r),_mm_set1_ps(one_over_255)); }
|
||||
__forceinline explicit Color4 ( const Col4f& other ) { m128 = _mm_set_ps(other.a,other.b,other.g,other.r); }
|
||||
|
||||
__forceinline Color4 ( const Color4& other ) : m128(other.m128) {}
|
||||
__forceinline Color4& operator=( const Color4& other ) { m128 = other.m128; return *this; }
|
||||
|
||||
__forceinline operator const __m128&() const { return m128; }
|
||||
__forceinline operator __m128&() { return m128; }
|
||||
__forceinline operator const __m128&( void ) const { return m128; }
|
||||
__forceinline operator __m128&( void ) { return m128; }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Set
|
||||
@@ -62,21 +62,8 @@ namespace embree
|
||||
|
||||
__forceinline void set(Col3f& d) const { d.r = r; d.g = g; d.b = b; }
|
||||
__forceinline void set(Col4f& d) const { d.r = r; d.g = g; d.b = b; d.a = a; }
|
||||
__forceinline void set(Col3uc& d) const
|
||||
{
|
||||
vfloat4 s = clamp(vfloat4(m128))*255.0f;
|
||||
d.r = (unsigned char)(s[0]);
|
||||
d.g = (unsigned char)(s[1]);
|
||||
d.b = (unsigned char)(s[2]);
|
||||
}
|
||||
__forceinline void set(Col4uc& d) const
|
||||
{
|
||||
vfloat4 s = clamp(vfloat4(m128))*255.0f;
|
||||
d.r = (unsigned char)(s[0]);
|
||||
d.g = (unsigned char)(s[1]);
|
||||
d.b = (unsigned char)(s[2]);
|
||||
d.a = (unsigned char)(s[3]);
|
||||
}
|
||||
__forceinline void set(Col3c& d) const { ssef s = clamp(ssef(m128))*255.0f; d.r = char(s[0]); d.g = char(s[1]); d.b = char(s[2]); }
|
||||
__forceinline void set(Col4c& d) const { ssef s = clamp(ssef(m128))*255.0f; d.r = char(s[0]); d.g = char(s[1]); d.b = char(s[2]); d.a = char(s[3]); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constants
|
||||
@@ -115,8 +102,8 @@ namespace embree
|
||||
__forceinline Color ( const Color4& other ) : m128(other.m128) {}
|
||||
__forceinline Color& operator=( const Color4& other ) { m128 = other.m128; return *this; }
|
||||
|
||||
__forceinline operator const __m128&() const { return m128; }
|
||||
__forceinline operator __m128&() { return m128; }
|
||||
__forceinline operator const __m128&( void ) const { return m128; }
|
||||
__forceinline operator __m128&( void ) { return m128; }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Set
|
||||
@@ -124,21 +111,8 @@ namespace embree
|
||||
|
||||
__forceinline void set(Col3f& d) const { d.r = r; d.g = g; d.b = b; }
|
||||
__forceinline void set(Col4f& d) const { d.r = r; d.g = g; d.b = b; d.a = 1.0f; }
|
||||
__forceinline void set(Col3uc& d) const
|
||||
{
|
||||
vfloat4 s = clamp(vfloat4(m128))*255.0f;
|
||||
d.r = (unsigned char)(s[0]);
|
||||
d.g = (unsigned char)(s[1]);
|
||||
d.b = (unsigned char)(s[2]);
|
||||
}
|
||||
__forceinline void set(Col4uc& d) const
|
||||
{
|
||||
vfloat4 s = clamp(vfloat4(m128))*255.0f;
|
||||
d.r = (unsigned char)(s[0]);
|
||||
d.g = (unsigned char)(s[1]);
|
||||
d.b = (unsigned char)(s[2]);
|
||||
d.a = 255;
|
||||
}
|
||||
__forceinline void set(Col3c& d) const { ssef s = clamp(ssef(m128))*255.0f; d.r = char(s[0]); d.g = char(s[1]); d.b = char(s[2]); }
|
||||
__forceinline void set(Col4c& d) const { ssef s = clamp(ssef(m128))*255.0f; d.r = char(s[0]); d.g = char(s[1]); d.b = char(s[2]); d.a = 1.0f; }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constants
|
||||
@@ -163,22 +137,12 @@ namespace embree
|
||||
const __m128 mask = _mm_castsi128_ps(_mm_set1_epi32(0x7fffffff));
|
||||
return _mm_and_ps(a.m128, mask);
|
||||
}
|
||||
__forceinline const Color rcp ( const Color& a )
|
||||
{
|
||||
#if defined(__AVX512VL__)
|
||||
const Color r = _mm_rcp14_ps(a.m128);
|
||||
#else
|
||||
__forceinline const Color rcp ( const Color& a ) {
|
||||
const Color r = _mm_rcp_ps(a.m128);
|
||||
#endif
|
||||
return _mm_sub_ps(_mm_add_ps(r, r), _mm_mul_ps(_mm_mul_ps(r, r), a));
|
||||
}
|
||||
__forceinline const Color rsqrt( const Color& a )
|
||||
{
|
||||
#if defined(__AVX512VL__)
|
||||
__m128 r = _mm_rsqrt14_ps(a.m128);
|
||||
#else
|
||||
__forceinline const Color rsqrt( const Color& a ) {
|
||||
__m128 r = _mm_rsqrt_ps(a.m128);
|
||||
#endif
|
||||
return _mm_add_ps(_mm_mul_ps(_mm_set1_ps(1.5f),r), _mm_mul_ps(_mm_mul_ps(_mm_mul_ps(a, _mm_set1_ps(-0.5f)), r), _mm_mul_ps(r, r)));
|
||||
}
|
||||
__forceinline const Color sqrt ( const Color& a ) { return _mm_sqrt_ps(a.m128); }
|
||||
@@ -237,7 +201,7 @@ namespace embree
|
||||
|
||||
__forceinline const Color select( bool s, const Color& t, const Color& f ) {
|
||||
__m128 mask = s ? _mm_castsi128_ps(_mm_cmpeq_epi32(_mm_setzero_si128(), _mm_setzero_si128())) : _mm_setzero_ps();
|
||||
return blendv_ps(f, t, mask);
|
||||
return _mm_blendv_ps(f, t, mask);
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
@@ -245,7 +209,11 @@ namespace embree
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
/*! computes luminance of a color */
|
||||
__forceinline float luminance (const Color& a) { return madd(0.212671f,a.r,madd(0.715160f,a.g,0.072169f*a.b)); }
|
||||
__forceinline float luminance (const Color& a) { return 0.212671f*a.r + 0.715160f*a.g + 0.072169f*a.b; }
|
||||
|
||||
__forceinline Color exp (const Color& a) { return Color(exp_ps(a.m128)); }
|
||||
__forceinline Color log (const Color& a) { return Color(log_ps(a.m128)); }
|
||||
__forceinline Color pow (const Color& a, const float& b) { return Color(select(ssef(a)<=ssef(zero),ssef(zero),ssef(exp_ps(log(a)*b)))); }
|
||||
|
||||
/*! output operator */
|
||||
inline std::ostream& operator<<(std::ostream& cout, const Color& a) {
|
||||
|
||||
@@ -1,231 +0,0 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#pragma once
|
||||
|
||||
#include "bbox.h"
|
||||
#include "range.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
template<typename T>
|
||||
__forceinline std::pair<T,T> globalLinear(const std::pair<T,T>& v, const BBox1f& dt)
|
||||
{
|
||||
const float rcp_dt_size = float(1.0f)/dt.size();
|
||||
const T g0 = lerp(v.first,v.second,-dt.lower*rcp_dt_size);
|
||||
const T g1 = lerp(v.first,v.second,(1.0f-dt.lower)*rcp_dt_size);
|
||||
return std::make_pair(g0,g1);
|
||||
}
|
||||
|
||||
template<typename T>
|
||||
struct LBBox
|
||||
{
|
||||
public:
|
||||
__forceinline LBBox () {}
|
||||
|
||||
__forceinline LBBox ( const LBBox& other ) {
|
||||
bounds0 = other.bounds0; bounds1 = other.bounds1;
|
||||
}
|
||||
__forceinline LBBox& operator= ( const LBBox& other ) {
|
||||
bounds0 = other.bounds0; bounds1 = other.bounds1; return *this;
|
||||
}
|
||||
|
||||
__forceinline LBBox (EmptyTy)
|
||||
: bounds0(EmptyTy()), bounds1(EmptyTy()) {}
|
||||
|
||||
__forceinline explicit LBBox ( const BBox<T>& bounds)
|
||||
: bounds0(bounds), bounds1(bounds) { }
|
||||
|
||||
__forceinline LBBox ( const BBox<T>& bounds0, const BBox<T>& bounds1)
|
||||
: bounds0(bounds0), bounds1(bounds1) { }
|
||||
|
||||
LBBox ( const avector<BBox<T>>& bounds )
|
||||
{
|
||||
assert(bounds.size());
|
||||
BBox<T> b0 = bounds.front();
|
||||
BBox<T> b1 = bounds.back();
|
||||
for (size_t i=1; i<bounds.size()-1; i++) {
|
||||
const float f = float(i)/float(bounds.size()-1);
|
||||
const BBox<T> bt = lerp(b0,b1,f);
|
||||
const T dlower = min(bounds[i].lower-bt.lower,T(zero));
|
||||
const T dupper = max(bounds[i].upper-bt.upper,T(zero));
|
||||
b0.lower += dlower; b1.lower += dlower;
|
||||
b0.upper += dupper; b1.upper += dupper;
|
||||
}
|
||||
bounds0 = b0;
|
||||
bounds1 = b1;
|
||||
}
|
||||
|
||||
/*! calculates the linear bounds of a primitive for the specified time range */
|
||||
template<typename BoundsFunc>
|
||||
__forceinline LBBox(const BoundsFunc& bounds, const BBox1f& time_range, float numTimeSegments)
|
||||
{
|
||||
const float lower = time_range.lower*numTimeSegments;
|
||||
const float upper = time_range.upper*numTimeSegments;
|
||||
const float ilowerf = floor(lower);
|
||||
const float iupperf = ceil(upper);
|
||||
const int ilower = (int)ilowerf;
|
||||
const int iupper = (int)iupperf;
|
||||
|
||||
const BBox<T> blower0 = bounds(ilower);
|
||||
const BBox<T> bupper1 = bounds(iupper);
|
||||
|
||||
if (iupper-ilower == 1) {
|
||||
bounds0 = lerp(blower0, bupper1, lower-ilowerf);
|
||||
bounds1 = lerp(bupper1, blower0, iupperf-upper);
|
||||
return;
|
||||
}
|
||||
|
||||
const BBox<T> blower1 = bounds(ilower+1);
|
||||
const BBox<T> bupper0 = bounds(iupper-1);
|
||||
BBox<T> b0 = lerp(blower0, blower1, lower-ilowerf);
|
||||
BBox<T> b1 = lerp(bupper1, bupper0, iupperf-upper);
|
||||
|
||||
for (int i = ilower+1; i < iupper; i++)
|
||||
{
|
||||
const float f = (float(i)/numTimeSegments - time_range.lower) / time_range.size();
|
||||
const BBox<T> bt = lerp(b0, b1, f);
|
||||
const BBox<T> bi = bounds(i);
|
||||
const T dlower = min(bi.lower-bt.lower, T(zero));
|
||||
const T dupper = max(bi.upper-bt.upper, T(zero));
|
||||
b0.lower += dlower; b1.lower += dlower;
|
||||
b0.upper += dupper; b1.upper += dupper;
|
||||
}
|
||||
|
||||
bounds0 = b0;
|
||||
bounds1 = b1;
|
||||
}
|
||||
|
||||
/*! calculates the linear bounds of a primitive for the specified time range */
|
||||
template<typename BoundsFunc>
|
||||
__forceinline LBBox(const BoundsFunc& bounds, const range<int>& time_range, int numTimeSegments)
|
||||
{
|
||||
const int ilower = time_range.begin();
|
||||
const int iupper = time_range.end();
|
||||
|
||||
BBox<T> b0 = bounds(ilower);
|
||||
BBox<T> b1 = bounds(iupper);
|
||||
|
||||
if (iupper-ilower == 1) {
|
||||
bounds0 = b0;
|
||||
bounds1 = b1;
|
||||
return;
|
||||
}
|
||||
|
||||
for (int i = ilower+1; i<iupper; i++)
|
||||
{
|
||||
const float f = float(i - time_range.begin()) / float(time_range.size());
|
||||
const BBox<T> bt = lerp(b0, b1, f);
|
||||
const BBox<T> bi = bounds(i);
|
||||
const T dlower = min(bi.lower-bt.lower, T(zero));
|
||||
const T dupper = max(bi.upper-bt.upper, T(zero));
|
||||
b0.lower += dlower; b1.lower += dlower;
|
||||
b0.upper += dupper; b1.upper += dupper;
|
||||
}
|
||||
|
||||
bounds0 = b0;
|
||||
bounds1 = b1;
|
||||
}
|
||||
|
||||
public:
|
||||
|
||||
__forceinline bool empty() const {
|
||||
return bounds().empty();
|
||||
}
|
||||
|
||||
__forceinline BBox<T> bounds () const {
|
||||
return merge(bounds0,bounds1);
|
||||
}
|
||||
|
||||
__forceinline BBox<T> interpolate( const float t ) const {
|
||||
return lerp(bounds0,bounds1,t);
|
||||
}
|
||||
|
||||
__forceinline LBBox<T> interpolate( const BBox1f& dt ) const {
|
||||
return LBBox<T>(interpolate(dt.lower),interpolate(dt.upper));
|
||||
}
|
||||
|
||||
__forceinline void extend( const LBBox& other ) {
|
||||
bounds0.extend(other.bounds0);
|
||||
bounds1.extend(other.bounds1);
|
||||
}
|
||||
|
||||
__forceinline float expectedHalfArea() const;
|
||||
|
||||
__forceinline float expectedHalfArea(const BBox1f& dt) const {
|
||||
return interpolate(dt).expectedHalfArea();
|
||||
}
|
||||
|
||||
__forceinline float expectedApproxHalfArea() const {
|
||||
return 0.5f*(halfArea(bounds0) + halfArea(bounds1));
|
||||
}
|
||||
|
||||
/* calculates bounds for [0,1] time range from bounds in dt time range */
|
||||
__forceinline LBBox global(const BBox1f& dt) const
|
||||
{
|
||||
const float rcp_dt_size = 1.0f/dt.size();
|
||||
const BBox<T> b0 = interpolate(-dt.lower*rcp_dt_size);
|
||||
const BBox<T> b1 = interpolate((1.0f-dt.lower)*rcp_dt_size);
|
||||
return LBBox(b0,b1);
|
||||
}
|
||||
|
||||
/*! Comparison Operators */
|
||||
template<typename TT> friend __forceinline bool operator==( const LBBox<TT>& a, const LBBox<TT>& b ) { return a.bounds0 == b.bounds0 && a.bounds1 == b.bounds1; }
|
||||
template<typename TT> friend __forceinline bool operator!=( const LBBox<TT>& a, const LBBox<TT>& b ) { return a.bounds0 != b.bounds0 || a.bounds1 != b.bounds1; }
|
||||
|
||||
/*! output operator */
|
||||
friend __forceinline std::ostream& operator<<(std::ostream& cout, const LBBox& box) {
|
||||
return cout << "LBBox { " << box.bounds0 << "; " << box.bounds1 << " }";
|
||||
}
|
||||
|
||||
public:
|
||||
BBox<T> bounds0, bounds1;
|
||||
};
|
||||
|
||||
template<typename T>
|
||||
__forceinline T expectedArea(const T& a0, const T& a1, const T& b0, const T& b1)
|
||||
{
|
||||
const T da = a1-a0;
|
||||
const T db = b1-b0;
|
||||
return a0*b0+(a0*db+da*b0)*T(0.5f) + da*db*T(1.0f/3.0f);
|
||||
}
|
||||
|
||||
template<> __forceinline float LBBox<Vec3fa>::expectedHalfArea() const
|
||||
{
|
||||
const Vec3fa d0 = bounds0.size();
|
||||
const Vec3fa d1 = bounds1.size();
|
||||
return reduce_add(expectedArea(Vec3fa(d0.x,d0.y,d0.z),
|
||||
Vec3fa(d1.x,d1.y,d1.z),
|
||||
Vec3fa(d0.y,d0.z,d0.x),
|
||||
Vec3fa(d1.y,d1.z,d1.x)));
|
||||
}
|
||||
|
||||
template<typename T>
|
||||
__forceinline float expectedApproxHalfArea(const LBBox<T>& box) {
|
||||
return box.expectedApproxHalfArea();
|
||||
}
|
||||
|
||||
template<typename T>
|
||||
__forceinline LBBox<T> merge(const LBBox<T>& a, const LBBox<T>& b) {
|
||||
return LBBox<T>(merge(a.bounds0, b.bounds0), merge(a.bounds1, b.bounds1));
|
||||
}
|
||||
|
||||
/*! default template instantiations */
|
||||
typedef LBBox<float> LBBox1f;
|
||||
typedef LBBox<Vec2f> LBBox2f;
|
||||
typedef LBBox<Vec3f> LBBox3f;
|
||||
typedef LBBox<Vec3fa> LBBox3fa;
|
||||
}
|
||||
@@ -1,160 +0,0 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#pragma once
|
||||
|
||||
#include "vec2.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// 2D Linear Transform (2x2 Matrix)
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
template<typename T> struct LinearSpace2
|
||||
{
|
||||
typedef T Vector;
|
||||
typedef typename T::Scalar Scalar;
|
||||
|
||||
/*! default matrix constructor */
|
||||
__forceinline LinearSpace2 ( ) {}
|
||||
__forceinline LinearSpace2 ( const LinearSpace2& other ) { vx = other.vx; vy = other.vy; }
|
||||
__forceinline LinearSpace2& operator=( const LinearSpace2& other ) { vx = other.vx; vy = other.vy; return *this; }
|
||||
|
||||
template<typename L1> __forceinline LinearSpace2( const LinearSpace2<L1>& s ) : vx(s.vx), vy(s.vy) {}
|
||||
|
||||
/*! matrix construction from column vectors */
|
||||
__forceinline LinearSpace2(const Vector& vx, const Vector& vy)
|
||||
: vx(vx), vy(vy) {}
|
||||
|
||||
/*! matrix construction from row mayor data */
|
||||
__forceinline LinearSpace2(const Scalar& m00, const Scalar& m01,
|
||||
const Scalar& m10, const Scalar& m11)
|
||||
: vx(m00,m10), vy(m01,m11) {}
|
||||
|
||||
/*! compute the determinant of the matrix */
|
||||
__forceinline const Scalar det() const { return vx.x*vy.y - vx.y*vy.x; }
|
||||
|
||||
/*! compute adjoint matrix */
|
||||
__forceinline const LinearSpace2 adjoint() const { return LinearSpace2(vy.y,-vy.x,-vx.y,vx.x); }
|
||||
|
||||
/*! compute inverse matrix */
|
||||
__forceinline const LinearSpace2 inverse() const { return adjoint()/det(); }
|
||||
|
||||
/*! compute transposed matrix */
|
||||
__forceinline const LinearSpace2 transposed() const { return LinearSpace2(vx.x,vx.y,vy.x,vy.y); }
|
||||
|
||||
/*! returns first row of matrix */
|
||||
__forceinline Vector row0() const { return Vector(vx.x,vy.x); }
|
||||
|
||||
/*! returns second row of matrix */
|
||||
__forceinline Vector row1() const { return Vector(vx.y,vy.y); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constants
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline LinearSpace2( ZeroTy ) : vx(zero), vy(zero) {}
|
||||
__forceinline LinearSpace2( OneTy ) : vx(one, zero), vy(zero, one) {}
|
||||
|
||||
/*! return matrix for scaling */
|
||||
static __forceinline LinearSpace2 scale(const Vector& s) {
|
||||
return LinearSpace2(s.x, 0,
|
||||
0 , s.y);
|
||||
}
|
||||
|
||||
/*! return matrix for rotation */
|
||||
static __forceinline LinearSpace2 rotate(const Scalar& r) {
|
||||
Scalar s = sin(r), c = cos(r);
|
||||
return LinearSpace2(c, -s,
|
||||
s, c);
|
||||
}
|
||||
|
||||
/*! return closest orthogonal matrix (i.e. a general rotation including reflection) */
|
||||
LinearSpace2 orthogonal() const
|
||||
{
|
||||
LinearSpace2 m = *this;
|
||||
|
||||
// mirrored?
|
||||
Scalar mirror(one);
|
||||
if (m.det() < Scalar(zero)) {
|
||||
m.vx = -m.vx;
|
||||
mirror = -mirror;
|
||||
}
|
||||
|
||||
// rotation
|
||||
for (int i = 0; i < 99; i++) {
|
||||
const LinearSpace2 m_next = 0.5 * (m + m.transposed().inverse());
|
||||
const LinearSpace2 d = m_next - m;
|
||||
m = m_next;
|
||||
// norm^2 of difference small enough?
|
||||
if (max(dot(d.vx, d.vx), dot(d.vy, d.vy)) < 1e-8)
|
||||
break;
|
||||
}
|
||||
|
||||
// rotation * mirror_x
|
||||
return LinearSpace2(mirror*m.vx, m.vy);
|
||||
}
|
||||
|
||||
public:
|
||||
|
||||
/*! the column vectors of the matrix */
|
||||
Vector vx,vy;
|
||||
};
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
// Unary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
template<typename T> __forceinline LinearSpace2<T> operator -( const LinearSpace2<T>& a ) { return LinearSpace2<T>(-a.vx,-a.vy); }
|
||||
template<typename T> __forceinline LinearSpace2<T> operator +( const LinearSpace2<T>& a ) { return LinearSpace2<T>(+a.vx,+a.vy); }
|
||||
template<typename T> __forceinline LinearSpace2<T> rcp ( const LinearSpace2<T>& a ) { return a.inverse(); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
// Binary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
template<typename T> __forceinline LinearSpace2<T> operator +( const LinearSpace2<T>& a, const LinearSpace2<T>& b ) { return LinearSpace2<T>(a.vx+b.vx,a.vy+b.vy); }
|
||||
template<typename T> __forceinline LinearSpace2<T> operator -( const LinearSpace2<T>& a, const LinearSpace2<T>& b ) { return LinearSpace2<T>(a.vx-b.vx,a.vy-b.vy); }
|
||||
|
||||
template<typename T> __forceinline LinearSpace2<T> operator*(const typename T::Scalar & a, const LinearSpace2<T>& b) { return LinearSpace2<T>(a*b.vx, a*b.vy); }
|
||||
template<typename T> __forceinline T operator*(const LinearSpace2<T>& a, const T & b) { return b.x*a.vx + b.y*a.vy; }
|
||||
template<typename T> __forceinline LinearSpace2<T> operator*(const LinearSpace2<T>& a, const LinearSpace2<T>& b) { return LinearSpace2<T>(a*b.vx, a*b.vy); }
|
||||
|
||||
template<typename T> __forceinline LinearSpace2<T> operator/(const LinearSpace2<T>& a, const typename T::Scalar & b) { return LinearSpace2<T>(a.vx/b, a.vy/b); }
|
||||
template<typename T> __forceinline LinearSpace2<T> operator/(const LinearSpace2<T>& a, const LinearSpace2<T>& b) { return a * rcp(b); }
|
||||
|
||||
template<typename T> __forceinline LinearSpace2<T>& operator *=( LinearSpace2<T>& a, const LinearSpace2<T>& b ) { return a = a * b; }
|
||||
template<typename T> __forceinline LinearSpace2<T>& operator /=( LinearSpace2<T>& a, const LinearSpace2<T>& b ) { return a = a / b; }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Comparison Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
template<typename T> __forceinline bool operator ==( const LinearSpace2<T>& a, const LinearSpace2<T>& b ) { return a.vx == b.vx && a.vy == b.vy; }
|
||||
template<typename T> __forceinline bool operator !=( const LinearSpace2<T>& a, const LinearSpace2<T>& b ) { return a.vx != b.vx || a.vy != b.vy; }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Output Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
template<typename T> static std::ostream& operator<<(std::ostream& cout, const LinearSpace2<T>& m) {
|
||||
return cout << "{ vx = " << m.vx << ", vy = " << m.vy << "}";
|
||||
}
|
||||
|
||||
/*! Shortcuts for common linear spaces. */
|
||||
typedef LinearSpace2<Vec2f> LinearSpace2f;
|
||||
}
|
||||
+16
-45
@@ -1,5 +1,5 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
@@ -35,7 +35,7 @@ namespace embree
|
||||
__forceinline LinearSpace3 ( const LinearSpace3& other ) { vx = other.vx; vy = other.vy; vz = other.vz; }
|
||||
__forceinline LinearSpace3& operator=( const LinearSpace3& other ) { vx = other.vx; vy = other.vy; vz = other.vz; return *this; }
|
||||
|
||||
template<typename L1> __forceinline LinearSpace3( const LinearSpace3<L1>& s ) : vx(s.vx), vy(s.vy), vz(s.vz) {}
|
||||
template<typename L1> __forceinline explicit LinearSpace3( const LinearSpace3<L1>& s ) : vx(s.vx), vy(s.vy), vz(s.vz) {}
|
||||
|
||||
/*! matrix construction from column vectors */
|
||||
__forceinline LinearSpace3(const Vector& vx, const Vector& vy, const Vector& vz)
|
||||
@@ -66,13 +66,13 @@ namespace embree
|
||||
__forceinline const LinearSpace3 transposed() const { return LinearSpace3(vx.x,vx.y,vx.z,vy.x,vy.y,vy.z,vz.x,vz.y,vz.z); }
|
||||
|
||||
/*! returns first row of matrix */
|
||||
__forceinline Vector row0() const { return Vector(vx.x,vy.x,vz.x); }
|
||||
__forceinline const Vector row0() const { return Vector(vx.x,vy.x,vz.x); }
|
||||
|
||||
/*! returns second row of matrix */
|
||||
__forceinline Vector row1() const { return Vector(vx.y,vy.y,vz.y); }
|
||||
__forceinline const Vector row1() const { return Vector(vx.y,vy.y,vz.y); }
|
||||
|
||||
/*! returns third row of matrix */
|
||||
__forceinline Vector row2() const { return Vector(vx.z,vy.z,vz.z); }
|
||||
__forceinline const Vector row2() const { return Vector(vx.z,vy.z,vz.z); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constants
|
||||
@@ -103,16 +103,13 @@ namespace embree
|
||||
Vector vx,vy,vz;
|
||||
};
|
||||
|
||||
#if !defined(__MIC__)
|
||||
/*! compute transposed matrix */
|
||||
template<> __forceinline const LinearSpace3<Vec3fa> LinearSpace3<Vec3fa>::transposed() const {
|
||||
vfloat4 rx,ry,rz; transpose((vfloat4&)vx,(vfloat4&)vy,(vfloat4&)vz,vfloat4(zero),rx,ry,rz);
|
||||
ssef rx,ry,rz; transpose((ssef&)vx,(ssef&)vy,(ssef&)vz,ssef(zero),rx,ry,rz);
|
||||
return LinearSpace3<Vec3fa>(Vec3fa(rx),Vec3fa(ry),Vec3fa(rz));
|
||||
}
|
||||
|
||||
template<typename T>
|
||||
__forceinline const LinearSpace3<T> transposed(const LinearSpace3<T>& xfm) {
|
||||
return xfm.transposed();
|
||||
}
|
||||
#endif
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
// Unary Operators
|
||||
@@ -122,25 +119,15 @@ namespace embree
|
||||
template<typename T> __forceinline LinearSpace3<T> operator +( const LinearSpace3<T>& a ) { return LinearSpace3<T>(+a.vx,+a.vy,+a.vz); }
|
||||
template<typename T> __forceinline LinearSpace3<T> rcp ( const LinearSpace3<T>& a ) { return a.inverse(); }
|
||||
|
||||
/* constructs a coordinate frame form a normalized normal */
|
||||
template<typename T> __forceinline LinearSpace3<T> frame(const T& N)
|
||||
{
|
||||
const T dx0(0,N.z,-N.y);
|
||||
const T dx1(-N.z,0,N.x);
|
||||
const T dx = normalize(select(dot(dx0,dx0) > dot(dx1,dx1),dx0,dx1));
|
||||
const T dy = normalize(cross(N,dx));
|
||||
/* constructs a coordinate frame form a normal */
|
||||
template<typename T> __forceinline LinearSpace3<T> frame(const T& N) {
|
||||
T dx0 = cross(T(one,zero,zero),N);
|
||||
T dx1 = cross(T(zero,one,zero),N);
|
||||
T dx = normalize(select(dot(dx0,dx0) > dot(dx1,dx1),dx0,dx1));
|
||||
T dy = normalize(cross(N,dx));
|
||||
return LinearSpace3<T>(dx,dy,N);
|
||||
}
|
||||
|
||||
/* constructs a coordinate frame from a normal and approximate x-direction */
|
||||
template<typename T> __forceinline LinearSpace3<T> frame(const T& N, const T& dxi)
|
||||
{
|
||||
if (abs(dot(dxi,N)) > 0.99f) return frame(N); // fallback in case N and dxi are very parallel
|
||||
const T dx = normalize(cross(dxi,N));
|
||||
const T dy = normalize(cross(N,dx));
|
||||
return LinearSpace3<T>(dx,dy,N);
|
||||
}
|
||||
|
||||
/* clamps linear space to range -1 to +1 */
|
||||
template<typename T> __forceinline LinearSpace3<T> clamp(const LinearSpace3<T>& space) {
|
||||
return LinearSpace3<T>(clamp(space.vx,T(-1.0f),T(1.0f)),
|
||||
@@ -156,7 +143,7 @@ namespace embree
|
||||
template<typename T> __forceinline LinearSpace3<T> operator -( const LinearSpace3<T>& a, const LinearSpace3<T>& b ) { return LinearSpace3<T>(a.vx-b.vx,a.vy-b.vy,a.vz-b.vz); }
|
||||
|
||||
template<typename T> __forceinline LinearSpace3<T> operator*(const typename T::Scalar & a, const LinearSpace3<T>& b) { return LinearSpace3<T>(a*b.vx, a*b.vy, a*b.vz); }
|
||||
template<typename T> __forceinline T operator*(const LinearSpace3<T>& a, const T & b) { return madd(T(b.x),a.vx,madd(T(b.y),a.vy,T(b.z)*a.vz)); }
|
||||
template<typename T> __forceinline T operator*(const LinearSpace3<T>& a, const T & b) { return b.x*a.vx + b.y*a.vy + b.z*a.vz; }
|
||||
template<typename T> __forceinline LinearSpace3<T> operator*(const LinearSpace3<T>& a, const LinearSpace3<T>& b) { return LinearSpace3<T>(a*b.vx, a*b.vy, a*b.vz); }
|
||||
|
||||
template<typename T> __forceinline LinearSpace3<T> operator/(const LinearSpace3<T>& a, const typename T::Scalar & b) { return LinearSpace3<T>(a.vx/b, a.vy/b, a.vz/b); }
|
||||
@@ -176,23 +163,6 @@ namespace embree
|
||||
template<typename T> __forceinline bool operator ==( const LinearSpace3<T>& a, const LinearSpace3<T>& b ) { return a.vx == b.vx && a.vy == b.vy && a.vz == b.vz; }
|
||||
template<typename T> __forceinline bool operator !=( const LinearSpace3<T>& a, const LinearSpace3<T>& b ) { return a.vx != b.vx || a.vy != b.vy || a.vz != b.vz; }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Select
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
template<typename T> __forceinline LinearSpace3<T> select ( const typename T::Scalar::Bool& s, const LinearSpace3<T>& t, const LinearSpace3<T>& f ) {
|
||||
return LinearSpace3<T>(select(s,t.vx,f.vx),select(s,t.vy,f.vy),select(s,t.vz,f.vz));
|
||||
}
|
||||
|
||||
/*! blending */
|
||||
template<typename T>
|
||||
__forceinline LinearSpace3<T> lerp(const LinearSpace3<T>& l0, const LinearSpace3<T>& l1, const float t)
|
||||
{
|
||||
return LinearSpace3<T>(lerp(l0.vx,l1.vx,t),
|
||||
lerp(l0.vy,l1.vy,t),
|
||||
lerp(l0.vz,l1.vz,t));
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Output Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
@@ -204,4 +174,5 @@ namespace embree
|
||||
/*! Shortcuts for common linear spaces. */
|
||||
typedef LinearSpace3<Vec3f> LinearSpace3f;
|
||||
typedef LinearSpace3<Vec3fa> LinearSpace3fa;
|
||||
|
||||
}
|
||||
|
||||
+91
-191
@@ -1,5 +1,5 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
@@ -16,40 +16,17 @@
|
||||
|
||||
#pragma once
|
||||
|
||||
#include "../sys/platform.h"
|
||||
#include "../sys/intrinsics.h"
|
||||
#include "constants.h"
|
||||
#include <cmath>
|
||||
#include "sys/platform.h"
|
||||
#include "sys/intrinsics.h"
|
||||
|
||||
#include <math.h>
|
||||
#include <cmath>
|
||||
#include <float.h>
|
||||
#include <emmintrin.h>
|
||||
#include <xmmintrin.h>
|
||||
#include <immintrin.h>
|
||||
|
||||
#if defined(__WIN32__)
|
||||
#if (__MSV_VER <= 1700)
|
||||
namespace std
|
||||
{
|
||||
__forceinline bool isinf ( const float x ) { return _finite(x) == 0; }
|
||||
__forceinline bool isnan ( const float x ) { return _isnan(x) != 0; }
|
||||
__forceinline bool isfinite (const float x) { return _finite(x) != 0; }
|
||||
}
|
||||
#endif
|
||||
#endif
|
||||
|
||||
namespace embree
|
||||
{
|
||||
__forceinline bool isvalid ( const float& v ) {
|
||||
return (v > -FLT_LARGE) & (v < +FLT_LARGE);
|
||||
}
|
||||
|
||||
__forceinline int cast_f2i(float f) {
|
||||
union { float f; int i; } v; v.f = f; return v.i;
|
||||
}
|
||||
|
||||
__forceinline float cast_i2f(int i) {
|
||||
union { float f; int i; } v; v.i = i; return v.f;
|
||||
}
|
||||
|
||||
#if defined(__WIN32__)
|
||||
__forceinline bool finite ( const float x ) { return _finite(x) != 0; }
|
||||
#endif
|
||||
@@ -57,54 +34,38 @@ namespace embree
|
||||
__forceinline float sign ( const float x ) { return x<0?-1.0f:1.0f; }
|
||||
__forceinline float sqr ( const float x ) { return x*x; }
|
||||
|
||||
__forceinline float rcp ( const float x )
|
||||
{
|
||||
const __m128 a = _mm_set_ss(x);
|
||||
|
||||
#if defined(__AVX512VL__)
|
||||
const __m128 r = _mm_rcp14_ss(_mm_set_ss(0.0f),a);
|
||||
#else
|
||||
const __m128 r = _mm_rcp_ss(a);
|
||||
#endif
|
||||
|
||||
#if defined(__AVX2__)
|
||||
return _mm_cvtss_f32(_mm_mul_ss(r,_mm_fnmadd_ss(r, a, _mm_set_ss(2.0f))));
|
||||
#else
|
||||
return _mm_cvtss_f32(_mm_mul_ss(r,_mm_sub_ss(_mm_set_ss(2.0f), _mm_mul_ss(r, a))));
|
||||
#endif
|
||||
#ifndef __MIC__
|
||||
__forceinline float rcp ( const float x )
|
||||
{
|
||||
const __m128 vx = _mm_set_ss(x);
|
||||
const __m128 r = _mm_rcp_ps(vx);
|
||||
return _mm_cvtss_f32(_mm_sub_ps(_mm_add_ps(r, r), _mm_mul_ps(_mm_mul_ps(r, r), vx)));
|
||||
}
|
||||
|
||||
__forceinline float signmsk ( const float x ) {
|
||||
__forceinline float signmsk ( const float x ) {
|
||||
return _mm_cvtss_f32(_mm_and_ps(_mm_set_ss(x),_mm_castsi128_ps(_mm_set1_epi32(0x80000000))));
|
||||
}
|
||||
__forceinline float xorf( const float x, const float y ) {
|
||||
__forceinline float xorf( const float x, const float y ) {
|
||||
return _mm_cvtss_f32(_mm_xor_ps(_mm_set_ss(x),_mm_set_ss(y)));
|
||||
}
|
||||
__forceinline float andf( const float x, const unsigned y ) {
|
||||
__forceinline float andf( const float x, const unsigned y ) {
|
||||
return _mm_cvtss_f32(_mm_and_ps(_mm_set_ss(x),_mm_castsi128_ps(_mm_set1_epi32(y))));
|
||||
}
|
||||
__forceinline float rsqrt( const float x )
|
||||
{
|
||||
__forceinline float rsqrt( const float x ) {
|
||||
const __m128 a = _mm_set_ss(x);
|
||||
#if defined(__AVX512VL__)
|
||||
const __m128 r = _mm_rsqrt14_ss(_mm_set_ss(0.0f),a);
|
||||
#else
|
||||
const __m128 r = _mm_rsqrt_ss(a);
|
||||
#endif
|
||||
const __m128 c = _mm_add_ss(_mm_mul_ss(_mm_set_ss(1.5f), r),
|
||||
_mm_mul_ss(_mm_mul_ss(_mm_mul_ss(a, _mm_set_ss(-0.5f)), r), _mm_mul_ss(r, r)));
|
||||
const __m128 r = _mm_rsqrt_ps(a);
|
||||
const __m128 c = _mm_add_ps(_mm_mul_ps(_mm_set_ps(1.5f, 1.5f, 1.5f, 1.5f), r),
|
||||
_mm_mul_ps(_mm_mul_ps(_mm_mul_ps(a, _mm_set_ps(-0.5f, -0.5f, -0.5f, -0.5f)), r), _mm_mul_ps(r, r)));
|
||||
return _mm_cvtss_f32(c);
|
||||
}
|
||||
|
||||
#if defined(__WIN32__) && (__MSC_VER <= 1700)
|
||||
__forceinline float nextafter(float x, float y) { if ((x<y) == (x>0)) return x*(1.1f+float(ulp)); else return x*(0.9f-float(ulp)); }
|
||||
__forceinline double nextafter(double x, double y) { return _nextafter(x, y); }
|
||||
__forceinline int roundf(float f) { return (int)(f + 0.5f); }
|
||||
#else
|
||||
__forceinline float nextafter(float x, float y) { return ::nextafterf(x, y); }
|
||||
__forceinline double nextafter(double x, double y) { return ::nextafter(x, y); }
|
||||
__forceinline float rcp ( const float x ) { return 1.0f/x; }
|
||||
__forceinline float signmsk ( const float x ) { return cast_i2f(cast_f2i(x)&0x80000000); }
|
||||
__forceinline float xorf( const float x, const float y ) { return cast_i2f(cast_f2i(x) ^ cast_f2i(y)); }
|
||||
__forceinline float andf( const float x, const float y ) { return cast_i2f(cast_f2i(x) & cast_f2i(y)); }
|
||||
__forceinline float rsqrt( const float x ) { return 1.0f/sqrtf(x); }
|
||||
#endif
|
||||
|
||||
#if !defined(__WIN32__)
|
||||
__forceinline float abs ( const float x ) { return ::fabsf(x); }
|
||||
__forceinline float acos ( const float x ) { return ::acosf (x); }
|
||||
__forceinline float asin ( const float x ) { return ::asinf (x); }
|
||||
@@ -124,7 +85,7 @@ namespace embree
|
||||
__forceinline float tanh ( const float x ) { return ::tanhf (x); }
|
||||
__forceinline float floor( const float x ) { return ::floorf (x); }
|
||||
__forceinline float ceil ( const float x ) { return ::ceilf (x); }
|
||||
__forceinline float frac ( const float x ) { return x-floor(x); }
|
||||
#endif
|
||||
|
||||
__forceinline double abs ( const double x ) { return ::fabs(x); }
|
||||
__forceinline double sign ( const double x ) { return x<0?-1.0:1.0; }
|
||||
@@ -151,7 +112,7 @@ namespace embree
|
||||
__forceinline double ceil ( const double x ) { return ::ceil (x); }
|
||||
|
||||
#if defined(__SSE4_1__)
|
||||
__forceinline float mini(float a, float b) {
|
||||
__forceinline float mini(float a, float b) {
|
||||
const __m128i ai = _mm_castps_si128(_mm_set_ss(a));
|
||||
const __m128i bi = _mm_castps_si128(_mm_set_ss(b));
|
||||
const __m128i ci = _mm_min_epi32(ai,bi);
|
||||
@@ -160,7 +121,7 @@ namespace embree
|
||||
#endif
|
||||
|
||||
#if defined(__SSE4_1__)
|
||||
__forceinline float maxi(float a, float b) {
|
||||
__forceinline float maxi(float a, float b) {
|
||||
const __m128i ai = _mm_castps_si128(_mm_set_ss(a));
|
||||
const __m128i bi = _mm_castps_si128(_mm_set_ss(b));
|
||||
const __m128i ci = _mm_max_epi32(ai,bi);
|
||||
@@ -168,60 +129,30 @@ namespace embree
|
||||
}
|
||||
#endif
|
||||
|
||||
template<typename T>
|
||||
__forceinline T twice(const T& a) { return a+a; }
|
||||
|
||||
__forceinline int min(int a, int b) { return a<b ? a:b; }
|
||||
__forceinline unsigned min(unsigned a, unsigned b) { return a<b ? a:b; }
|
||||
__forceinline int64_t min(int64_t a, int64_t b) { return a<b ? a:b; }
|
||||
__forceinline float min(float a, float b) { return a<b ? a:b; }
|
||||
__forceinline double min(double a, double b) { return a<b ? a:b; }
|
||||
#if defined(__X86_64__)
|
||||
__forceinline size_t min(size_t a, size_t b) { return a<b ? a:b; }
|
||||
__forceinline int min(int a, int b) { return a<b? a:b; }
|
||||
__forceinline int64 min(int64 a, int64 b) { return a<b? a:b; }
|
||||
__forceinline size_t min(size_t a, size_t b) { return a<b? a:b; }
|
||||
#if !defined(__WIN32__)
|
||||
__forceinline ssize_t min(ssize_t a, ssize_t b) { return a<b? a:b; }
|
||||
#endif
|
||||
|
||||
template<typename T> __forceinline T min(const T& a, const T& b, const T& c) { return min(min(a,b),c); }
|
||||
template<typename T> __forceinline T min(const T& a, const T& b, const T& c, const T& d) { return min(min(a,b),min(c,d)); }
|
||||
__forceinline float min(float a, float b) { return a<b? a:b; }
|
||||
__forceinline double min(double a, double b) { return a<b? a:b; }
|
||||
template<typename T> __forceinline T min(const T& a, const T& b, const T& c) { return min(min(a,b),c); }
|
||||
template<typename T> __forceinline T min(const T& a, const T& b, const T& c, const T& d) { return min(min(a,b),min(c,d)); }
|
||||
template<typename T> __forceinline T min(const T& a, const T& b, const T& c, const T& d, const T& e) { return min(min(min(a,b),min(c,d)),e); }
|
||||
|
||||
template<typename T> __forceinline T mini(const T& a, const T& b, const T& c) { return mini(mini(a,b),c); }
|
||||
template<typename T> __forceinline T mini(const T& a, const T& b, const T& c, const T& d) { return mini(mini(a,b),mini(c,d)); }
|
||||
template<typename T> __forceinline T mini(const T& a, const T& b, const T& c, const T& d, const T& e) { return mini(mini(mini(a,b),mini(c,d)),e); }
|
||||
|
||||
__forceinline int max(int a, int b) { return a<b ? b:a; }
|
||||
__forceinline unsigned max(unsigned a, unsigned b) { return a<b ? b:a; }
|
||||
__forceinline int64_t max(int64_t a, int64_t b) { return a<b ? b:a; }
|
||||
__forceinline float max(float a, float b) { return a<b ? b:a; }
|
||||
__forceinline double max(double a, double b) { return a<b ? b:a; }
|
||||
#if defined(__X86_64__)
|
||||
__forceinline size_t max(size_t a, size_t b) { return a<b ? b:a; }
|
||||
__forceinline int max(int a, int b) { return a<b? b:a; }
|
||||
__forceinline int64 max(int64 a, int64 b) { return a<b? b:a; }
|
||||
__forceinline size_t max(size_t a, size_t b) { return a<b? b:a; }
|
||||
#if !defined(__WIN32__)
|
||||
__forceinline ssize_t max(ssize_t a, ssize_t b) { return a<b? b:a; }
|
||||
#endif
|
||||
|
||||
template<typename T> __forceinline T max(const T& a, const T& b, const T& c) { return max(max(a,b),c); }
|
||||
template<typename T> __forceinline T max(const T& a, const T& b, const T& c, const T& d) { return max(max(a,b),max(c,d)); }
|
||||
__forceinline float max(float a, float b) { return a<b? b:a; }
|
||||
__forceinline double max(double a, double b) { return a<b? b:a; }
|
||||
template<typename T> __forceinline T max(const T& a, const T& b, const T& c) { return max(max(a,b),c); }
|
||||
template<typename T> __forceinline T max(const T& a, const T& b, const T& c, const T& d) { return max(max(a,b),max(c,d)); }
|
||||
template<typename T> __forceinline T max(const T& a, const T& b, const T& c, const T& d, const T& e) { return max(max(max(a,b),max(c,d)),e); }
|
||||
|
||||
template<typename T> __forceinline T maxi(const T& a, const T& b, const T& c) { return maxi(maxi(a,b),c); }
|
||||
template<typename T> __forceinline T maxi(const T& a, const T& b, const T& c, const T& d) { return maxi(maxi(a,b),maxi(c,d)); }
|
||||
template<typename T> __forceinline T maxi(const T& a, const T& b, const T& c, const T& d, const T& e) { return maxi(maxi(maxi(a,b),maxi(c,d)),e); }
|
||||
|
||||
#if defined(__MACOSX__)
|
||||
__forceinline ssize_t min(ssize_t a, ssize_t b) { return a<b ? a:b; }
|
||||
__forceinline ssize_t max(ssize_t a, ssize_t b) { return a<b ? b:a; }
|
||||
#endif
|
||||
|
||||
#if defined(__MACOSX__) && !defined(__INTEL_COMPILER)
|
||||
__forceinline void sincosf(float x, float *sin, float *cos) {
|
||||
__sincosf(x,sin,cos);
|
||||
}
|
||||
#endif
|
||||
|
||||
#if defined(__WIN32__) || defined(__FreeBSD__)
|
||||
__forceinline void sincosf(float x, float *s, float *c) {
|
||||
*s = sinf(x); *c = cosf(x);
|
||||
}
|
||||
#endif
|
||||
|
||||
template<typename T> __forceinline T clamp(const T& x, const T& lower = T(zero), const T& upper = T(one)) { return max(min(x,upper),lower); }
|
||||
template<typename T> __forceinline T clampz(const T& x, const T& upper) { return max(T(zero), min(x,upper)); }
|
||||
|
||||
@@ -238,48 +169,22 @@ namespace embree
|
||||
#else
|
||||
__forceinline float madd ( const float a, const float b, const float c) { return a*b+c; }
|
||||
__forceinline float msub ( const float a, const float b, const float c) { return a*b-c; }
|
||||
__forceinline float nmadd ( const float a, const float b, const float c) { return -a*b+c;}
|
||||
__forceinline float nmsub ( const float a, const float b, const float c) { return -a*b-c; }
|
||||
__forceinline float nmadd ( const float a, const float b, const float c) { return -a*b-c;}
|
||||
__forceinline float nmsub ( const float a, const float b, const float c) { return c-a*b; }
|
||||
#endif
|
||||
|
||||
/*! random functions */
|
||||
template<typename T> T random() { return T(0); }
|
||||
#if defined(_WIN32)
|
||||
template<> __forceinline int random() { return int(rand()) ^ (int(rand()) << 8) ^ (int(rand()) << 16); }
|
||||
template<> __forceinline uint32_t random() { return uint32_t(rand()) ^ (uint32_t(rand()) << 8) ^ (uint32_t(rand()) << 16); }
|
||||
#else
|
||||
template<> __forceinline int random() { return int(rand()); }
|
||||
template<> __forceinline uint32_t random() { return uint32_t(rand()) ^ (uint32_t(rand()) << 16); }
|
||||
#endif
|
||||
template<> __forceinline float random() { return rand()/float(RAND_MAX); }
|
||||
template<> __forceinline double random() { return rand()/double(RAND_MAX); }
|
||||
|
||||
#if _WIN32
|
||||
__forceinline double drand48() {
|
||||
return double(rand())/double(RAND_MAX);
|
||||
}
|
||||
|
||||
__forceinline void srand48(long seed) {
|
||||
return srand(seed);
|
||||
}
|
||||
#endif
|
||||
template<typename T> T random() { return T(0); }
|
||||
template<> __forceinline int random() { return int(rand()); }
|
||||
template<> __forceinline uint32 random() { return uint32(rand()); }
|
||||
template<> __forceinline float random() { return random<uint32>()/float(RAND_MAX); }
|
||||
template<> __forceinline double random() { return random<uint32>()/double(RAND_MAX); }
|
||||
|
||||
/*! selects */
|
||||
__forceinline bool select(bool s, bool t , bool f) { return s ? t : f; }
|
||||
__forceinline int select(bool s, int t, int f) { return s ? t : f; }
|
||||
__forceinline float select(bool s, float t, float f) { return s ? t : f; }
|
||||
|
||||
|
||||
template<typename V>
|
||||
__forceinline V lerp(const V& v0, const V& v1, const float t) {
|
||||
return madd(V(1.0f-t),v0,t*v1);
|
||||
}
|
||||
|
||||
template<typename T>
|
||||
__forceinline T lerp2(const float x0, const float x1, const float x2, const float x3, const T& u, const T& v) {
|
||||
return madd((1.0f-u),madd((1.0f-v),T(x0),v*T(x2)),u*madd((1.0f-v),T(x1),v*T(x3)));
|
||||
}
|
||||
|
||||
/*! exchange */
|
||||
template<typename T> __forceinline void xchg ( T& a, T& b ) { const T tmp = a; a = b; b = tmp; }
|
||||
|
||||
@@ -301,62 +206,57 @@ namespace embree
|
||||
__forceinline T bitInterleave(const T& xin, const T& yin, const T& zin)
|
||||
{
|
||||
T x = xin, y = yin, z = zin;
|
||||
x = (x | (x << 16)) & 0x030000FF;
|
||||
x = (x | (x << 8)) & 0x0300F00F;
|
||||
x = (x | (x << 4)) & 0x030C30C3;
|
||||
x = (x | (x << 2)) & 0x09249249;
|
||||
|
||||
y = (y | (y << 16)) & 0x030000FF;
|
||||
y = (y | (y << 8)) & 0x0300F00F;
|
||||
y = (y | (y << 4)) & 0x030C30C3;
|
||||
y = (y | (y << 2)) & 0x09249249;
|
||||
|
||||
z = (z | (z << 16)) & 0x030000FF;
|
||||
z = (z | (z << 8)) & 0x0300F00F;
|
||||
z = (z | (z << 4)) & 0x030C30C3;
|
||||
z = (z | (z << 2)) & 0x09249249;
|
||||
|
||||
x = (x | (x << 16)) & 0x030000FF;
|
||||
x = (x | (x << 8)) & 0x0300F00F;
|
||||
x = (x | (x << 4)) & 0x030C30C3;
|
||||
x = (x | (x << 2)) & 0x09249249;
|
||||
|
||||
y = (y | (y << 16)) & 0x030000FF;
|
||||
y = (y | (y << 8)) & 0x0300F00F;
|
||||
y = (y | (y << 4)) & 0x030C30C3;
|
||||
y = (y | (y << 2)) & 0x09249249;
|
||||
|
||||
z = (z | (z << 16)) & 0x030000FF;
|
||||
z = (z | (z << 8)) & 0x0300F00F;
|
||||
z = (z | (z << 4)) & 0x030C30C3;
|
||||
z = (z | (z << 2)) & 0x09249249;
|
||||
|
||||
return x | (y << 1) | (z << 2);
|
||||
}
|
||||
|
||||
#if defined(__AVX2__)
|
||||
|
||||
template<>
|
||||
__forceinline unsigned int bitInterleave(const unsigned int &xi, const unsigned int& yi, const unsigned int& zi)
|
||||
{
|
||||
const unsigned int xx = pdep(xi,0x49249249 /* 0b01001001001001001001001001001001 */ );
|
||||
const unsigned int yy = pdep(yi,0x92492492 /* 0b10010010010010010010010010010010 */);
|
||||
const unsigned int zz = pdep(zi,0x24924924 /* 0b00100100100100100100100100100100 */);
|
||||
return xx | yy | zz;
|
||||
}
|
||||
|
||||
#endif
|
||||
|
||||
/*! bit interleave operation for 64bit data types*/
|
||||
template<class T>
|
||||
__forceinline T bitInterleave64(const T& xin, const T& yin, const T& zin){
|
||||
T x = xin & 0x1fffff;
|
||||
T y = yin & 0x1fffff;
|
||||
T z = zin & 0x1fffff;
|
||||
T x = xin & 0x1fffff;
|
||||
T y = yin & 0x1fffff;
|
||||
T z = zin & 0x1fffff;
|
||||
|
||||
x = (x | x << 32) & 0x1f00000000ffff;
|
||||
x = (x | x << 16) & 0x1f0000ff0000ff;
|
||||
x = (x | x << 8) & 0x100f00f00f00f00f;
|
||||
x = (x | x << 4) & 0x10c30c30c30c30c3;
|
||||
x = (x | x << 32) & 0x1f00000000ffff;
|
||||
x = (x | x << 16) & 0x1f0000ff0000ff;
|
||||
x = (x | x << 8) & 0x100f00f00f00f00f;
|
||||
x = (x | x << 4) & 0x10c30c30c30c30c3;
|
||||
x = (x | x << 2) & 0x1249249249249249;
|
||||
|
||||
y = (y | y << 32) & 0x1f00000000ffff;
|
||||
y = (y | y << 16) & 0x1f0000ff0000ff;
|
||||
y = (y | y << 8) & 0x100f00f00f00f00f;
|
||||
y = (y | y << 4) & 0x10c30c30c30c30c3;
|
||||
y = (y | y << 32) & 0x1f00000000ffff;
|
||||
y = (y | y << 16) & 0x1f0000ff0000ff;
|
||||
y = (y | y << 8) & 0x100f00f00f00f00f;
|
||||
y = (y | y << 4) & 0x10c30c30c30c30c3;
|
||||
y = (y | y << 2) & 0x1249249249249249;
|
||||
|
||||
z = (z | z << 32) & 0x1f00000000ffff;
|
||||
z = (z | z << 16) & 0x1f0000ff0000ff;
|
||||
z = (z | z << 8) & 0x100f00f00f00f00f;
|
||||
z = (z | z << 4) & 0x10c30c30c30c30c3;
|
||||
z = (z | z << 32) & 0x1f00000000ffff;
|
||||
z = (z | z << 16) & 0x1f0000ff0000ff;
|
||||
z = (z | z << 8) & 0x100f00f00f00f00f;
|
||||
z = (z | z << 4) & 0x10c30c30c30c30c3;
|
||||
z = (z | z << 2) & 0x1249249249249249;
|
||||
|
||||
return x | (y << 1) | (z << 2);
|
||||
}
|
||||
|
||||
|
||||
#if _WIN32
|
||||
__forceinline double drand48() {
|
||||
return double(rand())/double(RAND_MAX);
|
||||
}
|
||||
#endif
|
||||
|
||||
}
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
@@ -21,32 +21,32 @@
|
||||
|
||||
namespace embree
|
||||
{
|
||||
/*! Oriented bounding box */
|
||||
/*! Non-axis aligned bounds */
|
||||
template<typename T>
|
||||
struct OBBox
|
||||
struct NAABBox
|
||||
{
|
||||
public:
|
||||
|
||||
__forceinline OBBox () {}
|
||||
__forceinline NAABBox () {}
|
||||
|
||||
__forceinline OBBox (EmptyTy)
|
||||
__forceinline NAABBox (EmptyTy)
|
||||
: space(one), bounds(empty) {}
|
||||
|
||||
__forceinline OBBox (const BBox<T>& bounds)
|
||||
__forceinline NAABBox (const BBox<T>& bounds)
|
||||
: space(one), bounds(bounds) {}
|
||||
|
||||
__forceinline OBBox (const LinearSpace3<T>& space, const BBox<T>& bounds)
|
||||
__forceinline NAABBox (const LinearSpace3<T>& space, const BBox<T>& bounds)
|
||||
: space(space), bounds(bounds) {}
|
||||
|
||||
friend std::ostream& operator<<(std::ostream& cout, const OBBox& p) {
|
||||
friend std::ostream& operator<<(std::ostream& cout, const NAABBox& p) {
|
||||
return std::cout << "{ space = " << p.space << ", bounds = " << p.bounds << "}";
|
||||
}
|
||||
|
||||
public:
|
||||
LinearSpace3<T> space; //!< orthonormal transformation
|
||||
BBox<T> bounds; //!< bounds in transformed space
|
||||
BBox<T> bounds; //!< bounds in transformed space
|
||||
};
|
||||
|
||||
typedef OBBox<Vec3f> OBBox3f;
|
||||
typedef OBBox<Vec3fa> OBBox3fa;
|
||||
typedef NAABBox<Vec3f> NAABBox3f;
|
||||
typedef NAABBox<Vec3fa> NAABBox3fa;
|
||||
}
|
||||
@@ -1,5 +1,5 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
@@ -16,12 +16,13 @@
|
||||
|
||||
#pragma once
|
||||
|
||||
#include "vec3.h"
|
||||
#include "sys/platform.h"
|
||||
#include "math/vec3.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
////////////////////////////////////////////////////////////////
|
||||
// Quaternion Struct
|
||||
// Quaterion Struct
|
||||
////////////////////////////////////////////////////////////////
|
||||
|
||||
template<typename T>
|
||||
@@ -33,13 +34,13 @@ namespace embree
|
||||
/// Construction
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline QuaternionT () { }
|
||||
__forceinline QuaternionT ( void ) { }
|
||||
__forceinline QuaternionT ( const QuaternionT& other ) { r = other.r; i = other.i; j = other.j; k = other.k; }
|
||||
__forceinline QuaternionT& operator=( const QuaternionT& other ) { r = other.r; i = other.i; j = other.j; k = other.k; return *this; }
|
||||
|
||||
__forceinline QuaternionT( const T& r ) : r(r), i(zero), j(zero), k(zero) {}
|
||||
__forceinline QuaternionT( const T& r ) : r(r), i(zero), j(zero), k(zero) {}
|
||||
__forceinline explicit QuaternionT( const Vec3<T>& v ) : r(zero), i(v.x), j(v.y), k(v.z) {}
|
||||
__forceinline QuaternionT( const T& r, const T& i, const T& j, const T& k ) : r(r), i(i), j(j), k(k) {}
|
||||
__forceinline QuaternionT( const T& r, const T& i, const T& j , const T& k ) : r(r), i(i), j(j), k(k) {}
|
||||
__forceinline QuaternionT( const T& r, const Vec3<T>& v ) : r(r), i(v.x), j(v.y), k(v.z) {}
|
||||
|
||||
__inline QuaternionT( const Vec3<T>& vx, const Vec3<T>& vy, const Vec3<T>& vz );
|
||||
@@ -58,7 +59,7 @@ namespace embree
|
||||
}
|
||||
|
||||
/*! returns the rotation axis of the quaternion as a vector */
|
||||
__forceinline Vec3<T> v( ) const { return Vec3<T>(i, j, k); }
|
||||
__forceinline const Vec3<T> v( ) const { return Vec3<T>(i, j, k); }
|
||||
|
||||
public:
|
||||
T r, i, j, k;
|
||||
|
||||
@@ -1,150 +0,0 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#pragma once
|
||||
|
||||
#include "../sys/platform.h"
|
||||
#include "../math/math.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
template<typename Ty>
|
||||
struct range
|
||||
{
|
||||
__forceinline range() {}
|
||||
|
||||
__forceinline range(const Ty& begin)
|
||||
: _begin(begin), _end(begin+1) {}
|
||||
|
||||
__forceinline range(const Ty& begin, const Ty& end)
|
||||
: _begin(begin), _end(end) {}
|
||||
|
||||
__forceinline range(const range& other)
|
||||
: _begin(other._begin), _end(other._end) {}
|
||||
|
||||
template<typename T1>
|
||||
__forceinline range(const range<T1>& other)
|
||||
: _begin(Ty(other._begin)), _end(Ty(other._end)) {}
|
||||
|
||||
template<typename T1>
|
||||
__forceinline range& operator =(const range<T1>& other) {
|
||||
_begin = other._begin;
|
||||
_end = other._end;
|
||||
return *this;
|
||||
}
|
||||
|
||||
__forceinline Ty begin() const {
|
||||
return _begin;
|
||||
}
|
||||
|
||||
__forceinline Ty end() const {
|
||||
return _end;
|
||||
}
|
||||
|
||||
__forceinline range intersect(const range& r) const {
|
||||
return range (max(_begin,r._begin),min(_end,r._end));
|
||||
}
|
||||
|
||||
__forceinline Ty size() const {
|
||||
return _end - _begin;
|
||||
}
|
||||
|
||||
__forceinline bool empty() const {
|
||||
return _end <= _begin;
|
||||
}
|
||||
|
||||
__forceinline Ty center() const {
|
||||
return (_begin + _end)/2;
|
||||
}
|
||||
|
||||
__forceinline std::pair<range,range> split() const
|
||||
{
|
||||
const Ty _center = center();
|
||||
return std::make_pair(range(_begin,_center),range(_center,_end));
|
||||
}
|
||||
|
||||
__forceinline void split(range& left_o, range& right_o) const
|
||||
{
|
||||
const Ty _center = center();
|
||||
left_o = range(_begin,_center);
|
||||
right_o = range(_center,_end);
|
||||
}
|
||||
|
||||
__forceinline friend bool operator< (const range& r0, const range& r1) {
|
||||
return r0.size() < r1.size();
|
||||
}
|
||||
|
||||
friend std::ostream& operator<<(std::ostream& cout, const range& r) {
|
||||
return cout << "range [" << r.begin() << ", " << r.end() << "]";
|
||||
}
|
||||
|
||||
Ty _begin, _end;
|
||||
};
|
||||
|
||||
template<typename Ty>
|
||||
range<Ty> make_range(const Ty& begin, const Ty& end) {
|
||||
return range<Ty>(begin,end);
|
||||
}
|
||||
|
||||
template<typename Ty>
|
||||
struct extended_range : public range<Ty>
|
||||
{
|
||||
__forceinline extended_range () {}
|
||||
|
||||
__forceinline extended_range (const Ty& begin)
|
||||
: range<Ty>(begin), _ext_end(begin+1) {}
|
||||
|
||||
__forceinline extended_range (const Ty& begin, const Ty& end)
|
||||
: range<Ty>(begin,end), _ext_end(end) {}
|
||||
|
||||
__forceinline extended_range (const Ty& begin, const Ty& end, const Ty& ext_end)
|
||||
: range<Ty>(begin,end), _ext_end(ext_end) {}
|
||||
|
||||
__forceinline Ty ext_end() const {
|
||||
return _ext_end;
|
||||
}
|
||||
|
||||
__forceinline Ty ext_size() const {
|
||||
return _ext_end - range<Ty>::_begin;
|
||||
}
|
||||
|
||||
__forceinline Ty ext_range_size() const {
|
||||
return _ext_end - range<Ty>::_end;
|
||||
}
|
||||
|
||||
__forceinline bool has_ext_range() const {
|
||||
assert(_ext_end >= range<Ty>::_end);
|
||||
return (_ext_end - range<Ty>::_end) > 0;
|
||||
}
|
||||
|
||||
__forceinline void set_ext_range(const size_t ext_end){
|
||||
assert(ext_end >= range<Ty>::_end);
|
||||
_ext_end = ext_end;
|
||||
}
|
||||
|
||||
__forceinline void move_right(const size_t plus){
|
||||
range<Ty>::_begin += plus;
|
||||
range<Ty>::_end += plus;
|
||||
_ext_end += plus;
|
||||
}
|
||||
|
||||
friend std::ostream& operator<<(std::ostream& cout, const extended_range& r) {
|
||||
return cout << "extended_range [" << r.begin() << ", " << r.end() << " (" << r.ext_end() << ")]";
|
||||
}
|
||||
|
||||
Ty _ext_end;
|
||||
};
|
||||
}
|
||||
+11
-45
@@ -1,5 +1,5 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
@@ -16,7 +16,8 @@
|
||||
|
||||
#pragma once
|
||||
|
||||
#include "math.h"
|
||||
#include "sys/platform.h"
|
||||
#include "math/math.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
@@ -35,14 +36,15 @@ namespace embree
|
||||
/// Construction
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline Vec2( ) {}
|
||||
__forceinline explicit Vec2( const T& a ) : x(a), y(a) {}
|
||||
__forceinline Vec2( const T& x, const T& y ) : x(x), y(y) {}
|
||||
|
||||
__forceinline Vec2( const Vec2& other ) { x = other.x; y = other.y; }
|
||||
__forceinline Vec2 ( ) { }
|
||||
__forceinline Vec2 ( const Vec2& other ) { x = other.x; y = other.y; }
|
||||
template<typename T1> __forceinline Vec2( const Vec2<T1>& a ) : x(T(a.x)), y(T(a.y)) {}
|
||||
template<typename T1> __forceinline Vec2& operator =( const Vec2<T1>& other ) { x = other.x; y = other.y; return *this; }
|
||||
|
||||
__forceinline explicit Vec2( const T& a ) : x(a), y(a) {}
|
||||
__forceinline explicit Vec2( const T& x, const T& y ) : x(x), y(y) {}
|
||||
__forceinline explicit Vec2( const T* const a, const ssize_t stride = 1 ) : x(a[0]), y(a[stride]) {}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constants
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
@@ -66,18 +68,13 @@ namespace embree
|
||||
template<typename T> __forceinline Vec2<T> rcp ( const Vec2<T>& a ) { return Vec2<T>(rcp (a.x), rcp (a.y)); }
|
||||
template<typename T> __forceinline Vec2<T> rsqrt ( const Vec2<T>& a ) { return Vec2<T>(rsqrt(a.x), rsqrt(a.y)); }
|
||||
template<typename T> __forceinline Vec2<T> sqrt ( const Vec2<T>& a ) { return Vec2<T>(sqrt (a.x), sqrt (a.y)); }
|
||||
template<typename T> __forceinline Vec2<T> frac ( const Vec2<T>& a ) { return Vec2<T>(frac (a.x), frac (a.y)); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Binary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
template<typename T> __forceinline Vec2<T> operator +( const Vec2<T>& a, const Vec2<T>& b ) { return Vec2<T>(a.x + b.x, a.y + b.y); }
|
||||
template<typename T> __forceinline Vec2<T> operator +( const Vec2<T>& a, const T& b ) { return Vec2<T>(a.x + b , a.y + b ); }
|
||||
template<typename T> __forceinline Vec2<T> operator +( const T& a, const Vec2<T>& b ) { return Vec2<T>(a + b.x, a + b.y); }
|
||||
template<typename T> __forceinline Vec2<T> operator -( const Vec2<T>& a, const Vec2<T>& b ) { return Vec2<T>(a.x - b.x, a.y - b.y); }
|
||||
template<typename T> __forceinline Vec2<T> operator -( const Vec2<T>& a, const T& b ) { return Vec2<T>(a.x - b , a.y - b ); }
|
||||
template<typename T> __forceinline Vec2<T> operator -( const T& a, const Vec2<T>& b ) { return Vec2<T>(a - b.x, a - b.y); }
|
||||
template<typename T> __forceinline Vec2<T> operator *( const Vec2<T>& a, const Vec2<T>& b ) { return Vec2<T>(a.x * b.x, a.y * b.y); }
|
||||
template<typename T> __forceinline Vec2<T> operator *( const T& a, const Vec2<T>& b ) { return Vec2<T>(a * b.x, a * b.y); }
|
||||
template<typename T> __forceinline Vec2<T> operator *( const Vec2<T>& a, const T& b ) { return Vec2<T>(a.x * b , a.y * b ); }
|
||||
@@ -88,20 +85,6 @@ namespace embree
|
||||
template<typename T> __forceinline Vec2<T> min(const Vec2<T>& a, const Vec2<T>& b) { return Vec2<T>(min(a.x, b.x), min(a.y, b.y)); }
|
||||
template<typename T> __forceinline Vec2<T> max(const Vec2<T>& a, const Vec2<T>& b) { return Vec2<T>(max(a.x, b.x), max(a.y, b.y)); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Ternary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
template<typename T> __forceinline Vec2<T> madd ( const Vec2<T>& a, const Vec2<T>& b, const Vec2<T>& c) { return Vec2<T>( madd(a.x,b.x,c.x), madd(a.y,b.y,c.y) ); }
|
||||
template<typename T> __forceinline Vec2<T> msub ( const Vec2<T>& a, const Vec2<T>& b, const Vec2<T>& c) { return Vec2<T>( msub(a.x,b.x,c.x), msub(a.y,b.y,c.y) ); }
|
||||
template<typename T> __forceinline Vec2<T> nmadd ( const Vec2<T>& a, const Vec2<T>& b, const Vec2<T>& c) { return Vec2<T>(nmadd(a.x,b.x,c.x),nmadd(a.y,b.y,c.y) ); }
|
||||
template<typename T> __forceinline Vec2<T> nmsub ( const Vec2<T>& a, const Vec2<T>& b, const Vec2<T>& c) { return Vec2<T>(nmsub(a.x,b.x,c.x),nmsub(a.y,b.y,c.y) ); }
|
||||
|
||||
template<typename T> __forceinline Vec2<T> madd ( const T& a, const Vec2<T>& b, const Vec2<T>& c) { return Vec2<T>( madd(a,b.x,c.x), madd(a,b.y,c.y) ); }
|
||||
template<typename T> __forceinline Vec2<T> msub ( const T& a, const Vec2<T>& b, const Vec2<T>& c) { return Vec2<T>( msub(a,b.x,c.x), msub(a,b.y,c.y) ); }
|
||||
template<typename T> __forceinline Vec2<T> nmadd ( const T& a, const Vec2<T>& b, const Vec2<T>& c) { return Vec2<T>(nmadd(a,b.x,c.x),nmadd(a,b.y,c.y) ); }
|
||||
template<typename T> __forceinline Vec2<T> nmsub ( const T& a, const Vec2<T>& b, const Vec2<T>& c) { return Vec2<T>(nmsub(a,b.x,c.x),nmsub(a,b.y,c.y) ); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Assignment Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
@@ -136,16 +119,10 @@ namespace embree
|
||||
/// Euclidian Space Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
template<typename T> __forceinline T dot ( const Vec2<T>& a, const Vec2<T>& b ) { return madd(a.x,b.x,a.y*b.y); }
|
||||
|
||||
template<typename T> __forceinline T dot ( const Vec2<T>& a, const Vec2<T>& b ) { return a.x*b.x + a.y*b.y; }
|
||||
template<typename T> __forceinline T length ( const Vec2<T>& a ) { return sqrt(dot(a,a)); }
|
||||
template<typename T> __forceinline Vec2<T> normalize( const Vec2<T>& a ) { return a*rsqrt(dot(a,a)); }
|
||||
template<typename T> __forceinline T distance ( const Vec2<T>& a, const Vec2<T>& b ) { return length(a-b); }
|
||||
template<typename T> __forceinline T det ( const Vec2<T>& a, const Vec2<T>& b ) { return a.x*b.y - a.y*b.x; }
|
||||
|
||||
template<typename T> __forceinline Vec2<T> normalize_safe( const Vec2<T>& a ) {
|
||||
const T d = dot(a,a); return select(d == T( zero ),a, a*rsqrt(d) );
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Select
|
||||
@@ -155,21 +132,10 @@ namespace embree
|
||||
return Vec2<T>(select(s,t.x,f.x),select(s,t.y,f.y));
|
||||
}
|
||||
|
||||
template<typename T> __forceinline Vec2<T> select ( const Vec2<bool>& s, const Vec2<T>& t, const Vec2<T>& f ) {
|
||||
return Vec2<T>(select(s.x,t.x,f.x),select(s.y,t.y,f.y));
|
||||
}
|
||||
|
||||
template<typename T> __forceinline Vec2<T> select ( const typename T::Bool& s, const Vec2<T>& t, const Vec2<T>& f ) {
|
||||
template<typename T> __forceinline Vec2<T> select ( const typename T::Mask& s, const Vec2<T>& t, const Vec2<T>& f ) {
|
||||
return Vec2<T>(select(s,t.x,f.x),select(s,t.y,f.y));
|
||||
}
|
||||
|
||||
template<typename T> __forceinline int maxDim ( const Vec2<T>& a )
|
||||
{
|
||||
const Vec2<T> b = abs(a);
|
||||
if (b.x > b.y) return 0;
|
||||
else return 1;
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Output Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
+79
-163
@@ -1,5 +1,5 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
@@ -16,7 +16,24 @@
|
||||
|
||||
#pragma once
|
||||
|
||||
#include "math.h"
|
||||
#include "sys/platform.h"
|
||||
#include "math/math.h"
|
||||
|
||||
#if defined __SSE__
|
||||
#include "simd/sse.h"
|
||||
#endif
|
||||
|
||||
#if defined __AVX__
|
||||
#include "simd/avx.h"
|
||||
#endif
|
||||
|
||||
#if defined __MIC__
|
||||
#include "simd/sse_mic.h"
|
||||
#endif
|
||||
|
||||
#if defined __MIC__
|
||||
#include "simd/mic.h"
|
||||
#endif
|
||||
|
||||
namespace embree
|
||||
{
|
||||
@@ -37,12 +54,12 @@ namespace embree
|
||||
/// Construction
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline Vec3( ) {}
|
||||
__forceinline explicit Vec3( const T& a ) : x(a), y(a), z(a) {}
|
||||
__forceinline Vec3( const T& x, const T& y, const T& z ) : x(x), y(y), z(z) {}
|
||||
__forceinline Vec3 ( ) {}
|
||||
__forceinline Vec3 ( const T& a ) : x(a), y(a), z(a) {}
|
||||
__forceinline Vec3 ( const T& x, const T& y, const T& z ) : x(x), y(y), z(z) {}
|
||||
|
||||
__forceinline Vec3( const Vec3& other ) { x = other.x; y = other.y; z = other.z; }
|
||||
__forceinline Vec3( const Vec3fa& other );
|
||||
__forceinline Vec3 ( const Vec3& other ) { x = other.x; y = other.y; z = other.z; }
|
||||
__forceinline Vec3 ( const Vec3fa& other );
|
||||
|
||||
template<typename T1> __forceinline Vec3( const Vec3<T1>& a ) : x(T(a.x)), y(T(a.y)), z(T(a.z)) {}
|
||||
template<typename T1> __forceinline Vec3& operator =(const Vec3<T1>& other) { x = other.x; y = other.y; z = other.z; return *this; }
|
||||
@@ -70,14 +87,10 @@ namespace embree
|
||||
template<typename T> __forceinline Vec3<T> rcp ( const Vec3<T>& a ) { return Vec3<T>(rcp (a.x), rcp (a.y), rcp (a.z)); }
|
||||
template<typename T> __forceinline Vec3<T> rsqrt ( const Vec3<T>& a ) { return Vec3<T>(rsqrt(a.x), rsqrt(a.y), rsqrt(a.z)); }
|
||||
template<typename T> __forceinline Vec3<T> sqrt ( const Vec3<T>& a ) { return Vec3<T>(sqrt (a.x), sqrt (a.y), sqrt (a.z)); }
|
||||
|
||||
template<typename T> __forceinline Vec3<T> zero_fix( const Vec3<T>& a )
|
||||
{
|
||||
return Vec3<T>(select(abs(a.x)<min_rcp_input,T(min_rcp_input),a.x),
|
||||
select(abs(a.y)<min_rcp_input,T(min_rcp_input),a.y),
|
||||
select(abs(a.z)<min_rcp_input,T(min_rcp_input),a.z));
|
||||
template<typename T> __forceinline Vec3<T> zero_fix( const Vec3<T>& a ) {
|
||||
return Vec3<T>(select(a.x==0.0f,T(1E-10f),a.x),select(a.y==0.0f,T(1E-10f),a.y),select(a.z==0.0f,T(1E-10f),a.z));
|
||||
}
|
||||
template<typename T> __forceinline Vec3<T> rcp_safe(const Vec3<T>& a) { return rcp(zero_fix(a)); }
|
||||
template<typename T> __forceinline const Vec3<T> rcp_safe(const Vec3<T>& a) { return rcp(zero_fix(a)); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Binary Operators
|
||||
@@ -102,25 +115,20 @@ namespace embree
|
||||
/// Ternary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
template<typename T> __forceinline Vec3<T> madd ( const Vec3<T>& a, const Vec3<T>& b, const Vec3<T>& c) { return Vec3<T>( madd(a.x,b.x,c.x), madd(a.y,b.y,c.y), madd(a.z,b.z,c.z)); }
|
||||
template<typename T> __forceinline Vec3<T> msub ( const Vec3<T>& a, const Vec3<T>& b, const Vec3<T>& c) { return Vec3<T>( msub(a.x,b.x,c.x), msub(a.y,b.y,c.y), msub(a.z,b.z,c.z)); }
|
||||
template<typename T> __forceinline Vec3<T> nmadd ( const Vec3<T>& a, const Vec3<T>& b, const Vec3<T>& c) { return Vec3<T>(nmadd(a.x,b.x,c.x),nmadd(a.y,b.y,c.y),nmadd(a.z,b.z,c.z));}
|
||||
template<typename T> __forceinline Vec3<T> nmsub ( const Vec3<T>& a, const Vec3<T>& b, const Vec3<T>& c) { return Vec3<T>(nmsub(a.x,b.x,c.x),nmsub(a.y,b.y,c.y),nmsub(a.z,b.z,c.z)); }
|
||||
|
||||
template<typename T> __forceinline Vec3<T> madd ( const T& a, const Vec3<T>& b, const Vec3<T>& c) { return Vec3<T>( madd(a,b.x,c.x), madd(a,b.y,c.y), madd(a,b.z,c.z)); }
|
||||
template<typename T> __forceinline Vec3<T> msub ( const T& a, const Vec3<T>& b, const Vec3<T>& c) { return Vec3<T>( msub(a,b.x,c.x), msub(a,b.y,c.y), msub(a,b.z,c.z)); }
|
||||
template<typename T> __forceinline Vec3<T> nmadd ( const T& a, const Vec3<T>& b, const Vec3<T>& c) { return Vec3<T>(nmadd(a,b.x,c.x),nmadd(a,b.y,c.y),nmadd(a,b.z,c.z));}
|
||||
template<typename T> __forceinline Vec3<T> nmsub ( const T& a, const Vec3<T>& b, const Vec3<T>& c) { return Vec3<T>(nmsub(a,b.x,c.x),nmsub(a,b.y,c.y),nmsub(a,b.z,c.z)); }
|
||||
template<typename T> __forceinline const Vec3<T> madd ( const Vec3<T>& a, const Vec3<T>& b, const Vec3<T>& c) { return a*b+c; }
|
||||
template<typename T> __forceinline const Vec3<T> msub ( const Vec3<T>& a, const Vec3<T>& b, const Vec3<T>& c) { return a*b-c; }
|
||||
template<typename T> __forceinline const Vec3<T> nmadd ( const Vec3<T>& a, const Vec3<T>& b, const Vec3<T>& c) { return -a*b-c;}
|
||||
template<typename T> __forceinline const Vec3<T> nmsub ( const Vec3<T>& a, const Vec3<T>& b, const Vec3<T>& c) { return c-a*b; }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Assignment Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
template<typename T> __forceinline Vec3<T>& operator +=( Vec3<T>& a, const T b ) { a.x += b; a.y += b; a.z += b; return a; }
|
||||
template<typename T> __forceinline Vec3<T>& operator +=( Vec3<T>& a, const Vec3<T>& b ) { a.x += b.x; a.y += b.y; a.z += b.z; return a; }
|
||||
template<typename T> __forceinline Vec3<T>& operator -=( Vec3<T>& a, const Vec3<T>& b ) { a.x -= b.x; a.y -= b.y; a.z -= b.z; return a; }
|
||||
template<typename T> __forceinline Vec3<T>& operator *=( Vec3<T>& a, const T& b ) { a.x *= b ; a.y *= b ; a.z *= b ; return a; }
|
||||
template<typename T> __forceinline Vec3<T>& operator /=( Vec3<T>& a, const T& b ) { a.x /= b ; a.y /= b ; a.z /= b ; return a; }
|
||||
template<typename T> __forceinline const Vec3<T>& operator +=( Vec3<T>& a, const T b ) { a.x += b; a.y += b; a.z += b; return a; }
|
||||
template<typename T> __forceinline const Vec3<T>& operator +=( Vec3<T>& a, const Vec3<T>& b ) { a.x += b.x; a.y += b.y; a.z += b.z; return a; }
|
||||
template<typename T> __forceinline const Vec3<T>& operator -=( Vec3<T>& a, const Vec3<T>& b ) { a.x -= b.x; a.y -= b.y; a.z -= b.z; return a; }
|
||||
template<typename T> __forceinline const Vec3<T>& operator *=( Vec3<T>& a, const T& b ) { a.x *= b ; a.y *= b ; a.z *= b ; return a; }
|
||||
template<typename T> __forceinline const Vec3<T>& operator /=( Vec3<T>& a, const T& b ) { a.x /= b ; a.y /= b ; a.z /= b ; return a; }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Reduction Operators
|
||||
@@ -144,6 +152,16 @@ namespace embree
|
||||
return false;
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Euclidian Space Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
template<typename T> __forceinline T dot ( const Vec3<T>& a, const Vec3<T>& b ) { return madd(a.x,b.x,madd(a.y,b.y,a.z*b.z)); }
|
||||
template<typename T> __forceinline T length ( const Vec3<T>& a ) { return sqrt(dot(a,a)); }
|
||||
template<typename T> __forceinline Vec3<T> normalize( const Vec3<T>& a ) { return a*rsqrt(dot(a,a)); }
|
||||
template<typename T> __forceinline T distance ( const Vec3<T>& a, const Vec3<T>& b ) { return length(a-b); }
|
||||
template<typename T> __forceinline Vec3<T> cross ( const Vec3<T>& a, const Vec3<T>& b ) { return Vec3<T>(msub(a.y,b.z,a.z*b.y), msub(a.z,b.x,a.x*b.z), msub(a.x,b.y,a.y*b.x)); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Select
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
@@ -156,29 +174,22 @@ namespace embree
|
||||
return Vec3<T>(select(s.x,t.x,f.x),select(s.y,t.y,f.y),select(s.z,t.z,f.z));
|
||||
}
|
||||
|
||||
template<typename T> __forceinline Vec3<T> select ( const typename T::Bool& s, const Vec3<T>& t, const Vec3<T>& f ) {
|
||||
template<typename T> __forceinline Vec3<T> select ( const typename T::Mask& s, const Vec3<T>& t, const Vec3<T>& f ) {
|
||||
return Vec3<T>(select(s,t.x,f.x),select(s,t.y,f.y),select(s,t.z,f.z));
|
||||
}
|
||||
|
||||
template<typename T>
|
||||
__forceinline Vec3<T> lerp(const Vec3<T>& v0, const Vec3<T>& v1, const T& t) {
|
||||
return madd(Vec3<T>(T(1.0f)-t),v0,t*v1);
|
||||
}
|
||||
|
||||
template<typename T> __forceinline int maxDim ( const Vec3<T>& a )
|
||||
{
|
||||
const Vec3<T> b = abs(a);
|
||||
if (b.x > b.y) {
|
||||
if (b.x > b.z) return 0; else return 2;
|
||||
template<typename T> __forceinline int maxDim ( const Vec3<T>& a )
|
||||
{
|
||||
if (a.x > a.y) {
|
||||
if (a.x > a.z) return 0; else return 2;
|
||||
} else {
|
||||
if (b.y > b.z) return 1; else return 2;
|
||||
if (a.y > a.z) return 1; else return 2;
|
||||
}
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Comparison Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
template<typename T> __forceinline Vec3<bool> eq_mask( const Vec3<T>& a, const Vec3<T>& b ) { return Vec3<bool>(a.x==b.x,a.y==b.y,a.z==b.z); }
|
||||
template<typename T> __forceinline Vec3<bool> neq_mask(const Vec3<T>& a, const Vec3<T>& b ) { return Vec3<bool>(a.x!=b.x,a.y!=b.y,a.z!=b.z); }
|
||||
template<typename T> __forceinline Vec3<bool> lt_mask( const Vec3<T>& a, const Vec3<T>& b ) { return Vec3<bool>(a.x< b.x,a.y< b.y,a.z< b.z); }
|
||||
@@ -186,54 +197,6 @@ namespace embree
|
||||
template<typename T> __forceinline Vec3<bool> gt_mask( const Vec3<T>& a, const Vec3<T>& b ) { return Vec3<bool>(a.x> b.x,a.y> b.y,a.z> b.z); }
|
||||
template<typename T> __forceinline Vec3<bool> ge_mask( const Vec3<T>& a, const Vec3<T>& b ) { return Vec3<bool>(a.x>=b.x,a.y>=b.y,a.z>=b.z); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Euclidian Space Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
template<typename T> __forceinline T dot ( const Vec3<T>& a, const Vec3<T>& b ) { return madd(a.x,b.x,madd(a.y,b.y,a.z*b.z)); }
|
||||
template<typename T> __forceinline T length ( const Vec3<T>& a ) { return sqrt(dot(a,a)); }
|
||||
template<typename T> __forceinline T rcp_length( const Vec3<T>& a ) { return rsqrt(dot(a,a)); }
|
||||
template<typename T> __forceinline Vec3<T> normalize( const Vec3<T>& a ) { return a*rsqrt(dot(a,a)); }
|
||||
template<typename T> __forceinline T distance ( const Vec3<T>& a, const Vec3<T>& b ) { return length(a-b); }
|
||||
template<typename T> __forceinline Vec3<T> cross ( const Vec3<T>& a, const Vec3<T>& b ) { return Vec3<T>(msub(a.y,b.z,a.z*b.y), msub(a.z,b.x,a.x*b.z), msub(a.x,b.y,a.y*b.x)); }
|
||||
|
||||
template<typename T> __forceinline Vec3<T> stable_triangle_normal( const Vec3<T>& a, const Vec3<T>& b, const Vec3<T>& c )
|
||||
{
|
||||
const T ab_x = a.z*b.y, ab_y = a.x*b.z, ab_z = a.y*b.x;
|
||||
const T bc_x = b.z*c.y, bc_y = b.x*c.z, bc_z = b.y*c.x;
|
||||
const Vec3<T> cross_ab(msub(a.y,b.z,ab_x), msub(a.z,b.x,ab_y), msub(a.x,b.y,ab_z));
|
||||
const Vec3<T> cross_bc(msub(b.y,c.z,bc_x), msub(b.z,c.x,bc_y), msub(b.x,c.y,bc_z));
|
||||
const auto sx = abs(ab_x) < abs(bc_x);
|
||||
const auto sy = abs(ab_y) < abs(bc_y);
|
||||
const auto sz = abs(ab_z) < abs(bc_z);
|
||||
return Vec3<T>(select(sx,cross_ab.x,cross_bc.x),
|
||||
select(sy,cross_ab.y,cross_bc.y),
|
||||
select(sz,cross_ab.z,cross_bc.z));
|
||||
}
|
||||
|
||||
template<typename T> __forceinline T sum ( const Vec3<T>& a ) { return a.x+a.y+a.z; }
|
||||
|
||||
template<typename T> __forceinline T halfArea ( const Vec3<T>& d ) { return madd(d.x,(d.y+d.z),d.y*d.z); }
|
||||
template<typename T> __forceinline T area ( const Vec3<T>& d ) { return 2.0f*halfArea(d); }
|
||||
|
||||
template<typename T> __forceinline Vec3<T> normalize_safe( const Vec3<T>& a ) {
|
||||
const T d = dot(a,a); return select(d == T( zero ), a , a*rsqrt(d) );
|
||||
}
|
||||
|
||||
template<typename T> __forceinline T sqr_point_to_line_distance(const Vec3<T>& P, const Vec3<T>& Q0, const Vec3<T>& Q1)
|
||||
{
|
||||
const Vec3<T> N = cross(P-Q0,Q1-Q0);
|
||||
const Vec3<T> D = Q1-Q0;
|
||||
return dot(N,N)*rcp(dot(D,D));
|
||||
}
|
||||
|
||||
template<typename T> __forceinline T sqr_point_to_line_distance(const Vec3<T>& PmQ0, const Vec3<T>& Q1mQ0)
|
||||
{
|
||||
const Vec3<T> N = cross(PmQ0,Q1mQ0);
|
||||
const Vec3<T> D = Q1mQ0;
|
||||
return dot(N,N)*rcp(dot(D,D));
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Output Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
@@ -247,93 +210,46 @@ namespace embree
|
||||
typedef Vec3<float> Vec3f;
|
||||
}
|
||||
|
||||
#include "vec3ba.h"
|
||||
#include "vec3ia.h"
|
||||
#include "vec3fa.h"
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// SSE / AVX / MIC specializations
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
#if defined __SSE__
|
||||
#include "../simd/sse.h"
|
||||
#if defined(__MIC__)
|
||||
#include "vec3ba_mic.h"
|
||||
#include "vec3ia_mic.h"
|
||||
#include "vec3fa_mic.h"
|
||||
#else
|
||||
#include "vec3ba.h"
|
||||
#include "vec3ia.h"
|
||||
#include "vec3fa.h"
|
||||
#endif
|
||||
|
||||
#if defined __AVX__
|
||||
#include "../simd/avx.h"
|
||||
#endif
|
||||
namespace embree
|
||||
{
|
||||
template<> __forceinline Vec3<float>::Vec3( const Vec3fa& a ) { x = a.x; y = a.y; z = a.z; }
|
||||
|
||||
#if defined(__AVX512F__)
|
||||
#include "../simd/avx512.h"
|
||||
#endif
|
||||
|
||||
namespace embree
|
||||
{
|
||||
template<typename Out, typename In>
|
||||
__forceinline Vec3<Out> broadcast(const Vec3<In>& a, const size_t k) {
|
||||
return Vec3<Out>(Out(a.x[k]), Out(a.y[k]), Out(a.z[k]));
|
||||
#if defined (__SSE__)
|
||||
template<> __forceinline Vec3<ssef>::Vec3( const Vec3fa& a ) {
|
||||
const ssef v = ssef(a); x = shuffle<0,0,0,0>(v); y = shuffle<1,1,1,1>(v); z = shuffle<2,2,2,2>(v);
|
||||
}
|
||||
|
||||
template<> __forceinline Vec3<float>::Vec3(const Vec3fa& a) { x = a.x; y = a.y; z = a.z; }
|
||||
|
||||
#if defined(__AVX__)
|
||||
template<> __forceinline Vec3<vfloat4>::Vec3(const Vec3fa& a) {
|
||||
x = a.x; y = a.y; z = a.z;
|
||||
}
|
||||
#elif defined(__SSE__)
|
||||
template<>
|
||||
__forceinline Vec3<vfloat4>::Vec3(const Vec3fa& a) {
|
||||
const vfloat4 v = vfloat4(a); x = shuffle<0,0,0,0>(v); y = shuffle<1,1,1,1>(v); z = shuffle<2,2,2,2>(v);
|
||||
}
|
||||
#endif
|
||||
|
||||
#if defined(__SSE__)
|
||||
__forceinline Vec3<vfloat4> broadcast4f(const Vec3<vfloat4>& a, const size_t k) {
|
||||
return Vec3<vfloat4>(vfloat4::broadcast(&a.x[k]), vfloat4::broadcast(&a.y[k]), vfloat4::broadcast(&a.z[k]));
|
||||
}
|
||||
|
||||
template<>
|
||||
__forceinline Vec3<vfloat4> broadcast<vfloat4,vfloat4>(const Vec3<vfloat4>& a, const size_t k) {
|
||||
return Vec3<vfloat4>(vfloat4::broadcast(&a.x[k]), vfloat4::broadcast(&a.y[k]), vfloat4::broadcast(&a.z[k]));
|
||||
}
|
||||
|
||||
template<int i0, int i1, int i2, int i3>
|
||||
__forceinline Vec3<vfloat4> shuffle(const Vec3<vfloat4>& b) {
|
||||
return Vec3<vfloat4>(shuffle<i0,i1,i2,i3>(b.x), shuffle<i0,i1,i2,i3>(b.y), shuffle<i0,i1,i2,i3>(b.z));
|
||||
__forceinline Vec3<ssef> broadcast4f( const Vec3<ssef>& a, const size_t k ) {
|
||||
return Vec3<ssef>(ssef::broadcast(&a.x[k]), ssef::broadcast(&a.y[k]), ssef::broadcast(&a.z[k]));
|
||||
}
|
||||
#endif
|
||||
|
||||
#if defined(__AVX__)
|
||||
template<>
|
||||
__forceinline Vec3<vfloat8>::Vec3(const Vec3fa& a) {
|
||||
x = a.x; y = a.y; z = a.z;
|
||||
template<> __forceinline Vec3<avxf>::Vec3( const Vec3fa& a ) {
|
||||
x = a.x; y = a.y; z = a.z;
|
||||
}
|
||||
__forceinline Vec3<vfloat4> broadcast4f(const Vec3<vfloat8>& a, const size_t k) {
|
||||
return Vec3<vfloat4>(vfloat4::broadcast(&a.x[k]), vfloat4::broadcast(&a.y[k]), vfloat4::broadcast(&a.z[k]));
|
||||
__forceinline Vec3<ssef> broadcast4f( const Vec3<avxf>& a, const size_t k ) {
|
||||
return Vec3<ssef>(ssef::broadcast(&a.x[k]), ssef::broadcast(&a.y[k]), ssef::broadcast(&a.z[k]));
|
||||
}
|
||||
__forceinline Vec3<vfloat8> broadcast8f(const Vec3<vfloat4>& a, const size_t k) {
|
||||
return Vec3<vfloat8>(vfloat8::broadcast(&a.x[k]), vfloat8::broadcast(&a.y[k]), vfloat8::broadcast(&a.z[k]));
|
||||
__forceinline Vec3<avxf> broadcast8f( const Vec3<ssef>& a, const size_t k ) {
|
||||
return Vec3<avxf>(avxf::broadcast(&a.x[k]), avxf::broadcast(&a.y[k]), avxf::broadcast(&a.z[k]));
|
||||
}
|
||||
__forceinline Vec3<vfloat8> broadcast8f(const Vec3<vfloat8>& a, const size_t k) {
|
||||
return Vec3<vfloat8>(vfloat8::broadcast(&a.x[k]), vfloat8::broadcast(&a.y[k]), vfloat8::broadcast(&a.z[k]));
|
||||
}
|
||||
|
||||
template<>
|
||||
__forceinline Vec3<vfloat8> broadcast<vfloat8,vfloat4>(const Vec3<vfloat4>& a, const size_t k) {
|
||||
return Vec3<vfloat8>(vfloat8::broadcast(&a.x[k]), vfloat8::broadcast(&a.y[k]), vfloat8::broadcast(&a.z[k]));
|
||||
}
|
||||
template<>
|
||||
__forceinline Vec3<vfloat8> broadcast<vfloat8,vfloat8>(const Vec3<vfloat8>& a, const size_t k) {
|
||||
return Vec3<vfloat8>(vfloat8::broadcast(&a.x[k]), vfloat8::broadcast(&a.y[k]), vfloat8::broadcast(&a.z[k]));
|
||||
}
|
||||
|
||||
template<int i0, int i1, int i2, int i3>
|
||||
__forceinline Vec3<vfloat8> shuffle(const Vec3<vfloat8>& b) {
|
||||
return Vec3<vfloat8>(shuffle<i0,i1,i2,i3>(b.x), shuffle<i0,i1,i2,i3>(b.y), shuffle<i0,i1,i2,i3>(b.z));
|
||||
__forceinline Vec3<avxf> broadcast8f( const Vec3<avxf>& a, const size_t k ) {
|
||||
return Vec3<avxf>(avxf::broadcast(&a.x[k]), avxf::broadcast(&a.y[k]), avxf::broadcast(&a.z[k]));
|
||||
}
|
||||
#endif
|
||||
|
||||
#if defined(__AVX512F__)
|
||||
template<> __forceinline Vec3<vfloat16>::Vec3(const Vec3fa& a) : x(a.x), y(a.y), z(a.z) {}
|
||||
#if defined(__MIC__)
|
||||
template<> __forceinline Vec3<ssef>::Vec3( const Vec3fa& a ) : x(a.x), y(a.y), z(a.z) {}
|
||||
template<> __forceinline Vec3<mic_f>::Vec3( const Vec3fa& a ) : x(a.x), y(a.y), z(a.z) {}
|
||||
#endif
|
||||
}
|
||||
|
||||
+13
-16
@@ -1,5 +1,5 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
@@ -16,9 +16,8 @@
|
||||
|
||||
#pragma once
|
||||
|
||||
#include "../sys/alloc.h"
|
||||
#include "simd/sse.h"
|
||||
#include "math.h"
|
||||
#include "../simd/sse.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
@@ -28,8 +27,6 @@ namespace embree
|
||||
|
||||
struct __aligned(16) Vec3ba
|
||||
{
|
||||
ALIGNED_STRUCT;
|
||||
|
||||
union {
|
||||
__m128 m128;
|
||||
struct { int x,y,z; int a; };
|
||||
@@ -47,13 +44,13 @@ namespace embree
|
||||
__forceinline Vec3ba( const Vec3ba& other ) : m128(other.m128) {}
|
||||
__forceinline Vec3ba& operator =(const Vec3ba& other) { m128 = other.m128; return *this; }
|
||||
|
||||
__forceinline explicit Vec3ba( bool a )
|
||||
__forceinline explicit Vec3ba ( bool a )
|
||||
: m128(_mm_lookupmask_ps[(size_t(a) << 3) | (size_t(a) << 2) | (size_t(a) << 1) | size_t(a)]) {}
|
||||
__forceinline Vec3ba( bool a, bool b, bool c)
|
||||
__forceinline explicit Vec3ba ( bool a, bool b, bool c)
|
||||
: m128(_mm_lookupmask_ps[(size_t(c) << 2) | (size_t(b) << 1) | size_t(a)]) {}
|
||||
|
||||
__forceinline operator const __m128&() const { return m128; }
|
||||
__forceinline operator __m128&() { return m128; }
|
||||
__forceinline operator const __m128&( void ) const { return m128; }
|
||||
__forceinline operator __m128&( void ) { return m128; }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constants
|
||||
@@ -75,23 +72,23 @@ namespace embree
|
||||
/// Unary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline Vec3ba operator !( const Vec3ba& a ) { return _mm_xor_ps(a.m128, Vec3ba(embree::True)); }
|
||||
__forceinline const Vec3ba operator !( const Vec3ba& a ) { return _mm_xor_ps(a.m128, Vec3ba(True)); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Binary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline Vec3ba operator &( const Vec3ba& a, const Vec3ba& b ) { return _mm_and_ps(a.m128, b.m128); }
|
||||
__forceinline Vec3ba operator |( const Vec3ba& a, const Vec3ba& b ) { return _mm_or_ps (a.m128, b.m128); }
|
||||
__forceinline Vec3ba operator ^( const Vec3ba& a, const Vec3ba& b ) { return _mm_xor_ps(a.m128, b.m128); }
|
||||
__forceinline const Vec3ba operator &( const Vec3ba& a, const Vec3ba& b ) { return _mm_and_ps(a.m128, b.m128); }
|
||||
__forceinline const Vec3ba operator |( const Vec3ba& a, const Vec3ba& b ) { return _mm_or_ps (a.m128, b.m128); }
|
||||
__forceinline const Vec3ba operator ^( const Vec3ba& a, const Vec3ba& b ) { return _mm_xor_ps(a.m128, b.m128); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Assignment Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline Vec3ba& operator &=( Vec3ba& a, const Vec3ba& b ) { return a = a & b; }
|
||||
__forceinline Vec3ba& operator |=( Vec3ba& a, const Vec3ba& b ) { return a = a | b; }
|
||||
__forceinline Vec3ba& operator ^=( Vec3ba& a, const Vec3ba& b ) { return a = a ^ b; }
|
||||
__forceinline const Vec3ba operator &=( Vec3ba& a, const Vec3ba& b ) { return a = a & b; }
|
||||
__forceinline const Vec3ba operator |=( Vec3ba& a, const Vec3ba& b ) { return a = a | b; }
|
||||
__forceinline const Vec3ba operator ^=( Vec3ba& a, const Vec3ba& b ) { return a = a ^ b; }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Comparison Operators + Select
|
||||
|
||||
@@ -0,0 +1,117 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#pragma once
|
||||
|
||||
#include "math.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// MIC Vec3ba Type
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
/*! 3-wide bool type emulated with 16-wide bit vectors. */
|
||||
struct Vec3ba
|
||||
{
|
||||
enum { N = 3 };
|
||||
__mmask16 v;
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constructors, Assignment & Cast Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline Vec3ba( ) {}
|
||||
__forceinline Vec3ba ( const __mmask16 a ) : v(a) {}
|
||||
__forceinline Vec3ba ( const Vec3ba& a ) : v(a.v) {}
|
||||
__forceinline Vec3ba& operator =(const Vec3ba& a) { v = a.v; return *this; }
|
||||
|
||||
__forceinline explicit Vec3ba ( bool x ) : v(x ? 0xF : 0x0) {}
|
||||
__forceinline explicit Vec3ba ( bool x, bool y, bool z ) : v(1*__mmask16(x)+2*__mmask16(y)+4*__mmask16(z)) {}
|
||||
|
||||
__forceinline operator const __mmask16&( void ) const { return v; }
|
||||
__forceinline operator __mmask16&( void ) { return v; }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constants
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline Vec3ba( FalseTy ) : v(0x0) {}
|
||||
__forceinline Vec3ba( TrueTy ) : v(0xF) {}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Array Access
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline bool operator []( const size_t index ) const { assert(index < 3); return (v >> index) & 1; }
|
||||
//__forceinline int& operator []( const size_t index ) { assert(index < 3); return (&x)[index]; }
|
||||
};
|
||||
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Unary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const Vec3ba operator !( const Vec3ba& a ) { return _mm512_knot(a); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Binary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const Vec3ba operator &( const Vec3ba& a, const Vec3ba& b ) { return _mm512_kand(a.v, b.v); }
|
||||
__forceinline const Vec3ba operator |( const Vec3ba& a, const Vec3ba& b ) { return _mm512_kor (a.v, b.v); }
|
||||
__forceinline const Vec3ba operator ^( const Vec3ba& a, const Vec3ba& b ) { return _mm512_kxor(a.v, b.v); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Assignment Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const Vec3ba operator &=( Vec3ba& a, const Vec3ba& b ) { return a = a & b; }
|
||||
__forceinline const Vec3ba operator |=( Vec3ba& a, const Vec3ba& b ) { return a = a | b; }
|
||||
__forceinline const Vec3ba operator ^=( Vec3ba& a, const Vec3ba& b ) { return a = a ^ b; }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Comparison Operators + Select
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline bool operator ==( const Vec3ba& a, const Vec3ba& b ) { return _mm512_kxnor(a, b); }
|
||||
__forceinline bool operator !=( const Vec3ba& a, const Vec3ba& b ) { return _mm512_kxor (a, b); }
|
||||
__forceinline bool operator < ( const Vec3ba& a, const Vec3ba& b ) {
|
||||
if (a[0] != b[0]) return a[0] < b[0];
|
||||
if (a[1] != b[1]) return a[1] < b[1];
|
||||
if (a[2] != b[2]) return a[2] < b[2];
|
||||
return false;
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Reduction Operations
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline bool reduce_and( const Vec3ba& a ) { return (a.v & 0x7) == 0x7; }
|
||||
__forceinline bool reduce_or ( const Vec3ba& a ) { return (a.v & 0x7) != 0x0; }
|
||||
|
||||
__forceinline int all(const Vec3ba& a) { return (a.v & 0x7) == 0x7; }
|
||||
__forceinline int any(const Vec3ba& a) { return (a.v & 0x7) != 0x0; }
|
||||
__forceinline int none(const Vec3ba& a) { return (a.v & 0x7) == 0x0; }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Output Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
inline std::ostream& operator<<(std::ostream& cout, const Vec3ba& a) {
|
||||
return cout << "(" << (a[0] ? "1" : "0") << ", " << (a[1] ? "1" : "0") << ", " << (a[2] ? "1" : "0") << ")";
|
||||
}
|
||||
}
|
||||
+72
-150
@@ -1,5 +1,5 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
@@ -16,9 +16,8 @@
|
||||
|
||||
#pragma once
|
||||
|
||||
#include "../sys/alloc.h"
|
||||
#include "simd/sse.h"
|
||||
#include "math.h"
|
||||
#include "../simd/sse.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
@@ -28,8 +27,6 @@ namespace embree
|
||||
|
||||
struct __aligned(16) Vec3fa
|
||||
{
|
||||
ALIGNED_STRUCT;
|
||||
|
||||
typedef float Scalar;
|
||||
enum { N = 3 };
|
||||
union {
|
||||
@@ -48,45 +45,22 @@ namespace embree
|
||||
__forceinline Vec3fa& operator =( const Vec3<float>& other ) { x = other.x; y = other.y; z = other.z; return *this; }
|
||||
|
||||
__forceinline Vec3fa ( const Vec3fa& other ) { m128 = other.m128; }
|
||||
|
||||
__forceinline Vec3fa& operator =( const Vec3fa& other ) { m128 = other.m128; return *this; }
|
||||
|
||||
__forceinline explicit Vec3fa( const float a ) : m128(_mm_set1_ps(a)) {}
|
||||
__forceinline Vec3fa( const float x, const float y, const float z) : m128(_mm_set_ps(z, z, y, x)) {}
|
||||
__forceinline Vec3fa( const float a ) : m128(_mm_set1_ps(a)) {}
|
||||
__forceinline Vec3fa( const float x, const float y, const float z) : m128(_mm_set_ps(z, z, y, x)) {}
|
||||
|
||||
__forceinline Vec3fa( const Vec3fa& other, const int a1) { m128 = other.m128; a = a1; }
|
||||
__forceinline Vec3fa( const Vec3fa& other, const unsigned a1) { m128 = other.m128; u = a1; }
|
||||
__forceinline Vec3fa( const Vec3fa& other, const float w1) { m128 = other.m128; w = w1; }
|
||||
//__forceinline Vec3fa( const float x, const float y, const float z, const int a) : x(x), y(y), z(z), a(a) {} // not working properly!
|
||||
//__forceinline Vec3fa( const float x, const float y, const float z, const unsigned a) : x(x), y(y), z(z), u(a) {} // not working properly!
|
||||
__forceinline Vec3fa( const float x, const float y, const float z, const float w) : m128(_mm_set_ps(w, z, y, x)) {}
|
||||
__forceinline explicit Vec3fa( const Vec3fa& other, const int a1) { m128 = other.m128; a = a1; }
|
||||
__forceinline explicit Vec3fa( const Vec3fa& other, const unsigned a1) { m128 = other.m128; a = a1; }
|
||||
__forceinline explicit Vec3fa( const Vec3fa& other, const float w1) { m128 = other.m128; w = w1; }
|
||||
__forceinline explicit Vec3fa( const float x, const float y, const float z, const int a) : x(x), y(y), z(z), a(a) {}
|
||||
__forceinline explicit Vec3fa( const float x, const float y, const float z, const unsigned a) : x(x), y(y), z(z), a(a) {}
|
||||
__forceinline explicit Vec3fa( const float x, const float y, const float z, const float w) : x(x), y(y), z(z), w(w) {}
|
||||
|
||||
__forceinline explicit Vec3fa( const __m128i a ) : m128(_mm_cvtepi32_ps(a)) {}
|
||||
|
||||
__forceinline operator const __m128&() const { return m128; }
|
||||
__forceinline operator __m128&() { return m128; }
|
||||
|
||||
#if defined (__SSE4_1__)
|
||||
friend __forceinline Vec3fa copy_a( const Vec3fa& a, const Vec3fa& b ) { return _mm_insert_ps(a, b, (3 << 4) | (3 << 6)); }
|
||||
#else
|
||||
friend __forceinline Vec3fa copy_a( const Vec3fa& a, const Vec3fa& b ) { Vec3fa c = a; c.a = b.a; return c; }
|
||||
#endif
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Loads and Stores
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
static __forceinline Vec3fa load( const void* const a ) {
|
||||
return Vec3fa(_mm_and_ps(_mm_load_ps((float*)a),_mm_castsi128_ps(_mm_set_epi32(0, -1, -1, -1))));
|
||||
}
|
||||
|
||||
static __forceinline Vec3fa loadu( const void* const a ) {
|
||||
return Vec3fa(_mm_and_ps(_mm_loadu_ps((float*)a),_mm_castsi128_ps(_mm_set_epi32(0, -1, -1, -1))));
|
||||
}
|
||||
|
||||
static __forceinline void storeu ( void* ptr, const Vec3fa& v ) {
|
||||
_mm_storeu_ps((float*)ptr,v);
|
||||
}
|
||||
__forceinline operator const __m128&( void ) const { return m128; }
|
||||
__forceinline operator __m128&( void ) { return m128; }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constants
|
||||
@@ -104,66 +78,41 @@ namespace embree
|
||||
__forceinline const float& operator []( const size_t index ) const { assert(index < 3); return (&x)[index]; }
|
||||
__forceinline float& operator []( const size_t index ) { assert(index < 3); return (&x)[index]; }
|
||||
};
|
||||
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Unary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline Vec3fa operator +( const Vec3fa& a ) { return a; }
|
||||
__forceinline Vec3fa operator -( const Vec3fa& a ) {
|
||||
__forceinline const Vec3fa operator +( const Vec3fa& a ) { return a; }
|
||||
__forceinline const Vec3fa operator -( const Vec3fa& a ) {
|
||||
const __m128 mask = _mm_castsi128_ps(_mm_set1_epi32(0x80000000));
|
||||
return _mm_xor_ps(a.m128, mask);
|
||||
}
|
||||
__forceinline Vec3fa abs ( const Vec3fa& a ) {
|
||||
__forceinline const Vec3fa abs ( const Vec3fa& a ) {
|
||||
const __m128 mask = _mm_castsi128_ps(_mm_set1_epi32(0x7fffffff));
|
||||
return _mm_and_ps(a.m128, mask);
|
||||
}
|
||||
__forceinline Vec3fa sign ( const Vec3fa& a ) {
|
||||
return blendv_ps(Vec3fa(one), -Vec3fa(one), _mm_cmplt_ps (a,Vec3fa(zero)));
|
||||
__forceinline const Vec3fa sign ( const Vec3fa& a ) {
|
||||
return _mm_blendv_ps(Vec3fa(one), -Vec3fa(one), _mm_cmplt_ps (a,Vec3fa(zero)));
|
||||
}
|
||||
|
||||
__forceinline Vec3fa rcp ( const Vec3fa& a )
|
||||
{
|
||||
#if defined(__AVX512VL__)
|
||||
const Vec3fa r = _mm_rcp14_ps(a.m128);
|
||||
#else
|
||||
__forceinline const Vec3fa rcp ( const Vec3fa& a ) {
|
||||
const Vec3fa r = _mm_rcp_ps(a.m128);
|
||||
#endif
|
||||
|
||||
#if defined(__AVX2__)
|
||||
const Vec3fa res = _mm_mul_ps(r,_mm_fnmadd_ps(r, a, vfloat4(2.0f)));
|
||||
#else
|
||||
const Vec3fa res = _mm_mul_ps(r,_mm_sub_ps(vfloat4(2.0f), _mm_mul_ps(r, a)));
|
||||
//return _mm_sub_ps(_mm_add_ps(r, r), _mm_mul_ps(_mm_mul_ps(r, r), a));
|
||||
#endif
|
||||
|
||||
return res;
|
||||
return _mm_sub_ps(_mm_add_ps(r, r), _mm_mul_ps(_mm_mul_ps(r, r), a));
|
||||
}
|
||||
|
||||
__forceinline Vec3fa sqrt ( const Vec3fa& a ) { return _mm_sqrt_ps(a.m128); }
|
||||
__forceinline Vec3fa sqr ( const Vec3fa& a ) { return _mm_mul_ps(a,a); }
|
||||
|
||||
__forceinline Vec3fa rsqrt( const Vec3fa& a )
|
||||
{
|
||||
#if defined(__AVX512VL__)
|
||||
__m128 r = _mm_rsqrt14_ps(a.m128);
|
||||
#else
|
||||
__forceinline const Vec3fa sqrt ( const Vec3fa& a ) { return _mm_sqrt_ps(a.m128); }
|
||||
__forceinline const Vec3fa sqr ( const Vec3fa& a ) { return _mm_mul_ps(a,a); }
|
||||
__forceinline const Vec3fa rsqrt( const Vec3fa& a ) {
|
||||
__m128 r = _mm_rsqrt_ps(a.m128);
|
||||
#endif
|
||||
return _mm_add_ps(_mm_mul_ps(_mm_set1_ps(1.5f),r), _mm_mul_ps(_mm_mul_ps(_mm_mul_ps(a, _mm_set1_ps(-0.5f)), r), _mm_mul_ps(r, r)));
|
||||
}
|
||||
__forceinline const Vec3fa zero_fix(const Vec3fa& a) { return _mm_blendv_ps(a, _mm_set1_ps(1E-10f), _mm_cmpeq_ps (a.m128, _mm_setzero_ps())); }
|
||||
__forceinline const Vec3fa rcp_safe(const Vec3fa& a) { return rcp(zero_fix(a)); }
|
||||
|
||||
__forceinline Vec3fa zero_fix(const Vec3fa& a) {
|
||||
return blendv_ps(a, _mm_set1_ps(min_rcp_input), _mm_cmplt_ps (abs(a).m128, _mm_set1_ps(min_rcp_input)));
|
||||
}
|
||||
__forceinline Vec3fa rcp_safe(const Vec3fa& a) {
|
||||
return rcp(zero_fix(a));
|
||||
}
|
||||
__forceinline Vec3fa log ( const Vec3fa& a ) {
|
||||
__forceinline Vec3fa log ( const Vec3fa& a ) {
|
||||
return Vec3fa(logf(a.x),logf(a.y),logf(a.z));
|
||||
}
|
||||
|
||||
__forceinline Vec3fa exp ( const Vec3fa& a ) {
|
||||
__forceinline Vec3fa exp ( const Vec3fa& a ) {
|
||||
return Vec3fa(expf(a.x),expf(a.y),expf(a.z));
|
||||
}
|
||||
|
||||
@@ -171,37 +120,37 @@ namespace embree
|
||||
/// Binary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline Vec3fa operator +( const Vec3fa& a, const Vec3fa& b ) { return _mm_add_ps(a.m128, b.m128); }
|
||||
__forceinline Vec3fa operator -( const Vec3fa& a, const Vec3fa& b ) { return _mm_sub_ps(a.m128, b.m128); }
|
||||
__forceinline Vec3fa operator *( const Vec3fa& a, const Vec3fa& b ) { return _mm_mul_ps(a.m128, b.m128); }
|
||||
__forceinline Vec3fa operator *( const Vec3fa& a, const float b ) { return a * Vec3fa(b); }
|
||||
__forceinline Vec3fa operator *( const float a, const Vec3fa& b ) { return Vec3fa(a) * b; }
|
||||
__forceinline Vec3fa operator /( const Vec3fa& a, const Vec3fa& b ) { return _mm_div_ps(a.m128,b.m128); }
|
||||
__forceinline Vec3fa operator /( const Vec3fa& a, const float b ) { return _mm_div_ps(a.m128,_mm_set1_ps(b)); }
|
||||
__forceinline Vec3fa operator /( const float a, const Vec3fa& b ) { return _mm_div_ps(_mm_set1_ps(a),b.m128); }
|
||||
__forceinline const Vec3fa operator +( const Vec3fa& a, const Vec3fa& b ) { return _mm_add_ps(a.m128, b.m128); }
|
||||
__forceinline const Vec3fa operator -( const Vec3fa& a, const Vec3fa& b ) { return _mm_sub_ps(a.m128, b.m128); }
|
||||
__forceinline const Vec3fa operator *( const Vec3fa& a, const Vec3fa& b ) { return _mm_mul_ps(a.m128, b.m128); }
|
||||
__forceinline const Vec3fa operator *( const Vec3fa& a, const float b ) { return a * Vec3fa(b); }
|
||||
__forceinline const Vec3fa operator *( const float a, const Vec3fa& b ) { return Vec3fa(a) * b; }
|
||||
__forceinline const Vec3fa operator /( const Vec3fa& a, const Vec3fa& b ) { return _mm_div_ps(a.m128,b.m128); }
|
||||
__forceinline const Vec3fa operator /( const Vec3fa& a, const float b ) { return _mm_div_ps(a.m128,_mm_set1_ps(b)); }
|
||||
__forceinline const Vec3fa operator /( const float a, const Vec3fa& b ) { return _mm_div_ps(_mm_set1_ps(a),b.m128); }
|
||||
|
||||
__forceinline Vec3fa min( const Vec3fa& a, const Vec3fa& b ) { return _mm_min_ps(a.m128,b.m128); }
|
||||
__forceinline Vec3fa max( const Vec3fa& a, const Vec3fa& b ) { return _mm_max_ps(a.m128,b.m128); }
|
||||
__forceinline const Vec3fa min( const Vec3fa& a, const Vec3fa& b ) { return _mm_min_ps(a.m128,b.m128); }
|
||||
__forceinline const Vec3fa max( const Vec3fa& a, const Vec3fa& b ) { return _mm_max_ps(a.m128,b.m128); }
|
||||
|
||||
#if defined(__SSE4_1__)
|
||||
__forceinline Vec3fa mini(const Vec3fa& a, const Vec3fa& b) {
|
||||
const vint4 ai = _mm_castps_si128(a);
|
||||
const vint4 bi = _mm_castps_si128(b);
|
||||
const vint4 ci = _mm_min_epi32(ai,bi);
|
||||
const ssei ai = _mm_castps_si128(a);
|
||||
const ssei bi = _mm_castps_si128(b);
|
||||
const ssei ci = _mm_min_epi32(ai,bi);
|
||||
return _mm_castsi128_ps(ci);
|
||||
}
|
||||
#endif
|
||||
|
||||
|
||||
#if defined(__SSE4_1__)
|
||||
__forceinline Vec3fa maxi(const Vec3fa& a, const Vec3fa& b) {
|
||||
const vint4 ai = _mm_castps_si128(a);
|
||||
const vint4 bi = _mm_castps_si128(b);
|
||||
const vint4 ci = _mm_max_epi32(ai,bi);
|
||||
const ssei ai = _mm_castps_si128(a);
|
||||
const ssei bi = _mm_castps_si128(b);
|
||||
const ssei ci = _mm_max_epi32(ai,bi);
|
||||
return _mm_castsi128_ps(ci);
|
||||
}
|
||||
#endif
|
||||
|
||||
__forceinline Vec3fa pow ( const Vec3fa& a, const float& b ) {
|
||||
__forceinline Vec3fa pow ( const Vec3fa& a, const float& b ) {
|
||||
return Vec3fa(powf(a.x,b),powf(a.y,b),powf(a.z,b));
|
||||
}
|
||||
|
||||
@@ -217,15 +166,10 @@ namespace embree
|
||||
#else
|
||||
__forceinline Vec3fa madd ( const Vec3fa& a, const Vec3fa& b, const Vec3fa& c) { return a*b+c; }
|
||||
__forceinline Vec3fa msub ( const Vec3fa& a, const Vec3fa& b, const Vec3fa& c) { return a*b-c; }
|
||||
__forceinline Vec3fa nmadd ( const Vec3fa& a, const Vec3fa& b, const Vec3fa& c) { return -a*b+c;}
|
||||
__forceinline Vec3fa nmsub ( const Vec3fa& a, const Vec3fa& b, const Vec3fa& c) { return -a*b-c; }
|
||||
__forceinline Vec3fa nmadd ( const Vec3fa& a, const Vec3fa& b, const Vec3fa& c) { return -a*b-c;}
|
||||
__forceinline Vec3fa nmsub ( const Vec3fa& a, const Vec3fa& b, const Vec3fa& c) { return c-a*b; }
|
||||
#endif
|
||||
|
||||
__forceinline Vec3fa madd ( const float a, const Vec3fa& b, const Vec3fa& c) { return madd(Vec3fa(a),b,c); }
|
||||
__forceinline Vec3fa msub ( const float a, const Vec3fa& b, const Vec3fa& c) { return msub(Vec3fa(a),b,c); }
|
||||
__forceinline Vec3fa nmadd ( const float a, const Vec3fa& b, const Vec3fa& c) { return nmadd(Vec3fa(a),b,c); }
|
||||
__forceinline Vec3fa nmsub ( const float a, const Vec3fa& b, const Vec3fa& c) { return nmsub(Vec3fa(a),b,c); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Assignment Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
@@ -260,14 +204,6 @@ namespace embree
|
||||
__forceinline Vec3ba gt_mask( const Vec3fa& a, const Vec3fa& b ) { return _mm_cmpnle_ps(a.m128, b.m128); }
|
||||
__forceinline Vec3ba ge_mask( const Vec3fa& a, const Vec3fa& b ) { return _mm_cmpnlt_ps(a.m128, b.m128); }
|
||||
|
||||
__forceinline bool isvalid ( const Vec3fa& v ) {
|
||||
return all(gt_mask(v,Vec3fa(-FLT_LARGE)) & lt_mask(v,Vec3fa(+FLT_LARGE)));
|
||||
}
|
||||
|
||||
__forceinline bool is_finite ( const Vec3fa& a ) {
|
||||
return all(ge_mask(a,Vec3fa(-FLT_MAX)) & le_mask(a,Vec3fa(+FLT_MAX)));
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Euclidian Space Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
@@ -282,56 +218,40 @@ namespace embree
|
||||
}
|
||||
#endif
|
||||
|
||||
__forceinline Vec3fa cross ( const Vec3fa& a, const Vec3fa& b )
|
||||
__forceinline Vec3fa cross ( const Vec3fa& a, const Vec3fa& b )
|
||||
{
|
||||
vfloat4 a0 = vfloat4(a);
|
||||
vfloat4 b0 = shuffle<1,2,0,3>(vfloat4(b));
|
||||
vfloat4 a1 = shuffle<1,2,0,3>(vfloat4(a));
|
||||
vfloat4 b1 = vfloat4(b);
|
||||
ssef a0 = ssef(a);
|
||||
ssef b0 = shuffle<1,2,0,3>(ssef(b));
|
||||
ssef a1 = shuffle<1,2,0,3>(ssef(a));
|
||||
ssef b1 = ssef(b);
|
||||
return Vec3fa(shuffle<1,2,0,3>(msub(a0,b0,a1*b1)));
|
||||
}
|
||||
|
||||
__forceinline float sqr_length ( const Vec3fa& a ) { return dot(a,a); }
|
||||
__forceinline float rcp_length ( const Vec3fa& a ) { return rsqrt(dot(a,a)); }
|
||||
__forceinline float rcp_length2( const Vec3fa& a ) { return rcp(dot(a,a)); }
|
||||
__forceinline float length ( const Vec3fa& a ) { return sqrt(dot(a,a)); }
|
||||
__forceinline Vec3fa normalize( const Vec3fa& a ) { return a*rsqrt(dot(a,a)); }
|
||||
__forceinline float distance ( const Vec3fa& a, const Vec3fa& b ) { return length(a-b); }
|
||||
__forceinline float halfArea ( const Vec3fa& d ) { return madd(d.x,(d.y+d.z),d.y*d.z); }
|
||||
__forceinline float area ( const Vec3fa& d ) { return 2.0f*halfArea(d); }
|
||||
|
||||
__forceinline Vec3fa normalize_safe( const Vec3fa& a ) {
|
||||
const float d = dot(a,a); if (unlikely(d == 0.0f)) return a; else return a*rsqrt(d);
|
||||
}
|
||||
|
||||
/*! differentiated normalization */
|
||||
__forceinline Vec3fa dnormalize(const Vec3fa& p, const Vec3fa& dp)
|
||||
{
|
||||
const float pp = dot(p,p);
|
||||
const float pdp = dot(p,dp);
|
||||
return (pp*dp-pdp*p)*rcp(pp)*rsqrt(pp);
|
||||
}
|
||||
__forceinline float halfArea ( const Vec3fa& d ) { return d.x*(d.y+d.z)+d.y*d.z; }
|
||||
__forceinline Vec3fa reflect (const Vec3fa& V, const Vec3fa& N) { return 2.0f*dot(V,N)*N-V; }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Select
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline Vec3fa select( bool s, const Vec3fa& t, const Vec3fa& f ) {
|
||||
__forceinline const Vec3fa select( bool s, const Vec3fa& t, const Vec3fa& f ) {
|
||||
__m128 mask = s ? _mm_castsi128_ps(_mm_cmpeq_epi32(_mm_setzero_si128(), _mm_setzero_si128())) : _mm_setzero_ps();
|
||||
return blendv_ps(f, t, mask);
|
||||
return _mm_blendv_ps(f, t, mask);
|
||||
}
|
||||
|
||||
__forceinline Vec3fa select( const Vec3ba& s, const Vec3fa& t, const Vec3fa& f ) {
|
||||
return blendv_ps(f, t, s);
|
||||
__forceinline const Vec3fa select( const Vec3ba& s, const Vec3fa& t, const Vec3fa& f ) {
|
||||
return _mm_blendv_ps(f, t, s);
|
||||
}
|
||||
|
||||
__forceinline int maxDim ( const Vec3fa& a )
|
||||
{
|
||||
const Vec3fa b = abs(a);
|
||||
if (b.x > b.y) {
|
||||
if (b.x > b.z) return 0; else return 2;
|
||||
__forceinline int maxDim ( const Vec3fa& a )
|
||||
{
|
||||
if (a.x > a.y) {
|
||||
if (a.x > a.z) return 0; else return 2;
|
||||
} else {
|
||||
if (b.y > b.z) return 1; else return 2;
|
||||
if (a.y > a.z) return 1; else return 2;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -340,13 +260,15 @@ namespace embree
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
#if defined (__SSE4_1__)
|
||||
//__forceinline Vec3fa trunc( const Vec3fa& a ) { return _mm_round_ps(a, _MM_FROUND_TO_NEAREST_INT); }
|
||||
__forceinline Vec3fa floor( const Vec3fa& a ) { return _mm_round_ps(a, _MM_FROUND_TO_NEG_INF ); }
|
||||
__forceinline Vec3fa ceil ( const Vec3fa& a ) { return _mm_round_ps(a, _MM_FROUND_TO_POS_INF ); }
|
||||
__forceinline const Vec3fa trunc( const Vec3fa& a ) { return _mm_round_ps(a, _MM_FROUND_TO_NEAREST_INT); }
|
||||
__forceinline const Vec3fa floor( const Vec3fa& a ) { return _mm_round_ps(a, _MM_FROUND_TO_NEG_INF ); }
|
||||
__forceinline const Vec3fa ceil ( const Vec3fa& a ) { return _mm_round_ps(a, _MM_FROUND_TO_POS_INF ); }
|
||||
#else
|
||||
//__forceinline Vec3fa trunc( const Vec3fa& a ) { return Vec3fa(truncf(a.x),truncf(a.y),truncf(a.z)); }
|
||||
__forceinline Vec3fa floor( const Vec3fa& a ) { return Vec3fa(floorf(a.x),floorf(a.y),floorf(a.z)); }
|
||||
__forceinline Vec3fa ceil ( const Vec3fa& a ) { return Vec3fa(ceilf (a.x),ceilf (a.y),ceilf (a.z)); }
|
||||
#if !defined(__WIN32__)
|
||||
__forceinline const Vec3fa trunc( const Vec3fa& a ) { return Vec3fa(truncf(a.x),truncf(a.y),truncf(a.z)); }
|
||||
__forceinline const Vec3fa floor( const Vec3fa& a ) { return Vec3fa(floorf(a.x),floorf(a.y),floorf(a.z)); }
|
||||
__forceinline const Vec3fa ceil ( const Vec3fa& a ) { return Vec3fa(ceilf (a.x),ceilf (a.y),ceilf (a.z)); }
|
||||
#endif
|
||||
#endif
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
@@ -356,6 +278,6 @@ namespace embree
|
||||
inline std::ostream& operator<<(std::ostream& cout, const Vec3fa& a) {
|
||||
return cout << "(" << a.x << ", " << a.y << ", " << a.z << ")";
|
||||
}
|
||||
|
||||
|
||||
typedef Vec3fa Vec3fa_t;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,258 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#pragma once
|
||||
|
||||
#include "math.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// MIC Vec3fa Type
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
struct Vec3fa_t;
|
||||
|
||||
/* 3 aligned floats as memory representation */
|
||||
struct __aligned(16) Vec3fa
|
||||
{
|
||||
typedef float Scalar;
|
||||
enum { N = 3 };
|
||||
float x,y,z; union { int a; unsigned u; float w; };
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constructors, Assignment & Cast Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline Vec3fa () {}
|
||||
__forceinline Vec3fa ( float x ) : x(x), y(x), z(x), a(0) {}
|
||||
__forceinline Vec3fa ( float x, float y, float z ) : x(x), y(y), z(z), a(0) {}
|
||||
__forceinline Vec3fa ( const Vec3f & o ) : x(o.x), y(o.y), z(o.z), a(0) {}
|
||||
__forceinline Vec3fa ( const Vec3fa& other, const int other_a ) : x(other.x), y(other.y), z(other.z), a(other_a) {}
|
||||
|
||||
__forceinline Vec3fa ( const Vec3ia_t& other );
|
||||
__forceinline Vec3fa ( const Vec3fa_t& other );
|
||||
__forceinline Vec3fa& operator=( const Vec3fa_t& other );
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constants
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline Vec3fa( ZeroTy ) : x(0.0f), y(0.0f), z(0.0f), a(0) {}
|
||||
__forceinline Vec3fa( OneTy ) : x(1.0f), y(1.0f), z(1.0f), a(0) {}
|
||||
__forceinline Vec3fa( PosInfTy ) : x(pos_inf), y(pos_inf), z(pos_inf), a(0) {}
|
||||
__forceinline Vec3fa( NegInfTy ) : x(neg_inf), y(neg_inf), z(neg_inf), a(0) {}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Array Access
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const float& operator []( const size_t index ) const { assert(index < 3); return (&x)[index]; }
|
||||
__forceinline float& operator []( const size_t index ) { assert(index < 3); return (&x)[index]; }
|
||||
};
|
||||
|
||||
/*! 3-wide vectors emulated with 16-wide vectors. */
|
||||
struct __aligned(64) Vec3fa_t
|
||||
{
|
||||
__m512 m512;
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constructors, Assignment & Cast Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline Vec3fa_t( const __m512 a ) : m512(a) {}
|
||||
__forceinline Vec3fa_t ( const Vec3fa_t& other ) { m512 = other.m512; }
|
||||
__forceinline Vec3fa_t& operator =( const Vec3fa_t& other ) { m512 = other.m512; return *this; }
|
||||
|
||||
__forceinline operator const __m512&( void ) const { return m512; }
|
||||
__forceinline operator __m512&( void ) { return m512; }
|
||||
|
||||
public:
|
||||
__forceinline explicit Vec3fa_t ( float a )
|
||||
: m512(_mm512_set1_ps(a)) {}
|
||||
|
||||
__forceinline Vec3fa_t( const Vec3fa& other ) {
|
||||
m512 = _mm512_extload_ps(&other,_MM_UPCONV_PS_NONE,_MM_BROADCAST_4X16,_MM_HINT_NONE);
|
||||
}
|
||||
};
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Assignment Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline Vec3ia::Vec3ia ( const Vec3fa_t& other ) {
|
||||
__m512i i = _mm512_cvtfxpnt_round_adjustps_epi32(other,_MM_ROUND_MODE_NEAREST,_MM_EXPADJ_NONE);
|
||||
_mm512_mask_extpackstorelo_epi32(this,0xf,i,_MM_DOWNCONV_EPI32_NONE, _MM_HINT_NONE);
|
||||
}
|
||||
|
||||
__forceinline Vec3fa::Vec3fa( const Vec3ia_t& v ) {
|
||||
__m512 f = _mm512_cvtfxpnt_round_adjustepi32_ps(v,_MM_ROUND_MODE_NEAREST,_MM_EXPADJ_NONE);
|
||||
_mm512_mask_extpackstorelo_ps(this,0xf,f,_MM_DOWNCONV_PS_NONE, _MM_HINT_NONE);
|
||||
}
|
||||
|
||||
__forceinline Vec3fa::Vec3fa ( const Vec3fa_t& other ) {
|
||||
_mm512_mask_extpackstorelo_ps(this,0xf,other.m512,_MM_DOWNCONV_PS_NONE, _MM_HINT_NONE);
|
||||
}
|
||||
|
||||
__forceinline Vec3fa& Vec3fa::operator=( const Vec3fa_t& other ) {
|
||||
_mm512_mask_extpackstorelo_ps(this,0xf,other.m512,_MM_DOWNCONV_PS_NONE, _MM_HINT_NONE);
|
||||
return *this;
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Unary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const Vec3fa_t operator +( const Vec3fa_t& a ) { return a; }
|
||||
__forceinline const Vec3fa_t operator -( const Vec3fa_t& a ) { return _mm512_castsi512_ps(_mm512_xor_epi32(_mm512_castps_si512(a.m512), _mm512_set1_epi32(0x80000000))); }
|
||||
__forceinline const Vec3fa_t abs ( const Vec3fa_t& a ) { return _mm512_gmaxabs_ps(a.m512,a.m512); }
|
||||
__forceinline const Vec3fa_t sign ( const Vec3fa_t& a ) { return _mm512_mask_blend_ps(_mm512_cmp_ps_mask(a,_mm512_setzero_ps(),_MM_CMPINT_LT), _mm512_set1_ps(1.0f), _mm512_set1_ps(-1.0f)); }
|
||||
|
||||
__forceinline const Vec3fa_t rcp ( const Vec3fa_t& a ) { return _mm512_rcp23_ps(a); }
|
||||
__forceinline const Vec3fa_t sqr ( const Vec3fa_t& a ) { return _mm512_mul_ps(a,a); }
|
||||
__forceinline const Vec3fa_t sqrt ( const Vec3fa_t& a ) { return _mm512_sqrt_ps(a.m512); }
|
||||
__forceinline const Vec3fa_t rsqrt( const Vec3fa_t& a ) { return _mm512_invsqrt_ps(a.m512); }
|
||||
__forceinline const Vec3fa_t zero_fix( const Vec3fa_t& a ) { return _mm512_mask_blend_ps(_mm512_cmp_ps_mask(a.m512,_mm512_setzero_ps(),_MM_CMPINT_EQ), a, _mm512_set1_ps(1E-10f)); }
|
||||
__forceinline const Vec3fa_t rcp_safe(const Vec3fa_t& a) { return rcp(zero_fix(a)); }
|
||||
|
||||
__forceinline Vec3fa log ( const Vec3fa& a ) {
|
||||
return Vec3fa(logf(a.x),logf(a.y),logf(a.z));
|
||||
}
|
||||
|
||||
__forceinline Vec3fa exp ( const Vec3fa& a ) {
|
||||
return Vec3fa(expf(a.x),expf(a.y),expf(a.z));
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Binary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const Vec3fa_t operator +( const Vec3fa_t& a, const Vec3fa_t& b ) { return _mm512_add_ps(a.m512, b.m512); }
|
||||
__forceinline const Vec3fa_t operator -( const Vec3fa_t& a, const Vec3fa_t& b ) { return _mm512_sub_ps(a.m512, b.m512); }
|
||||
__forceinline const Vec3fa_t operator *( const Vec3fa_t& a, const Vec3fa_t& b ) { return _mm512_mul_ps(a.m512, b.m512); }
|
||||
__forceinline const Vec3fa_t operator *( const Vec3fa_t& a, const float b ) { return a * Vec3fa_t(b); }
|
||||
__forceinline const Vec3fa_t operator *( const float a, const Vec3fa_t& b ) { return Vec3fa_t(a) * b; }
|
||||
__forceinline const Vec3fa_t operator /( const Vec3fa_t& a, const Vec3fa_t& b ) { return _mm512_div_ps(a.m512,b.m512); }
|
||||
__forceinline const Vec3fa_t operator /( const Vec3fa_t& a, const float b ) { return _mm512_div_ps(a.m512,_mm512_set1_ps(b)); }
|
||||
__forceinline const Vec3fa_t operator /( const float a, const Vec3fa_t& b ) { return _mm512_div_ps(_mm512_set1_ps(a),b.m512); }
|
||||
|
||||
__forceinline const Vec3fa_t min( const Vec3fa_t& a, const Vec3fa_t& b ) { return _mm512_gmin_ps(a.m512,b.m512); }
|
||||
__forceinline const Vec3fa_t max( const Vec3fa_t& a, const Vec3fa_t& b ) { return _mm512_gmax_ps(a.m512,b.m512); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Ternary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline Vec3fa_t madd ( const Vec3fa_t& a, const Vec3fa_t& b, const Vec3fa_t& c) { return _mm512_fmadd_ps(a,b,c); }
|
||||
__forceinline Vec3fa_t msub ( const Vec3fa_t& a, const Vec3fa_t& b, const Vec3fa_t& c) { return _mm512_fmsub_ps(a,b,c); }
|
||||
__forceinline Vec3fa_t nmadd ( const Vec3fa_t& a, const Vec3fa_t& b, const Vec3fa_t& c) { return _mm512_fnmadd_ps(a,b,c); }
|
||||
__forceinline Vec3fa_t nmsub ( const Vec3fa_t& a, const Vec3fa_t& b, const Vec3fa_t& c) { return _mm512_fnmsub_ps(a,b,c); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Assignment Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline Vec3fa& operator +=( Vec3fa& a, const Vec3fa_t& b ) { return a = a + b; }
|
||||
__forceinline Vec3fa& operator -=( Vec3fa& a, const Vec3fa_t& b ) { return a = a - b; }
|
||||
__forceinline Vec3fa& operator *=( Vec3fa& a, const float b ) { return a = a * b; }
|
||||
__forceinline Vec3fa& operator /=( Vec3fa& a, const float b ) { return a = a / b; }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Reductions
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline float reduce_add(const Vec3fa_t& a) {
|
||||
__m512 b = _mm512_mask_blend_ps(0x8,a.m512,_mm512_set1_ps(0.0f));
|
||||
__m512 c = _mm512_add_ps(b, _mm512_swizzle_ps(b, _MM_SWIZ_REG_BADC));
|
||||
__m512 d = _mm512_add_ps(c, _mm512_swizzle_ps(c, _MM_SWIZ_REG_CDAB));
|
||||
return _mm512_cvtss_f32(d);
|
||||
}
|
||||
|
||||
__forceinline float reduce_mul(const Vec3fa_t& a) {
|
||||
__m512 b = _mm512_mask_blend_ps(0x8,a.m512,_mm512_set1_ps(1.0f));
|
||||
__m512 c = _mm512_mul_ps(b, _mm512_swizzle_ps(b, _MM_SWIZ_REG_BADC));
|
||||
__m512 d = _mm512_mul_ps(c, _mm512_swizzle_ps(c, _MM_SWIZ_REG_CDAB));
|
||||
return _mm512_cvtss_f32(d);
|
||||
}
|
||||
|
||||
__forceinline float reduce_min(const Vec3fa_t& a) {
|
||||
__m512 b = _mm512_mask_blend_ps(0x8,a.m512,_mm512_set1_ps(pos_inf));
|
||||
__m512 c = _mm512_gmin_ps(b, _mm512_swizzle_ps(b, _MM_SWIZ_REG_BADC));
|
||||
__m512 d = _mm512_gmin_ps(c, _mm512_swizzle_ps(c, _MM_SWIZ_REG_CDAB));
|
||||
return _mm512_cvtss_f32(d);
|
||||
}
|
||||
|
||||
__forceinline float reduce_max(const Vec3fa_t& a) {
|
||||
__m512 b = _mm512_mask_blend_ps(0x8,a.m512,_mm512_set1_ps(neg_inf));
|
||||
__m512 c = _mm512_gmax_ps(b, _mm512_swizzle_ps(b, _MM_SWIZ_REG_BADC));
|
||||
__m512 d = _mm512_gmax_ps(c, _mm512_swizzle_ps(c, _MM_SWIZ_REG_CDAB));
|
||||
return _mm512_cvtss_f32(d);
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Comparison Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline bool operator ==( const Vec3fa_t& a, const Vec3fa_t& b ) { return (_mm512_cmp_ps_mask(a,b,_MM_CMPINT_EQ) & 7) == 7; }
|
||||
__forceinline bool operator !=( const Vec3fa_t& a, const Vec3fa_t& b ) { return (_mm512_cmp_ps_mask(a,b,_MM_CMPINT_NE) & 7) != 0; }
|
||||
|
||||
__forceinline Vec3ba eq_mask( const Vec3fa_t& a, const Vec3fa_t& b ) { return _mm512_cmp_ps_mask(a,b,_MM_CMPINT_EQ); }
|
||||
__forceinline Vec3ba neq_mask(const Vec3fa_t& a, const Vec3fa_t& b ) { return _mm512_cmp_ps_mask(a,b,_MM_CMPINT_NE); }
|
||||
__forceinline Vec3ba lt_mask( const Vec3fa_t& a, const Vec3fa_t& b ) { return _mm512_cmp_ps_mask(a,b,_MM_CMPINT_LT); }
|
||||
__forceinline Vec3ba le_mask( const Vec3fa_t& a, const Vec3fa_t& b ) { return _mm512_cmp_ps_mask(a,b,_MM_CMPINT_LE); }
|
||||
__forceinline Vec3ba gt_mask( const Vec3fa_t& a, const Vec3fa_t& b ) { return _mm512_cmp_ps_mask(a,b,_MM_CMPINT_GT); }
|
||||
__forceinline Vec3ba ge_mask( const Vec3fa_t& a, const Vec3fa_t& b ) { return _mm512_cmp_ps_mask(a,b,_MM_CMPINT_GE); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Euclidian Space Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline float dot ( const Vec3fa_t& a, const Vec3fa_t& b ) {
|
||||
return reduce_add(a*b);
|
||||
}
|
||||
|
||||
__forceinline Vec3fa_t cross ( const Vec3fa_t& a, const Vec3fa_t& b )
|
||||
{
|
||||
__m512 c = _mm512_mul_ps(b,_mm512_swizzle_ps(a,_MM_SWIZ_REG_DACB));
|
||||
c = _mm512_fmsub_ps(_mm512_swizzle_ps(b,_MM_SWIZ_REG_DACB),a,c);
|
||||
c = _mm512_swizzle_ps(c,_MM_SWIZ_REG_DACB);
|
||||
return c;
|
||||
}
|
||||
|
||||
__forceinline float length ( const Vec3fa_t& a ) { return sqrt(dot(a,a)); }
|
||||
__forceinline Vec3fa_t normalize( const Vec3fa_t& a ) { return a*rsqrt(dot(a,a)); }
|
||||
__forceinline float distance ( const Vec3fa_t& a, const Vec3fa_t& b ) { return length(a-b); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Select
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const Vec3fa_t select( bool s, const Vec3fa_t& t, const Vec3fa_t& f ) {
|
||||
return _mm512_mask_blend_ps(s ? 0xF : 0x0, f, t);
|
||||
}
|
||||
|
||||
__forceinline const Vec3fa_t select( const Vec3ba& s, const Vec3fa_t& t, const Vec3fa_t& f ) {
|
||||
return _mm512_mask_blend_ps(s, f, t);
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Output Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
inline std::ostream& operator<<(std::ostream& cout, const Vec3fa& a) {
|
||||
return cout << "(" << a.x << ", " << a.y << ", " << a.z << ", " << a.w << ")";
|
||||
}
|
||||
}
|
||||
+43
-47
@@ -1,5 +1,5 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
@@ -16,9 +16,8 @@
|
||||
|
||||
#pragma once
|
||||
|
||||
#include "../sys/alloc.h"
|
||||
#include "simd/sse.h"
|
||||
#include "math.h"
|
||||
#include "../simd/sse.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
@@ -28,8 +27,6 @@ namespace embree
|
||||
|
||||
struct __aligned(16) Vec3ia
|
||||
{
|
||||
ALIGNED_STRUCT;
|
||||
|
||||
union {
|
||||
__m128i m128;
|
||||
struct { int x,y,z; int a; };
|
||||
@@ -47,12 +44,12 @@ namespace embree
|
||||
__forceinline Vec3ia( const Vec3ia& other ) : m128(other.m128) {}
|
||||
__forceinline Vec3ia& operator =(const Vec3ia& other) { m128 = other.m128; return *this; }
|
||||
|
||||
__forceinline explicit Vec3ia( const int a ) : m128(_mm_set1_epi32(a)) {}
|
||||
__forceinline Vec3ia( const int x, const int y, const int z) : m128(_mm_set_epi32(z, z, y, x)) {}
|
||||
__forceinline Vec3ia( const int a ) : m128(_mm_set1_epi32(a)) {}
|
||||
__forceinline explicit Vec3ia( const int x, const int y, const int z) : m128(_mm_set_epi32(z, z, y, x)) {}
|
||||
__forceinline explicit Vec3ia( const __m128 a ) : m128(_mm_cvtps_epi32(a)) {}
|
||||
|
||||
__forceinline operator const __m128i&() const { return m128; }
|
||||
__forceinline operator __m128i&() { return m128; }
|
||||
__forceinline operator const __m128i&( void ) const { return m128; }
|
||||
__forceinline operator __m128i&( void ) { return m128; }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constants
|
||||
@@ -76,72 +73,71 @@ namespace embree
|
||||
/// Unary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline Vec3ia operator +( const Vec3ia& a ) { return a; }
|
||||
__forceinline Vec3ia operator -( const Vec3ia& a ) { return _mm_sub_epi32(_mm_setzero_si128(), a.m128); }
|
||||
__forceinline const Vec3ia operator +( const Vec3ia& a ) { return a; }
|
||||
__forceinline const Vec3ia operator -( const Vec3ia& a ) { return _mm_sub_epi32(_mm_setzero_si128(), a.m128); }
|
||||
#if defined(__SSSE3__)
|
||||
__forceinline Vec3ia abs ( const Vec3ia& a ) { return _mm_abs_epi32(a.m128); }
|
||||
__forceinline const Vec3ia abs ( const Vec3ia& a ) { return _mm_abs_epi32(a.m128); }
|
||||
#endif
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Binary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline Vec3ia operator +( const Vec3ia& a, const Vec3ia& b ) { return _mm_add_epi32(a.m128, b.m128); }
|
||||
__forceinline Vec3ia operator +( const Vec3ia& a, const int b ) { return a+Vec3ia(b); }
|
||||
__forceinline Vec3ia operator +( const int a, const Vec3ia& b ) { return Vec3ia(a)+b; }
|
||||
__forceinline const Vec3ia operator +( const Vec3ia& a, const Vec3ia& b ) { return _mm_add_epi32(a.m128, b.m128); }
|
||||
__forceinline const Vec3ia operator +( const Vec3ia& a, const int b ) { return a+Vec3ia(b); }
|
||||
__forceinline const Vec3ia operator +( const int a, const Vec3ia& b ) { return Vec3ia(a)+b; }
|
||||
|
||||
__forceinline Vec3ia operator -( const Vec3ia& a, const Vec3ia& b ) { return _mm_sub_epi32(a.m128, b.m128); }
|
||||
__forceinline Vec3ia operator -( const Vec3ia& a, const int b ) { return a-Vec3ia(b); }
|
||||
__forceinline Vec3ia operator -( const int a, const Vec3ia& b ) { return Vec3ia(a)-b; }
|
||||
__forceinline const Vec3ia operator -( const Vec3ia& a, const Vec3ia& b ) { return _mm_sub_epi32(a.m128, b.m128); }
|
||||
__forceinline const Vec3ia operator -( const Vec3ia& a, const int b ) { return a-Vec3ia(b); }
|
||||
__forceinline const Vec3ia operator -( const int a, const Vec3ia& b ) { return Vec3ia(a)-b; }
|
||||
|
||||
#if defined(__SSE4_1__)
|
||||
__forceinline Vec3ia operator *( const Vec3ia& a, const Vec3ia& b ) { return _mm_mullo_epi32(a.m128, b.m128); }
|
||||
__forceinline Vec3ia operator *( const Vec3ia& a, const int b ) { return a * Vec3ia(b); }
|
||||
__forceinline Vec3ia operator *( const int a, const Vec3ia& b ) { return Vec3ia(a) * b; }
|
||||
__forceinline const Vec3ia operator *( const Vec3ia& a, const Vec3ia& b ) { return _mm_mullo_epi32(a.m128, b.m128); }
|
||||
__forceinline const Vec3ia operator *( const Vec3ia& a, const int b ) { return a * Vec3ia(b); }
|
||||
__forceinline const Vec3ia operator *( const int a, const Vec3ia& b ) { return Vec3ia(a) * b; }
|
||||
#endif
|
||||
|
||||
__forceinline Vec3ia operator &( const Vec3ia& a, const Vec3ia& b ) { return _mm_and_si128(a.m128, b.m128); }
|
||||
__forceinline Vec3ia operator &( const Vec3ia& a, const int b ) { return a & Vec3ia(b); }
|
||||
__forceinline Vec3ia operator &( const int a, const Vec3ia& b ) { return Vec3ia(a) & b; }
|
||||
__forceinline const Vec3ia operator &( const Vec3ia& a, const Vec3ia& b ) { return _mm_and_si128(a.m128, b.m128); }
|
||||
__forceinline const Vec3ia operator &( const Vec3ia& a, const int b ) { return a & Vec3ia(b); }
|
||||
__forceinline const Vec3ia operator &( const int a, const Vec3ia& b ) { return Vec3ia(a) & b; }
|
||||
|
||||
__forceinline Vec3ia operator |( const Vec3ia& a, const Vec3ia& b ) { return _mm_or_si128(a.m128, b.m128); }
|
||||
__forceinline Vec3ia operator |( const Vec3ia& a, const int b ) { return a | Vec3ia(b); }
|
||||
__forceinline Vec3ia operator |( const int a, const Vec3ia& b ) { return Vec3ia(a) | b; }
|
||||
__forceinline const Vec3ia operator |( const Vec3ia& a, const Vec3ia& b ) { return _mm_or_si128(a.m128, b.m128); }
|
||||
__forceinline const Vec3ia operator |( const Vec3ia& a, const int b ) { return a | Vec3ia(b); }
|
||||
__forceinline const Vec3ia operator |( const int a, const Vec3ia& b ) { return Vec3ia(a) | b; }
|
||||
|
||||
__forceinline Vec3ia operator ^( const Vec3ia& a, const Vec3ia& b ) { return _mm_xor_si128(a.m128, b.m128); }
|
||||
__forceinline Vec3ia operator ^( const Vec3ia& a, const int b ) { return a ^ Vec3ia(b); }
|
||||
__forceinline Vec3ia operator ^( const int a, const Vec3ia& b ) { return Vec3ia(a) ^ b; }
|
||||
__forceinline const Vec3ia operator ^( const Vec3ia& a, const Vec3ia& b ) { return _mm_xor_si128(a.m128, b.m128); }
|
||||
__forceinline const Vec3ia operator ^( const Vec3ia& a, const int b ) { return a ^ Vec3ia(b); }
|
||||
__forceinline const Vec3ia operator ^( const int a, const Vec3ia& b ) { return Vec3ia(a) ^ b; }
|
||||
|
||||
__forceinline Vec3ia operator <<( const Vec3ia& a, const int n ) { return _mm_slli_epi32(a.m128, n); }
|
||||
__forceinline Vec3ia operator >>( const Vec3ia& a, const int n ) { return _mm_srai_epi32(a.m128, n); }
|
||||
__forceinline const Vec3ia operator <<( const Vec3ia& a, const int n ) { return _mm_slli_epi32(a.m128, n); }
|
||||
__forceinline const Vec3ia operator >>( const Vec3ia& a, const int n ) { return _mm_srai_epi32(a.m128, n); }
|
||||
|
||||
__forceinline Vec3ia sll ( const Vec3ia& a, const int b ) { return _mm_slli_epi32(a.m128, b); }
|
||||
__forceinline Vec3ia sra ( const Vec3ia& a, const int b ) { return _mm_srai_epi32(a.m128, b); }
|
||||
__forceinline Vec3ia srl ( const Vec3ia& a, const int b ) { return _mm_srli_epi32(a.m128, b); }
|
||||
__forceinline const Vec3ia sra ( const Vec3ia& a, const int b ) { return _mm_srai_epi32(a.m128, b); }
|
||||
__forceinline const Vec3ia srl ( const Vec3ia& a, const int b ) { return _mm_srli_epi32(a.m128, b); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Assignment Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline Vec3ia& operator +=( Vec3ia& a, const Vec3ia& b ) { return a = a + b; }
|
||||
__forceinline Vec3ia& operator +=( Vec3ia& a, const int& b ) { return a = a + b; }
|
||||
__forceinline Vec3ia& operator +=( Vec3ia& a, const int32& b ) { return a = a + b; }
|
||||
|
||||
__forceinline Vec3ia& operator -=( Vec3ia& a, const Vec3ia& b ) { return a = a - b; }
|
||||
__forceinline Vec3ia& operator -=( Vec3ia& a, const int& b ) { return a = a - b; }
|
||||
__forceinline Vec3ia& operator -=( Vec3ia& a, const int32& b ) { return a = a - b; }
|
||||
|
||||
#if defined(__SSE4_1__)
|
||||
__forceinline Vec3ia& operator *=( Vec3ia& a, const Vec3ia& b ) { return a = a * b; }
|
||||
__forceinline Vec3ia& operator *=( Vec3ia& a, const int& b ) { return a = a * b; }
|
||||
__forceinline Vec3ia& operator *=( Vec3ia& a, const int32& b ) { return a = a * b; }
|
||||
#endif
|
||||
|
||||
__forceinline Vec3ia& operator &=( Vec3ia& a, const Vec3ia& b ) { return a = a & b; }
|
||||
__forceinline Vec3ia& operator &=( Vec3ia& a, const int& b ) { return a = a & b; }
|
||||
__forceinline Vec3ia& operator &=( Vec3ia& a, const int32& b ) { return a = a & b; }
|
||||
|
||||
__forceinline Vec3ia& operator |=( Vec3ia& a, const Vec3ia& b ) { return a = a | b; }
|
||||
__forceinline Vec3ia& operator |=( Vec3ia& a, const int& b ) { return a = a | b; }
|
||||
__forceinline Vec3ia& operator |=( Vec3ia& a, const int32& b ) { return a = a | b; }
|
||||
|
||||
__forceinline Vec3ia& operator <<=( Vec3ia& a, const int& b ) { return a = a << b; }
|
||||
__forceinline Vec3ia& operator >>=( Vec3ia& a, const int& b ) { return a = a >> b; }
|
||||
__forceinline Vec3ia& operator <<=( Vec3ia& a, const int32& b ) { return a = a << b; }
|
||||
__forceinline Vec3ia& operator >>=( Vec3ia& a, const int32& b ) { return a = a >> b; }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Reductions
|
||||
@@ -173,7 +169,7 @@ namespace embree
|
||||
/// Select
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline Vec3ia select( const Vec3ba& m, const Vec3ia& t, const Vec3ia& f ) {
|
||||
__forceinline const Vec3ia select( const Vec3ba& m, const Vec3ia& t, const Vec3ia& f ) {
|
||||
#if defined(__SSE4_1__)
|
||||
return _mm_castps_si128(_mm_blendv_ps(_mm_castsi128_ps(f), _mm_castsi128_ps(t), m));
|
||||
#else
|
||||
@@ -182,11 +178,11 @@ namespace embree
|
||||
}
|
||||
|
||||
#if defined(__SSE4_1__)
|
||||
__forceinline Vec3ia min( const Vec3ia& a, const Vec3ia& b ) { return _mm_min_epi32(a.m128,b.m128); }
|
||||
__forceinline Vec3ia max( const Vec3ia& a, const Vec3ia& b ) { return _mm_max_epi32(a.m128,b.m128); }
|
||||
__forceinline const Vec3ia min( const Vec3ia& a, const Vec3ia& b ) { return _mm_min_epi32(a.m128,b.m128); }
|
||||
__forceinline const Vec3ia max( const Vec3ia& a, const Vec3ia& b ) { return _mm_max_epi32(a.m128,b.m128); }
|
||||
#else
|
||||
__forceinline Vec3ia min( const Vec3ia& a, const Vec3ia& b ) { return select(lt_mask(a,b),a,b); }
|
||||
__forceinline Vec3ia max( const Vec3ia& a, const Vec3ia& b ) { return select(gt_mask(a,b),a,b); }
|
||||
__forceinline const Vec3ia min( const Vec3ia& a, const Vec3ia& b ) { return select(lt_mask(a,b),a,b); }
|
||||
__forceinline const Vec3ia max( const Vec3ia& a, const Vec3ia& b ) { return select(gt_mask(a,b),a,b); }
|
||||
#endif
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
@@ -0,0 +1,229 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#pragma once
|
||||
|
||||
#include "math.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// MIC Vec3ia Type
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
struct Vec3ia_t;
|
||||
struct Vec3fa_t;
|
||||
|
||||
/* 3 aligned ints as memory representation */
|
||||
struct __aligned(16) Vec3ia
|
||||
{
|
||||
typedef int Scalar;
|
||||
enum { N = 3 };
|
||||
struct { int x,y,z; int a; };
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constructors, Assignment & Cast Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline Vec3ia ( ) {}
|
||||
__forceinline Vec3ia ( int x ) : x(x), y(x), z(x), a(0) {}
|
||||
__forceinline Vec3ia ( int x, int y, int z ) : x(x), y(y), z(z), a(0) {}
|
||||
|
||||
__forceinline Vec3ia ( const Vec3fa_t& other );
|
||||
__forceinline Vec3ia ( const Vec3ia_t& other );
|
||||
__forceinline Vec3ia& operator=( const Vec3ia_t& other );
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constants
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline Vec3ia( ZeroTy ) : x(0), y(0), z(0), a(0) {}
|
||||
__forceinline Vec3ia( OneTy ) : x(1), y(1), z(1), a(1) {}
|
||||
__forceinline Vec3ia( PosInfTy ) : x(pos_inf), y(pos_inf), z(pos_inf), a(pos_inf) {}
|
||||
__forceinline Vec3ia( NegInfTy ) : x(neg_inf), y(neg_inf), z(neg_inf), a(neg_inf) {}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Array Access
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const int& operator []( const size_t index ) const { assert(index < 3); return (&x)[index]; }
|
||||
__forceinline int& operator []( const size_t index ) { assert(index < 3); return (&x)[index]; }
|
||||
};
|
||||
|
||||
/*! 3-wide vectors emulated with 16-wide vectors. */
|
||||
struct __aligned(64) Vec3ia_t
|
||||
{
|
||||
__m512i m512;
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constructors, Assignment & Cast Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline Vec3ia_t ( const __m512i a ) : m512(a) {}
|
||||
__forceinline Vec3ia_t ( const Vec3ia_t& other ) { m512 = other.m512; }
|
||||
__forceinline Vec3ia_t& operator =( const Vec3ia_t& other ) { m512 = other.m512; return *this; }
|
||||
|
||||
__forceinline operator const __m512i&( void ) const { return m512; }
|
||||
__forceinline operator __m512i&( void ) { return m512; }
|
||||
|
||||
public:
|
||||
__forceinline explicit Vec3ia_t ( int a )
|
||||
: m512(_mm512_set1_epi32(a)) {}
|
||||
|
||||
__forceinline Vec3ia_t( const Vec3ia& other ) {
|
||||
m512 = _mm512_extload_epi32(&other,_MM_UPCONV_EPI32_NONE,_MM_BROADCAST_4X16,_MM_HINT_NONE);
|
||||
}
|
||||
};
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Assignment Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline Vec3ia::Vec3ia ( const Vec3ia_t& other ) {
|
||||
_mm512_mask_extpackstorelo_epi32(this,0xf,other.m512,_MM_DOWNCONV_EPI32_NONE, _MM_HINT_NONE);
|
||||
}
|
||||
|
||||
__forceinline Vec3ia& Vec3ia::operator=( const Vec3ia_t& other ) {
|
||||
_mm512_mask_extpackstorelo_epi32(this,0xf,other.m512,_MM_DOWNCONV_EPI32_NONE, _MM_HINT_NONE);
|
||||
return *this;
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Unary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const Vec3ia_t operator +( const Vec3ia_t& a ) { return a; }
|
||||
__forceinline const Vec3ia_t operator -( const Vec3ia_t& a ) { return _mm512_sub_epi32(_mm512_setzero_epi32(), a.m512); }
|
||||
//__forceinline const Vec3ia_t abs ( const Vec3ia_t& a ) { return _mm512_abs_epi32(a.m512); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Binary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const Vec3ia_t operator +( const Vec3ia_t& a, const Vec3ia_t& b ) { return _mm512_add_epi32(a.m512, b.m512); }
|
||||
__forceinline const Vec3ia_t operator +( const Vec3ia_t& a, const int b ) { return a+Vec3ia_t(b); }
|
||||
__forceinline const Vec3ia_t operator +( const int a, const Vec3ia_t& b ) { return Vec3ia_t(a)+b; }
|
||||
|
||||
__forceinline const Vec3ia_t operator -( const Vec3ia_t& a, const Vec3ia_t& b ) { return _mm512_sub_epi32(a.m512, b.m512); }
|
||||
__forceinline const Vec3ia_t operator -( const Vec3ia_t& a, const int b ) { return a-Vec3ia_t(b); }
|
||||
__forceinline const Vec3ia_t operator -( const int a, const Vec3ia_t& b ) { return Vec3ia_t(a)-b; }
|
||||
|
||||
__forceinline const Vec3ia_t operator *( const Vec3ia_t& a, const Vec3ia_t& b ) { return _mm512_mullo_epi32(a.m512, b.m512); }
|
||||
__forceinline const Vec3ia_t operator *( const Vec3ia_t& a, const int b ) { return a * Vec3ia_t(b); }
|
||||
__forceinline const Vec3ia_t operator *( const int a, const Vec3ia_t& b ) { return Vec3ia_t(a) * b; }
|
||||
|
||||
__forceinline const Vec3ia_t operator &( const Vec3ia_t& a, const Vec3ia_t& b ) { return _mm512_and_epi32(a.m512, b.m512); }
|
||||
__forceinline const Vec3ia_t operator &( const Vec3ia_t& a, const int b ) { return a & Vec3ia_t(b); }
|
||||
__forceinline const Vec3ia_t operator &( const int a, const Vec3ia_t& b ) { return Vec3ia_t(a) & b; }
|
||||
|
||||
__forceinline const Vec3ia_t operator |( const Vec3ia_t& a, const Vec3ia_t& b ) { return _mm512_or_epi32(a.m512, b.m512); }
|
||||
__forceinline const Vec3ia_t operator |( const Vec3ia_t& a, const int b ) { return a | Vec3ia_t(b); }
|
||||
__forceinline const Vec3ia_t operator |( const int a, const Vec3ia_t& b ) { return Vec3ia_t(a) | b; }
|
||||
|
||||
__forceinline const Vec3ia_t operator ^( const Vec3ia_t& a, const Vec3ia_t& b ) { return _mm512_xor_epi32(a.m512, b.m512); }
|
||||
__forceinline const Vec3ia_t operator ^( const Vec3ia_t& a, const int b ) { return a ^ Vec3ia_t(b); }
|
||||
__forceinline const Vec3ia_t operator ^( const int a, const Vec3ia_t& b ) { return Vec3ia_t(a) ^ b; }
|
||||
|
||||
__forceinline const Vec3ia_t operator <<( const Vec3ia_t& a, const int n ) { return _mm512_slli_epi32(a.m512, n); }
|
||||
__forceinline const Vec3ia_t operator >>( const Vec3ia_t& a, const int n ) { return _mm512_srai_epi32(a.m512, n); }
|
||||
|
||||
__forceinline const Vec3ia_t sra ( const Vec3ia_t& a, const int b ) { return _mm512_srai_epi32(a.m512, b); }
|
||||
__forceinline const Vec3ia_t srl ( const Vec3ia_t& a, const int b ) { return _mm512_srli_epi32(a.m512, b); }
|
||||
|
||||
__forceinline const Vec3ia_t min( const Vec3ia_t& a, const Vec3ia_t& b ) { return _mm512_min_epi32(a.m512,b.m512); }
|
||||
__forceinline const Vec3ia_t max( const Vec3ia_t& a, const Vec3ia_t& b ) { return _mm512_max_epi32(a.m512,b.m512); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Assignment Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline Vec3ia& operator +=( Vec3ia& a, const Vec3ia_t& b ) { return a = a + b; }
|
||||
__forceinline Vec3ia& operator +=( Vec3ia& a, const int32& b ) { return a = a + b; }
|
||||
|
||||
__forceinline Vec3ia& operator -=( Vec3ia& a, const Vec3ia_t& b ) { return a = a - b; }
|
||||
__forceinline Vec3ia& operator -=( Vec3ia& a, const int32& b ) { return a = a - b; }
|
||||
|
||||
__forceinline Vec3ia& operator *=( Vec3ia& a, const Vec3ia_t& b ) { return a = a * b; }
|
||||
__forceinline Vec3ia& operator *=( Vec3ia& a, const int32& b ) { return a = a * b; }
|
||||
|
||||
__forceinline Vec3ia& operator &=( Vec3ia& a, const Vec3ia_t& b ) { return a = a & b; }
|
||||
__forceinline Vec3ia& operator &=( Vec3ia& a, const int32& b ) { return a = a & b; }
|
||||
|
||||
__forceinline Vec3ia& operator |=( Vec3ia& a, const Vec3ia_t& b ) { return a = a | b; }
|
||||
__forceinline Vec3ia& operator |=( Vec3ia& a, const int32& b ) { return a = a | b; }
|
||||
|
||||
__forceinline Vec3ia& operator <<=( Vec3ia& a, const int32& b ) { return a = a << b; }
|
||||
__forceinline Vec3ia& operator >>=( Vec3ia& a, const int32& b ) { return a = a >> b; }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Reductions
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline int reduce_add(const Vec3ia_t& a) {
|
||||
__m512i b = _mm512_mask_blend_epi32(0x8,a.m512,_mm512_set1_epi32(0));
|
||||
__m512i c = _mm512_add_epi32(b, _mm512_swizzle_epi32(b, _MM_SWIZ_REG_BADC));
|
||||
__m512i d = _mm512_add_epi32(c, _mm512_swizzle_epi32(c, _MM_SWIZ_REG_CDAB));
|
||||
return *(int*)&d;
|
||||
}
|
||||
|
||||
__forceinline int reduce_mul(const Vec3ia_t& a) {
|
||||
__m512i b = _mm512_mask_blend_epi32(0x8,a.m512,_mm512_set1_epi32(1));
|
||||
__m512i c = _mm512_mullo_epi32(b, _mm512_swizzle_epi32(b, _MM_SWIZ_REG_BADC));
|
||||
__m512i d = _mm512_mullo_epi32(c, _mm512_swizzle_epi32(c, _MM_SWIZ_REG_CDAB));
|
||||
return *(int*)&d;
|
||||
}
|
||||
|
||||
__forceinline int reduce_min(const Vec3ia_t& a) {
|
||||
__m512i b = _mm512_mask_blend_epi32(0x8,a.m512,_mm512_set1_epi32(pos_inf));
|
||||
__m512i c = _mm512_min_epi32(b, _mm512_swizzle_epi32(b, _MM_SWIZ_REG_BADC));
|
||||
__m512i d = _mm512_min_epi32(c, _mm512_swizzle_epi32(c, _MM_SWIZ_REG_CDAB));
|
||||
return *(int*)&d;
|
||||
}
|
||||
|
||||
__forceinline int reduce_max(const Vec3ia_t& a) {
|
||||
__m512i b = _mm512_mask_blend_epi32(0x8,a.m512,_mm512_set1_epi32(neg_inf));
|
||||
__m512i c = _mm512_max_epi32(b, _mm512_swizzle_epi32(b, _MM_SWIZ_REG_BADC));
|
||||
__m512i d = _mm512_max_epi32(c, _mm512_swizzle_epi32(c, _MM_SWIZ_REG_CDAB));
|
||||
return *(int*)&d;
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Comparison Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline bool operator ==( const Vec3ia_t& a, const Vec3ia_t& b ) { return (_mm512_cmp_epi32_mask(a,b,_MM_CMPINT_EQ) & 7) == 7; }
|
||||
__forceinline bool operator !=( const Vec3ia_t& a, const Vec3ia_t& b ) { return (_mm512_cmp_epi32_mask(a,b,_MM_CMPINT_NE) & 7) != 0; }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Select
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const Vec3ia_t select( bool s, const Vec3ia_t& t, const Vec3ia_t& f ) {
|
||||
return _mm512_mask_blend_epi32(s ? 0xF : 0x0, f, t);
|
||||
}
|
||||
|
||||
__forceinline const Vec3ia_t select( const Vec3ba& s, const Vec3ia_t& t, const Vec3ia_t& f ) {
|
||||
return _mm512_mask_blend_epi32(s, f, t);
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Output Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
inline std::ostream& operator<<(std::ostream& cout, const Vec3ia& a) {
|
||||
return cout << "(" << a.x << ", " << a.y << ", " << a.z << ")";
|
||||
}
|
||||
}
|
||||
+35
-96
@@ -1,5 +1,5 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
@@ -16,8 +16,7 @@
|
||||
|
||||
#pragma once
|
||||
|
||||
#include "math.h"
|
||||
#include "vec3.h"
|
||||
#include "sys/platform.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
@@ -36,18 +35,16 @@ namespace embree
|
||||
/// Construction
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline Vec4( ) {}
|
||||
__forceinline explicit Vec4( const T& a ) : x(a), y(a), z(a), w(a) {}
|
||||
__forceinline Vec4( const T& x, const T& y, const T& z, const T& w ) : x(x), y(y), z(z), w(w) {}
|
||||
__forceinline Vec4( const Vec3<T>& xyz, const T& w ) : x(xyz.x), y(xyz.y), z(xyz.z), w(w) {}
|
||||
|
||||
__forceinline Vec4( const Vec4& other ) { x = other.x; y = other.y; z = other.z; w = other.w; }
|
||||
__forceinline Vec4( const Vec3fa& other );
|
||||
__forceinline Vec4 ( ) { }
|
||||
__forceinline Vec4 ( const Vec4& other ) { x = other.x; y = other.y; z = other.z; w = other.w; }
|
||||
__forceinline Vec4 ( const Vec3fa& other );
|
||||
|
||||
template<typename T1> __forceinline Vec4( const Vec4<T1>& a ) : x(T(a.x)), y(T(a.y)), z(T(a.z)), w(T(a.w)) {}
|
||||
template<typename T1> __forceinline Vec4& operator =(const Vec4<T1>& other) { x = other.x; y = other.y; z = other.z; w = other.w; return *this; }
|
||||
|
||||
__forceinline operator Vec3<T> () const { return Vec3<T>(x,y,z); }
|
||||
__forceinline explicit Vec4( const T& a ) : x(a), y(a), z(a), w(a) {}
|
||||
__forceinline explicit Vec4( const T& x, const T& y, const T& z, const T& w ) : x(x), y(y), z(z), w(w) {}
|
||||
__forceinline explicit Vec4( const T* const a, const size_t stride = 1 ) : x(a[0]), y(a[stride]), z(a[2*stride]), w(a[3*stride]) {}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constants
|
||||
@@ -60,12 +57,6 @@ namespace embree
|
||||
|
||||
__forceinline const T& operator []( const size_t axis ) const { assert(axis < 4); return (&x)[axis]; }
|
||||
__forceinline T& operator []( const size_t axis ) { assert(axis < 4); return (&x)[axis]; }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Swizzles
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline Vec3<T> xyz() const { return Vec3<T>(x, y, z); }
|
||||
};
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
@@ -95,20 +86,6 @@ namespace embree
|
||||
template<typename T> __forceinline Vec4<T> min(const Vec4<T>& a, const Vec4<T>& b) { return Vec4<T>(min(a.x, b.x), min(a.y, b.y), min(a.z, b.z), min(a.w, b.w)); }
|
||||
template<typename T> __forceinline Vec4<T> max(const Vec4<T>& a, const Vec4<T>& b) { return Vec4<T>(max(a.x, b.x), max(a.y, b.y), max(a.z, b.z), max(a.w, b.w)); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Ternary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
template<typename T> __forceinline Vec4<T> madd ( const Vec4<T>& a, const Vec4<T>& b, const Vec4<T>& c) { return Vec4<T>( madd(a.x,b.x,c.x), madd(a.y,b.y,c.y), madd(a.z,b.z,c.z), madd(a.w,b.w,c.w)); }
|
||||
template<typename T> __forceinline Vec4<T> msub ( const Vec4<T>& a, const Vec4<T>& b, const Vec4<T>& c) { return Vec4<T>( msub(a.x,b.x,c.x), msub(a.y,b.y,c.y), msub(a.z,b.z,c.z), msub(a.w,b.w,c.w)); }
|
||||
template<typename T> __forceinline Vec4<T> nmadd ( const Vec4<T>& a, const Vec4<T>& b, const Vec4<T>& c) { return Vec4<T>(nmadd(a.x,b.x,c.x),nmadd(a.y,b.y,c.y),nmadd(a.z,b.z,c.z),nmadd(a.w,b.w,c.w)); }
|
||||
template<typename T> __forceinline Vec4<T> nmsub ( const Vec4<T>& a, const Vec4<T>& b, const Vec4<T>& c) { return Vec4<T>(nmsub(a.x,b.x,c.x),nmsub(a.y,b.y,c.y),nmsub(a.z,b.z,c.z),nmsub(a.w,b.w,c.w)); }
|
||||
|
||||
template<typename T> __forceinline Vec4<T> madd ( const T& a, const Vec4<T>& b, const Vec4<T>& c) { return Vec4<T>( madd(a,b.x,c.x), madd(a,b.y,c.y), madd(a,b.z,c.z), madd(a,b.w,c.w)); }
|
||||
template<typename T> __forceinline Vec4<T> msub ( const T& a, const Vec4<T>& b, const Vec4<T>& c) { return Vec4<T>( msub(a,b.x,c.x), msub(a,b.y,c.y), msub(a,b.z,c.z), msub(a,b.w,c.w)); }
|
||||
template<typename T> __forceinline Vec4<T> nmadd ( const T& a, const Vec4<T>& b, const Vec4<T>& c) { return Vec4<T>(nmadd(a,b.x,c.x),nmadd(a,b.y,c.y),nmadd(a,b.z,c.z),nmadd(a,b.w,c.w)); }
|
||||
template<typename T> __forceinline Vec4<T> nmsub ( const T& a, const Vec4<T>& b, const Vec4<T>& c) { return Vec4<T>(nmsub(a,b.x,c.x),nmsub(a,b.y,c.y),nmsub(a,b.z,c.z),nmsub(a,b.w,c.w)); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Assignment Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
@@ -141,20 +118,11 @@ namespace embree
|
||||
return false;
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Shift Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
template<typename T> __forceinline Vec4<T> shift_right_1( const Vec4<T>& a ) {
|
||||
return Vec4<T>(shift_right_1(a.x),shift_right_1(a.y),shift_right_1(a.z),shift_right_1(a.w));
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Euclidian Space Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
template<typename T> __forceinline T dot ( const Vec4<T>& a, const Vec4<T>& b ) { return madd(a.x,b.x,madd(a.y,b.y,madd(a.z,b.z,a.w*b.w))); }
|
||||
|
||||
template<typename T> __forceinline T dot ( const Vec4<T>& a, const Vec4<T>& b ) { return a.x*b.x + a.y*b.y + a.z*b.z + a.w*b.w; }
|
||||
template<typename T> __forceinline T length ( const Vec4<T>& a ) { return sqrt(dot(a,a)); }
|
||||
template<typename T> __forceinline Vec4<T> normalize( const Vec4<T>& a ) { return a*rsqrt(dot(a,a)); }
|
||||
template<typename T> __forceinline T distance ( const Vec4<T>& a, const Vec4<T>& b ) { return length(a-b); }
|
||||
@@ -167,19 +135,6 @@ namespace embree
|
||||
return Vec4<T>(select(s,t.x,f.x),select(s,t.y,f.y),select(s,t.z,f.z),select(s,t.w,f.w));
|
||||
}
|
||||
|
||||
template<typename T> __forceinline Vec4<T> select ( const Vec4<bool>& s, const Vec4<T>& t, const Vec4<T>& f ) {
|
||||
return Vec4<T>(select(s.x,t.x,f.x),select(s.y,t.y,f.y),select(s.z,t.z,f.z),select(s.w,t.w,f.w));
|
||||
}
|
||||
|
||||
template<typename T> __forceinline Vec4<T> select ( const typename T::Bool& s, const Vec4<T>& t, const Vec4<T>& f ) {
|
||||
return Vec4<T>(select(s,t.x,f.x),select(s,t.y,f.y),select(s,t.z,f.z),select(s,t.w,f.w));
|
||||
}
|
||||
|
||||
template<typename T>
|
||||
__forceinline Vec4<T> lerp(const Vec4<T>& v0, const Vec4<T>& v1, const T& t) {
|
||||
return madd(Vec4<T>(T(1.0f)-t),v0,t*v1);
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Output Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
@@ -198,62 +153,46 @@ namespace embree
|
||||
typedef Vec4<float > Vec4f;
|
||||
}
|
||||
|
||||
#include "vec3ba.h"
|
||||
#include "vec3ia.h"
|
||||
#include "vec3fa.h"
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// SSE / AVX / MIC specializations
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
#if defined __SSE__
|
||||
#include "../simd/sse.h"
|
||||
#if defined(__MIC__)
|
||||
#include "vec3ba_mic.h"
|
||||
#include "vec3ia_mic.h"
|
||||
#include "vec3fa_mic.h"
|
||||
#else
|
||||
#include "vec3ba.h"
|
||||
#include "vec3ia.h"
|
||||
#include "vec3fa.h"
|
||||
#endif
|
||||
|
||||
#if defined __AVX__
|
||||
#include "../simd/avx.h"
|
||||
#endif
|
||||
|
||||
#if defined __AVX512F__
|
||||
#include "../simd/avx512.h"
|
||||
#endif
|
||||
|
||||
namespace embree
|
||||
{
|
||||
namespace embree
|
||||
{
|
||||
template<> __forceinline Vec4<float>::Vec4( const Vec3fa& a ) { x = a.x; y = a.y; z = a.z; w = a.w; }
|
||||
|
||||
#if defined(__AVX__)
|
||||
template<> __forceinline Vec4<vfloat4>::Vec4( const Vec3fa& a ) {
|
||||
x = a.x; y = a.y; z = a.z; w = a.w;
|
||||
#if defined (__SSE__)
|
||||
template<> __forceinline Vec4<ssef>::Vec4( const Vec3fa& a ) {
|
||||
const ssef v = ssef(a); x = shuffle<0,0,0,0>(v); y = shuffle<1,1,1,1>(v); z = shuffle<2,2,2,2>(v); w = shuffle<3,3,3,3>(v);
|
||||
}
|
||||
#elif defined(__SSE__)
|
||||
template<> __forceinline Vec4<vfloat4>::Vec4( const Vec3fa& a ) {
|
||||
const vfloat4 v = vfloat4(a); x = shuffle<0,0,0,0>(v); y = shuffle<1,1,1,1>(v); z = shuffle<2,2,2,2>(v); w = shuffle<3,3,3,3>(v);
|
||||
}
|
||||
#endif
|
||||
|
||||
#if defined(__SSE__)
|
||||
__forceinline Vec4<vfloat4> broadcast4f( const Vec4<vfloat4>& a, const size_t k ) {
|
||||
return Vec4<vfloat4>(vfloat4::broadcast(&a.x[k]), vfloat4::broadcast(&a.y[k]), vfloat4::broadcast(&a.z[k]), vfloat4::broadcast(&a.w[k]));
|
||||
__forceinline Vec4<ssef> broadcast4f( const Vec4<ssef>& a, const size_t k ) {
|
||||
return Vec4<ssef>(ssef::broadcast(&a.x[k]), ssef::broadcast(&a.y[k]), ssef::broadcast(&a.z[k]), ssef::broadcast(&a.w[k]));
|
||||
}
|
||||
#endif
|
||||
|
||||
#if defined(__AVX__)
|
||||
template<> __forceinline Vec4<vfloat8>::Vec4( const Vec3fa& a ) {
|
||||
x = a.x; y = a.y; z = a.z; w = a.w;
|
||||
template<> __forceinline Vec4<avxf>::Vec4( const Vec3fa& a ) {
|
||||
x = a.x; y = a.y; z = a.z; w = a.w;
|
||||
}
|
||||
__forceinline Vec4<vfloat4> broadcast4f( const Vec4<vfloat8>& a, const size_t k ) {
|
||||
return Vec4<vfloat4>(vfloat4::broadcast(&a.x[k]), vfloat4::broadcast(&a.y[k]), vfloat4::broadcast(&a.z[k]), vfloat4::broadcast(&a.w[k]));
|
||||
__forceinline Vec4<ssef> broadcast4f( const Vec4<avxf>& a, const size_t k ) {
|
||||
return Vec4<ssef>(ssef::broadcast(&a.x[k]), ssef::broadcast(&a.y[k]), ssef::broadcast(&a.z[k]), ssef::broadcast(&a.w[k]));
|
||||
}
|
||||
__forceinline Vec4<vfloat8> broadcast8f( const Vec4<vfloat4>& a, const size_t k ) {
|
||||
return Vec4<vfloat8>(vfloat8::broadcast(&a.x[k]), vfloat8::broadcast(&a.y[k]), vfloat8::broadcast(&a.z[k]), vfloat8::broadcast(&a.w[k]));
|
||||
__forceinline Vec4<avxf> broadcast8f( const Vec4<ssef>& a, const size_t k ) {
|
||||
return Vec4<avxf>(avxf::broadcast(&a.x[k]), avxf::broadcast(&a.y[k]), avxf::broadcast(&a.z[k]), avxf::broadcast(&a.w[k]));
|
||||
}
|
||||
__forceinline Vec4<vfloat8> broadcast8f( const Vec4<vfloat8>& a, const size_t k ) {
|
||||
return Vec4<vfloat8>(vfloat8::broadcast(&a.x[k]), vfloat8::broadcast(&a.y[k]), vfloat8::broadcast(&a.z[k]), vfloat8::broadcast(&a.w[k]));
|
||||
__forceinline Vec4<avxf> broadcast8f( const Vec4<avxf>& a, const size_t k ) {
|
||||
return Vec4<avxf>(avxf::broadcast(&a.x[k]), avxf::broadcast(&a.y[k]), avxf::broadcast(&a.z[k]), avxf::broadcast(&a.w[k]));
|
||||
}
|
||||
#endif
|
||||
|
||||
#if defined(__AVX512F__)
|
||||
template<> __forceinline Vec4<vfloat16>::Vec4( const Vec3fa& a ) : x(a.x), y(a.y), z(a.z), w(a.w) {}
|
||||
#if defined(__MIC__)
|
||||
template<> __forceinline Vec4<ssef>::Vec4( const Vec3fa& a ) : x(a.x), y(a.y), z(a.z), w(a.w) {}
|
||||
template<> __forceinline Vec4<mic_f>::Vec4( const Vec3fa& a ) : x(a.x), y(a.y), z(a.z), w(a.w) {}
|
||||
#endif
|
||||
}
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
## ======================================================================== ##
|
||||
## Copyright 2009-2017 Intel Corporation ##
|
||||
## Copyright 2009-2014 Intel Corporation ##
|
||||
## ##
|
||||
## Licensed under the Apache License, Version 2.0 (the "License"); ##
|
||||
## you may not use this file except in compliance with the License. ##
|
||||
@@ -14,6 +14,10 @@
|
||||
## limitations under the License. ##
|
||||
## ======================================================================== ##
|
||||
|
||||
ADD_LIBRARY(simd STATIC sse.cpp)
|
||||
SET_PROPERTY(TARGET simd PROPERTY FOLDER common)
|
||||
IF (__XEON__)
|
||||
ADD_LIBRARY(simd STATIC sse.cpp)
|
||||
ELSE ()
|
||||
INCLUDE(icc_xeonphi)
|
||||
ADD_LIBRARY(simd_xeonphi STATIC mic_i.cpp mic_f.cpp mic_m.cpp)
|
||||
ENDIF ()
|
||||
|
||||
|
||||
+14
-25
@@ -1,5 +1,5 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
@@ -16,30 +16,19 @@
|
||||
|
||||
#pragma once
|
||||
|
||||
#include "sse.h"
|
||||
#include "simd/sse.h"
|
||||
|
||||
#if defined(__AVX512VL__)
|
||||
#include "vboolf8_avx512.h"
|
||||
#include "vboold4_avx512.h"
|
||||
namespace embree
|
||||
{
|
||||
struct avxb;
|
||||
struct avxi;
|
||||
struct avxf;
|
||||
}
|
||||
|
||||
#include "simd/avxb.h"
|
||||
#if defined (__AVX2__)
|
||||
#include "simd/avxi.h"
|
||||
#else
|
||||
#include "vboolf8_avx.h"
|
||||
#include "vboold4_avx.h"
|
||||
#include "simd/avxi_emu.h"
|
||||
#endif
|
||||
|
||||
#if defined(__AVX2__)
|
||||
#include "vint8_avx2.h"
|
||||
#if defined(__X86_64__)
|
||||
#include "vllong4_avx2.h"
|
||||
#endif
|
||||
#else
|
||||
#include "vint8_avx.h"
|
||||
#endif
|
||||
#include "vfloat8_avx.h"
|
||||
#if defined(__X86_64__)
|
||||
#include "vdouble4_avx.h"
|
||||
#endif
|
||||
|
||||
#if defined(__AVX512F__)
|
||||
#include "avx512.h"
|
||||
#endif
|
||||
|
||||
#include "simd/avxf.h"
|
||||
|
||||
@@ -1,53 +0,0 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#pragma once
|
||||
|
||||
#include "../sys/platform.h"
|
||||
#include "../sys/intrinsics.h"
|
||||
#include "../math/constants.h"
|
||||
#include "varying.h"
|
||||
|
||||
#include "vboolf16_avx512.h"
|
||||
#include "vint16_avx512.h"
|
||||
#include "vfloat16_avx512.h"
|
||||
|
||||
#include "vuint16_avx512.h"
|
||||
#include "vboold8_avx512.h"
|
||||
#include "vllong8_avx512.h"
|
||||
#include "vdouble8_avx512.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Prefetching
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
#define PFHINT_L1 0
|
||||
#define PFHINT_L2 1
|
||||
#define PFHINT_NT 2
|
||||
|
||||
template<const unsigned int mode>
|
||||
__forceinline void prefetch(const void * __restrict__ const m)
|
||||
{
|
||||
if (mode == PFHINT_L1)
|
||||
_mm_prefetch((const char*)m,_MM_HINT_T0);
|
||||
else if (mode == PFHINT_L2)
|
||||
_mm_prefetch((const char*)m,_MM_HINT_T1);
|
||||
else if (mode == PFHINT_NT)
|
||||
_mm_prefetch((const char*)m,_MM_HINT_NTA);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,172 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#pragma once
|
||||
|
||||
namespace embree
|
||||
{
|
||||
/*! 8-wide AVX bool type. */
|
||||
struct avxb
|
||||
{
|
||||
typedef avxb Mask; // mask type for us
|
||||
enum { size = 8 }; // number of SIMD elements
|
||||
union { // data
|
||||
__m256 m256;
|
||||
struct { __m128 l,h; };
|
||||
int32 v[8];
|
||||
};
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constructors, Assignment & Cast Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline avxb () {}
|
||||
__forceinline avxb ( const avxb& a ) { m256 = a.m256; }
|
||||
__forceinline avxb& operator=( const avxb& a ) { m256 = a.m256; return *this; }
|
||||
|
||||
__forceinline avxb( const __m256 a ) : m256(a) {}
|
||||
__forceinline operator const __m256&( void ) const { return m256; }
|
||||
__forceinline operator const __m256i( void ) const { return _mm256_castps_si256(m256); }
|
||||
__forceinline operator const __m256d( void ) const { return _mm256_castps_pd(m256); }
|
||||
|
||||
__forceinline avxb ( const int a )
|
||||
{
|
||||
assert(a >= 0 && a <= 255);
|
||||
#if defined (__AVX2__)
|
||||
const __m256i mask = _mm256_set_epi32(0x80, 0x40, 0x20, 0x10, 0x8, 0x4, 0x2, 0x1);
|
||||
const __m256i b = _mm256_set1_epi32(a);
|
||||
const __m256i c = _mm256_and_si256(b,mask);
|
||||
m256 = _mm256_castsi256_ps(_mm256_cmpeq_epi32(c,mask));
|
||||
#else
|
||||
l = _mm_lookupmask_ps[a & 0xF];
|
||||
h = _mm_lookupmask_ps[a >> 4];
|
||||
#endif
|
||||
}
|
||||
|
||||
__forceinline avxb ( const sseb& a ) : m256(_mm256_insertf128_ps(_mm256_castps128_ps256(a),a,1)) {}
|
||||
__forceinline avxb ( const sseb& a, const sseb& b) : m256(_mm256_insertf128_ps(_mm256_castps128_ps256(a),b,1)) {}
|
||||
__forceinline avxb ( const __m128 a, const __m128 b) : l(a), h(b) {}
|
||||
|
||||
__forceinline avxb ( bool a ) : m256(avxb(sseb(a), sseb(a))) {}
|
||||
__forceinline avxb ( bool a, bool b) : m256(avxb(sseb(a), sseb(b))) {}
|
||||
__forceinline avxb ( bool a, bool b, bool c, bool d) : m256(avxb(sseb(a,b), sseb(c,d))) {}
|
||||
__forceinline avxb ( bool a, bool b, bool c, bool d, bool e, bool f, bool g, bool h ) : m256(avxb(sseb(a,b,c,d), sseb(e,f,g,h))) {}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constants
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline avxb( FalseTy ) : m256(_mm256_setzero_ps()) {}
|
||||
__forceinline avxb( TrueTy ) : m256(_mm256_cmp_ps(_mm256_setzero_ps(), _mm256_setzero_ps(), _CMP_EQ_OQ)) {}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Array Access
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline bool operator []( const size_t i ) const { assert(i < 8); return (_mm256_movemask_ps(m256) >> i) & 1; }
|
||||
__forceinline int32& operator []( const size_t i ) { assert(i < 8); return v[i]; }
|
||||
};
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Unary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const avxb operator !( const avxb& a ) { return _mm256_xor_ps(a, avxb(True)); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Binary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const avxb operator &( const avxb& a, const avxb& b ) { return _mm256_and_ps(a, b); }
|
||||
__forceinline const avxb operator |( const avxb& a, const avxb& b ) { return _mm256_or_ps (a, b); }
|
||||
__forceinline const avxb operator ^( const avxb& a, const avxb& b ) { return _mm256_xor_ps(a, b); }
|
||||
|
||||
__forceinline avxb operator &=( avxb& a, const avxb& b ) { return a = a & b; }
|
||||
__forceinline avxb operator |=( avxb& a, const avxb& b ) { return a = a | b; }
|
||||
__forceinline avxb operator ^=( avxb& a, const avxb& b ) { return a = a ^ b; }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Comparison Operators + Select
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const avxb operator !=( const avxb& a, const avxb& b ) { return _mm256_xor_ps(a, b); }
|
||||
__forceinline const avxb operator ==( const avxb& a, const avxb& b ) { return _mm256_xor_ps(_mm256_xor_ps(a,b),avxb(True)); }
|
||||
|
||||
__forceinline const avxb select( const avxb& mask, const avxb& t, const avxb& f ) {
|
||||
return _mm256_blendv_ps(f, t, mask);
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Movement/Shifting/Shuffling Functions
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline avxb unpacklo( const avxb& a, const avxb& b ) { return _mm256_unpacklo_ps(a.m256, b.m256); }
|
||||
__forceinline avxb unpackhi( const avxb& a, const avxb& b ) { return _mm256_unpackhi_ps(a.m256, b.m256); }
|
||||
|
||||
template<size_t i> __forceinline const avxb shuffle( const avxb& a ) {
|
||||
return _mm256_permute_ps(a, _MM_SHUFFLE(i, i, i, i));
|
||||
}
|
||||
|
||||
template<size_t i0, size_t i1> __forceinline const avxb shuffle( const avxb& a ) {
|
||||
return _mm256_permute2f128_ps(a, a, (i1 << 4) | (i0 << 0));
|
||||
}
|
||||
|
||||
template<size_t i0, size_t i1> __forceinline const avxb shuffle( const avxb& a, const avxb& b) {
|
||||
return _mm256_permute2f128_ps(a, b, (i1 << 4) | (i0 << 0));
|
||||
}
|
||||
|
||||
template<size_t i0, size_t i1, size_t i2, size_t i3> __forceinline const avxb shuffle( const avxb& a ) {
|
||||
return _mm256_permute_ps(a, _MM_SHUFFLE(i3, i2, i1, i0));
|
||||
}
|
||||
|
||||
template<size_t i0, size_t i1, size_t i2, size_t i3> __forceinline const avxb shuffle( const avxb& a, const avxb& b ) {
|
||||
return _mm256_shuffle_ps(a, b, _MM_SHUFFLE(i3, i2, i1, i0));
|
||||
}
|
||||
|
||||
template<> __forceinline const avxb shuffle<0, 0, 2, 2>( const avxb& b ) { return _mm256_moveldup_ps(b); }
|
||||
template<> __forceinline const avxb shuffle<1, 1, 3, 3>( const avxb& b ) { return _mm256_movehdup_ps(b); }
|
||||
template<> __forceinline const avxb shuffle<0, 1, 0, 1>( const avxb& b ) { return _mm256_castpd_ps(_mm256_movedup_pd(_mm256_castps_pd(b))); }
|
||||
|
||||
template<size_t i> __forceinline const avxb insert (const avxb& a, const sseb& b) { return _mm256_insertf128_ps (a,b,i); }
|
||||
template<size_t i> __forceinline const sseb extract(const avxb& a ) { return _mm256_extractf128_ps(a ,i); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Reduction Operations
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline bool reduce_and( const avxb& a ) { return _mm256_movemask_ps(a) == (unsigned int)0xff; }
|
||||
__forceinline bool reduce_or ( const avxb& a ) { return !_mm256_testz_ps(a,a); }
|
||||
|
||||
__forceinline bool all ( const avxb& a ) { return _mm256_movemask_ps(a) == (unsigned int)0xff; }
|
||||
__forceinline bool any ( const avxb& a ) { return !_mm256_testz_ps(a,a); }
|
||||
__forceinline bool none ( const avxb& a ) { return _mm256_testz_ps(a,a) != 0; }
|
||||
|
||||
__forceinline bool all ( const avxb& valid, const avxb& b ) { return all(!valid | b); }
|
||||
__forceinline bool any ( const avxb& valid, const avxb& b ) { return any( valid & b); }
|
||||
__forceinline bool none ( const avxb& valid, const avxb& b ) { return none(valid & b); }
|
||||
|
||||
__forceinline unsigned int movemask( const avxb& a ) { return _mm256_movemask_ps(a); }
|
||||
__forceinline size_t popcnt ( const avxb& a ) { return __popcnt(_mm256_movemask_ps(a)); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Output Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
inline std::ostream& operator<<(std::ostream& cout, const avxb& a) {
|
||||
return cout << "<" << a[0] << ", " << a[1] << ", " << a[2] << ", " << a[3] << ", "
|
||||
<< a[4] << ", " << a[5] << ", " << a[6] << ", " << a[7] << ">";
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,424 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#pragma once
|
||||
|
||||
namespace embree
|
||||
{
|
||||
/*! 8-wide AVX float type. */
|
||||
struct avxf
|
||||
{
|
||||
typedef avxb Mask; // mask type for us
|
||||
typedef avxi Int ; // int type for us
|
||||
enum { size = 8 }; // number of SIMD elements
|
||||
union { __m256 m256; float v[8]; }; // data
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constructors, Assignment & Cast Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline avxf ( ) {}
|
||||
__forceinline avxf ( const avxf& other ) { m256 = other.m256; }
|
||||
__forceinline avxf& operator=( const avxf& other ) { m256 = other.m256; return *this; }
|
||||
|
||||
__forceinline avxf( const __m256 a ) : m256(a) {}
|
||||
__forceinline operator const __m256&( void ) const { return m256; }
|
||||
__forceinline operator __m256&( void ) { return m256; }
|
||||
|
||||
__forceinline explicit avxf( const ssef& a ) : m256(_mm256_insertf128_ps(_mm256_castps128_ps256(a),a,1)) {}
|
||||
__forceinline avxf( const ssef& a, const ssef& b ) : m256(_mm256_insertf128_ps(_mm256_castps128_ps256(a),b,1)) {}
|
||||
|
||||
static __forceinline avxf load( const void* const ptr ) { return *(__m256*)ptr; }
|
||||
|
||||
__forceinline explicit avxf( const char* const a ) : m256(_mm256_loadu_ps((const float*)a)) {}
|
||||
__forceinline avxf( const float& a ) : m256(_mm256_broadcast_ss(&a)) {}
|
||||
__forceinline avxf( float a, float b) : m256(_mm256_set_ps(b, a, b, a, b, a, b, a)) {}
|
||||
__forceinline avxf( float a, float b, float c, float d ) : m256(_mm256_set_ps(d, c, b, a, d, c, b, a)) {}
|
||||
__forceinline avxf( float a, float b, float c, float d, float e, float f, float g, float h ) : m256(_mm256_set_ps(h, g, f, e, d, c, b, a)) {}
|
||||
|
||||
__forceinline explicit avxf( const __m256i a ) : m256(_mm256_cvtepi32_ps(a)) {}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constants
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline avxf( ZeroTy ) : m256(_mm256_setzero_ps()) {}
|
||||
__forceinline avxf( OneTy ) : m256(_mm256_set1_ps(1.0f)) {}
|
||||
__forceinline avxf( PosInfTy ) : m256(_mm256_set1_ps(pos_inf)) {}
|
||||
__forceinline avxf( NegInfTy ) : m256(_mm256_set1_ps(neg_inf)) {}
|
||||
__forceinline avxf( StepTy ) : m256(_mm256_set_ps(7.0f, 6.0f, 5.0f, 4.0f, 3.0f, 2.0f, 1.0f, 0.0f)) {}
|
||||
__forceinline avxf( NaNTy ) : m256(_mm256_set1_ps(nan)) {}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constants
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
static __forceinline avxf broadcast( const void* const a ) {
|
||||
return _mm256_broadcast_ss((float*)a);
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Array Access
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const float& operator []( const size_t i ) const { assert(i < 8); return v[i]; }
|
||||
__forceinline float& operator []( const size_t i ) { assert(i < 8); return v[i]; }
|
||||
};
|
||||
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Unary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const avxf cast (const avxi& a ) { return _mm256_castsi256_ps(a); }
|
||||
__forceinline const avxi cast (const avxf& a ) { return _mm256_castps_si256(a); }
|
||||
__forceinline const avxf operator +( const avxf& a ) { return a; }
|
||||
__forceinline const avxf operator -( const avxf& a ) {
|
||||
const __m256 mask = _mm256_castsi256_ps(_mm256_set1_epi32(0x80000000));
|
||||
return _mm256_xor_ps(a.m256, mask);
|
||||
}
|
||||
__forceinline const avxf abs ( const avxf& a ) {
|
||||
const __m256 mask = _mm256_castsi256_ps(_mm256_set1_epi32(0x7fffffff));
|
||||
return _mm256_and_ps(a.m256, mask);
|
||||
}
|
||||
__forceinline const avxf sign ( const avxf& a ) { return _mm256_blendv_ps(avxf(one), -avxf(one), _mm256_cmp_ps(a, avxf(zero), _CMP_NGE_UQ )); }
|
||||
__forceinline const avxf signmsk ( const avxf& a ) { return _mm256_and_ps(a.m256,_mm256_castsi256_ps(_mm256_set1_epi32(0x80000000))); }
|
||||
|
||||
__forceinline const avxf rcp ( const avxf& a ) {
|
||||
const avxf r = _mm256_rcp_ps(a.m256);
|
||||
return _mm256_sub_ps(_mm256_add_ps(r, r), _mm256_mul_ps(_mm256_mul_ps(r, r), a));
|
||||
}
|
||||
__forceinline const avxf sqr ( const avxf& a ) { return _mm256_mul_ps(a,a); }
|
||||
__forceinline const avxf sqrt ( const avxf& a ) { return _mm256_sqrt_ps(a.m256); }
|
||||
__forceinline const avxf rsqrt( const avxf& a ) {
|
||||
const avxf r = _mm256_rsqrt_ps(a.m256);
|
||||
return _mm256_add_ps(_mm256_mul_ps(_mm256_set1_ps(1.5f), r), _mm256_mul_ps(_mm256_mul_ps(_mm256_mul_ps(a, _mm256_set1_ps(-0.5f)), r), _mm256_mul_ps(r, r)));
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Binary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const avxf operator +( const avxf& a, const avxf& b ) { return _mm256_add_ps(a.m256, b.m256); }
|
||||
__forceinline const avxf operator +( const avxf& a, const float b ) { return a + avxf(b); }
|
||||
__forceinline const avxf operator +( const float a, const avxf& b ) { return avxf(a) + b; }
|
||||
|
||||
__forceinline const avxf operator -( const avxf& a, const avxf& b ) { return _mm256_sub_ps(a.m256, b.m256); }
|
||||
__forceinline const avxf operator -( const avxf& a, const float b ) { return a - avxf(b); }
|
||||
__forceinline const avxf operator -( const float a, const avxf& b ) { return avxf(a) - b; }
|
||||
|
||||
__forceinline const avxf operator *( const avxf& a, const avxf& b ) { return _mm256_mul_ps(a.m256, b.m256); }
|
||||
__forceinline const avxf operator *( const avxf& a, const float b ) { return a * avxf(b); }
|
||||
__forceinline const avxf operator *( const float a, const avxf& b ) { return avxf(a) * b; }
|
||||
|
||||
__forceinline const avxf operator /( const avxf& a, const avxf& b ) { return _mm256_div_ps(a.m256, b.m256); }
|
||||
__forceinline const avxf operator /( const avxf& a, const float b ) { return a / avxf(b); }
|
||||
__forceinline const avxf operator /( const float a, const avxf& b ) { return avxf(a) / b; }
|
||||
|
||||
__forceinline const avxf operator^( const avxf& a, const avxf& b ) { return _mm256_xor_ps(a.m256,b.m256); }
|
||||
__forceinline const avxf operator^( const avxf& a, const avxi& b ) { return _mm256_xor_ps(a.m256,_mm256_castsi256_ps(b.m256)); }
|
||||
|
||||
__forceinline const avxf operator&( const avxf& a, const avxf& b ) { return _mm256_and_ps(a.m256,b.m256); }
|
||||
|
||||
__forceinline const avxf min( const avxf& a, const avxf& b ) { return _mm256_min_ps(a.m256, b.m256); }
|
||||
__forceinline const avxf min( const avxf& a, const float b ) { return _mm256_min_ps(a.m256, avxf(b)); }
|
||||
__forceinline const avxf min( const float a, const avxf& b ) { return _mm256_min_ps(avxf(a), b.m256); }
|
||||
|
||||
__forceinline const avxf max( const avxf& a, const avxf& b ) { return _mm256_max_ps(a.m256, b.m256); }
|
||||
__forceinline const avxf max( const avxf& a, const float b ) { return _mm256_max_ps(a.m256, avxf(b)); }
|
||||
__forceinline const avxf max( const float a, const avxf& b ) { return _mm256_max_ps(avxf(a), b.m256); }
|
||||
|
||||
#if defined (__AVX2__)
|
||||
__forceinline avxf mini(const avxf& a, const avxf& b) {
|
||||
const avxi ai = _mm256_castps_si256(a);
|
||||
const avxi bi = _mm256_castps_si256(b);
|
||||
const avxi ci = _mm256_min_epi32(ai,bi);
|
||||
return _mm256_castsi256_ps(ci);
|
||||
}
|
||||
__forceinline avxf maxi(const avxf& a, const avxf& b) {
|
||||
const avxi ai = _mm256_castps_si256(a);
|
||||
const avxi bi = _mm256_castps_si256(b);
|
||||
const avxi ci = _mm256_max_epi32(ai,bi);
|
||||
return _mm256_castsi256_ps(ci);
|
||||
}
|
||||
#endif
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Ternary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
#if defined(__AVX2__)
|
||||
__forceinline const avxf madd ( const avxf& a, const avxf& b, const avxf& c) { return _mm256_fmadd_ps(a,b,c); }
|
||||
__forceinline const avxf msub ( const avxf& a, const avxf& b, const avxf& c) { return _mm256_fmsub_ps(a,b,c); }
|
||||
__forceinline const avxf nmadd ( const avxf& a, const avxf& b, const avxf& c) { return _mm256_fnmadd_ps(a,b,c); }
|
||||
__forceinline const avxf nmsub ( const avxf& a, const avxf& b, const avxf& c) { return _mm256_fnmsub_ps(a,b,c); }
|
||||
#else
|
||||
__forceinline const avxf madd ( const avxf& a, const avxf& b, const avxf& c) { return a*b+c; }
|
||||
__forceinline const avxf msub ( const avxf& a, const avxf& b, const avxf& c) { return a*b-c; }
|
||||
__forceinline const avxf nmadd ( const avxf& a, const avxf& b, const avxf& c) { return -a*b-c;}
|
||||
__forceinline const avxf nmsub ( const avxf& a, const avxf& b, const avxf& c) { return c-a*b; }
|
||||
#endif
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Assignment Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline avxf& operator +=( avxf& a, const avxf& b ) { return a = a + b; }
|
||||
__forceinline avxf& operator +=( avxf& a, const float b ) { return a = a + b; }
|
||||
|
||||
__forceinline avxf& operator -=( avxf& a, const avxf& b ) { return a = a - b; }
|
||||
__forceinline avxf& operator -=( avxf& a, const float b ) { return a = a - b; }
|
||||
|
||||
__forceinline avxf& operator *=( avxf& a, const avxf& b ) { return a = a * b; }
|
||||
__forceinline avxf& operator *=( avxf& a, const float b ) { return a = a * b; }
|
||||
|
||||
__forceinline avxf& operator /=( avxf& a, const avxf& b ) { return a = a / b; }
|
||||
__forceinline avxf& operator /=( avxf& a, const float b ) { return a = a / b; }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Comparison Operators + Select
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const avxb operator ==( const avxf& a, const avxf& b ) { return _mm256_cmp_ps(a.m256, b.m256, _CMP_EQ_OQ ); }
|
||||
__forceinline const avxb operator ==( const avxf& a, const float b ) { return _mm256_cmp_ps(a.m256, avxf(b), _CMP_EQ_OQ ); }
|
||||
__forceinline const avxb operator ==( const float a, const avxf& b ) { return _mm256_cmp_ps(avxf(a), b.m256, _CMP_EQ_OQ ); }
|
||||
|
||||
__forceinline const avxb operator !=( const avxf& a, const avxf& b ) { return _mm256_cmp_ps(a.m256, b.m256, _CMP_NEQ_OQ); }
|
||||
__forceinline const avxb operator !=( const avxf& a, const float b ) { return _mm256_cmp_ps(a.m256, avxf(b), _CMP_NEQ_OQ); }
|
||||
__forceinline const avxb operator !=( const float a, const avxf& b ) { return _mm256_cmp_ps(avxf(a), b.m256, _CMP_NEQ_OQ); }
|
||||
|
||||
__forceinline const avxb operator < ( const avxf& a, const avxf& b ) { return _mm256_cmp_ps(a.m256, b.m256, _CMP_LT_OQ ); }
|
||||
__forceinline const avxb operator < ( const avxf& a, const float b ) { return _mm256_cmp_ps(a.m256, avxf(b), _CMP_LT_OQ ); }
|
||||
__forceinline const avxb operator < ( const float a, const avxf& b ) { return _mm256_cmp_ps(avxf(a), b.m256, _CMP_LT_OQ ); }
|
||||
|
||||
__forceinline const avxb operator >=( const avxf& a, const avxf& b ) { return _mm256_cmp_ps(a.m256, b.m256, _CMP_GE_OQ); }
|
||||
__forceinline const avxb operator >=( const avxf& a, const float b ) { return _mm256_cmp_ps(a.m256, avxf(b), _CMP_GE_OQ); }
|
||||
__forceinline const avxb operator >=( const float a, const avxf& b ) { return _mm256_cmp_ps(avxf(a), b.m256, _CMP_GE_OQ); }
|
||||
|
||||
__forceinline const avxb operator > ( const avxf& a, const avxf& b ) { return _mm256_cmp_ps(a.m256, b.m256, _CMP_GT_OQ); }
|
||||
__forceinline const avxb operator > ( const avxf& a, const float b ) { return _mm256_cmp_ps(a.m256, avxf(b), _CMP_GT_OQ); }
|
||||
__forceinline const avxb operator > ( const float a, const avxf& b ) { return _mm256_cmp_ps(avxf(a), b.m256, _CMP_GT_OQ); }
|
||||
|
||||
__forceinline const avxb operator <=( const avxf& a, const avxf& b ) { return _mm256_cmp_ps(a.m256, b.m256, _CMP_LE_OQ ); }
|
||||
__forceinline const avxb operator <=( const avxf& a, const float b ) { return _mm256_cmp_ps(a.m256, avxf(b), _CMP_LE_OQ ); }
|
||||
__forceinline const avxb operator <=( const float a, const avxf& b ) { return _mm256_cmp_ps(avxf(a), b.m256, _CMP_LE_OQ ); }
|
||||
|
||||
__forceinline const avxf select( const avxb& m, const avxf& t, const avxf& f ) {
|
||||
return _mm256_blendv_ps(f, t, m);
|
||||
}
|
||||
|
||||
#if defined(__clang__) || defined(_MSC_VER) && !defined(__INTEL_COMPILER)
|
||||
__forceinline const avxf select(const int m, const avxf& t, const avxf& f) {
|
||||
return select(avxb(m), t, f); // workaround for clang and Microsoft compiler bugs
|
||||
}
|
||||
#else
|
||||
__forceinline const avxf select( const int m, const avxf& t, const avxf& f ) {
|
||||
return _mm256_blend_ps(f, t, m);
|
||||
}
|
||||
#endif
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Rounding Functions
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const avxf round_even( const avxf& a ) { return _mm256_round_ps(a, _MM_FROUND_TO_NEAREST_INT); }
|
||||
__forceinline const avxf round_down( const avxf& a ) { return _mm256_round_ps(a, _MM_FROUND_TO_NEG_INF ); }
|
||||
__forceinline const avxf round_up ( const avxf& a ) { return _mm256_round_ps(a, _MM_FROUND_TO_POS_INF ); }
|
||||
__forceinline const avxf round_zero( const avxf& a ) { return _mm256_round_ps(a, _MM_FROUND_TO_ZERO ); }
|
||||
__forceinline const avxf floor ( const avxf& a ) { return _mm256_round_ps(a, _MM_FROUND_TO_NEG_INF ); }
|
||||
__forceinline const avxf ceil ( const avxf& a ) { return _mm256_round_ps(a, _MM_FROUND_TO_POS_INF ); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Movement/Shifting/Shuffling Functions
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline avxf unpacklo( const avxf& a, const avxf& b ) { return _mm256_unpacklo_ps(a.m256, b.m256); }
|
||||
__forceinline avxf unpackhi( const avxf& a, const avxf& b ) { return _mm256_unpackhi_ps(a.m256, b.m256); }
|
||||
|
||||
template<size_t i> __forceinline const avxf shuffle( const avxf& a ) {
|
||||
return _mm256_permute_ps(a, _MM_SHUFFLE(i, i, i, i));
|
||||
}
|
||||
|
||||
template<size_t i0, size_t i1> __forceinline const avxf shuffle( const avxf& a ) {
|
||||
return _mm256_permute2f128_ps(a, a, (i1 << 4) | (i0 << 0));
|
||||
}
|
||||
|
||||
template<size_t i0, size_t i1> __forceinline const avxf shuffle( const avxf& a, const avxf& b) {
|
||||
return _mm256_permute2f128_ps(a, b, (i1 << 4) | (i0 << 0));
|
||||
}
|
||||
|
||||
template<size_t i0, size_t i1, size_t i2, size_t i3> __forceinline const avxf shuffle( const avxf& a ) {
|
||||
return _mm256_permute_ps(a, _MM_SHUFFLE(i3, i2, i1, i0));
|
||||
}
|
||||
|
||||
template<size_t i0, size_t i1, size_t i2, size_t i3> __forceinline const avxf shuffle( const avxf& a, const avxf& b ) {
|
||||
return _mm256_shuffle_ps(a, b, _MM_SHUFFLE(i3, i2, i1, i0));
|
||||
}
|
||||
|
||||
template<> __forceinline const avxf shuffle<0, 0, 2, 2>( const avxf& b ) { return _mm256_moveldup_ps(b); }
|
||||
template<> __forceinline const avxf shuffle<1, 1, 3, 3>( const avxf& b ) { return _mm256_movehdup_ps(b); }
|
||||
template<> __forceinline const avxf shuffle<0, 1, 0, 1>( const avxf& b ) { return _mm256_castpd_ps(_mm256_movedup_pd(_mm256_castps_pd(b))); }
|
||||
|
||||
__forceinline const avxf broadcast(const float* ptr) { return _mm256_broadcast_ss(ptr); }
|
||||
template<size_t i> __forceinline const avxf insert (const avxf& a, const ssef& b) { return _mm256_insertf128_ps (a,b,i); }
|
||||
template<size_t i> __forceinline const ssef extract (const avxf& a ) { return _mm256_extractf128_ps(a ,i); }
|
||||
template<> __forceinline const ssef extract<0>(const avxf& a ) { return _mm256_castps256_ps128(a); }
|
||||
|
||||
template<size_t i> __forceinline float fextract (const avxf& a ) { return _mm_cvtss_f32(_mm256_extractf128_ps(a ,i)); }
|
||||
|
||||
#if defined (__AVX2__)
|
||||
__forceinline avxf permute(const avxf &a, const __m256i &index) {
|
||||
return _mm256_permutevar8x32_ps(a,index);
|
||||
}
|
||||
|
||||
template<int i>
|
||||
__forceinline avxf alignr(const avxf &a, const avxf &b) {
|
||||
return _mm256_castsi256_ps(_mm256_alignr_epi8(_mm256_castps_si256(a), _mm256_castps_si256(b), i));
|
||||
}
|
||||
#endif
|
||||
|
||||
#if defined (__AVX_I__)
|
||||
template<const int mode>
|
||||
__forceinline ssei convert_to_hf16(const avxf &a) {
|
||||
return _mm256_cvtps_ph(a,mode);
|
||||
}
|
||||
|
||||
__forceinline avxf convert_from_hf16(const ssei &a) {
|
||||
return _mm256_cvtph_ps(a);
|
||||
}
|
||||
#endif
|
||||
|
||||
__forceinline ssef broadcast4f( const avxf& a, const size_t k ) {
|
||||
return ssef::broadcast(&a[k]);
|
||||
}
|
||||
|
||||
__forceinline avxf broadcast8f( const avxf& a, const size_t k ) {
|
||||
return avxf::broadcast(&a[k]);
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Transpose
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline void transpose4(const avxf& r0, const avxf& r1, const avxf& r2, const avxf& r3, avxf& c0, avxf& c1, avxf& c2, avxf& c3)
|
||||
{
|
||||
avxf l02 = unpacklo(r0,r2);
|
||||
avxf h02 = unpackhi(r0,r2);
|
||||
avxf l13 = unpacklo(r1,r3);
|
||||
avxf h13 = unpackhi(r1,r3);
|
||||
c0 = unpacklo(l02,l13);
|
||||
c1 = unpackhi(l02,l13);
|
||||
c2 = unpacklo(h02,h13);
|
||||
c3 = unpackhi(h02,h13);
|
||||
}
|
||||
|
||||
__forceinline void transpose(const avxf& r0, const avxf& r1, const avxf& r2, const avxf& r3, const avxf& r4, const avxf& r5, const avxf& r6, const avxf& r7,
|
||||
avxf& c0, avxf& c1, avxf& c2, avxf& c3, avxf& c4, avxf& c5, avxf& c6, avxf& c7)
|
||||
{
|
||||
avxf h0,h1,h2,h3; transpose4(r0,r1,r2,r3,h0,h1,h2,h3);
|
||||
avxf h4,h5,h6,h7; transpose4(r4,r5,r6,r7,h4,h5,h6,h7);
|
||||
c0 = shuffle<0,2>(h0,h4);
|
||||
c1 = shuffle<0,2>(h1,h5);
|
||||
c2 = shuffle<0,2>(h2,h6);
|
||||
c3 = shuffle<0,2>(h3,h7);
|
||||
c4 = shuffle<1,3>(h0,h4);
|
||||
c5 = shuffle<1,3>(h1,h5);
|
||||
c6 = shuffle<1,3>(h2,h6);
|
||||
c7 = shuffle<1,3>(h3,h7);
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Reductions
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const avxf vreduce_min2(const avxf& v) { return min(v,shuffle<1,0,3,2>(v)); }
|
||||
__forceinline const avxf vreduce_min4(const avxf& v) { avxf v1 = vreduce_min2(v); return min(v1,shuffle<2,3,0,1>(v1)); }
|
||||
__forceinline const avxf vreduce_min (const avxf& v) { avxf v1 = vreduce_min4(v); return min(v1,shuffle<1,0>(v1)); }
|
||||
|
||||
__forceinline const avxf vreduce_max2(const avxf& v) { return max(v,shuffle<1,0,3,2>(v)); }
|
||||
__forceinline const avxf vreduce_max4(const avxf& v) { avxf v1 = vreduce_max2(v); return max(v1,shuffle<2,3,0,1>(v1)); }
|
||||
__forceinline const avxf vreduce_max (const avxf& v) { avxf v1 = vreduce_max4(v); return max(v1,shuffle<1,0>(v1)); }
|
||||
|
||||
__forceinline const avxf vreduce_add2(const avxf& v) { return v + shuffle<1,0,3,2>(v); }
|
||||
__forceinline const avxf vreduce_add4(const avxf& v) { avxf v1 = vreduce_add2(v); return v1 + shuffle<2,3,0,1>(v1); }
|
||||
__forceinline const avxf vreduce_add (const avxf& v) { avxf v1 = vreduce_add4(v); return v1 + shuffle<1,0>(v1); }
|
||||
|
||||
__forceinline float reduce_min(const avxf& v) { return _mm_cvtss_f32(extract<0>(vreduce_min(v))); }
|
||||
__forceinline float reduce_max(const avxf& v) { return _mm_cvtss_f32(extract<0>(vreduce_max(v))); }
|
||||
__forceinline float reduce_add(const avxf& v) { return _mm_cvtss_f32(extract<0>(vreduce_add(v))); }
|
||||
|
||||
__forceinline size_t select_min(const avxf& v) { return __bsf(movemask(v == vreduce_min(v))); }
|
||||
__forceinline size_t select_max(const avxf& v) { return __bsf(movemask(v == vreduce_max(v))); }
|
||||
|
||||
__forceinline size_t select_min(const avxb& valid, const avxf& v) { const avxf a = select(valid,v,avxf(pos_inf)); return __bsf(movemask(valid & (a == vreduce_min(a)))); }
|
||||
__forceinline size_t select_max(const avxb& valid, const avxf& v) { const avxf a = select(valid,v,avxf(neg_inf)); return __bsf(movemask(valid & (a == vreduce_max(a)))); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Memory load and store operations
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline avxf load8f( const void* const a) {
|
||||
return _mm256_load_ps((const float*)a);
|
||||
}
|
||||
|
||||
__forceinline void store8f(void *ptr, const avxf& f ) {
|
||||
return _mm256_store_ps((float*)ptr,f);
|
||||
}
|
||||
|
||||
__forceinline void store8f( const avxb& mask, void *ptr, const avxf& f ) {
|
||||
return _mm256_maskstore_ps((float*)ptr,(__m256i)mask,f);
|
||||
}
|
||||
|
||||
#if defined (__AVX2__)
|
||||
__forceinline avxf load8f_nt(void* ptr) {
|
||||
return _mm256_castsi256_ps(_mm256_stream_load_si256((__m256i*)ptr));
|
||||
}
|
||||
|
||||
#endif
|
||||
|
||||
__forceinline void store8f_nt(void* ptr, const avxf& v) {
|
||||
_mm256_stream_ps((float*)ptr,v);
|
||||
}
|
||||
|
||||
__forceinline const avxf broadcast4f(const void* ptr) {
|
||||
return _mm256_broadcast_ps((__m128*)ptr);
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Euclidian Space Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline avxf dot ( const avxf& a, const avxf& b ) {
|
||||
return vreduce_add4(a*b);
|
||||
}
|
||||
|
||||
__forceinline avxf cross ( const avxf& a, const avxf& b )
|
||||
{
|
||||
const avxf a0 = a;
|
||||
const avxf b0 = shuffle<1,2,0,3>(b);
|
||||
const avxf a1 = shuffle<1,2,0,3>(a);
|
||||
const avxf b1 = b;
|
||||
return shuffle<1,2,0,3>(msub(a0,b0,a1*b1));
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Output Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
inline std::ostream& operator<<(std::ostream& cout, const avxf& a) {
|
||||
return cout << "<" << a[0] << ", " << a[1] << ", " << a[2] << ", " << a[3] << ", " << a[4] << ", " << a[5] << ", " << a[6] << ", " << a[7] << ">";
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,289 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#pragma once
|
||||
|
||||
namespace embree
|
||||
{
|
||||
/*! 8-wide AVX integer type. */
|
||||
struct avxi
|
||||
{
|
||||
typedef avxb Mask; // mask type for us
|
||||
enum { size = 8 }; // number of SIMD elements
|
||||
union { // data
|
||||
__m256i m256;
|
||||
int32 v[8];
|
||||
};
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constructors, Assignment & Cast Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline avxi ( ) {}
|
||||
__forceinline avxi ( const avxi& a ) { m256 = a.m256; }
|
||||
__forceinline avxi& operator=( const avxi& a ) { m256 = a.m256; return *this; }
|
||||
|
||||
__forceinline avxi( const __m256i a ) : m256(a) {}
|
||||
__forceinline operator const __m256i&( void ) const { return m256; }
|
||||
__forceinline operator __m256i&( void ) { return m256; }
|
||||
|
||||
__forceinline explicit avxi( const ssei& a ) : m256(_mm256_insertf128_si256(_mm256_castsi128_si256(a),a,1)) {}
|
||||
__forceinline avxi( const ssei& a, const ssei& b ) : m256(_mm256_insertf128_si256(_mm256_castsi128_si256(a),b,1)) {}
|
||||
__forceinline avxi( const __m128i& a, const __m128i& b ) : m256(_mm256_insertf128_si256(_mm256_castsi128_si256(a),b,1)) {}
|
||||
|
||||
__forceinline explicit avxi ( const int32* const a ) : m256(_mm256_castps_si256(_mm256_loadu_ps((const float*)a))) {}
|
||||
__forceinline avxi ( int32 a ) : m256(_mm256_set1_epi32(a)) {}
|
||||
__forceinline avxi ( int32 a, int32 b) : m256(_mm256_set_epi32(b, a, b, a, b, a, b, a)) {}
|
||||
__forceinline avxi ( int32 a, int32 b, int32 c, int32 d) : m256(_mm256_set_epi32(d, c, b, a, d, c, b, a)) {}
|
||||
__forceinline avxi ( int32 a, int32 b, int32 c, int32 d, int32 e, int32 f, int32 g, int32 h) : m256(_mm256_set_epi32(h, g, f, e, d, c, b, a)) {}
|
||||
|
||||
__forceinline explicit avxi( const __m256 a ) : m256(_mm256_cvtps_epi32(a)) {}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constants
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline avxi( ZeroTy ) : m256(_mm256_setzero_si256()) {}
|
||||
__forceinline avxi( OneTy ) : m256(_mm256_set1_epi32(1)) {}
|
||||
__forceinline avxi( PosInfTy ) : m256(_mm256_set1_epi32(pos_inf)) {}
|
||||
__forceinline avxi( NegInfTy ) : m256(_mm256_set1_epi32(neg_inf)) {}
|
||||
__forceinline avxi( StepTy ) : m256(_mm256_set_epi32(7, 6, 5, 4, 3, 2, 1, 0)) {}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Array Access
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const int32& operator []( const size_t i ) const { assert(i < 8); return v[i]; }
|
||||
__forceinline int32& operator []( const size_t i ) { assert(i < 8); return v[i]; }
|
||||
};
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Unary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const avxi operator +( const avxi& a ) { return a; }
|
||||
__forceinline const avxi operator -( const avxi& a ) { return _mm256_sub_epi32(_mm256_setzero_si256(), a.m256); }
|
||||
__forceinline const avxi abs ( const avxi& a ) { return _mm256_abs_epi32(a.m256); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Binary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const avxi operator +( const avxi& a, const avxi& b ) { return _mm256_add_epi32(a.m256, b.m256); }
|
||||
__forceinline const avxi operator +( const avxi& a, const int32 b ) { return a + avxi(b); }
|
||||
__forceinline const avxi operator +( const int32 a, const avxi& b ) { return avxi(a) + b; }
|
||||
|
||||
__forceinline const avxi operator -( const avxi& a, const avxi& b ) { return _mm256_sub_epi32(a.m256, b.m256); }
|
||||
__forceinline const avxi operator -( const avxi& a, const int32 b ) { return a - avxi(b); }
|
||||
__forceinline const avxi operator -( const int32 a, const avxi& b ) { return avxi(a) - b; }
|
||||
|
||||
__forceinline const avxi operator *( const avxi& a, const avxi& b ) { return _mm256_mullo_epi32(a.m256, b.m256); }
|
||||
__forceinline const avxi operator *( const avxi& a, const int32 b ) { return a * avxi(b); }
|
||||
__forceinline const avxi operator *( const int32 a, const avxi& b ) { return avxi(a) * b; }
|
||||
|
||||
__forceinline const avxi operator &( const avxi& a, const avxi& b ) { return _mm256_and_si256(a.m256, b.m256); }
|
||||
__forceinline const avxi operator &( const avxi& a, const int32 b ) { return a & avxi(b); }
|
||||
__forceinline const avxi operator &( const int32 a, const avxi& b ) { return avxi(a) & b; }
|
||||
|
||||
__forceinline const avxi operator |( const avxi& a, const avxi& b ) { return _mm256_or_si256(a.m256, b.m256); }
|
||||
__forceinline const avxi operator |( const avxi& a, const int32 b ) { return a | avxi(b); }
|
||||
__forceinline const avxi operator |( const int32 a, const avxi& b ) { return avxi(a) | b; }
|
||||
|
||||
__forceinline const avxi operator ^( const avxi& a, const avxi& b ) { return _mm256_xor_si256(a.m256, b.m256); }
|
||||
__forceinline const avxi operator ^( const avxi& a, const int32 b ) { return a ^ avxi(b); }
|
||||
__forceinline const avxi operator ^( const int32 a, const avxi& b ) { return avxi(a) ^ b; }
|
||||
|
||||
__forceinline const avxi operator <<( const avxi& a, const int32 n ) { return _mm256_slli_epi32(a.m256, n); }
|
||||
__forceinline const avxi operator >>( const avxi& a, const int32 n ) { return _mm256_srai_epi32(a.m256, n); }
|
||||
|
||||
__forceinline const avxi sra ( const avxi& a, const int32 b ) { return _mm256_srai_epi32(a.m256, b); }
|
||||
__forceinline const avxi srl ( const avxi& a, const int32 b ) { return _mm256_srli_epi32(a.m256, b); }
|
||||
|
||||
__forceinline const avxi min( const avxi& a, const avxi& b ) { return _mm256_min_epi32(a.m256, b.m256); }
|
||||
__forceinline const avxi min( const avxi& a, const int32 b ) { return min(a,avxi(b)); }
|
||||
__forceinline const avxi min( const int32 a, const avxi& b ) { return min(avxi(a),b); }
|
||||
|
||||
__forceinline const avxi max( const avxi& a, const avxi& b ) { return _mm256_max_epi32(a.m256, b.m256); }
|
||||
__forceinline const avxi max( const avxi& a, const int32 b ) { return max(a,avxi(b)); }
|
||||
__forceinline const avxi max( const int32 a, const avxi& b ) { return max(avxi(a),b); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Assignment Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline avxi& operator +=( avxi& a, const avxi& b ) { return a = a + b; }
|
||||
__forceinline avxi& operator +=( avxi& a, const int32 b ) { return a = a + b; }
|
||||
|
||||
__forceinline avxi& operator -=( avxi& a, const avxi& b ) { return a = a - b; }
|
||||
__forceinline avxi& operator -=( avxi& a, const int32 b ) { return a = a - b; }
|
||||
|
||||
__forceinline avxi& operator *=( avxi& a, const avxi& b ) { return a = a * b; }
|
||||
__forceinline avxi& operator *=( avxi& a, const int32 b ) { return a = a * b; }
|
||||
|
||||
__forceinline avxi& operator &=( avxi& a, const avxi& b ) { return a = a & b; }
|
||||
__forceinline avxi& operator &=( avxi& a, const int32 b ) { return a = a & b; }
|
||||
|
||||
__forceinline avxi& operator |=( avxi& a, const avxi& b ) { return a = a | b; }
|
||||
__forceinline avxi& operator |=( avxi& a, const int32 b ) { return a = a | b; }
|
||||
|
||||
__forceinline avxi& operator <<=( avxi& a, const int32 b ) { return a = a << b; }
|
||||
__forceinline avxi& operator >>=( avxi& a, const int32 b ) { return a = a >> b; }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Comparison Operators + Select
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const avxb operator ==( const avxi& a, const avxi& b ) { return _mm256_castsi256_ps(_mm256_cmpeq_epi32 (a.m256, b.m256)); }
|
||||
__forceinline const avxb operator ==( const avxi& a, const int32 b ) { return a == avxi(b); }
|
||||
__forceinline const avxb operator ==( const int32 a, const avxi& b ) { return avxi(a) == b; }
|
||||
|
||||
__forceinline const avxb operator !=( const avxi& a, const avxi& b ) { return !(a == b); }
|
||||
__forceinline const avxb operator !=( const avxi& a, const int32 b ) { return a != avxi(b); }
|
||||
__forceinline const avxb operator !=( const int32 a, const avxi& b ) { return avxi(a) != b; }
|
||||
|
||||
__forceinline const avxb operator < ( const avxi& a, const avxi& b ) { return _mm256_castsi256_ps(_mm256_cmpgt_epi32 (b.m256, a.m256)); }
|
||||
__forceinline const avxb operator < ( const avxi& a, const int32 b ) { return a < avxi(b); }
|
||||
__forceinline const avxb operator < ( const int32 a, const avxi& b ) { return avxi(a) < b; }
|
||||
|
||||
__forceinline const avxb operator >=( const avxi& a, const avxi& b ) { return !(a < b); }
|
||||
__forceinline const avxb operator >=( const avxi& a, const int32 b ) { return a >= avxi(b); }
|
||||
__forceinline const avxb operator >=( const int32 a, const avxi& b ) { return avxi(a) >= b; }
|
||||
|
||||
__forceinline const avxb operator > ( const avxi& a, const avxi& b ) { return _mm256_castsi256_ps(_mm256_cmpgt_epi32 (a.m256, b.m256)); }
|
||||
__forceinline const avxb operator > ( const avxi& a, const int32 b ) { return a > avxi(b); }
|
||||
__forceinline const avxb operator > ( const int32 a, const avxi& b ) { return avxi(a) > b; }
|
||||
|
||||
__forceinline const avxb operator <=( const avxi& a, const avxi& b ) { return !(a > b); }
|
||||
__forceinline const avxb operator <=( const avxi& a, const int32 b ) { return a <= avxi(b); }
|
||||
__forceinline const avxb operator <=( const int32 a, const avxi& b ) { return avxi(a) <= b; }
|
||||
|
||||
__forceinline const avxi select( const avxb& m, const avxi& t, const avxi& f ) {
|
||||
return _mm256_castps_si256(_mm256_blendv_ps(_mm256_castsi256_ps(f), _mm256_castsi256_ps(t), m));
|
||||
}
|
||||
|
||||
#if defined(__clang__) || defined(_MSC_VER) && !defined(__INTEL_COMPILER)
|
||||
__forceinline const avxi select(const int m, const avxi& t, const avxi& f) {
|
||||
return select(avxb(m), t, f); // workaround for clang and Microsoft compiler bugs
|
||||
}
|
||||
#else
|
||||
__forceinline const avxi select(const int m, const avxi& t, const avxi& f) {
|
||||
return _mm256_blend_epi32(f, t, m);
|
||||
}
|
||||
#endif
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Movement/Shifting/Shuffling Functions
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline avxi unpacklo( const avxi& a, const avxi& b ) { return _mm256_unpacklo_epi32(a.m256, b.m256); }
|
||||
__forceinline avxi unpackhi( const avxi& a, const avxi& b ) { return _mm256_unpackhi_epi32(a.m256, b.m256); }
|
||||
|
||||
template<size_t i> __forceinline const avxi shuffle( const avxi& a ) {
|
||||
return _mm256_castps_si256(_mm256_permute_ps(_mm256_castsi256_ps(a), _MM_SHUFFLE(i, i, i, i)));
|
||||
}
|
||||
|
||||
template<size_t i0, size_t i1> __forceinline const avxi shuffle( const avxi& a ) {
|
||||
return _mm256_permute2f128_si256(a, a, (i1 << 4) | (i0 << 0));
|
||||
}
|
||||
|
||||
template<size_t i0, size_t i1> __forceinline const avxi shuffle( const avxi& a, const avxi& b) {
|
||||
return _mm256_permute2f128_si256(a, b, (i1 << 4) | (i0 << 0));
|
||||
}
|
||||
|
||||
template<size_t i0, size_t i1, size_t i2, size_t i3> __forceinline const avxi shuffle( const avxi& a ) {
|
||||
return _mm256_castps_si256(_mm256_permute_ps(_mm256_castsi256_ps(a), _MM_SHUFFLE(i3, i2, i1, i0)));
|
||||
}
|
||||
|
||||
template<size_t i0, size_t i1, size_t i2, size_t i3> __forceinline const avxi shuffle( const avxi& a, const avxi& b ) {
|
||||
return _mm256_castps_si256(_mm256_shuffle_ps(_mm256_castsi256_ps(a), _mm256_castsi256_ps(b), _MM_SHUFFLE(i3, i2, i1, i0)));
|
||||
}
|
||||
|
||||
template<> __forceinline const avxi shuffle<0, 0, 2, 2>( const avxi& b ) { return _mm256_castps_si256(_mm256_moveldup_ps(_mm256_castsi256_ps(b))); }
|
||||
template<> __forceinline const avxi shuffle<1, 1, 3, 3>( const avxi& b ) { return _mm256_castps_si256(_mm256_movehdup_ps(_mm256_castsi256_ps(b))); }
|
||||
template<> __forceinline const avxi shuffle<0, 1, 0, 1>( const avxi& b ) { return _mm256_castps_si256(_mm256_castpd_ps(_mm256_movedup_pd(_mm256_castps_pd(_mm256_castsi256_ps(b))))); }
|
||||
|
||||
__forceinline const avxi broadcast(const int* ptr) { return _mm256_castps_si256(_mm256_broadcast_ss((const float*)ptr)); }
|
||||
template<size_t i> __forceinline const avxi insert (const avxi& a, const ssei& b) { return _mm256_insertf128_si256 (a,b,i); }
|
||||
template<size_t i> __forceinline const ssei extract(const avxi& a ) { return _mm256_extractf128_si256(a ,i); }
|
||||
|
||||
__forceinline avxi permute(const avxi& a, const __m256i& index) {
|
||||
return _mm256_permutevar8x32_epi32(a,index);
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Reductions
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const avxi vreduce_min2(const avxi& v) { return min(v,shuffle<1,0,3,2>(v)); }
|
||||
__forceinline const avxi vreduce_min4(const avxi& v) { avxi v1 = vreduce_min2(v); return min(v1,shuffle<2,3,0,1>(v1)); }
|
||||
__forceinline const avxi vreduce_min (const avxi& v) { avxi v1 = vreduce_min4(v); return min(v1,shuffle<1,0>(v1)); }
|
||||
|
||||
__forceinline const avxi vreduce_max2(const avxi& v) { return max(v,shuffle<1,0,3,2>(v)); }
|
||||
__forceinline const avxi vreduce_max4(const avxi& v) { avxi v1 = vreduce_max2(v); return max(v1,shuffle<2,3,0,1>(v1)); }
|
||||
__forceinline const avxi vreduce_max (const avxi& v) { avxi v1 = vreduce_max4(v); return max(v1,shuffle<1,0>(v1)); }
|
||||
|
||||
__forceinline const avxi vreduce_add2(const avxi& v) { return v + shuffle<1,0,3,2>(v); }
|
||||
__forceinline const avxi vreduce_add4(const avxi& v) { avxi v1 = vreduce_add2(v); return v1 + shuffle<2,3,0,1>(v1); }
|
||||
__forceinline const avxi vreduce_add (const avxi& v) { avxi v1 = vreduce_add4(v); return v1 + shuffle<1,0>(v1); }
|
||||
|
||||
__forceinline int reduce_min(const avxi& v) { return extract<0>(extract<0>(vreduce_min(v))); }
|
||||
__forceinline int reduce_max(const avxi& v) { return extract<0>(extract<0>(vreduce_max(v))); }
|
||||
__forceinline int reduce_add(const avxi& v) { return extract<0>(extract<0>(vreduce_add(v))); }
|
||||
|
||||
__forceinline size_t select_min(const avxi& v) { return __bsf(movemask(v == vreduce_min(v))); }
|
||||
__forceinline size_t select_max(const avxi& v) { return __bsf(movemask(v == vreduce_max(v))); }
|
||||
|
||||
__forceinline size_t select_min(const avxb& valid, const avxi& v) { const avxi a = select(valid,v,avxi(pos_inf)); return __bsf(movemask(valid & (a == vreduce_min(a)))); }
|
||||
__forceinline size_t select_max(const avxb& valid, const avxi& v) { const avxi a = select(valid,v,avxi(neg_inf)); return __bsf(movemask(valid & (a == vreduce_max(a)))); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Memory load and store operations
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const avxi load8i(const int* const i) {
|
||||
return _mm256_load_si256((__m256i*)i);
|
||||
}
|
||||
|
||||
__forceinline const avxi uload8i(const int* const i) {
|
||||
return _mm256_loadu_si256((__m256i*)i);
|
||||
}
|
||||
|
||||
__forceinline void store8i(void *ptr, const avxi& i ) {
|
||||
_mm256_store_si256((__m256i*)ptr,i);
|
||||
}
|
||||
|
||||
__forceinline void store8i( const avxb &mask, void *ptr, const avxi& i ) {
|
||||
_mm256_maskstore_epi32((int*)ptr,mask,i);
|
||||
}
|
||||
|
||||
#if defined (__AVX2__)
|
||||
__forceinline avxi load8i_nt(void* ptr) {
|
||||
return _mm256_stream_load_si256((__m256i*)ptr);
|
||||
}
|
||||
#endif
|
||||
|
||||
__forceinline void store8i_nt(void* ptr, const avxi& v) {
|
||||
_mm256_stream_ps((float*)ptr,_mm256_castsi256_ps(v));
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Output Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
inline std::ostream& operator<<(std::ostream& cout, const avxi& a) {
|
||||
return cout << "<" << a[0] << ", " << a[1] << ", " << a[2] << ", " << a[3] << ", " << a[4] << ", " << a[5] << ", " << a[6] << ", " << a[7] << ">";
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,266 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#pragma once
|
||||
|
||||
namespace embree
|
||||
{
|
||||
/*! 8-wide AVX integer type. */
|
||||
struct avxi
|
||||
{
|
||||
typedef avxb Mask; // mask type for us
|
||||
enum { size = 8 }; // number of SIMD elements
|
||||
union { // data
|
||||
__m256i m256;
|
||||
struct { __m128i l,h; };
|
||||
int32 v[8];
|
||||
};
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constructors, Assignment & Cast Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline avxi ( ) {}
|
||||
__forceinline avxi ( const avxi& a ) { m256 = a.m256; }
|
||||
__forceinline avxi& operator=( const avxi& a ) { m256 = a.m256; return *this; }
|
||||
|
||||
__forceinline avxi( const __m256i a ) : m256(a) {}
|
||||
__forceinline operator const __m256i&( void ) const { return m256; }
|
||||
__forceinline operator __m256i&( void ) { return m256; }
|
||||
|
||||
__forceinline explicit avxi( const ssei& a ) : m256(_mm256_insertf128_si256(_mm256_castsi128_si256(a),a,1)) {}
|
||||
__forceinline avxi( const ssei& a, const ssei& b ) : m256(_mm256_insertf128_si256(_mm256_castsi128_si256(a),b,1)) {}
|
||||
__forceinline avxi( const __m128i& a, const __m128i& b ) : l(a), h(b) {}
|
||||
|
||||
__forceinline explicit avxi ( const int32* const a ) : m256(_mm256_castps_si256(_mm256_loadu_ps((const float*)a))) {}
|
||||
__forceinline avxi ( int32 a ) : m256(_mm256_set1_epi32(a)) {}
|
||||
__forceinline avxi ( int32 a, int32 b) : m256(_mm256_set_epi32(b, a, b, a, b, a, b, a)) {}
|
||||
__forceinline avxi ( int32 a, int32 b, int32 c, int32 d) : m256(_mm256_set_epi32(d, c, b, a, d, c, b, a)) {}
|
||||
__forceinline avxi ( int32 a, int32 b, int32 c, int32 d, int32 e, int32 f, int32 g, int32 h) : m256(_mm256_set_epi32(h, g, f, e, d, c, b, a)) {}
|
||||
|
||||
__forceinline explicit avxi( const __m256 a ) : m256(_mm256_cvtps_epi32(a)) {}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constants
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline avxi( ZeroTy ) : m256(_mm256_setzero_si256()) {}
|
||||
__forceinline avxi( OneTy ) : m256(_mm256_set_epi32(1,1,1,1,1,1,1,1)) {}
|
||||
__forceinline avxi( PosInfTy ) : m256(_mm256_set_epi32(pos_inf,pos_inf,pos_inf,pos_inf,pos_inf,pos_inf,pos_inf,pos_inf)) {}
|
||||
__forceinline avxi( NegInfTy ) : m256(_mm256_set_epi32(neg_inf,neg_inf,neg_inf,neg_inf,neg_inf,neg_inf,neg_inf,neg_inf)) {}
|
||||
__forceinline avxi( StepTy ) : m256(_mm256_set_epi32(7, 6, 5, 4, 3, 2, 1, 0)) {}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Array Access
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const int32& operator []( const size_t i ) const { assert(i < 8); return v[i]; }
|
||||
__forceinline int32& operator []( const size_t i ) { assert(i < 8); return v[i]; }
|
||||
};
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Unary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const avxi operator +( const avxi& a ) { return a; }
|
||||
__forceinline const avxi operator -( const avxi& a ) { return avxi(_mm_sub_epi32(_mm_setzero_si128(), a.l), _mm_sub_epi32(_mm_setzero_si128(), a.h)); }
|
||||
__forceinline const avxi abs ( const avxi& a ) { return avxi(_mm_abs_epi32(a.l), _mm_abs_epi32(a.h)); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Binary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const avxi operator +( const avxi& a, const avxi& b ) { return avxi(_mm_add_epi32(a.l, b.l), _mm_add_epi32(a.h, b.h)); }
|
||||
__forceinline const avxi operator +( const avxi& a, const int32 b ) { return a + avxi(b); }
|
||||
__forceinline const avxi operator +( const int32 a, const avxi& b ) { return avxi(a) + b; }
|
||||
|
||||
__forceinline const avxi operator -( const avxi& a, const avxi& b ) { return avxi(_mm_sub_epi32(a.l, b.l), _mm_sub_epi32(a.h, b.h)); }
|
||||
__forceinline const avxi operator -( const avxi& a, const int32 b ) { return a - avxi(b); }
|
||||
__forceinline const avxi operator -( const int32 a, const avxi& b ) { return avxi(a) - b; }
|
||||
|
||||
__forceinline const avxi operator *( const avxi& a, const avxi& b ) { return avxi(_mm_mullo_epi32(a.l, b.l), _mm_mullo_epi32(a.h, b.h)); }
|
||||
__forceinline const avxi operator *( const avxi& a, const int32 b ) { return a * avxi(b); }
|
||||
__forceinline const avxi operator *( const int32 a, const avxi& b ) { return avxi(a) * b; }
|
||||
|
||||
__forceinline const avxi operator &( const avxi& a, const avxi& b ) { return _mm256_castps_si256(_mm256_and_ps(_mm256_castsi256_ps(a), _mm256_castsi256_ps(b))); }
|
||||
__forceinline const avxi operator &( const avxi& a, const int32 b ) { return a & avxi(b); }
|
||||
__forceinline const avxi operator &( const int32 a, const avxi& b ) { return avxi(a) & b; }
|
||||
|
||||
__forceinline const avxi operator |( const avxi& a, const avxi& b ) { return _mm256_castps_si256(_mm256_or_ps (_mm256_castsi256_ps(a), _mm256_castsi256_ps(b))); }
|
||||
__forceinline const avxi operator |( const avxi& a, const int32 b ) { return a | avxi(b); }
|
||||
__forceinline const avxi operator |( const int32 a, const avxi& b ) { return avxi(a) | b; }
|
||||
|
||||
__forceinline const avxi operator ^( const avxi& a, const avxi& b ) { return _mm256_castps_si256(_mm256_xor_ps(_mm256_castsi256_ps(a), _mm256_castsi256_ps(b))); }
|
||||
__forceinline const avxi operator ^( const avxi& a, const int32 b ) { return a ^ avxi(b); }
|
||||
__forceinline const avxi operator ^( const int32 a, const avxi& b ) { return avxi(a) ^ b; }
|
||||
|
||||
__forceinline const avxi operator <<( const avxi& a, const int32 n ) { return avxi(_mm_slli_epi32(a.l, n), _mm_slli_epi32(a.h, n)); }
|
||||
__forceinline const avxi operator >>( const avxi& a, const int32 n ) { return avxi(_mm_srai_epi32(a.l, n), _mm_srai_epi32(a.h, n)); }
|
||||
|
||||
__forceinline const avxi sra ( const avxi& a, const int32 b ) { return avxi(_mm_srai_epi32(a.l, b), _mm_srai_epi32(a.h, b)); }
|
||||
__forceinline const avxi srl ( const avxi& a, const int32 b ) { return avxi(_mm_srli_epi32(a.l, b), _mm_srli_epi32(a.h, b)); }
|
||||
|
||||
__forceinline const avxi min( const avxi& a, const avxi& b ) { return avxi(_mm_min_epi32(a.l, b.l), _mm_min_epi32(a.h, b.h)); }
|
||||
__forceinline const avxi min( const avxi& a, const int32 b ) { return min(a,avxi(b)); }
|
||||
__forceinline const avxi min( const int32 a, const avxi& b ) { return min(avxi(a),b); }
|
||||
|
||||
__forceinline const avxi max( const avxi& a, const avxi& b ) { return avxi(_mm_max_epi32(a.l, b.l), _mm_max_epi32(a.h, b.h)); }
|
||||
__forceinline const avxi max( const avxi& a, const int32 b ) { return max(a,avxi(b)); }
|
||||
__forceinline const avxi max( const int32 a, const avxi& b ) { return max(avxi(a),b); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Assignment Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline avxi& operator +=( avxi& a, const avxi& b ) { return a = a + b; }
|
||||
__forceinline avxi& operator +=( avxi& a, const int32 b ) { return a = a + b; }
|
||||
|
||||
__forceinline avxi& operator -=( avxi& a, const avxi& b ) { return a = a - b; }
|
||||
__forceinline avxi& operator -=( avxi& a, const int32 b ) { return a = a - b; }
|
||||
|
||||
__forceinline avxi& operator *=( avxi& a, const avxi& b ) { return a = a * b; }
|
||||
__forceinline avxi& operator *=( avxi& a, const int32 b ) { return a = a * b; }
|
||||
|
||||
__forceinline avxi& operator &=( avxi& a, const avxi& b ) { return a = a & b; }
|
||||
__forceinline avxi& operator &=( avxi& a, const int32 b ) { return a = a & b; }
|
||||
|
||||
__forceinline avxi& operator |=( avxi& a, const avxi& b ) { return a = a | b; }
|
||||
__forceinline avxi& operator |=( avxi& a, const int32 b ) { return a = a | b; }
|
||||
|
||||
__forceinline avxi& operator <<=( avxi& a, const int32 b ) { return a = a << b; }
|
||||
__forceinline avxi& operator >>=( avxi& a, const int32 b ) { return a = a >> b; }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Comparison Operators + Select
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const avxb operator ==( const avxi& a, const avxi& b ) { return avxb(_mm_castsi128_ps(_mm_cmpeq_epi32 (a.l, b.l)),
|
||||
_mm_castsi128_ps(_mm_cmpeq_epi32 (a.h, b.h))); }
|
||||
__forceinline const avxb operator ==( const avxi& a, const int32 b ) { return a == avxi(b); }
|
||||
__forceinline const avxb operator ==( const int32 a, const avxi& b ) { return avxi(a) == b; }
|
||||
|
||||
__forceinline const avxb operator !=( const avxi& a, const avxi& b ) { return !(a == b); }
|
||||
__forceinline const avxb operator !=( const avxi& a, const int32 b ) { return a != avxi(b); }
|
||||
__forceinline const avxb operator !=( const int32 a, const avxi& b ) { return avxi(a) != b; }
|
||||
|
||||
__forceinline const avxb operator < ( const avxi& a, const avxi& b ) { return avxb(_mm_castsi128_ps(_mm_cmplt_epi32 (a.l, b.l)),
|
||||
_mm_castsi128_ps(_mm_cmplt_epi32 (a.h, b.h))); }
|
||||
__forceinline const avxb operator < ( const avxi& a, const int32 b ) { return a < avxi(b); }
|
||||
__forceinline const avxb operator < ( const int32 a, const avxi& b ) { return avxi(a) < b; }
|
||||
|
||||
__forceinline const avxb operator >=( const avxi& a, const avxi& b ) { return !(a < b); }
|
||||
__forceinline const avxb operator >=( const avxi& a, const int32 b ) { return a >= avxi(b); }
|
||||
__forceinline const avxb operator >=( const int32 a, const avxi& b ) { return avxi(a) >= b; }
|
||||
|
||||
__forceinline const avxb operator > ( const avxi& a, const avxi& b ) { return avxb(_mm_castsi128_ps(_mm_cmpgt_epi32 (a.l, b.l)),
|
||||
_mm_castsi128_ps(_mm_cmpgt_epi32 (a.h, b.h))); }
|
||||
__forceinline const avxb operator > ( const avxi& a, const int32 b ) { return a > avxi(b); }
|
||||
__forceinline const avxb operator > ( const int32 a, const avxi& b ) { return avxi(a) > b; }
|
||||
|
||||
__forceinline const avxb operator <=( const avxi& a, const avxi& b ) { return !(a > b); }
|
||||
__forceinline const avxb operator <=( const avxi& a, const int32 b ) { return a <= avxi(b); }
|
||||
__forceinline const avxb operator <=( const int32 a, const avxi& b ) { return avxi(a) <= b; }
|
||||
|
||||
__forceinline const avxi select( const avxb& m, const avxi& t, const avxi& f ) {
|
||||
return _mm256_castps_si256(_mm256_blendv_ps(_mm256_castsi256_ps(f), _mm256_castsi256_ps(t), m));
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Movement/Shifting/Shuffling Functions
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline avxi unpacklo( const avxi& a, const avxi& b ) { return _mm256_castps_si256(_mm256_unpacklo_ps(_mm256_castsi256_ps(a), _mm256_castsi256_ps(b))); }
|
||||
__forceinline avxi unpackhi( const avxi& a, const avxi& b ) { return _mm256_castps_si256(_mm256_unpackhi_ps(_mm256_castsi256_ps(a), _mm256_castsi256_ps(b))); }
|
||||
|
||||
template<size_t i> __forceinline const avxi shuffle( const avxi& a ) {
|
||||
return _mm256_castps_si256(_mm256_permute_ps(_mm256_castsi256_ps(a), _MM_SHUFFLE(i, i, i, i)));
|
||||
}
|
||||
|
||||
template<size_t i0, size_t i1> __forceinline const avxi shuffle( const avxi& a ) {
|
||||
return _mm256_permute2f128_si256(a, a, (i1 << 4) | (i0 << 0));
|
||||
}
|
||||
|
||||
template<size_t i0, size_t i1> __forceinline const avxi shuffle( const avxi& a, const avxi& b) {
|
||||
return _mm256_permute2f128_si256(a, b, (i1 << 4) | (i0 << 0));
|
||||
}
|
||||
|
||||
template<size_t i0, size_t i1, size_t i2, size_t i3> __forceinline const avxi shuffle( const avxi& a ) {
|
||||
return _mm256_castps_si256(_mm256_permute_ps(_mm256_castsi256_ps(a), _MM_SHUFFLE(i3, i2, i1, i0)));
|
||||
}
|
||||
|
||||
template<size_t i0, size_t i1, size_t i2, size_t i3> __forceinline const avxi shuffle( const avxi& a, const avxi& b ) {
|
||||
return _mm256_castps_si256(_mm256_shuffle_ps(_mm256_castsi256_ps(a), _mm256_castsi256_ps(b), _MM_SHUFFLE(i3, i2, i1, i0)));
|
||||
}
|
||||
|
||||
template<> __forceinline const avxi shuffle<0, 0, 2, 2>( const avxi& b ) { return _mm256_castps_si256(_mm256_moveldup_ps(_mm256_castsi256_ps(b))); }
|
||||
template<> __forceinline const avxi shuffle<1, 1, 3, 3>( const avxi& b ) { return _mm256_castps_si256(_mm256_movehdup_ps(_mm256_castsi256_ps(b))); }
|
||||
template<> __forceinline const avxi shuffle<0, 1, 0, 1>( const avxi& b ) { return _mm256_castps_si256(_mm256_castpd_ps(_mm256_movedup_pd(_mm256_castps_pd(_mm256_castsi256_ps(b))))); }
|
||||
|
||||
__forceinline const avxi broadcast(const int* ptr) { return _mm256_castps_si256(_mm256_broadcast_ss((const float*)ptr)); }
|
||||
template<size_t i> __forceinline const avxi insert (const avxi& a, const ssei& b) { return _mm256_insertf128_si256 (a,b,i); }
|
||||
template<size_t i> __forceinline const ssei extract(const avxi& a ) { return _mm256_extractf128_si256(a ,i); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Reductions
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const avxi vreduce_min2(const avxi& v) { return min(v,shuffle<1,0,3,2>(v)); }
|
||||
__forceinline const avxi vreduce_min4(const avxi& v) { avxi v1 = vreduce_min2(v); return min(v1,shuffle<2,3,0,1>(v1)); }
|
||||
__forceinline const avxi vreduce_min (const avxi& v) { avxi v1 = vreduce_min4(v); return min(v1,shuffle<1,0>(v1)); }
|
||||
|
||||
__forceinline const avxi vreduce_max2(const avxi& v) { return max(v,shuffle<1,0,3,2>(v)); }
|
||||
__forceinline const avxi vreduce_max4(const avxi& v) { avxi v1 = vreduce_max2(v); return max(v1,shuffle<2,3,0,1>(v1)); }
|
||||
__forceinline const avxi vreduce_max (const avxi& v) { avxi v1 = vreduce_max4(v); return max(v1,shuffle<1,0>(v1)); }
|
||||
|
||||
__forceinline const avxi vreduce_add2(const avxi& v) { return v + shuffle<1,0,3,2>(v); }
|
||||
__forceinline const avxi vreduce_add4(const avxi& v) { avxi v1 = vreduce_add2(v); return v1 + shuffle<2,3,0,1>(v1); }
|
||||
__forceinline const avxi vreduce_add (const avxi& v) { avxi v1 = vreduce_add4(v); return v1 + shuffle<1,0>(v1); }
|
||||
|
||||
__forceinline int reduce_min(const avxi& v) { return extract<0>(extract<0>(vreduce_min(v))); }
|
||||
__forceinline int reduce_max(const avxi& v) { return extract<0>(extract<0>(vreduce_max(v))); }
|
||||
__forceinline int reduce_add(const avxi& v) { return extract<0>(extract<0>(vreduce_add(v))); }
|
||||
|
||||
__forceinline size_t select_min(const avxi& v) { return __bsf(movemask(v == vreduce_min(v))); }
|
||||
__forceinline size_t select_max(const avxi& v) { return __bsf(movemask(v == vreduce_max(v))); }
|
||||
|
||||
__forceinline size_t select_min(const avxb& valid, const avxi& v) { const avxi a = select(valid,v,avxi(pos_inf)); return __bsf(movemask(valid & (a == vreduce_min(a)))); }
|
||||
__forceinline size_t select_max(const avxb& valid, const avxi& v) { const avxi a = select(valid,v,avxi(neg_inf)); return __bsf(movemask(valid & (a == vreduce_max(a)))); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// New stuff
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline void store8i(void* ptr, const avxi& i) {
|
||||
_mm256_store_ps((float*)ptr,_mm256_castsi256_ps(i));
|
||||
}
|
||||
|
||||
__forceinline void store8i( const avxb &mask, void *ptr, const avxi& i ) {
|
||||
_mm256_maskstore_ps((float*)ptr,(__m256i)mask,_mm256_castsi256_ps(i));
|
||||
}
|
||||
|
||||
__forceinline void store8i_nt(void* ptr, const avxi& v) {
|
||||
store4i_nt((int*)ptr+0,v.l);
|
||||
store4i_nt((int*)ptr+4,v.h);
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Output Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
inline std::ostream& operator<<(std::ostream& cout, const avxi& a) {
|
||||
return cout << "<" << a[0] << ", " << a[1] << ", " << a[2] << ", " << a[3] << ", " << a[4] << ", " << a[5] << ", " << a[6] << ", " << a[7] << ">";
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,140 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#pragma once
|
||||
|
||||
#include "sys/platform.h"
|
||||
#include "sys/intrinsics.h"
|
||||
#include "sse_mic.h"
|
||||
|
||||
#include <zmmintrin.h>
|
||||
|
||||
#define _MM_SHUF_PERM(e3, e2, e1, e0) \
|
||||
((_MM_PERM_ENUM)((e3)*64 + (e2)*16 + (e1)*4 + (e0)))
|
||||
|
||||
#define _MM_SHUF_PERM_NONE _MM_SHUF_PERM(3,2,1,0)
|
||||
|
||||
namespace embree
|
||||
{
|
||||
class mic_m;
|
||||
class mic_i;
|
||||
class mic_f;
|
||||
}
|
||||
|
||||
#include "mic_m.h"
|
||||
#include "mic_i.h"
|
||||
#include "mic_f.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Prefetching
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
#define PFHINT_L1 0
|
||||
#define PFHINT_L2 1
|
||||
#define PFHINT_NT 2
|
||||
#define PFHINT_L1EX 3
|
||||
#define PFHINT_L2EX 4
|
||||
#define PFHINT_NTEX 5
|
||||
|
||||
template<const unsigned int mode>
|
||||
__forceinline void prefetch(const void * __restrict__ const m)
|
||||
{
|
||||
if (mode == PFHINT_L1)
|
||||
_mm_prefetch((const char*)m,_MM_HINT_T0);
|
||||
else if (mode == PFHINT_L2)
|
||||
_mm_prefetch((const char*)m,_MM_HINT_T1);
|
||||
else if (mode == PFHINT_NT)
|
||||
_mm_prefetch((const char*)m,_MM_HINT_NTA);
|
||||
else if (mode == PFHINT_L1EX)
|
||||
_mm_prefetch((const char*)m,_MM_HINT_ET0);
|
||||
else if (mode == PFHINT_L2EX)
|
||||
_mm_prefetch((const char*)m,_MM_HINT_ET2);
|
||||
else if (mode == PFHINT_NTEX)
|
||||
_mm_prefetch((const char*)m,_MM_HINT_ENTA);
|
||||
}
|
||||
|
||||
__forceinline void gather_prefetch(const mic_m &m_active,
|
||||
const void *const ptr,
|
||||
const mic_i index,
|
||||
const int mode = _MM_HINT_T2,
|
||||
const _MM_INDEX_SCALE_ENUM scale = _MM_SCALE_4,
|
||||
const _MM_UPCONV_PS_ENUM up = _MM_UPCONV_PS_NONE)
|
||||
{
|
||||
_mm512_mask_prefetch_i32extgather_ps(index,m_active,ptr,up,scale,mode);
|
||||
}
|
||||
|
||||
__forceinline void scatter_prefetch(const mic_m &m_active,
|
||||
void *const ptr,
|
||||
const mic_i index,
|
||||
const int mode = _MM_HINT_ET2,
|
||||
const _MM_INDEX_SCALE_ENUM scale = _MM_SCALE_4,
|
||||
const _MM_UPCONV_PS_ENUM up = _MM_UPCONV_PS_NONE)
|
||||
{
|
||||
_mm512_mask_prefetch_i32extscatter_ps(ptr,m_active,index,up,scale,mode);
|
||||
}
|
||||
|
||||
__forceinline void evictL1(const void * __restrict__ m) {
|
||||
_mm_clevict(m,_MM_HINT_T0);
|
||||
}
|
||||
|
||||
__forceinline void evictL2(const void * __restrict__ m) {
|
||||
_mm_clevict(m,_MM_HINT_T1);
|
||||
}
|
||||
|
||||
|
||||
#if !defined(_MM_SHUF_PERM)
|
||||
#define _MM_SHUF_PERM(e3, e2, e1, e0) ((_MM_PERM_ENUM)((e3)*64 + (e2)*16 + (e1)*4 + (e0)))
|
||||
#define _MM_SHUF_PERM_NONE _MM_SHUF_PERM(3,2,1,0)
|
||||
#endif
|
||||
|
||||
template<const int D, const int C, const int B, const int A>
|
||||
__forceinline mic_f lshuf(const mic_f &in)
|
||||
{
|
||||
return _mm512_permute4f128_ps(in,(_MM_PERM_ENUM)_MM_SHUF_PERM(D,C,B,A));
|
||||
}
|
||||
|
||||
|
||||
template<const int D, const int C, const int B, const int A>
|
||||
__forceinline mic_f lshuf(const mic_m &mask, mic_f &dest, const mic_f &in)
|
||||
{
|
||||
return _mm512_mask_permute4f128_ps(dest,mask,in,(_MM_PERM_ENUM) _MM_SHUF_PERM(D,C,B,A));
|
||||
}
|
||||
|
||||
template<const int lane>
|
||||
__forceinline mic_f lane_shuffle_gather(const mic_f &v0,const mic_f &v1,const mic_f &v2,const mic_f &v3)
|
||||
{
|
||||
mic_f t = lshuf<lane,lane,lane,lane>(v0);
|
||||
t = lshuf<lane,lane,lane,lane>(0xf0,t,v1);
|
||||
t = lshuf<lane,lane,lane,lane>(0xf00,t,v2);
|
||||
t = lshuf<lane,lane,lane,lane>(0xf000,t,v3);
|
||||
return t;
|
||||
}
|
||||
|
||||
__forceinline mic_f convert(const ssef &v)
|
||||
{
|
||||
return broadcast4to16f(&v);
|
||||
}
|
||||
|
||||
__forceinline mic_i mul_uint64( const mic_i& a, const mic_i& b) {
|
||||
const mic_i low = _mm512_mullo_epi32(a, b);
|
||||
const mic_i high = _mm512_mulhi_epu32(a, b);
|
||||
return select(0x5555,low,high);
|
||||
}
|
||||
|
||||
|
||||
}
|
||||
@@ -1,5 +1,5 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
@@ -14,6 +14,4 @@
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#pragma once
|
||||
|
||||
#include "../../../common/math/math.h"
|
||||
#include "mic.h"
|
||||
@@ -0,0 +1,756 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#pragma once
|
||||
|
||||
namespace embree
|
||||
{
|
||||
/*! 16-wide MIC float type. */
|
||||
class mic_f
|
||||
{
|
||||
public:
|
||||
|
||||
union {
|
||||
__m512 v;
|
||||
float f[16];
|
||||
};
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constructors, Assignment & Cast Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline mic_f() {}
|
||||
__forceinline mic_f(const mic_f& t) { v = t; };
|
||||
__forceinline mic_f& operator=(const mic_f& f) { v = f.v; return *this; };
|
||||
|
||||
__forceinline mic_f(const __m512& t) { v = t; };
|
||||
__forceinline operator __m512 () const { return v; };
|
||||
|
||||
__forceinline mic_f(const float& f) {
|
||||
v = _mm512_set_1to16_ps(f);
|
||||
}
|
||||
__forceinline mic_f(const float& a, const float& b, const float& c, const float& d) {
|
||||
v = _mm512_set_4to16_ps(a,b,c,d);
|
||||
}
|
||||
|
||||
__forceinline explicit mic_f(const __m512i& a) {
|
||||
v = _mm512_cvtfxpnt_round_adjustepi32_ps(a, _MM_FROUND_NO_EXC,_MM_EXPADJ_NONE);
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constants
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline mic_f( ZeroTy ) : v(_mm512_setzero_ps()) {}
|
||||
__forceinline mic_f( OneTy ) : v(_mm512_set_1to16_ps(1.0f)) {}
|
||||
__forceinline mic_f( PosInfTy ) : v(_mm512_set_1to16_ps(pos_inf)) {}
|
||||
__forceinline mic_f( NegInfTy ) : v(_mm512_set_1to16_ps(neg_inf)) {}
|
||||
__forceinline mic_f( StepTy ) : v(_mm512_set_16to16_ps(15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,0)) {}
|
||||
__forceinline mic_f( NaNTy ) : v(_mm512_set_1to16_ps(nan)) {}
|
||||
|
||||
__forceinline static mic_f undefined() { return _mm512_undefined(); }
|
||||
__forceinline static mic_f zero() { return _mm512_setzero_ps(); }
|
||||
__forceinline static mic_f one () { return _mm512_set_1to16_ps(1.0f); }
|
||||
__forceinline static mic_f ulp () { return _mm512_set_1to16_ps(embree::ulp); }
|
||||
__forceinline static mic_f inf () { return _mm512_set_1to16_ps((float)pos_inf); }
|
||||
__forceinline static mic_f minus_inf () { return _mm512_set_1to16_ps((float)neg_inf); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Array Access
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline float& operator[](const size_t index) { return f[index]; };
|
||||
__forceinline const float& operator[](const size_t index) const { return f[index]; };
|
||||
};
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Unary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const mic_f cast (const __m512i& a) { return _mm512_castsi512_ps(a); }
|
||||
__forceinline const mic_f operator +( const mic_f& a ) { return a; }
|
||||
__forceinline const mic_f operator -( const mic_f& a ) { return _mm512_mul_ps(a,mic_f(-1)); }
|
||||
__forceinline const mic_f abs ( const mic_f& a ) { return _mm512_gmaxabs_ps(a,a); }
|
||||
__forceinline const mic_f signmsk ( const mic_f& a ) { return _mm512_castsi512_ps(_mm512_and_epi32(_mm512_castps_si512(a),_mm512_set1_epi32(0x80000000))); }
|
||||
|
||||
__forceinline const mic_f rcp ( const mic_f& a ) { return _mm512_rcp23_ps(a); };
|
||||
|
||||
__forceinline const mic_f sqr ( const mic_f& a ) { return _mm512_mul_ps(a,a); }
|
||||
__forceinline const mic_f sqrt ( const mic_f& a ) { return _mm512_sqrt_ps(a); }
|
||||
__forceinline const mic_f rsqrt( const mic_f& a ) { return _mm512_invsqrt_ps(a); }
|
||||
|
||||
|
||||
//__forceinline mic_f floor(const mic_f& a) { return _mm512_floor_ps(a); }
|
||||
//__forceinline mic_f ceil(const mic_f& a) { return _mm512_ceil_ps(a); }
|
||||
__forceinline mic_f trunc(const mic_f& a) { return _mm512_trunc_ps(a); }
|
||||
|
||||
__forceinline mic_f exp(const mic_f& a) { return _mm512_exp_ps(a); }
|
||||
__forceinline mic_f exp2(const mic_f& a) { return _mm512_exp2_ps(a); }
|
||||
__forceinline mic_f pow(const mic_f& a, mic_f b) { return _mm512_pow_ps(a,b); }
|
||||
|
||||
__forceinline mic_f log(const mic_f& a) { return _mm512_log_ps(a); }
|
||||
__forceinline mic_f log2(const mic_f& a) { return _mm512_log2_ps(a); }
|
||||
__forceinline mic_f log10(const mic_f& a) { return _mm512_log10_ps(a); }
|
||||
|
||||
__forceinline mic_f sin(const mic_f& a) { return _mm512_sin_ps(a); }
|
||||
__forceinline mic_f cos(const mic_f& a) { return _mm512_cos_ps(a); }
|
||||
__forceinline mic_f tan(const mic_f& a) { return _mm512_tan_ps(a); }
|
||||
|
||||
__forceinline mic_f asin(const mic_f& a) { return _mm512_asin_ps(a); }
|
||||
__forceinline mic_f acos(const mic_f& a) { return _mm512_acos_ps(a); }
|
||||
__forceinline mic_f atan(const mic_f& a) { return _mm512_atan_ps(a); }
|
||||
__forceinline mic_f atan2(const mic_f& a, mic_f b) { return _mm512_atan2_ps(a,b); }
|
||||
|
||||
__forceinline mic_f sinh(const mic_f& a) { return _mm512_sinh_ps(a); }
|
||||
__forceinline mic_f cosh(const mic_f& a) { return _mm512_cosh_ps(a); }
|
||||
__forceinline mic_f tanh(const mic_f& a) { return _mm512_tan_ps(a); }
|
||||
|
||||
__forceinline mic_f asinh(const mic_f& a) { return _mm512_asinh_ps(a); }
|
||||
__forceinline mic_f acosh(const mic_f& a) { return _mm512_acosh_ps(a); }
|
||||
__forceinline mic_f atanh(const mic_f& a) { return _mm512_atanh_ps(a); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Binary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const mic_f operator +( const mic_f& a, const mic_f& b ) { return _mm512_add_ps(a, b); }
|
||||
__forceinline const mic_f operator +( const mic_f& a, const float& b ) { return a + mic_f(b); }
|
||||
__forceinline const mic_f operator +( const float& a, const mic_f& b ) { return mic_f(a) + b; }
|
||||
|
||||
__forceinline const mic_f operator -( const mic_f& a, const mic_f& b ) { return _mm512_sub_ps(a, b); }
|
||||
__forceinline const mic_f operator -( const mic_f& a, const float& b ) { return a - mic_f(b); }
|
||||
__forceinline const mic_f operator -( const float& a, const mic_f& b ) { return mic_f(a) - b; }
|
||||
|
||||
__forceinline const mic_f operator *( const mic_f& a, const mic_f& b ) { return _mm512_mul_ps(a, b); }
|
||||
__forceinline const mic_f operator *( const mic_f& a, const float& b ) { return a * mic_f(b); }
|
||||
__forceinline const mic_f operator *( const float& a, const mic_f& b ) { return mic_f(a) * b; }
|
||||
|
||||
__forceinline const mic_f operator /( const mic_f& a, const mic_f& b ) { return _mm512_div_ps(a,b); }
|
||||
__forceinline const mic_f operator /( const mic_f& a, const float& b ) { return a/mic_f(b); }
|
||||
__forceinline const mic_f operator /( const float& a, const mic_f& b ) { return mic_f(a)/b; }
|
||||
|
||||
__forceinline const mic_f operator^(const mic_f& a, const mic_f& b) {
|
||||
return _mm512_castsi512_ps(_mm512_xor_epi32(_mm512_castps_si512(a),_mm512_castps_si512(b)));
|
||||
}
|
||||
|
||||
__forceinline const mic_f min( const mic_f& a, const mic_f& b ) { return _mm512_gmin_ps(a,b); }
|
||||
__forceinline const mic_f min( const mic_f& a, const float& b ) { return _mm512_gmin_ps(a,mic_f(b)); }
|
||||
__forceinline const mic_f min( const float& a, const mic_f& b ) { return _mm512_gmin_ps(mic_f(a),b); }
|
||||
|
||||
__forceinline const mic_f max( const mic_f& a, const mic_f& b ) { return _mm512_gmax_ps(a,b); }
|
||||
__forceinline const mic_f max( const mic_f& a, const float& b ) { return _mm512_gmax_ps(a,mic_f(b)); }
|
||||
__forceinline const mic_f max( const float& a, const mic_f& b ) { return _mm512_gmax_ps(mic_f(a),b); }
|
||||
|
||||
__forceinline mic_f mask_add(const mic_m& mask, const mic_f& c, const mic_f& a, const mic_f& b) { return _mm512_mask_add_ps (c,mask,a,b); };
|
||||
__forceinline mic_f mask_min(const mic_m& mask, const mic_f& c, const mic_f& a, const mic_f& b) { return _mm512_mask_gmin_ps(c,mask,a,b); };
|
||||
__forceinline mic_f mask_max(const mic_m& mask, const mic_f& c, const mic_f& a, const mic_f& b) { return _mm512_mask_gmax_ps(c,mask,a,b); };
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Ternary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline mic_f madd (const mic_f& a, const mic_f& b, const mic_f& c) { return _mm512_fmadd_ps(a,b,c); }
|
||||
__forceinline mic_f msub (const mic_f& a, const mic_f& b, const mic_f& c) { return _mm512_fmsub_ps(a,b,c); }
|
||||
__forceinline mic_f nmadd (const mic_f& a, const mic_f& b, const mic_f& c) { return _mm512_fnmadd_ps(a,b,c); }
|
||||
__forceinline mic_f nmsub (const mic_f& a, const mic_f& b, const mic_f& c) { return _mm512_fnmsub_ps(a,b,c); }
|
||||
|
||||
__forceinline mic_f mask_msub (const mic_m& mask,const mic_f& a, const mic_f& b, const mic_f& c) { return _mm512_mask_fmsub_ps(a,mask,b,c); }
|
||||
|
||||
__forceinline mic_f madd231 (const mic_f& a, const mic_f& b, const mic_f& c) { return _mm512_fmadd_ps(c,b,a); }
|
||||
__forceinline mic_f msub213 (const mic_f& a, const mic_f& b, const mic_f& c) { return _mm512_fmsub_ps(a,b,c); }
|
||||
__forceinline mic_f msub231 (const mic_f& a, const mic_f& b, const mic_f& c) { return _mm512_fmsub_ps(c,b,a); }
|
||||
__forceinline mic_f msubr231(const mic_f& a, const mic_f& b, const mic_f& c) { return _mm512_fnmadd_ps(c,b,a); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Assignment Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline mic_f& operator +=( mic_f& a, const mic_f& b ) { return a = a + b; }
|
||||
__forceinline mic_f& operator +=( mic_f& a, const float& b ) { return a = a + b; }
|
||||
|
||||
__forceinline mic_f& operator -=( mic_f& a, const mic_f& b ) { return a = a - b; }
|
||||
__forceinline mic_f& operator -=( mic_f& a, const float& b ) { return a = a - b; }
|
||||
|
||||
__forceinline mic_f& operator *=( mic_f& a, const mic_f& b ) { return a = a * b; }
|
||||
__forceinline mic_f& operator *=( mic_f& a, const float& b ) { return a = a * b; }
|
||||
|
||||
__forceinline mic_f& operator /=( mic_f& a, const mic_f& b ) { return a = a / b; }
|
||||
__forceinline mic_f& operator /=( mic_f& a, const float& b ) { return a = a / b; }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Comparison Operators + Select
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const mic_m operator ==( const mic_f& a, const mic_f& b ) { return _mm512_cmp_ps_mask(a,b,_MM_CMPINT_EQ); }
|
||||
__forceinline const mic_m operator ==( const mic_f& a, const float& b ) { return a == mic_f(b); }
|
||||
__forceinline const mic_m operator ==( const float& a, const mic_f& b ) { return mic_f(a) == b; }
|
||||
|
||||
__forceinline const mic_m operator !=( const mic_f& a, const mic_f& b ) { return _mm512_cmp_ps_mask(a,b,_MM_CMPINT_NE); }
|
||||
__forceinline const mic_m operator !=( const mic_f& a, const float& b ) { return a != mic_f(b); }
|
||||
__forceinline const mic_m operator !=( const float& a, const mic_f& b ) { return mic_f(a) != b; }
|
||||
|
||||
__forceinline const mic_m operator < ( const mic_f& a, const mic_f& b ) { return _mm512_cmp_ps_mask(a,b,_MM_CMPINT_LT); }
|
||||
__forceinline const mic_m operator < ( const mic_f& a, const float& b ) { return a < mic_f(b); }
|
||||
__forceinline const mic_m operator < ( const float& a, const mic_f& b ) { return mic_f(a) < b; }
|
||||
|
||||
__forceinline const mic_m operator >=( const mic_f& a, const mic_f& b ) { return _mm512_cmp_ps_mask(a,b,_MM_CMPINT_GE); }
|
||||
__forceinline const mic_m operator >=( const mic_f& a, const float& b ) { return a >= mic_f(b); }
|
||||
__forceinline const mic_m operator >=( const float& a, const mic_f& b ) { return mic_f(a) >= b; }
|
||||
|
||||
__forceinline const mic_m operator > ( const mic_f& a, const mic_f& b ) { return _mm512_cmp_ps_mask(a,b,_MM_CMPINT_GT); }
|
||||
__forceinline const mic_m operator > ( const mic_f& a, const float& b ) { return a > mic_f(b); }
|
||||
__forceinline const mic_m operator > ( const float& a, const mic_f& b ) { return mic_f(a) > b; }
|
||||
|
||||
__forceinline const mic_m operator <=( const mic_f& a, const mic_f& b ) { return _mm512_cmp_ps_mask(a,b,_MM_CMPINT_LE); }
|
||||
__forceinline const mic_m operator <=( const mic_f& a, const float& b ) { return a <= mic_f(b); }
|
||||
__forceinline const mic_m operator <=( const float& a, const mic_f& b ) { return mic_f(a) <= b; }
|
||||
|
||||
__forceinline mic_m eq( const mic_f& a, const mic_f& b) { return _mm512_cmp_ps_mask(a,b,_MM_CMPINT_EQ); }
|
||||
__forceinline mic_m eq(const mic_m& mask, const mic_f& a, const mic_f& b) { return _mm512_mask_cmp_ps_mask(mask,a,b,_MM_CMPINT_EQ); }
|
||||
|
||||
__forceinline mic_m ne( const mic_f& a, const mic_f& b) { return _mm512_cmp_ps_mask(a,b,_MM_CMPINT_NE); }
|
||||
__forceinline mic_m ne(const mic_m& mask, const mic_f& a, const mic_f& b) { return _mm512_mask_cmp_ps_mask(mask,a,b,_MM_CMPINT_NE); }
|
||||
|
||||
__forceinline mic_m lt( const mic_f& a, const mic_f& b) { return _mm512_cmp_ps_mask(a,b,_MM_CMPINT_LT); }
|
||||
__forceinline mic_m lt(const mic_m& mask, const mic_f& a, const mic_f& b) { return _mm512_mask_cmp_ps_mask(mask,a,b,_MM_CMPINT_LT); }
|
||||
|
||||
__forceinline mic_m ge( const mic_f& a, const mic_f& b) { return _mm512_cmp_ps_mask(a,b,_MM_CMPINT_GE); }
|
||||
__forceinline mic_m ge(const mic_m& mask, const mic_f& a, const mic_f& b) { return _mm512_mask_cmp_ps_mask(mask,a,b,_MM_CMPINT_GE); }
|
||||
|
||||
__forceinline mic_m gt( const mic_f& a, const mic_f& b) { return _mm512_cmp_ps_mask(a,b,_MM_CMPINT_GT); }
|
||||
__forceinline mic_m gt(const mic_m& mask, const mic_f& a, const mic_f& b) { return _mm512_mask_cmp_ps_mask(mask,a,b,_MM_CMPINT_GT); }
|
||||
|
||||
__forceinline mic_m le( const mic_f& a, const mic_f& b) { return _mm512_cmp_ps_mask(a,b,_MM_CMPINT_LE); }
|
||||
__forceinline mic_m le(const mic_m& mask, const mic_f& a, const mic_f& b) { return _mm512_mask_cmp_ps_mask(mask,a,b,_MM_CMPINT_LE); }
|
||||
|
||||
__forceinline const mic_f select( const mic_m& s, const mic_f& t, const mic_f& f ) {
|
||||
return _mm512_mask_blend_ps(s, f, t);
|
||||
}
|
||||
|
||||
|
||||
__forceinline void xchg(mic_m m, mic_f& a, mic_f& b)
|
||||
{
|
||||
mic_f c = a;
|
||||
a = select(m,b,a);
|
||||
b = select(m,c,b);
|
||||
}
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Rounding Functions
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline mic_f vround(const mic_f& f,
|
||||
const _MM_ROUND_MODE_ENUM mode,
|
||||
const _MM_EXP_ADJ_ENUM exp = _MM_EXPADJ_NONE)
|
||||
{
|
||||
return _mm512_round_ps(f,mode,exp);
|
||||
}
|
||||
|
||||
__forceinline mic_f floor(const mic_f& a) { return _mm512_round_ps(a,_MM_ROUND_MODE_DOWN, _MM_EXPADJ_NONE); }
|
||||
__forceinline mic_f ceil (const mic_f& a) { return _mm512_round_ps(a,_MM_ROUND_MODE_UP , _MM_EXPADJ_NONE); }
|
||||
|
||||
__forceinline const mic_f rcp_nr ( const mic_f& a ) {
|
||||
const mic_f ra = _mm512_rcp23_ps(a);
|
||||
return (ra+ra) - (ra * a * ra);
|
||||
};
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Movement/Shifting/Shuffling Functions
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline mic_f swizzle(const mic_f& x,_MM_SWIZZLE_ENUM perm32 ) { return _mm512_swizzle_ps(x,perm32); }
|
||||
__forceinline mic_f permute(const mic_f& x,_MM_PERM_ENUM perm128) { return _mm512_permute4f128_ps(x,perm128); }
|
||||
|
||||
template<int D, int C, int B, int A> __forceinline mic_f swizzle (const mic_f& v) { return _mm512_shuffle_ps(v,_MM_SHUF_PERM(D,C,B,A)); }
|
||||
template<int A> __forceinline mic_f swizzle (const mic_f& x) { return swizzle<A,A,A,A>(v); }
|
||||
template<> __forceinline mic_f swizzle<0>(const mic_f& x) { return swizzle(x,_MM_SWIZ_REG_AAAA); }
|
||||
template<> __forceinline mic_f swizzle<1>(const mic_f& x) { return swizzle(x,_MM_SWIZ_REG_BBBB); }
|
||||
template<> __forceinline mic_f swizzle<2>(const mic_f& x) { return swizzle(x,_MM_SWIZ_REG_CCCC); }
|
||||
template<> __forceinline mic_f swizzle<3>(const mic_f& x) { return swizzle(x,_MM_SWIZ_REG_DDDD); }
|
||||
|
||||
template<int D, int C, int B, int A> __forceinline mic_f permute(const mic_f& v) { return permute(v,_MM_SHUF_PERM(D,C,B,A)); }
|
||||
template<int A> __forceinline mic_f permute(const mic_f& x) { return permute<A,A,A,A>(x); }
|
||||
|
||||
__forceinline mic_f shuffle(const mic_f& x,_MM_PERM_ENUM perm128, _MM_SWIZZLE_ENUM perm32) { return swizzle(permute(x,perm128),perm32); }
|
||||
|
||||
__forceinline mic_f shuffle(const mic_m& mask, mic_f& v, const mic_f& x,_MM_PERM_ENUM perm128, _MM_SWIZZLE_ENUM perm32) {
|
||||
return _mm512_mask_swizzle_ps(_mm512_mask_permute4f128_ps(v,mask,x,perm128),mask,x,perm32);
|
||||
}
|
||||
|
||||
__forceinline mic_f swAAAA(const mic_f &x) {
|
||||
return swizzle(x,_MM_SWIZ_REG_AAAA);
|
||||
}
|
||||
|
||||
__forceinline mic_f swBBBB(const mic_f &x) {
|
||||
return swizzle(x,_MM_SWIZ_REG_BBBB);
|
||||
}
|
||||
|
||||
__forceinline mic_f swCCCC(const mic_f &x) {
|
||||
return swizzle(x,_MM_SWIZ_REG_CCCC);
|
||||
}
|
||||
|
||||
__forceinline mic_f swDDDD(const mic_f &x) {
|
||||
return swizzle(x,_MM_SWIZ_REG_DDDD);
|
||||
}
|
||||
|
||||
__forceinline mic_f _mm512_permutev_ps(__m512i index, mic_f v)
|
||||
{
|
||||
return _mm512_castsi512_ps(_mm512_permutev_epi32(index,_mm512_castps_si512(v)));
|
||||
}
|
||||
|
||||
template<int i>
|
||||
__forceinline mic_f align_shift_right(const mic_f &a, const mic_f &b)
|
||||
{
|
||||
return _mm512_castsi512_ps(_mm512_alignr_epi32(_mm512_castps_si512(a),_mm512_castps_si512(b),i));
|
||||
};
|
||||
|
||||
template<int i>
|
||||
__forceinline mic_f mask_align_shift_right(const mic_m &mask,mic_f &c,const mic_f &a, const mic_f &b)
|
||||
{
|
||||
return _mm512_castsi512_ps(_mm512_mask_alignr_epi32(_mm512_castps_si512(c),mask,_mm512_castps_si512(a),_mm512_castps_si512(b),i));
|
||||
};
|
||||
|
||||
__forceinline mic_f shl1_zero_extend(const mic_f &a)
|
||||
{
|
||||
mic_f z = mic_f::zero();
|
||||
return mask_align_shift_right<15>(0xfffe,z,a,a);
|
||||
}
|
||||
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Reductions
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline float reduce_add(mic_f a) { return _mm512_reduce_add_ps(a); }
|
||||
__forceinline float reduce_mul(mic_f a) { return _mm512_reduce_mul_ps(a); }
|
||||
__forceinline float reduce_min(mic_f a) { return _mm512_reduce_min_ps(a); }
|
||||
__forceinline float reduce_max(mic_f a) { return _mm512_reduce_max_ps(a); }
|
||||
|
||||
__forceinline mic_f vreduce_min2(mic_f x) { return min(x,swizzle(x,_MM_SWIZ_REG_BADC)); }
|
||||
__forceinline mic_f vreduce_min4(mic_f x) { x = vreduce_min2(x); return min(x,swizzle(x,_MM_SWIZ_REG_CDAB)); }
|
||||
__forceinline mic_f vreduce_min8(mic_f x) { x = vreduce_min4(x); return min(x,permute(x,_MM_SHUF_PERM(2,3,0,1))); }
|
||||
__forceinline mic_f vreduce_min (mic_f x) { x = vreduce_min8(x); return min(x,permute(x,_MM_SHUF_PERM(1,0,3,2))); }
|
||||
|
||||
__forceinline mic_f vreduce_max2(mic_f x) { return max(x,swizzle(x,_MM_SWIZ_REG_BADC)); }
|
||||
__forceinline mic_f vreduce_max4(mic_f x) { x = vreduce_max2(x); return max(x,swizzle(x,_MM_SWIZ_REG_CDAB)); }
|
||||
__forceinline mic_f vreduce_max8(mic_f x) { x = vreduce_max4(x); return max(x,permute(x,_MM_SHUF_PERM(2,3,0,1))); }
|
||||
__forceinline mic_f vreduce_max (mic_f x) { x = vreduce_max8(x); return max(x,permute(x,_MM_SHUF_PERM(1,0,3,2))); }
|
||||
|
||||
__forceinline mic_f vreduce_add2(mic_f x) { return x + swizzle(x,_MM_SWIZ_REG_BADC); }
|
||||
__forceinline mic_f vreduce_add4(mic_f x) { x = vreduce_add2(x); return x + swizzle(x,_MM_SWIZ_REG_CDAB); }
|
||||
__forceinline mic_f vreduce_add8(mic_f x) { x = vreduce_add4(x); return x + permute(x,_MM_SHUF_PERM(2,3,0,1)); }
|
||||
__forceinline mic_f vreduce_add (mic_f x) { x = vreduce_add8(x); return x + permute(x,_MM_SHUF_PERM(1,0,3,2)); }
|
||||
|
||||
__forceinline size_t select_min(const mic_f& v) { return __bsf(movemask(v == vreduce_min(v))); }
|
||||
__forceinline size_t select_max(const mic_f& v) { return __bsf(movemask(v == vreduce_max(v))); }
|
||||
|
||||
__forceinline size_t select_min(const mic_m& valid, const mic_f& v, const mic_f &max_value) { const mic_f a = select(valid,v,max_value); return __bsf(movemask(a == vreduce_min(a))); }
|
||||
|
||||
__forceinline size_t select_max(const mic_m& valid, const mic_f& v, const mic_f &min_value) { const mic_f a = select(valid,v,min_value); return __bsf(movemask(a == vreduce_max(a))); }
|
||||
|
||||
__forceinline size_t select_max(const mic_m& valid, const mic_f& v) { const mic_f a = select(valid,v,mic_f(neg_inf)); return __bsf(movemask(valid & (a == vreduce_max(a)))); }
|
||||
|
||||
__forceinline size_t select_min(const mic_m& valid, const mic_f& v) { const mic_f a = select(valid,v,mic_f(pos_inf)); return __bsf(movemask(valid & (a == vreduce_min(a)))); }
|
||||
|
||||
__forceinline mic_f prefix_sum(const mic_f& a)
|
||||
{
|
||||
mic_f v = a;
|
||||
v = mask_add(0xaaaa,v,v,swizzle(v,_MM_SWIZ_REG_CDAB));
|
||||
v = mask_add(0xcccc,v,v,swizzle(v,_MM_SWIZ_REG_BBBB));
|
||||
const mic_f shuf_v0 = shuffle(v,_MM_SHUF_PERM(2,2,0,0),_MM_SWIZ_REG_DDDD);
|
||||
v = mask_add(0xf0f0,v,v,shuf_v0);
|
||||
const mic_f shuf_v1 = shuffle(v,_MM_SHUF_PERM(1,1,0,0),_MM_SWIZ_REG_DDDD);
|
||||
v = mask_add(0xff00,v,v,shuf_v1);
|
||||
return v;
|
||||
}
|
||||
|
||||
__forceinline mic_f prefix_min(const mic_f& a)
|
||||
{
|
||||
mic_f v = a;
|
||||
v = mask_min(0xaaaa,v,v,swizzle(v,_MM_SWIZ_REG_CDAB));
|
||||
v = mask_min(0xcccc,v,v,swizzle(v,_MM_SWIZ_REG_BBBB));
|
||||
const mic_f shuf_v0 = shuffle(v,(_MM_PERM_ENUM)_MM_SHUF_PERM(2,2,0,0),_MM_SWIZ_REG_DDDD);
|
||||
v = mask_min(0xf0f0,v,v,shuf_v0);
|
||||
const mic_f shuf_v1 = shuffle(v,(_MM_PERM_ENUM)_MM_SHUF_PERM(1,1,0,0),_MM_SWIZ_REG_DDDD);
|
||||
v = mask_min(0xff00,v,v,shuf_v1);
|
||||
return v;
|
||||
}
|
||||
|
||||
__forceinline mic_f prefix_max(const mic_f& a)
|
||||
{
|
||||
mic_f v = a;
|
||||
v = mask_max(0xaaaa,v,v,swizzle(v,_MM_SWIZ_REG_CDAB));
|
||||
v = mask_max(0xcccc,v,v,swizzle(v,_MM_SWIZ_REG_BBBB));
|
||||
const mic_f shuf_v0 = shuffle(v,(_MM_PERM_ENUM)_MM_SHUF_PERM(2,2,0,0),_MM_SWIZ_REG_DDDD);
|
||||
v = mask_max(0xf0f0,v,v,shuf_v0);
|
||||
const mic_f shuf_v1 = shuffle(v,(_MM_PERM_ENUM)_MM_SHUF_PERM(1,1,0,0),_MM_SWIZ_REG_DDDD);
|
||||
v = mask_max(0xff00,v,v,shuf_v1);
|
||||
return v;
|
||||
}
|
||||
|
||||
__forceinline mic_f set_min4(mic_f x) {
|
||||
x = min(x,swizzle(x,_MM_SWIZ_REG_BADC));
|
||||
x = min(x,swizzle(x,_MM_SWIZ_REG_CDAB));
|
||||
return x;
|
||||
}
|
||||
|
||||
__forceinline mic_f set_min_lanes(mic_f x) {
|
||||
x = min(x,_mm512_permute4f128_ps(x, _MM_PERM_CDAB));
|
||||
x = min(x,_mm512_permute4f128_ps(x, _MM_PERM_BADC));
|
||||
return x;
|
||||
}
|
||||
|
||||
__forceinline mic_f set_max_lanes(mic_f x) {
|
||||
x = max(x,_mm512_permute4f128_ps(x, _MM_PERM_CDAB));
|
||||
x = max(x,_mm512_permute4f128_ps(x, _MM_PERM_BADC));
|
||||
return x;
|
||||
}
|
||||
|
||||
__forceinline mic_f set_min16(mic_f x) {
|
||||
return set_min_lanes(set_min4(x));
|
||||
}
|
||||
|
||||
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Memory load and store operations
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline mic_f load1f(const void *f) {
|
||||
return _mm512_extload_ps(f,_MM_UPCONV_PS_NONE,_MM_BROADCAST_1X16,_MM_HINT_NONE);
|
||||
}
|
||||
|
||||
__forceinline mic_f load16f(const void *f) {
|
||||
return _mm512_extload_ps(f,_MM_UPCONV_PS_NONE,_MM_BROADCAST_16X16,_MM_HINT_NONE);
|
||||
}
|
||||
|
||||
__forceinline mic_f load16f(const mic_f d, const mic_m valid, const void *f) {
|
||||
return _mm512_mask_extload_ps(d,valid,f,_MM_UPCONV_PS_NONE,_MM_BROADCAST_16X16,_MM_HINT_NONE);
|
||||
}
|
||||
|
||||
__forceinline mic_f broadcast1to16f(const void *f) {
|
||||
return _mm512_extload_ps(f,_MM_UPCONV_PS_NONE,_MM_BROADCAST_1X16,_MM_HINT_NONE);
|
||||
}
|
||||
|
||||
__forceinline mic_f broadcast4to16f(const void *f) {
|
||||
return _mm512_extload_ps(f,_MM_UPCONV_PS_NONE,_MM_BROADCAST_4X16,0);
|
||||
}
|
||||
|
||||
__forceinline mic_f broadcast8to16f(const void *f) {
|
||||
return _mm512_castpd_ps(_mm512_extload_pd(f,_MM_UPCONV_PD_NONE,_MM_BROADCAST_4X8,0));
|
||||
}
|
||||
|
||||
__forceinline mic_f load16f_uint8(const unsigned char *const ptr) {
|
||||
return _mm512_mul_ps(_mm512_extload_ps(ptr,_MM_UPCONV_PS_UINT8,_MM_BROADCAST_16X16,_MM_HINT_NONE),mic_f(1.0f/255.0f));
|
||||
}
|
||||
|
||||
__forceinline mic_f uload16f_low_uint8(const mic_m& mask, const void* addr, const mic_f& v1) {
|
||||
return _mm512_mask_extloadunpacklo_ps(v1, mask, addr, _MM_UPCONV_PS_UINT8, _MM_HINT_NONE);
|
||||
}
|
||||
|
||||
__forceinline mic_f load16f_int8(const char *const ptr) {
|
||||
return _mm512_mul_ps(_mm512_extload_ps(ptr,_MM_UPCONV_PS_SINT8,_MM_BROADCAST_16X16,_MM_HINT_NONE),mic_f(1.0f/127.0f));
|
||||
}
|
||||
|
||||
|
||||
__forceinline mic_f gather16f_4f(const float *__restrict__ const ptr0,
|
||||
const float *__restrict__ const ptr1,
|
||||
const float *__restrict__ const ptr2,
|
||||
const float *__restrict__ const ptr3)
|
||||
{
|
||||
mic_f v = broadcast4to16f(ptr0);
|
||||
v = select((mic_m)0x00f0,broadcast4to16f(ptr1),v);
|
||||
v = select((mic_m)0x0f00,broadcast4to16f(ptr2),v);
|
||||
v = select((mic_m)0xf000,broadcast4to16f(ptr3),v);
|
||||
return v;
|
||||
}
|
||||
|
||||
__forceinline mic_f gather16f_4f(const mic_f& v0,
|
||||
const mic_f& v1,
|
||||
const mic_f& v2,
|
||||
const mic_f& v3)
|
||||
{
|
||||
mic_f v = v0;
|
||||
v = select((mic_m)0xf0 ,v1,v);
|
||||
v = select((mic_m)0xf00 ,v2,v);
|
||||
v = select((mic_m)0xf000,v3,v);
|
||||
return v;
|
||||
}
|
||||
|
||||
__forceinline mic_f gather_4f_zlc(const mic_i &v_mask,
|
||||
const void *__restrict__ const ptr0,
|
||||
const void *__restrict__ const ptr1,
|
||||
const void *__restrict__ const ptr2,
|
||||
const void *__restrict__ const ptr3)
|
||||
{
|
||||
const mic_m m_00f0 = 0x00f0;
|
||||
const mic_m m_0f00 = 0x0f00;
|
||||
const mic_m m_f000 = 0xf000;
|
||||
|
||||
mic_i v = v_mask & broadcast4to16i((const int*)ptr0);
|
||||
v = mask_and(m_00f0,v,v_mask, broadcast4to16i((const int*)ptr1));
|
||||
v = mask_and(m_0f00,v,v_mask, broadcast4to16i((const int*)ptr2));
|
||||
v = mask_and(m_f000,v,v_mask, broadcast4to16i((const int*)ptr3));
|
||||
return cast(v);
|
||||
}
|
||||
|
||||
__forceinline mic_f gather_2f_zlc(const mic_i &v_mask,
|
||||
const mic_m &mask,
|
||||
const void *__restrict__ const ptr0,
|
||||
const void *__restrict__ const ptr1)
|
||||
{
|
||||
mic_i v = v_mask & broadcast4to16i((const int*)ptr0);
|
||||
v = mask_and(mask,v,v_mask, broadcast4to16i((const int*)ptr1));
|
||||
return cast(v);
|
||||
}
|
||||
|
||||
|
||||
__forceinline mic_f gather16f_4f_align(const void *__restrict__ const ptr0,
|
||||
const void *__restrict__ const ptr1,
|
||||
const void *__restrict__ const ptr2,
|
||||
const void *__restrict__ const ptr3)
|
||||
{
|
||||
mic_f v = broadcast4to16f(ptr3);
|
||||
v = align_shift_right<12>(v,broadcast4to16f(ptr2));
|
||||
v = align_shift_right<12>(v,broadcast4to16f(ptr1));
|
||||
v = align_shift_right<12>(v,broadcast4to16f(ptr0));
|
||||
return v;
|
||||
}
|
||||
|
||||
|
||||
__forceinline mic_f gather16f_4f_align(const mic_f& v0,
|
||||
const mic_f& v1,
|
||||
const mic_f& v2,
|
||||
const mic_f& v3)
|
||||
{
|
||||
mic_f v = v3;
|
||||
v = align_shift_right<12>(v,v2);
|
||||
v = align_shift_right<12>(v,v1);
|
||||
v = align_shift_right<12>(v,v0);
|
||||
return v;
|
||||
}
|
||||
|
||||
__forceinline mic_f gather_4f_zlc_align(const mic_i &v_mask,
|
||||
const void *__restrict__ const ptr0,
|
||||
const void *__restrict__ const ptr1,
|
||||
const void *__restrict__ const ptr2,
|
||||
const void *__restrict__ const ptr3)
|
||||
{
|
||||
mic_f v = gather16f_4f_align(ptr0,ptr1,ptr2,ptr3);
|
||||
return cast(cast(v) & v_mask);
|
||||
}
|
||||
|
||||
|
||||
__forceinline mic_f uload16f(const float *const addr) {
|
||||
mic_f r = mic_f::undefined();
|
||||
r =_mm512_extloadunpacklo_ps(r, addr, _MM_UPCONV_PS_NONE, _MM_HINT_NONE);
|
||||
return _mm512_extloadunpackhi_ps(r, addr+16, _MM_UPCONV_PS_NONE, _MM_HINT_NONE);
|
||||
}
|
||||
|
||||
__forceinline mic_f uload16f_low(const float *const addr) {
|
||||
mic_f r = mic_f::undefined();
|
||||
return _mm512_extloadunpacklo_ps(r, addr, _MM_UPCONV_PS_NONE, _MM_HINT_NONE);
|
||||
}
|
||||
|
||||
__forceinline mic_f uload16f_low(const mic_m& mask, const void* addr, const mic_f& v1) {
|
||||
return _mm512_mask_extloadunpacklo_ps(v1, mask, addr, _MM_UPCONV_PS_NONE, _MM_HINT_NONE);
|
||||
}
|
||||
|
||||
__forceinline void ustore16f(float *addr, const mic_f& reg) {
|
||||
_mm512_extpackstorelo_ps(addr+0 ,reg, _MM_DOWNCONV_PS_NONE , 0);
|
||||
_mm512_extpackstorehi_ps(addr+16 ,reg, _MM_DOWNCONV_PS_NONE , 0);
|
||||
}
|
||||
|
||||
/* pass by value to avoid compiler generating inefficient code */
|
||||
__forceinline void compactustore16f(const mic_m& mask, float *addr, const mic_f reg) {
|
||||
_mm512_mask_extpackstorelo_ps(addr+0 ,mask, reg, _MM_DOWNCONV_PS_NONE , 0);
|
||||
_mm512_mask_extpackstorehi_ps(addr+16 ,mask, reg, _MM_DOWNCONV_PS_NONE , 0);
|
||||
}
|
||||
|
||||
__forceinline void compactustore16f_low(const mic_m& mask, float * addr, const mic_f ®) {
|
||||
_mm512_mask_extpackstorelo_ps(addr+0 ,mask, reg, _MM_DOWNCONV_PS_NONE , 0);
|
||||
}
|
||||
|
||||
__forceinline void compactustore16f_low_uint8(const mic_m& mask, void * addr, const mic_f ®) {
|
||||
_mm512_mask_extpackstorelo_ps(addr+0 ,mask, reg, _MM_DOWNCONV_PS_UINT8 , 0);
|
||||
}
|
||||
|
||||
__forceinline void ustore16f_low(float * addr, const mic_f& reg) {
|
||||
_mm512_extpackstorelo_ps(addr+0 ,reg, _MM_DOWNCONV_PS_NONE , 0);
|
||||
}
|
||||
|
||||
__forceinline void compactustore16f_high(const mic_m& mask, float *addr, const mic_f& reg) {
|
||||
_mm512_extpackstorehi_ps(addr+0 ,reg, _MM_DOWNCONV_PS_NONE , 0);
|
||||
}
|
||||
|
||||
__forceinline void store16f(const mic_m& mask, void* addr, const mic_f& v2) {
|
||||
_mm512_mask_extstore_ps(addr,mask,v2,_MM_DOWNCONV_PS_NONE,0);
|
||||
}
|
||||
|
||||
__forceinline void store16f(void* addr, const mic_f& v2) {
|
||||
_mm512_extstore_ps(addr,v2,_MM_DOWNCONV_PS_NONE,0);
|
||||
}
|
||||
|
||||
__forceinline void store16f_int8(void* addr, const mic_f& v2) {
|
||||
_mm512_extstore_ps(addr,v2,_MM_DOWNCONV_PS_SINT8,0);
|
||||
}
|
||||
|
||||
__forceinline void store4f_int8(void* addr, const mic_f& v1) {
|
||||
assert((unsigned long)addr % 4 == 0);
|
||||
_mm512_mask_extpackstorelo_ps(addr,0xf, v1, _MM_DOWNCONV_PS_SINT8 , 0);
|
||||
}
|
||||
|
||||
__forceinline void store4f(void* addr, const mic_f& v1) {
|
||||
assert((unsigned long)addr % 16 == 0);
|
||||
_mm512_mask_extpackstorelo_ps(addr,0xf, v1, _MM_DOWNCONV_PS_NONE , 0);
|
||||
}
|
||||
|
||||
__forceinline void store3f(void* addr, const mic_f& v1) {
|
||||
assert((unsigned long)addr % 16 == 0);
|
||||
_mm512_mask_extpackstorelo_ps(addr,0x7, v1, _MM_DOWNCONV_PS_NONE , 0);
|
||||
}
|
||||
|
||||
__forceinline void store4f_nt(void* addr, const mic_f& v1) {
|
||||
assert((unsigned long)addr % 16 == 0);
|
||||
_mm512_mask_extpackstorelo_ps(addr,0xf, v1, _MM_DOWNCONV_PS_NONE , _MM_HINT_NT);
|
||||
}
|
||||
|
||||
__forceinline void store16f_nt(void *__restrict__ ptr, const mic_f& a) {
|
||||
_mm512_storenr_ps(ptr,a);
|
||||
}
|
||||
|
||||
__forceinline void store16f_ngo(void *__restrict__ ptr, const mic_f& a) {
|
||||
_mm512_storenrngo_ps(ptr,a);
|
||||
}
|
||||
|
||||
__forceinline void store1f(void *addr, const mic_f& reg) {
|
||||
_mm512_mask_extpackstorelo_ps((float*)addr+0 ,(mic_m)1, reg, _MM_DOWNCONV_PS_NONE, _MM_HINT_NONE);
|
||||
}
|
||||
|
||||
__forceinline mic_f gather16f(const mic_m& mask, const float *const ptr, __m512i index, const _MM_INDEX_SCALE_ENUM scale) {
|
||||
mic_f r = mic_f::undefined();
|
||||
return _mm512_mask_i32extgather_ps(r,mask,index,ptr,_MM_UPCONV_PS_NONE,scale,0);
|
||||
}
|
||||
|
||||
__forceinline void scatter16f(const mic_m& mask,const float *const ptr, const __m512i index,const mic_f v, const _MM_INDEX_SCALE_ENUM scale) {
|
||||
_mm512_mask_i32extscatter_ps((void*)ptr,mask,index,v,_MM_DOWNCONV_PS_NONE,scale,0);
|
||||
}
|
||||
|
||||
__forceinline mic_f loadAOS4to16f(const float& x,const float& y, const float& z)
|
||||
{
|
||||
mic_f f = mic_f::zero();
|
||||
f = select(0x1111,broadcast1to16f(&x),f);
|
||||
f = select(0x2222,broadcast1to16f(&y),f);
|
||||
f = select(0x4444,broadcast1to16f(&z),f);
|
||||
return f;
|
||||
}
|
||||
|
||||
__forceinline mic_f loadAOS4to16f(const unsigned int index,
|
||||
const mic_f &x,
|
||||
const mic_f &y,
|
||||
const mic_f &z)
|
||||
{
|
||||
mic_f f = mic_f::zero();
|
||||
f = select(0x1111,broadcast1to16f((float*)&x + index),f);
|
||||
f = select(0x2222,broadcast1to16f((float*)&y + index),f);
|
||||
f = select(0x4444,broadcast1to16f((float*)&z + index),f);
|
||||
return f;
|
||||
}
|
||||
|
||||
__forceinline mic_f loadAOS4to16f(const unsigned int index,
|
||||
const mic_f &x,
|
||||
const mic_f &y,
|
||||
const mic_f &z,
|
||||
const mic_f &fill)
|
||||
{
|
||||
mic_f f = fill;
|
||||
f = select(0x1111,broadcast1to16f((float*)&x + index),f);
|
||||
f = select(0x2222,broadcast1to16f((float*)&y + index),f);
|
||||
f = select(0x4444,broadcast1to16f((float*)&z + index),f);
|
||||
return f;
|
||||
}
|
||||
|
||||
__forceinline mic_f rcp_safe( const mic_f& a ) { return select(a != mic_f::zero(),_mm512_rcp23_ps(a),mic_f(1E-10f)); };
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Euclidian Space Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline mic_f lcross_zxy(const mic_f &ao, const mic_f &bo) {
|
||||
mic_f ao_bo = bo * swizzle(ao,_MM_SWIZ_REG_DACB);
|
||||
ao_bo = msub231(ao_bo,ao,swizzle(bo,_MM_SWIZ_REG_DACB));
|
||||
return ao_bo;
|
||||
}
|
||||
|
||||
__forceinline mic_f ldot16_zxy(const mic_f &a,const mic_f &v0, const mic_f &v1, const mic_f &v2)
|
||||
{
|
||||
mic_f v = v0 * swizzle(a,_MM_SWIZ_REG_BBBB);
|
||||
v = madd231(v,v1,swizzle(a,_MM_SWIZ_REG_CCCC));
|
||||
v = madd231(v,v2,swizzle(a,_MM_SWIZ_REG_AAAA));
|
||||
return v;
|
||||
}
|
||||
|
||||
__forceinline mic_f ldot16_xyz(const mic_f &a,const mic_f &v0, const mic_f &v1, const mic_f &v2)
|
||||
{
|
||||
mic_f v = v0 * swizzle(a,_MM_SWIZ_REG_AAAA);
|
||||
v = madd231(v, v1,swizzle(a,_MM_SWIZ_REG_BBBB));
|
||||
v = madd231(v, v2,swizzle(a,_MM_SWIZ_REG_CCCC));
|
||||
return v;
|
||||
}
|
||||
|
||||
__forceinline mic_f lcross_xyz(const mic_f &a, const mic_f &b)
|
||||
{
|
||||
mic_f c = b * swizzle(a,_MM_SWIZ_REG_DACB);
|
||||
c = msub231(c,a,swizzle(b,_MM_SWIZ_REG_DACB));
|
||||
c = swizzle(c,_MM_SWIZ_REG_DACB);
|
||||
return c;
|
||||
}
|
||||
|
||||
__forceinline mic_f ldot3_xyz(const mic_f &ao, const mic_f &normal)
|
||||
{
|
||||
mic_f vv = ao * normal;
|
||||
vv = _mm512_add_ps(vv,swizzle(vv,_MM_SWIZ_REG_CDAB));
|
||||
vv = _mm512_add_ps(vv,swizzle(vv,_MM_SWIZ_REG_BADC));
|
||||
return vv;
|
||||
}
|
||||
|
||||
__forceinline mic_f ldot3_xyz(const mic_m &m_mask, const mic_f &ao, const mic_f &normal)
|
||||
{
|
||||
mic_f vv = _mm512_mask_mul_ps(ao,m_mask,ao,normal);
|
||||
vv = _mm512_add_ps(vv,swizzle(vv,_MM_SWIZ_REG_CDAB));
|
||||
vv = _mm512_add_ps(vv,swizzle(vv,_MM_SWIZ_REG_BADC));
|
||||
return vv;
|
||||
}
|
||||
|
||||
__forceinline mic_f ldot3_zxy(const mic_f &ao, const mic_f &normal)
|
||||
{
|
||||
mic_f vv = ao * swizzle(normal,_MM_SWIZ_REG_DACB);
|
||||
vv = _mm512_add_ps(vv,swizzle(vv,_MM_SWIZ_REG_CDAB));
|
||||
vv = _mm512_add_ps(vv,swizzle(vv,_MM_SWIZ_REG_BADC));
|
||||
return vv;
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Output Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline std::ostream &operator<<(std::ostream& cout, const mic_f& v)
|
||||
{
|
||||
cout << "<" << v[0];
|
||||
for (int i=1; i<16; i++) cout << ", " << v[i];
|
||||
cout << ">";
|
||||
return cout;
|
||||
}
|
||||
}
|
||||
@@ -1,5 +1,5 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
@@ -14,7 +14,4 @@
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#pragma once
|
||||
|
||||
#include "../../../common/math/linearspace3.h"
|
||||
|
||||
#include "mic.h"
|
||||
@@ -0,0 +1,482 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#pragma once
|
||||
|
||||
namespace embree
|
||||
{
|
||||
/*! 16-wide MIC integer type. */
|
||||
class mic_i
|
||||
{
|
||||
public:
|
||||
|
||||
union {
|
||||
__m512i v;
|
||||
int32 i[16];
|
||||
};
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constructors, Assignment & Cast Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline mic_i() {};
|
||||
__forceinline mic_i(const mic_i& t) { v = t.v; };
|
||||
__forceinline mic_i& operator=(const mic_i& f) { v = f.v; return *this; };
|
||||
|
||||
__forceinline mic_i(const __m512i& t) { v = t; };
|
||||
__forceinline operator __m512i () const { return v; };
|
||||
|
||||
__forceinline mic_i(const int& i) {
|
||||
v = _mm512_set_1to16_epi32(i);
|
||||
}
|
||||
|
||||
__forceinline mic_i(const int& a, const int& b, const int& c, const int& d) {
|
||||
v = _mm512_set_4to16_epi32(a,b,c,d);
|
||||
}
|
||||
|
||||
__forceinline explicit mic_i(const __m512 f) {
|
||||
v = _mm512_cvtfxpnt_round_adjustps_epi32(f,_MM_FROUND_FLOOR,_MM_EXPADJ_NONE);
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constants
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline mic_i( ZeroTy ) : v(_mm512_setzero_epi32()) {}
|
||||
__forceinline mic_i( OneTy ) : v(_mm512_set_1to16_epi32(1)) {}
|
||||
__forceinline mic_i( PosInfTy ) : v(_mm512_set_1to16_epi32(pos_inf)) {}
|
||||
__forceinline mic_i( NegInfTy ) : v(_mm512_set_1to16_epi32(neg_inf)) {}
|
||||
__forceinline mic_i( StepTy ) : v(_mm512_set_16to16_epi32(15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,0)) {}
|
||||
|
||||
__forceinline static mic_i zero() { return _mm512_setzero_epi32(); }
|
||||
__forceinline static mic_i one () { return _mm512_set_1to16_epi32(1); }
|
||||
__forceinline static mic_i neg_one () { return _mm512_set_1to16_epi32(-1); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Array Access
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline int& operator[](const size_t index) { return i[index]; };
|
||||
__forceinline const int& operator[](const size_t index) const { return i[index]; };
|
||||
|
||||
__forceinline unsigned int& uint(const size_t index) const { assert(index < 16); return ((unsigned int*)i)[index]; };
|
||||
__forceinline size_t& uint64(const size_t index) const { assert(index < 8); return ((size_t*)i)[index]; };
|
||||
|
||||
|
||||
};
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Unary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const mic_i cast ( const __m512& a) { return _mm512_castps_si512(a); }
|
||||
__forceinline const mic_i operator +( const mic_i& a ) { return a; }
|
||||
__forceinline const mic_i operator -( const mic_i& a ) { return _mm512_sub_epi32(_mm512_setzero_epi32(), a); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Binary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const mic_i operator +( const mic_i& a, const mic_i& b ) { return _mm512_add_epi32(a, b); }
|
||||
__forceinline const mic_i operator +( const mic_i& a, const int32& b ) { return a + mic_i(b); }
|
||||
__forceinline const mic_i operator +( const int32& a, const mic_i& b ) { return mic_i(a) + b; }
|
||||
|
||||
__forceinline const mic_i operator -( const mic_i& a, const mic_i& b ) { return _mm512_sub_epi32(a, b); }
|
||||
__forceinline const mic_i operator -( const mic_i& a, const int32& b ) { return a - mic_i(b); }
|
||||
__forceinline const mic_i operator -( const int32& a, const mic_i& b ) { return mic_i(a) - b; }
|
||||
|
||||
__forceinline const mic_i operator *( const mic_i& a, const mic_i& b ) { return _mm512_mullo_epi32(a, b); }
|
||||
__forceinline const mic_i operator *( const mic_i& a, const int32& b ) { return a * mic_i(b); }
|
||||
__forceinline const mic_i operator *( const int32& a, const mic_i& b ) { return mic_i(a) * b; }
|
||||
|
||||
__forceinline const mic_i operator &( const mic_i& a, const mic_i& b ) { return _mm512_and_epi32(a, b); }
|
||||
__forceinline const mic_i operator &( const mic_i& a, const int32& b ) { return a & mic_i(b); }
|
||||
__forceinline const mic_i operator &( const int32& a, const mic_i& b ) { return mic_i(a) & b; }
|
||||
|
||||
__forceinline const mic_i operator |( const mic_i& a, const mic_i& b ) { return _mm512_or_epi32(a, b); }
|
||||
__forceinline const mic_i operator |( const mic_i& a, const int32& b ) { return a | mic_i(b); }
|
||||
__forceinline const mic_i operator |( const int32& a, const mic_i& b ) { return mic_i(a) | b; }
|
||||
|
||||
__forceinline const mic_i operator ^( const mic_i& a, const mic_i& b ) { return _mm512_xor_epi32(a, b); }
|
||||
__forceinline const mic_i operator ^( const mic_i& a, const int32& b ) { return a ^ mic_i(b); }
|
||||
__forceinline const mic_i operator ^( const int32& a, const mic_i& b ) { return mic_i(a) ^ b; }
|
||||
|
||||
__forceinline const mic_i operator <<( const mic_i& a, const int32& n ) { return _mm512_slli_epi32(a, n); }
|
||||
__forceinline const mic_i operator >>( const mic_i& a, const int32& n ) { return _mm512_srai_epi32(a, n); }
|
||||
|
||||
__forceinline const mic_i operator <<( const mic_i& a, const mic_i& n ) { return _mm512_sllv_epi32(a, n); }
|
||||
__forceinline const mic_i operator >>( const mic_i& a, const mic_i& n ) { return _mm512_srav_epi32(a, n); }
|
||||
|
||||
__forceinline const mic_i sra ( const mic_i& a, const int32& b ) { return _mm512_srai_epi32(a, b); }
|
||||
__forceinline const mic_i srl ( const mic_i& a, const int32& b ) { return _mm512_srli_epi32(a, b); }
|
||||
|
||||
__forceinline const mic_i min( const mic_i& a, const mic_i& b ) { return _mm512_min_epi32(a, b); }
|
||||
__forceinline const mic_i min( const mic_i& a, const int32& b ) { return min(a,mic_i(b)); }
|
||||
__forceinline const mic_i min( const int32& a, const mic_i& b ) { return min(mic_i(a),b); }
|
||||
|
||||
__forceinline const mic_i max( const mic_i& a, const mic_i& b ) { return _mm512_max_epi32(a, b); }
|
||||
__forceinline const mic_i max( const mic_i& a, const int32& b ) { return max(a,mic_i(b)); }
|
||||
__forceinline const mic_i max( const int32& a, const mic_i& b ) { return max(mic_i(a),b); }
|
||||
|
||||
__forceinline const mic_i mask_add(const mic_m& mask, mic_i& c, const mic_i& a, const mic_i& b) { return _mm512_mask_add_epi32(c,mask,a,b); };
|
||||
__forceinline const mic_i mask_sub(const mic_m& mask, mic_i& c, const mic_i& a, const mic_i& b) { return _mm512_mask_sub_epi32(c,mask,a,b); };
|
||||
|
||||
__forceinline const mic_i mask_and(const mic_m& m,mic_i& c, const mic_i& a, const mic_i& b) { return _mm512_mask_and_epi32(c,m,a,b); };
|
||||
__forceinline const mic_i mask_or (const mic_m& m,mic_i& c, const mic_i& a, const mic_i& b) { return _mm512_mask_or_epi32(c,m,a,b); };
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Assignment Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline mic_i& operator +=( mic_i& a, const mic_i& b ) { return a = a + b; }
|
||||
__forceinline mic_i& operator +=( mic_i& a, const int32& b ) { return a = a + b; }
|
||||
|
||||
__forceinline mic_i& operator -=( mic_i& a, const mic_i& b ) { return a = a - b; }
|
||||
__forceinline mic_i& operator -=( mic_i& a, const int32& b ) { return a = a - b; }
|
||||
|
||||
__forceinline mic_i& operator *=( mic_i& a, const mic_i& b ) { return a = a * b; }
|
||||
__forceinline mic_i& operator *=( mic_i& a, const int32& b ) { return a = a * b; }
|
||||
|
||||
__forceinline mic_i& operator &=( mic_i& a, const mic_i& b ) { return a = a & b; }
|
||||
__forceinline mic_i& operator &=( mic_i& a, const int32& b ) { return a = a & b; }
|
||||
|
||||
__forceinline mic_i& operator |=( mic_i& a, const mic_i& b ) { return a = a | b; }
|
||||
__forceinline mic_i& operator |=( mic_i& a, const int32& b ) { return a = a | b; }
|
||||
|
||||
__forceinline mic_i& operator <<=( mic_i& a, const int32& b ) { return a = a << b; }
|
||||
__forceinline mic_i& operator >>=( mic_i& a, const int32& b ) { return a = a >> b; }
|
||||
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Comparison Operators + Select
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const mic_m operator ==( const mic_i& a, const mic_i& b ) { return _mm512_cmp_epi32_mask(a,b,_MM_CMPINT_EQ); }
|
||||
__forceinline const mic_m operator ==( const mic_i& a, const int32& b ) { return a == mic_i(b); }
|
||||
__forceinline const mic_m operator ==( const int32& a, const mic_i& b ) { return mic_i(a) == b; }
|
||||
|
||||
__forceinline const mic_m operator !=( const mic_i& a, const mic_i& b ) { return _mm512_cmp_epi32_mask(a,b,_MM_CMPINT_NE); }
|
||||
__forceinline const mic_m operator !=( const mic_i& a, const int32& b ) { return a != mic_i(b); }
|
||||
__forceinline const mic_m operator !=( const int32& a, const mic_i& b ) { return mic_i(a) != b; }
|
||||
|
||||
__forceinline const mic_m operator < ( const mic_i& a, const mic_i& b ) { return _mm512_cmp_epi32_mask(a,b,_MM_CMPINT_LT); }
|
||||
__forceinline const mic_m operator < ( const mic_i& a, const int32& b ) { return a < mic_i(b); }
|
||||
__forceinline const mic_m operator < ( const int32& a, const mic_i& b ) { return mic_i(a) < b; }
|
||||
|
||||
__forceinline const mic_m operator >=( const mic_i& a, const mic_i& b ) { return _mm512_cmp_epi32_mask(a,b,_MM_CMPINT_GE); }
|
||||
__forceinline const mic_m operator >=( const mic_i& a, const int32& b ) { return a >= mic_i(b); }
|
||||
__forceinline const mic_m operator >=( const int32& a, const mic_i& b ) { return mic_i(a) >= b; }
|
||||
|
||||
__forceinline const mic_m operator > ( const mic_i& a, const mic_i& b ) { return _mm512_cmp_epi32_mask(a,b,_MM_CMPINT_GT); }
|
||||
__forceinline const mic_m operator > ( const mic_i& a, const int32& b ) { return a > mic_i(b); }
|
||||
__forceinline const mic_m operator > ( const int32& a, const mic_i& b ) { return mic_i(a) > b; }
|
||||
|
||||
__forceinline const mic_m operator <=( const mic_i& a, const mic_i& b ) { return _mm512_cmp_epi32_mask(a,b,_MM_CMPINT_LE); }
|
||||
__forceinline const mic_m operator <=( const mic_i& a, const int32& b ) { return a <= mic_i(b); }
|
||||
__forceinline const mic_m operator <=( const int32& a, const mic_i& b ) { return mic_i(a) <= b; }
|
||||
|
||||
__forceinline mic_m eq( const mic_i& a, const mic_i& b) { return _mm512_cmp_epi32_mask(a,b,_MM_CMPINT_EQ); };
|
||||
__forceinline mic_m eq(const mic_m mask, const mic_i& a, const mic_i& b) { return _mm512_mask_cmp_epi32_mask(mask,a,b,_MM_CMPINT_EQ); };
|
||||
|
||||
__forceinline mic_m ne( const mic_i& a, const mic_i& b) { return _mm512_cmp_epi32_mask(a,b,_MM_CMPINT_NE); };
|
||||
__forceinline mic_m ne(const mic_m mask, const mic_i& a, const mic_i& b) { return _mm512_mask_cmp_epi32_mask(mask,a,b,_MM_CMPINT_NE); };
|
||||
|
||||
__forceinline mic_m lt( const mic_i& a, const mic_i& b) { return _mm512_cmp_epi32_mask(a,b,_MM_CMPINT_LT); };
|
||||
__forceinline mic_m lt(const mic_m mask, const mic_i& a, const mic_i& b) { return _mm512_mask_cmp_epi32_mask(mask,a,b,_MM_CMPINT_LT); };
|
||||
|
||||
__forceinline mic_m ge( const mic_i& a, const mic_i& b) { return _mm512_cmp_epi32_mask(a,b,_MM_CMPINT_GE); };
|
||||
__forceinline mic_m ge(const mic_m mask, const mic_i& a, const mic_i& b) { return _mm512_mask_cmp_epi32_mask(mask,a,b,_MM_CMPINT_GE); };
|
||||
|
||||
__forceinline mic_m gt( const mic_i& a, const mic_i& b) { return _mm512_cmp_epi32_mask(a,b,_MM_CMPINT_GT); };
|
||||
__forceinline mic_m gt(const mic_m mask, const mic_i& a, const mic_i& b) { return _mm512_mask_cmp_epi32_mask(mask,a,b,_MM_CMPINT_GT); };
|
||||
|
||||
__forceinline mic_m le( const mic_i& a, const mic_i& b) { return _mm512_cmp_epi32_mask(a,b,_MM_CMPINT_LE); };
|
||||
__forceinline mic_m le(const mic_m mask, const mic_i& a, const mic_i& b) { return _mm512_mask_cmp_epi32_mask(mask,a,b,_MM_CMPINT_LE); };
|
||||
|
||||
|
||||
__forceinline const mic_i select( const mic_m& m, const mic_i& t, const mic_i& f ) {
|
||||
return _mm512_mask_or_epi32(f,m,t,t);
|
||||
}
|
||||
|
||||
__forceinline void xchg(const mic_m &m, mic_i& a, mic_i& b) {
|
||||
const mic_i c = a; a = select(m,b,a); b = select(m,c,b);
|
||||
}
|
||||
|
||||
__forceinline mic_m test(const mic_m &m, const mic_i& a, const mic_i& b) {
|
||||
return _mm512_mask_test_epi32_mask(m,a,b);
|
||||
}
|
||||
|
||||
__forceinline mic_m test(const mic_i& a, const mic_i& b) {
|
||||
return _mm512_test_epi32_mask(a,b);
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
// Movement/Shifting/Shuffling Functions
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline mic_i swizzle(const mic_i& x,_MM_SWIZZLE_ENUM perm32 ) { return _mm512_swizzle_epi32(x,perm32); }
|
||||
__forceinline mic_i permute(const mic_i& x,_MM_PERM_ENUM perm128) { return _mm512_permute4f128_epi32(x,perm128); }
|
||||
|
||||
template<int D, int C, int B, int A> __forceinline mic_i swizzle (const mic_i& v) { return _mm512_shuffle_epi32(v,_MM_SHUF_PERM(D,C,B,A)); }
|
||||
template<int A> __forceinline mic_i swizzle (const mic_i& x) { return swizzle<A,A,A,A>(v); }
|
||||
template<> __forceinline mic_i swizzle<0>(const mic_i& x) { return swizzle(x,_MM_SWIZ_REG_AAAA); }
|
||||
template<> __forceinline mic_i swizzle<1>(const mic_i& x) { return swizzle(x,_MM_SWIZ_REG_BBBB); }
|
||||
template<> __forceinline mic_i swizzle<2>(const mic_i& x) { return swizzle(x,_MM_SWIZ_REG_CCCC); }
|
||||
template<> __forceinline mic_i swizzle<3>(const mic_i& x) { return swizzle(x,_MM_SWIZ_REG_DDDD); }
|
||||
|
||||
template<int D, int C, int B, int A> __forceinline mic_i permute(const mic_i& v) { return permute(v,_MM_SHUF_PERM(D,C,B,A)); }
|
||||
template<int A> __forceinline mic_i permute(const mic_i& x) { return permute<A,A,A,A>(x); }
|
||||
|
||||
__forceinline mic_i shuffle(const mic_i& x,_MM_PERM_ENUM perm128, _MM_SWIZZLE_ENUM perm32) { return swizzle(permute(x,perm128),perm32); }
|
||||
|
||||
__forceinline mic_i shuffle(const mic_m& mask, mic_i& v, const mic_i& x,_MM_PERM_ENUM perm128, _MM_SWIZZLE_ENUM perm32) {
|
||||
return _mm512_mask_swizzle_epi32(_mm512_mask_permute4f128_epi32(v,mask,x,perm128),mask,x,perm32);
|
||||
}
|
||||
|
||||
__forceinline mic_i swAAAA(const mic_i &x) {
|
||||
return swizzle(x,_MM_SWIZ_REG_AAAA);
|
||||
}
|
||||
|
||||
__forceinline mic_i swBBBB(const mic_i &x) {
|
||||
return swizzle(x,_MM_SWIZ_REG_BBBB);
|
||||
}
|
||||
|
||||
__forceinline mic_i swCCCC(const mic_i &x) {
|
||||
return swizzle(x,_MM_SWIZ_REG_CCCC);
|
||||
}
|
||||
|
||||
__forceinline mic_i swDDDD(const mic_i &x) {
|
||||
return swizzle(x,_MM_SWIZ_REG_DDDD);
|
||||
}
|
||||
|
||||
template<int i>
|
||||
__forceinline mic_i align_shift_right(const mic_i &a, const mic_i &b)
|
||||
{
|
||||
return _mm512_alignr_epi32(a,b,i);
|
||||
};
|
||||
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Reductions
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline int reduce_add(mic_i a) { return _mm512_reduce_add_epi32(a); }
|
||||
__forceinline int reduce_mul(mic_i a) { return _mm512_reduce_mul_epi32(a); }
|
||||
__forceinline int reduce_min(mic_i a) { return _mm512_reduce_min_epi32(a); }
|
||||
__forceinline int reduce_max(mic_i a) { return _mm512_reduce_max_epi32(a); }
|
||||
__forceinline int reduce_and(mic_i a) { return _mm512_reduce_and_epi32(a); }
|
||||
|
||||
__forceinline mic_i vreduce_min2(mic_i x) { return min(x,swizzle(x,_MM_SWIZ_REG_BADC)); }
|
||||
__forceinline mic_i vreduce_min4(mic_i x) { x = vreduce_min2(x); return min(x,swizzle(x,_MM_SWIZ_REG_CDAB)); }
|
||||
__forceinline mic_i vreduce_min8(mic_i x) { x = vreduce_min4(x); return min(x,permute(x,_MM_SHUF_PERM(2,3,0,1))); }
|
||||
__forceinline mic_i vreduce_min (mic_i x) { x = vreduce_min8(x); return min(x,permute(x,_MM_SHUF_PERM(1,0,3,2))); }
|
||||
|
||||
__forceinline mic_i vreduce_max2(mic_i x) { return max(x,swizzle(x,_MM_SWIZ_REG_BADC)); }
|
||||
__forceinline mic_i vreduce_max4(mic_i x) { x = vreduce_max2(x); return max(x,swizzle(x,_MM_SWIZ_REG_CDAB)); }
|
||||
__forceinline mic_i vreduce_max8(mic_i x) { x = vreduce_max4(x); return max(x,permute(x,_MM_SHUF_PERM(2,3,0,1))); }
|
||||
__forceinline mic_i vreduce_max (mic_i x) { x = vreduce_max8(x); return max(x,permute(x,_MM_SHUF_PERM(1,0,3,2))); }
|
||||
|
||||
__forceinline mic_i vreduce_and2(mic_i x) { return x & swizzle(x,_MM_SWIZ_REG_BADC); }
|
||||
__forceinline mic_i vreduce_and4(mic_i x) { x = vreduce_and2(x); return x & swizzle(x,_MM_SWIZ_REG_CDAB); }
|
||||
__forceinline mic_i vreduce_and8(mic_i x) { x = vreduce_and4(x); return x & permute(x,_MM_SHUF_PERM(2,3,0,1)); }
|
||||
__forceinline mic_i vreduce_and (mic_i x) { x = vreduce_and8(x); return x & permute(x,_MM_SHUF_PERM(1,0,3,2)); }
|
||||
|
||||
__forceinline mic_i vreduce_or2(mic_i x) { return x | swizzle(x,_MM_SWIZ_REG_BADC); }
|
||||
__forceinline mic_i vreduce_or4(mic_i x) { x = vreduce_or2(x); return x | swizzle(x,_MM_SWIZ_REG_CDAB); }
|
||||
__forceinline mic_i vreduce_or8(mic_i x) { x = vreduce_or4(x); return x | permute(x,_MM_SHUF_PERM(2,3,0,1)); }
|
||||
__forceinline mic_i vreduce_or (mic_i x) { x = vreduce_or8(x); return x | permute(x,_MM_SHUF_PERM(1,0,3,2)); }
|
||||
|
||||
__forceinline mic_i vreduce_add2(mic_i x) { return x + swizzle(x,_MM_SWIZ_REG_BADC); }
|
||||
__forceinline mic_i vreduce_add4(mic_i x) { x = vreduce_add2(x); return x + swizzle(x,_MM_SWIZ_REG_CDAB); }
|
||||
__forceinline mic_i vreduce_add8(mic_i x) { x = vreduce_add4(x); return x + permute(x,_MM_SHUF_PERM(2,3,0,1)); }
|
||||
__forceinline mic_i vreduce_add (mic_i x) { x = vreduce_add8(x); return x + permute(x,_MM_SHUF_PERM(1,0,3,2)); }
|
||||
|
||||
__forceinline mic_i prefix_sum(const mic_i& a)
|
||||
{
|
||||
mic_i v = a;
|
||||
v = mask_add(0xaaaa,v,v,swizzle(v,_MM_SWIZ_REG_CDAB));
|
||||
v = mask_add(0xcccc,v,v,swizzle(v,_MM_SWIZ_REG_BBBB));
|
||||
const mic_i shuf_v0 = shuffle(v,(_MM_PERM_ENUM)_MM_SHUF_PERM(2,2,0,0),_MM_SWIZ_REG_DDDD);
|
||||
v = mask_add(0xf0f0,v,v,shuf_v0);
|
||||
const mic_i shuf_v1 = shuffle(v,(_MM_PERM_ENUM)_MM_SHUF_PERM(1,1,0,0),_MM_SWIZ_REG_DDDD);
|
||||
v = mask_add(0xff00,v,v,shuf_v1);
|
||||
return v;
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Memory load and store operations
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline mic_i load1i(const int *const ptr) {
|
||||
return _mm512_extload_epi32(ptr,_MM_UPCONV_EPI32_NONE,_MM_BROADCAST_1X16,_MM_HINT_NONE);
|
||||
}
|
||||
|
||||
__forceinline mic_i load16i(const int *const ptr) {
|
||||
return _mm512_extload_epi32(ptr,_MM_UPCONV_EPI32_NONE,_MM_BROADCAST_16X16,_MM_HINT_NONE);
|
||||
}
|
||||
|
||||
__forceinline mic_i load1i_uint8(const unsigned char *const ptr) {
|
||||
return _mm512_extload_epi32(ptr,_MM_UPCONV_EPI32_UINT8,_MM_BROADCAST_1X16,_MM_HINT_NONE);
|
||||
}
|
||||
|
||||
__forceinline mic_i load16i_uint8(const unsigned char *const ptr) {
|
||||
return _mm512_extload_epi32(ptr,_MM_UPCONV_EPI32_UINT8,_MM_BROADCAST32_NONE,_MM_HINT_NONE);
|
||||
}
|
||||
|
||||
__forceinline mic_i broadcast4to16i(const int *const ptr) {
|
||||
return _mm512_extload_epi32(ptr,_MM_UPCONV_EPI32_NONE,_MM_BROADCAST_4X16,_MM_HINT_NONE);
|
||||
}
|
||||
|
||||
__forceinline mic_i broadcast1to16i(const int *const ptr) {
|
||||
return _mm512_extload_epi32(ptr,_MM_UPCONV_EPI32_NONE,_MM_BROADCAST_1X16,_MM_HINT_NONE);
|
||||
}
|
||||
|
||||
__forceinline mic_i uload16i(const int *const addr) {
|
||||
mic_i r = _mm512_undefined_epi32();
|
||||
r =_mm512_extloadunpacklo_epi32(r, addr, _MM_UPCONV_EPI32_NONE, _MM_HINT_NONE);
|
||||
return _mm512_extloadunpackhi_epi32(r, addr+16, _MM_UPCONV_EPI32_NONE, _MM_HINT_NONE);
|
||||
}
|
||||
|
||||
__forceinline mic_i uload16i_low(const mic_m& mask, const void* addr) {
|
||||
mic_i v = _mm512_undefined_epi32();
|
||||
return _mm512_mask_extloadunpacklo_epi32(v, mask, addr, _MM_UPCONV_EPI32_NONE, _MM_HINT_NONE);
|
||||
}
|
||||
|
||||
__forceinline mic_i uload16i(const mic_m& mask,const int *const addr) {
|
||||
mic_i r = _mm512_undefined_epi32();
|
||||
r =_mm512_mask_extloadunpacklo_epi32(r, mask,addr, _MM_UPCONV_EPI32_NONE, _MM_HINT_NONE);
|
||||
return _mm512_mask_extloadunpackhi_epi32(r, mask,addr+16, _MM_UPCONV_EPI32_NONE, _MM_HINT_NONE);
|
||||
}
|
||||
|
||||
|
||||
__forceinline mic_i gather16i_4i(const int *__restrict__ const ptr0,
|
||||
const int *__restrict__ const ptr1,
|
||||
const int *__restrict__ const ptr2,
|
||||
const int *__restrict__ const ptr3)
|
||||
{
|
||||
mic_i v = broadcast4to16i(ptr0);
|
||||
v = select((mic_m)0xf0 , broadcast4to16i(ptr1),v);
|
||||
v = select((mic_m)0xf00 , broadcast4to16i(ptr2),v);
|
||||
v = select((mic_m)0xf000, broadcast4to16i(ptr3),v);
|
||||
return v;
|
||||
}
|
||||
|
||||
|
||||
__forceinline mic_i gather16i_4i_align(const void *__restrict__ const ptr0,
|
||||
const void *__restrict__ const ptr1,
|
||||
const void *__restrict__ const ptr2,
|
||||
const void *__restrict__ const ptr3)
|
||||
{
|
||||
mic_i v = broadcast4to16i((const int*)ptr3);
|
||||
v = align_shift_right<12>(v,broadcast4to16i((const int*)ptr2));
|
||||
v = align_shift_right<12>(v,broadcast4to16i((const int*)ptr1));
|
||||
v = align_shift_right<12>(v,broadcast4to16i((const int*)ptr0));
|
||||
return v;
|
||||
}
|
||||
|
||||
__forceinline mic_i gather16i_4i_align(const mic_i &v0,
|
||||
const mic_i &v1,
|
||||
const mic_i &v2,
|
||||
const mic_i &v3)
|
||||
{
|
||||
mic_i v = v3;
|
||||
v = align_shift_right<12>(v,v2);
|
||||
v = align_shift_right<12>(v,v1);
|
||||
v = align_shift_right<12>(v,v0);
|
||||
return v;
|
||||
}
|
||||
|
||||
|
||||
__forceinline mic_i gather16i(const mic_m& mask, const int *const ptr, const mic_i& index,const _MM_INDEX_SCALE_ENUM scale) {
|
||||
return _mm512_mask_i32extgather_epi32(_mm512_undefined_epi32(),mask,index,ptr,_MM_UPCONV_EPI32_NONE,scale,0);
|
||||
}
|
||||
|
||||
__forceinline mic_i gather16i(const mic_m& mask, mic_i& dest, const int *const ptr, const mic_i& index,const _MM_INDEX_SCALE_ENUM scale) {
|
||||
return _mm512_mask_i32extgather_epi32(dest,mask,index,ptr,_MM_UPCONV_EPI32_NONE,scale,0);
|
||||
}
|
||||
|
||||
__forceinline void scatter16i(const mic_m& mask,int *const ptr, const mic_i& index,const mic_i& v, const _MM_INDEX_SCALE_ENUM scale) {
|
||||
_mm512_mask_i32extscatter_epi32((int*)ptr,mask,index,v,_MM_DOWNCONV_EPI32_NONE,scale,0);
|
||||
}
|
||||
|
||||
__forceinline void ustore16i(void *addr, const mic_i& reg) {
|
||||
_mm512_extpackstorelo_epi32((int*)addr+0 ,reg, _MM_DOWNCONV_EPI32_NONE, _MM_HINT_NONE);
|
||||
_mm512_extpackstorehi_epi32((int*)addr+16 ,reg, _MM_DOWNCONV_EPI32_NONE, _MM_HINT_NONE);
|
||||
}
|
||||
|
||||
/* pass by value to avoid compiler generating inefficient code */
|
||||
__forceinline void compactustore16i(const mic_m mask,void * addr, const mic_i reg) {
|
||||
_mm512_mask_extpackstorelo_epi32((int*)addr+0 ,mask, reg, _MM_DOWNCONV_EPI32_NONE, _MM_HINT_NONE);
|
||||
_mm512_mask_extpackstorehi_epi32((int*)addr+16 ,mask, reg, _MM_DOWNCONV_EPI32_NONE, _MM_HINT_NONE);
|
||||
}
|
||||
|
||||
|
||||
__forceinline void compactustore16i_low(const mic_m mask, void *addr, const mic_i ®) {
|
||||
_mm512_mask_extpackstorelo_epi32((int*)addr+0 ,mask, reg, _MM_DOWNCONV_EPI32_NONE, _MM_HINT_NONE);
|
||||
}
|
||||
|
||||
__forceinline void store1i(void *addr, const mic_i& reg) {
|
||||
_mm512_mask_extpackstorelo_epi32((int*)addr+0 ,(mic_m)1, reg, _MM_DOWNCONV_EPI32_NONE, _MM_HINT_NONE);
|
||||
}
|
||||
|
||||
__forceinline void ustore16i_low(void *addr, const mic_i& reg) {
|
||||
_mm512_extpackstorelo_epi32((int*)addr+0 ,reg, _MM_DOWNCONV_EPI32_NONE, _MM_HINT_NONE);
|
||||
}
|
||||
|
||||
__forceinline void compactustore16i_high(const mic_m mask, int *addr, const mic_i& reg) {
|
||||
_mm512_mask_extpackstorehi_epi32((int*)addr+16 ,mask, reg, _MM_DOWNCONV_EPI32_NONE, _MM_HINT_NONE);
|
||||
}
|
||||
|
||||
__forceinline void store16i_nr(void *__restrict__ ptr, const mic_i& a) {
|
||||
_mm512_storenr_ps(ptr,_mm512_castsi512_ps(a));
|
||||
}
|
||||
|
||||
__forceinline void store16i_ngo(void *__restrict__ ptr, const mic_i& a) {
|
||||
_mm512_storenrngo_ps(ptr,_mm512_castsi512_ps(a));
|
||||
}
|
||||
|
||||
__forceinline void store16i(const mic_m& mask, void* __restrict__ addr, const mic_i& v2) {
|
||||
_mm512_mask_extstore_epi32(addr,mask,v2,_MM_DOWNCONV_EPI32_NONE,_MM_HINT_NONE);
|
||||
}
|
||||
|
||||
__forceinline void store16i_nt(void* __restrict__ addr, const mic_i& v2) {
|
||||
_mm512_extstore_epi32(addr,v2,_MM_DOWNCONV_EPI32_NONE,_MM_HINT_NT);
|
||||
}
|
||||
|
||||
__forceinline void store16i(void* __restrict__ addr, const mic_i& v2) {
|
||||
_mm512_extstore_epi32(addr,v2,_MM_DOWNCONV_EPI32_NONE,_MM_HINT_NONE);
|
||||
}
|
||||
|
||||
__forceinline void store16i_uint8(const mic_m& mask, void* __restrict__ addr, const mic_i& v2) {
|
||||
_mm512_mask_extstore_epi32(addr,mask,v2,_MM_DOWNCONV_EPI32_UINT8,_MM_HINT_NONE);
|
||||
}
|
||||
|
||||
__forceinline mic_i convert_uint32(const __m512 f) {
|
||||
return _mm512_cvtfxpnt_round_adjustps_epu32(f,_MM_FROUND_TO_ZERO,_MM_EXPADJ_NONE);
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Output Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline std::ostream &operator<<(std::ostream& cout, const mic_i& v)
|
||||
{
|
||||
cout << "<" << v[0];
|
||||
for (int i=1; i<16; i++) cout << ", " << v[i];
|
||||
cout << ">";
|
||||
return cout;
|
||||
}
|
||||
}
|
||||
@@ -1,5 +1,5 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
@@ -14,43 +14,44 @@
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#include "parallel_set.h"
|
||||
#include "../sys/regression.h"
|
||||
#include "mic.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
struct parallel_set_regression_test : public RegressionTest
|
||||
{
|
||||
parallel_set_regression_test(const char* name) : RegressionTest(name) {
|
||||
registerRegressionTest(this);
|
||||
}
|
||||
|
||||
bool run ()
|
||||
{
|
||||
bool passed = true;
|
||||
|
||||
/* create vector with random numbers */
|
||||
const size_t N = 10000;
|
||||
std::vector<uint32_t> unsorted(N);
|
||||
for (size_t i=0; i<N; i++) unsorted[i] = 2*rand();
|
||||
|
||||
/* created set from numbers */
|
||||
parallel_set<uint32_t> sorted;
|
||||
sorted.init(unsorted);
|
||||
|
||||
/* check that all elements are in the set */
|
||||
for (size_t i=0; i<N; i++) {
|
||||
passed &= sorted.lookup(unsorted[i]);
|
||||
}
|
||||
|
||||
/* check that these elements are not in the set */
|
||||
for (size_t i=0; i<N; i++) {
|
||||
passed &= !sorted.lookup(unsorted[i]+1);
|
||||
}
|
||||
|
||||
return passed;
|
||||
}
|
||||
__aligned(64) unsigned int mic_m::shift1[32] = {
|
||||
((unsigned int)1 << 0),
|
||||
((unsigned int)1 << 1),
|
||||
((unsigned int)1 << 2),
|
||||
((unsigned int)1 << 3),
|
||||
((unsigned int)1 << 4),
|
||||
((unsigned int)1 << 5),
|
||||
((unsigned int)1 << 6),
|
||||
((unsigned int)1 << 7),
|
||||
((unsigned int)1 << 8),
|
||||
((unsigned int)1 << 9),
|
||||
((unsigned int)1 << 10),
|
||||
((unsigned int)1 << 11),
|
||||
((unsigned int)1 << 12),
|
||||
((unsigned int)1 << 13),
|
||||
((unsigned int)1 << 14),
|
||||
((unsigned int)1 << 15),
|
||||
((unsigned int)1 << 16),
|
||||
((unsigned int)1 << 17),
|
||||
((unsigned int)1 << 18),
|
||||
((unsigned int)1 << 19),
|
||||
((unsigned int)1 << 20),
|
||||
((unsigned int)1 << 21),
|
||||
((unsigned int)1 << 22),
|
||||
((unsigned int)1 << 23),
|
||||
((unsigned int)1 << 24),
|
||||
((unsigned int)1 << 25),
|
||||
((unsigned int)1 << 26),
|
||||
((unsigned int)1 << 27),
|
||||
((unsigned int)1 << 28),
|
||||
((unsigned int)1 << 29),
|
||||
((unsigned int)1 << 30),
|
||||
((unsigned int)1 << 31)
|
||||
};
|
||||
|
||||
parallel_set_regression_test parallel_set_regression("parallel_set_regression_test");
|
||||
}
|
||||
};
|
||||
@@ -0,0 +1,121 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#pragma once
|
||||
|
||||
namespace embree
|
||||
{
|
||||
/*! 16-wide MIC bool type. */
|
||||
class mic_m
|
||||
{
|
||||
public:
|
||||
__mmask v;
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constructors, Assignment & Cast Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline mic_m () {};
|
||||
__forceinline mic_m (const mic_m &t) { v = t.v; };
|
||||
__forceinline mic_m& operator=(const mic_m &f) { v = f.v; return *this; };
|
||||
|
||||
__forceinline mic_m (const __mmask &t) { v = t; };
|
||||
__forceinline operator __mmask () const { return v; };
|
||||
|
||||
__forceinline mic_m(bool b) { v = b ? 0xFFFF : 0x0000; };
|
||||
__forceinline mic_m(int t ) { v = (__mmask)t; };
|
||||
__forceinline mic_m(unsigned int t ) { v = (__mmask)t; };
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constants
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline mic_m( FalseTy ) : v(0x0000) {}
|
||||
__forceinline mic_m( TrueTy ) : v(0xffff) {}
|
||||
|
||||
static unsigned int shift1[32];
|
||||
};
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Unary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline mic_m operator!(const mic_m &a) { return _mm512_knot(a); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Binary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline mic_m operator&(const mic_m &a, const mic_m &b) { return _mm512_kand(a,b); };
|
||||
__forceinline mic_m operator|(const mic_m &a, const mic_m &b) { return _mm512_kor(a,b); };
|
||||
__forceinline mic_m operator^(const mic_m &a, const mic_m &b) { return _mm512_kxor(a,b); };
|
||||
|
||||
__forceinline mic_m andn(const mic_m &a, const mic_m &b) { return _mm512_kandn(b,a); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Assignment Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const mic_m operator &=( mic_m& a, const mic_m& b ) { return a = a & b; }
|
||||
__forceinline const mic_m operator |=( mic_m& a, const mic_m& b ) { return a = a | b; }
|
||||
__forceinline const mic_m operator ^=( mic_m& a, const mic_m& b ) { return a = a ^ b; }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Comparison Operators + Select
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const mic_m operator !=( const mic_m& a, const mic_m& b ) { return _mm512_kxor(a, b); }
|
||||
__forceinline const mic_m operator ==( const mic_m& a, const mic_m& b ) { return _mm512_kxnor(a, b); }
|
||||
|
||||
__forceinline mic_m select (const mic_m &s, const mic_m &a, const mic_m &b) {
|
||||
return _mm512_kor(_mm512_kand(s,a),_mm512_kandn(s,b));
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Reduction Operations
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline int all(const mic_m &a) { return _mm512_kortestc(a,a) != 0; }
|
||||
__forceinline int any(const mic_m &a) { return _mm512_kortestz(a,a) == 0; }
|
||||
__forceinline int none(const mic_m &a) { return _mm512_kortestz(a,a) != 0; }
|
||||
|
||||
__forceinline int all ( const mic_m& valid, const mic_m& b ) { return all(!valid | b); }
|
||||
__forceinline int any ( const mic_m& valid, const mic_m& b ) { return any( valid & b); }
|
||||
__forceinline int none ( const mic_m& valid, const mic_m& b ) { return none(valid & b); }
|
||||
|
||||
__forceinline size_t movemask( const mic_m& a ) { return _mm512_kmov(a); }
|
||||
__forceinline size_t popcnt ( const mic_m& a ) { return _mm_countbits_32(a.v); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Convertion Operations
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline unsigned int toInt (const mic_m &a) { return _mm512_mask2int(a); };
|
||||
__forceinline mic_m toMask(const int &a) { return _mm512_int2mask(a); };
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Output Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
inline std::ostream& operator<<(std::ostream& cout, const mic_m& a)
|
||||
{
|
||||
cout << "<";
|
||||
for (size_t i=0; i<16; i++) {
|
||||
if ((a.v >> i) & 1) cout << "1"; else cout << "0";
|
||||
}
|
||||
return cout << ">";
|
||||
}
|
||||
}
|
||||
+13
-72
@@ -1,5 +1,5 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
@@ -16,85 +16,26 @@
|
||||
|
||||
#pragma once
|
||||
|
||||
#include "../math/math.h"
|
||||
#include "math/math.h"
|
||||
|
||||
/* include SSE emulation for Xeon Phi */
|
||||
#if defined (__MIC__)
|
||||
# include "simd/sse_mic.h"
|
||||
# include "simd/mic.h"
|
||||
#endif
|
||||
|
||||
/* include SSE wrapper classes */
|
||||
#if defined(__SSE__)
|
||||
# include "sse.h"
|
||||
# include "simd/sse.h"
|
||||
#endif
|
||||
|
||||
/* include AVX wrapper classes */
|
||||
#if defined(__AVX__)
|
||||
# include "avx.h"
|
||||
#include "simd/avx.h"
|
||||
#endif
|
||||
|
||||
/* include AVX512 wrapper classes */
|
||||
#if defined (__AVX512F__)
|
||||
# include "avx512.h"
|
||||
#endif
|
||||
|
||||
#if defined(__AVX512F__)
|
||||
# define AVX_ZERO_UPPER()
|
||||
#elif defined (__AVX__)
|
||||
# define AVX_ZERO_UPPER() _mm256_zeroupper()
|
||||
#if defined (__AVX__)
|
||||
#define AVX_ZERO_UPPER() _mm256_zeroupper()
|
||||
#else
|
||||
# define AVX_ZERO_UPPER()
|
||||
#define AVX_ZERO_UPPER()
|
||||
#endif
|
||||
|
||||
namespace embree
|
||||
{
|
||||
/* foreach unique */
|
||||
template<typename vbool, typename vint, typename Closure>
|
||||
__forceinline void foreach_unique(const vbool& valid0, const vint& vi, const Closure& closure)
|
||||
{
|
||||
vbool valid1 = valid0;
|
||||
while (any(valid1)) {
|
||||
const int j = int(__bsf(movemask(valid1)));
|
||||
const int i = vi[j];
|
||||
const vbool valid2 = valid1 & (i == vi);
|
||||
valid1 = valid1 & !valid2;
|
||||
closure(valid2,i);
|
||||
}
|
||||
}
|
||||
|
||||
/* foreach unique */
|
||||
template<typename vbool, typename vint, typename Closure>
|
||||
__forceinline void foreach_unique_index(const vbool& valid0, const vint& vi, const Closure& closure)
|
||||
{
|
||||
vbool valid1 = valid0;
|
||||
while (any(valid1)) {
|
||||
const int j = (int) __bsf(movemask(valid1));
|
||||
const int i = vi[j];
|
||||
const vbool valid2 = valid1 & (i == vi);
|
||||
valid1 = valid1 & !valid2;
|
||||
closure(valid2,i,j);
|
||||
}
|
||||
}
|
||||
|
||||
template<typename Closure>
|
||||
__forceinline void foreach2(int x0, int x1, int y0, int y1, const Closure& closure)
|
||||
{
|
||||
__aligned(64) int U[2*VSIZEX];
|
||||
__aligned(64) int V[2*VSIZEX];
|
||||
int index = 0;
|
||||
for (int y=y0; y<y1; y++) {
|
||||
const bool lasty = y+1>=y1;
|
||||
const vintx vy = y;
|
||||
for (int x=x0; x<x1; ) { //x+=VSIZEX) {
|
||||
const bool lastx = x+VSIZEX >= x1;
|
||||
vintx vx = x+vintx(step);
|
||||
vintx::storeu(&U[index],vx);
|
||||
vintx::storeu(&V[index],vy);
|
||||
const int dx = min(x1-x,VSIZEX);
|
||||
index += dx;
|
||||
x += dx;
|
||||
if (index >= VSIZEX || (lastx && lasty)) {
|
||||
const vboolx valid = vintx(step) < vintx(index);
|
||||
closure(valid,vintx::load(U),vintx::load(V));
|
||||
x-= max(0,index-VSIZEX);
|
||||
index = 0;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,114 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#pragma once
|
||||
|
||||
#ifndef __GNUC__
|
||||
#pragma message (" --- Intel remark: SSE4 intrinsics are emulated with SSE3 ---")
|
||||
#endif
|
||||
|
||||
#define _MM_FROUND_TO_NEAREST_INT 0x00
|
||||
#define _MM_FROUND_TO_NEG_INF 0x01
|
||||
#define _MM_FROUND_TO_POS_INF 0x02
|
||||
#define _MM_FROUND_TO_ZERO 0x03
|
||||
#define _MM_FROUND_CUR_DIRECTION 0x04
|
||||
|
||||
#define _mm_blendv_ps __emu_mm_blendv_ps
|
||||
__forceinline __m128 _mm_blendv_ps( __m128 value, __m128 input, __m128 mask ) {
|
||||
return _mm_or_ps(_mm_and_ps(mask, input), _mm_andnot_ps(mask, value));
|
||||
}
|
||||
|
||||
#define _mm_blend_ps __emu_mm_blend_ps
|
||||
__forceinline __m128 _mm_blend_ps( __m128 value, __m128 input, const int mask ) {
|
||||
assert(mask < 0x10); return _mm_blendv_ps(value, input, _mm_lookupmask_ps[mask]);
|
||||
}
|
||||
|
||||
#define _mm_blendv_epi8 __emu_mm_blendv_epi8
|
||||
__forceinline __m128i _mm_blendv_epi8( __m128i value, __m128i input, __m128i mask ) {
|
||||
return _mm_or_si128(_mm_and_si128(mask, input), _mm_andnot_si128(mask, value));
|
||||
}
|
||||
|
||||
#define _mm_mullo_epi32 __emu_mm_mullo_epi32
|
||||
__forceinline __m128i _mm_mullo_epi32( __m128i value, __m128i input ) {
|
||||
__m128i rvalue;
|
||||
char* _r = (char*)(&rvalue + 1);
|
||||
char* _v = (char*)(& value + 1);
|
||||
char* _i = (char*)(& input + 1);
|
||||
for ( ssize_t i = -16 ; i != 0 ; i += 4 ) *((int32*)(_r + i)) = *((int32*)(_v + i))* *((int32*)(_i + i));
|
||||
return rvalue;
|
||||
}
|
||||
|
||||
|
||||
#define _mm_min_epi32 __emu_mm_min_epi32
|
||||
__forceinline __m128i _mm_min_epi32( __m128i value, __m128i input ) {
|
||||
return _mm_blendv_epi8(input, value, _mm_cmplt_epi32(value, input));
|
||||
}
|
||||
|
||||
#define _mm_max_epi32 __emu_mm_max_epi32
|
||||
__forceinline __m128i _mm_max_epi32( __m128i value, __m128i input ) {
|
||||
return _mm_blendv_epi8(value, input, _mm_cmplt_epi32(value, input));
|
||||
}
|
||||
|
||||
#define _mm_extract_epi32 __emu_mm_extract_epi32
|
||||
__forceinline int _mm_extract_epi32( __m128i input, const int index ) {
|
||||
switch ( index ) {
|
||||
case 0: return _mm_cvtsi128_si32(input);
|
||||
case 1: return _mm_cvtsi128_si32(_mm_shuffle_epi32(input, _MM_SHUFFLE(1, 1, 1, 1)));
|
||||
case 2: return _mm_cvtsi128_si32(_mm_shuffle_epi32(input, _MM_SHUFFLE(2, 2, 2, 2)));
|
||||
case 3: return _mm_cvtsi128_si32(_mm_shuffle_epi32(input, _MM_SHUFFLE(3, 3, 3, 3)));
|
||||
default: assert(false); return 0;
|
||||
}
|
||||
}
|
||||
|
||||
#define _mm_insert_epi32 __emu_mm_insert_epi32
|
||||
__forceinline __m128i _mm_insert_epi32( __m128i value, int input, const int index ) {
|
||||
assert(index >= 0 && index < 4); ((int*)&value)[index] = input; return value;
|
||||
}
|
||||
|
||||
#define _mm_extract_ps __emu_mm_extract_ps
|
||||
__forceinline int _mm_extract_ps( __m128 input, const int index ) {
|
||||
int32* ptr = (int32*)&input; return ptr[index];
|
||||
}
|
||||
|
||||
#define _mm_insert_ps __emu_mm_insert_ps
|
||||
__forceinline __m128 _mm_insert_ps( __m128 value, __m128 input, const int index )
|
||||
{ assert(index < 0x100); ((float*)&value)[(index >> 4)&0x3] = ((float*)&input)[index >> 6]; return _mm_andnot_ps(_mm_lookupmask_ps[index&0xf], value); }
|
||||
|
||||
#define _mm_round_ps __emu_mm_round_ps
|
||||
__forceinline __m128 _mm_round_ps( __m128 value, const int flags )
|
||||
{
|
||||
switch ( flags )
|
||||
{
|
||||
case _MM_FROUND_TO_NEAREST_INT: return _mm_cvtepi32_ps(_mm_cvtps_epi32(value));
|
||||
case _MM_FROUND_TO_NEG_INF : return _mm_cvtepi32_ps(_mm_cvtps_epi32(_mm_add_ps(value, _mm_set1_ps(-0.5f))));
|
||||
case _MM_FROUND_TO_POS_INF : return _mm_cvtepi32_ps(_mm_cvtps_epi32(_mm_add_ps(value, _mm_set1_ps( 0.5f))));
|
||||
case _MM_FROUND_TO_ZERO : return _mm_cvtepi32_ps(_mm_cvttps_epi32(value));
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
#ifdef _M_X64
|
||||
#define _mm_insert_epi64 __emu_mm_insert_epi64
|
||||
__forceinline __m128i _mm_insert_epi64( __m128i value, __int64 input, const int index ) {
|
||||
assert(size_t(index) < 4); ((__int64*)&value)[index] = input; return value;
|
||||
}
|
||||
|
||||
#define _mm_extract_epi64 __emu_mm_extract_epi64
|
||||
__forceinline __int64 _mm_extract_epi64( __m128i input, const int index ) {
|
||||
assert(size_t(index) < 2);
|
||||
return index == 0 ? _mm_cvtsi128_si64x(input) : _mm_cvtsi128_si64x(_mm_unpackhi_epi64(input, input));
|
||||
}
|
||||
#endif
|
||||
+2
-11
@@ -1,5 +1,5 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
@@ -16,8 +16,7 @@
|
||||
|
||||
#include "sse.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
namespace embree {
|
||||
const __m128 _mm_lookupmask_ps[16] = {
|
||||
_mm_castsi128_ps(_mm_set_epi32( 0, 0, 0, 0)),
|
||||
_mm_castsi128_ps(_mm_set_epi32( 0, 0, 0,-1)),
|
||||
@@ -36,12 +35,4 @@ namespace embree
|
||||
_mm_castsi128_ps(_mm_set_epi32(-1,-1,-1, 0)),
|
||||
_mm_castsi128_ps(_mm_set_epi32(-1,-1,-1,-1))
|
||||
};
|
||||
|
||||
const __m128d _mm_lookupmask_pd[4] = {
|
||||
_mm_castsi128_pd(_mm_set_epi32( 0, 0, 0, 0)),
|
||||
_mm_castsi128_pd(_mm_set_epi32( 0, 0,-1,-1)),
|
||||
_mm_castsi128_pd(_mm_set_epi32(-1,-1, 0, 0)),
|
||||
_mm_castsi128_pd(_mm_set_epi32(-1,-1,-1,-1))
|
||||
};
|
||||
|
||||
}
|
||||
|
||||
+34
-23
@@ -1,5 +1,5 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
@@ -16,32 +16,43 @@
|
||||
|
||||
#pragma once
|
||||
|
||||
#include "../sys/platform.h"
|
||||
#include "../sys/intrinsics.h"
|
||||
#include "../math/constants.h"
|
||||
#include "varying.h"
|
||||
#include "sys/platform.h"
|
||||
#include "sys/intrinsics.h"
|
||||
#include "sse_special.h"
|
||||
|
||||
namespace embree
|
||||
{
|
||||
#if defined(__SSE4_1__)
|
||||
__forceinline __m128 blendv_ps(__m128 f, __m128 t, __m128 mask) {
|
||||
return _mm_blendv_ps(f,t,mask);
|
||||
}
|
||||
#else
|
||||
__forceinline __m128 blendv_ps(__m128 f, __m128 t, __m128 mask) {
|
||||
/* Workaround for Compiler bug in VS2008 */
|
||||
#if !defined(__SSE4_1__) || defined(_MSC_VER) && (_MSC_VER < 1600) && !defined(__INTEL_COMPILER)
|
||||
#define _mm_blendv_ps __emu_mm_blendv_ps
|
||||
__forceinline __m128 _mm_blendv_ps( __m128 f, __m128 t, __m128 mask ) {
|
||||
return _mm_or_ps(_mm_and_ps(mask, t), _mm_andnot_ps(mask, f));
|
||||
}
|
||||
#endif
|
||||
|
||||
extern const __m128 _mm_lookupmask_ps[16];
|
||||
extern const __m128d _mm_lookupmask_pd[4];
|
||||
/* Workaround for Compiler bug in VS2008 */
|
||||
#if defined(_MSC_VER) && (_MSC_VER < 1600) && !defined(__INTEL_COMPILER)
|
||||
#define _mm_extract_epi32 __emu_mm_extract_epi32
|
||||
__forceinline int _mm_extract_epi32( __m128i input, const int i ) {
|
||||
return input.m128i_i32[i];
|
||||
}
|
||||
#endif
|
||||
|
||||
namespace embree
|
||||
{
|
||||
extern const __m128 _mm_lookupmask_ps[16];
|
||||
|
||||
struct sseb;
|
||||
struct ssei;
|
||||
struct ssef;
|
||||
|
||||
#if !defined(__MIC__)
|
||||
typedef ssef ssef_t;
|
||||
typedef ssei ssei_t;
|
||||
|
||||
typedef ssef ssef_m;
|
||||
typedef ssei ssei_m;
|
||||
#endif
|
||||
}
|
||||
|
||||
#if defined(__AVX512VL__)
|
||||
#include "vboolf4_avx512.h"
|
||||
#else
|
||||
#include "vboolf4_sse2.h"
|
||||
#endif
|
||||
#include "vint4_sse2.h"
|
||||
#include "vuint4_sse2.h"
|
||||
#include "vfloat4_sse2.h"
|
||||
#include "simd/sseb.h"
|
||||
#include "simd/ssei.h"
|
||||
#include "simd/ssef.h"
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2017 Intel Corporation //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
@@ -14,20 +14,38 @@
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#include "light.h"
|
||||
#pragma once
|
||||
|
||||
#include "sys/platform.h"
|
||||
#include "sys/intrinsics.h"
|
||||
|
||||
#include <immintrin.h>
|
||||
|
||||
namespace embree {
|
||||
|
||||
Light_EvalRes Light_eval(const Light* uniform,
|
||||
const DifferentialGeometry&,
|
||||
const Vec3fa&)
|
||||
{
|
||||
Light_EvalRes res;
|
||||
res.value = Vec3fa(0.f);
|
||||
res.dist = inf;
|
||||
res.pdf = 0.f;
|
||||
return res;
|
||||
struct sseb_t;
|
||||
struct ssei_t;
|
||||
struct ssef_t;
|
||||
struct sseb_m;
|
||||
struct ssei_m;
|
||||
struct ssef_m;
|
||||
}
|
||||
extern "C" void dummy() {} // just to avoid linker warning under MacOSX
|
||||
|
||||
} // namespace embree
|
||||
#include "simd/sseb_mic.h"
|
||||
#include "simd/ssei_mic.h"
|
||||
#include "simd/ssef_mic.h"
|
||||
|
||||
namespace embree {
|
||||
typedef sseb_m sseb;
|
||||
typedef ssei_m ssei;
|
||||
typedef ssef_m ssef;
|
||||
|
||||
__forceinline const ssei cast( const ssef& a ) { return *(ssei*)&a; }
|
||||
__forceinline const ssef cast( const ssei& a ) { return *(ssef*)&a; }
|
||||
|
||||
__forceinline ssei_t floor_i( const ssef_t& other )
|
||||
{
|
||||
const __m512i m512 = _mm512_cvtfxpnt_round_adjustps_epi32(other.m512,_MM_FROUND_FLOOR,_MM_EXPADJ_NONE);
|
||||
return ssei_t(m512);
|
||||
}
|
||||
|
||||
}
|
||||
@@ -0,0 +1,761 @@
|
||||
/* SIMD (SSE1+MMX or SSE2) implementation of sin, cos, exp and log
|
||||
|
||||
Inspired by Intel Approximate Math library, and based on the
|
||||
corresponding algorithms of the cephes math library
|
||||
|
||||
The default is to use the SSE1 version. If you define USE_SSE2 the
|
||||
the SSE2 intrinsics will be used in place of the MMX intrinsics. Do
|
||||
not expect any significant performance improvement with SSE2.
|
||||
*/
|
||||
|
||||
/* Copyright (C) 2007 Julien Pommier
|
||||
|
||||
This software is provided 'as-is', without any express or implied
|
||||
warranty. In no event will the authors be held liable for any damages
|
||||
arising from the use of this software.
|
||||
|
||||
Permission is granted to anyone to use this software for any purpose,
|
||||
including commercial applications, and to alter it and redistribute it
|
||||
freely, subject to the following restrictions:
|
||||
|
||||
1. The origin of this software must not be misrepresented; you must not
|
||||
claim that you wrote the original software. If you use this software
|
||||
in a product, an acknowledgment in the product documentation would be
|
||||
appreciated but is not required.
|
||||
2. Altered source versions must be plainly marked as such, and must not be
|
||||
misrepresented as being the original software.
|
||||
3. This notice may not be removed or altered from any source distribution.
|
||||
|
||||
(this is the zlib license)
|
||||
*/
|
||||
|
||||
#include <xmmintrin.h>
|
||||
|
||||
/* yes I know, the top of this file is quite ugly */
|
||||
|
||||
#ifdef _MSC_VER /* visual c++ */
|
||||
# define ALIGN16_BEG __declspec(align(16))
|
||||
# define ALIGN16_END
|
||||
# define USE_SSE2
|
||||
#else /* gcc or icc */
|
||||
# define ALIGN16_BEG
|
||||
# define ALIGN16_END __attribute__((aligned(16)))
|
||||
#endif
|
||||
|
||||
/* __m128 is ugly to write */
|
||||
typedef __m128 v4sf; // vector of 4 float (sse1)
|
||||
|
||||
#ifdef USE_SSE2
|
||||
# include <emmintrin.h>
|
||||
typedef __m128i v4si; // vector of 4 int (sse2)
|
||||
#else
|
||||
typedef __m64 v2si; // vector of 2 int (mmx)
|
||||
#endif
|
||||
|
||||
/* declare some SSE constants -- why can't I figure a better way to do that? */
|
||||
#define _PS_CONST(Name, Val) \
|
||||
static const ALIGN16_BEG float _ps_##Name[4] ALIGN16_END = { Val, Val, Val, Val }
|
||||
#define _PI32_CONST(Name, Val) \
|
||||
static const ALIGN16_BEG int _pi32_##Name[4] ALIGN16_END = { Val, Val, Val, Val }
|
||||
#define _PS_CONST_TYPE(Name, Type, Val) \
|
||||
static const ALIGN16_BEG Type _ps_##Name[4] ALIGN16_END = { Val, Val, Val, Val }
|
||||
|
||||
_PS_CONST(1 , 1.0f);
|
||||
_PS_CONST(0p5, 0.5f);
|
||||
/* the smallest non denormalized float number */
|
||||
_PS_CONST_TYPE(min_norm_pos, int, 0x00800000);
|
||||
_PS_CONST_TYPE(mant_mask, int, 0x7f800000);
|
||||
_PS_CONST_TYPE(inv_mant_mask, int, ~0x7f800000);
|
||||
|
||||
_PS_CONST_TYPE(sign_mask, int, 0x80000000);
|
||||
_PS_CONST_TYPE(inv_sign_mask, int, ~0x80000000);
|
||||
|
||||
_PI32_CONST(1, 1);
|
||||
_PI32_CONST(inv1, ~1);
|
||||
_PI32_CONST(2, 2);
|
||||
_PI32_CONST(4, 4);
|
||||
_PI32_CONST(0x7f, 0x7f);
|
||||
|
||||
_PS_CONST(cephes_SQRTHF, 0.707106781186547524f);
|
||||
_PS_CONST(cephes_log_p0, 7.0376836292E-2f);
|
||||
_PS_CONST(cephes_log_p1, - 1.1514610310E-1f);
|
||||
_PS_CONST(cephes_log_p2, 1.1676998740E-1f);
|
||||
_PS_CONST(cephes_log_p3, - 1.2420140846E-1f);
|
||||
_PS_CONST(cephes_log_p4, + 1.4249322787E-1f);
|
||||
_PS_CONST(cephes_log_p5, - 1.6668057665E-1f);
|
||||
_PS_CONST(cephes_log_p6, + 2.0000714765E-1f);
|
||||
_PS_CONST(cephes_log_p7, - 2.4999993993E-1f);
|
||||
_PS_CONST(cephes_log_p8, + 3.3333331174E-1f);
|
||||
_PS_CONST(cephes_log_q1, -2.12194440e-4f);
|
||||
_PS_CONST(cephes_log_q2, 0.693359375f);
|
||||
|
||||
#if defined (__MINGW32__)
|
||||
|
||||
/* the ugly part below: many versions of gcc used to be completely buggy with respect to some intrinsics
|
||||
The movehl_ps is fixed in mingw 3.4.5, but I found out that all the _mm_cmp* intrinsics were completely
|
||||
broken on my mingw gcc 3.4.5 ...
|
||||
|
||||
Note that the bug on _mm_cmp* does occur only at -O0 optimization level
|
||||
*/
|
||||
|
||||
inline __m128 my_movehl_ps(__m128 a, const __m128 b) {
|
||||
asm (
|
||||
"movhlps %2,%0\n\t"
|
||||
: "=x" (a)
|
||||
: "0" (a), "x"(b)
|
||||
);
|
||||
return a; }
|
||||
#warning "redefined _mm_movehl_ps (see gcc bug 21179)"
|
||||
#define _mm_movehl_ps my_movehl_ps
|
||||
|
||||
inline __m128 my_cmplt_ps(__m128 a, const __m128 b) {
|
||||
asm (
|
||||
"cmpltps %2,%0\n\t"
|
||||
: "=x" (a)
|
||||
: "0" (a), "x"(b)
|
||||
);
|
||||
return a;
|
||||
}
|
||||
inline __m128 my_cmpgt_ps(__m128 a, const __m128 b) {
|
||||
asm (
|
||||
"cmpnleps %2,%0\n\t"
|
||||
: "=x" (a)
|
||||
: "0" (a), "x"(b)
|
||||
);
|
||||
return a;
|
||||
}
|
||||
inline __m128 my_cmpeq_ps(__m128 a, const __m128 b) {
|
||||
asm (
|
||||
"cmpeqps %2,%0\n\t"
|
||||
: "=x" (a)
|
||||
: "0" (a), "x"(b)
|
||||
);
|
||||
return a;
|
||||
}
|
||||
#warning "redefined _mm_cmpxx_ps functions..."
|
||||
#define _mm_cmplt_ps my_cmplt_ps
|
||||
#define _mm_cmpgt_ps my_cmpgt_ps
|
||||
#define _mm_cmpeq_ps my_cmpeq_ps
|
||||
#endif
|
||||
|
||||
#ifndef USE_SSE2
|
||||
typedef union xmm_mm_union {
|
||||
__m128 xmm;
|
||||
__m64 mm[2];
|
||||
} xmm_mm_union;
|
||||
|
||||
#define COPY_XMM_TO_MM(xmm_, mm0_, mm1_) { \
|
||||
xmm_mm_union u; u.xmm = xmm_; \
|
||||
mm0_ = u.mm[0]; \
|
||||
mm1_ = u.mm[1]; \
|
||||
}
|
||||
|
||||
#define COPY_MM_TO_XMM(mm0_, mm1_, xmm_) { \
|
||||
xmm_mm_union u; u.mm[0]=mm0_; u.mm[1]=mm1_; xmm_ = u.xmm; \
|
||||
}
|
||||
|
||||
#endif // USE_SSE2
|
||||
|
||||
/* natural logarithm computed for 4 simultaneous float
|
||||
return NaN for x <= 0
|
||||
*/
|
||||
inline v4sf log_ps(v4sf x) {
|
||||
#ifdef USE_SSE2
|
||||
v4si emm0;
|
||||
#else
|
||||
v2si mm0, mm1;
|
||||
#endif
|
||||
v4sf one = *(v4sf*)_ps_1;
|
||||
|
||||
v4sf invalid_mask = _mm_cmple_ps(x, _mm_setzero_ps());
|
||||
|
||||
x = _mm_max_ps(x, *(v4sf*)_ps_min_norm_pos); /* cut off denormalized stuff */
|
||||
|
||||
#ifndef USE_SSE2
|
||||
/* part 1: x = frexpf(x, &e); */
|
||||
COPY_XMM_TO_MM(x, mm0, mm1);
|
||||
mm0 = _mm_srli_pi32(mm0, 23);
|
||||
mm1 = _mm_srli_pi32(mm1, 23);
|
||||
#else
|
||||
emm0 = _mm_srli_epi32(_mm_castps_si128(x), 23);
|
||||
#endif
|
||||
/* keep only the fractional part */
|
||||
x = _mm_and_ps(x, *(v4sf*)_ps_inv_mant_mask);
|
||||
x = _mm_or_ps(x, *(v4sf*)_ps_0p5);
|
||||
|
||||
#ifndef USE_SSE2
|
||||
/* now e=mm0:mm1 contain the really base-2 exponent */
|
||||
mm0 = _mm_sub_pi32(mm0, *(v2si*)_pi32_0x7f);
|
||||
mm1 = _mm_sub_pi32(mm1, *(v2si*)_pi32_0x7f);
|
||||
v4sf e = _mm_cvtpi32x2_ps(mm0, mm1);
|
||||
_mm_empty(); /* bye bye mmx */
|
||||
#else
|
||||
emm0 = _mm_sub_epi32(emm0, *(v4si*)_pi32_0x7f);
|
||||
v4sf e = _mm_cvtepi32_ps(emm0);
|
||||
#endif
|
||||
|
||||
e = _mm_add_ps(e, one);
|
||||
|
||||
/* part2:
|
||||
if( x < SQRTHF ) {
|
||||
e -= 1;
|
||||
x = x + x - 1.0;
|
||||
} else { x = x - 1.0; }
|
||||
*/
|
||||
v4sf mask = _mm_cmplt_ps(x, *(v4sf*)_ps_cephes_SQRTHF);
|
||||
v4sf tmp = _mm_and_ps(x, mask);
|
||||
x = _mm_sub_ps(x, one);
|
||||
e = _mm_sub_ps(e, _mm_and_ps(one, mask));
|
||||
x = _mm_add_ps(x, tmp);
|
||||
|
||||
|
||||
v4sf z = _mm_mul_ps(x,x);
|
||||
|
||||
v4sf y = *(v4sf*)_ps_cephes_log_p0;
|
||||
y = _mm_mul_ps(y, x);
|
||||
y = _mm_add_ps(y, *(v4sf*)_ps_cephes_log_p1);
|
||||
y = _mm_mul_ps(y, x);
|
||||
y = _mm_add_ps(y, *(v4sf*)_ps_cephes_log_p2);
|
||||
y = _mm_mul_ps(y, x);
|
||||
y = _mm_add_ps(y, *(v4sf*)_ps_cephes_log_p3);
|
||||
y = _mm_mul_ps(y, x);
|
||||
y = _mm_add_ps(y, *(v4sf*)_ps_cephes_log_p4);
|
||||
y = _mm_mul_ps(y, x);
|
||||
y = _mm_add_ps(y, *(v4sf*)_ps_cephes_log_p5);
|
||||
y = _mm_mul_ps(y, x);
|
||||
y = _mm_add_ps(y, *(v4sf*)_ps_cephes_log_p6);
|
||||
y = _mm_mul_ps(y, x);
|
||||
y = _mm_add_ps(y, *(v4sf*)_ps_cephes_log_p7);
|
||||
y = _mm_mul_ps(y, x);
|
||||
y = _mm_add_ps(y, *(v4sf*)_ps_cephes_log_p8);
|
||||
y = _mm_mul_ps(y, x);
|
||||
|
||||
y = _mm_mul_ps(y, z);
|
||||
|
||||
|
||||
tmp = _mm_mul_ps(e, *(v4sf*)_ps_cephes_log_q1);
|
||||
y = _mm_add_ps(y, tmp);
|
||||
|
||||
|
||||
tmp = _mm_mul_ps(z, *(v4sf*)_ps_0p5);
|
||||
y = _mm_sub_ps(y, tmp);
|
||||
|
||||
tmp = _mm_mul_ps(e, *(v4sf*)_ps_cephes_log_q2);
|
||||
x = _mm_add_ps(x, y);
|
||||
x = _mm_add_ps(x, tmp);
|
||||
x = _mm_or_ps(x, invalid_mask); // negative arg will be NAN
|
||||
return x;
|
||||
}
|
||||
|
||||
_PS_CONST(exp_hi, 88.3762626647949f);
|
||||
_PS_CONST(exp_lo, -88.3762626647949f);
|
||||
|
||||
_PS_CONST(cephes_LOG2EF, 1.44269504088896341f);
|
||||
_PS_CONST(cephes_exp_C1, 0.693359375f);
|
||||
_PS_CONST(cephes_exp_C2, -2.12194440e-4f);
|
||||
|
||||
_PS_CONST(cephes_exp_p0, 1.9875691500E-4f);
|
||||
_PS_CONST(cephes_exp_p1, 1.3981999507E-3f);
|
||||
_PS_CONST(cephes_exp_p2, 8.3334519073E-3f);
|
||||
_PS_CONST(cephes_exp_p3, 4.1665795894E-2f);
|
||||
_PS_CONST(cephes_exp_p4, 1.6666665459E-1f);
|
||||
_PS_CONST(cephes_exp_p5, 5.0000001201E-1f);
|
||||
|
||||
inline v4sf exp_ps(v4sf x) {
|
||||
v4sf tmp = _mm_setzero_ps(), fx;
|
||||
#ifdef USE_SSE2
|
||||
v4si emm0;
|
||||
#else
|
||||
v2si mm0, mm1;
|
||||
#endif
|
||||
v4sf one = *(v4sf*)_ps_1;
|
||||
|
||||
x = _mm_min_ps(x, *(v4sf*)_ps_exp_hi);
|
||||
x = _mm_max_ps(x, *(v4sf*)_ps_exp_lo);
|
||||
|
||||
/* express exp(x) as exp(g + n*log(2)) */
|
||||
fx = _mm_mul_ps(x, *(v4sf*)_ps_cephes_LOG2EF);
|
||||
fx = _mm_add_ps(fx, *(v4sf*)_ps_0p5);
|
||||
|
||||
/* how to perform a floorf with SSE: just below */
|
||||
#ifndef USE_SSE2
|
||||
/* step 1 : cast to int */
|
||||
tmp = _mm_movehl_ps(tmp, fx);
|
||||
mm0 = _mm_cvttps_pi32(fx);
|
||||
mm1 = _mm_cvttps_pi32(tmp);
|
||||
/* step 2 : cast back to float */
|
||||
tmp = _mm_cvtpi32x2_ps(mm0, mm1);
|
||||
#else
|
||||
emm0 = _mm_cvttps_epi32(fx);
|
||||
tmp = _mm_cvtepi32_ps(emm0);
|
||||
#endif
|
||||
/* if greater, substract 1 */
|
||||
v4sf mask = _mm_cmpgt_ps(tmp, fx);
|
||||
mask = _mm_and_ps(mask, one);
|
||||
fx = _mm_sub_ps(tmp, mask);
|
||||
|
||||
tmp = _mm_mul_ps(fx, *(v4sf*)_ps_cephes_exp_C1);
|
||||
v4sf z = _mm_mul_ps(fx, *(v4sf*)_ps_cephes_exp_C2);
|
||||
x = _mm_sub_ps(x, tmp);
|
||||
x = _mm_sub_ps(x, z);
|
||||
|
||||
z = _mm_mul_ps(x,x);
|
||||
|
||||
v4sf y = *(v4sf*)_ps_cephes_exp_p0;
|
||||
y = _mm_mul_ps(y, x);
|
||||
y = _mm_add_ps(y, *(v4sf*)_ps_cephes_exp_p1);
|
||||
y = _mm_mul_ps(y, x);
|
||||
y = _mm_add_ps(y, *(v4sf*)_ps_cephes_exp_p2);
|
||||
y = _mm_mul_ps(y, x);
|
||||
y = _mm_add_ps(y, *(v4sf*)_ps_cephes_exp_p3);
|
||||
y = _mm_mul_ps(y, x);
|
||||
y = _mm_add_ps(y, *(v4sf*)_ps_cephes_exp_p4);
|
||||
y = _mm_mul_ps(y, x);
|
||||
y = _mm_add_ps(y, *(v4sf*)_ps_cephes_exp_p5);
|
||||
y = _mm_mul_ps(y, z);
|
||||
y = _mm_add_ps(y, x);
|
||||
y = _mm_add_ps(y, one);
|
||||
|
||||
/* build 2^n */
|
||||
#ifndef USE_SSE2
|
||||
z = _mm_movehl_ps(z, fx);
|
||||
mm0 = _mm_cvttps_pi32(fx);
|
||||
mm1 = _mm_cvttps_pi32(z);
|
||||
mm0 = _mm_add_pi32(mm0, *(v2si*)_pi32_0x7f);
|
||||
mm1 = _mm_add_pi32(mm1, *(v2si*)_pi32_0x7f);
|
||||
mm0 = _mm_slli_pi32(mm0, 23);
|
||||
mm1 = _mm_slli_pi32(mm1, 23);
|
||||
|
||||
v4sf pow2n;
|
||||
COPY_MM_TO_XMM(mm0, mm1, pow2n);
|
||||
_mm_empty();
|
||||
#else
|
||||
emm0 = _mm_cvttps_epi32(fx);
|
||||
emm0 = _mm_add_epi32(emm0, *(v4si*)_pi32_0x7f);
|
||||
emm0 = _mm_slli_epi32(emm0, 23);
|
||||
v4sf pow2n = _mm_castsi128_ps(emm0);
|
||||
#endif
|
||||
y = _mm_mul_ps(y, pow2n);
|
||||
return y;
|
||||
}
|
||||
|
||||
_PS_CONST(minus_cephes_DP1, -0.78515625f);
|
||||
_PS_CONST(minus_cephes_DP2, -2.4187564849853515625e-4f);
|
||||
_PS_CONST(minus_cephes_DP3, -3.77489497744594108e-8f);
|
||||
_PS_CONST(sincof_p0, -1.9515295891E-4f);
|
||||
_PS_CONST(sincof_p1, 8.3321608736E-3f);
|
||||
_PS_CONST(sincof_p2, -1.6666654611E-1f);
|
||||
_PS_CONST(coscof_p0, 2.443315711809948E-005f);
|
||||
_PS_CONST(coscof_p1, -1.388731625493765E-003f);
|
||||
_PS_CONST(coscof_p2, 4.166664568298827E-002f);
|
||||
_PS_CONST(cephes_FOPI, 1.27323954473516f); // 4 / M_PI
|
||||
|
||||
|
||||
/* evaluation of 4 sines at onces, using only SSE1+MMX intrinsics so
|
||||
it runs also on old athlons XPs and the pentium III of your grand
|
||||
mother.
|
||||
|
||||
The code is the exact rewriting of the cephes sinf function.
|
||||
Precision is excellent as long as x < 8192 (I did not bother to
|
||||
take into account the special handling they have for greater values
|
||||
-- it does not return garbage for arguments over 8192, though, but
|
||||
the extra precision is missing).
|
||||
|
||||
Note that it is such that sinf((float)M_PI) = 8.74e-8, which is the
|
||||
surprising but correct result.
|
||||
|
||||
Performance is also surprisingly good, 1.33 times faster than the
|
||||
macos vsinf SSE2 function, and 1.5 times faster than the
|
||||
__vrs4_sinf of amd's ACML (which is only available in 64 bits). Not
|
||||
too bad for an SSE1 function (with no special tuning) !
|
||||
However the latter libraries probably have a much better handling of NaN,
|
||||
Inf, denormalized and other special arguments..
|
||||
|
||||
On my core 1 duo, the execution of this function takes approximately 95 cycles.
|
||||
|
||||
From what I have observed on the experiments with Intel AMath lib, switching to an
|
||||
SSE2 version would improve the perf by only 10%.
|
||||
|
||||
Since it is based on SSE intrinsics, it has to be compiled at -O2 to
|
||||
deliver full speed.
|
||||
*/
|
||||
inline v4sf sin_ps(v4sf x) { // any x
|
||||
v4sf xmm1, xmm2 = _mm_setzero_ps(), xmm3, sign_bit, y;
|
||||
|
||||
#ifdef USE_SSE2
|
||||
v4si emm0, emm2;
|
||||
#else
|
||||
v2si mm0, mm1, mm2, mm3;
|
||||
#endif
|
||||
sign_bit = x;
|
||||
/* take the absolute value */
|
||||
x = _mm_and_ps(x, *(v4sf*)_ps_inv_sign_mask);
|
||||
/* extract the sign bit (upper one) */
|
||||
sign_bit = _mm_and_ps(sign_bit, *(v4sf*)_ps_sign_mask);
|
||||
|
||||
/* scale by 4/Pi */
|
||||
y = _mm_mul_ps(x, *(v4sf*)_ps_cephes_FOPI);
|
||||
|
||||
//printf("plop:"); print4(y);
|
||||
#ifdef USE_SSE2
|
||||
/* store the integer part of y in mm0 */
|
||||
emm2 = _mm_cvttps_epi32(y);
|
||||
/* j=(j+1) & (~1) (see the cephes sources) */
|
||||
emm2 = _mm_add_epi32(emm2, *(v4si*)_pi32_1);
|
||||
emm2 = _mm_and_si128(emm2, *(v4si*)_pi32_inv1);
|
||||
y = _mm_cvtepi32_ps(emm2);
|
||||
/* get the swap sign flag */
|
||||
emm0 = _mm_and_si128(emm2, *(v4si*)_pi32_4);
|
||||
emm0 = _mm_slli_epi32(emm0, 29);
|
||||
/* get the polynom selection mask
|
||||
there is one polynom for 0 <= x <= Pi/4
|
||||
and another one for Pi/4<x<=Pi/2
|
||||
|
||||
Both branches will be computed.
|
||||
*/
|
||||
emm2 = _mm_and_si128(emm2, *(v4si*)_pi32_2);
|
||||
emm2 = _mm_cmpeq_epi32(emm2, _mm_setzero_si128());
|
||||
|
||||
v4sf swap_sign_bit = _mm_castsi128_ps(emm0);
|
||||
v4sf poly_mask = _mm_castsi128_ps(emm2);
|
||||
sign_bit = _mm_xor_ps(sign_bit, swap_sign_bit);
|
||||
#else
|
||||
/* store the integer part of y in mm0:mm1 */
|
||||
xmm2 = _mm_movehl_ps(xmm2, y);
|
||||
mm2 = _mm_cvttps_pi32(y);
|
||||
mm3 = _mm_cvttps_pi32(xmm2);
|
||||
/* j=(j+1) & (~1) (see the cephes sources) */
|
||||
mm2 = _mm_add_pi32(mm2, *(v2si*)_pi32_1);
|
||||
mm3 = _mm_add_pi32(mm3, *(v2si*)_pi32_1);
|
||||
mm2 = _mm_and_si64(mm2, *(v2si*)_pi32_inv1);
|
||||
mm3 = _mm_and_si64(mm3, *(v2si*)_pi32_inv1);
|
||||
y = _mm_cvtpi32x2_ps(mm2, mm3);
|
||||
/* get the swap sign flag */
|
||||
mm0 = _mm_and_si64(mm2, *(v2si*)_pi32_4);
|
||||
mm1 = _mm_and_si64(mm3, *(v2si*)_pi32_4);
|
||||
mm0 = _mm_slli_pi32(mm0, 29);
|
||||
mm1 = _mm_slli_pi32(mm1, 29);
|
||||
/* get the polynom selection mask */
|
||||
mm2 = _mm_and_si64(mm2, *(v2si*)_pi32_2);
|
||||
mm3 = _mm_and_si64(mm3, *(v2si*)_pi32_2);
|
||||
mm2 = _mm_cmpeq_pi32(mm2, _mm_setzero_si64());
|
||||
mm3 = _mm_cmpeq_pi32(mm3, _mm_setzero_si64());
|
||||
v4sf swap_sign_bit, poly_mask;
|
||||
COPY_MM_TO_XMM(mm0, mm1, swap_sign_bit);
|
||||
COPY_MM_TO_XMM(mm2, mm3, poly_mask);
|
||||
sign_bit = _mm_xor_ps(sign_bit, swap_sign_bit);
|
||||
_mm_empty(); /* good-bye mmx */
|
||||
#endif
|
||||
|
||||
/* The magic pass: "Extended precision modular arithmetic"
|
||||
x = ((x - y * DP1) - y * DP2) - y * DP3; */
|
||||
xmm1 = *(v4sf*)_ps_minus_cephes_DP1;
|
||||
xmm2 = *(v4sf*)_ps_minus_cephes_DP2;
|
||||
xmm3 = *(v4sf*)_ps_minus_cephes_DP3;
|
||||
xmm1 = _mm_mul_ps(y, xmm1);
|
||||
xmm2 = _mm_mul_ps(y, xmm2);
|
||||
xmm3 = _mm_mul_ps(y, xmm3);
|
||||
x = _mm_add_ps(x, xmm1);
|
||||
x = _mm_add_ps(x, xmm2);
|
||||
x = _mm_add_ps(x, xmm3);
|
||||
|
||||
/* Evaluate the first polynom (0 <= x <= Pi/4) */
|
||||
y = *(v4sf*)_ps_coscof_p0;
|
||||
v4sf z = _mm_mul_ps(x,x);
|
||||
|
||||
y = _mm_mul_ps(y, z);
|
||||
y = _mm_add_ps(y, *(v4sf*)_ps_coscof_p1);
|
||||
y = _mm_mul_ps(y, z);
|
||||
y = _mm_add_ps(y, *(v4sf*)_ps_coscof_p2);
|
||||
y = _mm_mul_ps(y, z);
|
||||
y = _mm_mul_ps(y, z);
|
||||
v4sf tmp = _mm_mul_ps(z, *(v4sf*)_ps_0p5);
|
||||
y = _mm_sub_ps(y, tmp);
|
||||
y = _mm_add_ps(y, *(v4sf*)_ps_1);
|
||||
|
||||
/* Evaluate the second polynom (Pi/4 <= x <= 0) */
|
||||
|
||||
v4sf y2 = *(v4sf*)_ps_sincof_p0;
|
||||
y2 = _mm_mul_ps(y2, z);
|
||||
y2 = _mm_add_ps(y2, *(v4sf*)_ps_sincof_p1);
|
||||
y2 = _mm_mul_ps(y2, z);
|
||||
y2 = _mm_add_ps(y2, *(v4sf*)_ps_sincof_p2);
|
||||
y2 = _mm_mul_ps(y2, z);
|
||||
y2 = _mm_mul_ps(y2, x);
|
||||
y2 = _mm_add_ps(y2, x);
|
||||
|
||||
/* select the correct result from the two polynoms */
|
||||
xmm3 = poly_mask;
|
||||
y2 = _mm_and_ps(xmm3, y2); //, xmm3);
|
||||
y = _mm_andnot_ps(xmm3, y);
|
||||
y = _mm_add_ps(y,y2);
|
||||
/* update the sign */
|
||||
y = _mm_xor_ps(y, sign_bit);
|
||||
|
||||
return y;
|
||||
}
|
||||
|
||||
/* almost the same as sin_ps */
|
||||
inline v4sf cos_ps(v4sf x) { // any x
|
||||
v4sf xmm1, xmm2 = _mm_setzero_ps(), xmm3, y;
|
||||
#ifdef USE_SSE2
|
||||
v4si emm0, emm2;
|
||||
#else
|
||||
v2si mm0, mm1, mm2, mm3;
|
||||
#endif
|
||||
/* take the absolute value */
|
||||
x = _mm_and_ps(x, *(v4sf*)_ps_inv_sign_mask);
|
||||
|
||||
/* scale by 4/Pi */
|
||||
y = _mm_mul_ps(x, *(v4sf*)_ps_cephes_FOPI);
|
||||
|
||||
#ifdef USE_SSE2
|
||||
/* store the integer part of y in mm0 */
|
||||
emm2 = _mm_cvttps_epi32(y);
|
||||
/* j=(j+1) & (~1) (see the cephes sources) */
|
||||
emm2 = _mm_add_epi32(emm2, *(v4si*)_pi32_1);
|
||||
emm2 = _mm_and_si128(emm2, *(v4si*)_pi32_inv1);
|
||||
y = _mm_cvtepi32_ps(emm2);
|
||||
|
||||
emm2 = _mm_sub_epi32(emm2, *(v4si*)_pi32_2);
|
||||
|
||||
/* get the swap sign flag */
|
||||
emm0 = _mm_andnot_si128(emm2, *(v4si*)_pi32_4);
|
||||
emm0 = _mm_slli_epi32(emm0, 29);
|
||||
/* get the polynom selection mask */
|
||||
emm2 = _mm_and_si128(emm2, *(v4si*)_pi32_2);
|
||||
emm2 = _mm_cmpeq_epi32(emm2, _mm_setzero_si128());
|
||||
|
||||
v4sf sign_bit = _mm_castsi128_ps(emm0);
|
||||
v4sf poly_mask = _mm_castsi128_ps(emm2);
|
||||
#else
|
||||
/* store the integer part of y in mm0:mm1 */
|
||||
xmm2 = _mm_movehl_ps(xmm2, y);
|
||||
mm2 = _mm_cvttps_pi32(y);
|
||||
mm3 = _mm_cvttps_pi32(xmm2);
|
||||
|
||||
/* j=(j+1) & (~1) (see the cephes sources) */
|
||||
mm2 = _mm_add_pi32(mm2, *(v2si*)_pi32_1);
|
||||
mm3 = _mm_add_pi32(mm3, *(v2si*)_pi32_1);
|
||||
mm2 = _mm_and_si64(mm2, *(v2si*)_pi32_inv1);
|
||||
mm3 = _mm_and_si64(mm3, *(v2si*)_pi32_inv1);
|
||||
|
||||
y = _mm_cvtpi32x2_ps(mm2, mm3);
|
||||
|
||||
|
||||
mm2 = _mm_sub_pi32(mm2, *(v2si*)_pi32_2);
|
||||
mm3 = _mm_sub_pi32(mm3, *(v2si*)_pi32_2);
|
||||
|
||||
/* get the swap sign flag in mm0:mm1 and the
|
||||
polynom selection mask in mm2:mm3 */
|
||||
|
||||
mm0 = _mm_andnot_si64(mm2, *(v2si*)_pi32_4);
|
||||
mm1 = _mm_andnot_si64(mm3, *(v2si*)_pi32_4);
|
||||
mm0 = _mm_slli_pi32(mm0, 29);
|
||||
mm1 = _mm_slli_pi32(mm1, 29);
|
||||
|
||||
mm2 = _mm_and_si64(mm2, *(v2si*)_pi32_2);
|
||||
mm3 = _mm_and_si64(mm3, *(v2si*)_pi32_2);
|
||||
|
||||
mm2 = _mm_cmpeq_pi32(mm2, _mm_setzero_si64());
|
||||
mm3 = _mm_cmpeq_pi32(mm3, _mm_setzero_si64());
|
||||
|
||||
v4sf sign_bit, poly_mask;
|
||||
COPY_MM_TO_XMM(mm0, mm1, sign_bit);
|
||||
COPY_MM_TO_XMM(mm2, mm3, poly_mask);
|
||||
_mm_empty(); /* good-bye mmx */
|
||||
#endif
|
||||
/* The magic pass: "Extended precision modular arithmetic"
|
||||
x = ((x - y * DP1) - y * DP2) - y * DP3; */
|
||||
xmm1 = *(v4sf*)_ps_minus_cephes_DP1;
|
||||
xmm2 = *(v4sf*)_ps_minus_cephes_DP2;
|
||||
xmm3 = *(v4sf*)_ps_minus_cephes_DP3;
|
||||
xmm1 = _mm_mul_ps(y, xmm1);
|
||||
xmm2 = _mm_mul_ps(y, xmm2);
|
||||
xmm3 = _mm_mul_ps(y, xmm3);
|
||||
x = _mm_add_ps(x, xmm1);
|
||||
x = _mm_add_ps(x, xmm2);
|
||||
x = _mm_add_ps(x, xmm3);
|
||||
|
||||
/* Evaluate the first polynom (0 <= x <= Pi/4) */
|
||||
y = *(v4sf*)_ps_coscof_p0;
|
||||
v4sf z = _mm_mul_ps(x,x);
|
||||
|
||||
y = _mm_mul_ps(y, z);
|
||||
y = _mm_add_ps(y, *(v4sf*)_ps_coscof_p1);
|
||||
y = _mm_mul_ps(y, z);
|
||||
y = _mm_add_ps(y, *(v4sf*)_ps_coscof_p2);
|
||||
y = _mm_mul_ps(y, z);
|
||||
y = _mm_mul_ps(y, z);
|
||||
v4sf tmp = _mm_mul_ps(z, *(v4sf*)_ps_0p5);
|
||||
y = _mm_sub_ps(y, tmp);
|
||||
y = _mm_add_ps(y, *(v4sf*)_ps_1);
|
||||
|
||||
/* Evaluate the second polynom (Pi/4 <= x <= 0) */
|
||||
|
||||
v4sf y2 = *(v4sf*)_ps_sincof_p0;
|
||||
y2 = _mm_mul_ps(y2, z);
|
||||
y2 = _mm_add_ps(y2, *(v4sf*)_ps_sincof_p1);
|
||||
y2 = _mm_mul_ps(y2, z);
|
||||
y2 = _mm_add_ps(y2, *(v4sf*)_ps_sincof_p2);
|
||||
y2 = _mm_mul_ps(y2, z);
|
||||
y2 = _mm_mul_ps(y2, x);
|
||||
y2 = _mm_add_ps(y2, x);
|
||||
|
||||
/* select the correct result from the two polynoms */
|
||||
xmm3 = poly_mask;
|
||||
y2 = _mm_and_ps(xmm3, y2); //, xmm3);
|
||||
y = _mm_andnot_ps(xmm3, y);
|
||||
y = _mm_add_ps(y,y2);
|
||||
/* update the sign */
|
||||
y = _mm_xor_ps(y, sign_bit);
|
||||
|
||||
return y;
|
||||
}
|
||||
|
||||
/* since sin_ps and cos_ps are almost identical, sincos_ps could replace both of them..
|
||||
it is almost as fast, and gives you a free cosine with your sine */
|
||||
inline void sincos_ps(v4sf x, v4sf *s, v4sf *c) {
|
||||
v4sf xmm1, xmm2, xmm3 = _mm_setzero_ps(), sign_bit_sin, y;
|
||||
#ifdef USE_SSE2
|
||||
v4si emm0, emm2, emm4;
|
||||
#else
|
||||
v2si mm0, mm1, mm2, mm3, mm4, mm5;
|
||||
#endif
|
||||
sign_bit_sin = x;
|
||||
/* take the absolute value */
|
||||
x = _mm_and_ps(x, *(v4sf*)_ps_inv_sign_mask);
|
||||
/* extract the sign bit (upper one) */
|
||||
sign_bit_sin = _mm_and_ps(sign_bit_sin, *(v4sf*)_ps_sign_mask);
|
||||
|
||||
/* scale by 4/Pi */
|
||||
y = _mm_mul_ps(x, *(v4sf*)_ps_cephes_FOPI);
|
||||
|
||||
#ifdef USE_SSE2
|
||||
/* store the integer part of y in emm2 */
|
||||
emm2 = _mm_cvttps_epi32(y);
|
||||
|
||||
/* j=(j+1) & (~1) (see the cephes sources) */
|
||||
emm2 = _mm_add_epi32(emm2, *(v4si*)_pi32_1);
|
||||
emm2 = _mm_and_si128(emm2, *(v4si*)_pi32_inv1);
|
||||
y = _mm_cvtepi32_ps(emm2);
|
||||
|
||||
emm4 = emm2;
|
||||
|
||||
/* get the swap sign flag for the sine */
|
||||
emm0 = _mm_and_si128(emm2, *(v4si*)_pi32_4);
|
||||
emm0 = _mm_slli_epi32(emm0, 29);
|
||||
v4sf swap_sign_bit_sin = _mm_castsi128_ps(emm0);
|
||||
|
||||
/* get the polynom selection mask for the sine*/
|
||||
emm2 = _mm_and_si128(emm2, *(v4si*)_pi32_2);
|
||||
emm2 = _mm_cmpeq_epi32(emm2, _mm_setzero_si128());
|
||||
v4sf poly_mask = _mm_castsi128_ps(emm2);
|
||||
#else
|
||||
/* store the integer part of y in mm2:mm3 */
|
||||
xmm3 = _mm_movehl_ps(xmm3, y);
|
||||
mm2 = _mm_cvttps_pi32(y);
|
||||
mm3 = _mm_cvttps_pi32(xmm3);
|
||||
|
||||
/* j=(j+1) & (~1) (see the cephes sources) */
|
||||
mm2 = _mm_add_pi32(mm2, *(v2si*)_pi32_1);
|
||||
mm3 = _mm_add_pi32(mm3, *(v2si*)_pi32_1);
|
||||
mm2 = _mm_and_si64(mm2, *(v2si*)_pi32_inv1);
|
||||
mm3 = _mm_and_si64(mm3, *(v2si*)_pi32_inv1);
|
||||
|
||||
y = _mm_cvtpi32x2_ps(mm2, mm3);
|
||||
|
||||
mm4 = mm2;
|
||||
mm5 = mm3;
|
||||
|
||||
/* get the swap sign flag for the sine */
|
||||
mm0 = _mm_and_si64(mm2, *(v2si*)_pi32_4);
|
||||
mm1 = _mm_and_si64(mm3, *(v2si*)_pi32_4);
|
||||
mm0 = _mm_slli_pi32(mm0, 29);
|
||||
mm1 = _mm_slli_pi32(mm1, 29);
|
||||
v4sf swap_sign_bit_sin;
|
||||
COPY_MM_TO_XMM(mm0, mm1, swap_sign_bit_sin);
|
||||
|
||||
/* get the polynom selection mask for the sine */
|
||||
|
||||
mm2 = _mm_and_si64(mm2, *(v2si*)_pi32_2);
|
||||
mm3 = _mm_and_si64(mm3, *(v2si*)_pi32_2);
|
||||
mm2 = _mm_cmpeq_pi32(mm2, _mm_setzero_si64());
|
||||
mm3 = _mm_cmpeq_pi32(mm3, _mm_setzero_si64());
|
||||
v4sf poly_mask;
|
||||
COPY_MM_TO_XMM(mm2, mm3, poly_mask);
|
||||
#endif
|
||||
|
||||
/* The magic pass: "Extended precision modular arithmetic"
|
||||
x = ((x - y * DP1) - y * DP2) - y * DP3; */
|
||||
xmm1 = *(v4sf*)_ps_minus_cephes_DP1;
|
||||
xmm2 = *(v4sf*)_ps_minus_cephes_DP2;
|
||||
xmm3 = *(v4sf*)_ps_minus_cephes_DP3;
|
||||
xmm1 = _mm_mul_ps(y, xmm1);
|
||||
xmm2 = _mm_mul_ps(y, xmm2);
|
||||
xmm3 = _mm_mul_ps(y, xmm3);
|
||||
x = _mm_add_ps(x, xmm1);
|
||||
x = _mm_add_ps(x, xmm2);
|
||||
x = _mm_add_ps(x, xmm3);
|
||||
|
||||
#ifdef USE_SSE2
|
||||
emm4 = _mm_sub_epi32(emm4, *(v4si*)_pi32_2);
|
||||
emm4 = _mm_andnot_si128(emm4, *(v4si*)_pi32_4);
|
||||
emm4 = _mm_slli_epi32(emm4, 29);
|
||||
v4sf sign_bit_cos = _mm_castsi128_ps(emm4);
|
||||
#else
|
||||
/* get the sign flag for the cosine */
|
||||
mm4 = _mm_sub_pi32(mm4, *(v2si*)_pi32_2);
|
||||
mm5 = _mm_sub_pi32(mm5, *(v2si*)_pi32_2);
|
||||
mm4 = _mm_andnot_si64(mm4, *(v2si*)_pi32_4);
|
||||
mm5 = _mm_andnot_si64(mm5, *(v2si*)_pi32_4);
|
||||
mm4 = _mm_slli_pi32(mm4, 29);
|
||||
mm5 = _mm_slli_pi32(mm5, 29);
|
||||
v4sf sign_bit_cos;
|
||||
COPY_MM_TO_XMM(mm4, mm5, sign_bit_cos);
|
||||
_mm_empty(); /* good-bye mmx */
|
||||
#endif
|
||||
|
||||
sign_bit_sin = _mm_xor_ps(sign_bit_sin, swap_sign_bit_sin);
|
||||
|
||||
|
||||
/* Evaluate the first polynom (0 <= x <= Pi/4) */
|
||||
v4sf z = _mm_mul_ps(x,x);
|
||||
y = *(v4sf*)_ps_coscof_p0;
|
||||
|
||||
y = _mm_mul_ps(y, z);
|
||||
y = _mm_add_ps(y, *(v4sf*)_ps_coscof_p1);
|
||||
y = _mm_mul_ps(y, z);
|
||||
y = _mm_add_ps(y, *(v4sf*)_ps_coscof_p2);
|
||||
y = _mm_mul_ps(y, z);
|
||||
y = _mm_mul_ps(y, z);
|
||||
v4sf tmp = _mm_mul_ps(z, *(v4sf*)_ps_0p5);
|
||||
y = _mm_sub_ps(y, tmp);
|
||||
y = _mm_add_ps(y, *(v4sf*)_ps_1);
|
||||
|
||||
/* Evaluate the second polynom (Pi/4 <= x <= 0) */
|
||||
|
||||
v4sf y2 = *(v4sf*)_ps_sincof_p0;
|
||||
y2 = _mm_mul_ps(y2, z);
|
||||
y2 = _mm_add_ps(y2, *(v4sf*)_ps_sincof_p1);
|
||||
y2 = _mm_mul_ps(y2, z);
|
||||
y2 = _mm_add_ps(y2, *(v4sf*)_ps_sincof_p2);
|
||||
y2 = _mm_mul_ps(y2, z);
|
||||
y2 = _mm_mul_ps(y2, x);
|
||||
y2 = _mm_add_ps(y2, x);
|
||||
|
||||
/* select the correct result from the two polynoms */
|
||||
xmm3 = poly_mask;
|
||||
v4sf ysin2 = _mm_and_ps(xmm3, y2);
|
||||
v4sf ysin1 = _mm_andnot_ps(xmm3, y);
|
||||
y2 = _mm_sub_ps(y2,ysin2);
|
||||
y = _mm_sub_ps(y, ysin1);
|
||||
|
||||
xmm1 = _mm_add_ps(ysin1,ysin2);
|
||||
xmm2 = _mm_add_ps(y,y2);
|
||||
|
||||
/* update the sign */
|
||||
*s = _mm_xor_ps(xmm1, sign_bit_sin);
|
||||
*c = _mm_xor_ps(xmm2, sign_bit_cos);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,163 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#pragma once
|
||||
|
||||
namespace embree
|
||||
{
|
||||
/*! 4-wide SSE bool type. */
|
||||
struct sseb
|
||||
{
|
||||
typedef sseb Mask; // mask type
|
||||
typedef ssei Int; // int type
|
||||
typedef ssef Float; // float type
|
||||
|
||||
enum { size = 4 }; // number of SIMD elements
|
||||
union { __m128 m128; int32 v[4]; }; // data
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constructors, Assignment & Cast Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline sseb ( ) {}
|
||||
__forceinline sseb ( const sseb& other ) { m128 = other.m128; }
|
||||
__forceinline sseb& operator=( const sseb& other ) { m128 = other.m128; return *this; }
|
||||
|
||||
__forceinline sseb( const __m128 input ) : m128(input) {}
|
||||
__forceinline operator const __m128&( void ) const { return m128; }
|
||||
__forceinline operator const __m128i( void ) const { return _mm_castps_si128(m128); }
|
||||
__forceinline operator const __m128d( void ) const { return _mm_castps_pd(m128); }
|
||||
|
||||
__forceinline sseb ( bool a )
|
||||
: m128(_mm_lookupmask_ps[(size_t(a) << 3) | (size_t(a) << 2) | (size_t(a) << 1) | size_t(a)]) {}
|
||||
__forceinline sseb ( bool a, bool b)
|
||||
: m128(_mm_lookupmask_ps[(size_t(b) << 3) | (size_t(a) << 2) | (size_t(b) << 1) | size_t(a)]) {}
|
||||
__forceinline sseb ( bool a, bool b, bool c, bool d)
|
||||
: m128(_mm_lookupmask_ps[(size_t(d) << 3) | (size_t(c) << 2) | (size_t(b) << 1) | size_t(a)]) {}
|
||||
__forceinline sseb(int mask) {
|
||||
assert(mask >= 0 && mask < 16);
|
||||
m128 = _mm_lookupmask_ps[mask];
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constants
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline sseb( FalseTy ) : m128(_mm_setzero_ps()) {}
|
||||
__forceinline sseb( TrueTy ) : m128(_mm_castsi128_ps(_mm_cmpeq_epi32(_mm_setzero_si128(), _mm_setzero_si128()))) {}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Array Access
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline bool operator []( const size_t i ) const { assert(i < 4); return (_mm_movemask_ps(m128) >> i) & 1; }
|
||||
__forceinline int32& operator []( const size_t i ) { assert(i < 4); return v[i]; }
|
||||
};
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Unary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const sseb operator !( const sseb& a ) { return _mm_xor_ps(a, sseb(True)); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Binary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const sseb operator &( const sseb& a, const sseb& b ) { return _mm_and_ps(a, b); }
|
||||
__forceinline const sseb operator |( const sseb& a, const sseb& b ) { return _mm_or_ps (a, b); }
|
||||
__forceinline const sseb operator ^( const sseb& a, const sseb& b ) { return _mm_xor_ps(a, b); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Assignment Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const sseb operator &=( sseb& a, const sseb& b ) { return a = a & b; }
|
||||
__forceinline const sseb operator |=( sseb& a, const sseb& b ) { return a = a | b; }
|
||||
__forceinline const sseb operator ^=( sseb& a, const sseb& b ) { return a = a ^ b; }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Comparison Operators + Select
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const sseb operator !=( const sseb& a, const sseb& b ) { return _mm_xor_ps(a, b); }
|
||||
__forceinline const sseb operator ==( const sseb& a, const sseb& b ) { return _mm_castsi128_ps(_mm_cmpeq_epi32(a, b)); }
|
||||
|
||||
__forceinline const sseb select( const sseb& m, const sseb& t, const sseb& f ) {
|
||||
#if defined(__SSE4_1__)
|
||||
return _mm_blendv_ps(f, t, m);
|
||||
#else
|
||||
return _mm_or_ps(_mm_and_ps(m, t), _mm_andnot_ps(m, f));
|
||||
#endif
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Movement/Shifting/Shuffling Functions
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const sseb unpacklo( const sseb& a, const sseb& b ) { return _mm_unpacklo_ps(a, b); }
|
||||
__forceinline const sseb unpackhi( const sseb& a, const sseb& b ) { return _mm_unpackhi_ps(a, b); }
|
||||
|
||||
template<size_t i0, size_t i1, size_t i2, size_t i3> __forceinline const sseb shuffle( const sseb& a ) {
|
||||
return _mm_shuffle_epi32(a, _MM_SHUFFLE(i3, i2, i1, i0));
|
||||
}
|
||||
|
||||
template<size_t i0, size_t i1, size_t i2, size_t i3> __forceinline const sseb shuffle( const sseb& a, const sseb& b ) {
|
||||
return _mm_shuffle_ps(a, b, _MM_SHUFFLE(i3, i2, i1, i0));
|
||||
}
|
||||
|
||||
#if defined(__SSE3__)
|
||||
template<> __forceinline const sseb shuffle<0, 0, 2, 2>( const sseb& a ) { return _mm_moveldup_ps(a); }
|
||||
template<> __forceinline const sseb shuffle<1, 1, 3, 3>( const sseb& a ) { return _mm_movehdup_ps(a); }
|
||||
template<> __forceinline const sseb shuffle<0, 1, 0, 1>( const sseb& a ) { return _mm_castpd_ps(_mm_movedup_pd (a)); }
|
||||
#endif
|
||||
|
||||
#if defined(__SSE4_1__)
|
||||
template<size_t dst, size_t src, size_t clr> __forceinline const sseb insert( const sseb& a, const sseb& b ) { return _mm_insert_ps(a, b, (dst << 4) | (src << 6) | clr); }
|
||||
template<size_t dst, size_t src> __forceinline const sseb insert( const sseb& a, const sseb& b ) { return insert<dst, src, 0>(a, b); }
|
||||
template<size_t dst> __forceinline const sseb insert( const sseb& a, const bool b ) { return insert<dst,0>(a, sseb(b)); }
|
||||
#endif
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Reduction Operations
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline bool reduce_and( const sseb& a ) { return _mm_movemask_ps(a) == 0xf; }
|
||||
__forceinline bool reduce_or ( const sseb& a ) { return _mm_movemask_ps(a) != 0x0; }
|
||||
|
||||
__forceinline bool all ( const sseb& b ) { return _mm_movemask_ps(b) == 0xf; }
|
||||
__forceinline bool any ( const sseb& b ) { return _mm_movemask_ps(b) != 0x0; }
|
||||
__forceinline bool none ( const sseb& b ) { return _mm_movemask_ps(b) == 0x0; }
|
||||
|
||||
__forceinline bool all ( const sseb& valid, const sseb& b ) { return all(!valid | b); }
|
||||
__forceinline bool any ( const sseb& valid, const sseb& b ) { return any( valid & b); }
|
||||
__forceinline bool none ( const sseb& valid, const sseb& b ) { return none(valid & b); }
|
||||
|
||||
__forceinline size_t movemask( const sseb& a ) { return _mm_movemask_ps(a); }
|
||||
#if defined(__SSE4_2__)
|
||||
__forceinline size_t popcnt( const sseb& a ) { return __popcnt(_mm_movemask_ps(a)); }
|
||||
#else
|
||||
__forceinline size_t popcnt( const sseb& a ) { return bool(a[0])+bool(a[1])+bool(a[2])+bool(a[3]); }
|
||||
#endif
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Output Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
inline std::ostream& operator<<(std::ostream& cout, const sseb& a) {
|
||||
return cout << "<" << a[0] << ", " << a[1] << ", " << a[2] << ", " << a[3] << ">";
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,148 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#pragma once
|
||||
|
||||
namespace embree
|
||||
{
|
||||
/* memory representation as bit vector */
|
||||
struct __aligned(16) sseb_m
|
||||
{
|
||||
enum { size = 4 }; // number of SIMD elements
|
||||
int32 v[4];
|
||||
|
||||
__forceinline sseb_m ( ) {}
|
||||
|
||||
__forceinline sseb_m ( bool a ) {
|
||||
v[0] = a ? -1 : 0;
|
||||
v[1] = a ? -1 : 0;
|
||||
v[2] = a ? -1 : 0;
|
||||
v[3] = a ? -1 : 0;
|
||||
}
|
||||
__forceinline sseb_m ( bool a, bool b ) {
|
||||
v[0] = a ? -1 : 0;
|
||||
v[1] = b ? -1 : 0;
|
||||
v[2] = a ? -1 : 0;
|
||||
v[3] = b ? -1 : 0;
|
||||
}
|
||||
__forceinline sseb_m ( bool a, bool b, bool c, bool d ) {
|
||||
v[0] = a ? -1 : 0;
|
||||
v[1] = b ? -1 : 0;
|
||||
v[2] = c ? -1 : 0;
|
||||
v[3] = d ? -1 : 0;
|
||||
}
|
||||
|
||||
__forceinline sseb_m ( const sseb_t& other );
|
||||
__forceinline sseb_m& operator=( const sseb_t& other );
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constants
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline sseb_m( FalseTy ) { v[0] = v[1] = v[2] = v[3] = 0; }
|
||||
__forceinline sseb_m( TrueTy ) { v[0] = v[1] = v[2] = v[3] = -1; }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Array Access
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline bool operator []( const size_t i ) const { assert(i < 4); return v[i] != 0; }
|
||||
__forceinline int32& operator []( const size_t i ) { assert(i < 4); return v[i]; }
|
||||
};
|
||||
|
||||
/*! 4-wide bool type emulated with 16-wide bit vectors. */
|
||||
struct sseb_t
|
||||
{
|
||||
__mmask16 v;
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constructors, Assignment & Cast Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline sseb_t ( const __mmask16 a ) : v(a) {}
|
||||
__forceinline sseb_t ( const sseb_t& other ) { v = other.v; }
|
||||
__forceinline sseb_t& operator=( const sseb_t& other ) { v = other.v; return *this; }
|
||||
__forceinline sseb_t ( const sseb_m& other ) {
|
||||
__m512i i = _mm512_extload_epi32(&other,_MM_UPCONV_EPI32_NONE,_MM_BROADCAST_4X16,_MM_HINT_NONE);
|
||||
v = _mm512_cmp_epi32_mask(i,_mm512_set1_epi32(0),_MM_CMPINT_NE);
|
||||
}
|
||||
};
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Assignment Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline sseb_m::sseb_m ( const sseb_t& other ) {
|
||||
__m512i i = _mm512_mask_blend_epi32(other.v, _mm512_set1_epi32(0), _mm512_set1_epi32(-1));
|
||||
_mm512_mask_extpackstorelo_epi32(this,0xf,i,_MM_DOWNCONV_EPI32_NONE, _MM_HINT_NONE);
|
||||
}
|
||||
__forceinline sseb_m& sseb_m::operator=( const sseb_t& other ) {
|
||||
__m512i i = _mm512_mask_blend_epi32(other.v, _mm512_set1_epi32(0), _mm512_set1_epi32(-1));
|
||||
_mm512_mask_extpackstorelo_epi32(this,0xf,i,_MM_DOWNCONV_EPI32_NONE, _MM_HINT_NONE);
|
||||
return *this;
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Unary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const sseb_t operator !( const sseb_t& a ) { return _mm512_knot(a.v); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Binary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const sseb_t operator &( const sseb_t& a, const sseb_t& b ) { return _mm512_kand(a.v, b.v); }
|
||||
__forceinline const sseb_t operator |( const sseb_t& a, const sseb_t& b ) { return _mm512_kor (a.v, b.v); }
|
||||
__forceinline const sseb_t operator ^( const sseb_t& a, const sseb_t& b ) { return _mm512_kxor(a.v, b.v); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Assignment Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const sseb_t operator &=( sseb_m& a, const sseb_t& b ) { return a = a & b; }
|
||||
__forceinline const sseb_t operator |=( sseb_m& a, const sseb_t& b ) { return a = a | b; }
|
||||
__forceinline const sseb_t operator ^=( sseb_m& a, const sseb_t& b ) { return a = a ^ b; }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Comparison Operators + Select
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const sseb_t operator !=( const sseb_t& a, const sseb_t& b ) { return _mm512_kxor (a.v, b.v); }
|
||||
__forceinline const sseb_t operator ==( const sseb_t& a, const sseb_t& b ) { return _mm512_kxnor(a.v, b.v); }
|
||||
__forceinline const sseb_t select( const sseb_t& m, const sseb_t& a, const sseb_t& b ) { return _mm512_kor(_mm512_kand(m.v, a.v), _mm512_kandn(m.v, b.v)); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Reduction Operations
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline bool reduce_and( const sseb_t& a ) { return (a.v & 0xf) == 0xf; }
|
||||
__forceinline bool reduce_or ( const sseb_t& a ) { return (a.v & 0xf) != 0x0; }
|
||||
__forceinline bool all ( const sseb_t& a ) { return (a.v & 0xf) == 0xf; }
|
||||
__forceinline bool any ( const sseb_t& a ) { return (a.v & 0xf) != 0x0; }
|
||||
__forceinline bool none ( const sseb_t& a ) { return (a.v & 0xf) == 0x0; }
|
||||
|
||||
__forceinline size_t movemask( const sseb_t& a ) { return a.v & 0xf; }
|
||||
__forceinline size_t popcnt( const sseb_t& a ) { return __popcnt(a.v); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Output Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
inline std::ostream& operator<<(std::ostream& cout, const sseb_m& a) {
|
||||
return cout << "<" << a[0] << ", " << a[1] << ", " << a[2] << ", " << a[3] << ">";
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,415 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#pragma once
|
||||
|
||||
namespace embree
|
||||
{
|
||||
/*! 4-wide SSE float type. */
|
||||
struct ssef
|
||||
{
|
||||
typedef sseb Mask; // mask type
|
||||
typedef ssei Int; // int type
|
||||
typedef ssef Float; // float type
|
||||
|
||||
enum { size = 4 }; // number of SIMD elements
|
||||
union { __m128 m128; float f[4]; int i[4]; }; // data
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constructors, Assignment & Cast Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline ssef ( ) {}
|
||||
__forceinline ssef ( const ssef& other ) { m128 = other.m128; }
|
||||
__forceinline ssef& operator=( const ssef& other ) { m128 = other.m128; return *this; }
|
||||
|
||||
__forceinline ssef( const __m128 a ) : m128(a) {}
|
||||
__forceinline operator const __m128&( void ) const { return m128; }
|
||||
__forceinline operator __m128&( void ) { return m128; }
|
||||
|
||||
__forceinline ssef ( float a ) : m128(_mm_set1_ps(a)) {}
|
||||
__forceinline ssef ( float a, float b, float c, float d) : m128(_mm_set_ps(d, c, b, a)) {}
|
||||
|
||||
__forceinline explicit ssef( const __m128i a ) : m128(_mm_cvtepi32_ps(a)) {}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constants
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline ssef( ZeroTy ) : m128(_mm_setzero_ps()) {}
|
||||
__forceinline ssef( OneTy ) : m128(_mm_set1_ps(1.0f)) {}
|
||||
__forceinline ssef( PosInfTy ) : m128(_mm_set1_ps(pos_inf)) {}
|
||||
__forceinline ssef( NegInfTy ) : m128(_mm_set1_ps(neg_inf)) {}
|
||||
__forceinline ssef( StepTy ) : m128(_mm_set_ps(3.0f, 2.0f, 1.0f, 0.0f)) {}
|
||||
__forceinline ssef( NaNTy ) : m128(_mm_set1_ps(nan)) {}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Loads and Stores
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
#if defined(__AVX__)
|
||||
static __forceinline ssef broadcast( const void* const a ) { return _mm_broadcast_ss((float*)a); }
|
||||
#else
|
||||
static __forceinline ssef broadcast( const void* const a ) { return _mm_set1_ps(*(float*)a); }
|
||||
#endif
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Array Access
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const float& operator []( const size_t i ) const { assert(i < 4); return f[i]; }
|
||||
__forceinline float& operator []( const size_t i ) { assert(i < 4); return f[i]; }
|
||||
};
|
||||
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Unary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const ssef cast (const __m128i& a) { return _mm_castsi128_ps(a); }
|
||||
__forceinline const ssef operator +( const ssef& a ) { return a; }
|
||||
__forceinline const ssef operator -( const ssef& a ) { return _mm_xor_ps(a.m128, _mm_castsi128_ps(_mm_set1_epi32(0x80000000))); }
|
||||
__forceinline const ssef abs ( const ssef& a ) { return _mm_and_ps(a.m128, _mm_castsi128_ps(_mm_set1_epi32(0x7fffffff))); }
|
||||
__forceinline const ssef sign ( const ssef& a ) { return _mm_blendv_ps(ssef(one), -ssef(one), _mm_cmplt_ps (a,ssef(zero))); }
|
||||
__forceinline const ssef signmsk ( const ssef& a ) { return _mm_and_ps(a.m128,_mm_castsi128_ps(_mm_set1_epi32(0x80000000))); }
|
||||
|
||||
__forceinline const ssef rcp ( const ssef& a ) {
|
||||
const ssef r = _mm_rcp_ps(a.m128);
|
||||
return _mm_sub_ps(_mm_add_ps(r, r), _mm_mul_ps(_mm_mul_ps(r, r), a));
|
||||
}
|
||||
__forceinline const ssef sqr ( const ssef& a ) { return _mm_mul_ps(a,a); }
|
||||
__forceinline const ssef sqrt ( const ssef& a ) { return _mm_sqrt_ps(a.m128); }
|
||||
__forceinline const ssef rsqrt( const ssef& a ) {
|
||||
const ssef r = _mm_rsqrt_ps(a.m128);
|
||||
return _mm_add_ps(_mm_mul_ps(_mm_set_ps(1.5f, 1.5f, 1.5f, 1.5f), r),
|
||||
_mm_mul_ps(_mm_mul_ps(_mm_mul_ps(a, _mm_set_ps(-0.5f, -0.5f, -0.5f, -0.5f)), r), _mm_mul_ps(r, r)));
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Binary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const ssef operator +( const ssef& a, const ssef& b ) { return _mm_add_ps(a.m128, b.m128); }
|
||||
__forceinline const ssef operator +( const ssef& a, const float& b ) { return a + ssef(b); }
|
||||
__forceinline const ssef operator +( const float& a, const ssef& b ) { return ssef(a) + b; }
|
||||
|
||||
__forceinline const ssef operator -( const ssef& a, const ssef& b ) { return _mm_sub_ps(a.m128, b.m128); }
|
||||
__forceinline const ssef operator -( const ssef& a, const float& b ) { return a - ssef(b); }
|
||||
__forceinline const ssef operator -( const float& a, const ssef& b ) { return ssef(a) - b; }
|
||||
|
||||
__forceinline const ssef operator *( const ssef& a, const ssef& b ) { return _mm_mul_ps(a.m128, b.m128); }
|
||||
__forceinline const ssef operator *( const ssef& a, const float& b ) { return a * ssef(b); }
|
||||
__forceinline const ssef operator *( const float& a, const ssef& b ) { return ssef(a) * b; }
|
||||
|
||||
__forceinline const ssef operator /( const ssef& a, const ssef& b ) { return _mm_div_ps(a.m128,b.m128); }
|
||||
__forceinline const ssef operator /( const ssef& a, const float& b ) { return a/ssef(b); }
|
||||
__forceinline const ssef operator /( const float& a, const ssef& b ) { return ssef(a)/b; }
|
||||
|
||||
__forceinline const ssef operator^( const ssef& a, const ssef& b ) { return _mm_xor_ps(a.m128,b.m128); }
|
||||
__forceinline const ssef operator^( const ssef& a, const ssei& b ) { return _mm_xor_ps(a.m128,_mm_castsi128_ps(b.m128)); }
|
||||
|
||||
__forceinline const ssef min( const ssef& a, const ssef& b ) { return _mm_min_ps(a.m128,b.m128); }
|
||||
__forceinline const ssef min( const ssef& a, const float& b ) { return _mm_min_ps(a.m128,ssef(b)); }
|
||||
__forceinline const ssef min( const float& a, const ssef& b ) { return _mm_min_ps(ssef(a),b.m128); }
|
||||
|
||||
__forceinline const ssef max( const ssef& a, const ssef& b ) { return _mm_max_ps(a.m128,b.m128); }
|
||||
__forceinline const ssef max( const ssef& a, const float& b ) { return _mm_max_ps(a.m128,ssef(b)); }
|
||||
__forceinline const ssef max( const float& a, const ssef& b ) { return _mm_max_ps(ssef(a),b.m128); }
|
||||
|
||||
#if defined(__SSE4_1__)
|
||||
__forceinline ssef mini(const ssef& a, const ssef& b) {
|
||||
const ssei ai = _mm_castps_si128(a);
|
||||
const ssei bi = _mm_castps_si128(b);
|
||||
const ssei ci = _mm_min_epi32(ai,bi);
|
||||
return _mm_castsi128_ps(ci);
|
||||
}
|
||||
#endif
|
||||
|
||||
#if defined(__SSE4_1__)
|
||||
__forceinline ssef maxi(const ssef& a, const ssef& b) {
|
||||
const ssei ai = _mm_castps_si128(a);
|
||||
const ssei bi = _mm_castps_si128(b);
|
||||
const ssei ci = _mm_max_epi32(ai,bi);
|
||||
return _mm_castsi128_ps(ci);
|
||||
}
|
||||
#endif
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Ternary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
#if defined(__AVX2__)
|
||||
__forceinline const ssef madd ( const ssef& a, const ssef& b, const ssef& c) { return _mm_fmadd_ps(a,b,c); }
|
||||
__forceinline const ssef msub ( const ssef& a, const ssef& b, const ssef& c) { return _mm_fmsub_ps(a,b,c); }
|
||||
__forceinline const ssef nmadd ( const ssef& a, const ssef& b, const ssef& c) { return _mm_fnmadd_ps(a,b,c); }
|
||||
__forceinline const ssef nmsub ( const ssef& a, const ssef& b, const ssef& c) { return _mm_fnmsub_ps(a,b,c); }
|
||||
#else
|
||||
__forceinline const ssef madd ( const ssef& a, const ssef& b, const ssef& c) { return a*b+c; }
|
||||
__forceinline const ssef msub ( const ssef& a, const ssef& b, const ssef& c) { return a*b-c; }
|
||||
__forceinline const ssef nmadd ( const ssef& a, const ssef& b, const ssef& c) { return -a*b-c;}
|
||||
__forceinline const ssef nmsub ( const ssef& a, const ssef& b, const ssef& c) { return c-a*b; }
|
||||
#endif
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Assignment Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline ssef& operator +=( ssef& a, const ssef& b ) { return a = a + b; }
|
||||
__forceinline ssef& operator +=( ssef& a, const float& b ) { return a = a + b; }
|
||||
|
||||
__forceinline ssef& operator -=( ssef& a, const ssef& b ) { return a = a - b; }
|
||||
__forceinline ssef& operator -=( ssef& a, const float& b ) { return a = a - b; }
|
||||
|
||||
__forceinline ssef& operator *=( ssef& a, const ssef& b ) { return a = a * b; }
|
||||
__forceinline ssef& operator *=( ssef& a, const float& b ) { return a = a * b; }
|
||||
|
||||
__forceinline ssef& operator /=( ssef& a, const ssef& b ) { return a = a / b; }
|
||||
__forceinline ssef& operator /=( ssef& a, const float& b ) { return a = a / b; }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Comparison Operators + Select
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const sseb operator ==( const ssef& a, const ssef& b ) { return _mm_cmpeq_ps (a.m128, b.m128); }
|
||||
__forceinline const sseb operator ==( const ssef& a, const float& b ) { return a == ssef(b); }
|
||||
__forceinline const sseb operator ==( const float& a, const ssef& b ) { return ssef(a) == b; }
|
||||
|
||||
__forceinline const sseb operator !=( const ssef& a, const ssef& b ) { return _mm_cmpneq_ps(a.m128, b.m128); }
|
||||
__forceinline const sseb operator !=( const ssef& a, const float& b ) { return a != ssef(b); }
|
||||
__forceinline const sseb operator !=( const float& a, const ssef& b ) { return ssef(a) != b; }
|
||||
|
||||
__forceinline const sseb operator < ( const ssef& a, const ssef& b ) { return _mm_cmplt_ps (a.m128, b.m128); }
|
||||
__forceinline const sseb operator < ( const ssef& a, const float& b ) { return a < ssef(b); }
|
||||
__forceinline const sseb operator < ( const float& a, const ssef& b ) { return ssef(a) < b; }
|
||||
|
||||
__forceinline const sseb operator >=( const ssef& a, const ssef& b ) { return _mm_cmpnlt_ps(a.m128, b.m128); }
|
||||
__forceinline const sseb operator >=( const ssef& a, const float& b ) { return a >= ssef(b); }
|
||||
__forceinline const sseb operator >=( const float& a, const ssef& b ) { return ssef(a) >= b; }
|
||||
|
||||
__forceinline const sseb operator > ( const ssef& a, const ssef& b ) { return _mm_cmpnle_ps(a.m128, b.m128); }
|
||||
__forceinline const sseb operator > ( const ssef& a, const float& b ) { return a > ssef(b); }
|
||||
__forceinline const sseb operator > ( const float& a, const ssef& b ) { return ssef(a) > b; }
|
||||
|
||||
__forceinline const sseb operator <=( const ssef& a, const ssef& b ) { return _mm_cmple_ps (a.m128, b.m128); }
|
||||
__forceinline const sseb operator <=( const ssef& a, const float& b ) { return a <= ssef(b); }
|
||||
__forceinline const sseb operator <=( const float& a, const ssef& b ) { return ssef(a) <= b; }
|
||||
|
||||
__forceinline const ssef select( const sseb& m, const ssef& t, const ssef& f ) {
|
||||
#if defined(__SSE4_1__)
|
||||
return _mm_blendv_ps(f, t, m);
|
||||
#else
|
||||
return _mm_or_ps(_mm_and_ps(m, t), _mm_andnot_ps(m, f));
|
||||
#endif
|
||||
}
|
||||
|
||||
#if defined(__SSE4_1__)
|
||||
#if defined(__clang__) || defined(_MSC_VER) && !defined(__INTEL_COMPILER)
|
||||
__forceinline const ssef select(const int mask, const ssef& t, const ssef& f) {
|
||||
return select(sseb(mask), t, f);
|
||||
}
|
||||
#else
|
||||
__forceinline const ssef select(const int mask, const ssef& t, const ssef& f) {
|
||||
return _mm_blend_ps(f, t, mask);
|
||||
}
|
||||
#endif
|
||||
#endif
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Rounding Functions
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
#if defined (__SSE4_1__)
|
||||
__forceinline const ssef round_even( const ssef& a ) { return _mm_round_ps(a, _MM_FROUND_TO_NEAREST_INT); }
|
||||
__forceinline const ssef round_down( const ssef& a ) { return _mm_round_ps(a, _MM_FROUND_TO_NEG_INF ); }
|
||||
__forceinline const ssef round_up ( const ssef& a ) { return _mm_round_ps(a, _MM_FROUND_TO_POS_INF ); }
|
||||
__forceinline const ssef round_zero( const ssef& a ) { return _mm_round_ps(a, _MM_FROUND_TO_ZERO ); }
|
||||
__forceinline const ssef floor ( const ssef& a ) { return _mm_round_ps(a, _MM_FROUND_TO_NEG_INF ); }
|
||||
__forceinline const ssef ceil ( const ssef& a ) { return _mm_round_ps(a, _MM_FROUND_TO_POS_INF ); }
|
||||
#endif
|
||||
|
||||
__forceinline ssei floori (const ssef& a) {
|
||||
#if defined (__SSE4_1__)
|
||||
return ssei(floor(a));
|
||||
#else
|
||||
return ssei(a-ssef(0.5f));
|
||||
#endif
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Movement/Shifting/Shuffling Functions
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline ssef unpacklo( const ssef& a, const ssef& b ) { return _mm_unpacklo_ps(a.m128, b.m128); }
|
||||
__forceinline ssef unpackhi( const ssef& a, const ssef& b ) { return _mm_unpackhi_ps(a.m128, b.m128); }
|
||||
|
||||
template<size_t i0, size_t i1, size_t i2, size_t i3> __forceinline const ssef shuffle( const ssef& b ) {
|
||||
return _mm_castsi128_ps(_mm_shuffle_epi32(_mm_castps_si128(b), _MM_SHUFFLE(i3, i2, i1, i0)));
|
||||
}
|
||||
|
||||
template<size_t i0, size_t i1, size_t i2, size_t i3> __forceinline const ssef shuffle( const ssef& a, const ssef& b ) {
|
||||
return _mm_shuffle_ps(a, b, _MM_SHUFFLE(i3, i2, i1, i0));
|
||||
}
|
||||
|
||||
#if defined (__SSSE3__)
|
||||
__forceinline const ssef shuffle8(const ssef& a, const ssei& shuf) {
|
||||
return _mm_castsi128_ps(_mm_shuffle_epi8(_mm_castps_si128(a), shuf));
|
||||
}
|
||||
#endif
|
||||
|
||||
#if defined(__SSE3__)
|
||||
template<> __forceinline const ssef shuffle<0, 0, 2, 2>( const ssef& b ) { return _mm_moveldup_ps(b); }
|
||||
template<> __forceinline const ssef shuffle<1, 1, 3, 3>( const ssef& b ) { return _mm_movehdup_ps(b); }
|
||||
template<> __forceinline const ssef shuffle<0, 1, 0, 1>( const ssef& b ) { return _mm_castpd_ps(_mm_movedup_pd(_mm_castps_pd(b))); }
|
||||
#endif
|
||||
|
||||
template<size_t i0> __forceinline const ssef shuffle( const ssef& b ) {
|
||||
return shuffle<i0,i0,i0,i0>(b);
|
||||
}
|
||||
|
||||
#if defined (__SSE4_1__) && !defined(__GNUC__)
|
||||
template<size_t i> __forceinline float extract ( const ssef& a ) { return _mm_cvtss_f32(_mm_extract_ps(a,i)); }
|
||||
#else
|
||||
template<size_t i> __forceinline float extract ( const ssef& a ) { return _mm_cvtss_f32(shuffle<i,i,i,i>(a)); }
|
||||
#endif
|
||||
template<> __forceinline float extract<0>( const ssef& a ) { return _mm_cvtss_f32(a); }
|
||||
|
||||
#if defined (__SSE4_1__)
|
||||
template<size_t dst, size_t src, size_t clr> __forceinline const ssef insert( const ssef& a, const ssef& b ) { return _mm_insert_ps(a, b, (dst << 4) | (src << 6) | clr); }
|
||||
template<size_t dst, size_t src> __forceinline const ssef insert( const ssef& a, const ssef& b ) { return insert<dst, src, 0>(a, b); }
|
||||
template<size_t dst> __forceinline const ssef insert( const ssef& a, const float b ) { return insert<dst, 0>(a, _mm_set_ss(b)); }
|
||||
#else
|
||||
template<size_t dst> __forceinline const ssef insert( const ssef& a, const float b ) { ssef c = a; c[dst] = b; return c; }
|
||||
#endif
|
||||
|
||||
__forceinline ssef broadcast4f( const ssef& a, const size_t k ) {
|
||||
return ssef::broadcast(&a[k]);
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Transpose
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline void transpose(const ssef& r0, const ssef& r1, const ssef& r2, const ssef& r3, ssef& c0, ssef& c1, ssef& c2, ssef& c3)
|
||||
{
|
||||
ssef l02 = unpacklo(r0,r2);
|
||||
ssef h02 = unpackhi(r0,r2);
|
||||
ssef l13 = unpacklo(r1,r3);
|
||||
ssef h13 = unpackhi(r1,r3);
|
||||
c0 = unpacklo(l02,l13);
|
||||
c1 = unpackhi(l02,l13);
|
||||
c2 = unpacklo(h02,h13);
|
||||
c3 = unpackhi(h02,h13);
|
||||
}
|
||||
|
||||
__forceinline void transpose(const ssef& r0, const ssef& r1, const ssef& r2, const ssef& r3, ssef& c0, ssef& c1, ssef& c2)
|
||||
{
|
||||
ssef l02 = unpacklo(r0,r2);
|
||||
ssef h02 = unpackhi(r0,r2);
|
||||
ssef l13 = unpacklo(r1,r3);
|
||||
ssef h13 = unpackhi(r1,r3);
|
||||
c0 = unpacklo(l02,l13);
|
||||
c1 = unpackhi(l02,l13);
|
||||
c2 = unpacklo(h02,h13);
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Reductions
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const ssef vreduce_min(const ssef& v) { ssef h = min(shuffle<1,0,3,2>(v),v); return min(shuffle<2,3,0,1>(h),h); }
|
||||
__forceinline const ssef vreduce_max(const ssef& v) { ssef h = max(shuffle<1,0,3,2>(v),v); return max(shuffle<2,3,0,1>(h),h); }
|
||||
__forceinline const ssef vreduce_add(const ssef& v) { ssef h = shuffle<1,0,3,2>(v) + v ; return shuffle<2,3,0,1>(h) + h ; }
|
||||
|
||||
__forceinline float reduce_min(const ssef& v) { return _mm_cvtss_f32(vreduce_min(v)); }
|
||||
__forceinline float reduce_max(const ssef& v) { return _mm_cvtss_f32(vreduce_max(v)); }
|
||||
__forceinline float reduce_add(const ssef& v) { return _mm_cvtss_f32(vreduce_add(v)); }
|
||||
|
||||
__forceinline size_t select_min(const ssef& v) { return __bsf(movemask(v == vreduce_min(v))); }
|
||||
__forceinline size_t select_max(const ssef& v) { return __bsf(movemask(v == vreduce_max(v))); }
|
||||
|
||||
__forceinline size_t select_min(const sseb& valid, const ssef& v) { const ssef a = select(valid,v,ssef(pos_inf)); return __bsf(movemask(valid & (a == vreduce_min(a)))); }
|
||||
__forceinline size_t select_max(const sseb& valid, const ssef& v) { const ssef a = select(valid,v,ssef(neg_inf)); return __bsf(movemask(valid & (a == vreduce_max(a)))); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Memory load and store operations
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline ssef load4f( const void* const a ) {
|
||||
return _mm_load_ps((float*)a);
|
||||
}
|
||||
|
||||
__forceinline void store4f ( void* ptr, const ssef& v ) {
|
||||
_mm_store_ps((float*)ptr,v);
|
||||
}
|
||||
|
||||
__forceinline ssef loadu4f( const void* const a ) {
|
||||
return _mm_loadu_ps((float*)a);
|
||||
}
|
||||
|
||||
__forceinline void storeu4f ( void* ptr, const ssef& v ) {
|
||||
_mm_storeu_ps((float*)ptr,v);
|
||||
}
|
||||
|
||||
__forceinline void store4f ( const sseb& mask, void* ptr, const ssef& f ) {
|
||||
#if defined (__AVX__)
|
||||
_mm_maskstore_ps((float*)ptr,(__m128i)mask,f);
|
||||
#else
|
||||
*(ssef*)ptr = select(mask,f,*(ssef*)ptr);
|
||||
#endif
|
||||
}
|
||||
|
||||
__forceinline ssef load4f_nt (void* ptr) {
|
||||
#if defined (__SSE4_1__)
|
||||
return _mm_castsi128_ps(_mm_stream_load_si128((__m128i*)ptr));
|
||||
#else
|
||||
return _mm_load_ps((float*)ptr);
|
||||
#endif
|
||||
}
|
||||
|
||||
__forceinline void store4f_nt (void* ptr, const ssef& v) {
|
||||
#if defined (__SSE4_1__)
|
||||
_mm_stream_ps((float*)ptr,v);
|
||||
#else
|
||||
_mm_store_ps((float*)ptr,v);
|
||||
#endif
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Euclidian Space Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline float dot ( const ssef& a, const ssef& b ) {
|
||||
return reduce_add(a*b);
|
||||
}
|
||||
|
||||
__forceinline ssef cross ( const ssef& a, const ssef& b )
|
||||
{
|
||||
const ssef a0 = a;
|
||||
const ssef b0 = shuffle<1,2,0,3>(b);
|
||||
const ssef a1 = shuffle<1,2,0,3>(a);
|
||||
const ssef b1 = b;
|
||||
return shuffle<1,2,0,3>(msub(a0,b0,a1*b1));
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Output Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
inline std::ostream& operator<<(std::ostream& cout, const ssef& a) {
|
||||
return cout << "<" << a[0] << ", " << a[1] << ", " << a[2] << ", " << a[3] << ">";
|
||||
}
|
||||
|
||||
}
|
||||
@@ -0,0 +1,352 @@
|
||||
// ======================================================================== //
|
||||
// Copyright 2009-2014 Intel Corporation //
|
||||
// //
|
||||
// Licensed under the Apache License, Version 2.0 (the "License"); //
|
||||
// you may not use this file except in compliance with the License. //
|
||||
// You may obtain a copy of the License at //
|
||||
// //
|
||||
// http://www.apache.org/licenses/LICENSE-2.0 //
|
||||
// //
|
||||
// Unless required by applicable law or agreed to in writing, software //
|
||||
// distributed under the License is distributed on an "AS IS" BASIS, //
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. //
|
||||
// See the License for the specific language governing permissions and //
|
||||
// limitations under the License. //
|
||||
// ======================================================================== //
|
||||
|
||||
#pragma once
|
||||
|
||||
namespace embree
|
||||
{
|
||||
/* memory representation as 4 aligned floats */
|
||||
struct __aligned(16) ssef_m
|
||||
{
|
||||
typedef sseb_m Mask;
|
||||
typedef ssei_m Int;
|
||||
typedef ssef_m Float;
|
||||
enum { size = 4 }; // number of SIMD elements
|
||||
union {
|
||||
float f[4];
|
||||
int i[4];
|
||||
};
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constructors, Assignment & Cast Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline ssef_m ( ) {}
|
||||
__forceinline ssef_m ( float a ) { f[0] = a; f[1] = a; f[2] = a; f[3] = a; }
|
||||
__forceinline ssef_m ( float a, float b, float c, float d) { f[0] = a; f[1] = b; f[2] = c; f[3] = d; }
|
||||
|
||||
__forceinline ssef_m ( const ssef_t& other );
|
||||
__forceinline ssef_m& operator=( const ssef_t& other );
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constants
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline ssef_m( ZeroTy ) { f[0] = f[1] = f[2] = f[3] = zero; }
|
||||
__forceinline ssef_m( OneTy ) { f[0] = f[1] = f[2] = f[3] = one; }
|
||||
__forceinline ssef_m( NegInfTy ) { f[0] = f[1] = f[2] = f[3] = neg_inf; }
|
||||
__forceinline ssef_m( PosInfTy ) { f[0] = f[1] = f[2] = f[3] = pos_inf; }
|
||||
__forceinline ssef_m( StepTy ) { f[0] = 0.0f; f[1] = 1.0f; f[2] = 2.0f; f[3] = 3.0f; }
|
||||
__forceinline ssef_m( NaNTy ) { f[0] = f[1] = f[2] = f[3] = nan; }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Array Access
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const float& operator []( const size_t index ) const { assert(index < 4); return f[index]; }
|
||||
__forceinline float& operator []( const size_t index ) { assert(index < 4); return f[index]; }
|
||||
};
|
||||
|
||||
/*! 4-wide SSE float type emulated with 16-wide vectors. */
|
||||
struct __aligned(64) ssef_t
|
||||
{
|
||||
union {
|
||||
__m512 m512;
|
||||
float f[4];
|
||||
int i[4];
|
||||
};
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Constructors, Assignment & Cast Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline ssef_t( const __m512 a ) : m512(a) {}
|
||||
__forceinline ssef_t ( const ssef_t& other ) { m512 = other.m512; }
|
||||
__forceinline ssef_t& operator=( const ssef_t& other ) { m512 = other.m512; return *this; }
|
||||
|
||||
__forceinline operator const __m512&( void ) const { return m512; }
|
||||
__forceinline operator __m512&( void ) { return m512; }
|
||||
|
||||
__forceinline ssef_t( const ssei_t& v );
|
||||
|
||||
|
||||
public:
|
||||
__forceinline explicit ssef_t ( float a )
|
||||
: m512(_mm512_set1_ps(a)) {}
|
||||
|
||||
__forceinline ssef_t ( const ssef_m& other ) {
|
||||
m512 = _mm512_extload_ps(&other,_MM_UPCONV_PS_NONE,_MM_BROADCAST_4X16,_MM_HINT_NONE);
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Array Access
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const float& operator []( const size_t index ) const { assert(index < 4); return f[index]; }
|
||||
__forceinline float& operator []( const size_t index ) { assert(index < 4); return f[index]; }
|
||||
};
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Assignment Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
/* __forceinline ssei_m::ssei_m ( const ssef_t& other ) { */
|
||||
/* __m512i i = _mm512_cvtfxpnt_round_adjustps_epi32(other,_MM_ROUND_MODE_NEAREST,_MM_EXPADJ_NONE); */
|
||||
/* _mm512_mask_extpackstorelo_epi32(this,0xf,i,_MM_DOWNCONV_EPI32_NONE, _MM_HINT_NONE); */
|
||||
/* } */
|
||||
|
||||
__forceinline ssef_t::ssef_t( const ssei_t& v ) {
|
||||
m512 = _mm512_cvtfxpnt_round_adjustepi32_ps(v,_MM_ROUND_MODE_NEAREST,_MM_EXPADJ_NONE);
|
||||
}
|
||||
|
||||
__forceinline ssei_t::ssei_t ( const ssef_t& other )
|
||||
{
|
||||
m512 = _mm512_cvtfxpnt_round_adjustps_epi32(other.m512,_MM_ROUND_NEAREST,_MM_EXPADJ_NONE);
|
||||
}
|
||||
|
||||
__forceinline ssef_m::ssef_m ( const ssef_t& other ) {
|
||||
_mm512_mask_extpackstorelo_ps(this,0xf,other.m512,_MM_DOWNCONV_PS_NONE, _MM_HINT_NONE);
|
||||
}
|
||||
|
||||
__forceinline ssef_m& ssef_m::operator=( const ssef_t& other ) {
|
||||
_mm512_mask_extpackstorelo_ps(this,0xf,other.m512,_MM_DOWNCONV_PS_NONE, _MM_HINT_NONE);
|
||||
return *this;
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Unary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const ssef_t operator +( const ssef_t& a ) { return a; }
|
||||
__forceinline const ssef_t operator -( const ssef_t& a ) { return _mm512_castsi512_ps(_mm512_xor_epi32(_mm512_castps_si512(a.m512), _mm512_set1_epi32(0x80000000))); }
|
||||
__forceinline const ssef_t abs ( const ssef_t& a ) { return _mm512_gmaxabs_ps(a.m512,a.m512); }
|
||||
__forceinline const ssef_t sign ( const ssef_t& a ) { return _mm512_mask_blend_ps(_mm512_cmp_ps_mask(a,ssef_t(ssef_m(zero)),_MM_CMPINT_LT), ssef_t(ssef_m(one)), -ssef_t(ssef_m(one))); }
|
||||
__forceinline const ssef_t signmsk ( const ssef_t& a ) { return _mm512_castsi512_ps(_mm512_and_epi32(_mm512_castps_si512(a.m512),_mm512_set1_epi32(0x80000000))); }
|
||||
|
||||
__forceinline const ssef_t rcp ( const ssef_t& a ) { return _mm512_rcp23_ps(a.m512); }
|
||||
__forceinline const ssef_t sqr ( const ssef_t& a ) { return _mm512_mul_ps(a.m512,a.m512); }
|
||||
__forceinline const ssef_t sqrt ( const ssef_t& a ) { return _mm512_sqrt_ps(a.m512); }
|
||||
__forceinline const ssef_t rsqrt( const ssef_t& a ) { return _mm512_invsqrt_ps(a.m512); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Binary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const ssef_t operator +( const ssef_t& a, const ssef_t& b ) { return _mm512_add_ps(a.m512, b.m512); }
|
||||
__forceinline const ssef_t operator +( const ssef_t& a, const float & b ) { return a + ssef_t(b); }
|
||||
__forceinline const ssef_t operator +( const float & a, const ssef_t& b ) { return ssef_t(a) + b; }
|
||||
|
||||
__forceinline const ssef_t operator -( const ssef_t& a, const ssef_t& b ) { return _mm512_sub_ps(a.m512, b.m512); }
|
||||
__forceinline const ssef_t operator -( const ssef_t& a, const float & b ) { return a - ssef_t(b); }
|
||||
__forceinline const ssef_t operator -( const float & a, const ssef_t& b ) { return ssef_t(a) - b; }
|
||||
|
||||
__forceinline const ssef_t operator *( const ssef_t& a, const ssef_t& b ) { return _mm512_mul_ps(a.m512, b.m512); }
|
||||
__forceinline const ssef_t operator *( const ssef_t& a, const float & b ) { return a * ssef_t(b); }
|
||||
__forceinline const ssef_t operator *( const float & a, const ssef_t& b ) { return ssef_t(a) * b; }
|
||||
|
||||
__forceinline const ssef_t operator /( const ssef_t& a, const ssef_t& b ) { return _mm512_div_ps(a.m512,b.m512); }
|
||||
__forceinline const ssef_t operator /( const ssef_t& a, const float & b ) { return a/ssef_t(b); }
|
||||
__forceinline const ssef_t operator /( const float & a, const ssef_t& b ) { return ssef_t(a)/b; }
|
||||
|
||||
__forceinline const ssef_t operator^( const ssef_t& a, const ssef_t& b ) { return _mm512_castsi512_ps(_mm512_xor_epi32(_mm512_castps_si512(a.m512),_mm512_castps_si512(b.m512))); }
|
||||
__forceinline const ssef_t operator^( const ssef_t& a, const ssei_t& b ) { return _mm512_castsi512_ps(_mm512_xor_epi32(_mm512_castps_si512(a.m512),b.m512)); }
|
||||
|
||||
__forceinline const ssef_t min( const ssef_t& a, const ssef_t& b ) { return _mm512_gmin_ps(a.m512,b.m512); }
|
||||
__forceinline const ssef_t min( const ssef_t& a, const float & b ) { return _mm512_gmin_ps(a.m512,ssef_t(b)); }
|
||||
__forceinline const ssef_t min( const float & a, const ssef_t& b ) { return _mm512_gmin_ps(ssef_t(a),b.m512); }
|
||||
|
||||
__forceinline const ssef_t max( const ssef_t& a, const ssef_t& b ) { return _mm512_gmax_ps(a.m512,b.m512); }
|
||||
__forceinline const ssef_t max( const ssef_t& a, const float & b ) { return _mm512_gmax_ps(a.m512,ssef_t(b)); }
|
||||
__forceinline const ssef_t max( const float & a, const ssef_t& b ) { return _mm512_gmax_ps(ssef_t(a),b.m512); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Ternary Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const ssef_t madd ( const ssef_t& a, const ssef_t& b, const ssef_t& c) { return _mm512_fmadd_ps(a,b,c); }
|
||||
__forceinline const ssef_t msub ( const ssef_t& a, const ssef_t& b, const ssef_t& c) { return _mm512_fmsub_ps(a,b,c); }
|
||||
__forceinline const ssef_t nmadd ( const ssef_t& a, const ssef_t& b, const ssef_t& c) { return _mm512_fnmadd_ps(a,b,c); }
|
||||
__forceinline const ssef_t nmsub ( const ssef_t& a, const ssef_t& b, const ssef_t& c) { return _mm512_fnmsub_ps(a,b,c); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Assignment Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline ssef_t& operator +=( ssef_t& a, const ssef_t& b ) { return a = a + b; }
|
||||
__forceinline ssef_t& operator +=( ssef_t& a, const float & b ) { return a = a + b; }
|
||||
|
||||
__forceinline ssef_t& operator -=( ssef_t& a, const ssef_t& b ) { return a = a - b; }
|
||||
__forceinline ssef_t& operator -=( ssef_t& a, const float & b ) { return a = a - b; }
|
||||
|
||||
__forceinline ssef_t& operator *=( ssef_t& a, const ssef_t& b ) { return a = a * b; }
|
||||
__forceinline ssef_t& operator *=( ssef_t& a, const float & b ) { return a = a * b; }
|
||||
|
||||
__forceinline ssef_t& operator /=( ssef_t& a, const ssef_t& b ) { return a = a / b; }
|
||||
__forceinline ssef_t& operator /=( ssef_t& a, const float & b ) { return a = a / b; }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Comparison Operators + Select
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const sseb_t operator ==( const ssef_t& a, const ssef_t& b ) { return _mm512_cmp_ps_mask(a,b,_MM_CMPINT_EQ); }
|
||||
__forceinline const sseb_t operator ==( const ssef_t& a, const float & b ) { return a == ssef_t(b); }
|
||||
__forceinline const sseb_t operator ==( const float & a, const ssef_t& b ) { return ssef_t(a) == b; }
|
||||
|
||||
__forceinline const sseb_t operator !=( const ssef_t& a, const ssef_t& b ) { return _mm512_cmp_ps_mask(a,b,_MM_CMPINT_NE); }
|
||||
__forceinline const sseb_t operator !=( const ssef_t& a, const float & b ) { return a != ssef_t(b); }
|
||||
__forceinline const sseb_t operator !=( const float & a, const ssef_t& b ) { return ssef_t(a) != b; }
|
||||
|
||||
__forceinline const sseb_t operator < ( const ssef_t& a, const ssef_t& b ) { return _mm512_cmp_ps_mask(a,b,_MM_CMPINT_LT); }
|
||||
__forceinline const sseb_t operator < ( const ssef_t& a, const float & b ) { return a < ssef_t(b); }
|
||||
__forceinline const sseb_t operator < ( const float & a, const ssef_t& b ) { return ssef_t(a) < b; }
|
||||
|
||||
__forceinline const sseb_t operator >=( const ssef_t& a, const ssef_t& b ) { return _mm512_cmp_ps_mask(a,b,_MM_CMPINT_GE); }
|
||||
__forceinline const sseb_t operator >=( const ssef_t& a, const float & b ) { return a >= ssef_t(b); }
|
||||
__forceinline const sseb_t operator >=( const float & a, const ssef_t& b ) { return ssef_t(a) >= b; }
|
||||
|
||||
__forceinline const sseb_t operator > ( const ssef_t& a, const ssef_t& b ) { return _mm512_cmp_ps_mask(a,b,_MM_CMPINT_GT); }
|
||||
__forceinline const sseb_t operator > ( const ssef_t& a, const float & b ) { return a > ssef_t(b); }
|
||||
__forceinline const sseb_t operator > ( const float & a, const ssef_t& b ) { return ssef_t(a) > b; }
|
||||
|
||||
__forceinline const sseb_t operator <=( const ssef_t& a, const ssef_t& b ) { return _mm512_cmp_ps_mask(a,b,_MM_CMPINT_LE); }
|
||||
__forceinline const sseb_t operator <=( const ssef_t& a, const float & b ) { return a <= ssef_t(b); }
|
||||
__forceinline const sseb_t operator <=( const float & a, const ssef_t& b ) { return ssef_t(a) <= b; }
|
||||
|
||||
__forceinline const ssef_t select( const sseb_t& mask, const ssef_t& t, const ssef_t& f ) { return _mm512_mask_blend_ps(mask.v, f, t); }
|
||||
|
||||
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Rounding Functions
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const ssef_t round_even( const ssef_t& a ) { return _mm512_round_ps(a, _MM_ROUND_MODE_NEAREST , _MM_EXPADJ_NONE); }
|
||||
__forceinline const ssef_t round_down( const ssef_t& a ) { return _mm512_round_ps(a, _MM_ROUND_MODE_DOWN , _MM_EXPADJ_NONE); }
|
||||
__forceinline const ssef_t round_up ( const ssef_t& a ) { return _mm512_round_ps(a, _MM_ROUND_MODE_UP , _MM_EXPADJ_NONE); }
|
||||
__forceinline const ssef_t round_zero( const ssef_t& a ) { return _mm512_round_ps(a, _MM_ROUND_MODE_TOWARD_ZERO, _MM_EXPADJ_NONE); }
|
||||
__forceinline const ssef_t floor ( const ssef_t& a ) { return _mm512_round_ps(a, _MM_ROUND_MODE_DOWN , _MM_EXPADJ_NONE); }
|
||||
__forceinline const ssef_t ceil ( const ssef_t& a ) { return _mm512_round_ps(a, _MM_ROUND_MODE_UP , _MM_EXPADJ_NONE); }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Movement/Shifting/Shuffling Functions
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
//__forceinline ssef_t unpacklo( const ssef_t& a, const ssef_t& b ) { return _mm512_unpacklo_ps(a.m512, b.m512); }
|
||||
//__forceinline ssef_t unpackhi( const ssef_t& a, const ssef_t& b ) { return _mm512_unpackhi_ps(a.m512, b.m512); }
|
||||
|
||||
template<size_t i0, size_t i1, size_t i2, size_t i3> __forceinline const ssef_t shuffle( const ssef_t& b ) {
|
||||
return _mm512_castsi512_ps(_mm512_shuffle_epi32(_mm512_castps_si512(b),(_MM_PERM_ENUM)_MM_SHUFFLE(i3, i2, i1, i0)));
|
||||
}
|
||||
|
||||
//template<size_t i0, size_t i1, size_t i2, size_t i3> __forceinline const ssef_t shuffle( const ssef_t& a, const ssef_t& b ) {
|
||||
// return _mm512_shuffle_ps(a, b, _MM512_SHUFFLE(i3, i2, i1, i0));
|
||||
//}
|
||||
|
||||
//__forceinline const ssef_t shuffle8(const ssef_t& a, const ssei& shuf) {
|
||||
// return _mm512_castsi512_ps(_mm512_shuffle_epi8(_mm512_castps_si512(a), shuf));
|
||||
//}
|
||||
|
||||
template<size_t i0> __forceinline const ssef_t shuffle( const ssef_t& b ) {
|
||||
return shuffle<i0,i0,i0,i0>(b);
|
||||
}
|
||||
|
||||
template<size_t i> __forceinline float extract( const ssef_t& a ) { return a[i]; }
|
||||
//template<size_t dst, size_t src, size_t clr> __forceinline const ssef_t insert( const ssef_t& a, const ssef_t& b ) { return _mm512_insert_ps(a, b, (dst << 4) | (src << 6) | clr); }
|
||||
template<size_t dst, size_t src> __forceinline const ssef_t insert( const ssef_t& a, const ssef_t& b ) { ssef_t c = a; c[dst] = b[src]; return c; }
|
||||
template<size_t dst> __forceinline const ssef_t insert( const ssef_t& a, const float b ) { ssef_t c = a; c[dst] = b; return c; }
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Transpose
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline void transpose(const ssef_t& r0, const ssef_t& r1, const ssef_t& r2, const ssef_t& r3, ssef_m& c0, ssef_m& c1, ssef_m& c2, ssef_m& c3)
|
||||
{
|
||||
c0[0] = r0[0]; c0[1] = r1[0]; c0[2] = r2[0]; c0[3] = r3[0];
|
||||
c1[0] = r0[1]; c1[1] = r1[1]; c1[2] = r2[1]; c1[3] = r3[1];
|
||||
c2[0] = r0[2]; c2[1] = r1[2]; c2[2] = r2[2]; c2[3] = r3[2];
|
||||
c3[0] = r0[3]; c3[1] = r1[3]; c3[2] = r2[3]; c3[3] = r3[3];
|
||||
}
|
||||
|
||||
__forceinline void transpose(const ssef_t& r0, const ssef_t& r1, const ssef_t& r2, const ssef_t& r3, ssef_m& c0, ssef_m& c1, ssef_m& c2)
|
||||
{
|
||||
c0[0] = r0[0]; c0[1] = r1[0]; c0[2] = r2[0]; c0[3] = r3[0];
|
||||
c1[0] = r0[1]; c1[1] = r1[1]; c1[2] = r2[1]; c1[3] = r3[1];
|
||||
c2[0] = r0[2]; c2[1] = r1[2]; c2[2] = r2[2]; c2[3] = r3[2];
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Reductions
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline const ssef_t vreduce_min(const ssef_t& v) { ssef_t h = min(shuffle<1,0,3,2>(v),v); return min(shuffle<2,3,0,1>(h),h); }
|
||||
__forceinline const ssef_t vreduce_max(const ssef_t& v) { ssef_t h = max(shuffle<1,0,3,2>(v),v); return max(shuffle<2,3,0,1>(h),h); }
|
||||
__forceinline const ssef_t vreduce_add(const ssef_t& v) { ssef_t h = shuffle<1,0,3,2>(v) + v ; return shuffle<2,3,0,1>(h) + h ; }
|
||||
|
||||
__forceinline float reduce_min(const ssef_t& v) { return _mm512_cvtss_f32(vreduce_min(v)); }
|
||||
__forceinline float reduce_max(const ssef_t& v) { return _mm512_cvtss_f32(vreduce_max(v)); }
|
||||
__forceinline float reduce_add(const ssef_t& v) { return _mm512_cvtss_f32(vreduce_add(v)); }
|
||||
|
||||
__forceinline size_t select_min(const ssef_t& v) { return bitscan(movemask(v == vreduce_min(v))); }
|
||||
__forceinline size_t select_max(const ssef_t& v) { return bitscan(movemask(v == vreduce_max(v))); }
|
||||
|
||||
__forceinline size_t select_min(const sseb_t& valid, const ssef_t& v) {
|
||||
const ssef_t a = select(valid,v,ssef_m(pos_inf));
|
||||
return bitscan(movemask(valid & (a == vreduce_min(a))));
|
||||
}
|
||||
__forceinline size_t select_max(const sseb_t& valid, const ssef_t& v) {
|
||||
const ssef_t a = select(valid,v,ssef_m(neg_inf));
|
||||
return bitscan(movemask(valid & (a == vreduce_max(a))));
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Memory load and store operations
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
__forceinline ssef_t load4f( const void* const a ) {
|
||||
return _mm512_extload_ps(a,_MM_UPCONV_PS_NONE,_MM_BROADCAST_4X16,_MM_HINT_NONE);
|
||||
}
|
||||
|
||||
__forceinline void store4f( void* ptr, const ssef_t& v ) {
|
||||
assert((size_t)ptr % 16 == 0);
|
||||
_mm512_mask_extpackstorelo_ps(ptr,0xf,v.m512,_MM_DOWNCONV_PS_NONE, _MM_HINT_NONE);
|
||||
}
|
||||
|
||||
__forceinline void storeu4f( void* ptr, const ssef_t& v ) {
|
||||
_mm512_mask_extpackstorelo_ps(ptr,0xf,v.m512,_MM_DOWNCONV_PS_NONE, _MM_HINT_NONE);
|
||||
_mm512_mask_extpackstorehi_ps(ptr,0xf,v.m512,_MM_DOWNCONV_PS_NONE, _MM_HINT_NONE);
|
||||
}
|
||||
|
||||
__forceinline void store4f( const sseb_t& mask, void* ptr, const ssef_t& f ) {
|
||||
store4f(ptr,select(mask,f,*(ssef_m*)ptr));
|
||||
}
|
||||
|
||||
__forceinline ssef_t load4f_nt(void* ptr) {
|
||||
return _mm512_extload_ps(ptr,_MM_UPCONV_PS_NONE,_MM_BROADCAST_4X16,_MM_HINT_NT);
|
||||
}
|
||||
|
||||
__forceinline void store4f_nt(void* ptr, const ssef_t& v) {
|
||||
assert((size_t)ptr % 16 == 0);
|
||||
_mm512_mask_extpackstorelo_ps(ptr,0xf,v.m512,_MM_DOWNCONV_PS_NONE, _MM_HINT_NT);
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// Output Operators
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
inline std::ostream& operator<<(std::ostream& cout, const ssef_m& a) {
|
||||
return cout << "<" << a[0] << ", " << a[1] << ", " << a[2] << ", " << a[3] << ">";
|
||||
}
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user