BRL-CAD/mkl-dnn
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
7,276 Commits 3 Branches 36 Tags
oidn
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Roy Oursler 7d78eb240d gpu: prevent segfaults from unchecked kernel creation failure 2021-04-27 17:08:06 -07:00
Roy Oursler 0fee9c7640 gpu: conv: winograd: fix cast to avoid kernel failure on gen11 
Prevent implicit conversion from VECT_BLOCK_DATA_T to BLOCK_DATA_T which fails
on windows on gen11.
 2021-02-17 10:54:22 -08:00
Roy Oursler d5bdfd5107 gpu: prelu: modify prelu fwd to zero padded memory 2021-02-16 16:37:48 -08:00
Roy Oursler 9e0a008621 gpu: ocl: winograd: implement nhwc format support 2021-02-10 17:27:55 -08:00
Roy Oursler b5fc62e783 gpu: ocl: winograd: remove unsupported dimension logic 
Additionally changes weights format to avoid likely reorders due to switching
between OIhw16i16 and IOhw16i16o formats.
 2021-02-10 17:27:55 -08:00
Roy Oursler 0fc2a40a30 gpu: ocl: merge gen9_conv_fwd and gen9_conv_wino offset calculation 2021-02-10 17:27:55 -08:00
Roy Oursler 9cdf6f828d gpu: ocl: cross_engine_reorder: remove unnecessary clean storage 
Removes dst from clean storage as reorder operations will make sure that the
blocked memory is zero padded correctly. This fixes an issue for gpu2cpu
reorders as well since dst could be a cpu buffer but the reorder is executed on
gpu.
 2021-01-22 13:05:19 -08:00
Roy Oursler 2954aa65ac tests: conv: wino: reduce f16 weights range to prevent testing error 
Reduce weights range to prevent the f16 winograd benchdnn tests from failing
with the Intel compiler. In this case, the data generated using icc results in
the winograd computation accumulating enough error to cause the test to fail,
despite computation proceeding as expected.
 2021-01-20 10:19:23 -08:00
Roy Oursler af199aa4b3 tests: benchdnn: matmul: remove unsupported 3d binary post_op test 2021-01-19 09:46:30 -08:00
Roy Oursler 34338803ef src: update copyright 
Update copyright for files used in modifying zero pad behavior.
 2021-01-13 14:22:59 -08:00
Roy Oursler 7580a3f4b4 common: memory: modify to zero pad destination buffers at execution 
This commits removes the now unused interface memory_t::zero_pad(stream). Note
this interface currently appears to be broken if it were used with SYCL on cpu.
 2021-01-13 14:22:58 -08:00
Roy Oursler 4f3b414fe5 tests: zeropad: modify test to call zeropad directly 2021-01-13 14:22:57 -08:00
Roy Oursler f1767fa52f common: create exec_ctx_t zero pad helper 2021-01-13 14:22:57 -08:00
Roy Oursler 84b1821cfa cpu: setup zero padded dst requirement at primitive execution 2021-01-13 14:22:57 -08:00
Roy Oursler 6deeb555d9 gpu: setup zero padded dst requirement at primitive execution 2021-01-13 14:22:56 -08:00
Roy Oursler 9e7ef78a3f common: memory: implement macros related to zero padded dst 2021-01-13 13:38:53 -08:00
Roy Oursler ae75cdbad1 tests: wino_iface: fix test for larger padding enabled on gpu 2021-01-12 19:40:51 -08:00
Roy Oursler 298322c55c fixup! src: gpu: ocl: rename x8s8s32x kernels to x8s8x 2021-01-12 15:59:33 -08:00
Roy Oursler 1902bb5d44 tests: benchdnn: update copyright 2021-01-12 13:14:08 -08:00
Roy Oursler 496ad4b071 tests: benchdnn: restrict gpu winograd test to limited padding size 2021-01-12 13:14:08 -08:00
Roy Oursler 90d29559ff tests: benchdnn: relax accuracy requirement for f16 gpu winograd conv 2021-01-12 13:14:08 -08:00
Roy Oursler ae719b3eb4 gpu: conv: wino: remove unnecessary winograd padding restriction 2021-01-12 13:14:08 -08:00
Roy Oursler b67f2acb8b gpu: conv: wino: update copyright 2021-01-12 13:14:08 -08:00
Roy Oursler a58ee0804d gpu: conv: wino: make scl a power of two to reduce computation error 2021-01-12 13:14:07 -08:00
Roy Oursler 01e69f9a6d gpu: conv: wino: implement initial fused 4x3 and 2x3 tile support 2021-01-12 13:14:07 -08:00
Roy Oursler fbb012d007 gpu: conv: wino: improve V transformation threading 2021-01-12 13:14:07 -08:00
Roy Oursler fcc68bd197 gpu: conv: wino: refactor threading logic into functions 2021-01-12 13:14:07 -08:00
Roy Oursler b3bb9cff5d gpu: conv: wino: refactor data transforms into functions 2021-01-12 13:14:07 -08:00
Roy Oursler 60059278ab gpu: conv: wino: reduce blocking on ow for small ow 2021-01-12 13:14:07 -08:00
Roy Oursler 5c7d2b1ddd gpu: conv: wino: refactor TRANS_DATA_T as VTRANS_DATA_T 2021-01-12 13:14:07 -08:00
Roy Oursler ecf7a2b268 gpu: conv: wino: implement vector sized data types 2021-01-12 13:14:07 -08:00
Roy Oursler 0da6c318a9 gpu: conf: wino: make 6x3 fused implementation more flexible 
Replace vector variables and manually unrolled computation with arrays and
unroll hinted for loops.
 2021-01-12 13:14:07 -08:00
Roy Oursler 54a869fc6c gpu: zero_pad: match verbose logging fields to other primitives 2020-12-11 09:59:42 -08:00
Roy Oursler 5c2b8c5890 gpu: conv: disable winograd convolution when alg_kind is auto 
There is no good heuristic to choose the optimal algorithm kind yet
 2020-12-10 17:03:21 -08:00
Roy Oursler 243a04509a gpu: ocl: set f32 winograd to default to 6x3 method 
This patch creates quite a bit of dead code, namely the logic for handling
non-fused Winograd algorithms, along with the 2x3 non-fused algorithm is no
longer use. This code is not removed at this time as Winograd is a work in
progress and some of this code may be needed again in the near future.
 2020-11-30 09:01:20 -08:00
Roy Oursler 5ac26697ec gpu: enable f32 data type for 6x3 winograd algorithm 2020-11-30 09:01:20 -08:00
Roy Oursler 09c5424079 tests: benchdnn: inputs: add gpu winograd tests 2020-11-25 10:52:40 -08:00
Roy Oursler 9cc8b5ea87 gpu: fix apply post ops in 6x3 fused winograd convolution 2020-11-25 10:52:40 -08:00
Roy Oursler 6581561888 tests: benchdnn: avoid unnecessary calls to zero_points.get() 
Avoid unnecessary calls to zero_points.get() as this calls the find() function
which is a relatively expensive operations.
 2020-11-25 10:51:54 -08:00
Roy Oursler e542f25a71 tests: benchdnn: use unordered_map for zero_points 
Change map to unordered map for zero_points since unordered_map is faster for
our lookup heavy usecase.
 2020-11-25 10:51:54 -08:00
Roy Oursler 6380a24e6b tests: benchdnn: create zero pad tests 2020-11-17 12:24:33 -08:00
Roy Oursler a1ecde1ba9 tests: benchdnn: add zero pad to benchdnn 2020-11-17 12:24:33 -08:00
Roy Oursler 6928069455 gpu: compute: fix race condition on zero_pad primitive creation 2020-11-12 15:54:38 -08:00
Roy Oursler ba540f14db gpu: compute: move zero pad resource allocation to primitive creation 2020-11-12 15:54:38 -08:00
Roy Oursler 04ab3e0b4e gpu: ocl: implement 6x3 fused Winograd convolution for f16 data type 2020-10-29 16:06:11 -07:00
Roy Oursler 20c79ada8b gpu: ocl: make winograd 2x3 labeling explicit 2020-10-29 16:06:11 -07:00
Roy Oursler adcc8cf604 gpu: ocl: fix incorrect divisibility condition for zero pad 2020-10-29 13:16:26 -07:00
Roy Oursler 52f7f75d80 gpu: ocl: use subgroups to write contiguous output in zero_pad 2020-10-23 14:05:13 -07:00
Roy Oursler ce70b7c165 gpu: ocl: balance zero_pad work unit size with number of work units 2020-10-23 14:05:13 -07:00
Roy Oursler f4dde5e712 gpu: ocl: specialize zero pad based on step_nelems 2020-10-23 14:05:13 -07:00
Roy Oursler 50ce2c2b49 gpu: compute: add verbose print to zero_pad primitive 2020-10-23 14:05:13 -07:00
Roy Oursler dfe3c149b6 tests: fix missing initialization for iface_wino_convolution test 2020-09-25 17:31:36 -07:00
Roy Oursler ce6e2f6431 gpu: ocl: disable f16 winograd due to poor performance 2020-09-23 12:51:01 -07:00
Roy Oursler d269b1844d gpu: ocl: fix winograd kernel to work with f16 data 2020-09-23 12:51:01 -07:00
Roy Oursler 7c28f32ecf tests: update test_iface_wino_convolution for gpu winograd support 2020-09-23 12:51:01 -07:00
Roy Oursler b7f030ef4c ocl: implement post ops and bias for winograd convolutions 2020-09-17 12:54:58 -07:00
Roy Oursler 8c2bbfedf7 ocl: implement initial optimized winograd version 2020-09-17 12:54:57 -07:00
Roy Oursler c7b91a87d2 gpu: ocl: implement initial gpu winograd convolution functionality 2020-09-17 12:54:57 -07:00
Roy Oursler 7a909cdbcb build: remove long-double flag from windows icl build 
The windows header file xlocnum which is a dependency for Gen
kernels (gen_gemm_kernel.cpp in particular) requires
sizeof(double) == sizeof(long double).
 2020-08-21 10:13:08 -07:00
Roy Oursler 57ceb24431 gpu: Implement lazy service stream initialization 
The service stream may never be used. Delay initialization until a service
stream is actually required.
 2020-07-06 10:15:40 -07:00
Roy Oursler 0730dbfe8d gpu: ocl: implement opencl zero_pad functionality 2020-06-24 15:51:00 -07:00
Roy Oursler 9a6d92e591 include: add max value to dnnl_primtive_kind_t and dnnl_query_t enum 
Add a max enum value to prevent UB associated with casts required internal to
dnnl when implementing internal only enum values. The value 0x7fff was chosen to
be portable within the C++ standard. While a larger value should not be an issue
as we are unlikely to support architectures with 2 byte int, it is also unlikely
that over 16,000 enum values will be required.
 2020-06-24 15:51:00 -07:00
Roy Oursler 218edc8dd8 gpu: compute: detach storage from kernel_arg_t struct 
Move value storage form kernel_arg_t to kernel_arg_list_t to enable more
range in data type sizes without using much more data than necessary
 2020-06-24 15:51:00 -07:00
Roy Oursler 77b818b48b common: create zero_pad function within stream_t 
Move zero_pad functionality into stream_t to abstract the zero_pad functionality
away from the memory. This is to enable implementing a gpu specific zero_pad
function.
 2020-06-24 15:50:59 -07:00
Roy OurslerandWierschem, Keola 8e74b950ae benchdnn: conv: add tests for memory format tag combinations 
Co-authored-by: Wierschem, Keola <keola.wierschem@intel.com>
 2020-05-29 20:34:21 -07:00
Roy Oursler 286eb60a8c fixup! cpu: conv: x64: avoid multiple calls to match_one_of_tag in bwd_d 
Add missing line initializing the weight tag
 2020-05-20 15:53:38 -07:00
Roy Oursler 778121af13 cpu: conv: x64: Implement bwd_w algorithm targeting nxc data formats 
This implements bwd_w algorithm in C++ using a loop order targeting the nxc data
format. This is currently implemented as a microkernel to improve development
speed as there may be some more optimization features that should be added such
as blocking on ic/oc for large ic/oc and threading on groups and possibly on the
kh/kw dimensions. After this implementation is feature complete, it may be
worthwhile to move more loops into the kernel to decrease DNNL's dependency on
an optimizing compiler as the performance can vary by around 10% from minor
changes to the C++ driver code.

The compute portion of this kernel unrolls on ow to amortize updates to the
weights buffers and unrolls on kw and ic to maximize the amount of computation
relative to loads from the src buffer.
 2020-05-20 15:15:44 -07:00
Roy Oursler 39e5b03a01 cpu: conv: x64: avoid multiple calls to match_one_of_tag in bwd_d 2020-05-20 13:44:45 -07:00
Roy Oursler 09707a332f cpu: conv: x64: add nxc format to avx2 f32 bwd_w convolutions 2020-05-20 13:44:44 -07:00
Roy Oursler 4bc173db6d cpu: conv: x64: add nxc format to avx2 f32 bwd_d convolutions 2020-05-20 13:44:44 -07:00
Roy Oursler 9d6b51443f cpu: conv: x64: add nxc format to avx2 f32 fwd convolutions 2020-05-20 13:44:44 -07:00
Roy Oursler 10f475e6e2 cpu: x64: conv: move avx2 bwd_w dst offset calculations to function 2020-05-20 13:44:44 -07:00
Roy Oursler d6219b7647 cpu: x64: conv: move avx2 bwd_w src offset calculations to function 2020-05-20 13:44:44 -07:00
Roy Oursler 3288c037df cpu: x64: conv: move avx2 bwd_w kernel offset calculations to function 2020-05-20 13:44:43 -07:00
Roy Oursler 094a1e5dc3 cpu: x64: conv: move avx2 bwd_d ddst offset calculations to function 2020-05-20 13:44:43 -07:00
Roy Oursler 159d666239 cpu: x64: conv: move avx2 bwd_d dsrc offset calculations to function 2020-05-20 13:44:43 -07:00
Roy Oursler 152292d727 cpu: x64: conv: move avx2 bwd_d kernel offset calculations to function 2020-05-20 13:44:43 -07:00
Roy Oursler 9a0482f270 cpu: x64: conv: move avx2 fwd dst offset calculations to function 2020-05-20 13:44:43 -07:00
Roy Oursler 05d9d5db22 cpu: x64: conv: move avx2 fwd src offset calculations to function 2020-05-20 13:44:42 -07:00
Roy Oursler a50017a94a cpu: x64: conv: move avx2 fwd kernel offset calculations to function 2020-05-20 13:44:42 -07:00
Roy Oursler 8ea1ba8879 cpu: x64: remove unused oc_blocks tag from avx2 forward conv 2020-05-20 13:44:42 -07:00
Roy Oursler 3074d098aa cpu: x64: remove unused pad_tag from avx2 fwd conv functions 2020-05-20 13:44:42 -07:00
Roy Oursler 765676e410 cpu: x64: remove dead code from avx2 fwd conv init_conf 2020-05-20 13:44:42 -07:00
Roy Oursler 500ff083a3 common: memory_debug: reduce small buffer memory initialization 2020-04-23 15:42:15 -07:00
Roy Oursler 517dc85745 common: memory_debug: remove implicit const cast in protect_buffer 2020-04-23 15:42:15 -07:00
Roy Oursler 959ba74b89 common: memory_tracking: add non-const iterator to registry_t 2020-04-23 15:42:15 -07:00
Roy Oursler ff2bdb3fba tests: Implement memory debug mode 
This initial implementation mainly supports cpu on linux.
 2020-04-21 14:51:50 -07:00
Roy Oursler 099e67b8cb memory_tracking: Implement iterator for registry_t 2020-04-21 14:51:50 -07:00
Roy Oursler 662a604077 scratchpad: Modify scratchpad book to track minimum data alignment 
Implement minimum data alignment tracking in scratchpad to aid memory debug
functionality.
 2020-04-21 14:51:50 -07:00
Roy Oursler d7907e20ee utils: Add getpagesize function 2020-04-21 14:51:49 -07:00
Roy Oursler 41076f50ee cpu: matmul: Fix calculation of ldc 
For scrathpad and dnnl allocated memory, the acc buffer
has dimension m x n, so ldc must have the value n.
 2020-04-06 16:26:53 -07:00
Roy Oursler b45b1160d9 cpu: matmul: Fix calculation of ldc 
For scrathpad and dnnl allocated memory, the acc buffer
has dimension m x n, so ldc must have the value n.
 2020-04-06 16:24:56 -07:00
Roy Oursler 0d5529b560 cpu: conv: int8: Fix scratchpad buffer overflow 
Fix overflow of the key_conv_adjusted_scales buffer on the scratchpad by fixing
initialization and size allocation.
 2020-04-01 16:22:42 -07:00
Roy Oursler baec2bc65f cpu: conv: bf16 bwd_w: Fix zeroing to prevent uninitialized weights 2020-03-13 16:13:44 -07:00
Roy Oursler 2c9f45f45c cpu: conv: bf16 bwd_w: Remove ur_ow conditions in full_spat kernel 
Fixes an issue where ur_ow conditions cause a buffer overflow on the transposed
source buffer.
 2020-03-13 16:13:44 -07:00
Roy Oursler 0d283894be cpu: conv: bf16 bwd_w: Fix incorrect tr_pad calculation 
Increase tr_pad in some cases to prevent overflow read causing invalid data to
be included in bwd_w computation
 2020-03-06 13:18:49 -08:00
Roy Oursler c391a63464 cpu: conv: bf16 bwd_w: Fix incorrect tr_pad calculation 
Increase tr_pad in some cases to prevent overflow read causing invalid data to
be included in bwd_w computation
 2020-03-06 13:09:48 -08:00
Roy Oursler e46a4ce71d cpu: conv: bf16 bwd_w: Implement height and depth threading 2020-03-04 16:36:49 -08:00
Roy Oursler 8bfb42f442 cpu: conv: bf16 bwd_w: Replace define with dnnl_thr_syncable 2020-03-04 16:36:49 -08:00
Roy Oursler b96a52fbf0 cpu: conv: bf16 bwd_w: Remove setting of unused jcp.ur_w value 2020-03-04 16:36:49 -08:00