Merge RAPIDS-Triton into developer tools - #13

Draft

wphicks wants to merge 216 commits into

triton-inference-server:mainfrom

wphicks:fea-backend

Commits on Sep 7, 2021

Add readme
wphicks
committed
6043492
Add README
wphicks
committed
fde4f80
Move basic parts into new repo
wphicks
committed
b4fb8da
Add license and readme
wphicks
committed
124b192
Add Triton ModelState interface
wphicks
committed
f0e1691
Provide API interface for everything except execution
wphicks
committed
c20fa75
Begin batch implementation
wphicks
committed
c340158
Use collector and responder for Batch
wphicks
committed
f4afbbb
Add configuration reading to shared state
wphicks
committed
ce34244
Provide basics of API functions for models
wphicks
committed
7369b4e
Set up output synchronization
wphicks
committed
995b70b
Set up CMake infrastructure
wphicks
committed
27e9b8a
Add basic test
wphicks
committed
4f29cc7
Add build control test
wphicks
committed
6581f45
Bring in Triton dependencies
wphicks
committed
347fb1d
Add tests for exceptions
wphicks
committed
58ab757
Test narrow
wphicks
committed
2e318df
Test memory allocation functions
wphicks
committed
ee6c09e
Test memory copy functions
wphicks
committed
f05351f
Add buffer tests
wphicks
committed
28cccdd
Test dtype
wphicks
committed
b3300f6
Test Tensor objects
wphicks
committed
a46a503
Add test skeleton for all Triton headers
wphicks
committed
56de966
Add tests for logging
wphicks
committed
c0a4023
Add tests for Batch
wphicks
committed
d8177cd
Properly encapsulate state types in Batch
wphicks
committed
23041d8
Begin adding statistics reporting
wphicks
committed
2933d20
Refactor API logic into library
wphicks
committed
ec7c42f
Correct batch statistics reporting call
wphicks
committed
b8178dc
Add Triton API tests
wphicks
committed
af258c9
Re-enable execute tests
wphicks
committed
e70732a
Correctly handle const in input types
wphicks
committed
75295c3
Clean up log message construction
wphicks
committed
f561f4c
Reorganize and simplify example backend
wphicks
committed
541d682
Add complete compilation of backend library
wphicks
committed
b951775
Do not access moved-from object
wphicks
committed
5927547
Improve logging for type mismatches
wphicks
committed
d0ecbc9
Redo output shape determination code
wphicks
committed
abd6e9e
Default to running example server in Dockerfile
wphicks
committed
75e55d8
Begin filling out docs
wphicks
committed
482b157
Add basics from identity example
wphicks
committed
ddcd1c0

Commits on Sep 8, 2021

Correct parameter fetching logic
wphicks
committed
c256fe1
Add stream interface for logging
wphicks
committed
78b0777
Clean up const declarations in logging
wphicks
committed
ed47a54
Provide shared state example docs
wphicks
committed
ff2032a
Add missing load in instance initialization
wphicks
committed
c91cd94
Add move assignment operator to Buffer
wphicks
committed
17e9024
Add Buffer move assignment test
wphicks
committed
78dcb56

Commits on Sep 9, 2021

Fill out model loading implementation
wphicks
committed
95f2068

Commits on Sep 10, 2021

Use default move operators for Buffer
wphicks
committed
2d6d53c
Add full prediction logic
wphicks
committed
7f67234
Add Python testing package
wphicks
committed
6d42da2
Remove circular import
wphicks
committed
19fa354
Use find_packages correctly
wphicks
committed
15b36ed
Add example models
wphicks
committed
720c64f
Add conclusion
wphicks
committed
74adfe5
Correct logging message in get_random_seed
wphicks
committed
deaa77f
Add links to Triton docs
wphicks
committed
7f5533d
Add E2E tests
wphicks
committed
c29835f
Remove copied comments
wphicks
committed
2d97788
Add contributing and usage docs
wphicks
committed
0a6e939
Fix style
wphicks
committed
1241f0b

Commits on Sep 20, 2021

Use remove_const_t where useful

wphicks
and
divyegala
authored
cf75925
Use is_same_v where useful

wphicks
and
divyegala
authored
dc6f4fe
Use is_same_v where useful

wphicks
and
divyegala
authored
a83f136
Use braced initialization more consistently
wphicks
committed
4fc3028
Add C++ linting infrastructure
wphicks
committed
17b8499
Use braced initialization where possible
wphicks
committed
31d60fc
Correct copyright year in cpp/CMakeLists.txt

wphicks
and
robertmaynard
authored
b7787c1
Make use of rapids_cmake_install_lib_dir
wphicks
committed
78f219c
Add comment on thrust inclusion
wphicks
committed
b746943
Make use of rapids-cmake for gtest and rmm
wphicks
committed
467360e
Eliminate unnecessary Thrust inclusion
wphicks
committed
bb73c2a
Link to Triton from README
wphicks
committed
d1c29ec
Add logging of Triton-specific build options
wphicks
committed
39d5e5d
Correct copyright year range

wphicks
and
dantegd
authored
b7e0abc
Add CUDA 11.2 development environment
wphicks
committed
4b3d857
Force double linebreak with spaces in DCO
wphicks
committed
0859765
Correct mismatched copyright headers
wphicks
committed
1b83010

Commits on Sep 21, 2021

Add missing client definition in README code
wphicks
committed
0b6d1d3

Commits on Sep 22, 2021

Pass optional by reference

wphicks
and
lowener
authored
e0aa081
Pass optionals by reference

wphicks
and
lowener
authored
8a2e3c3
Correct typo in Tensor documentation

wphicks
and
lowener
authored
207d154
Explicitly label auto pointers

wphicks
and
lowener
authored
7ddc0af
Explicitly label auto pointer

wphicks
and
lowener
authored
9786d1a
Explicitly label auto pointer
wphicks
committed
07ea40b
Allow modifications on state in get_stream
wphicks
committed
e2d3026

Commits on Sep 23, 2021

Construct closures for Batch separately
wphicks
committed
05b50b2
Change const model to allow for stream-related state
wphicks
committed
b9a9ef6

Commits on Sep 27, 2021

Commits on Sep 28, 2021

Update Greek characters for MD rendering
wphicks
committed
e8e4032
Add option to squeeze output shape
wphicks
committed
420f621
Actually erase elements during squeeze
wphicks
committed
11ff73b

Commits on Oct 5, 2021

Commits on Oct 7, 2021

Commits on Oct 8, 2021

Commits on Oct 11, 2021

Update to Triton 21.09
wphicks
committed
a08afc6
Update Triton bug warning
wphicks
committed
b4cd889
Update rapids::copy to allow template param omission
wphicks
committed
334b504
Fix style
wphicks
committed
54f1815
Provide complete initial implementation of RAPIDS-Triton
wphicks
authored
c6c2aa7

Commits on Oct 12, 2021

Provide basic usage information
wphicks
committed
2c4c1d1
Provide basic usage information
wphicks
authored
5421381
Update default install prefix
wphicks
committed
6ba3e8e
Upgrade to Triton 21.09
wphicks
committed
9cbfe04
Use AAA style in loop
wphicks
committed
b7b5165
Disable shared memory tests until after 21.10
wphicks
committed
81846b0
Specify Triton repo versions in Dockerfile
wphicks
committed
eba3429
Correct stream handling
wphicks
committed
8c67ff4
Always set device for execution and initialization
wphicks
committed
c04d920
Revert "Update to Triton 21.09"
wphicks
committed
1fe80cb
Revert to Triton 21.08
wphicks
authored
6e3d231

Commits on Oct 13, 2021

Revert to Triton 21.08 and pin RAPIDS-Triton to main
wphicks
committed
9086c34
Update docs based on RAPIDS-Triton main
wphicks
committed
1a0c2b7
Provide initial implementation of full example
wphicks
authored
9956c9f

Commits on Oct 19, 2021

Update batch logic for new pinned memory default
wphicks
committed
f963b46
Update batch logic for new pinned memory default
wphicks
authored
5419f29

Commits on Oct 21, 2021

Pin to RAPIDS-Triton 21.10
wphicks
committed
dd2b509

Commits on Oct 22, 2021

Merge pull request #2 from rapidsai/fea-pin_2110
wphicks
authored
4dfe79e

Commits on Oct 26, 2021

Remove unnecessary cudaSetDevice calls
wphicks
committed
6d3d7cc

Commits on Oct 27, 2021

Merge pull request #3 from rapidsai/fea-unnecessary_set_device
wphicks
authored
bb89a57

Commits on Nov 15, 2021

Provide initial CPU-only implementation
wphicks
committed
6522ce6

Commits on Nov 16, 2021

Update unit tests for CPU-only build
wphicks
committed
c43b35f
Add usage information on CPU-only helpers
wphicks
committed
394e2a7
Add build script
wphicks
committed
e9cf10d

Commits on Nov 18, 2021

Provide working build/tests for CPU-only mode
wphicks
committed
5b87634
Remove logs directory
wphicks
committed
30bb810
Enable ccache
wphicks
committed
f6b02cd

Commits on Nov 22, 2021

Make cleaner use of cmake generator expressions
wphicks
committed
f01823c
Remove debug ccache reporting
wphicks
committed
2aed622

Commits on Nov 23, 2021

Remove GPU restriction on C++ standard
wphicks
committed
47c808d

Commits on Dec 1, 2021

Remove old commented-out code
wphicks
committed
4d01da3
Remove unnecessary capture from lambda
wphicks
committed
2009803

Commits on Dec 2, 2021

Enable CPU-only build
wphicks
authored
4efd4de

Commits on Dec 8, 2021

Commits on Dec 9, 2021

Guard against CUDA shared memory import on CPU-only machine
wphicks
committed
47782dc
Guard against missed cuda_shared_memory import
wphicks
committed
03a5977

Commits on Dec 10, 2021

Provide better error messages for import failures
wphicks
committed
3a05c3b

Commits on Dec 13, 2021

Merge pull request #11 from rapidsai/bug-cpu_cmake
wphicks
authored
51bb797
Update version numbers for 21.12 release
wphicks
committed
ec154f1

Commits on Dec 17, 2021

Merge pull request #12 from rapidsai/fea-2112_version_bumps
wphicks
authored
ff82f51

Commits on Jan 21, 2022

Update versions of upstream RAPIDS libraries
wphicks
committed
1b953ba
Remove no-cache debug lines
wphicks
committed
4aac005

Commits on Jan 24, 2022

Merge pull request #14 from rapidsai/fea-2202_version_bump
wphicks
authored
8ff2a8d

Commits on Jan 27, 2022

Add asynchronous prediction to RAPIDS-Triton client
wphicks
committed
c77b633

Commits on Feb 2, 2022

Add multimodel evaluation helper
wphicks
committed
0817313
Add multimodel tests
wphicks
committed
037770e
Add placeholder files for model directories
wphicks
committed
c44ff36
Remove unused parameter
wphicks
committed
af10cf7
Remove unused device_id parameters
wphicks
committed
5bc5164

Commits on Feb 4, 2022

Only add callback when necessary to release I/O
wphicks
committed
34c9def
Fix syntax error
wphicks
committed
775eadc

Commits on Feb 7, 2022

Update python/rapids_triton/utils/safe_import.py

wphicks
and
lowener
authored
44c57a8
Update function name to reflect asynchronicity
wphicks
committed
301cdd5

Commits on Feb 9, 2022

Initial code
divyegala
committed
43771e9

Commits on Feb 22, 2022

Commits on Feb 24, 2022

review comments
divyegala
committed
3bd5f1e
Merge pull request #18 from divyegala/input-by-name
wphicks
authored
a2a6413

Commits on Mar 10, 2022

Merge pull request #15 from rapidsai/fea-async
wphicks
authored
7c4592e

Commits on Oct 5, 2022

Upstream rapid-json
GuanLuo
committed
3f61d1a
Replace RAPIDS keyword with developer_tools::backend
GuanLuo
committed
2861873
Add CI test for backend unit test
GuanLuo
committed
ebbb073

Commits on Oct 7, 2022

Merge RAPIDS-Triton project into developer_tools
wphicks
committed
4b7cd17
Add initial group of codeowners
wphicks
committed
f7ebb6a
Merge branch 'gluo-backend' into fea-backend
wphicks
committed
c9d8cdd
Remove incorrectly-moved files
wphicks
committed
5654d51
Remove redundant test script
wphicks
committed
5720f30
Merge remote-tracking branch 'rapids-triton-le/main' into fea-backend
wphicks
committed
2aa2d12
Move linear example to examples directory
wphicks
committed
0ee9531
Restore README
wphicks
committed
b497f91
Correct global CODEOWNER configuration
wphicks
committed
fb5a3db

Commits on Oct 10, 2022

Replace rapids_triton with triton_backend
wphicks
committed
cb46f12
Replace rapids namespace with dev_tools
wphicks
committed
15a690d
Replace Rapids template names with Backend
wphicks
committed
0abaca1
Rewrite comments referencing RAPIDS
wphicks
committed
da604ca
Revert "Replace Rapids template names with Backend"
wphicks
committed
d476da9
Replace Rapids with DevTools in template names
wphicks
committed
17bebde
Update include path for dev_tools
wphicks
committed
e69a803
Update template names to DevTools in identity
wphicks
committed
399fe74
Update name of identity backend
wphicks
committed
fa82f5b
Update rapids namespace to dev_tools in identity
wphicks
committed
aff27a0
Update CMake variable names
wphicks
committed
f86e69b
Update include path in tests
wphicks
committed
386a8eb
Update remaining names in tests
wphicks
committed
c6da4f2
Update names in cmake
wphicks
committed
3143d9a
Update names in Dockerfile and CMakeLists
wphicks
committed
793b57d
Update naming in build scripts
wphicks
committed
fad694a
Update conda names
wphicks
committed
47a83f6
Bring in updates from post 22.08 RAPIDS-Triton
wphicks
committed
3fa14a7
Begin filling out top-level README
wphicks
committed
2dbff7e

Commits on Dec 15, 2022

Update conda dependencies
wphicks
committed
a039d7a
Correctly fetch backend dependencies
wphicks
committed
b618621
Update include paths
wphicks
committed
3d2cf74
Update namespace for developer_tools
wphicks
committed
507e44e

Commits on Jan 17, 2023

Bump version numbers
wphicks
committed
1d95ae2