From fbea7eb57874ca8165dcdf74beb55c34a87115c2 Mon Sep 17 00:00:00 2001 From: Aakarsh Kashyap Date: Thu, 2 Apr 2026 02:38:02 +0530 Subject: [PATCH 01/27] Added time field in make.sh --- make.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/make.sh b/make.sh index 23c57b2..27ac66a 100755 --- a/make.sh +++ b/make.sh @@ -1,2 +1,2 @@ -cmake -B build -DCMAKE_BUILD_TYPE=Debug -cmake --build build +time cmake -B build -DCMAKE_BUILD_TYPE=Debug +time cmake --build build -j16 From 7ca90a4d8e13c3e598f07f732ce55c438aedf199 Mon Sep 17 00:00:00 2001 From: Aakarsh Kashyap Date: Thu, 2 Apr 2026 11:22:37 +0530 Subject: [PATCH 02/27] chore/ Running summarizer --- scripts/coldstart/coldstart.prev | 40 ++++++++++++++++++++++++++++++++ scripts/coldstart/coldstart.sh | 4 ++-- 2 files changed, 42 insertions(+), 2 deletions(-) diff --git a/scripts/coldstart/coldstart.prev b/scripts/coldstart/coldstart.prev index e69de29..2abb2e0 100644 --- a/scripts/coldstart/coldstart.prev +++ b/scripts/coldstart/coldstart.prev @@ -0,0 +1,40 @@ +Based on the recent logs and the state of the TODO list, here is the reconstruction of your mental context: + +### 1. What I Was Doing +I have just finished the basic implementation of **ReLU and Bias addition** (Commit `3b68101`). The project is currently transitioning from a simple tensor container to a functional compute graph. The "Cold start agent" is now functional, and I’ve been running a "summarizer" (likely an automated documentation or analysis tool) to track progress. My main goal now is **stabilizing the GPU backend assignment** and implementing core math kernels. + +### 2. Active TODOs (Grouped) + +**GPU & Backend Assignment (Critical)** +* **Memory Management:** Implement the logic for copying memory from parent nodes to child nodes in the GPU backend (`assignBackend.cu`). +* **Logic Refinement:** Build out the execution logic for the GPU part of the graph and resolve how to access execution nodes within the graph context. +* **Config Parser:** Implement the `CONFIG.soft` parser to allow dynamic backend assignment instead of hardcoding. +* **Stream Optimization:** Review the execution flow to potentially use CUDA streams for better performance. + +**Math Kernels (High Priority)** +* **Matrix Multiplication:** Currently a placeholder; needs a full implementation in `mul.cpp`. +* **Transpose Optimization:** Implement "Loop Tiling" in `transpose.cpp` to improve cache locality/performance on the CPU. + +**Technical Debt & Refactoring** +* **Multi-Dtype Support:** The system is currently "hardcoded" to `float32`. I need to refactor ReLU, Mul, and the GPU assignment logic to use `tensor_dtype_sizeof`. +* **Memory Layout:** Implement "contiguous logic" for tensors to ensure kernels don't break on sliced or non-contiguous data. + +### 3. Current State of the Code +* **Implemented:** Basic Tensor structure, ReLU, Bias addition, and the "Cold Start" agent. +* **Partially Done:** The GPU backend assignment (`assignBackend.cu`) exists but lacks the data-flow logic (parent-to-child memory copies). +* **Missing:** Matrix multiplication (the engine of the framework), multi-type support, and efficient GPU memory handling. + +### 4. My Likely Next Steps +1. **Complete the GPU Data Flow:** Finish the logic in `assignBackend.cu` that handles how children nodes fetch data from parents. This is the biggest blocker for running a full graph on GPU. +2. **Implement Matrix Multiplication:** The framework can't do much without `matmul`. I should implement a basic version first, then worry about optimization. +3. **Address the `float32` assumption:** Before the codebase gets too large, I need to start replacing `float` with the dtype abstraction I planned. +4. **Loop Tiling for Transpose:** A quick win for CPU performance that I've already noted down. + +### 5. Important Context I Should Not Forget +* **Dtype Assumption:** Almost everywhere in the math backends, we are assuming `float32_t`. If I start working on an op and it crashes with other types, this is why. +* **Backend Assignment:** The current logic for deciding if a node is CPU or GPU is still primitive. I was planning to use a "soft" configuration file to drive this. + +### 6. Risks / Tricky Areas +* **Graph Node Access:** In `assignBackend.cu`, there is uncertainty about how to correctly reference an execution node's place in the graph when we only have partial metadata. +* **Memory Contiguity:** If I implement kernels assuming contiguous memory without finishing the "contagious logic" TODO, I will run into silent data corruption or segfaults during slicing/reshaping. +* **GPU Streams:** If I move to streams for performance, I need to be careful about synchronization points so children don't start before parents finish. diff --git a/scripts/coldstart/coldstart.sh b/scripts/coldstart/coldstart.sh index 8b3c45f..78a0e9c 100755 --- a/scripts/coldstart/coldstart.sh +++ b/scripts/coldstart/coldstart.sh @@ -1,6 +1,6 @@ cp coldstart.tmp coldstart.prev 2>/dev/null -git add .. +git add ../.. git commit -m "chore/ Running summarizer" date=$(date +%Y-%m-%d_%H-%M-%S) @@ -19,6 +19,6 @@ filename="agent_summary_${date}.log" cat logs/${filename} > coldstart.tmp -git add .. +git add . git commit -m "chore/ Summarizer completion logging" From ccfc4a2f675fb35940b7de0806e025377d27b239 Mon Sep 17 00:00:00 2001 From: Aakarsh Kashyap Date: Thu, 2 Apr 2026 15:47:08 +0530 Subject: [PATCH 03/27] Completed MSE, passed smoke tests Written SUB, SQUARE, MEAN --- include/soft-cuda/tensor/api.h | 15 ++++++- main.cpp | 74 +++++++++---------------------- src/backend_cpu/include/mean.h | 2 + src/backend_cpu/include/sub.h | 6 +++ src/backend_cpu/math/add_bias.cpp | 0 src/backend_cpu/math/mean.cpp | 26 +++++++++++ src/backend_cpu/math/mse.cpp | 3 ++ src/backend_cpu/math/square.cpp | 5 +++ src/backend_cpu/math/sub.cpp | 27 +++++++++++ src/core/include/tensor/tensor.h | 2 + src/core/tensor/tensor.cpp | 10 +++++ src/internal_header.h | 2 + 12 files changed, 117 insertions(+), 55 deletions(-) create mode 100644 src/backend_cpu/include/mean.h create mode 100644 src/backend_cpu/include/sub.h delete mode 100644 src/backend_cpu/math/add_bias.cpp create mode 100644 src/backend_cpu/math/mean.cpp create mode 100644 src/backend_cpu/math/mse.cpp create mode 100644 src/backend_cpu/math/square.cpp create mode 100644 src/backend_cpu/math/sub.cpp diff --git a/include/soft-cuda/tensor/api.h b/include/soft-cuda/tensor/api.h index befb6e9..eaf4488 100644 --- a/include/soft-cuda/tensor/api.h +++ b/include/soft-cuda/tensor/api.h @@ -186,6 +186,7 @@ void tensor_print_data(tensor_t *t); // DONE tensor_t *tensor_mul(tensor_pool_t *pool, tensor_t *x, tensor_t *y); +tensor_t *tensor_square(tensor_pool_t *pool, tensor_t *x); /* The naive version of matrix multiplication * @param pool Pointer to the tensor pool * @param x Pointer to the tensor which will be mulptiplied @@ -221,6 +222,16 @@ tensor_t *tensor_add(tensor_pool_t *pool, tensor_t *x, tensor_t *y); // DONE tensor_t *tensor_add_bias(tensor_pool_t *pool, const tensor_t *xw, const tensor_t *bias); +/* + * Do matrix subtraction + * @param out Pointer to the tensor where result will be stored + * @param x Pointer to the tensor which will be subtracted from + * @param y Pointer to the tensor which will be subtracted + * + * @return Returns a tensor object with operation set. + * */ +// DONE +tensor_t *tensor_sub(tensor_pool_t *pool, tensor_t *a, tensor_t *b); ///////////////////////////////////////////////////////////// /// DEPRECEATED tensor_mul operation handles it automatically // /* @@ -242,6 +253,8 @@ tensor_t *tensor_add_bias(tensor_pool_t *pool, const tensor_t *xw, const tensor_ // DONE tensor_t *tensor_relu(tensor_pool_t *pool, tensor_t *a); +// DONE +tensor_t *tensor_mean(tensor_pool_t *pool, tensor_t *a); // Compares result // return how correct we were b/w 0-1 tensor_t *tensor_mse_loss(tensor_pool_t *pool, tensor_t *predictions, tensor_t *target); @@ -258,7 +271,7 @@ tensor_t *tensor_cross_entropy_loss(tensor_pool_t *pool, const tensor_t *predict // Evalutes the operation(Forward) with depth=1 // @return boolean flag for status // DONE -bool tensor_evaluate( tensor_pool_t *pool,tensor_t *t, float *d_a, float *d_b, float *d_res); +bool tensor_evaluate( tensor_pool_t *pool,tensor_t *t, float *d_a = nullptr, float *d_b = nullptr, float *d_res= nullptr); // DONE bool tensor_evaluate_GPU( tensor_pool_t *pool,tensor_t *t, float *d_a, float *d_b, float *d_res); diff --git a/main.cpp b/main.cpp index 4ede92f..0835779 100644 --- a/main.cpp +++ b/main.cpp @@ -6,71 +6,37 @@ using namespace std; int main() { - // Create pools tensor_pool_t *pool = tensor_pool_create(1024 * 1024); - tensor_pool_t *pool_2 = tensor_pool_create(1024 * 1024); - tensor_pool_t *pool_gpu = tensor_pool_create(1024 * 1024, true); assert(pool != NULL); - assert(pool_gpu != NULL); cout << "=========================================== \n"; - cout << "=============TESTING DAG VERIFICATION============== \n"; - - float val_a[900]{}; - float val_b[900]{}; - float val_c[900]{}; - float val_d[900]{}; - - // TACTICAL FIX: Make 'e' the exact same size to avoid a GPU segfault - // before we implement a dedicated GPU broadcasting kernel! - float val_e[900]{}; + cout << "=============TESTING SUBTRACTION============== \n"; + float val_a[] = {1.0f, 2.0f, 3.0f, 4.0f}; + float val_b[] = {1.0f, 2.0f, 3.0f, 4.0f}; - uint32_t dims_a[] = {30, 30}; - uint32_t dims_b[] = {30, 30}; - uint32_t dims_c[] = {30, 30}; - uint32_t dims_d[] = {30, 30}; - uint32_t dims_e[] = {30, 30}; // Matched dimensions! + uint32_t dims_a[] = {2, 2}; + uint32_t dims_b[] = {2, 2}; tensor_t *a = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_a, val_a); tensor_t *b = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_b, val_b); - tensor_t *c = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_c, val_c); - tensor_t *d = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_d, val_d); - tensor_t *e = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_e, val_e); - tensor_fill_random_normal(a, 10.1, 5.7); - tensor_fill_random_normal(b, 10.1, 5.7); - tensor_fill_random_normal(c, 10.1, 5.7); - tensor_fill_random_normal(d, 10.1, 5.7); - tensor_fill_random_normal(e, 10.1, 5.7); - tensor_t *f = tensor_mul(pool, a, b); - tensor_t *g = tensor_mul(pool, b, c); - tensor_t *h = tensor_mul(pool, e, f); - tensor_t *i = tensor_mul(pool, h, a); - tensor_t *j = tensor_mul(pool, i, g); - + + tensor_t *c = tensor_mean(pool, a); cout << "=========================================== \n"; cout << "==============LAZY EVAL DONE=============== \n"; - - std::vector seq; - bool oki = verifyIfDAG(pool_2, j, seq); - - // Ensure assignDevice is returning device_type::GPU under the hood! - assignBackendGraph(pool_gpu, seq); - tensor_graph_forward_evaluate(pool, pool_gpu, seq); - - if (oki) { - cout << "=========================================== \n"; - cout << "========= VRAM TRACE ============= \n"; - - execution_node_t *node = seq.back(); - - // Pull it back to the CPU - execution_node_to_host(node); - printExecutionNode(node); - cout << "\n"; - } + bool ok = tensor_evaluate(pool, c); + assert(ok); + std::cout << "============================================= \n"; + std::cout << "===============ORIGINAL TENSOR=============== \n"; + tensor_print_data(a); + std::cout << "============================================= \n"; + std::cout << "===============SQUARED TENSOR=============== \n"; + // tensor_print_data(c); + float* data = (float*)tensor_get_data(c); + std::cout << data[0] << "\n"; + std::cout << "============================================= \n"; + // debug("result[0] = %f\n", result[0]); + // debug("result[1] = %f\n", result[1]); tensor_pool_destroy(pool); - tensor_pool_destroy(pool_2); - tensor_pool_destroy(pool_gpu); return 0; } diff --git a/src/backend_cpu/include/mean.h b/src/backend_cpu/include/mean.h new file mode 100644 index 0000000..e82f37f --- /dev/null +++ b/src/backend_cpu/include/mean.h @@ -0,0 +1,2 @@ +#pragma once +bool tensor_op_mean(tensor_pool_t *pool, tensor_t *t); diff --git a/src/backend_cpu/include/sub.h b/src/backend_cpu/include/sub.h new file mode 100644 index 0000000..a82defb --- /dev/null +++ b/src/backend_cpu/include/sub.h @@ -0,0 +1,6 @@ +#pragma once + +tensor_t *tensor_sub(tensor_pool_t *pool, tensor_t *a, tensor_t *b); + + +bool tensor_op_sub(tensor_pool_t *pool, tensor_t *t); diff --git a/src/backend_cpu/math/add_bias.cpp b/src/backend_cpu/math/add_bias.cpp deleted file mode 100644 index e69de29..0000000 diff --git a/src/backend_cpu/math/mean.cpp b/src/backend_cpu/math/mean.cpp new file mode 100644 index 0000000..ffb9ef0 --- /dev/null +++ b/src/backend_cpu/math/mean.cpp @@ -0,0 +1,26 @@ +#include "internal_header.h" + +tensor_t *tensor_mean(tensor_pool_t *pool, tensor_t *a) { + assert(pool != NULL); + assert(a != NULL); + + tensor_t *t = tensor_dtype_create(pool, a->dtype, 0, 0, NULL); + if (t == NULL) { + return NULL; + } + + t->op = tensor_op_t::MEAN; + t->a = a; + return t; +} + +bool tensor_op_mean(tensor_pool_t *pool, tensor_t *t) { + assert(pool != NULL); + assert(t != NULL); + float sum{}; + for (uint32_t i = 0; i < t->a->nvalues; i++) { + sum += ((float *)t->a->data)[i]; + } + ((float *)t->data)[0] = sum/(float)(t->a->nvalues); + return true; +} diff --git a/src/backend_cpu/math/mse.cpp b/src/backend_cpu/math/mse.cpp new file mode 100644 index 0000000..9a133d8 --- /dev/null +++ b/src/backend_cpu/math/mse.cpp @@ -0,0 +1,3 @@ +#include "internal_header.h" + + diff --git a/src/backend_cpu/math/square.cpp b/src/backend_cpu/math/square.cpp new file mode 100644 index 0000000..f33715a --- /dev/null +++ b/src/backend_cpu/math/square.cpp @@ -0,0 +1,5 @@ +#include "internal_header.h" + +tensor_t *tensor_square(tensor_pool_t *pool, tensor_t *x) { + return tensor_mul(pool, x, x); +} diff --git a/src/backend_cpu/math/sub.cpp b/src/backend_cpu/math/sub.cpp new file mode 100644 index 0000000..3bd133c --- /dev/null +++ b/src/backend_cpu/math/sub.cpp @@ -0,0 +1,27 @@ +#include "internal_header.h" + +tensor_t *tensor_sub(tensor_pool_t *pool, tensor_t *a, tensor_t *b) { + assert(pool != NULL); + assert(a != NULL); + assert(b != NULL); + + tensor_t *t = tensor_dtype_create(pool, a->dtype, a->ndims, a->dims, NULL); + if (t == NULL) { + return NULL; + } + + t->op = tensor_op_t::SUB; + t->a = a; + t->b = b; + return t; +} + +bool tensor_op_sub(tensor_pool_t *pool, tensor_t *t) { + assert(pool != NULL); + assert(t != NULL); + + for (uint32_t i = 0; i < t->dims[0] * t->dims[1]; i++) { + ((float *)t->data)[i] = ((float *)t->a->data)[i] - ((float *)t->b->data)[i]; + } + return true; +} diff --git a/src/core/include/tensor/tensor.h b/src/core/include/tensor/tensor.h index c6f05f9..58b2a3f 100644 --- a/src/core/include/tensor/tensor.h +++ b/src/core/include/tensor/tensor.h @@ -12,6 +12,8 @@ enum class tensor_op_t { ADD, // Add a and b BROADCAST_ADD, // Performs broadcasting during addition RELU, // Activation function + SUB, // Subtract two tensor of same shape + MEAN, // Returns a scalar value mean of the tensor }; struct tensor_instance { diff --git a/src/core/tensor/tensor.cpp b/src/core/tensor/tensor.cpp index b8761de..2e5ad40 100644 --- a/src/core/tensor/tensor.cpp +++ b/src/core/tensor/tensor.cpp @@ -161,6 +161,16 @@ bool tensor_evaluate(tensor_pool_t *pool, tensor_t *t, [[maybe_unused]]float *d assert(t->a != NULL); success = tensor_op_relu(pool, t); break; + case tensor_op_t::SUB: + assert(t->a != NULL); + assert(t->b != NULL); + assert(t->a->dtype == t->b->dtype); + success = tensor_op_sub(pool, t); + break; + case tensor_op_t::MEAN: + assert(t->a != NULL); + success = tensor_op_mean(pool, t); + break; default: assert(false); } diff --git a/src/internal_header.h b/src/internal_header.h index eb2f722..638a335 100644 --- a/src/internal_header.h +++ b/src/internal_header.h @@ -7,6 +7,8 @@ #include "./core/include/tensor/tensor.h" #include "./core/include/pool/pool.h" #include "backend_cpu/include/add.h" +#include "backend_cpu/include/sub.h" +#include "backend_cpu/include/mean.h" #include "backend_cpu/include/debug.h" #include "backend_cpu/include/mul.h" #include "backend_cpu/include/scalar.h" From 66d4dd9e4c92642ba38d544011b2ccee6dda5155 Mon Sep 17 00:00:00 2001 From: Aakarsh Kashyap Date: Thu, 2 Apr 2026 15:50:57 +0530 Subject: [PATCH 04/27] chore/ Running summarizer --- scripts/coldstart/coldstart.prev | 40 -------------------------------- 1 file changed, 40 deletions(-) diff --git a/scripts/coldstart/coldstart.prev b/scripts/coldstart/coldstart.prev index 2abb2e0..e69de29 100644 --- a/scripts/coldstart/coldstart.prev +++ b/scripts/coldstart/coldstart.prev @@ -1,40 +0,0 @@ -Based on the recent logs and the state of the TODO list, here is the reconstruction of your mental context: - -### 1. What I Was Doing -I have just finished the basic implementation of **ReLU and Bias addition** (Commit `3b68101`). The project is currently transitioning from a simple tensor container to a functional compute graph. The "Cold start agent" is now functional, and I’ve been running a "summarizer" (likely an automated documentation or analysis tool) to track progress. My main goal now is **stabilizing the GPU backend assignment** and implementing core math kernels. - -### 2. Active TODOs (Grouped) - -**GPU & Backend Assignment (Critical)** -* **Memory Management:** Implement the logic for copying memory from parent nodes to child nodes in the GPU backend (`assignBackend.cu`). -* **Logic Refinement:** Build out the execution logic for the GPU part of the graph and resolve how to access execution nodes within the graph context. -* **Config Parser:** Implement the `CONFIG.soft` parser to allow dynamic backend assignment instead of hardcoding. -* **Stream Optimization:** Review the execution flow to potentially use CUDA streams for better performance. - -**Math Kernels (High Priority)** -* **Matrix Multiplication:** Currently a placeholder; needs a full implementation in `mul.cpp`. -* **Transpose Optimization:** Implement "Loop Tiling" in `transpose.cpp` to improve cache locality/performance on the CPU. - -**Technical Debt & Refactoring** -* **Multi-Dtype Support:** The system is currently "hardcoded" to `float32`. I need to refactor ReLU, Mul, and the GPU assignment logic to use `tensor_dtype_sizeof`. -* **Memory Layout:** Implement "contiguous logic" for tensors to ensure kernels don't break on sliced or non-contiguous data. - -### 3. Current State of the Code -* **Implemented:** Basic Tensor structure, ReLU, Bias addition, and the "Cold Start" agent. -* **Partially Done:** The GPU backend assignment (`assignBackend.cu`) exists but lacks the data-flow logic (parent-to-child memory copies). -* **Missing:** Matrix multiplication (the engine of the framework), multi-type support, and efficient GPU memory handling. - -### 4. My Likely Next Steps -1. **Complete the GPU Data Flow:** Finish the logic in `assignBackend.cu` that handles how children nodes fetch data from parents. This is the biggest blocker for running a full graph on GPU. -2. **Implement Matrix Multiplication:** The framework can't do much without `matmul`. I should implement a basic version first, then worry about optimization. -3. **Address the `float32` assumption:** Before the codebase gets too large, I need to start replacing `float` with the dtype abstraction I planned. -4. **Loop Tiling for Transpose:** A quick win for CPU performance that I've already noted down. - -### 5. Important Context I Should Not Forget -* **Dtype Assumption:** Almost everywhere in the math backends, we are assuming `float32_t`. If I start working on an op and it crashes with other types, this is why. -* **Backend Assignment:** The current logic for deciding if a node is CPU or GPU is still primitive. I was planning to use a "soft" configuration file to drive this. - -### 6. Risks / Tricky Areas -* **Graph Node Access:** In `assignBackend.cu`, there is uncertainty about how to correctly reference an execution node's place in the graph when we only have partial metadata. -* **Memory Contiguity:** If I implement kernels assuming contiguous memory without finishing the "contagious logic" TODO, I will run into silent data corruption or segfaults during slicing/reshaping. -* **GPU Streams:** If I move to streams for performance, I need to be careful about synchronization points so children don't start before parents finish. From 656da45c9a9f71d3867722d275c40e3335920b5b Mon Sep 17 00:00:00 2001 From: Aakarsh Kashyap Date: Thu, 9 Apr 2026 12:38:56 +0530 Subject: [PATCH 05/27] made assign grad memory function --- main.cpp | 74 +++++++++++++++++++-------- notes/Backward/MemoryModel.md | 53 +++++++++++++++++++ src/backend_cpu/backprop/matmul_b.cpp | 3 ++ src/core/graph/assignBackend.cu | 24 +++++++++ src/core/include/graph/DAGbuild.h | 3 ++ src/core/include/tensor/tensor.h | 6 ++- 6 files changed, 142 insertions(+), 21 deletions(-) create mode 100644 notes/Backward/MemoryModel.md create mode 100644 src/backend_cpu/backprop/matmul_b.cpp diff --git a/main.cpp b/main.cpp index 0835779..4ede92f 100644 --- a/main.cpp +++ b/main.cpp @@ -6,37 +6,71 @@ using namespace std; int main() { + // Create pools tensor_pool_t *pool = tensor_pool_create(1024 * 1024); + tensor_pool_t *pool_2 = tensor_pool_create(1024 * 1024); + tensor_pool_t *pool_gpu = tensor_pool_create(1024 * 1024, true); assert(pool != NULL); + assert(pool_gpu != NULL); cout << "=========================================== \n"; - cout << "=============TESTING SUBTRACTION============== \n"; - float val_a[] = {1.0f, 2.0f, 3.0f, 4.0f}; - float val_b[] = {1.0f, 2.0f, 3.0f, 4.0f}; + cout << "=============TESTING DAG VERIFICATION============== \n"; + + float val_a[900]{}; + float val_b[900]{}; + float val_c[900]{}; + float val_d[900]{}; + + // TACTICAL FIX: Make 'e' the exact same size to avoid a GPU segfault + // before we implement a dedicated GPU broadcasting kernel! + float val_e[900]{}; - uint32_t dims_a[] = {2, 2}; - uint32_t dims_b[] = {2, 2}; + uint32_t dims_a[] = {30, 30}; + uint32_t dims_b[] = {30, 30}; + uint32_t dims_c[] = {30, 30}; + uint32_t dims_d[] = {30, 30}; + uint32_t dims_e[] = {30, 30}; // Matched dimensions! tensor_t *a = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_a, val_a); tensor_t *b = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_b, val_b); - - tensor_t *c = tensor_mean(pool, a); + tensor_t *c = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_c, val_c); + tensor_t *d = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_d, val_d); + tensor_t *e = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_e, val_e); + tensor_fill_random_normal(a, 10.1, 5.7); + tensor_fill_random_normal(b, 10.1, 5.7); + tensor_fill_random_normal(c, 10.1, 5.7); + tensor_fill_random_normal(d, 10.1, 5.7); + tensor_fill_random_normal(e, 10.1, 5.7); + tensor_t *f = tensor_mul(pool, a, b); + tensor_t *g = tensor_mul(pool, b, c); + tensor_t *h = tensor_mul(pool, e, f); + tensor_t *i = tensor_mul(pool, h, a); + tensor_t *j = tensor_mul(pool, i, g); + cout << "=========================================== \n"; cout << "==============LAZY EVAL DONE=============== \n"; - bool ok = tensor_evaluate(pool, c); - assert(ok); - std::cout << "============================================= \n"; - std::cout << "===============ORIGINAL TENSOR=============== \n"; - tensor_print_data(a); - std::cout << "============================================= \n"; - std::cout << "===============SQUARED TENSOR=============== \n"; - // tensor_print_data(c); - float* data = (float*)tensor_get_data(c); - std::cout << data[0] << "\n"; + + std::vector seq; + bool oki = verifyIfDAG(pool_2, j, seq); + + // Ensure assignDevice is returning device_type::GPU under the hood! + assignBackendGraph(pool_gpu, seq); + tensor_graph_forward_evaluate(pool, pool_gpu, seq); + + if (oki) { + cout << "=========================================== \n"; + cout << "========= VRAM TRACE ============= \n"; + + execution_node_t *node = seq.back(); + + // Pull it back to the CPU + execution_node_to_host(node); + printExecutionNode(node); + cout << "\n"; + } - std::cout << "============================================= \n"; - // debug("result[0] = %f\n", result[0]); - // debug("result[1] = %f\n", result[1]); tensor_pool_destroy(pool); + tensor_pool_destroy(pool_2); + tensor_pool_destroy(pool_gpu); return 0; } diff --git a/notes/Backward/MemoryModel.md b/notes/Backward/MemoryModel.md new file mode 100644 index 0000000..86f3724 --- /dev/null +++ b/notes/Backward/MemoryModel.md @@ -0,0 +1,53 @@ +## Memory model for backward pass(Unstable) + +The problem we have is that when we do hybrid forward pass, it causes some data to be on GPU and some to be on CPU, this I thought +would be a problem but that is not the case. + +### Observation +> Since data flow is only one directional and due to our contagious logic + +``` + assignPlaceOnDeviceMemory(pool, (int32_t)(node->pos), nodes); + if (node->t->a == NULL) + goto trp; + if (node->t->a->device == device_type::CPU) { + int32_t a_idx = getTheExecutionNodeIndex(node, 0); + assert(a_idx != -1); + nodes[((size_t)a_idx)]->to_device_needed = true; + assignPlaceOnDeviceMemory(pool, a_idx, nodes); + } + trp: + if (node->t->b == NULL) + continue; + if (node->t->b->device == device_type::CPU) { + int32_t b_idx = getTheExecutionNodeIndex(node, 1); + assert(b_idx != -1); + nodes[((size_t)b_idx)]->to_device_needed = true; + assignPlaceOnDeviceMemory(pool, b_idx, nodes); + } + + +``` +We can conclude that all the data we need for grad computation is available on each device. and since we copy the parents we can +confidently say that we don't need to retransfer tensors inbetween eliminating the thrashing. + +* So here comes the part where what we will do in +`void assignBackendGraph(tensor_pool_t *pool,std::vector &nodes);` +is we will allocate the grad space for tensors on GPU as `pool_gpu_grad` and on CPU as `pool_cpu_grad` then we assign it to the home tensor(refering to the current tensor) +then during backward pass data is assigned to each pool and is written as it is. +since we don't want transfer overhead during computation path we will construct another data transfering fucnction which will get grad data from gpu to host. +and then we do optimization and then free resources. + +Also the device_ptr_gpu is stored in execution_node_t; + +=================================================================================================== + +actually i think we can't do grad placement in the AssignBackendGraph function which we are implementing the contagious logic we will need a 4th pass i think where so we can do this maneuvour + +P1->GPU, P2->GPU, C1->GPU + +but now if P1 and P2 was bought to GPU and it's copy exist on cpu we can jump to it's DRAM storage and i think since it's address we will know due to well how we can detect it you may ask it would be if it have address on execution node and also on tensor instance right then we switch to CPU allocation right. + +Now for this to work we will need to walk reverse in graph so one more reason for 4th seperate pass plus this also helps that we will not be touching much of things. + +Now if we are already doing 4th pass it would be better to instead have a seperate explict function for this so func params also don't get bloated and confusing. diff --git a/src/backend_cpu/backprop/matmul_b.cpp b/src/backend_cpu/backprop/matmul_b.cpp new file mode 100644 index 0000000..9a133d8 --- /dev/null +++ b/src/backend_cpu/backprop/matmul_b.cpp @@ -0,0 +1,3 @@ +#include "internal_header.h" + + diff --git a/src/core/graph/assignBackend.cu b/src/core/graph/assignBackend.cu index 007c09c..ea0a7ff 100644 --- a/src/core/graph/assignBackend.cu +++ b/src/core/graph/assignBackend.cu @@ -208,3 +208,27 @@ bool execution_node_to_host(execution_node_t *node) { } return true; } + +// TODO: Will need to edit the tensor create func signature so to allow the user to assign grad_compute boolean. +// What we can do is keep it default and then have user declare leaf to NULL. As otherwise it would be difficult to implement the +// propogation logic + +void assignGradMemory(tensor_pool_t *pool_grad_cpu, tensor_pool_t *pool_grad_gpu, std::vector &nodes) { + for(auto &node : nodes) { + // We are not assigning grad instance during tensor create. + // Maybe we can do that but I feel it might cause recursion problem even if with assigning base case of leaf nodes + // there is the problem that during creation time it could cause function call overhead and whatnot. + // So we are going to make a tensor instance here and assign it here i guess. + + if(node->t->grad_compute == false) continue; + + node->t->grad = tensor_dtype_create(pool_grad_cpu, node->t->dtype, node->t->ndims, node->t->dims, NULL); + + if(node->t->device == device_type::GPU) { + size_t size = node->t->nvalues * sizeof(float) ; + uint32_t id; + node->device_ptr_grad = tensor_pool_alloc(pool_grad_gpu, size, &id); + node->t->grad = tensor_dtype_create(pool_grad_cpu, node->t->dtype, node->t->ndims, node->t->dims, NULL); + } + } +} diff --git a/src/core/include/graph/DAGbuild.h b/src/core/include/graph/DAGbuild.h index 707d324..180a1c6 100644 --- a/src/core/include/graph/DAGbuild.h +++ b/src/core/include/graph/DAGbuild.h @@ -14,6 +14,9 @@ struct execution_node { // Pointer to device VRAM alloc, NULL if not needed void *device_ptr; + // Pointer to device VRAM alloc, will be the gradient pointer storing the result of backward pass for data i.e on GPU + void *device_ptr_grad; + // Boolean flag storing weather it will need to be transfered based upon reading the childs OPS bool to_device_needed; diff --git a/src/core/include/tensor/tensor.h b/src/core/include/tensor/tensor.h index 58b2a3f..6e23fed 100644 --- a/src/core/include/tensor/tensor.h +++ b/src/core/include/tensor/tensor.h @@ -49,7 +49,11 @@ struct tensor_instance { // If tensor is transposed bool is_transposed; - + + // Writing it for common interface and ease of access + // grad is a data-only tensor. Fields op, a, b, + // stateTracker, grad_compute, grad are always NULL/zero. + // Only data, nvalues, ndims, dims, dtype are valid. // For storing autograd result tensor_t *grad; From 0348243d167a28e6f4e37de69691107c470764f7 Mon Sep 17 00:00:00 2001 From: Aakarsh Kashyap Date: Mon, 13 Apr 2026 11:56:58 +0530 Subject: [PATCH 06/27] chore/ Running summarizer --- include/soft-cuda/tensor/api.h | 3 +- main.cpp | 22 ++++++++++----- scripts/coldstart/coldstart.prev | 38 ++++++++++++++++++++++++++ src/backend_cpu/backprop/matmul_b.cpp | 2 -- src/core/graph/assignBackend.cu | 4 +-- src/core/include/graph/assignBackend.h | 5 ++++ src/core/include/tensor/tensor.h | 4 +-- src/core/tensor/tensor.cpp | 8 +++--- 8 files changed, 68 insertions(+), 18 deletions(-) diff --git a/include/soft-cuda/tensor/api.h b/include/soft-cuda/tensor/api.h index eaf4488..3ff22e1 100644 --- a/include/soft-cuda/tensor/api.h +++ b/include/soft-cuda/tensor/api.h @@ -84,7 +84,7 @@ uint32_t tensor_id(tensor_t *t); */ // DONE tensor_t *tensor_create(tensor_pool_t *pool, tensor_dtype_t dtype, uint32_t num_dims, - uint32_t *dims, void *elems); + uint32_t *dims, void *elems, bool grad_status = true); /* * Establish a new memory arena @@ -334,6 +334,7 @@ bool verifyIfDAG(tensor_pool_t *pool, tensor_t *t, std::vector &nodes); +void assignGradMemory(tensor_pool_t *pool_grad_cpu, tensor_pool_t *pool_grad_gpu, std::vector &nodes); /* @params Take execution_node_t which you wanna know * @returns the postion of the execution_node_t after verifyIfDAG * */ diff --git a/main.cpp b/main.cpp index 4ede92f..2a5605a 100644 --- a/main.cpp +++ b/main.cpp @@ -1,15 +1,19 @@ #include "soft-cuda/tensor/api.h" -#include "soft-cuda/tensor/debug_api.h" +// #include "soft-cuda/tensor/debug_api.h" #include #include using namespace std; - int main() { // Create pools + // CPU ALLOCATION tensor_pool_t *pool = tensor_pool_create(1024 * 1024); tensor_pool_t *pool_2 = tensor_pool_create(1024 * 1024); + tensor_pool_t *pool_grad_cpu = tensor_pool_create(1024 * 1024); + + // GPU ALLOCATION tensor_pool_t *pool_gpu = tensor_pool_create(1024 * 1024, true); + tensor_pool_t *pool_grad_gpu = tensor_pool_create(1024 * 1024, true); assert(pool != NULL); assert(pool_gpu != NULL); @@ -31,11 +35,11 @@ int main() { uint32_t dims_d[] = {30, 30}; uint32_t dims_e[] = {30, 30}; // Matched dimensions! - tensor_t *a = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_a, val_a); - tensor_t *b = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_b, val_b); - tensor_t *c = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_c, val_c); - tensor_t *d = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_d, val_d); - tensor_t *e = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_e, val_e); + tensor_t *a = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_a, val_a, false); + tensor_t *b = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_b, val_b, false); + tensor_t *c = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_c, val_c, false); + tensor_t *d = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_d, val_d, false); + tensor_t *e = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_e, val_e, false); tensor_fill_random_normal(a, 10.1, 5.7); tensor_fill_random_normal(b, 10.1, 5.7); tensor_fill_random_normal(c, 10.1, 5.7); @@ -55,6 +59,8 @@ int main() { // Ensure assignDevice is returning device_type::GPU under the hood! assignBackendGraph(pool_gpu, seq); + + assignGradMemory(pool_grad_cpu,pool_grad_gpu, seq); tensor_graph_forward_evaluate(pool, pool_gpu, seq); if (oki) { @@ -71,6 +77,8 @@ int main() { tensor_pool_destroy(pool); tensor_pool_destroy(pool_2); + tensor_pool_destroy(pool_grad_cpu); + tensor_pool_destroy(pool_grad_gpu); tensor_pool_destroy(pool_gpu); return 0; } diff --git a/scripts/coldstart/coldstart.prev b/scripts/coldstart/coldstart.prev index e69de29..a9308cf 100644 --- a/scripts/coldstart/coldstart.prev +++ b/scripts/coldstart/coldstart.prev @@ -0,0 +1,38 @@ +Based on the recent commits and the transition from basic tensor operations to loss functions and backend management, here is your reconstructed context: + +### 1. What I Was Doing +I was expanding the core library's mathematical capabilities and infrastructure. Specifically, I just finished implementing the **Mean Squared Error (MSE)** loss function and verified it with basic tests. Parallel to this, I began abstraction work to allow the library to **switch between different backends** (likely preparing for optimized BLAS or GPU support) and added a **naive SGEMM** (Single-precision General Matrix Multiply) as a baseline. + +### 2. Active TODOs (Grouped) +**Loss Functions & Backprop** +* [ ] Implement the **Backward Pass** for MSE (Gradient calculation). +* [ ] Add **CrossEntropyLoss** (the logical next step after MSE for classification tasks). + +**Backend & Optimization** +* [ ] Optimize SGEMM: Replace the naive triple-loop with tiled or SIMD-accelerated versions. +* [ ] Validate Backend Switching: Ensure the `backend switch logic` correctly routes operations without data corruption. + +**Infrastructure** +* [ ] Refine `make.sh`: Standardize the `time` field reporting for benchmarking different backends. +* [ ] Regression testing: Ensure the ReLU data bug fixes (commits `#30`/`#31`) stay fixed during backend transitions. + +### 3. Current State of the Code +* **Implemented:** Naive SGEMM, Backend switching logic, ReLU (bug-fixed), MSE Loss (Forward pass/Smoke tests passed). +* **Partially Done:** Infrastructure for performance timing (added to `make.sh`). +* **Missing:** Optimized kernels (currently relying on naive loops), actual multi-backend support (e.g., CUDA or OpenBLAS), and automated benchmarking suite. + +### 4. My Likely Next Steps +1. **Verify the Backend Switch:** Write a test case that swaps backends mid-execution to ensure tensor data remains consistent. +2. **MSE Gradient:** Implement the derivative for MSE so the library can actually be used for training. +3. **Benchmarking:** Use the newly updated `make.sh` to get a baseline timing for the naive SGEMM vs. basic operations. +4. **Optimized GEMM:** Start a "fast" backend path using a more efficient matrix multiplication algorithm. + +### 5. Important Context I Should Not Forget +* **Timing Logic:** The `make.sh` changes suggest a shift in focus toward performance monitoring. Don't forget to check if the `time` command is portable across the dev environments. +* **ReLU Bug:** There were two consecutive commits for a "ReLU data bug." This suggests an edge case (likely related to memory alignment or in-place modifications) that might reappear in other activation functions. +* **Backend Abstraction:** The "switch logic" implies that tensors now need to be aware of their "location" or "engine" before operations are called. + +### 6. Risks / Tricky Areas +* **Backend Divergence:** If the naive SGEMM and a future optimized version yield slightly different precision results, the smoke tests might fail on different backends. +* **Pointer Handling in Switching:** Ensure that switching backends doesn't lead to memory leaks or double-frees if pointers are being re-allocated or cast during the switch. +* **The ReLU Fix:** Since it took two tries to fix (commits `00c6494` to `28377df`), the logic there might be fragile. Check this area first if unexpected data corruption occurs in deeper networks. diff --git a/src/backend_cpu/backprop/matmul_b.cpp b/src/backend_cpu/backprop/matmul_b.cpp index 9a133d8..518ad62 100644 --- a/src/backend_cpu/backprop/matmul_b.cpp +++ b/src/backend_cpu/backprop/matmul_b.cpp @@ -1,3 +1 @@ #include "internal_header.h" - - diff --git a/src/core/graph/assignBackend.cu b/src/core/graph/assignBackend.cu index ea0a7ff..6af9e70 100644 --- a/src/core/graph/assignBackend.cu +++ b/src/core/graph/assignBackend.cu @@ -185,6 +185,7 @@ void printExecutionNode(execution_node_t *et) { tensor_print_data(et->t); std::cout << et->to_device_needed << "\n"; std::cout << et->device_ptr << "\n"; + std::cout << et->device_ptr_grad << "\n"; std::cout << (et->backend_fn != NULL) << "\n"; } @@ -224,11 +225,10 @@ void assignGradMemory(tensor_pool_t *pool_grad_cpu, tensor_pool_t *pool_grad_gpu node->t->grad = tensor_dtype_create(pool_grad_cpu, node->t->dtype, node->t->ndims, node->t->dims, NULL); - if(node->t->device == device_type::GPU) { + if(node->backend_fn == tensor_evaluate_GPU) { size_t size = node->t->nvalues * sizeof(float) ; uint32_t id; node->device_ptr_grad = tensor_pool_alloc(pool_grad_gpu, size, &id); - node->t->grad = tensor_dtype_create(pool_grad_cpu, node->t->dtype, node->t->ndims, node->t->dims, NULL); } } } diff --git a/src/core/include/graph/assignBackend.h b/src/core/include/graph/assignBackend.h index ee34501..91f8d3f 100644 --- a/src/core/include/graph/assignBackend.h +++ b/src/core/include/graph/assignBackend.h @@ -1,7 +1,9 @@ #pragma once #include #include "../third_party/json.hpp" + using json = nlohmann::json; + void assignBackend(execution_node_t *e); device_type assignDevice(uint8_t ndims, uint32_t *dims, tensor_op_t op, uint32_t nvalues, json &data); @@ -9,3 +11,6 @@ device_type assignDevice(uint8_t ndims, uint32_t *dims, tensor_op_t op, uint32_t int32_t getTheExecutionNodeIndex(uint32_t id, std::vector &nodes); void assignBackendGraph(tensor_pool_t *pool, std::vector &nodes); + + +void assignGradMemory(tensor_pool_t *pool_grad_cpu, tensor_pool_t *pool_grad_gpu, std::vector &nodes); diff --git a/src/core/include/tensor/tensor.h b/src/core/include/tensor/tensor.h index 6e23fed..9e89b75 100644 --- a/src/core/include/tensor/tensor.h +++ b/src/core/include/tensor/tensor.h @@ -65,7 +65,7 @@ struct tensor_instance { // a scalar is created. If elems is NULL then the tensor is created with zeros. // If elems is not NULL then it is assigned as initial values. tensor_t *tensor_dtype_create(tensor_pool_t *pool, tensor_dtype_t dtype, uint32_t num_dims, - uint32_t *dims, void *elems); + uint32_t *dims, void *elems, bool grad_status = true); // Evaluate the tensor, return true on success bool tensor_evaluate( tensor_pool_t *pool,tensor_t *t, float *d_a, float *d_b, float *d_res); @@ -74,5 +74,5 @@ bool tensor_evaluate( tensor_pool_t *pool,tensor_t *t, float *d_a, float *d_b, size_t tensor_dtype_sizeof(tensor_dtype_t dtype); tensor_t *tensor_create(tensor_pool_t *pool, tensor_dtype_t dtype, uint32_t num_dims, - uint32_t *dims, void *elems); + uint32_t *dims, void *elems, bool grad_status); #endif // !PRIVATE_TENSOR_H diff --git a/src/core/tensor/tensor.cpp b/src/core/tensor/tensor.cpp index 2e5ad40..efa2a47 100644 --- a/src/core/tensor/tensor.cpp +++ b/src/core/tensor/tensor.cpp @@ -33,7 +33,7 @@ size_t tensor_dtype_sizeof(tensor_dtype_t dtype) { } tensor_t *tensor_dtype_create(tensor_pool_t *pool, tensor_dtype_t dtype, uint32_t ndims, - uint32_t *dims, void *elems) { + uint32_t *dims, void *elems, bool grad_status) { assert(pool != NULL); assert(tensor_dtype_sizeof(dtype) > 0); @@ -99,7 +99,7 @@ tensor_t *tensor_dtype_create(tensor_pool_t *pool, tensor_dtype_t dtype, uint32_ t->b = NULL; t->stateTracker = 0; t->device = device_type::CPU; - t->grad_compute = false; + t->grad_compute = grad_status; t->is_transposed = false; // Return success return t; @@ -107,8 +107,8 @@ tensor_t *tensor_dtype_create(tensor_pool_t *pool, tensor_dtype_t dtype, uint32_ // Create float32 tensor tensor_t *tensor_create(tensor_pool_t *pool, tensor_dtype_t dtype, uint32_t num_dims, - uint32_t *dims, void *elems) { - return tensor_dtype_create(pool, dtype, num_dims, dims, elems); + uint32_t *dims, void *elems, bool grad_status) { + return tensor_dtype_create(pool, dtype, num_dims, dims, elems, grad_status); } bool tensor_evaluate(tensor_pool_t *pool, tensor_t *t, [[maybe_unused]]float *d_a, [[maybe_unused]]float *d_b, [[maybe_unused]]float *d_res) { From 215c541176170694e8515a466527c8d52aa3c464 Mon Sep 17 00:00:00 2001 From: Aakarsh Kashyap Date: Mon, 13 Apr 2026 11:57:20 +0530 Subject: [PATCH 07/27] chore/ Running summarizer --- scripts/coldstart/coldstart.prev | 38 -------------------------------- 1 file changed, 38 deletions(-) diff --git a/scripts/coldstart/coldstart.prev b/scripts/coldstart/coldstart.prev index a9308cf..e69de29 100644 --- a/scripts/coldstart/coldstart.prev +++ b/scripts/coldstart/coldstart.prev @@ -1,38 +0,0 @@ -Based on the recent commits and the transition from basic tensor operations to loss functions and backend management, here is your reconstructed context: - -### 1. What I Was Doing -I was expanding the core library's mathematical capabilities and infrastructure. Specifically, I just finished implementing the **Mean Squared Error (MSE)** loss function and verified it with basic tests. Parallel to this, I began abstraction work to allow the library to **switch between different backends** (likely preparing for optimized BLAS or GPU support) and added a **naive SGEMM** (Single-precision General Matrix Multiply) as a baseline. - -### 2. Active TODOs (Grouped) -**Loss Functions & Backprop** -* [ ] Implement the **Backward Pass** for MSE (Gradient calculation). -* [ ] Add **CrossEntropyLoss** (the logical next step after MSE for classification tasks). - -**Backend & Optimization** -* [ ] Optimize SGEMM: Replace the naive triple-loop with tiled or SIMD-accelerated versions. -* [ ] Validate Backend Switching: Ensure the `backend switch logic` correctly routes operations without data corruption. - -**Infrastructure** -* [ ] Refine `make.sh`: Standardize the `time` field reporting for benchmarking different backends. -* [ ] Regression testing: Ensure the ReLU data bug fixes (commits `#30`/`#31`) stay fixed during backend transitions. - -### 3. Current State of the Code -* **Implemented:** Naive SGEMM, Backend switching logic, ReLU (bug-fixed), MSE Loss (Forward pass/Smoke tests passed). -* **Partially Done:** Infrastructure for performance timing (added to `make.sh`). -* **Missing:** Optimized kernels (currently relying on naive loops), actual multi-backend support (e.g., CUDA or OpenBLAS), and automated benchmarking suite. - -### 4. My Likely Next Steps -1. **Verify the Backend Switch:** Write a test case that swaps backends mid-execution to ensure tensor data remains consistent. -2. **MSE Gradient:** Implement the derivative for MSE so the library can actually be used for training. -3. **Benchmarking:** Use the newly updated `make.sh` to get a baseline timing for the naive SGEMM vs. basic operations. -4. **Optimized GEMM:** Start a "fast" backend path using a more efficient matrix multiplication algorithm. - -### 5. Important Context I Should Not Forget -* **Timing Logic:** The `make.sh` changes suggest a shift in focus toward performance monitoring. Don't forget to check if the `time` command is portable across the dev environments. -* **ReLU Bug:** There were two consecutive commits for a "ReLU data bug." This suggests an edge case (likely related to memory alignment or in-place modifications) that might reappear in other activation functions. -* **Backend Abstraction:** The "switch logic" implies that tensors now need to be aware of their "location" or "engine" before operations are called. - -### 6. Risks / Tricky Areas -* **Backend Divergence:** If the naive SGEMM and a future optimized version yield slightly different precision results, the smoke tests might fail on different backends. -* **Pointer Handling in Switching:** Ensure that switching backends doesn't lead to memory leaks or double-frees if pointers are being re-allocated or cast during the switch. -* **The ReLU Fix:** Since it took two tries to fix (commits `00c6494` to `28377df`), the logic there might be fragile. Check this area first if unexpected data corruption occurs in deeper networks. From f3d569c5db89fa44c37c8546965228c8b9dd2b4d Mon Sep 17 00:00:00 2001 From: Aakarsh Kashyap Date: Mon, 13 Apr 2026 12:01:11 +0530 Subject: [PATCH 08/27] chore/ Running summarizer --- scripts/coldstart/coldstart.prev | 36 ++++++++++++++++++++++++++++++++ scripts/coldstart/coldstart.sh | 2 +- 2 files changed, 37 insertions(+), 1 deletion(-) diff --git a/scripts/coldstart/coldstart.prev b/scripts/coldstart/coldstart.prev index e69de29..e48b1cd 100644 --- a/scripts/coldstart/coldstart.prev +++ b/scripts/coldstart/coldstart.prev @@ -0,0 +1,36 @@ +Based on the git log and the recent activity, here is the reconstruction of your mental context. + +### 1. What I Was Doing +I have been building out the core components of a tensor/autograd library. I just successfully finished the **Mean Squared Error (MSE)** loss function and verified it with smoke tests. Immediately after that, I shifted focus to **gradient memory management**, specifically writing a function to handle how gradient memory is assigned/allocated to tensors. + +### 2. Active TODOs (Grouped) +* **Autograd Engine/Memory:** + * Integrate `assign_grad_memory` into the tensor initialization or the backward pass flow. + * Ensure gradients are properly zeroed out before the next backward pass (standard `zero_grad` functionality). +* **Loss Functions:** + * (Optional) Implement Cross-Entropy or other standard loss functions now that the MSE template is proven. +* **Tooling/DevOps:** + * The repetitive "Running summarizer" commits suggest I am fine-tuning an automated workflow or documentation script. + +### 3. Current State of the Code +* **Implemented:** + * MSE Loss function (logic is solid and passed basic verification). + * `make.sh` utility now includes a time field for performance tracking. + * A function to assign gradient memory (`assign_grad_memory`). +* **Partially Done:** + * The plumbing for backpropagation. While the loss exists, the full "Backward" chain for a model likely needs the newly created gradient memory function to be fully integrated. +* **Missing:** + * A comprehensive test for a full training step (Forward -> MSE -> Backward -> Optimizer Step). + +### 4. My Likely Next Steps +1. **Verify Memory Assignment:** Write a small test script to ensure `assign_grad_memory` actually allocates the correct shapes and doesn't leak memory. +2. **Connect MSE to Backprop:** Implement the `.backward()` method for the MSE node, ensuring it utilizes the new gradient memory allocation. +3. **End-to-End Smoke Test:** Run a simple linear regression or a single-neuron test to see if the gradient flows from the MSE loss back to the input weights correctly. + +### 5. Important Context I Should Not Forget +* **Performance Tracking:** I added a `time` field to `make.sh`, which implies I'm starting to care about how long the build or the tests take. I should keep an eye on this as memory management becomes more complex. +* **Manual Memory Management:** Since I'm writing `assign_grad_memory` manually, I need to be careful about whether gradients are being allocated lazily or eagerly at tensor creation. + +### 6. Risks / Tricky Areas +* **Gradient Accumulation:** If `assign_grad_memory` is called multiple times or incorrectly, I might overwrite gradients instead of accumulating them, or double-allocate memory. +* **Smoke Test Scope:** Passing "smoke tests" for MSE proves the math is right, but it doesn't prove the engine can handle a complex graph yet. Be wary of moving to complex architectures too fast. diff --git a/scripts/coldstart/coldstart.sh b/scripts/coldstart/coldstart.sh index 78a0e9c..dc394d7 100755 --- a/scripts/coldstart/coldstart.sh +++ b/scripts/coldstart/coldstart.sh @@ -11,7 +11,7 @@ filename="agent_summary_${date}.log" echo "=== GIT DIFF ===" git diff HEAD echo "=== TODO ===" - grep -r TODO ../src/ 2>/dev/null + grep -r TODO ../../src 2>/dev/null echo "=== PREVIOUS SESSION SUMMARY ===" [ -f coldstart.prev ] && cat coldstart.prev } | python coldstart_agent.py > logs/${filename} From 5533ad3d840c8eb7fba3a84747ac731527f23b8f Mon Sep 17 00:00:00 2001 From: Aakarsh Kashyap Date: Tue, 14 Apr 2026 16:11:39 +0530 Subject: [PATCH 09/27] Somewhat work not tested --- src/backend_cpu/backprop/matmul_b.cpp | 138 ++++++++++++++++++++++++++ src/core/graph/assignBackend.cu | 15 +-- 2 files changed, 147 insertions(+), 6 deletions(-) diff --git a/src/backend_cpu/backprop/matmul_b.cpp b/src/backend_cpu/backprop/matmul_b.cpp index 518ad62..3a926d9 100644 --- a/src/backend_cpu/backprop/matmul_b.cpp +++ b/src/backend_cpu/backprop/matmul_b.cpp @@ -1 +1,139 @@ #include "internal_header.h" + +// We are gonna write a backprop fucntion for matmul cause that is easy thing to do +// Since our backprop is an eagar evaluation we will make tensor objects inside the function +// transpose it execute it as well and then work with it + +// Now I am slightly confused but I think it would have gone something like this +// Since we are going to get execution node we could have a decision made here if it's data will be +// written on device_ptr of node_.grad->data +// We are gonna do this cause making decsion on higher level will require us to make two kernels for similar work. + +// Since this will be eagar evalution we are gonna execute it here now and we are gonna return a +// boolean. +// @return: boolean +// @params: execution_node_t +// Since all the memory is already executed we just need the execution node +bool backprop__(std::vector &nodes) { + + for(auto &node : nodes) { + if (node->t->grad_compute) { + if (node->device_ptr_grad != NULL) { + bool success = backprop_gpu(node); + return success; + } else { + bool success = backprop_cpu(node); + return sucess; + } + } + } +} + +bool backprop_cpu(execution_node_t *node) { + assert(node->t != NULL); + bool success = false; + + switch (t->op) { + case tensor_grad_op_t::NONE: + success = true; + break; + case tensor_grad_op_t::CAST: + break; + // TODO: Implement here + case tensor_grad_op_t::MUL_MATRIX: + assert(t->a != NULL); + assert(t->b != NULL); + assert(t->a->dtype == t->b->dtype); + success = tensor_mul_op_matrix(pool, t); + break; + case tensor_grad_op_t::MUL_SCALAR: + assert(t->a != NULL); + assert(t->b != NULL); + assert(t->a->dtype == t->b->dtype); + success = tensor_mul_op_scalar(pool, t); + break; + case tensor_grad_op_t::TRANSPOSE: + assert(t->a != NULL); + success = tensor_tranpose_op_matrix(pool, t); + break; + case tensor_grad_op_t::NAIVE_MATRIX_MUL: + assert(t->a != NULL); + assert(t->b != NULL); + assert(t->a->dtype == t->b->dtype); + success = tensor_mul_op_matrix_naive(pool, t); + break; + case tensor_grad_op_t::ADD: + assert(t->a != NULL); + assert(t->b != NULL); + assert(t->a->dtype == t->b->dtype); + success = tensor_op_add(pool, t); + break; + case tensor_grad_op_t::BROADCAST_ADD: + assert(t->a != NULL); + assert(t->b != NULL); + assert(t->a->dtype == t->b->dtype); + success = tensor_op_broadcasting_add(pool, t); + break; + case tensor_grad_op_t::RELU: + assert(t->a != NULL); + success = tensor_op_relu(pool, t); + break; + case tensor_grad_op_t::SUB: + assert(t->a != NULL); + assert(t->b != NULL); + assert(t->a->dtype == t->b->dtype); + success = tensor_op_sub(pool, t); + break; + case tensor_grad_op_t::MEAN: + assert(t->a != NULL); + success = tensor_op_mean(pool, t); + break; + default: + assert(false); + } + if (success) { + debug("tensor_evaluate: success\n"); + } else { + debug("tensor_evaluate: FUBAR\n"); + } + return success; +} + +/*************************************************************/ +/*************************************************************/ +/*************************************************************/ +// TODO: Implement GPU module +bool backprop_gpu(execution_node_t *node) { + assert(pool != NULL); + assert(t != NULL); + bool success = false; + + switch (t->op) { + case tensor_op_t::NONE: + success = true; + break; + case tensor_op_t::CAST: + break; + // TODO: Implement here + case tensor_op_t::ADD: + assert(t->a != NULL); + assert(t->b != NULL); + assert(t->a->dtype == t->b->dtype); + success = tensor_add_op_cuda(t, d_a, d_b, d_res); + break; + case tensor_op_t::MUL_MATRIX: + assert(t->a != NULL); + assert(t->b != NULL); + assert(t->a->dtype == t->b->dtype); + success = tensor_mul_op_cuda(t, d_a, d_b, d_res); + break; + default: + assert(false); + } + if (success) { + debug("tensor_evaluate_GPU: success\n"); + } else { + debug("tensor_evaluate_GPU: FUBAR\n"); + } + return success; +} diff --git a/src/core/graph/assignBackend.cu b/src/core/graph/assignBackend.cu index 6af9e70..d472a33 100644 --- a/src/core/graph/assignBackend.cu +++ b/src/core/graph/assignBackend.cu @@ -1,5 +1,5 @@ #include "internal_header.h" -#include "vector" +// #include "vector" #include #include @@ -210,10 +210,13 @@ bool execution_node_to_host(execution_node_t *node) { return true; } -// TODO: Will need to edit the tensor create func signature so to allow the user to assign grad_compute boolean. -// What we can do is keep it default and then have user declare leaf to NULL. As otherwise it would be difficult to implement the -// propogation logic - +// What we can do is keep it default and then have user declare leaf to NULL. As otherwise it would +// be difficult to implement the propogation logic +// NOTE: Since we are making the memory here and device_ptr_grad is here to what we will do is take +// the whole execution node and then pass it to the eagar OPS and then we will see if it does have +// `device_ptr_grad` if so then it's fine right other wise we will directly write to +// node->t->grad->data right. Then after all the ops we will make a new function which will scan the +// whole graph and then if it sees the device_ptr_grad is not null it will initiate data transfer. void assignGradMemory(tensor_pool_t *pool_grad_cpu, tensor_pool_t *pool_grad_gpu, std::vector &nodes) { for(auto &node : nodes) { // We are not assigning grad instance during tensor create. @@ -224,7 +227,7 @@ void assignGradMemory(tensor_pool_t *pool_grad_cpu, tensor_pool_t *pool_grad_gpu if(node->t->grad_compute == false) continue; node->t->grad = tensor_dtype_create(pool_grad_cpu, node->t->dtype, node->t->ndims, node->t->dims, NULL); - + if(node->backend_fn == tensor_evaluate_GPU) { size_t size = node->t->nvalues * sizeof(float) ; uint32_t id; From db3009d7c6283f5b873bbb252f033a67496200e2 Mon Sep 17 00:00:00 2001 From: Aakarsh Kashyap Date: Wed, 15 Apr 2026 01:52:05 +0530 Subject: [PATCH 10/27] Implemented brackprop_b.cpp and added switch statements for op handling --- .../backprop/{matmul_b.cpp => backprop_b.cpp} | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) rename src/backend_cpu/backprop/{matmul_b.cpp => backprop_b.cpp} (94%) diff --git a/src/backend_cpu/backprop/matmul_b.cpp b/src/backend_cpu/backprop/backprop_b.cpp similarity index 94% rename from src/backend_cpu/backprop/matmul_b.cpp rename to src/backend_cpu/backprop/backprop_b.cpp index 3a926d9..a888e6f 100644 --- a/src/backend_cpu/backprop/matmul_b.cpp +++ b/src/backend_cpu/backprop/backprop_b.cpp @@ -28,7 +28,7 @@ bool backprop__(std::vector &nodes) { } } } - +// TODO: To implement each and every kernel for each op backprop bool backprop_cpu(execution_node_t *node) { assert(node->t != NULL); bool success = false; @@ -102,26 +102,26 @@ bool backprop_cpu(execution_node_t *node) { /*************************************************************/ /*************************************************************/ /*************************************************************/ -// TODO: Implement GPU module +// TODO: Implement GPU GRADIENT module bool backprop_gpu(execution_node_t *node) { assert(pool != NULL); assert(t != NULL); bool success = false; switch (t->op) { - case tensor_op_t::NONE: + case tensor_grad_t::NONE: success = true; break; - case tensor_op_t::CAST: + case tensor_grad_op_t::CAST: break; // TODO: Implement here - case tensor_op_t::ADD: + case tensor_grad_op_t::ADD: assert(t->a != NULL); assert(t->b != NULL); assert(t->a->dtype == t->b->dtype); success = tensor_add_op_cuda(t, d_a, d_b, d_res); break; - case tensor_op_t::MUL_MATRIX: + case tensor_grad_op_t::MUL_MATRIX: assert(t->a != NULL); assert(t->b != NULL); assert(t->a->dtype == t->b->dtype); From ed64d53f8e5623a6c317ab0ec18dacea445ee830 Mon Sep 17 00:00:00 2001 From: Aakarsh Kashyap Date: Wed, 15 Apr 2026 11:38:25 +0530 Subject: [PATCH 11/27] Corrected the enum missnaming mistake --- src/backend_cpu/backprop/backprop_b.cpp | 70 ++++++++++++------------- src/backend_cpu/backprop/matmul_b.cpp | 1 + 2 files changed, 36 insertions(+), 35 deletions(-) create mode 100644 src/backend_cpu/backprop/matmul_b.cpp diff --git a/src/backend_cpu/backprop/backprop_b.cpp b/src/backend_cpu/backprop/backprop_b.cpp index a888e6f..e1476c2 100644 --- a/src/backend_cpu/backprop/backprop_b.cpp +++ b/src/backend_cpu/backprop/backprop_b.cpp @@ -28,73 +28,74 @@ bool backprop__(std::vector &nodes) { } } } -// TODO: To implement each and every kernel for each op backprop + bool backprop_cpu(execution_node_t *node) { + assert(node->t != NULL); bool success = false; switch (t->op) { - case tensor_grad_op_t::NONE: + case tensor_op_t::NONE: success = true; break; - case tensor_grad_op_t::CAST: + case tensor_op_t::CAST: break; // TODO: Implement here - case tensor_grad_op_t::MUL_MATRIX: + case tensor_op_t::MUL_MATRIX: assert(t->a != NULL); assert(t->b != NULL); assert(t->a->dtype == t->b->dtype); - success = tensor_mul_op_matrix(pool, t); + success = tensor_mul_grad_op_matrix(pool, t); break; - case tensor_grad_op_t::MUL_SCALAR: + case tensor_op_t::MUL_SCALAR: assert(t->a != NULL); assert(t->b != NULL); assert(t->a->dtype == t->b->dtype); - success = tensor_mul_op_scalar(pool, t); + success = tensor_mul_grad_op_scalar(pool, t); break; - case tensor_grad_op_t::TRANSPOSE: + case tensor_op_t::TRANSPOSE: assert(t->a != NULL); - success = tensor_tranpose_op_matrix(pool, t); + success = tensor_tranpose_grad_op_matrix(pool, t); break; - case tensor_grad_op_t::NAIVE_MATRIX_MUL: + case tensor_op_t::NAIVE_MATRIX_MUL: assert(t->a != NULL); assert(t->b != NULL); assert(t->a->dtype == t->b->dtype); - success = tensor_mul_op_matrix_naive(pool, t); + success = tensor_mul_grad_op_matrix_naive(pool, t); break; - case tensor_grad_op_t::ADD: + case tensor_op_t::ADD: assert(t->a != NULL); assert(t->b != NULL); assert(t->a->dtype == t->b->dtype); - success = tensor_op_add(pool, t); + success = tensor_grad_op_add(pool, t); break; - case tensor_grad_op_t::BROADCAST_ADD: + case tensor_op_t::BROADCAST_ADD: assert(t->a != NULL); assert(t->b != NULL); assert(t->a->dtype == t->b->dtype); - success = tensor_op_broadcasting_add(pool, t); + success = tensor_grad_op_broadcasting_add(pool, t); break; - case tensor_grad_op_t::RELU: + case tensor_op_t::RELU: assert(t->a != NULL); - success = tensor_op_relu(pool, t); + success = tensor_grad_op_relu(pool, t); break; - case tensor_grad_op_t::SUB: + case tensor_op_t::SUB: assert(t->a != NULL); assert(t->b != NULL); assert(t->a->dtype == t->b->dtype); - success = tensor_op_sub(pool, t); + success = tensor_grad_op_sub(pool, t); break; - case tensor_grad_op_t::MEAN: + case tensor_op_t::MEAN: assert(t->a != NULL); - success = tensor_op_mean(pool, t); + success = tensor_grad_op_mean(pool, t); break; default: assert(false); } if (success) { - debug("tensor_evaluate: success\n"); + debug("backprop_cpu: success\n"); } else { - debug("tensor_evaluate: FUBAR\n"); + debug("backprop_cpu: FUBAR\n"); } return success; } @@ -102,38 +103,37 @@ bool backprop_cpu(execution_node_t *node) { /*************************************************************/ /*************************************************************/ /*************************************************************/ -// TODO: Implement GPU GRADIENT module +// TODO: Implement GPU module bool backprop_gpu(execution_node_t *node) { - assert(pool != NULL); - assert(t != NULL); + assert(node->t != NULL); bool success = false; - switch (t->op) { - case tensor_grad_t::NONE: + switch (node->t->op) { + case tensor_op_t::NONE: success = true; break; - case tensor_grad_op_t::CAST: + case tensor_op_t::CAST: break; // TODO: Implement here - case tensor_grad_op_t::ADD: + case tensor_op_t::ADD: assert(t->a != NULL); assert(t->b != NULL); assert(t->a->dtype == t->b->dtype); - success = tensor_add_op_cuda(t, d_a, d_b, d_res); + success = tensor_grad_add_op_cuda(t, d_a, d_b, d_res); break; - case tensor_grad_op_t::MUL_MATRIX: + case tensor_op_t::MUL_MATRIX: assert(t->a != NULL); assert(t->b != NULL); assert(t->a->dtype == t->b->dtype); - success = tensor_mul_op_cuda(t, d_a, d_b, d_res); + success = tensor_grad_mul_op_cuda(t, d_a, d_b, d_res); break; default: assert(false); } if (success) { - debug("tensor_evaluate_GPU: success\n"); + debug("backprop_gpu: success\n"); } else { - debug("tensor_evaluate_GPU: FUBAR\n"); + debug("backprop_gpu: FUBAR\n"); } return success; } diff --git a/src/backend_cpu/backprop/matmul_b.cpp b/src/backend_cpu/backprop/matmul_b.cpp new file mode 100644 index 0000000..6d5ed80 --- /dev/null +++ b/src/backend_cpu/backprop/matmul_b.cpp @@ -0,0 +1 @@ +#include "matmul_b.h" From c082e3c174dc62425228a21a669f86153c64e4fd Mon Sep 17 00:00:00 2001 From: Aakarsh Kashyap Date: Thu, 16 Apr 2026 02:01:51 +0530 Subject: [PATCH 12/27] BROKEN : THE FUCKING NN IS NOT LEARNING WRRRRRYYYYYYYYY FUCK --- error.txt | 46 +++ include/soft-cuda/tensor/api.h | 29 +- main.cpp | 121 +++--- src/backend_cpu/backprop/backprop_b.cpp | 344 ++++++++++++++---- src/backend_cpu/backprop/matmul_b.cpp | 1 - src/backend_cpu/include/backprop/backprop_b.h | 25 ++ src/backend_cpu/math/mean.cpp | 1 + src/backend_cpu/math/mse.cpp | 1 - src/backend_cpu/math/square.cpp | 2 +- src/core/graph/assignBackend.cu | 1 + src/core/graph/train.cpp | 26 ++ src/internal_header.h | 1 + 12 files changed, 448 insertions(+), 150 deletions(-) create mode 100644 error.txt create mode 100644 src/backend_cpu/include/backprop/backprop_b.h create mode 100644 src/core/graph/train.cpp diff --git a/error.txt b/error.txt new file mode 100644 index 0000000..e7afcf6 --- /dev/null +++ b/error.txt @@ -0,0 +1,46 @@ +-- Configuring done (0.3s) +-- Generating done (0.0s) +-- Build files have been written to: /home/wslarch/Documents/Coding/DEV/soft/soft-cuda/build + +real 0m0.317s +user 0m0.134s +sys 0m0.177s +[ 5%] Built target gtest +[ 21%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/mse.cpp.o +[ 21%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/bias_add.cpp.o +[ 21%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/mean.cpp.o +[ 21%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/backprop/backprop_b.cpp.o +[ 21%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/base/debug.cpp.o +[ 21%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/mul.cpp.o +[ 24%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/add.cpp.o +[ 27%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/relu.cpp.o +[ 29%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/scalar.cpp.o +[ 32%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/square.cpp.o +[ 37%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/transpose.cpp.o +[ 37%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/sub.cpp.o +[ 40%] Building CUDA object CMakeFiles/soft_lib.dir/src/backend_gpu/math/matmul.cu.o +[ 43%] Building CUDA object CMakeFiles/soft_lib.dir/src/backend_gpu/math/add.cu.o +[ 48%] Built target gmock +[ 54%] Built target gtest_main +[ 56%] Building CXX object CMakeFiles/soft_lib.dir/src/core/JSON/json_utils.cpp.o +[ 62%] Built target gmock_main +[ 64%] Building CXX object CMakeFiles/soft_lib.dir/src/core/graph/DAGbuild.cpp.o +[ 67%] Building CUDA object CMakeFiles/soft_lib.dir/src/core/graph/assignBackend.cu.o +[ 70%] Building CXX object CMakeFiles/soft_lib.dir/src/core/graph/train.cpp.o +[ 72%] Building CUDA object CMakeFiles/soft_lib.dir/src/core/pool/pool.cu.o +[ 75%] Building CXX object CMakeFiles/soft_lib.dir/src/core/tensor/tensor.cpp.o +[ 78%] Linking CUDA device code CMakeFiles/soft_lib.dir/cmake_device_link.o +[ 81%] Linking CXX shared library libsoft_lib.so +[ 83%] Built target soft_lib +[ 86%] Building CXX object CMakeFiles/soft.dir/main.cpp.o +[ 94%] Building CXX object tests/CMakeFiles/tensor_tests.dir/test_ops.cpp.o +[ 94%] Building CXX object tests/CMakeFiles/tensor_tests.dir/soft-cuda/tensor/test_tensor.cpp.o +[ 94%] Building CXX object tests/CMakeFiles/tensor_tests.dir/test_mul.cpp.o +[ 97%] Linking CXX executable soft +[ 97%] Built target soft +[100%] Linking CXX executable tensor_tests +[100%] Built target tensor_tests + +real 0m14.130s +user 1m18.485s +sys 0m19.782s diff --git a/include/soft-cuda/tensor/api.h b/include/soft-cuda/tensor/api.h index 3ff22e1..9ab3db1 100644 --- a/include/soft-cuda/tensor/api.h +++ b/include/soft-cuda/tensor/api.h @@ -259,12 +259,12 @@ tensor_t *tensor_mean(tensor_pool_t *pool, tensor_t *a); // return how correct we were b/w 0-1 tensor_t *tensor_mse_loss(tensor_pool_t *pool, tensor_t *predictions, tensor_t *target); -// Fills an existing tensor with normally distributed random numbers. +// // Fills an existing tensor with normally distributed random numbers. bool tensor_fill_random_normal(tensor_t *t, float mean, float std_dev); - -// Fused operation combining Softmax and Cross-Entropy for stability -tensor_t *tensor_cross_entropy_loss(tensor_pool_t *pool, const tensor_t *predictions, - const tensor_t *targets); +// +// // Fused operation combining Softmax and Cross-Entropy for stability +// tensor_t *tensor_cross_entropy_loss(tensor_pool_t *pool, const tensor_t *predictions, +// const tensor_t *targets); // *********************************************************************************** // TODO: HAVE TO UPDATED FUNC SIG SPEC @@ -286,9 +286,7 @@ bool tensor_evaluate_GPU( tensor_pool_t *pool,tensor_t *t, float *d_a, float *d bool tensor_backward(tensor_pool_t *pool, tensor_t *t); // The Optimizer -void tensor_sgd_template(tensor_pool_t *static_weights_pool, double learning_rate); - -// API UNSTABLE +void tensor_sgd(std::vector &nodes, float learning_rate); ///////////////////////////////////////////////////////////// // GRAPH OPERATIONS @@ -354,14 +352,15 @@ void printExecutionNode(execution_node_t *et); * */ bool tensor_graph_forward_evaluate(tensor_pool_t *pool_cpu, tensor_pool_t *pool_gpu, std::vector &nodes); -/* - * Evaluate the whole graph backward operation. - * @params graph struct for ops sequence - * @return boolean status flag - * */ -bool tensor_graph_backward_evaluate(tensor_graph_t *g); - ////////////////////////////////////////////////////////////////////////// +///////////////////////////////////////////////////////////////////////// +// BACKWARD PASS FUNCTIONS + +void gradInitializer(std::vector &nodes); + +bool tensor_graph_backward(std::vector &nodes); + +void assignGradMemory(tensor_pool_t *pool_grad_cpu, tensor_pool_t *pool_grad_gpu, std::vector &nodes); #ifdef __cplusplus } diff --git a/main.cpp b/main.cpp index 2a5605a..2cc3650 100644 --- a/main.cpp +++ b/main.cpp @@ -1,84 +1,99 @@ #include "soft-cuda/tensor/api.h" -// #include "soft-cuda/tensor/debug_api.h" +#include "soft-cuda/tensor/debug_api.h" #include #include +#include using namespace std; + int main() { - // Create pools // CPU ALLOCATION tensor_pool_t *pool = tensor_pool_create(1024 * 1024); - tensor_pool_t *pool_2 = tensor_pool_create(1024 * 1024); + tensor_pool_t *pool_meta = tensor_pool_create(1024 * 1024); tensor_pool_t *pool_grad_cpu = tensor_pool_create(1024 * 1024); - + + // GPU ALLOCATION tensor_pool_t *pool_gpu = tensor_pool_create(1024 * 1024, true); tensor_pool_t *pool_grad_gpu = tensor_pool_create(1024 * 1024, true); + assert(pool != NULL); assert(pool_gpu != NULL); - + cout << "=========================================== \n"; - cout << "=============TESTING DAG VERIFICATION============== \n"; + cout << "============= XOR IMPLEMENTATION ============== \n"; - float val_a[900]{}; - float val_b[900]{}; - float val_c[900]{}; - float val_d[900]{}; + float val_X[8]{0,0,0,1,1,0,1,1}; + float val_Y[4]{0,1,1,0}; + float val_W1[4]{}; + float val_W2[2]{}; + float val_b1[2]{}; + float val_b2[1]{}; + uint32_t dims_X[] = {4,2}; + uint32_t dims_Y[] = {4,1}; + uint32_t dims_W1[] = {2, 2}; + uint32_t dims_b1[] = {1, 2}; + uint32_t dims_b2[] = {1, 1}; + uint32_t dims_W2[] = {2, 1}; - // TACTICAL FIX: Make 'e' the exact same size to avoid a GPU segfault - // before we implement a dedicated GPU broadcasting kernel! - float val_e[900]{}; - - uint32_t dims_a[] = {30, 30}; - uint32_t dims_b[] = {30, 30}; - uint32_t dims_c[] = {30, 30}; - uint32_t dims_d[] = {30, 30}; - uint32_t dims_e[] = {30, 30}; // Matched dimensions! - - tensor_t *a = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_a, val_a, false); - tensor_t *b = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_b, val_b, false); - tensor_t *c = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_c, val_c, false); - tensor_t *d = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_d, val_d, false); - tensor_t *e = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_e, val_e, false); - tensor_fill_random_normal(a, 10.1, 5.7); - tensor_fill_random_normal(b, 10.1, 5.7); - tensor_fill_random_normal(c, 10.1, 5.7); - tensor_fill_random_normal(d, 10.1, 5.7); - tensor_fill_random_normal(e, 10.1, 5.7); - tensor_t *f = tensor_mul(pool, a, b); - tensor_t *g = tensor_mul(pool, b, c); - tensor_t *h = tensor_mul(pool, e, f); - tensor_t *i = tensor_mul(pool, h, a); - tensor_t *j = tensor_mul(pool, i, g); + tensor_t *X = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_X, val_X); + tensor_t *Y = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_Y, val_Y); + tensor_t *W1 = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_W1, val_W1); + tensor_t *W2 = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_W2, val_W2); + tensor_t *b1 = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_b1, val_b1); + tensor_t *b2 = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_b2, val_b2); + tensor_fill_random_normal(W1, 1, 0.01); + tensor_fill_random_normal(W2, 1.1, 0.01); + tensor_fill_random_normal(b1, 0.1, 0.01); + tensor_fill_random_normal(b2, 0.11, 0.01); + cout << "=========================================== \n"; - cout << "==============LAZY EVAL DONE=============== \n"; + cout << "============== DEFINING OPS =============== \n"; - std::vector seq; - bool oki = verifyIfDAG(pool_2, j, seq); + tensor_t *H_pred_bef = tensor_mul_naive(pool, X, W1); + tensor_t *H_pred = tensor_add(pool, H_pred_bef, b1); + tensor_t *H = tensor_relu(pool, H_pred); + tensor_t *Y_pred_bef = tensor_mul_naive(pool, H, W2); + tensor_t *Y_pred = tensor_add(pool, Y_pred_bef, b2); + tensor_t *L_sub = tensor_sub(pool, Y_pred, Y); + tensor_t *L_squ = tensor_square(pool, L_sub); + tensor_t *mse = tensor_mean(pool, L_squ); + cout << "=========================================== \n"; + cout << "============== LAZY EVAL DONE =============== \n"; - // Ensure assignDevice is returning device_type::GPU under the hood! - assignBackendGraph(pool_gpu, seq); - assignGradMemory(pool_grad_cpu,pool_grad_gpu, seq); - tensor_graph_forward_evaluate(pool, pool_gpu, seq); + cout << "=========================================== \n"; + cout << "============== PREPARING THE KITCHEN =============== \n"; + std::vector seq; + bool oki = verifyIfDAG(pool_meta, mse, seq); + assignBackendGraph(pool_gpu, seq); + assignGradMemory(pool_grad_cpu, pool_grad_gpu, seq); + if (oki) { - cout << "=========================================== \n"; - cout << "========= VRAM TRACE ============= \n"; - - execution_node_t *node = seq.back(); - - // Pull it back to the CPU - execution_node_to_host(node); - printExecutionNode(node); - cout << "\n"; + cout << "=========================================== \n"; + cout << "============== STARTING TRAINING =============== \n"; + for (int i = 0; i < 10000; i++) { + tensor_graph_forward_evaluate(pool, pool_gpu, seq); + gradInitializer(seq); + tensor_graph_backward(seq); + if (i % 1000 == 0) { + std::cout << i << "EPOCH"; + execution_node_t *mse = seq.back(); + printExecutionNode(mse); + } + tensor_sgd(seq, 0.01); + } + } else { + cout << "WARNING: DAG Verification failed. Aborting expedition.\n"; } - + tensor_pool_destroy(pool); - tensor_pool_destroy(pool_2); + tensor_pool_destroy(pool_meta); tensor_pool_destroy(pool_grad_cpu); tensor_pool_destroy(pool_grad_gpu); tensor_pool_destroy(pool_gpu); + return 0; } diff --git a/src/backend_cpu/backprop/backprop_b.cpp b/src/backend_cpu/backprop/backprop_b.cpp index e1476c2..3511e65 100644 --- a/src/backend_cpu/backprop/backprop_b.cpp +++ b/src/backend_cpu/backprop/backprop_b.cpp @@ -1,5 +1,6 @@ #include "internal_header.h" + // We are gonna write a backprop fucntion for matmul cause that is easy thing to do // Since our backprop is an eagar evaluation we will make tensor objects inside the function // transpose it execute it as well and then work with it @@ -8,132 +9,317 @@ // Since we are going to get execution node we could have a decision made here if it's data will be // written on device_ptr of node_.grad->data // We are gonna do this cause making decsion on higher level will require us to make two kernels for similar work. - +void gradInitializer(std::vector &nodes) { + for (auto &node : nodes) { + if (!node->t->grad_compute) continue; + memset(node->t->grad->data, 0,node->t->grad->nvalues * sizeof(float)); + // TODO: We are not zeroing the GPU will have to do that + } + if (!nodes.empty()) { + execution_node_t *root = nodes.back(); + if (root->t != nullptr && root->t->grad != nullptr && root->t->grad->data != nullptr) { + ((float*)root->t->grad->data)[0] = 1.0f; + } + } +} // Since this will be eagar evalution we are gonna execute it here now and we are gonna return a // boolean. // @return: boolean // @params: execution_node_t // Since all the memory is already executed we just need the execution node -bool backprop__(std::vector &nodes) { - for(auto &node : nodes) { - if (node->t->grad_compute) { - if (node->device_ptr_grad != NULL) { - bool success = backprop_gpu(node); - return success; - } else { - bool success = backprop_cpu(node); - return sucess; - } +bool backprop__(std::vector &nodes) { + for (int i = (int)nodes.size() - 1; i >= 0; i--) { + execution_node_t *node = nodes[(size_t)i]; + // Cause it's not needed + if (!node->t->grad_compute) + continue; + // Cause it's super child + if (node->t->op == tensor_op_t::NONE) + continue; + bool success{false}; + if (node->device_ptr_grad != NULL) { + success = backprop_gpu(node); + } else { + success = backprop_cpu(node); + } + if (!success) { + debug("backprop__: FUBAR at node pos=%d\n", (int)node->pos); + return false; } } + return true; } -bool backprop_cpu(execution_node_t *node) { +// ================================================================================ +// CPU BACKWARD DISPATCHER +// ================================================================================ +bool backprop_cpu(execution_node_t *node) { + assert(node != NULL); assert(node->t != NULL); bool success = false; - switch (t->op) { + switch (node->t->op) { case tensor_op_t::NONE: + // Leaf node success = true; break; case tensor_op_t::CAST: - break; // TODO: Implement here - case tensor_op_t::MUL_MATRIX: - assert(t->a != NULL); - assert(t->b != NULL); - assert(t->a->dtype == t->b->dtype); - success = tensor_mul_grad_op_matrix(pool, t); + success = false; break; + // case tensor_op_t::MUL_MATRIX: + // assert(node->t->a != NULL); + // assert(node->t->b != NULL); + // assert(node->t->a->dtype == node->t->b->dtype); + // success = tensor_mul_grad_op_matrix(node); + // break; case tensor_op_t::MUL_SCALAR: - assert(t->a != NULL); - assert(t->b != NULL); - assert(t->a->dtype == t->b->dtype); - success = tensor_mul_grad_op_scalar(pool, t); + assert(node->t->a != NULL); + assert(node->t->b != NULL); + assert(node->t->a->dtype == node->t->b->dtype); + success = tensor_mul_grad_op_scalar(node); break; case tensor_op_t::TRANSPOSE: - assert(t->a != NULL); - success = tensor_tranpose_grad_op_matrix(pool, t); + assert(node->t->a != NULL); + success = tensor_tranpose_grad_op_matrix(node); break; case tensor_op_t::NAIVE_MATRIX_MUL: - assert(t->a != NULL); - assert(t->b != NULL); - assert(t->a->dtype == t->b->dtype); - success = tensor_mul_grad_op_matrix_naive(pool, t); + assert(node->t->a != NULL); + assert(node->t->b != NULL); + assert(node->t->a->dtype == node->t->b->dtype); + success = tensor_mul_grad_op_matrix_naive(node); break; case tensor_op_t::ADD: - assert(t->a != NULL); - assert(t->b != NULL); - assert(t->a->dtype == t->b->dtype); - success = tensor_grad_op_add(pool, t); + assert(node->t->a != NULL); + assert(node->t->b != NULL); + assert(node->t->a->dtype == node->t->b->dtype); + success = tensor_grad_op_add(node); break; case tensor_op_t::BROADCAST_ADD: - assert(t->a != NULL); - assert(t->b != NULL); - assert(t->a->dtype == t->b->dtype); - success = tensor_grad_op_broadcasting_add(pool, t); + assert(node->t->a != NULL); + assert(node->t->b != NULL); + assert(node->t->a->dtype == node->t->b->dtype); + success = tensor_grad_op_broadcasting_add(node); break; case tensor_op_t::RELU: - assert(t->a != NULL); - success = tensor_grad_op_relu(pool, t); + assert(node->t->a != NULL); + success = tensor_grad_op_relu(node); break; case tensor_op_t::SUB: - assert(t->a != NULL); - assert(t->b != NULL); - assert(t->a->dtype == t->b->dtype); - success = tensor_grad_op_sub(pool, t); + assert(node->t->a != NULL); + assert(node->t->b != NULL); + assert(node->t->a->dtype == node->t->b->dtype); + success = tensor_grad_op_sub(node); break; case tensor_op_t::MEAN: - assert(t->a != NULL); - success = tensor_grad_op_mean(pool, t); + assert(node->t->a != NULL); + success = tensor_grad_op_mean(node); break; default: assert(false); } if (success) { - debug("backprop_cpu: success\n"); + debug("backprop_cpu: success for op=%d\n", (int)node->t->op); } else { - debug("backprop_cpu: FUBAR\n"); + debug("backprop_cpu: FUBAR for op=%d\n", (int)node->t->op); } return success; } -/*************************************************************/ -/*************************************************************/ -/*************************************************************/ -// TODO: Implement GPU module -bool backprop_gpu(execution_node_t *node) { +// ============================================================================= +// GPU BACKWARD DISPATCHER (STUB) +// ============================================================================= +bool backprop_gpu([[maybe_unused]] execution_node_t *node) { + assert(node != NULL); assert(node->t != NULL); - bool success = false; + // TODO: Implement GPU backward kernels + debug("backprop_gpu: not yet implemented, falling back\n"); + return false; +} +// TODO: ONE PROBLEM IS WE NEED MULTIPLE CASE FOR +bool tensor_grad_op_add(execution_node_t *node) { + tensor_t *out_grad = node->t->grad; + tensor_t *a = node->t->a; + tensor_t *b = node->t->b; - switch (node->t->op) { - case tensor_op_t::NONE: - success = true; - break; - case tensor_op_t::CAST: - break; - // TODO: Implement here - case tensor_op_t::ADD: - assert(t->a != NULL); - assert(t->b != NULL); - assert(t->a->dtype == t->b->dtype); - success = tensor_grad_add_op_cuda(t, d_a, d_b, d_res); - break; - case tensor_op_t::MUL_MATRIX: - assert(t->a != NULL); - assert(t->b != NULL); - assert(t->a->dtype == t->b->dtype); - success = tensor_grad_mul_op_cuda(t, d_a, d_b, d_res); - break; - default: - assert(false); + assert(out_grad != NULL && a != NULL && b != NULL); + assert(a->grad != NULL && b->grad != NULL); + float *g_out = (float *)out_grad->data; + float *g_a = (float *)a->grad; + float *g_b = (float *)b->grad; + uint32_t n = node->t->nvalues; + + for (uint32_t i = 0; i < n; i++) { + g_a[i] += g_out[i]; + g_b[i] += g_out[i]; } - if (success) { - debug("backprop_gpu: success\n"); - } else { - debug("backprop_gpu: FUBAR\n"); + return true; +} + +bool tensor_grad_op_broadcasting_add(execution_node_t *node) { + tensor_t *out_grad = node->t->grad; + tensor_t *a = node->t->a; + tensor_t *b = node->t->b; + + assert(out_grad != NULL && a != NULL && b != NULL); + assert(a->grad != NULL && b->grad != NULL); + float *g_out = (float *)out_grad->data; + float *g_a = (float *)a->grad; + float *g_b = (float *)b->grad; + + uint32_t rows = node->t->dims[0]; + uint32_t cols = node->t->dims[1]; + // We are assuming that b would be the broadcasted array + uint32_t val = node->t->nvalues; + for (uint32_t i = 0; i < val; i++) { + g_a[i] = g_out[i]; } - return success; + + // For broadcasting we accumulate btw + for (uint32_t i = 0; i < rows; i++) { + for (uint32_t j = 0; j < cols; j++) { + uint32_t b_idx = i * b->broadcast_stride[0] + j * b->broadcast_stride[1]; + g_b[b_idx] += g_out[i*cols + j]; + } + } + return true; +} + +bool tensor_grad_op_sub(execution_node_t *node) { + tensor_t *out_grad = node->t->grad; + tensor_t *a = node->t->a; + tensor_t *b = node->t->b; + assert(out_grad != NULL && a != NULL && b != NULL); + assert(a->grad != NULL && b->grad != NULL); + + float *g_out = (float *)out_grad->data; + float *g_a = (float *)a->grad->data; + float *g_b = (float *)b->grad->data; + uint32_t n = node->t->nvalues; + + for (uint32_t i = 0; i < n; i++) { + g_a[i] += g_out[i]; + g_b[i] -= g_out[i]; + } + return true; +} + + + +bool tensor_grad_op_relu(execution_node_t *node) { + tensor_t *out_grad = node->t->grad; + tensor_t *a = node->t->a; + assert(out_grad != NULL && a != NULL); + assert(a->grad != NULL); + + float *g_out = (float *)out_grad->data; + float *g_a = (float *)a->grad->data; + float *a_data = (float *)a->data; + uint32_t n = node->t->nvalues; + + for (uint32_t i = 0; i < n; i++) { + if (a_data[i] > 0.0f) { + g_a[i] += g_out[i]; + } + } + return true; +} + +bool tensor_grad_op_mean(execution_node_t *node) { + tensor_t *out_grad = node->t->grad; + tensor_t *a = node->t->a; + assert(out_grad != NULL && a != NULL); + assert(a->grad != NULL); + + // out->grad is a scalar — one value + float upstream = ((float *)out_grad->data)[0]; + float *g_a = (float *)a->grad->data; + uint32_t n = a->nvalues; + float scale = upstream / (float)n; + + for (uint32_t i = 0; i < n; i++) { + g_a[i] += scale; + } + return true; +} + +bool tensor_mul_grad_op_scalar(execution_node_t *node) { + tensor_t *out_grad = node->t->grad; + tensor_t *a = node->t->a; + tensor_t *b = node->t->b; + assert(out_grad != NULL && a != NULL && b != NULL); + assert(a->grad != NULL); + + float *g_out = (float *)out_grad->data; + float *g_a = (float *)a->grad->data; + float s = tensor_float32_value(b); + uint32_t n = a->nvalues; + + for (uint32_t i = 0; i < n; i++) { + g_a[i] += g_out[i] * s; + } + return true; +} + + +bool tensor_tranpose_grad_op_matrix(execution_node_t *node) { + tensor_t *out_grad = node->t->grad; + tensor_t *a = node->t->a; + assert(out_grad != NULL && a != NULL); + assert(a->grad != NULL); + + uint32_t rows = a->dims[0]; + uint32_t cols = a->dims[1]; + float *g_out = (float *)out_grad->data; + float *g_a = (float *)a->grad->data; + + for (uint32_t i = 0; i < rows; i++) { + for (uint32_t j = 0; j < cols; j++) { + g_a[i * cols + j] += g_out[j * rows + i]; + } + } + return true; +} + +bool tensor_mul_grad_op_matrix_naive(execution_node_t *node) { + tensor_t *out_grad = node->t->grad; + tensor_t *a = node->t->a; + tensor_t *b = node->t->b; + assert(out_grad != NULL && a != NULL && b != NULL); + assert(a->grad != NULL && b->grad != NULL); + + uint32_t M = a->dims[0]; + uint32_t K = a->dims[1]; + uint32_t N = b->dims[1]; + + float *g_out = (float *)out_grad->data; // (M, N) + float *a_data = (float *)a->data; // (M, K) + float *b_data = (float *)b->data; // (K, N) + float *g_a = (float *)a->grad->data; // (M, K) + float *g_b = (float *)b->grad->data; // (K, N) + + // dL/dA[i,k] += sum_j dL/dOut[i,j] * B[k,j] + for (uint32_t i = 0; i < M; i++) { + for (uint32_t k = 0; k < K; k++) { + float sum = 0.0f; + for (uint32_t j = 0; j < N; j++) { + sum += g_out[i * N + j] * b_data[k * N + j]; + } + g_a[i * K + k] += sum; + } + } + + // dL/dB[k,j] += sum_i A[i,k] * dL/dOut[i,j] + for (uint32_t k = 0; k < K; k++) { + for (uint32_t j = 0; j < N; j++) { + float sum = 0.0f; + for (uint32_t i = 0; i < M; i++) { + sum += a_data[i * K + k] * g_out[i * N + j]; + } + g_b[k * N + j] += sum; + } + } + return true; } diff --git a/src/backend_cpu/backprop/matmul_b.cpp b/src/backend_cpu/backprop/matmul_b.cpp index 6d5ed80..e69de29 100644 --- a/src/backend_cpu/backprop/matmul_b.cpp +++ b/src/backend_cpu/backprop/matmul_b.cpp @@ -1 +0,0 @@ -#include "matmul_b.h" diff --git a/src/backend_cpu/include/backprop/backprop_b.h b/src/backend_cpu/include/backprop/backprop_b.h new file mode 100644 index 0000000..6e11e50 --- /dev/null +++ b/src/backend_cpu/include/backprop/backprop_b.h @@ -0,0 +1,25 @@ +#pragma once + +void gradInitializer(std::vector &nodes); + +bool backprop__(std::vector &nodes); + +bool backprop_cpu(execution_node_t *node); + +bool backprop_gpu([[maybe_unused]] execution_node_t *node); + +bool tensor_grad_op_add(execution_node_t *node); + +bool tensor_grad_op_broadcasting_add(execution_node_t *node); + +bool tensor_grad_op_sub(execution_node_t *node); + +bool tensor_grad_op_relu(execution_node_t *node); + +bool tensor_grad_op_mean(execution_node_t *node); + +bool tensor_mul_grad_op_scalar(execution_node_t *node); + +bool tensor_tranpose_grad_op_matrix(execution_node_t *node); + +bool tensor_mul_grad_op_matrix_naive(execution_node_t *node); diff --git a/src/backend_cpu/math/mean.cpp b/src/backend_cpu/math/mean.cpp index ffb9ef0..7fddeb6 100644 --- a/src/backend_cpu/math/mean.cpp +++ b/src/backend_cpu/math/mean.cpp @@ -22,5 +22,6 @@ bool tensor_op_mean(tensor_pool_t *pool, tensor_t *t) { sum += ((float *)t->a->data)[i]; } ((float *)t->data)[0] = sum/(float)(t->a->nvalues); + ((float *)t->grad->data)[0] = 1.0f; return true; } diff --git a/src/backend_cpu/math/mse.cpp b/src/backend_cpu/math/mse.cpp index 9a133d8..82bba6a 100644 --- a/src/backend_cpu/math/mse.cpp +++ b/src/backend_cpu/math/mse.cpp @@ -1,3 +1,2 @@ #include "internal_header.h" - diff --git a/src/backend_cpu/math/square.cpp b/src/backend_cpu/math/square.cpp index f33715a..945e8c5 100644 --- a/src/backend_cpu/math/square.cpp +++ b/src/backend_cpu/math/square.cpp @@ -1,5 +1,5 @@ #include "internal_header.h" tensor_t *tensor_square(tensor_pool_t *pool, tensor_t *x) { - return tensor_mul(pool, x, x); + return tensor_mul_naive(pool, x, x); } diff --git a/src/core/graph/assignBackend.cu b/src/core/graph/assignBackend.cu index d472a33..4d285d1 100644 --- a/src/core/graph/assignBackend.cu +++ b/src/core/graph/assignBackend.cu @@ -23,6 +23,7 @@ void assignBackend(execution_node_t *e, json &data) { */ device_type assignDevice([[maybe_unused]] uint8_t ndims, [[maybe_unused]] uint32_t *dims, [[maybe_unused]] tensor_op_t op, uint32_t nvalues, json &data) { + return device_type::CPU; if(op == tensor_op_t::NONE) { return device_type::CPU; } diff --git a/src/core/graph/train.cpp b/src/core/graph/train.cpp new file mode 100644 index 0000000..f0c01a9 --- /dev/null +++ b/src/core/graph/train.cpp @@ -0,0 +1,26 @@ +#include "internal_header.h" +#include + +bool tensor_graph_backward(std::vector &nodes) { + return backprop__(nodes); +} + + +void tensor_sgd(std::vector &nodes, float learning_rate) { + for (auto &node : nodes) { + tensor_t *t = node->t; + + if (t->op != tensor_op_t::NONE) continue; + if (!t->grad_compute) continue; + if (t->grad == NULL) continue; + + float *w = (float *)t->data; + float *grad = (float *)t->grad->data; + uint32_t n = t->nvalues; + + for (uint32_t i = 0; i < n; i++) { + w[i] -= learning_rate * grad[i]; + // grad[i] = 0.0f; + } + } +} diff --git a/src/internal_header.h b/src/internal_header.h index 638a335..a426e07 100644 --- a/src/internal_header.h +++ b/src/internal_header.h @@ -14,6 +14,7 @@ #include "backend_cpu/include/scalar.h" #include "backend_cpu/include/transpose.h" #include "backend_cpu/include/relu.h" +#include "backend_cpu/include/backprop/backprop_b.h" #include "./core/include/graph/DAGbuild.h" #include "./core/include/graph/assignBackend.h" #include "./core/include/JSON/json_utils.h" From 39a67cee22f672e42a565059e5884fcc1b198bb5 Mon Sep 17 00:00:00 2001 From: Aakarsh Kashyap Date: Thu, 16 Apr 2026 02:54:50 +0530 Subject: [PATCH 13/27] BROKEN 2 : THE FUCKING NN IS NOT LEARNING WRRRRRYYYYYYYYY FUCK --- error.txt | 52 +++++++++---------- main.cpp | 4 +- src/backend_cpu/backprop/backprop_b.cpp | 24 ++++++++- src/backend_cpu/include/backprop/backprop_b.h | 2 + src/backend_cpu/include/square.h | 7 +++ src/backend_cpu/math/square.cpp | 29 ++++++++++- src/core/include/tensor/tensor.h | 1 + src/core/tensor/tensor.cpp | 4 ++ src/internal_header.h | 1 + summary.txt | 10 ++++ 10 files changed, 103 insertions(+), 31 deletions(-) create mode 100644 src/backend_cpu/include/square.h create mode 100644 summary.txt diff --git a/error.txt b/error.txt index e7afcf6..cd529e6 100644 --- a/error.txt +++ b/error.txt @@ -2,26 +2,26 @@ -- Generating done (0.0s) -- Build files have been written to: /home/wslarch/Documents/Coding/DEV/soft/soft-cuda/build -real 0m0.317s -user 0m0.134s -sys 0m0.177s +real 0m0.356s +user 0m0.157s +sys 0m0.181s [ 5%] Built target gtest -[ 21%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/mse.cpp.o -[ 21%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/bias_add.cpp.o -[ 21%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/mean.cpp.o -[ 21%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/backprop/backprop_b.cpp.o -[ 21%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/base/debug.cpp.o -[ 21%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/mul.cpp.o +[ 10%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/base/debug.cpp.o +[ 10%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/bias_add.cpp.o +[ 13%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/mean.cpp.o +[ 16%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/mse.cpp.o +[ 18%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/mul.cpp.o +[ 21%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/relu.cpp.o [ 24%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/add.cpp.o -[ 27%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/relu.cpp.o +[ 29%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/backprop/backprop_b.cpp.o [ 29%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/scalar.cpp.o [ 32%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/square.cpp.o -[ 37%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/transpose.cpp.o +[ 35%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/transpose.cpp.o [ 37%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/sub.cpp.o [ 40%] Building CUDA object CMakeFiles/soft_lib.dir/src/backend_gpu/math/matmul.cu.o [ 43%] Building CUDA object CMakeFiles/soft_lib.dir/src/backend_gpu/math/add.cu.o -[ 48%] Built target gmock -[ 54%] Built target gtest_main +[ 48%] Built target gtest_main +[ 54%] Built target gmock [ 56%] Building CXX object CMakeFiles/soft_lib.dir/src/core/JSON/json_utils.cpp.o [ 62%] Built target gmock_main [ 64%] Building CXX object CMakeFiles/soft_lib.dir/src/core/graph/DAGbuild.cpp.o @@ -29,18 +29,16 @@ sys 0m0.177s [ 70%] Building CXX object CMakeFiles/soft_lib.dir/src/core/graph/train.cpp.o [ 72%] Building CUDA object CMakeFiles/soft_lib.dir/src/core/pool/pool.cu.o [ 75%] Building CXX object CMakeFiles/soft_lib.dir/src/core/tensor/tensor.cpp.o -[ 78%] Linking CUDA device code CMakeFiles/soft_lib.dir/cmake_device_link.o -[ 81%] Linking CXX shared library libsoft_lib.so -[ 83%] Built target soft_lib -[ 86%] Building CXX object CMakeFiles/soft.dir/main.cpp.o -[ 94%] Building CXX object tests/CMakeFiles/tensor_tests.dir/test_ops.cpp.o -[ 94%] Building CXX object tests/CMakeFiles/tensor_tests.dir/soft-cuda/tensor/test_tensor.cpp.o -[ 94%] Building CXX object tests/CMakeFiles/tensor_tests.dir/test_mul.cpp.o -[ 97%] Linking CXX executable soft -[ 97%] Built target soft -[100%] Linking CXX executable tensor_tests -[100%] Built target tensor_tests +/home/wslarch/Documents/Coding/DEV/soft/soft-cuda/src/core/tensor/tensor.cpp: In function ‘bool tensor_evaluate(tensor_pool_t*, tensor_t*, float*, float*, float*)’: +/home/wslarch/Documents/Coding/DEV/soft/soft-cuda/src/core/tensor/tensor.cpp:176:19: error: ‘tensor_op_square’ was not declared in this scope; did you mean ‘tensor_square’? + 176 | success = tensor_op_square(pool, t); + | ^~~~~~~~~~~~~~~~ + | tensor_square +make[2]: *** [CMakeFiles/soft_lib.dir/build.make:363: CMakeFiles/soft_lib.dir/src/core/tensor/tensor.cpp.o] Error 1 +make[2]: *** Waiting for unfinished jobs.... +make[1]: *** [CMakeFiles/Makefile2:181: CMakeFiles/soft_lib.dir/all] Error 2 +make: *** [Makefile:146: all] Error 2 -real 0m14.130s -user 1m18.485s -sys 0m19.782s +real 0m11.154s +user 1m11.334s +sys 0m16.624s diff --git a/main.cpp b/main.cpp index 2cc3650..952f0d3 100644 --- a/main.cpp +++ b/main.cpp @@ -43,8 +43,8 @@ int main() { tensor_t *b1 = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_b1, val_b1); tensor_t *b2 = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_b2, val_b2); - tensor_fill_random_normal(W1, 1, 0.01); - tensor_fill_random_normal(W2, 1.1, 0.01); + tensor_fill_random_normal(W1, 0, 0.01); + tensor_fill_random_normal(W2, 0.1, 0.01); tensor_fill_random_normal(b1, 0.1, 0.01); tensor_fill_random_normal(b2, 0.11, 0.01); diff --git a/src/backend_cpu/backprop/backprop_b.cpp b/src/backend_cpu/backprop/backprop_b.cpp index 3511e65..04e72bc 100644 --- a/src/backend_cpu/backprop/backprop_b.cpp +++ b/src/backend_cpu/backprop/backprop_b.cpp @@ -117,6 +117,10 @@ bool backprop_cpu(execution_node_t *node) { assert(node->t->a != NULL); success = tensor_grad_op_mean(node); break; + case tensor_op_t::SQUARE: + assert(node->t->a != NULL); + success = tensor_grad_op_square(node); + break; default: assert(false); } @@ -174,7 +178,7 @@ bool tensor_grad_op_broadcasting_add(execution_node_t *node) { // We are assuming that b would be the broadcasted array uint32_t val = node->t->nvalues; for (uint32_t i = 0; i < val; i++) { - g_a[i] = g_out[i]; + g_a[i] += g_out[i]; } // For broadcasting we accumulate btw @@ -323,3 +327,21 @@ bool tensor_mul_grad_op_matrix_naive(execution_node_t *node) { } return true; } + +bool tensor_grad_op_square(execution_node_t *node) { + tensor_t *out_grad = node->t->grad; + tensor_t *a = node->t->a; + assert(out_grad != NULL && a != NULL); + assert(a->grad != NULL); + + float *g_out = (float *)out_grad->data; + float *g_a = (float *)a->grad->data; + float *a_data = (float *)a->data; + uint32_t n = node->t->nvalues; + + for (uint32_t i = 0; i < n; i++) { + g_a[i] += g_out[i] * 2.0f * a_data[i]; + } + return true; +} + diff --git a/src/backend_cpu/include/backprop/backprop_b.h b/src/backend_cpu/include/backprop/backprop_b.h index 6e11e50..6dd53fb 100644 --- a/src/backend_cpu/include/backprop/backprop_b.h +++ b/src/backend_cpu/include/backprop/backprop_b.h @@ -23,3 +23,5 @@ bool tensor_mul_grad_op_scalar(execution_node_t *node); bool tensor_tranpose_grad_op_matrix(execution_node_t *node); bool tensor_mul_grad_op_matrix_naive(execution_node_t *node); + +bool tensor_grad_op_square(execution_node_t *node); diff --git a/src/backend_cpu/include/square.h b/src/backend_cpu/include/square.h new file mode 100644 index 0000000..ceb9c33 --- /dev/null +++ b/src/backend_cpu/include/square.h @@ -0,0 +1,7 @@ +#pragma once + +#include "internal_header.h" + +tensor_t *tensor_square(tensor_pool_t *pool, tensor_t *x); + +bool tensor_op_square(tensor_pool_t *pool, tensor_t *t); diff --git a/src/backend_cpu/math/square.cpp b/src/backend_cpu/math/square.cpp index 945e8c5..a828114 100644 --- a/src/backend_cpu/math/square.cpp +++ b/src/backend_cpu/math/square.cpp @@ -1,5 +1,32 @@ #include "internal_header.h" tensor_t *tensor_square(tensor_pool_t *pool, tensor_t *x) { - return tensor_mul_naive(pool, x, x); + assert(pool != NULL); + assert(x != NULL); + + tensor_t *t = tensor_dtype_create(pool, x->dtype, x->ndims, x->dims, NULL); + if (t == NULL) { + return NULL; + } + + t->op = tensor_op_t::SQUARE; + t->a = x; + + return t; } + +bool tensor_op_square(tensor_pool_t *pool, tensor_t *t) { + assert(pool != NULL); + assert(t != NULL); + assert(t->a != NULL); + + float *in = (float *)t->a->data; + float *out = (float *)t->data; + uint32_t n = t->nvalues; + + for (uint32_t i = 0; i < n; i++) { + out[i] = in[i] * in[i]; + } + return true; +} + diff --git a/src/core/include/tensor/tensor.h b/src/core/include/tensor/tensor.h index 9e89b75..05ee054 100644 --- a/src/core/include/tensor/tensor.h +++ b/src/core/include/tensor/tensor.h @@ -14,6 +14,7 @@ enum class tensor_op_t { RELU, // Activation function SUB, // Subtract two tensor of same shape MEAN, // Returns a scalar value mean of the tensor + SQUARE, // Square the tensor element-wise }; struct tensor_instance { diff --git a/src/core/tensor/tensor.cpp b/src/core/tensor/tensor.cpp index efa2a47..76cac9b 100644 --- a/src/core/tensor/tensor.cpp +++ b/src/core/tensor/tensor.cpp @@ -171,6 +171,10 @@ bool tensor_evaluate(tensor_pool_t *pool, tensor_t *t, [[maybe_unused]]float *d assert(t->a != NULL); success = tensor_op_mean(pool, t); break; + case tensor_op_t::SQUARE: + assert(t->a != NULL); + success = tensor_op_square(pool, t); + break; default: assert(false); } diff --git a/src/internal_header.h b/src/internal_header.h index a426e07..625597a 100644 --- a/src/internal_header.h +++ b/src/internal_header.h @@ -10,6 +10,7 @@ #include "backend_cpu/include/sub.h" #include "backend_cpu/include/mean.h" #include "backend_cpu/include/debug.h" +#include "backend_cpu/include/square.h" #include "backend_cpu/include/mul.h" #include "backend_cpu/include/scalar.h" #include "backend_cpu/include/transpose.h" diff --git a/summary.txt b/summary.txt new file mode 100644 index 0000000..d009af4 --- /dev/null +++ b/summary.txt @@ -0,0 +1,10 @@ +0EPOCH13 +1000EPOCH13 +2000EPOCH13 +3000EPOCH13 +4000EPOCH13 +5000EPOCH13 +6000EPOCH13 +7000EPOCH13 +8000EPOCH13 +9000EPOCH13 From 90ab4d07b1596ae44a17424ee377d47ccd368103 Mon Sep 17 00:00:00 2001 From: Aakarsh Kashyap Date: Thu, 16 Apr 2026 03:24:29 +0530 Subject: [PATCH 14/27] Working operations there is still problem with routing but will resolve that in some time Auto grad almost complete --- main.cpp | 22 ++++++++++++++++++++-- src/core/tensor/tensor.cpp | 4 ++++ 2 files changed, 24 insertions(+), 2 deletions(-) diff --git a/main.cpp b/main.cpp index 952f0d3..46a06e8 100644 --- a/main.cpp +++ b/main.cpp @@ -36,8 +36,8 @@ int main() { uint32_t dims_b2[] = {1, 1}; uint32_t dims_W2[] = {2, 1}; - tensor_t *X = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_X, val_X); - tensor_t *Y = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_Y, val_Y); + tensor_t *X = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_X, val_X, false); + tensor_t *Y = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_Y, val_Y, false); tensor_t *W1 = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_W1, val_W1); tensor_t *W2 = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_W2, val_W2); tensor_t *b1 = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_b1, val_b1); @@ -85,6 +85,24 @@ int main() { } tensor_sgd(seq, 0.01); } + tensor_graph_forward_evaluate(pool, pool_gpu, seq); + + float* inputs = (float*)tensor_get_data(X); + float* targets = (float*)tensor_get_data(Y); + float* predictions = (float*)tensor_get_data(Y_pred); + + cout << "X1\tX2\t|\tTarget\t|\tPredicted\n"; + cout << "---------------------------------------------------\n"; + + for (int i = 0; i < 4; i++) { + float x1 = inputs[i * 2 + 0]; + float x2 = inputs[i * 2 + 1]; + float y_true = targets[i]; + float y_pred = predictions[i]; + + cout << x1 << "\t" << x2 << "\t|\t" << y_true << "\t|\t" << y_pred << "\n"; + } + cout << "=========================================== \n"; } else { cout << "WARNING: DAG Verification failed. Aborting expedition.\n"; } diff --git a/src/core/tensor/tensor.cpp b/src/core/tensor/tensor.cpp index 76cac9b..f90cfe9 100644 --- a/src/core/tensor/tensor.cpp +++ b/src/core/tensor/tensor.cpp @@ -235,6 +235,10 @@ uint8_t tensor_get_ndims(tensor_t *t) { return t->ndims; } uint32_t *tensor_get_dims(tensor_t *t) { return t->dims; } void tensor_print_data(tensor_t *t) { + if (t->ndims == 0) { + std::cout << ((float *)t->data)[0] << "\n"; + return; + } for (uint32_t i = 0; i < t->dims[0]; i++) { for (uint32_t j = 0; j < t->dims[1]; j++) { uint32_t index = i * t->dims[1] + j; From 85be285cc7256d2a450bd8b8f954d1378cdeeaec Mon Sep 17 00:00:00 2001 From: Aakarsh Kashyap Date: Thu, 16 Apr 2026 03:57:10 +0530 Subject: [PATCH 15/27] FAAAAAAAAAAAAAAAAAAAAAAAA: I HAVE COMPLETED THE AUTOGRAD FAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AUTOGRAD WORKS 4NN XOR WORKS AND LIFE IS BEST --- main.cpp | 123 +++++++++--------- src/backend_cpu/backprop/backprop_b.cpp | 164 +++++++++++++----------- src/backend_cpu/math/add.cpp | 160 ++++++++++++----------- 3 files changed, 234 insertions(+), 213 deletions(-) diff --git a/main.cpp b/main.cpp index 46a06e8..28a19cb 100644 --- a/main.cpp +++ b/main.cpp @@ -7,111 +7,118 @@ using namespace std; int main() { - // CPU ALLOCATION - tensor_pool_t *pool = tensor_pool_create(1024 * 1024); - tensor_pool_t *pool_meta = tensor_pool_create(1024 * 1024); + tensor_pool_t *pool = tensor_pool_create(1024 * 1024); + tensor_pool_t *pool_meta = tensor_pool_create(1024 * 1024); tensor_pool_t *pool_grad_cpu = tensor_pool_create(1024 * 1024); - - // GPU ALLOCATION - tensor_pool_t *pool_gpu = tensor_pool_create(1024 * 1024, true); + tensor_pool_t *pool_gpu = tensor_pool_create(1024 * 1024, true); tensor_pool_t *pool_grad_gpu = tensor_pool_create(1024 * 1024, true); - + assert(pool != NULL); assert(pool_gpu != NULL); - + cout << "=========================================== \n"; - cout << "============= XOR IMPLEMENTATION ============== \n"; + cout << "========= XOR IMPLEMENTATION (4-NEURON) ==== \n"; + + float val_X[8]{0,0, 0,1, 1,0, 1,1}; + float val_Y[4]{0, 1, 1, 0}; - float val_X[8]{0,0,0,1,1,0,1,1}; - float val_Y[4]{0,1,1,0}; - float val_W1[4]{}; - float val_W2[2]{}; - float val_b1[2]{}; - float val_b2[1]{}; - uint32_t dims_X[] = {4,2}; - uint32_t dims_Y[] = {4,1}; - uint32_t dims_W1[] = {2, 2}; - uint32_t dims_b1[] = {1, 2}; + float val_W1[8]{}; + float val_W2[4]{}; + float val_b1[4]{}; + float val_b2[1]{}; + + uint32_t dims_X[] = {4, 2}; + uint32_t dims_Y[] = {4, 1}; + uint32_t dims_W1[] = {2, 4}; + uint32_t dims_b1[] = {1, 4}; + uint32_t dims_W2[] = {4, 1}; uint32_t dims_b2[] = {1, 1}; - uint32_t dims_W2[] = {2, 1}; - + tensor_t *X = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_X, val_X, false); tensor_t *Y = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_Y, val_Y, false); - tensor_t *W1 = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_W1, val_W1); - tensor_t *W2 = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_W2, val_W2); - tensor_t *b1 = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_b1, val_b1); - tensor_t *b2 = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_b2, val_b2); - - tensor_fill_random_normal(W1, 0, 0.01); - tensor_fill_random_normal(W2, 0.1, 0.01); - tensor_fill_random_normal(b1, 0.1, 0.01); - tensor_fill_random_normal(b2, 0.11, 0.01); + tensor_t *W1 = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_W1, val_W1, true); + tensor_t *W2 = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_W2, val_W2, true); + tensor_t *b1 = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_b1, val_b1, true); + tensor_t *b2 = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_b2, val_b2, true); + + tensor_fill_random_normal(W1, 0.5, 0.2); + tensor_fill_random_normal(W2, 0.5, 0.2); + tensor_fill_random_normal(b1, 0.0, 0.1); + tensor_fill_random_normal(b2, 0.0, 0.1); + cout << "=========================================== \n"; cout << "============== DEFINING OPS =============== \n"; - + tensor_t *H_pred_bef = tensor_mul_naive(pool, X, W1); - tensor_t *H_pred = tensor_add(pool, H_pred_bef, b1); - tensor_t *H = tensor_relu(pool, H_pred); + tensor_t *H_pred = tensor_add(pool, H_pred_bef, b1); + tensor_t *H = tensor_relu(pool, H_pred); tensor_t *Y_pred_bef = tensor_mul_naive(pool, H, W2); - tensor_t *Y_pred = tensor_add(pool, Y_pred_bef, b2); - tensor_t *L_sub = tensor_sub(pool, Y_pred, Y); - tensor_t *L_squ = tensor_square(pool, L_sub); - tensor_t *mse = tensor_mean(pool, L_squ); - cout << "=========================================== \n"; - cout << "============== LAZY EVAL DONE =============== \n"; + tensor_t *Y_pred = tensor_add(pool, Y_pred_bef, b2); + tensor_t *L_sub = tensor_sub(pool, Y_pred, Y); + tensor_t *L_squ = tensor_square(pool, L_sub); + tensor_t *mse = tensor_mean(pool, L_squ); + + cout << "=========================================== \n"; + cout << "============== LAZY EVAL DONE ============= \n"; cout << "=========================================== \n"; - cout << "============== PREPARING THE KITCHEN =============== \n"; + cout << "=========== PREPARING THE KITCHEN ========= \n"; std::vector seq; bool oki = verifyIfDAG(pool_meta, mse, seq); assignBackendGraph(pool_gpu, seq); assignGradMemory(pool_grad_cpu, pool_grad_gpu, seq); - - + if (oki) { - cout << "=========================================== \n"; - cout << "============== STARTING TRAINING =============== \n"; - for (int i = 0; i < 10000; i++) { + cout << "=========================================== \n"; + cout << "============= STARTING TRAINING =========== \n"; + + for (int i = 0; i <= 10000; i++) { tensor_graph_forward_evaluate(pool, pool_gpu, seq); gradInitializer(seq); tensor_graph_backward(seq); + if (i % 1000 == 0) { - std::cout << i << "EPOCH"; - execution_node_t *mse = seq.back(); - printExecutionNode(mse); + std::cout << "EPOCH " << i << "\n"; + execution_node_t *mse_node = seq.back(); + printExecutionNode(mse_node); } - tensor_sgd(seq, 0.01); + + tensor_sgd(seq, 0.05); } + tensor_graph_forward_evaluate(pool, pool_gpu, seq); - - float* inputs = (float*)tensor_get_data(X); - float* targets = (float*)tensor_get_data(Y); + + float* inputs = (float*)tensor_get_data(X); + float* targets = (float*)tensor_get_data(Y); float* predictions = (float*)tensor_get_data(Y_pred); + cout << "=========================================== \n"; + cout << "============= XOR(4 NEURONS RESULT) ============= \n"; cout << "X1\tX2\t|\tTarget\t|\tPredicted\n"; cout << "---------------------------------------------------\n"; - + for (int i = 0; i < 4; i++) { - float x1 = inputs[i * 2 + 0]; - float x2 = inputs[i * 2 + 1]; + float x1 = inputs[i * 2 + 0]; + float x2 = inputs[i * 2 + 1]; float y_true = targets[i]; float y_pred = predictions[i]; - + cout << x1 << "\t" << x2 << "\t|\t" << y_true << "\t|\t" << y_pred << "\n"; } cout << "=========================================== \n"; + } else { cout << "WARNING: DAG Verification failed. Aborting expedition.\n"; } - + tensor_pool_destroy(pool); tensor_pool_destroy(pool_meta); tensor_pool_destroy(pool_grad_cpu); tensor_pool_destroy(pool_grad_gpu); tensor_pool_destroy(pool_gpu); - + return 0; } diff --git a/src/backend_cpu/backprop/backprop_b.cpp b/src/backend_cpu/backprop/backprop_b.cpp index 04e72bc..c8f2712 100644 --- a/src/backend_cpu/backprop/backprop_b.cpp +++ b/src/backend_cpu/backprop/backprop_b.cpp @@ -149,15 +149,14 @@ bool tensor_grad_op_add(execution_node_t *node) { tensor_t *b = node->t->b; assert(out_grad != NULL && a != NULL && b != NULL); - assert(a->grad != NULL && b->grad != NULL); float *g_out = (float *)out_grad->data; - float *g_a = (float *)a->grad; - float *g_b = (float *)b->grad; + float *g_a = a->grad != NULL ? (float *)a->grad->data : NULL; + float *g_b = b->grad != NULL ? (float *)b->grad->data : NULL; uint32_t n = node->t->nvalues; for (uint32_t i = 0; i < n; i++) { - g_a[i] += g_out[i]; - g_b[i] += g_out[i]; + if (g_a) g_a[i] += g_out[i]; + if (g_b) g_b[i] += g_out[i]; } return true; } @@ -168,25 +167,28 @@ bool tensor_grad_op_broadcasting_add(execution_node_t *node) { tensor_t *b = node->t->b; assert(out_grad != NULL && a != NULL && b != NULL); - assert(a->grad != NULL && b->grad != NULL); float *g_out = (float *)out_grad->data; - float *g_a = (float *)a->grad; - float *g_b = (float *)b->grad; + float *g_a = a->grad != NULL ? (float *)a->grad->data : NULL; + float *g_b = b->grad != NULL ? (float *)b->grad->data : NULL; uint32_t rows = node->t->dims[0]; uint32_t cols = node->t->dims[1]; // We are assuming that b would be the broadcasted array uint32_t val = node->t->nvalues; - for (uint32_t i = 0; i < val; i++) { - g_a[i] += g_out[i]; + if (g_a) { + for (uint32_t i = 0; i < val; i++) { + g_a[i] += g_out[i]; + } } - // For broadcasting we accumulate btw - for (uint32_t i = 0; i < rows; i++) { - for (uint32_t j = 0; j < cols; j++) { - uint32_t b_idx = i * b->broadcast_stride[0] + j * b->broadcast_stride[1]; - g_b[b_idx] += g_out[i*cols + j]; - } + if (g_b) { + // For broadcasting we accumulate btw + for (uint32_t i = 0; i < rows; i++) { + for (uint32_t j = 0; j < cols; j++) { + uint32_t b_idx = i * b->broadcast_stride[0] + j * b->broadcast_stride[1]; + g_b[b_idx] += g_out[i*cols + j]; + } + } } return true; } @@ -196,16 +198,15 @@ bool tensor_grad_op_sub(execution_node_t *node) { tensor_t *a = node->t->a; tensor_t *b = node->t->b; assert(out_grad != NULL && a != NULL && b != NULL); - assert(a->grad != NULL && b->grad != NULL); float *g_out = (float *)out_grad->data; - float *g_a = (float *)a->grad->data; - float *g_b = (float *)b->grad->data; + float *g_a = a->grad != NULL ? (float *)a->grad->data : NULL; + float *g_b = b->grad != NULL ? (float *)b->grad->data : NULL; uint32_t n = node->t->nvalues; for (uint32_t i = 0; i < n; i++) { - g_a[i] += g_out[i]; - g_b[i] -= g_out[i]; + if (g_a) g_a[i] += g_out[i]; + if (g_b) g_b[i] -= g_out[i]; } return true; } @@ -216,16 +217,17 @@ bool tensor_grad_op_relu(execution_node_t *node) { tensor_t *out_grad = node->t->grad; tensor_t *a = node->t->a; assert(out_grad != NULL && a != NULL); - assert(a->grad != NULL); - float *g_out = (float *)out_grad->data; - float *g_a = (float *)a->grad->data; - float *a_data = (float *)a->data; - uint32_t n = node->t->nvalues; + if (a->grad != NULL) { + float *g_out = (float *)out_grad->data; + float *g_a = (float *)a->grad->data; + float *a_data = (float *)a->data; + uint32_t n = node->t->nvalues; - for (uint32_t i = 0; i < n; i++) { - if (a_data[i] > 0.0f) { - g_a[i] += g_out[i]; + for (uint32_t i = 0; i < n; i++) { + if (a_data[i] > 0.0f) { + g_a[i] += g_out[i]; + } } } return true; @@ -235,16 +237,17 @@ bool tensor_grad_op_mean(execution_node_t *node) { tensor_t *out_grad = node->t->grad; tensor_t *a = node->t->a; assert(out_grad != NULL && a != NULL); - assert(a->grad != NULL); - // out->grad is a scalar — one value - float upstream = ((float *)out_grad->data)[0]; - float *g_a = (float *)a->grad->data; - uint32_t n = a->nvalues; - float scale = upstream / (float)n; + if (a->grad != NULL) { + // out->grad is a scalar — one value + float upstream = ((float *)out_grad->data)[0]; + float *g_a = (float *)a->grad->data; + uint32_t n = a->nvalues; + float scale = upstream / (float)n; - for (uint32_t i = 0; i < n; i++) { - g_a[i] += scale; + for (uint32_t i = 0; i < n; i++) { + g_a[i] += scale; + } } return true; } @@ -254,15 +257,16 @@ bool tensor_mul_grad_op_scalar(execution_node_t *node) { tensor_t *a = node->t->a; tensor_t *b = node->t->b; assert(out_grad != NULL && a != NULL && b != NULL); - assert(a->grad != NULL); - float *g_out = (float *)out_grad->data; - float *g_a = (float *)a->grad->data; - float s = tensor_float32_value(b); - uint32_t n = a->nvalues; + if (a->grad != NULL) { + float *g_out = (float *)out_grad->data; + float *g_a = (float *)a->grad->data; + float s = tensor_float32_value(b); + uint32_t n = a->nvalues; - for (uint32_t i = 0; i < n; i++) { - g_a[i] += g_out[i] * s; + for (uint32_t i = 0; i < n; i++) { + g_a[i] += g_out[i] * s; + } } return true; } @@ -272,16 +276,17 @@ bool tensor_tranpose_grad_op_matrix(execution_node_t *node) { tensor_t *out_grad = node->t->grad; tensor_t *a = node->t->a; assert(out_grad != NULL && a != NULL); - assert(a->grad != NULL); - uint32_t rows = a->dims[0]; - uint32_t cols = a->dims[1]; - float *g_out = (float *)out_grad->data; - float *g_a = (float *)a->grad->data; + if (a->grad != NULL) { + uint32_t rows = a->dims[0]; + uint32_t cols = a->dims[1]; + float *g_out = (float *)out_grad->data; + float *g_a = (float *)a->grad->data; - for (uint32_t i = 0; i < rows; i++) { - for (uint32_t j = 0; j < cols; j++) { - g_a[i * cols + j] += g_out[j * rows + i]; + for (uint32_t i = 0; i < rows; i++) { + for (uint32_t j = 0; j < cols; j++) { + g_a[i * cols + j] += g_out[j * rows + i]; + } } } return true; @@ -292,7 +297,6 @@ bool tensor_mul_grad_op_matrix_naive(execution_node_t *node) { tensor_t *a = node->t->a; tensor_t *b = node->t->b; assert(out_grad != NULL && a != NULL && b != NULL); - assert(a->grad != NULL && b->grad != NULL); uint32_t M = a->dims[0]; uint32_t K = a->dims[1]; @@ -301,28 +305,32 @@ bool tensor_mul_grad_op_matrix_naive(execution_node_t *node) { float *g_out = (float *)out_grad->data; // (M, N) float *a_data = (float *)a->data; // (M, K) float *b_data = (float *)b->data; // (K, N) - float *g_a = (float *)a->grad->data; // (M, K) - float *g_b = (float *)b->grad->data; // (K, N) - - // dL/dA[i,k] += sum_j dL/dOut[i,j] * B[k,j] - for (uint32_t i = 0; i < M; i++) { - for (uint32_t k = 0; k < K; k++) { - float sum = 0.0f; - for (uint32_t j = 0; j < N; j++) { - sum += g_out[i * N + j] * b_data[k * N + j]; + float *g_a = a->grad != NULL ? (float *)a->grad->data : NULL; // (M, K) + float *g_b = b->grad != NULL ? (float *)b->grad->data : NULL; // (K, N) + + if (g_a) { + // dL/dA[i,k] += sum_j dL/dOut[i,j] * B[k,j] + for (uint32_t i = 0; i < M; i++) { + for (uint32_t k = 0; k < K; k++) { + float sum = 0.0f; + for (uint32_t j = 0; j < N; j++) { + sum += g_out[i * N + j] * b_data[k * N + j]; + } + g_a[i * K + k] += sum; } - g_a[i * K + k] += sum; } } - // dL/dB[k,j] += sum_i A[i,k] * dL/dOut[i,j] - for (uint32_t k = 0; k < K; k++) { - for (uint32_t j = 0; j < N; j++) { - float sum = 0.0f; - for (uint32_t i = 0; i < M; i++) { - sum += a_data[i * K + k] * g_out[i * N + j]; + if (g_b) { + // dL/dB[k,j] += sum_i A[i,k] * dL/dOut[i,j] + for (uint32_t k = 0; k < K; k++) { + for (uint32_t j = 0; j < N; j++) { + float sum = 0.0f; + for (uint32_t i = 0; i < M; i++) { + sum += a_data[i * K + k] * g_out[i * N + j]; + } + g_b[k * N + j] += sum; } - g_b[k * N + j] += sum; } } return true; @@ -332,16 +340,18 @@ bool tensor_grad_op_square(execution_node_t *node) { tensor_t *out_grad = node->t->grad; tensor_t *a = node->t->a; assert(out_grad != NULL && a != NULL); - assert(a->grad != NULL); - float *g_out = (float *)out_grad->data; - float *g_a = (float *)a->grad->data; - float *a_data = (float *)a->data; - uint32_t n = node->t->nvalues; + if (a->grad != NULL) { + float *g_out = (float *)out_grad->data; + float *g_a = (float *)a->grad->data; + float *a_data = (float *)a->data; + uint32_t n = node->t->nvalues; - for (uint32_t i = 0; i < n; i++) { - g_a[i] += g_out[i] * 2.0f * a_data[i]; + for (uint32_t i = 0; i < n; i++) { + g_a[i] += g_out[i] * 2.0f * a_data[i]; + } } return true; } + diff --git a/src/backend_cpu/math/add.cpp b/src/backend_cpu/math/add.cpp index 54b5d77..a110607 100644 --- a/src/backend_cpu/math/add.cpp +++ b/src/backend_cpu/math/add.cpp @@ -1,78 +1,82 @@ -#include "internal_header.h" - -static tensor_t *tensor_broadcast_add(tensor_pool_t *pool, tensor_t *a, tensor_t *b) { - assert(pool != NULL); - assert(a != NULL); - assert(b != NULL); - - if (b->dims[0] == 1) { - b->broadcast_stride[0] = 0; - b->broadcast_stride[1] = b->stride[1]; - } else { - b->broadcast_stride[1] = 0; - b->broadcast_stride[0] = b->stride[0]; - } - - tensor_t *t = tensor_dtype_create(pool, a->dtype, a->ndims, a->dims, NULL); - if (t == NULL) { - return NULL; - } - - t->op = tensor_op_t::BROADCAST_ADD; - t->a = a; - t->b = b; - return t; -} - -// We are baking lots of assumptions in this one haha -tensor_t *tensor_add(tensor_pool_t *pool, tensor_t *x, tensor_t *y) { - assert(pool != NULL); - assert(x != NULL); - assert(y != NULL); - - if (x->dims[0] == 1 || x->dims[1] == 1) { - return tensor_broadcast_add(pool, y, x); - } - if (y->dims[0] == 1 || y->dims[1] == 1) { - return tensor_broadcast_add(pool, x, y); - } - - tensor_t *t = tensor_dtype_create(pool, x->dtype, x->ndims, x->dims, NULL); - if (t == NULL) { - return NULL; - } - t->op = tensor_op_t::ADD; - t->a = x; - t->b = y; - return t; -} - -bool tensor_op_add(tensor_pool_t *pool, tensor_t *t) { - assert(pool != NULL); - assert(t != NULL); - - for (uint32_t i = 0; i < t->dims[0] * t->dims[1]; i++) { - ((float *)t->data)[i] = ((float *)t->a->data)[i] + ((float *)t->b->data)[i]; - } - return true; -} - -bool tensor_op_broadcasting_add(tensor_pool_t *pool, tensor_t *t) { - assert(pool != NULL); - assert(t != NULL); - assert(t->a != NULL); - assert(t->b != NULL); - - for (uint32_t i = 0; i < t->dims[0] * t->dims[1]; i++) { - ((float *)t->data)[i] = ((float *)t->a->data)[i] + get_sum_for_col_op(t->b, i, t); - } - return true; -} - -float get_sum_for_col_op(tensor_t *b, uint32_t i, tensor_t *t) { - uint32_t row = i / (t->dims[1]); - uint32_t col = i % (t->dims[1]); - float val = - ((float *)b->data)[row * ((b->broadcast_stride)[0]) + col * ((b->broadcast_stride)[1])]; - return val; -} +#include "internal_header.h" + +static tensor_t *tensor_broadcast_add(tensor_pool_t *pool, tensor_t *a, tensor_t *b) { + assert(pool != NULL); + assert(a != NULL); + assert(b != NULL); + + if (b->dims[0] == 1) { + b->broadcast_stride[0] = 0; + b->broadcast_stride[1] = b->stride[1]; + } else { + b->broadcast_stride[1] = 0; + b->broadcast_stride[0] = b->stride[0]; + } + + tensor_t *t = tensor_dtype_create(pool, a->dtype, a->ndims, a->dims, NULL); + if (t == NULL) { + return NULL; + } + + t->op = tensor_op_t::BROADCAST_ADD; + t->a = a; + t->b = b; + return t; +} + +// We are baking lots of assumptions in this one haha +tensor_t *tensor_add(tensor_pool_t *pool, tensor_t *x, tensor_t *y) { + assert(pool != NULL); + assert(x != NULL); + assert(y != NULL); + + if (x->nvalues == y->nvalues) { + tensor_t *t = tensor_dtype_create(pool, x->dtype, x->ndims, x->dims, NULL); + if (t == NULL) return NULL; + t->op = tensor_op_t::ADD; + t->a = x; + t->b = y; + return t; + } + + if (y->nvalues < x->nvalues && (y->dims[0] == 1 || y->dims[1] == 1)) { + return tensor_broadcast_add(pool, x, y); + } + + if (x->nvalues < y->nvalues && (x->dims[0] == 1 || x->dims[1] == 1)) { + return tensor_broadcast_add(pool, y, x); + } + + return NULL; // Unsupported broadcast shape +} + +bool tensor_op_add(tensor_pool_t *pool, tensor_t *t) { + assert(pool != NULL); + assert(t != NULL); + + for (uint32_t i = 0; i < t->dims[0] * t->dims[1]; i++) { + ((float *)t->data)[i] = ((float *)t->a->data)[i] + ((float *)t->b->data)[i]; + } + return true; +} + +bool tensor_op_broadcasting_add(tensor_pool_t *pool, tensor_t *t) { + assert(pool != NULL); + assert(t != NULL); + assert(t->a != NULL); + assert(t->b != NULL); + + for (uint32_t i = 0; i < t->dims[0] * t->dims[1]; i++) { + ((float *)t->data)[i] = ((float *)t->a->data)[i] + get_sum_for_col_op(t->b, i, t); + } + return true; +} + +float get_sum_for_col_op(tensor_t *b, uint32_t i, tensor_t *t) { + uint32_t row = i / (t->dims[1]); + uint32_t col = i % (t->dims[1]); + float val = + ((float *)b->data)[row * ((b->broadcast_stride)[0]) + col * ((b->broadcast_stride)[1])]; + return val; +} + From ee6b5b35c19721eb1a99ed72ce891f358b4ed7f4 Mon Sep 17 00:00:00 2001 From: Aakarsh Kashyap Date: Sat, 18 Apr 2026 01:47:07 +0530 Subject: [PATCH 16/27] GPU fallback is working but i feel something is wrong with hybrid path will need to see --- include/soft-cuda/tensor/api.h | 11 +++- main.cpp | 91 +++++++++++++++++---------------- src/core/graph/assignBackend.cu | 58 ++++++++++++++++++--- src/core/tensor/tensor.cpp | 4 +- 4 files changed, 111 insertions(+), 53 deletions(-) diff --git a/include/soft-cuda/tensor/api.h b/include/soft-cuda/tensor/api.h index 9ab3db1..03a9edd 100644 --- a/include/soft-cuda/tensor/api.h +++ b/include/soft-cuda/tensor/api.h @@ -30,7 +30,12 @@ enum class device_type { GPU, CPU, }; - +// Added this for assignBackend function so that we can adjust according to user requirements +enum class backend_mode { + GPU, + CPU, + HYBRID, +}; // DONE // Opaque tensor typedef struct tensor_instance tensor_t; @@ -330,7 +335,7 @@ bool verifyIfDAG(tensor_pool_t *pool, tensor_t *t, std::vector &nodes); +void assignBackendGraph(tensor_pool_t *pool,std::vector &nodes, backend_mode value = backend_mode::CPU); void assignGradMemory(tensor_pool_t *pool_grad_cpu, tensor_pool_t *pool_grad_gpu, std::vector &nodes); /* @params Take execution_node_t which you wanna know @@ -362,6 +367,8 @@ bool tensor_graph_backward(std::vector &nodes); void assignGradMemory(tensor_pool_t *pool_grad_cpu, tensor_pool_t *pool_grad_gpu, std::vector &nodes); +void autogradGpuMemTranfer(std::vector &nodes); + #ifdef __cplusplus } #endif diff --git a/main.cpp b/main.cpp index 28a19cb..e9974c4 100644 --- a/main.cpp +++ b/main.cpp @@ -68,51 +68,54 @@ int main() { cout << "=========== PREPARING THE KITCHEN ========= \n"; std::vector seq; bool oki = verifyIfDAG(pool_meta, mse, seq); - assignBackendGraph(pool_gpu, seq); + assignBackendGraph(pool_gpu, seq, backend_mode::HYBRID); + std::cout << "TESTING OUT "; assignGradMemory(pool_grad_cpu, pool_grad_gpu, seq); - - if (oki) { - cout << "=========================================== \n"; - cout << "============= STARTING TRAINING =========== \n"; - - for (int i = 0; i <= 10000; i++) { - tensor_graph_forward_evaluate(pool, pool_gpu, seq); - gradInitializer(seq); - tensor_graph_backward(seq); - - if (i % 1000 == 0) { - std::cout << "EPOCH " << i << "\n"; - execution_node_t *mse_node = seq.back(); - printExecutionNode(mse_node); - } - - tensor_sgd(seq, 0.05); - } - - tensor_graph_forward_evaluate(pool, pool_gpu, seq); - - float* inputs = (float*)tensor_get_data(X); - float* targets = (float*)tensor_get_data(Y); - float* predictions = (float*)tensor_get_data(Y_pred); - - cout << "=========================================== \n"; - cout << "============= XOR(4 NEURONS RESULT) ============= \n"; - cout << "X1\tX2\t|\tTarget\t|\tPredicted\n"; - cout << "---------------------------------------------------\n"; - - for (int i = 0; i < 4; i++) { - float x1 = inputs[i * 2 + 0]; - float x2 = inputs[i * 2 + 1]; - float y_true = targets[i]; - float y_pred = predictions[i]; - - cout << x1 << "\t" << x2 << "\t|\t" << y_true << "\t|\t" << y_pred << "\n"; - } - cout << "=========================================== \n"; - - } else { - cout << "WARNING: DAG Verification failed. Aborting expedition.\n"; - } + tensor_graph_forward_evaluate(pool, pool_gpu, seq); + + // if (oki) { + // cout << "=========================================== \n"; + // cout << "============= STARTING TRAINING =========== \n"; + // + // for (int i = 0; i <= 10000; i++) { + // tensor_graph_forward_evaluate(pool, pool_gpu, seq); + // autogradGpuMemTranfer(seq); + // gradInitializer(seq); + // tensor_graph_backward(seq); + // + // if (i % 1000 == 0) { + // std::cout << "EPOCH " << i << "\n"; + // execution_node_t *mse_node = seq.back(); + // printExecutionNode(mse_node); + // } + // + // tensor_sgd(seq, 0.05); + // } + // + // tensor_graph_forward_evaluate(pool, pool_gpu, seq); + // + // float* inputs = (float*)tensor_get_data(X); + // float* targets = (float*)tensor_get_data(Y); + // float* predictions = (float*)tensor_get_data(Y_pred); + // + // cout << "=========================================== \n"; + // cout << "============= XOR(4 NEURONS RESULT) ============= \n"; + // cout << "X1\tX2\t|\tTarget\t|\tPredicted\n"; + // cout << "---------------------------------------------------\n"; + // + // for (int i = 0; i < 4; i++) { + // float x1 = inputs[i * 2 + 0]; + // float x2 = inputs[i * 2 + 1]; + // float y_true = targets[i]; + // float y_pred = predictions[i]; + // + // cout << x1 << "\t" << x2 << "\t|\t" << y_true << "\t|\t" << y_pred << "\n"; + // } + // cout << "=========================================== \n"; + // + // } else { + // cout << "WARNING: DAG Verification failed. Aborting expedition.\n"; + // } tensor_pool_destroy(pool); tensor_pool_destroy(pool_meta); diff --git a/src/core/graph/assignBackend.cu b/src/core/graph/assignBackend.cu index 4d285d1..59d2b79 100644 --- a/src/core/graph/assignBackend.cu +++ b/src/core/graph/assignBackend.cu @@ -16,6 +16,22 @@ void assignBackend(execution_node_t *e, json &data) { e->device_ptr = NULL; // pre-allocation happens in Pass 2 } } +void assignBackendCpu(execution_node_t *e) { + e->backend_fn = tensor_evaluate; + e->device_ptr = NULL; +} + +void assignBackendGpu(execution_node_t *e) { + if (e->t->op == tensor_op_t::NONE) { + e->backend_fn = tensor_evaluate; + e->device_ptr = NULL; + return; + + } + e->backend_fn = tensor_evaluate_GPU; + e->device_ptr = NULL; + return; +} // TODO: Implement CONFIG.soft parser and assignment on the basis of that /* NOTE: Remember that when op is tensor_op_t::NONE you assign it to CPU, @@ -23,7 +39,6 @@ void assignBackend(execution_node_t *e, json &data) { */ device_type assignDevice([[maybe_unused]] uint8_t ndims, [[maybe_unused]] uint32_t *dims, [[maybe_unused]] tensor_op_t op, uint32_t nvalues, json &data) { - return device_type::CPU; if(op == tensor_op_t::NONE) { return device_type::CPU; } @@ -77,13 +92,24 @@ void assignPlaceOnDeviceMemory(tensor_pool_t *pool, int32_t a_idx, std::vector &nodes) { +void assignBackendGraph(tensor_pool_t *pool,std::vector &nodes, backend_mode backend) { setUpParentReference(nodes); - json data = readJsonToMap("/home/wslarch/Documents/Coding/DEV/soft/soft-cuda/src/init/config/CONFIG.soft"); - for (auto node : nodes) { - assignBackend(node, data); + if (backend == backend_mode::CPU) { + for (auto node : nodes) { + assignBackendCpu(node); + } + } else if (backend == backend_mode::GPU) { + for (auto node : nodes) { + assignBackendGpu(node); + } + } else if (backend == backend_mode::HYBRID) { + json data = readJsonToMap("/home/wslarch/Documents/Coding/DEV/soft/soft-cuda/src/init/config/CONFIG.soft"); + for (auto node : nodes) { + assignBackend(node, data); + } + } else { + throw std::invalid_argument("This backend is not supported"); } - for (auto node : nodes) { if (node->backend_fn == tensor_evaluate_GPU) { // TODO: Implement contagious logic @@ -236,3 +262,23 @@ void assignGradMemory(tensor_pool_t *pool_grad_cpu, tensor_pool_t *pool_grad_gpu } } } + +// Since autograd is not ready I am making two function one for device to host copy and one which iterate over graph and transfer them back to cpu + +void fromDeviceToHost(execution_node_t *node) { + cudaMemcpy(node->t->data, node->device_ptr, node->t->nvalues * sizeof(float), cudaMemcpyDeviceToHost); + return; +} + +void autogradGpuMemTranfer(std::vector &nodes) { + for(auto node : nodes) { + if (node->backend_fn == tensor_evaluate_GPU) { + // cause we do implict fallback to cpu in case we don't have that op on gpu + if (node->t->data == NULL) { + fromDeviceToHost(node); + } + // Not doing backend_fn mutation cause then we will have to run assignBackend again and again + node->device_ptr_grad = NULL; + } + } +} diff --git a/src/core/tensor/tensor.cpp b/src/core/tensor/tensor.cpp index f90cfe9..cfdeed7 100644 --- a/src/core/tensor/tensor.cpp +++ b/src/core/tensor/tensor.cpp @@ -215,7 +215,9 @@ bool tensor_evaluate_GPU([[maybe_unused]] tensor_pool_t *pool, [[maybe_unused]]t success = tensor_mul_op_cuda(t, d_a, d_b, d_res); break; default: - assert(false); + // We are having a stopgap where if we have not written that op on GPU then CPU will handle it + std::cout << "OP NOT AVAILABLE FOR GPU SWITCHING TO CPU"; + success = tensor_evaluate(pool, t, d_a, d_b, d_res); } if (success) { debug("tensor_evaluate_GPU: success\n"); From e1eb99fe9d5a816dadcdada6dce7adc0a3f7379c Mon Sep 17 00:00:00 2001 From: Aakarsh Kashyap Date: Sat, 18 Apr 2026 03:04:16 +0530 Subject: [PATCH 17/27] Broken push don't try it it's super broken likely --- error.txt | 44 ----------- main.cpp | 90 +++++++++++------------ src/core/graph/assignBackend.cu | 20 +++++ src/core/tensor/{tensor.cpp => tensor.cu} | 13 +++- 4 files changed, 76 insertions(+), 91 deletions(-) delete mode 100644 error.txt rename src/core/tensor/{tensor.cpp => tensor.cu} (94%) diff --git a/error.txt b/error.txt deleted file mode 100644 index cd529e6..0000000 --- a/error.txt +++ /dev/null @@ -1,44 +0,0 @@ --- Configuring done (0.3s) --- Generating done (0.0s) --- Build files have been written to: /home/wslarch/Documents/Coding/DEV/soft/soft-cuda/build - -real 0m0.356s -user 0m0.157s -sys 0m0.181s -[ 5%] Built target gtest -[ 10%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/base/debug.cpp.o -[ 10%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/bias_add.cpp.o -[ 13%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/mean.cpp.o -[ 16%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/mse.cpp.o -[ 18%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/mul.cpp.o -[ 21%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/relu.cpp.o -[ 24%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/add.cpp.o -[ 29%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/backprop/backprop_b.cpp.o -[ 29%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/scalar.cpp.o -[ 32%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/square.cpp.o -[ 35%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/transpose.cpp.o -[ 37%] Building CXX object CMakeFiles/soft_lib.dir/src/backend_cpu/math/sub.cpp.o -[ 40%] Building CUDA object CMakeFiles/soft_lib.dir/src/backend_gpu/math/matmul.cu.o -[ 43%] Building CUDA object CMakeFiles/soft_lib.dir/src/backend_gpu/math/add.cu.o -[ 48%] Built target gtest_main -[ 54%] Built target gmock -[ 56%] Building CXX object CMakeFiles/soft_lib.dir/src/core/JSON/json_utils.cpp.o -[ 62%] Built target gmock_main -[ 64%] Building CXX object CMakeFiles/soft_lib.dir/src/core/graph/DAGbuild.cpp.o -[ 67%] Building CUDA object CMakeFiles/soft_lib.dir/src/core/graph/assignBackend.cu.o -[ 70%] Building CXX object CMakeFiles/soft_lib.dir/src/core/graph/train.cpp.o -[ 72%] Building CUDA object CMakeFiles/soft_lib.dir/src/core/pool/pool.cu.o -[ 75%] Building CXX object CMakeFiles/soft_lib.dir/src/core/tensor/tensor.cpp.o -/home/wslarch/Documents/Coding/DEV/soft/soft-cuda/src/core/tensor/tensor.cpp: In function ‘bool tensor_evaluate(tensor_pool_t*, tensor_t*, float*, float*, float*)’: -/home/wslarch/Documents/Coding/DEV/soft/soft-cuda/src/core/tensor/tensor.cpp:176:19: error: ‘tensor_op_square’ was not declared in this scope; did you mean ‘tensor_square’? - 176 | success = tensor_op_square(pool, t); - | ^~~~~~~~~~~~~~~~ - | tensor_square -make[2]: *** [CMakeFiles/soft_lib.dir/build.make:363: CMakeFiles/soft_lib.dir/src/core/tensor/tensor.cpp.o] Error 1 -make[2]: *** Waiting for unfinished jobs.... -make[1]: *** [CMakeFiles/Makefile2:181: CMakeFiles/soft_lib.dir/all] Error 2 -make: *** [Makefile:146: all] Error 2 - -real 0m11.154s -user 1m11.334s -sys 0m16.624s diff --git a/main.cpp b/main.cpp index e9974c4..f56d1cd 100644 --- a/main.cpp +++ b/main.cpp @@ -71,51 +71,51 @@ int main() { assignBackendGraph(pool_gpu, seq, backend_mode::HYBRID); std::cout << "TESTING OUT "; assignGradMemory(pool_grad_cpu, pool_grad_gpu, seq); - tensor_graph_forward_evaluate(pool, pool_gpu, seq); - - // if (oki) { - // cout << "=========================================== \n"; - // cout << "============= STARTING TRAINING =========== \n"; - // - // for (int i = 0; i <= 10000; i++) { - // tensor_graph_forward_evaluate(pool, pool_gpu, seq); - // autogradGpuMemTranfer(seq); - // gradInitializer(seq); - // tensor_graph_backward(seq); - // - // if (i % 1000 == 0) { - // std::cout << "EPOCH " << i << "\n"; - // execution_node_t *mse_node = seq.back(); - // printExecutionNode(mse_node); - // } - // - // tensor_sgd(seq, 0.05); - // } - // - // tensor_graph_forward_evaluate(pool, pool_gpu, seq); - // - // float* inputs = (float*)tensor_get_data(X); - // float* targets = (float*)tensor_get_data(Y); - // float* predictions = (float*)tensor_get_data(Y_pred); - // - // cout << "=========================================== \n"; - // cout << "============= XOR(4 NEURONS RESULT) ============= \n"; - // cout << "X1\tX2\t|\tTarget\t|\tPredicted\n"; - // cout << "---------------------------------------------------\n"; - // - // for (int i = 0; i < 4; i++) { - // float x1 = inputs[i * 2 + 0]; - // float x2 = inputs[i * 2 + 1]; - // float y_true = targets[i]; - // float y_pred = predictions[i]; - // - // cout << x1 << "\t" << x2 << "\t|\t" << y_true << "\t|\t" << y_pred << "\n"; - // } - // cout << "=========================================== \n"; - // - // } else { - // cout << "WARNING: DAG Verification failed. Aborting expedition.\n"; - // } + // tensor_graph_forward_evaluate(pool, pool_gpu, seq); + + if (oki) { + cout << "=========================================== \n"; + cout << "============= STARTING TRAINING =========== \n"; + + for (int i = 0; i <= 10000; i++) { + tensor_graph_forward_evaluate(pool, pool_gpu, seq); + autogradGpuMemTranfer(seq); + gradInitializer(seq); + tensor_graph_backward(seq); + + if (i % 1000 == 0) { + std::cout << "EPOCH " << i << "\n"; + execution_node_t *mse_node = seq.back(); + printExecutionNode(mse_node); + } + + tensor_sgd(seq, 0.05); + } + + tensor_graph_forward_evaluate(pool, pool_gpu, seq); + + float* inputs = (float*)tensor_get_data(X); + float* targets = (float*)tensor_get_data(Y); + float* predictions = (float*)tensor_get_data(Y_pred); + + cout << "=========================================== \n"; + cout << "============= XOR(4 NEURONS RESULT) ============= \n"; + cout << "X1\tX2\t|\tTarget\t|\tPredicted\n"; + cout << "---------------------------------------------------\n"; + + for (int i = 0; i < 4; i++) { + float x1 = inputs[i * 2 + 0]; + float x2 = inputs[i * 2 + 1]; + float y_true = targets[i]; + float y_pred = predictions[i]; + + cout << x1 << "\t" << x2 << "\t|\t" << y_true << "\t|\t" << y_pred << "\n"; + } + cout << "=========================================== \n"; + + } else { + cout << "WARNING: DAG Verification failed. Aborting expedition.\n"; + } tensor_pool_destroy(pool); tensor_pool_destroy(pool_meta); diff --git a/src/core/graph/assignBackend.cu b/src/core/graph/assignBackend.cu index 59d2b79..9a5cb6a 100644 --- a/src/core/graph/assignBackend.cu +++ b/src/core/graph/assignBackend.cu @@ -156,6 +156,7 @@ void assignBackendGraph(tensor_pool_t *pool,std::vector &nod bool tensor_graph_forward_evaluate(tensor_pool_t *pool_cpu, tensor_pool_t *pool_gpu, std::vector &nodes) { for(auto node : nodes) { +changedToGPU: if(node->backend_fn == tensor_evaluate_GPU){ // TODO: Write the memory copy for parents when child have this as well as evaluating it // NOTE: We have to check maybe parents are already on GPU @@ -198,6 +199,25 @@ bool tensor_graph_forward_evaluate(tensor_pool_t *pool_cpu, tensor_pool_t *pool_ node->t->device = device_type::GPU; // device_type device; } else if (node->backend_fn == tensor_evaluate) { + // I had forgotten to take into account that if parents are on GPU but child is assigned as CPU operation(which is unlikely but could be possible) + // We will handle it by changing the backend_fn to GPU so things work out + + int32_t a_idx = getTheExecutionNodeIndex(node, 0); + if(a_idx != -1){ + auto parent_a = nodes[(size_t)a_idx]; + if (parent_a->t->device == device_type::GPU) { + node->backend_fn = tensor_evaluate_GPU; + goto changedToGPU; + } + } + int32_t b_idx = getTheExecutionNodeIndex(node, 1); + if(b_idx != -1){ + auto parent_b = nodes[(size_t)b_idx]; + if (parent_b->t->device == device_type::CPU) { + node->backend_fn = tensor_evaluate_GPU; + goto changedToGPU; + } + } float *dummy = nullptr; (*(node->backend_fn))(pool_cpu, node->t, dummy, dummy, dummy); } diff --git a/src/core/tensor/tensor.cpp b/src/core/tensor/tensor.cu similarity index 94% rename from src/core/tensor/tensor.cpp rename to src/core/tensor/tensor.cu index cfdeed7..e439c96 100644 --- a/src/core/tensor/tensor.cpp +++ b/src/core/tensor/tensor.cu @@ -216,9 +216,18 @@ bool tensor_evaluate_GPU([[maybe_unused]] tensor_pool_t *pool, [[maybe_unused]]t break; default: // We are having a stopgap where if we have not written that op on GPU then CPU will handle it - std::cout << "OP NOT AVAILABLE FOR GPU SWITCHING TO CPU"; + std::cout << "OP NOT AVAILABLE FOR GPU SWITCHING TO CPU\n"; + if (t->a != nullptr && d_a != nullptr) { + cudaMemcpy(t->a->data, d_a, t->a->nvalues * sizeof(float), cudaMemcpyDeviceToHost); + } + if (t->b != nullptr && d_b != nullptr) { + cudaMemcpy(t->b->data, d_b, t->b->nvalues * sizeof(float), cudaMemcpyDeviceToHost); + } success = tensor_evaluate(pool, t, d_a, d_b, d_res); - } + if (success && d_res != nullptr) { + cudaMemcpy(d_res, t->data, t->nvalues * sizeof(float), cudaMemcpyHostToDevice); + } +} if (success) { debug("tensor_evaluate_GPU: success\n"); } else { From c8ec0440d4d469bb0da236f4a8175d57cf0fcfb0 Mon Sep 17 00:00:00 2001 From: Aakarsh Kashyap Date: Sat, 18 Apr 2026 21:37:52 +0530 Subject: [PATCH 18/27] CORRECTED: The backend switch and GPU CPU thrashing was resolved, it was some config and default return error --- src/core/graph/assignBackend.cu | 43 +++++++++++++++++---------------- src/init/config/CONFIG.soft | 7 +++--- 2 files changed, 25 insertions(+), 25 deletions(-) diff --git a/src/core/graph/assignBackend.cu b/src/core/graph/assignBackend.cu index 9a5cb6a..779bd58 100644 --- a/src/core/graph/assignBackend.cu +++ b/src/core/graph/assignBackend.cu @@ -59,7 +59,7 @@ device_type assignDevice([[maybe_unused]] uint8_t ndims, [[maybe_unused]] uint32 if (op == tensor_op_t::ADD) { auto params = data["ops"]["add"]; for(auto param: params) { - if(!param.contains("min") || param["min"] <= nvalues && param["max"] > nvalues) { + if(!param.contains("min") || (param["min"] <= nvalues && param["max"] > nvalues)) { if(param["backend"] == "cpu") { return device_type::CPU; } else { @@ -71,7 +71,7 @@ device_type assignDevice([[maybe_unused]] uint8_t ndims, [[maybe_unused]] uint32 } catch (const std::exception& e) { debug("Error: %s\n", e.what()); } - return device_type::GPU; + return device_type::CPU; } int32_t getTheExecutionNodeIndex(execution_node_t *node, uint32_t idx) { @@ -111,6 +111,7 @@ void assignBackendGraph(tensor_pool_t *pool,std::vector &nod throw std::invalid_argument("This backend is not supported"); } for (auto node : nodes) { +changedToGPU: if (node->backend_fn == tensor_evaluate_GPU) { // TODO: Implement contagious logic // TODO: How to get access to the execution_node place in graph when we just know the @@ -133,6 +134,25 @@ void assignBackendGraph(tensor_pool_t *pool,std::vector &nod nodes[((size_t)b_idx)]->to_device_needed = true; assignPlaceOnDeviceMemory(pool, b_idx, nodes); } + } else { + // I had forgotten to take into account that if parents are on GPU but child is assigned as CPU operation(which is unlikely but could be possible) + // We will handle it by changing the backend_fn to GPU so things work out + int32_t a_idx = getTheExecutionNodeIndex(node, 0); + if(a_idx != -1){ + auto parent_a = nodes[(size_t)a_idx]; + if (parent_a->t->device == device_type::GPU) { + node->backend_fn = tensor_evaluate_GPU; + goto changedToGPU; + } + } + int32_t b_idx = getTheExecutionNodeIndex(node, 1); + if(b_idx != -1){ + auto parent_b = nodes[(size_t)b_idx]; + if (parent_b->t->device == device_type::GPU) { + node->backend_fn = tensor_evaluate_GPU; + goto changedToGPU; + } + } } } // Yea no thrid check would be done after the eval graph had been learn so we can just see the @@ -156,7 +176,6 @@ void assignBackendGraph(tensor_pool_t *pool,std::vector &nod bool tensor_graph_forward_evaluate(tensor_pool_t *pool_cpu, tensor_pool_t *pool_gpu, std::vector &nodes) { for(auto node : nodes) { -changedToGPU: if(node->backend_fn == tensor_evaluate_GPU){ // TODO: Write the memory copy for parents when child have this as well as evaluating it // NOTE: We have to check maybe parents are already on GPU @@ -199,25 +218,7 @@ changedToGPU: node->t->device = device_type::GPU; // device_type device; } else if (node->backend_fn == tensor_evaluate) { - // I had forgotten to take into account that if parents are on GPU but child is assigned as CPU operation(which is unlikely but could be possible) - // We will handle it by changing the backend_fn to GPU so things work out - int32_t a_idx = getTheExecutionNodeIndex(node, 0); - if(a_idx != -1){ - auto parent_a = nodes[(size_t)a_idx]; - if (parent_a->t->device == device_type::GPU) { - node->backend_fn = tensor_evaluate_GPU; - goto changedToGPU; - } - } - int32_t b_idx = getTheExecutionNodeIndex(node, 1); - if(b_idx != -1){ - auto parent_b = nodes[(size_t)b_idx]; - if (parent_b->t->device == device_type::CPU) { - node->backend_fn = tensor_evaluate_GPU; - goto changedToGPU; - } - } float *dummy = nullptr; (*(node->backend_fn))(pool_cpu, node->t, dummy, dummy, dummy); } diff --git a/src/init/config/CONFIG.soft b/src/init/config/CONFIG.soft index 7dd9f44..03d98ee 100644 --- a/src/init/config/CONFIG.soft +++ b/src/init/config/CONFIG.soft @@ -33,11 +33,10 @@ } ], "add": [ - { - "backend": "cuda" - } + { "min": 0, "max": 127, "backend": "cpu" }, + { "min": 128, "max": 4096, "backend": "cuda" } ] - }, + }, "pool": { "device": "cuda", "block_size": 2097152 From 7dbe92f5de49662dcea78517cafe441d4fc74218 Mon Sep 17 00:00:00 2001 From: Aakarsh Kashyap Date: Tue, 21 Apr 2026 01:32:07 +0530 Subject: [PATCH 19/27] Some works and config fallbakc --- CMakeLists.txt | 7 +++ include/soft-cuda/tensor/api.h | 18 +++++--- include/soft-cuda/tensor/tensor.h | 31 ++++++++++++++ src/core/JSON/json_utils.cpp | 22 ++++++++++ src/core/graph/CONFIG.soft | 44 +++++++++++++++++++ src/core/graph/assignBackend.cu | 59 +++++++++++++++++++++++++- src/core/graph/saveLoad.cpp | 36 ++++++++++++++++ src/core/include/JSON/json_utils.h | 3 ++ src/core/include/graph/assignBackend.h | 2 + src/init/config/soft_init.cpp | 0 src/init/config/soft_init.h | 1 + 11 files changed, 215 insertions(+), 8 deletions(-) create mode 100644 include/soft-cuda/tensor/tensor.h create mode 100644 src/core/graph/CONFIG.soft create mode 100644 src/core/graph/saveLoad.cpp create mode 100644 src/init/config/soft_init.cpp create mode 100644 src/init/config/soft_init.h diff --git a/CMakeLists.txt b/CMakeLists.txt index 354f1d9..72e9864 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -7,6 +7,11 @@ set(CMAKE_CUDA_ARCHITECTURES native) add_library(soft_lib SHARED ${LIB_SOURCE}) +set_target_properties(soft_lib PROPERTIES + CXX_STANDARD 17 + CXX_STANDARD_REQUIRED ON + CXX_EXTENSIONS OFF +) target_include_directories(soft_lib PUBLIC "${CMAKE_CURRENT_SOURCE_DIR}/include" PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/src/backend_cpu/include" @@ -23,6 +28,8 @@ target_compile_options(soft_lib PRIVATE set_target_properties(soft_lib PROPERTIES CUDA_SEPARABLE_COMPILATION ON + CXX_STANDARD 17 + CXX_STANDARD_REQUIRED ON ) # # target_compile_options(soft_lib PUBLIC diff --git a/include/soft-cuda/tensor/api.h b/include/soft-cuda/tensor/api.h index 03a9edd..808013e 100644 --- a/include/soft-cuda/tensor/api.h +++ b/include/soft-cuda/tensor/api.h @@ -19,7 +19,7 @@ extern "C" { // DONE enum class tensor_dtype_t { UINT32_T, - INT32_T, + INT33_T, UINT64_T, INT64_T, FLOAT32_T, @@ -355,19 +355,23 @@ void printExecutionNode(execution_node_t *et); * @params graph struct for ops sequence * @return boolean status flag * */ -bool tensor_graph_forward_evaluate(tensor_pool_t *pool_cpu, tensor_pool_t *pool_gpu, std::vector &nodes); +// bool tensor_graph_forward_evaluate(tensor_pool_t *pool_cpu, tensor_pool_t *pool_gpu, std::vector &nodes); ////////////////////////////////////////////////////////////////////////// ///////////////////////////////////////////////////////////////////////// // BACKWARD PASS FUNCTIONS -void gradInitializer(std::vector &nodes); - -bool tensor_graph_backward(std::vector &nodes); +// void gradInitializer(std::vector &nodes); +// +// bool tensor_graph_backward(std::vector &nodes); +// +// void assignGradMemory(tensor_pool_t *pool_grad_cpu, tensor_pool_t *pool_grad_gpu, std::vector &nodes); +// +// void autogradGpuMemTranfer(std::vector &nodes); -void assignGradMemory(tensor_pool_t *pool_grad_cpu, tensor_pool_t *pool_grad_gpu, std::vector &nodes); -void autogradGpuMemTranfer(std::vector &nodes); +// bool save_model(const std::string& filepath, const std::vector& weights); +// bool load_model(const std::string& filepath, const std::vector& weights); #ifdef __cplusplus } diff --git a/include/soft-cuda/tensor/tensor.h b/include/soft-cuda/tensor/tensor.h new file mode 100644 index 0000000..1870a19 --- /dev/null +++ b/include/soft-cuda/tensor/tensor.h @@ -0,0 +1,31 @@ +#pragme once + +void tensor_sgd(std::vector &nodes, float learning_rate); + +bool verifyIfDAG(tensor_pool_t *pool, tensor_t *t, std::vector &seq); + +void assignBackendGraph(tensor_pool_t *pool,std::vector &nodes, backend_mode value = backend_mode::CPU); + + +void assignGradMemory(tensor_pool_t *pool_grad_cpu, tensor_pool_t *pool_grad_gpu, std::vector &nodes); + + +bool tensor_graph_forward_evaluate(tensor_pool_t *pool_cpu, tensor_pool_t *pool_gpu, std::vector &nodes); + + +void gradInitializer(std::vector &nodes); + + +bool tensor_graph_backward(std::vector &nodes); + + +void assignGradMemory(tensor_pool_t *pool_grad_cpu, tensor_pool_t *pool_grad_gpu, std::vector &nodes); + + +void autogradGpuMemTranfer(std::vector &nodes); + + +bool save_model(const std::string& filepath, const std::vector& weights); + + +bool load_model(const std::string& filepath, const std::vector& weights); diff --git a/src/core/JSON/json_utils.cpp b/src/core/JSON/json_utils.cpp index fce2b4c..78b2320 100644 --- a/src/core/JSON/json_utils.cpp +++ b/src/core/JSON/json_utils.cpp @@ -18,6 +18,28 @@ readJsonToMap(const std::string& filePath) { json j; file >> j; + // Check if JSON is object + if (!j.is_object()) { + debug("JSON root is not an object"); + } + + // Convert JSON to unordered_map + for (auto& [key, value] : j.items()) { + result[key] = value; + } + + return result; +} + +std::unordered_map +readDefaultToMap(std::string_view conf) { + + std::unordered_map result; + + // Read JSON + json j = json::parse(conf); + + // Check if JSON is object if (!j.is_object()) { debug("JSON root is not an object"); diff --git a/src/core/graph/CONFIG.soft b/src/core/graph/CONFIG.soft new file mode 100644 index 0000000..03d98ee --- /dev/null +++ b/src/core/graph/CONFIG.soft @@ -0,0 +1,44 @@ +{ + "meta": { + "soft_version": "0.1.0", + "device_hash": "3f8a9c2d7b1e6f4a8c0d123456789abcdeffedcba9876543210abcdef12345678", + "generated_at": "2026-03-18T09:00:00Z" + }, + "device": { + "type": "cuda", + "compute_capability": 8.6, + "vram_mb": 8192 + }, + "ops": { + "matmul": [ + { + "min": 0, + "max": 127, + "backend": "cpu" + }, + { + "min": 128, + "max": 511, + "backend": "cuda" + }, + { + "min": 512, + "max": 4096, + "backend": "cuda" + } + ], + "relu": [ + { + "backend": "cuda" + } + ], + "add": [ + { "min": 0, "max": 127, "backend": "cpu" }, + { "min": 128, "max": 4096, "backend": "cuda" } + ] + }, + "pool": { + "device": "cuda", + "block_size": 2097152 + } +} diff --git a/src/core/graph/assignBackend.cu b/src/core/graph/assignBackend.cu index 779bd58..8ac18b8 100644 --- a/src/core/graph/assignBackend.cu +++ b/src/core/graph/assignBackend.cu @@ -1,4 +1,5 @@ #include "internal_header.h" +#include // #include "vector" #include #include @@ -92,6 +93,53 @@ void assignPlaceOnDeviceMemory(tensor_pool_t *pool, int32_t a_idx, std::vector &nodes, backend_mode backend) { setUpParentReference(nodes); if (backend == backend_mode::CPU) { @@ -103,7 +151,16 @@ void assignBackendGraph(tensor_pool_t *pool,std::vector &nod assignBackendGpu(node); } } else if (backend == backend_mode::HYBRID) { - json data = readJsonToMap("/home/wslarch/Documents/Coding/DEV/soft/soft-cuda/src/init/config/CONFIG.soft"); + json data{}; + std::string path{}; + if (getenv("HOME") != nullptr) { + static std::string path = std::string(getenv("HOME")) + "/.config/soft-cuda/CONFIG.soft"; + } + if (std::filesystem::exists(path)) { + data = readJsonToMap(path); // passing the char pointer + } else { + data = readDefaultToMap(conf); + } for (auto node : nodes) { assignBackend(node, data); } diff --git a/src/core/graph/saveLoad.cpp b/src/core/graph/saveLoad.cpp new file mode 100644 index 0000000..a9d2743 --- /dev/null +++ b/src/core/graph/saveLoad.cpp @@ -0,0 +1,36 @@ +#include "internal_header.h" + +#include +#include +#include + +bool save_model(const std::string& filepath, const std::vector& weights) { + std::ofstream out(filepath, std::ios::binary); + if (!out.is_open()) return false; + for (auto t : weights) { + uint32_t num_elements = 1; + uint8_t ndims = tensor_get_ndims(t); + uint32_t* dims = tensor_get_dims(t); + for (int i = 0; i < ndims; i++) { + num_elements *= dims[i]; + } + out.write(reinterpret_cast(tensor_get_data(t)), num_elements * sizeof(float)); + } + return true; +} + +bool load_model(const std::string& filepath, const std::vector& weights) { + std::ifstream in(filepath, std::ios::binary); + if (!in.is_open()) return false; + for (auto t : weights) { + uint32_t num_elements = 1; + uint8_t ndims = tensor_get_ndims(t); + uint32_t* dims = tensor_get_dims(t); + for (int i = 0; i < ndims; i++) { + num_elements *= dims[i]; + } + in.read(reinterpret_cast(tensor_get_data(t)), num_elements * sizeof(float)); + } + return true; +} + diff --git a/src/core/include/JSON/json_utils.h b/src/core/include/JSON/json_utils.h index 14e4ee3..24a02e9 100644 --- a/src/core/include/JSON/json_utils.h +++ b/src/core/include/JSON/json_utils.h @@ -4,3 +4,6 @@ #include "internal_header.h" std::unordered_map readJsonToMap(const std::string& filePath); + +std::unordered_map +readDefaultToMap(std::string_view conf); diff --git a/src/core/include/graph/assignBackend.h b/src/core/include/graph/assignBackend.h index 91f8d3f..b880e7f 100644 --- a/src/core/include/graph/assignBackend.h +++ b/src/core/include/graph/assignBackend.h @@ -2,8 +2,10 @@ #include #include "../third_party/json.hpp" + using json = nlohmann::json; + void assignBackend(execution_node_t *e); device_type assignDevice(uint8_t ndims, uint32_t *dims, tensor_op_t op, uint32_t nvalues, json &data); diff --git a/src/init/config/soft_init.cpp b/src/init/config/soft_init.cpp new file mode 100644 index 0000000..e69de29 diff --git a/src/init/config/soft_init.h b/src/init/config/soft_init.h new file mode 100644 index 0000000..3ecf8ea --- /dev/null +++ b/src/init/config/soft_init.h @@ -0,0 +1 @@ +#embed "CONFIG.soft" From aab6ec98bf1c6b8c3e8db7984dc40b62dad16269 Mon Sep 17 00:00:00 2001 From: Aakarsh Kashyap <24f2000831@ds.study.iitm.ac.in> Date: Sun, 19 Apr 2026 22:13:50 +0530 Subject: [PATCH 20/27] Feat/backend dispatch mode (#35) * Added time field in make.sh * chore/ Running summarizer * Completed MSE, passed smoke tests Written SUB, SQUARE, MEAN * chore/ Running summarizer * made assign grad memory function * chore/ Running summarizer * chore/ Running summarizer * chore/ Running summarizer * Somewhat work not tested * Implemented brackprop_b.cpp and added switch statements for op handling * Corrected the enum missnaming mistake * BROKEN : THE FUCKING NN IS NOT LEARNING WRRRRRYYYYYYYYY FUCK * BROKEN 2 : THE FUCKING NN IS NOT LEARNING WRRRRRYYYYYYYYY FUCK * Working operations there is still problem with routing but will resolve that in some time Auto grad almost complete * FAAAAAAAAAAAAAAAAAAAAAAAA: I HAVE COMPLETED THE AUTOGRAD FAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AUTOGRAD WORKS 4NN XOR WORKS AND LIFE IS BEST * GPU fallback is working but i feel something is wrong with hybrid path will need to see * Broken push don't try it it's super broken likely * CORRECTED: The backend switch and GPU CPU thrashing was resolved, it was some config and default return error --------- Co-authored-by: Aakarsh Kashyap --- include/soft-cuda/tensor/api.h | 17 +++++++---------- src/core/graph/assignBackend.cu | 3 +++ src/core/include/graph/assignBackend.h | 1 - 3 files changed, 10 insertions(+), 11 deletions(-) diff --git a/include/soft-cuda/tensor/api.h b/include/soft-cuda/tensor/api.h index 808013e..9f80f6f 100644 --- a/include/soft-cuda/tensor/api.h +++ b/include/soft-cuda/tensor/api.h @@ -355,23 +355,20 @@ void printExecutionNode(execution_node_t *et); * @params graph struct for ops sequence * @return boolean status flag * */ -// bool tensor_graph_forward_evaluate(tensor_pool_t *pool_cpu, tensor_pool_t *pool_gpu, std::vector &nodes); +bool tensor_graph_forward_evaluate(tensor_pool_t *pool_cpu, tensor_pool_t *pool_gpu, std::vector &nodes); ////////////////////////////////////////////////////////////////////////// ///////////////////////////////////////////////////////////////////////// // BACKWARD PASS FUNCTIONS -// void gradInitializer(std::vector &nodes); -// -// bool tensor_graph_backward(std::vector &nodes); -// -// void assignGradMemory(tensor_pool_t *pool_grad_cpu, tensor_pool_t *pool_grad_gpu, std::vector &nodes); -// -// void autogradGpuMemTranfer(std::vector &nodes); +void gradInitializer(std::vector &nodes); + +bool tensor_graph_backward(std::vector &nodes); +void autogradGpuMemTranfer(std::vector &nodes); -// bool save_model(const std::string& filepath, const std::vector& weights); -// bool load_model(const std::string& filepath, const std::vector& weights); +bool save_model(const std::string& filepath, const std::vector& weights); +bool load_model(const std::string& filepath, const std::vector& weights); #ifdef __cplusplus } diff --git a/src/core/graph/assignBackend.cu b/src/core/graph/assignBackend.cu index 8ac18b8..d0a0f49 100644 --- a/src/core/graph/assignBackend.cu +++ b/src/core/graph/assignBackend.cu @@ -93,6 +93,7 @@ void assignPlaceOnDeviceMemory(tensor_pool_t *pool, int32_t a_idx, std::vector>>>>>> 5155f61 (Feat/backend dispatch mode (#35)) void assignBackendGraph(tensor_pool_t *pool,std::vector &nodes, backend_mode backend) { setUpParentReference(nodes); if (backend == backend_mode::CPU) { diff --git a/src/core/include/graph/assignBackend.h b/src/core/include/graph/assignBackend.h index b880e7f..2547549 100644 --- a/src/core/include/graph/assignBackend.h +++ b/src/core/include/graph/assignBackend.h @@ -2,7 +2,6 @@ #include #include "../third_party/json.hpp" - using json = nlohmann::json; From c38517222e03d0e9404b3b8f2bdf2f17e0c0db5d Mon Sep 17 00:00:00 2001 From: Souls Syntax Nix Date: Mon, 20 Apr 2026 16:48:41 +0530 Subject: [PATCH 21/27] R --- include/soft-cuda/tensor/api.h | 111 +++++++++++++++++++-------------- 1 file changed, 64 insertions(+), 47 deletions(-) diff --git a/include/soft-cuda/tensor/api.h b/include/soft-cuda/tensor/api.h index 9f80f6f..1394417 100644 --- a/include/soft-cuda/tensor/api.h +++ b/include/soft-cuda/tensor/api.h @@ -46,15 +46,6 @@ typedef struct tensor_pool_instance tensor_pool_t; typedef struct execution_node execution_node_t; -/******************************************************************************* - * !!!!!!!! DISCARDED !!!!!!!! - * Discarding this in favour of vector of execution_node - * - * Opaque graph - * typedef struct tensor_graph_instance tensor_graph_t; - * ***************************************************************************** - */ -typedef struct tensor_graph_instance tensor_graph_t; /////////////////////////////////////////////// // RETURN TENSOR INFORMATION @@ -132,33 +123,24 @@ size_t tensor_pool_size(tensor_pool_t *pool); // DONE size_t tensor_pool_used(tensor_pool_t *pool); -//////////////////////////////////////////////////////////////////////////////////////////// -// WILL LIKELY BE DEPRECEATED /* - * Move the memory pool between device_type + * Move the node from device to host * - * @param device Name of the device to move the pool to. GPU/CPU @param t The - * tensor to move - * @pool Pool where the tensor will be allocated + * @param node The node which we want to move from device to host * */ - -// DEPRECEATED -// bool tensor_move_device(tensor_t *t, device_type target_device, tensor_pool_t *pool); -// NEW IMPLEMENTATION - // DONE bool execution_node_to_host(execution_node_t *node); -//////////////////////////////////////////////////////////////////////////////////////////// + /* * Fetches the data of the tensor. - * @return Returns a void pointer to the array + * @return Returns a void pointer to the data array * */ // DONE void *tensor_get_data(tensor_t *t); /* - * @return returns the dimension of the tensor + * @return the dimension of the tensor * */ // DONE uint8_t tensor_get_ndims(tensor_t *t); @@ -188,10 +170,20 @@ void tensor_print_data(tensor_t *t); * @Note tensor_matmul expects B to be transposed and * contiguous. Call tensor_transpose(B) first. * */ + // DONE tensor_t *tensor_mul(tensor_pool_t *pool, tensor_t *x, tensor_t *y); +/* + * @param pool Takes the tensor pool. + * @param x The tensor we want to square + * + * @return Squared tensor object + * */ + +// DONE tensor_t *tensor_square(tensor_pool_t *pool, tensor_t *x); + /* The naive version of matrix multiplication * @param pool Pointer to the tensor pool * @param x Pointer to the tensor which will be mulptiplied @@ -209,6 +201,7 @@ tensor_t *tensor_mul_naive(tensor_pool_t *pool, tensor_t *x, tensor_t *y); * * @return Returns a tensor object with operation set. * */ + // DONE tensor_t *tensor_transpose(tensor_pool_t *pool, tensor_t *a); @@ -220,6 +213,7 @@ tensor_t *tensor_transpose(tensor_pool_t *pool, tensor_t *a); * * @return Returns a tensor object with operation set. * */ + // DONE tensor_t *tensor_add(tensor_pool_t *pool, tensor_t *x, tensor_t *y); @@ -237,19 +231,6 @@ tensor_t *tensor_add_bias(tensor_pool_t *pool, const tensor_t *xw, const tensor_ * */ // DONE tensor_t *tensor_sub(tensor_pool_t *pool, tensor_t *a, tensor_t *b); -///////////////////////////////////////////////////////////// -/// DEPRECEATED tensor_mul operation handles it automatically -// /* -// * Do scalar matrix multiplication -// * @param out Pointer to the tensor where result will be stored -// * @param x Pointer to the tensor which will be mulptiplied -// * @param y Scalar with which the matrix will be multiplied -// * -// * */ -// // void tensor_scalar_mul(tensor_t *out, tensor_t *x, double y); -// -// upon request it can be exposed seperately -////////////////////////////////////////////////////////////// // The activation function // @params out Output tensor @@ -258,39 +239,60 @@ tensor_t *tensor_sub(tensor_pool_t *pool, tensor_t *a, tensor_t *b); // DONE tensor_t *tensor_relu(tensor_pool_t *pool, tensor_t *a); + +// Calculates the mean +// @params pool Tensor pool +// @a Tensor whose mean we want to find +// +// @return Returns the mean of the tensor // DONE tensor_t *tensor_mean(tensor_pool_t *pool, tensor_t *a); + // Compares result // return how correct we were b/w 0-1 + +// DONE tensor_t *tensor_mse_loss(tensor_pool_t *pool, tensor_t *predictions, tensor_t *target); -// // Fills an existing tensor with normally distributed random numbers. +// Fills an existing tensor with normally distributed random numbers. +// @params t Tensor we want to fill with random no. +// @params mean The mean around which we want to have the value +// @params std_dev Standard deviation + +// DONE bool tensor_fill_random_normal(tensor_t *t, float mean, float std_dev); -// + + +// TODO: HAVE TO DO THIS IF WE WANT CLASSIFICATION // // Fused operation combining Softmax and Cross-Entropy for stability // tensor_t *tensor_cross_entropy_loss(tensor_pool_t *pool, const tensor_t *predictions, // const tensor_t *targets); // *********************************************************************************** -// TODO: HAVE TO UPDATED FUNC SIG SPEC // Evalutes the operation(Forward) with depth=1 // @return boolean flag for status +// +// @note This is atomic operations and is used internally +// is for CPU computation // DONE bool tensor_evaluate( tensor_pool_t *pool,tensor_t *t, float *d_a = nullptr, float *d_b = nullptr, float *d_res= nullptr); + +// Is for GPU computation // DONE bool tensor_evaluate_GPU( tensor_pool_t *pool,tensor_t *t, float *d_a, float *d_b, float *d_res); -///////////////////////////////////////////////////////////// -// NO DESIGNING DONE HENCE NOT RECOMMENDED TO WORK AROUND USE IT JUST AS PLACEHOLDER BUT BE READY TO -// UPDATE API below it is still unstable ////////////////////// // THE BACKWARD PASS // Evalutes the operation(BACKWARD) with depth=1 // @return boolean flag for status +// @note Inplictly handle the GPU CPU transfer is a single operation bool tensor_backward(tensor_pool_t *pool, tensor_t *t); // The Optimizer +// @params nodes The graph +// @learning_rate The size of step +// void tensor_sgd(std::vector &nodes, float learning_rate); ///////////////////////////////////////////////////////////// // GRAPH OPERATIONS @@ -310,7 +312,7 @@ void tensor_sgd(std::vector &nodes, float learning_rate); ///////////////////////////////////////////////////////////////// -/* !!!!! SIGNATURE WAS CHANGED !!!!! +/* * Topologically sort the tensors, detect dependency and return a vector of . * * @params pool The data pool where the execution nodes will be @@ -327,7 +329,7 @@ bool verifyIfDAG(tensor_pool_t *pool, tensor_t *t, std::vector &nodes, backend_mode value = backend_mode::CPU); +void assignBackendGraph(tensor_pool_t *pool_gpu,std::vector &nodes, backend_mode value = backend_mode::CPU); +/* This function is used for assigining temporary storage on GPU VRAM for GPU -> GPU part so we don't have + * transfer overhead and thrashing + * + * @params pool_grad_cpu Pool for CPU grad allocation + * @params pool_grad_gpu Pool for GPU grad allocation + * + * @params nodes The graph + * */ +//DONE void assignGradMemory(tensor_pool_t *pool_grad_cpu, tensor_pool_t *pool_grad_gpu, std::vector &nodes); + + /* @params Take execution_node_t which you wanna know * @returns the postion of the execution_node_t after verifyIfDAG * */ @@ -345,10 +358,11 @@ void assignGradMemory(tensor_pool_t *pool_grad_cpu, tensor_pool_t *pool_grad_gpu int32_t getPosOfNode(execution_node_t *et); // Prints all data about execution_node_t as well as tensor data +// internally calls tensor print hence prints tensor data too + // DONE void printExecutionNode(execution_node_t *et); -// Previous SIGNATURE -// bool tensor_graph_build(tensor_graph_t *g, tensor_t *t); + /* * Evaluate the whole graph forward operation. @@ -361,10 +375,13 @@ bool tensor_graph_forward_evaluate(tensor_pool_t *pool_cpu, tensor_pool_t *pool_ ///////////////////////////////////////////////////////////////////////// // BACKWARD PASS FUNCTIONS +// Takes nodes and then sets the grads data to 0 using memset void gradInitializer(std::vector &nodes); +// Walk over the graph(nodes) and then run autograd bool tensor_graph_backward(std::vector &nodes); +// Used to transfer the gradient data from device to host void autogradGpuMemTranfer(std::vector &nodes); bool save_model(const std::string& filepath, const std::vector& weights); From dcc6c978039872797311a343ff20d623b73c1477 Mon Sep 17 00:00:00 2001 From: souls-syntax Date: Tue, 21 Apr 2026 01:59:05 +0530 Subject: [PATCH 22/27] feat(python): Add flat-C Python bridge layer (sc_bridge) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Introduces include/soft-cuda/python/ with five headers: - soft_cuda_python.h : master include - tensor_pool.h : sc_pool_* arena wrappers - tensor_core.h : sc_tensor_* lifecycle + SC_DTYPE_* constants - tensor_ops.h : sc_tensor_mul/add/relu/... + sc_tensor_evaluate* - tensor_graph.h : sc_graph_t opaque handle, Layer 1 + Layer 2 API - tensor_io.h : sc_save_model / sc_load_model Adds src/python/sc_bridge.cpp — single C++ TU that implements all sc_* symbols with extern C linkage, wrapping std::vector via placement-new in sc_graph_t so Python ctypes/cffi never sees any C++ types. Updates CMakeLists.txt: - Excludes src/python/ from the core GLOB (prevents duplicate symbols) - Adds soft_cuda_python SHARED target linked against soft_lib --- CMakeLists.txt | 27 ++ include/soft-cuda/python/soft_cuda_python.h | 43 +++ include/soft-cuda/python/tensor_core.h | 108 ++++++ include/soft-cuda/python/tensor_graph.h | 205 +++++++++++ include/soft-cuda/python/tensor_io.h | 53 +++ include/soft-cuda/python/tensor_ops.h | 102 ++++++ include/soft-cuda/python/tensor_pool.h | 75 +++++ src/python/sc_bridge.cpp | 356 ++++++++++++++++++++ 8 files changed, 969 insertions(+) create mode 100644 include/soft-cuda/python/soft_cuda_python.h create mode 100644 include/soft-cuda/python/tensor_core.h create mode 100644 include/soft-cuda/python/tensor_graph.h create mode 100644 include/soft-cuda/python/tensor_io.h create mode 100644 include/soft-cuda/python/tensor_ops.h create mode 100644 include/soft-cuda/python/tensor_pool.h create mode 100644 src/python/sc_bridge.cpp diff --git a/CMakeLists.txt b/CMakeLists.txt index 72e9864..693b0d6 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -1,7 +1,9 @@ cmake_minimum_required(VERSION 3.16) project(soft-cuda LANGUAGES CXX CUDA) +# Core library sources — explicitly exclude the Python bridge subdir file(GLOB_RECURSE LIB_SOURCE src/*.cu src/*.cpp src/*.hpp src/*.h) +list(FILTER LIB_SOURCE EXCLUDE REGEX ".*/python/.*") set(CMAKE_CUDA_ARCHITECTURES native) @@ -44,5 +46,30 @@ add_executable(soft main.cpp) target_link_libraries(soft PRIVATE soft_lib) +# ───────────────────────────────────────────────────────────────────────────── +# Python C bridge — flat-C shared library for use with ctypes / cffi +# ───────────────────────────────────────────────────────────────────────────── +add_library(soft_cuda_python SHARED src/python/sc_bridge.cpp) + +set_target_properties(soft_cuda_python PROPERTIES + CXX_STANDARD 17 + CXX_STANDARD_REQUIRED ON + CXX_EXTENSIONS OFF +) + +target_include_directories(soft_cuda_python + PUBLIC "${CMAKE_CURRENT_SOURCE_DIR}/include" + PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/src/backend_cpu/include" + PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/src/core/include" + PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/src" +) + +target_link_libraries(soft_cuda_python PRIVATE soft_lib) + +# Ensure all sc_* symbols are exported on Linux/macOS +if(NOT WIN32) + target_compile_options(soft_cuda_python PRIVATE -fvisibility=default) +endif() + enable_testing() add_subdirectory(tests) diff --git a/include/soft-cuda/python/soft_cuda_python.h b/include/soft-cuda/python/soft_cuda_python.h new file mode 100644 index 0000000..6ce212f --- /dev/null +++ b/include/soft-cuda/python/soft_cuda_python.h @@ -0,0 +1,43 @@ +/** + * @file soft_cuda_python.h + * @brief Master include for the soft-cuda Python/ctypes bridge layer. + * + * Include only this header in your Python binding or cffi cdef block. + * It pulls in the five sub-headers in dependency order: + * + * tensor_pool.h — arena management (create, destroy, zero, alloc, size, used) + * tensor_core.h — tensor lifecycle (create, id, get_data, get_ndims, get_dims, + * print_data, fill_random_normal) + * tensor_ops.h — forward ops + evaluate + * (mul, mul_naive, add, add_bias, sub, relu, + * mean, mse_loss, square, transpose, evaluate, evaluate_gpu) + * tensor_graph.h — graph building + training primitives + * Layer 1: verify_dag, assign_backend, assign_grad_memory, + * graph_forward, autograd_gpu_transfer, + * grad_initializer, backward, sgd, node_to_host + * Layer 2: build_graph, graph_step, graph_get_loss, graph_size + * tensor_io.h — persistence (save_model, load_model) + * + * All symbols use the "sc_" prefix and expose only C-safe types so that + * Python ctypes / cffi can parse and call them directly. + * + * Shared library: link against libsoft_cuda_python.so (or .dll on Windows). + * + * Quick Python example: + * @code + * import ctypes, numpy as np + * lib = ctypes.CDLL("./libsoft_cuda_python.so") + * + * lib.sc_pool_create.restype = ctypes.c_void_p + * lib.sc_pool_create.argtypes = [ctypes.c_size_t, ctypes.c_int] + * pool = lib.sc_pool_create(4 * 1024 * 1024, 0) # 4 MB CPU pool + * @endcode + */ + +#pragma once + +#include "tensor_pool.h" +#include "tensor_core.h" +#include "tensor_ops.h" +#include "tensor_graph.h" +#include "tensor_io.h" diff --git a/include/soft-cuda/python/tensor_core.h b/include/soft-cuda/python/tensor_core.h new file mode 100644 index 0000000..23f04b1 --- /dev/null +++ b/include/soft-cuda/python/tensor_core.h @@ -0,0 +1,108 @@ +/** + * @file tensor_core.h + * @brief Flat-C tensor lifecycle API — Python/ctypes compatible. + * + * Uses plain int dtype constants instead of enum class tensor_dtype_t + * so Python ctypes can pass them directly. + * Include via soft_cuda_python.h — do not include directly. + */ + +#pragma once + +#include +#include "tensor_pool.h" + +#ifdef __cplusplus +extern "C" { +#endif + +/* Opaque tensor handle. */ +typedef struct tensor_instance sc_tensor_t; + +/* ----------------------------------------------------------------------- + * Data-type constants (mirrors tensor_dtype_t enum class values) + * ----------------------------------------------------------------------- */ +#define SC_DTYPE_UINT32 0 +#define SC_DTYPE_INT32 1 +#define SC_DTYPE_UINT64 2 +#define SC_DTYPE_INT64 3 +#define SC_DTYPE_FLOAT32 4 +#define SC_DTYPE_FLOAT64 5 + +/* ----------------------------------------------------------------------- + * Tensor lifecycle + * ----------------------------------------------------------------------- */ + +/** + * Create a tensor. + * + * @param pool Memory pool to allocate from. + * @param dtype One of the SC_DTYPE_* constants above. + * @param num_dims Rank of the tensor (number of dimensions). + * @param dims Array of length num_dims containing each dimension size. + * @param elems Initial data buffer, or NULL for a zero-initialised tensor. + * The buffer is copied; the caller retains ownership. + * @param grad Non-zero → autograd will track this tensor. + * @return New tensor pointer, or NULL on allocation failure. + */ +sc_tensor_t *sc_tensor_create(sc_pool_t *pool, + int dtype, + uint32_t num_dims, + uint32_t *dims, + void *elems, + int grad); + +/** + * Return the unique identifier for a tensor. + * IDs are unique within the same pool. + * + * @param t The tensor. + * @return Unique uint32 id. + */ +uint32_t sc_tensor_id(sc_tensor_t *t); + +/** + * Fetch a raw pointer to the tensor's data buffer. + * + * @param t The tensor. + * @return void* to the underlying data array. + */ +void *sc_tensor_get_data(sc_tensor_t *t); + +/** + * Return the rank (number of dimensions) of the tensor. + * + * @param t The tensor. + * @return Number of dimensions (0 = scalar). + */ +uint8_t sc_tensor_get_ndims(sc_tensor_t *t); + +/** + * Return a pointer to the dimension array. + * The array has sc_tensor_get_ndims(t) valid entries. + * + * @param t The tensor. + * @return Pointer to dims[TENSOR_MAX_DIMS+1]. + */ +uint32_t *sc_tensor_get_dims(sc_tensor_t *t); + +/** + * Print tensor data to stdout (mirrors tensor_print_data). + * + * @param t The tensor to print. + */ +void sc_tensor_print_data(sc_tensor_t *t); + +/** + * Fill an existing tensor with normally distributed random floats. + * + * @param t Tensor to fill (must be FLOAT32). + * @param mean Distribution mean. + * @param std_dev Standard deviation. + * @return Non-zero on success. + */ +int sc_tensor_fill_random_normal(sc_tensor_t *t, float mean, float std_dev); + +#ifdef __cplusplus +} /* extern "C" */ +#endif diff --git a/include/soft-cuda/python/tensor_graph.h b/include/soft-cuda/python/tensor_graph.h new file mode 100644 index 0000000..d53166f --- /dev/null +++ b/include/soft-cuda/python/tensor_graph.h @@ -0,0 +1,205 @@ +/** + * @file tensor_graph.h + * @brief Flat-C graph building and training primitives — Python/ctypes compatible. + * + * Two-layer API: + * Layer 1 — low-level primitives that mirror the internal C++ graph functions. + * Layer 2 — high-level convenience wrappers for typical Python usage. + * + * Key design: sc_graph_t is an opaque handle that wraps + * std::vector so Python never sees C++ containers. + * + * Include via soft_cuda_python.h — do not include directly. + */ + +#pragma once + +#include +#include +#include "tensor_pool.h" +#include "tensor_core.h" + +#ifdef __cplusplus +extern "C" { +#endif + +/* ----------------------------------------------------------------------- + * Backend mode constants (mirrors backend_mode enum class) + * ----------------------------------------------------------------------- */ +#define SC_BACKEND_GPU 0 +#define SC_BACKEND_CPU 1 +#define SC_BACKEND_HYBRID 2 + +/* ----------------------------------------------------------------------- + * Opaque graph handle + * Internally holds a std::vector allocated on the heap. + * ----------------------------------------------------------------------- */ +typedef struct sc_graph sc_graph_t; + +/* ----------------------------------------------------------------------- + * Layer 1 — low-level graph primitives + * ----------------------------------------------------------------------- */ + +/** + * Allocate a new, empty graph handle. + * Must be freed with sc_graph_destroy when no longer needed. + * + * @return New sc_graph_t*, or NULL on allocation failure. + */ +sc_graph_t *sc_graph_create(void); + +/** + * Free the graph handle and the internal node vector. + * Does NOT free the tensors or pools themselves. + * + * @param g Graph to destroy. + */ +void sc_graph_destroy(sc_graph_t *g); + +/** + * Topologically sort the tensor computation graph starting from tensor `t` + * and populate the graph handle with execution nodes. + * Detects cycles (returns 0 if a cycle is found). + * + * @param meta_pool Pool used to allocate execution_node_t objects. + * Recommended: a dedicated metadata pool. + * @param t Root tensor (e.g. loss node). + * @param g Graph handle to populate. + * @return Non-zero on success (valid DAG), 0 if cycle detected. + */ +int sc_verify_dag(sc_pool_t *meta_pool, sc_tensor_t *t, sc_graph_t *g); + +/** + * Assign a backend (CPU / GPU / HYBRID) to each node in the graph. + * For GPU nodes, allocates device memory in pool_gpu. + * + * @param pool_gpu GPU VRAM pool for device memory allocation. + * @param g Populated graph handle. + * @param mode One of SC_BACKEND_CPU / SC_BACKEND_GPU / SC_BACKEND_HYBRID. + */ +void sc_assign_backend(sc_pool_t *pool_gpu, sc_graph_t *g, int mode); + +/** + * Allocate gradient memory for each trainable node. + * CPU grad tensors go into pool_grad_cpu; GPU grad buffers into pool_grad_gpu. + * + * @param pool_grad_cpu CPU pool for gradient tensors. + * @param pool_grad_gpu GPU pool for gradient device buffers. + * @param g Graph handle. + */ +void sc_assign_grad_memory(sc_pool_t *pool_grad_cpu, + sc_pool_t *pool_grad_gpu, + sc_graph_t *g); + +/** + * Run the full forward pass over the graph (all nodes, in topological order). + * Handles CPU↔GPU data transfer as needed by the assigned backends. + * + * @param pool_cpu CPU pool (for CPU-side evaluate calls). + * @param pool_gpu GPU pool (metadata, used by GPU evaluate). + * @param g Graph handle. + * @return Non-zero on success. + */ +int sc_graph_forward(sc_pool_t *pool_cpu, sc_pool_t *pool_gpu, sc_graph_t *g); + +/** + * Copy GPU gradient data back to CPU host memory after the forward pass. + * Must be called before sc_backward if any nodes ran on GPU. + * + * @param g Graph handle. + */ +void sc_autograd_gpu_transfer(sc_graph_t *g); + +/** + * Zero out all gradient buffers in the graph (call at the start of each step). + * + * @param g Graph handle. + */ +void sc_grad_initializer(sc_graph_t *g); + +/** + * Run backward pass (autograd) over the entire graph. + * + * @param g Graph handle. + * @return Non-zero on success. + */ +int sc_backward(sc_graph_t *g); + +/** + * Stochastic gradient descent update for all leaf (op == NONE) trainable tensors. + * + * @param g Graph handle. + * @param learning_rate Step size for parameter update. + */ +void sc_sgd(sc_graph_t *g, float learning_rate); + +/** + * Copy the data of a specific node from device (GPU) to host (CPU). + * Useful to read a specific intermediate result after the forward pass. + * + * @param g Graph handle. + * @param node_idx Index of the node in the topological order (0-based). + * @return Non-zero on success. + */ +int sc_node_to_host(sc_graph_t *g, size_t node_idx); + +/* ----------------------------------------------------------------------- + * Layer 2 — high-level convenience API + * ----------------------------------------------------------------------- */ + +/** + * Build a ready-to-train graph from a loss tensor in one call. + * Internally calls sc_verify_dag → sc_assign_backend → sc_assign_grad_memory. + * + * Caller must still supply pool_grad_cpu and pool_grad_gpu for gradient + * memory (they are passed through to sc_assign_grad_memory). + * + * @param meta_pool Pool for execution_node_t allocation. + * @param pool_gpu GPU VRAM pool (device data + grad buffers). + * @param pool_grad_cpu CPU pool for gradient tensors. + * @param pool_grad_gpu GPU pool for gradient device buffers. + * @param loss Root loss tensor (usually the output of sc_tensor_mse_loss). + * @param backend_mode One of SC_BACKEND_* constants. + * @return Ready sc_graph_t*, or NULL on failure. + */ +sc_graph_t *sc_build_graph(sc_pool_t *meta_pool, + sc_pool_t *pool_gpu, + sc_pool_t *pool_grad_cpu, + sc_pool_t *pool_grad_gpu, + sc_tensor_t *loss, + int backend_mode); + +/** + * Execute a single full training step: + * forward → autograd_gpu_transfer → grad_initializer → backward → sgd + * + * @param pool_cpu CPU execution pool. + * @param pool_gpu GPU execution pool. + * @param g Graph handle. + * @param learning_rate SGD step size. + */ +void sc_graph_step(sc_pool_t *pool_cpu, + sc_pool_t *pool_gpu, + sc_graph_t *g, + float learning_rate); + +/** + * Read the scalar loss value from the last node in the graph. + * Returns NaN if the last node is not a scalar or has not been evaluated. + * + * @param g Graph handle. + * @return float loss value. + */ +float sc_graph_get_loss(sc_graph_t *g); + +/** + * Return the number of nodes in the graph (useful for iteration / debugging). + * + * @param g Graph handle. + * @return Node count. + */ +size_t sc_graph_size(sc_graph_t *g); + +#ifdef __cplusplus +} /* extern "C" */ +#endif diff --git a/include/soft-cuda/python/tensor_io.h b/include/soft-cuda/python/tensor_io.h new file mode 100644 index 0000000..892d34e --- /dev/null +++ b/include/soft-cuda/python/tensor_io.h @@ -0,0 +1,53 @@ +/** + * @file tensor_io.h + * @brief Flat-C model persistence API — Python/ctypes compatible. + * + * Wraps the C++ save_model / load_model (from src/core/graph/saveLoad.cpp) + * using only C-safe primitives (const char* path, pointer array + count). + * + * Include via soft_cuda_python.h — do not include directly. + */ + +#pragma once + +#include +#include "tensor_core.h" + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * Save model weights to a binary file. + * + * Writes all tensor data sequentially as raw float32 bytes. + * Tensors must already have been evaluated (data populated) before saving. + * + * @param path Null-terminated file path. + * @param tensors Array of tensor pointers to save (parameter tensors). + * @param count Number of tensors in the array. + * @return Non-zero on success, 0 on failure (e.g. file not writable). + * + * @note Only the raw float data is saved — shape/dtype metadata is NOT + * persisted. The caller must reconstruct tensors of the correct + * shape before calling sc_load_model. + */ +int sc_save_model(const char *path, sc_tensor_t **tensors, size_t count); + +/** + * Load model weights from a binary file into pre-allocated tensors. + * + * Reads raw float32 bytes and fills each tensor in-order. + * Tensors must already be created with the correct shapes before loading. + * + * @param path Null-terminated file path. + * @param tensors Array of pre-allocated tensor pointers to fill. + * @param count Number of tensors in the array. + * @return Non-zero on success, 0 on failure (e.g. file not found, + * size mismatch). + */ +int sc_load_model(const char *path, sc_tensor_t **tensors, size_t count); + +#ifdef __cplusplus +} /* extern "C" */ +#endif diff --git a/include/soft-cuda/python/tensor_ops.h b/include/soft-cuda/python/tensor_ops.h new file mode 100644 index 0000000..13c753a --- /dev/null +++ b/include/soft-cuda/python/tensor_ops.h @@ -0,0 +1,102 @@ +/** + * @file tensor_ops.h + * @brief Flat-C forward ops + evaluate — Python/ctypes compatible. + * + * Every function returns a new sc_tensor_t* allocated in the given pool + * (lazy / define-graph style), except sc_tensor_evaluate* which trigger + * the actual computation for a single node. + * Include via soft_cuda_python.h — do not include directly. + */ + +#pragma once + +#include "tensor_pool.h" +#include "tensor_core.h" + +#ifdef __cplusplus +extern "C" { +#endif + +/* ----------------------------------------------------------------------- + * Matrix / element-wise ops (all lazy — return a new op-node tensor) + * ----------------------------------------------------------------------- */ + +/** Cache-optimised matrix multiplication (calls tensor_mul internally). */ +sc_tensor_t *sc_tensor_mul(sc_pool_t *pool, sc_tensor_t *x, sc_tensor_t *y); + +/** Naive O(n³) matrix multiplication (calls tensor_mul_naive). */ +sc_tensor_t *sc_tensor_mul_naive(sc_pool_t *pool, sc_tensor_t *x, sc_tensor_t *y); + +/** Element-wise / matrix addition. */ +sc_tensor_t *sc_tensor_add(sc_pool_t *pool, sc_tensor_t *x, sc_tensor_t *y); + +/** + * Explicit bias-broadcast addition (e.g., Y = XW + b where b has fewer dims). + * Calls tensor_add_bias internally. + */ +sc_tensor_t *sc_tensor_add_bias(sc_pool_t *pool, sc_tensor_t *xw, sc_tensor_t *bias); + +/** Element-wise / matrix subtraction. */ +sc_tensor_t *sc_tensor_sub(sc_pool_t *pool, sc_tensor_t *a, sc_tensor_t *b); + +/** ReLU activation. */ +sc_tensor_t *sc_tensor_relu(sc_pool_t *pool, sc_tensor_t *a); + +/** Scalar mean of all elements. Returns a rank-0 (scalar) tensor. */ +sc_tensor_t *sc_tensor_mean(sc_pool_t *pool, sc_tensor_t *a); + +/** + * Mean-squared-error loss. + * + * @param predictions Model output tensor. + * @param target Ground-truth tensor (same shape as predictions). + * @return Scalar MSE tensor. + */ +sc_tensor_t *sc_tensor_mse_loss(sc_pool_t *pool, + sc_tensor_t *predictions, + sc_tensor_t *target); + +/** Element-wise square. */ +sc_tensor_t *sc_tensor_square(sc_pool_t *pool, sc_tensor_t *x); + +/** + * Transpose a 2-D matrix. + * The returned tensor shares no memory with the input; it is a new node. + */ +sc_tensor_t *sc_tensor_transpose(sc_pool_t *pool, sc_tensor_t *a); + +/* ----------------------------------------------------------------------- + * Evaluate (eager execution for a single node) + * ----------------------------------------------------------------------- */ + +/** + * Evaluate a single tensor node on the CPU. + * Internally calls tensor_evaluate(pool, t, NULL, NULL, NULL). + * + * @param pool CPU memory pool. + * @param t Tensor node to evaluate. + * @return Non-zero on success. + */ +int sc_tensor_evaluate(sc_pool_t *pool, sc_tensor_t *t); + +/** + * Evaluate a single tensor node on the GPU. + * Caller is responsible for providing device pointers to parent data + * (d_a, d_b) and a device pointer for the output (d_res). + * + * @param pool CPU pool (used for metadata). + * @param t Tensor node to evaluate. + * @param d_a GPU pointer to operand A data (float*), or NULL. + * @param d_b GPU pointer to operand B data (float*), or NULL. + * @param d_res GPU pointer to output buffer (float*). + * @return Non-zero on success. + */ +int sc_tensor_evaluate_gpu(sc_pool_t *pool, + sc_tensor_t *t, + float *d_a, + float *d_b, + float *d_res); + +#ifdef __cplusplus +} /* extern "C" */ +#endif diff --git a/include/soft-cuda/python/tensor_pool.h b/include/soft-cuda/python/tensor_pool.h new file mode 100644 index 0000000..c70e0a4 --- /dev/null +++ b/include/soft-cuda/python/tensor_pool.h @@ -0,0 +1,75 @@ +/** + * @file tensor_pool.h + * @brief Flat-C arena management API — Python/ctypes compatible. + * + * All functions use only C-safe primitives: void*, size_t, int. + * This header is part of the soft_cuda_python bridge layer. + * Include via soft_cuda_python.h — do not include directly. + */ + +#pragma once + +#include +#include + +#ifdef __cplusplus +extern "C" { +#endif + +/* Opaque alias so C and Python both see the same type name. */ +typedef struct tensor_pool_instance sc_pool_t; + +/** + * Create a new memory arena. + * + * @param capacity_bytes Total bytes to pre-allocate. + * @param on_device Non-zero → allocate on GPU VRAM; zero → CPU RAM. + * @return Pointer to the new pool, or NULL on failure. + */ +sc_pool_t *sc_pool_create(size_t capacity_bytes, int on_device); + +/** + * Completely destroy the pool and return memory to the system. + * + * @param pool Pool to destroy. Must not be used after this call. + */ +void sc_pool_destroy(sc_pool_t *pool); + +/** + * Reset the bump pointer to zero, invalidating all tensors in the pool + * without releasing the underlying memory block. + * Highly efficient for resetting temporary/gradient pools each step. + * + * @param pool Pool to reset. + */ +void sc_pool_zero(sc_pool_t *pool); + +/** + * Allocate raw bytes from the pool. + * + * @param pool The pool to allocate from. + * @param size Number of bytes to allocate. + * @param out_id Output: unique id assigned to this allocation. + * @return Pointer to the allocated block, or NULL if exhausted. + */ +void *sc_pool_alloc(sc_pool_t *pool, size_t size, uint32_t *out_id); + +/** + * Return the total capacity of the pool in bytes. + * + * @param pool The pool to query. + * @return Capacity in bytes. + */ +size_t sc_pool_size(sc_pool_t *pool); + +/** + * Return the number of bytes currently in use. + * + * @param pool The pool to query. + * @return Bytes consumed so far. + */ +size_t sc_pool_used(sc_pool_t *pool); + +#ifdef __cplusplus +} /* extern "C" */ +#endif diff --git a/src/python/sc_bridge.cpp b/src/python/sc_bridge.cpp new file mode 100644 index 0000000..beeb2f8 --- /dev/null +++ b/src/python/sc_bridge.cpp @@ -0,0 +1,356 @@ +/** + * @file sc_bridge.cpp + * @brief Implementation of the soft-cuda Python C bridge. + * + * Compiled as C++ (so it can call the internal C++ API that uses + * std::vector, enum class, etc.), but all exported symbols have C linkage + * so Python ctypes / cffi can dlopen and call them by name. + * + * Build as a shared library: + * cmake --build build --target soft_cuda_python + * + * Every exported function is declared in: + * include/soft-cuda/python/soft_cuda_python.h + * + * The internal C++ API lives in: + * include/soft-cuda/tensor/api.h (referenced via internal_header.h) + */ + +// Pull in everything: public + private headers & all internal APIs +#include "../include/soft-cuda/tensor/api.h" +#include "../include/soft-cuda/python/soft_cuda_python.h" + +#include +#include /* memcpy */ +#include /* NAN */ +#include +#include /* malloc/free */ +#include + +/* ======================================================================== + * sc_graph_t — opaque handle wrapping std::vector + * ======================================================================== + * We heap-allocate the struct itself via malloc so it has a stable address + * and no C++ constructors are visible at the ABI boundary. + */ +struct sc_graph { + std::vector nodes; +}; + +/* ======================================================================== + * Helper: cast int backend constant → backend_mode enum class + * ======================================================================== */ +static backend_mode to_backend_mode(int mode) { + switch (mode) { + case 0: return backend_mode::GPU; + case 2: return backend_mode::HYBRID; + default: return backend_mode::CPU; + } +} + +/* ======================================================================== + * Helper: cast int dtype constant → tensor_dtype_t enum class + * ======================================================================== */ +static tensor_dtype_t to_tensor_dtype(int d) { + switch (d) { + case 0: return tensor_dtype_t::UINT32_T; + case 1: return tensor_dtype_t::INT32_T; + case 2: return tensor_dtype_t::UINT64_T; + case 3: return tensor_dtype_t::INT64_T; + case 5: return tensor_dtype_t::FLOAT64_T; + default: return tensor_dtype_t::FLOAT32_T; /* SC_DTYPE_FLOAT32 = 4 */ + } +} + +/* ------------------------------------------------------------------------ */ +/* Pool wrappers */ +/* ------------------------------------------------------------------------ */ + +extern "C" sc_pool_t *sc_pool_create(size_t capacity_bytes, int on_device) { + return tensor_pool_create(capacity_bytes, (bool)on_device); +} + +extern "C" void sc_pool_destroy(sc_pool_t *pool) { + tensor_pool_destroy(pool); +} + +extern "C" void sc_pool_zero(sc_pool_t *pool) { + tensor_pool_zero(pool); +} + +extern "C" void *sc_pool_alloc(sc_pool_t *pool, size_t size, uint32_t *out_id) { + return tensor_pool_alloc(pool, size, out_id); +} + +extern "C" size_t sc_pool_size(sc_pool_t *pool) { + return tensor_pool_size(pool); +} + +extern "C" size_t sc_pool_used(sc_pool_t *pool) { + return tensor_pool_used(pool); +} + +/* ------------------------------------------------------------------------ */ +/* Tensor core wrappers */ +/* ------------------------------------------------------------------------ */ + +extern "C" sc_tensor_t *sc_tensor_create(sc_pool_t *pool, + int dtype, + uint32_t num_dims, + uint32_t *dims, + void *elems, + int grad) { + return tensor_create(pool, + to_tensor_dtype(dtype), + num_dims, + dims, + elems, + (bool)grad); +} + +extern "C" uint32_t sc_tensor_id(sc_tensor_t *t) { + return tensor_id(t); +} + +extern "C" void *sc_tensor_get_data(sc_tensor_t *t) { + return tensor_get_data(t); +} + +extern "C" uint8_t sc_tensor_get_ndims(sc_tensor_t *t) { + return tensor_get_ndims(t); +} + +extern "C" uint32_t *sc_tensor_get_dims(sc_tensor_t *t) { + return tensor_get_dims(t); +} + +extern "C" void sc_tensor_print_data(sc_tensor_t *t) { + tensor_print_data(t); +} + +extern "C" int sc_tensor_fill_random_normal(sc_tensor_t *t, + float mean, + float std_dev) { + return (int)tensor_fill_random_normal(t, mean, std_dev); +} + +/* ------------------------------------------------------------------------ */ +/* Op wrappers */ +/* ------------------------------------------------------------------------ */ + +extern "C" sc_tensor_t *sc_tensor_mul(sc_pool_t *pool, + sc_tensor_t *x, + sc_tensor_t *y) { + return tensor_mul(pool, x, y); +} + +extern "C" sc_tensor_t *sc_tensor_mul_naive(sc_pool_t *pool, + sc_tensor_t *x, + sc_tensor_t *y) { + return tensor_mul_naive(pool, x, y); +} + +extern "C" sc_tensor_t *sc_tensor_add(sc_pool_t *pool, + sc_tensor_t *x, + sc_tensor_t *y) { + return tensor_add(pool, x, y); +} + +extern "C" sc_tensor_t *sc_tensor_add_bias(sc_pool_t *pool, + sc_tensor_t *xw, + sc_tensor_t *bias) { + return tensor_add_bias(pool, xw, bias); +} + +extern "C" sc_tensor_t *sc_tensor_sub(sc_pool_t *pool, + sc_tensor_t *a, + sc_tensor_t *b) { + return tensor_sub(pool, a, b); +} + +extern "C" sc_tensor_t *sc_tensor_relu(sc_pool_t *pool, sc_tensor_t *a) { + return tensor_relu(pool, a); +} + +extern "C" sc_tensor_t *sc_tensor_mean(sc_pool_t *pool, sc_tensor_t *a) { + return tensor_mean(pool, a); +} + +extern "C" sc_tensor_t *sc_tensor_mse_loss(sc_pool_t *pool, + sc_tensor_t *predictions, + sc_tensor_t *target) { + return tensor_mse_loss(pool, predictions, target); +} + +extern "C" sc_tensor_t *sc_tensor_square(sc_pool_t *pool, sc_tensor_t *x) { + return tensor_square(pool, x); +} + +extern "C" sc_tensor_t *sc_tensor_transpose(sc_pool_t *pool, sc_tensor_t *a) { + return tensor_transpose(pool, a); +} + +extern "C" int sc_tensor_evaluate(sc_pool_t *pool, sc_tensor_t *t) { + /* Pass null device pointers — CPU path */ + return (int)tensor_evaluate(pool, t, nullptr, nullptr, nullptr); +} + +extern "C" int sc_tensor_evaluate_gpu(sc_pool_t *pool, + sc_tensor_t *t, + float *d_a, + float *d_b, + float *d_res) { + return (int)tensor_evaluate_GPU(pool, t, d_a, d_b, d_res); +} + +/* ------------------------------------------------------------------------ */ +/* Graph — Layer 1 */ +/* ------------------------------------------------------------------------ */ + +extern "C" sc_graph_t *sc_graph_create(void) { + sc_graph_t *g = (sc_graph_t *)malloc(sizeof(sc_graph_t)); + if (g == nullptr) return nullptr; + /* Placement-new to properly construct the std::vector */ + new (&g->nodes) std::vector(); + return g; +} + +extern "C" void sc_graph_destroy(sc_graph_t *g) { + if (g == nullptr) return; + /* Explicitly destroy the vector before freeing the raw memory */ + g->nodes.~vector(); + free(g); +} + +extern "C" int sc_verify_dag(sc_pool_t *meta_pool, + sc_tensor_t *t, + sc_graph_t *g) { + if (g == nullptr) return 0; + return (int)verifyIfDAG(meta_pool, t, g->nodes); +} + +extern "C" void sc_assign_backend(sc_pool_t *pool_gpu, + sc_graph_t *g, + int mode) { + if (g == nullptr) return; + assignBackendGraph(pool_gpu, g->nodes, to_backend_mode(mode)); +} + +extern "C" void sc_assign_grad_memory(sc_pool_t *pool_grad_cpu, + sc_pool_t *pool_grad_gpu, + sc_graph_t *g) { + if (g == nullptr) return; + assignGradMemory(pool_grad_cpu, pool_grad_gpu, g->nodes); +} + +extern "C" int sc_graph_forward(sc_pool_t *pool_cpu, + sc_pool_t *pool_gpu, + sc_graph_t *g) { + if (g == nullptr) return 0; + return (int)tensor_graph_forward_evaluate(pool_cpu, pool_gpu, g->nodes); +} + +extern "C" void sc_autograd_gpu_transfer(sc_graph_t *g) { + if (g == nullptr) return; + autogradGpuMemTranfer(g->nodes); +} + +extern "C" void sc_grad_initializer(sc_graph_t *g) { + if (g == nullptr) return; + gradInitializer(g->nodes); +} + +extern "C" int sc_backward(sc_graph_t *g) { + if (g == nullptr) return 0; + return (int)tensor_graph_backward(g->nodes); +} + +extern "C" void sc_sgd(sc_graph_t *g, float learning_rate) { + if (g == nullptr) return; + tensor_sgd(g->nodes, learning_rate); +} + +extern "C" int sc_node_to_host(sc_graph_t *g, size_t node_idx) { + if (g == nullptr || node_idx >= g->nodes.size()) return 0; + return (int)execution_node_to_host(g->nodes[node_idx]); +} + +/* ------------------------------------------------------------------------ */ +/* Graph — Layer 2 (convenience wrappers) */ +/* ------------------------------------------------------------------------ */ + +extern "C" sc_graph_t *sc_build_graph(sc_pool_t *meta_pool, + sc_pool_t *pool_gpu, + sc_pool_t *pool_grad_cpu, + sc_pool_t *pool_grad_gpu, + sc_tensor_t *loss, + int backend_mode_val) { + sc_graph_t *g = sc_graph_create(); + if (g == nullptr) return nullptr; + + if (!verifyIfDAG(meta_pool, loss, g->nodes)) { + sc_graph_destroy(g); + return nullptr; + } + + assignBackendGraph(pool_gpu, g->nodes, to_backend_mode(backend_mode_val)); + assignGradMemory(pool_grad_cpu, pool_grad_gpu, g->nodes); + + return g; +} + +extern "C" void sc_graph_step(sc_pool_t *pool_cpu, + sc_pool_t *pool_gpu, + sc_graph_t *g, + float learning_rate) { + if (g == nullptr) return; + tensor_graph_forward_evaluate(pool_cpu, pool_gpu, g->nodes); + autogradGpuMemTranfer(g->nodes); + gradInitializer(g->nodes); + tensor_graph_backward(g->nodes); + tensor_sgd(g->nodes, learning_rate); +} + +extern "C" float sc_graph_get_loss(sc_graph_t *g) { + if (g == nullptr || g->nodes.empty()) return NAN; + + execution_node_t *last = g->nodes.back(); + if (last == nullptr || last->t == nullptr) return NAN; + + /* Bring the loss node back to host if it ran on GPU */ + if (last->t->device == device_type::GPU) { + execution_node_to_host(last); + } + + if (last->t->data == nullptr) return NAN; + + /* Loss node must be a scalar (1 element) */ + float val; + memcpy(&val, last->t->data, sizeof(float)); + return val; +} + +extern "C" size_t sc_graph_size(sc_graph_t *g) { + if (g == nullptr) return 0; + return g->nodes.size(); +} + +/* ------------------------------------------------------------------------ */ +/* IO wrappers */ +/* ------------------------------------------------------------------------ */ + +extern "C" int sc_save_model(const char *path, + sc_tensor_t **tensors, + size_t count) { + if (path == nullptr || tensors == nullptr) return 0; + std::vector vec(tensors, tensors + count); + return (int)save_model(std::string(path), vec); +} + +extern "C" int sc_load_model(const char *path, + sc_tensor_t **tensors, + size_t count) { + if (path == nullptr || tensors == nullptr) return 0; + std::vector vec(tensors, tensors + count); + return (int)load_model(std::string(path), vec); +} From 68010213de7e371e5f34ce63a901fefff15b78a8 Mon Sep 17 00:00:00 2001 From: souls-syntax Date: Tue, 21 Apr 2026 02:00:13 +0530 Subject: [PATCH 23/27] fix: Remove stray conflict markers from assignBackend.cu --- src/core/graph/assignBackend.cu | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/src/core/graph/assignBackend.cu b/src/core/graph/assignBackend.cu index d0a0f49..c4b8c41 100644 --- a/src/core/graph/assignBackend.cu +++ b/src/core/graph/assignBackend.cu @@ -93,7 +93,6 @@ void assignPlaceOnDeviceMemory(tensor_pool_t *pool, int32_t a_idx, std::vector>>>>>> 5155f61 (Feat/backend dispatch mode (#35)) + void assignBackendGraph(tensor_pool_t *pool,std::vector &nodes, backend_mode backend) { setUpParentReference(nodes); if (backend == backend_mode::CPU) { From c2187c6a45abc36784ffe6df50fc2a8d4332eab7 Mon Sep 17 00:00:00 2001 From: souls-syntax Date: Tue, 21 Apr 2026 02:05:53 +0530 Subject: [PATCH 24/27] docs: Rewrite main.cpp as sc_* API tutorial, add PYTHON_BRIDGE.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit main.cpp: - Complete rewrite using only sc_* bridge functions - 9 numbered sections: pools, data, weights, lazy ops, build_graph, training loop, inference, save/load, cleanup - Pure C (no iostream, no std::vector visible) - Demonstrates both Layer 2 convenience API (sc_build_graph, sc_graph_step) and data readback (sc_tensor_get_data) docs/PYTHON_BRIDGE.md: - Architecture diagram (Python → sc_bridge → soft_lib) - Build instructions (cmake targets) - Full API reference for all 5 headers with function signatures - Complete Python ctypes example (XOR training from Python) - Memory model explanation (arena/bump allocation) - Design decision rationale (why placement-new, why SC_DTYPE, etc.) --- docs/PYTHON_BRIDGE.md | 554 ++++++++++++++++++++++++++++++++++++++++++ main.cpp | 415 +++++++++++++++++++++++-------- 2 files changed, 866 insertions(+), 103 deletions(-) create mode 100644 docs/PYTHON_BRIDGE.md diff --git a/docs/PYTHON_BRIDGE.md b/docs/PYTHON_BRIDGE.md new file mode 100644 index 0000000..0cd87a7 --- /dev/null +++ b/docs/PYTHON_BRIDGE.md @@ -0,0 +1,554 @@ +# soft-cuda Python Bridge — API Reference + +> **`include/soft-cuda/python/soft_cuda_python.h`** — the only header you need. +> **`libsoft_cuda_python.so`** (or `.dll`) — the shared library to load from Python. + +The Python bridge wraps the internal C++ tensor library behind a flat-C +(`extern "C"`) interface. Every symbol uses the **`sc_`** prefix and +only C-safe primitives — no `std::vector`, no `enum class`, no default +arguments. This makes it directly callable from Python via `ctypes` or +`cffi`. + +--- + +## Table of Contents + +- [Architecture](#architecture) +- [Building](#building) +- [Headers at a Glance](#headers-at-a-glance) +- [API Reference](#api-reference) + - [Pool Management — `tensor_pool.h`](#pool-management) + - [Tensor Lifecycle — `tensor_core.h`](#tensor-lifecycle) + - [Forward Ops — `tensor_ops.h`](#forward-ops) + - [Graph & Training — `tensor_graph.h`](#graph--training) + - [Persistence — `tensor_io.h`](#persistence) +- [Python ctypes Example](#python-ctypes-example) +- [C/C++ Usage Example](#cc-usage-example) +- [Memory Model](#memory-model) +- [Design Decisions](#design-decisions) + +--- + +## Architecture + +``` +┌────────────────────────────────────────────────────────────────────────┐ +│ Python (ctypes.CDLL / cffi) │ +│ │ +│ import ctypes │ +│ lib = ctypes.CDLL("./libsoft_cuda_python.so") │ +│ pool = lib.sc_pool_create(4*1024*1024, 0) │ +│ │ +├────────────────────────── extern "C" ABI ──────────────────────────────┤ +│ │ +│ sc_bridge.cpp │ +│ ┌──────────────────────────────────────────────────────────────────┐ │ +│ │ sc_pool_create() → tensor_pool_create() │ │ +│ │ sc_tensor_create() → tensor_create() + enum cast │ │ +│ │ sc_tensor_mul() → tensor_mul() │ │ +│ │ sc_build_graph() → verifyIfDAG + assignBackendGraph + ... │ │ +│ │ sc_graph_step() → forward + gpu_transfer + grad + bwd + sgd│ │ +│ │ sc_graph_t → wraps std::vector │ │ +│ └──────────────────────────────────────────────────────────────────┘ │ +│ │ +├────────────────────────── C++ internals ───────────────────────────────┤ +│ │ +│ soft_lib (shared library) │ +│ ┌───────────────────────────────────────────────────────────────────┐ │ +│ │ tensor.h, pool.h, DAGbuild.cpp, assignBackend.cu, train.cpp ... │ │ +│ │ backend_cpu/*, backend_gpu/*, saveLoad.cpp │ │ +│ └───────────────────────────────────────────────────────────────────┘ │ +│ │ +└────────────────────────────────────────────────────────────────────────┘ +``` + +--- + +## Building + +```bash +# Configure (once) +cmake -B build -DCMAKE_BUILD_TYPE=Release + +# Build everything (main executable + python bridge) +cmake --build build + +# Build only the Python bridge shared library +cmake --build build --target soft_cuda_python +``` + +The output is: + +| Target | File | Description | +|---|---|---| +| `soft` | `build/soft` | Main executable (demo) | +| `soft_lib` | `build/libsoft_lib.so` | Core C++ tensor library | +| `soft_cuda_python` | `build/libsoft_cuda_python.so` | Python-compatible bridge | + +--- + +## Headers at a Glance + +``` +include/soft-cuda/python/ +├── soft_cuda_python.h ← master include (just #includes the five below) +│ +├── tensor_pool.h ← arena management +│ sc_pool_create, sc_pool_destroy, sc_pool_zero, +│ sc_pool_alloc, sc_pool_size, sc_pool_used +│ +├── tensor_core.h ← tensor lifecycle + dtype constants +│ SC_DTYPE_FLOAT32, SC_DTYPE_INT32, ... +│ sc_tensor_create, sc_tensor_id, sc_tensor_get_data, +│ sc_tensor_get_ndims, sc_tensor_get_dims, +│ sc_tensor_print_data, sc_tensor_fill_random_normal +│ +├── tensor_ops.h ← forward ops + evaluate +│ sc_tensor_mul, sc_tensor_mul_naive, +│ sc_tensor_add, sc_tensor_add_bias, sc_tensor_sub, +│ sc_tensor_relu, sc_tensor_mean, sc_tensor_mse_loss, +│ sc_tensor_square, sc_tensor_transpose, +│ sc_tensor_evaluate, sc_tensor_evaluate_gpu +│ +├── tensor_graph.h ← graph building + training +│ (Layer 1) sc_graph_create, sc_graph_destroy, +│ sc_verify_dag, sc_assign_backend, +│ sc_assign_grad_memory, sc_graph_forward, +│ sc_autograd_gpu_transfer, +│ sc_grad_initializer, sc_backward, +│ sc_sgd, sc_node_to_host +│ (Layer 2) sc_build_graph, sc_graph_step, +│ sc_graph_get_loss, sc_graph_size +│ +└── tensor_io.h ← persistence + sc_save_model, sc_load_model +``` + +--- + +## API Reference + +### Pool Management + +> Defined in `tensor_pool.h` + +```c +/* Opaque type — never dereference, just pass the pointer around. */ +typedef struct tensor_pool_instance sc_pool_t; + +/* Create a new arena. on_device=1 → GPU VRAM, on_device=0 → CPU RAM. */ +sc_pool_t *sc_pool_create(size_t capacity_bytes, int on_device); + +/* Completely free the arena and return memory to the OS. */ +void sc_pool_destroy(sc_pool_t *pool); + +/* Reset bump pointer to zero (invalidates all tensors, keeps memory). */ +void sc_pool_zero(sc_pool_t *pool); + +/* Raw allocation from the pool. Returns NULL if exhausted. */ +void *sc_pool_alloc(sc_pool_t *pool, size_t size, uint32_t *out_id); + +/* Query capacity and usage. */ +size_t sc_pool_size(sc_pool_t *pool); +size_t sc_pool_used(sc_pool_t *pool); +``` + +> [!TIP] +> Use **separate pools** for data, metadata, GPU memory, and gradients. +> This lets you `sc_pool_zero(pool_grad)` between epochs without +> affecting your weights. + +--- + +### Tensor Lifecycle + +> Defined in `tensor_core.h` + +#### Data-type Constants + +```c +#define SC_DTYPE_UINT32 0 +#define SC_DTYPE_INT32 1 +#define SC_DTYPE_UINT64 2 +#define SC_DTYPE_INT64 3 +#define SC_DTYPE_FLOAT32 4 /* ← the one you'll use 99% of the time */ +#define SC_DTYPE_FLOAT64 5 +``` + +#### Functions + +```c +typedef struct tensor_instance sc_tensor_t; + +/* Create a tensor. Pass NULL for elems to get zero-initialised data. + grad = 1 → autograd will track this tensor (for weights). */ +sc_tensor_t *sc_tensor_create(sc_pool_t *pool, int dtype, + uint32_t num_dims, uint32_t *dims, + void *elems, int grad); + +uint32_t sc_tensor_id(sc_tensor_t *t); +void *sc_tensor_get_data(sc_tensor_t *t); +uint8_t sc_tensor_get_ndims(sc_tensor_t *t); +uint32_t *sc_tensor_get_dims(sc_tensor_t *t); +void sc_tensor_print_data(sc_tensor_t *t); +int sc_tensor_fill_random_normal(sc_tensor_t *t, float mean, float std_dev); +``` + +> [!IMPORTANT] +> Set `grad = 0` for input data and targets. +> Set `grad = 1` for trainable parameters (weights, biases). + +--- + +### Forward Ops + +> Defined in `tensor_ops.h` + +All ops are **lazy** — they don't compute anything. They create a new +tensor node that records the operation and its operands. Actual +computation happens when the graph's `forward` is called. + +```c +/* Arithmetic */ +sc_tensor_t *sc_tensor_mul(sc_pool_t *pool, sc_tensor_t *x, sc_tensor_t *y); +sc_tensor_t *sc_tensor_mul_naive(sc_pool_t *pool, sc_tensor_t *x, sc_tensor_t *y); +sc_tensor_t *sc_tensor_add(sc_pool_t *pool, sc_tensor_t *x, sc_tensor_t *y); +sc_tensor_t *sc_tensor_add_bias(sc_pool_t *pool, sc_tensor_t *xw, sc_tensor_t *bias); +sc_tensor_t *sc_tensor_sub(sc_pool_t *pool, sc_tensor_t *a, sc_tensor_t *b); +sc_tensor_t *sc_tensor_square(sc_pool_t *pool, sc_tensor_t *x); +sc_tensor_t *sc_tensor_transpose(sc_pool_t *pool, sc_tensor_t *a); + +/* Activation */ +sc_tensor_t *sc_tensor_relu(sc_pool_t *pool, sc_tensor_t *a); + +/* Reduction */ +sc_tensor_t *sc_tensor_mean(sc_pool_t *pool, sc_tensor_t *a); + +/* Loss */ +sc_tensor_t *sc_tensor_mse_loss(sc_pool_t *pool, + sc_tensor_t *predictions, + sc_tensor_t *target); + +/* Single-node evaluate (for debugging — you don't need these normally) */ +int sc_tensor_evaluate(sc_pool_t *pool, sc_tensor_t *t); +int sc_tensor_evaluate_gpu(sc_pool_t *pool, sc_tensor_t *t, + float *d_a, float *d_b, float *d_res); +``` + +> [!NOTE] +> `sc_tensor_mul` internally transposes B and does a cache-optimised +> matmul. `sc_tensor_mul_naive` is the straightforward O(n³) version, +> safer for small matrices (e.g. XOR). + +--- + +### Graph & Training + +> Defined in `tensor_graph.h` + +#### Backend Constants + +```c +#define SC_BACKEND_GPU 0 +#define SC_BACKEND_CPU 1 +#define SC_BACKEND_HYBRID 2 /* auto-dispatch per CONFIG.soft */ +``` + +#### Opaque Graph Handle + +```c +typedef struct sc_graph sc_graph_t; +``` + +This hides `std::vector` from Python/C. + +#### Layer 1 — Low-level Primitives + +```c +sc_graph_t *sc_graph_create(void); +void sc_graph_destroy(sc_graph_t *g); + +int sc_verify_dag(sc_pool_t *meta_pool, sc_tensor_t *t, sc_graph_t *g); +void sc_assign_backend(sc_pool_t *pool_gpu, sc_graph_t *g, int mode); +void sc_assign_grad_memory(sc_pool_t *pool_grad_cpu, + sc_pool_t *pool_grad_gpu, + sc_graph_t *g); + +int sc_graph_forward(sc_pool_t *pool_cpu, sc_pool_t *pool_gpu, sc_graph_t *g); +void sc_autograd_gpu_transfer(sc_graph_t *g); +void sc_grad_initializer(sc_graph_t *g); +int sc_backward(sc_graph_t *g); +void sc_sgd(sc_graph_t *g, float learning_rate); +int sc_node_to_host(sc_graph_t *g, size_t node_idx); +``` + +#### Layer 2 — Convenience API + +```c +/* One-liner: verify DAG → assign backend → alloc grads. Returns ready graph. */ +sc_graph_t *sc_build_graph(sc_pool_t *meta_pool, + sc_pool_t *pool_gpu, + sc_pool_t *pool_grad_cpu, + sc_pool_t *pool_grad_gpu, + sc_tensor_t *loss, + int backend_mode); + +/* One-liner: forward → gpu_transfer → zero_grads → backward → sgd. */ +void sc_graph_step(sc_pool_t *pool_cpu, + sc_pool_t *pool_gpu, + sc_graph_t *g, + float learning_rate); + +/* Read the scalar loss value (auto-transfers from GPU if needed). */ +float sc_graph_get_loss(sc_graph_t *g); + +/* Number of nodes in the topological order. */ +size_t sc_graph_size(sc_graph_t *g); +``` + +> [!TIP] +> **For Python users**, the Layer 2 API is almost always what you want. +> Two function calls cover the entire training loop: +> ```python +> graph = lib.sc_build_graph(meta, gpu, gcpu, ggpu, loss, SC_BACKEND_CPU) +> for epoch in range(10000): +> lib.sc_graph_step(pool, gpu, graph, ctypes.c_float(0.05)) +> ``` + +--- + +### Persistence + +> Defined in `tensor_io.h` + +```c +/* Saves raw float32 data sequentially. No shape metadata stored. */ +int sc_save_model(const char *path, sc_tensor_t **tensors, size_t count); + +/* Loads raw float32 data into pre-allocated tensors (shapes must match). */ +int sc_load_model(const char *path, sc_tensor_t **tensors, size_t count); +``` + +> [!WARNING] +> Only the raw float bytes are persisted — **not** the tensor shapes or +> dtypes. You must recreate tensors with the correct shapes before +> calling `sc_load_model`. + +--- + +## Python ctypes Example + +```python +#!/usr/bin/env python3 +"""XOR training using soft-cuda from Python via ctypes.""" + +import ctypes +import os + +# ── Load the shared library ──────────────────────────────────── +lib = ctypes.CDLL("./build/libsoft_cuda_python.so") + +# ── Declare return types (ctypes defaults to c_int otherwise) ── +lib.sc_pool_create.restype = ctypes.c_void_p +lib.sc_tensor_create.restype = ctypes.c_void_p +lib.sc_tensor_get_data.restype = ctypes.c_void_p +lib.sc_tensor_get_dims.restype = ctypes.POINTER(ctypes.c_uint32) +lib.sc_tensor_mul_naive.restype = ctypes.c_void_p +lib.sc_tensor_add.restype = ctypes.c_void_p +lib.sc_tensor_relu.restype = ctypes.c_void_p +lib.sc_tensor_sub.restype = ctypes.c_void_p +lib.sc_tensor_square.restype = ctypes.c_void_p +lib.sc_tensor_mean.restype = ctypes.c_void_p +lib.sc_build_graph.restype = ctypes.c_void_p +lib.sc_graph_get_loss.restype = ctypes.c_float +lib.sc_graph_size.restype = ctypes.c_size_t + +# ── Constants ────────────────────────────────────────────────── +SC_DTYPE_FLOAT32 = 4 +SC_BACKEND_CPU = 1 + +# ── Helper to create a uint32 array ─────────────────────────── +def dims(*d): + return (ctypes.c_uint32 * len(d))(*d) + +def floats(*f): + return (ctypes.c_float * len(f))(*f) + +# ── 1. Create pools ─────────────────────────────────────────── +MB = 1024 * 1024 +pool = lib.sc_pool_create(MB, 0) +meta = lib.sc_pool_create(MB, 0) +gcpu = lib.sc_pool_create(MB, 0) +gpu = lib.sc_pool_create(MB, 1) +ggpu = lib.sc_pool_create(MB, 1) + +# ── 2. Create tensors ───────────────────────────────────────── +X = lib.sc_tensor_create(pool, SC_DTYPE_FLOAT32, 2, dims(4,2), + floats(0,0, 0,1, 1,0, 1,1), 0) +Y = lib.sc_tensor_create(pool, SC_DTYPE_FLOAT32, 2, dims(4,1), + floats(0, 1, 1, 0), 0) + +W1 = lib.sc_tensor_create(pool, SC_DTYPE_FLOAT32, 2, dims(2,4), None, 1) +b1 = lib.sc_tensor_create(pool, SC_DTYPE_FLOAT32, 2, dims(1,4), None, 1) +W2 = lib.sc_tensor_create(pool, SC_DTYPE_FLOAT32, 2, dims(4,1), None, 1) +b2 = lib.sc_tensor_create(pool, SC_DTYPE_FLOAT32, 2, dims(1,1), None, 1) + +lib.sc_tensor_fill_random_normal(W1, ctypes.c_float(0.5), ctypes.c_float(0.2)) +lib.sc_tensor_fill_random_normal(W2, ctypes.c_float(0.5), ctypes.c_float(0.2)) +lib.sc_tensor_fill_random_normal(b1, ctypes.c_float(0.0), ctypes.c_float(0.1)) +lib.sc_tensor_fill_random_normal(b2, ctypes.c_float(0.0), ctypes.c_float(0.1)) + +# ── 3. Define computation graph ─────────────────────────────── +H = lib.sc_tensor_relu(pool, lib.sc_tensor_add(pool, + lib.sc_tensor_mul_naive(pool, X, W1), b1)) +Y_pred = lib.sc_tensor_add(pool, lib.sc_tensor_mul_naive(pool, H, W2), b2) +diff = lib.sc_tensor_sub(pool, Y_pred, Y) +loss = lib.sc_tensor_mean(pool, lib.sc_tensor_square(pool, diff)) + +# ── 4. Build graph (one call) ───────────────────────────────── +graph = lib.sc_build_graph(meta, gpu, gcpu, ggpu, loss, SC_BACKEND_CPU) +assert graph is not None +print(f"Graph built: {lib.sc_graph_size(graph)} nodes") + +# ── 5. Train ────────────────────────────────────────────────── +for epoch in range(10001): + lib.sc_graph_step(pool, gpu, graph, ctypes.c_float(0.05)) + if epoch % 2000 == 0: + l = lib.sc_graph_get_loss(graph) + print(f" epoch {epoch:5d} loss = {l:.8f}") + +# ── 6. Read predictions ────────────────────────────────────── +lib.sc_graph_forward(pool, gpu, graph) +pred_ptr = ctypes.cast(lib.sc_tensor_get_data(Y_pred), + ctypes.POINTER(ctypes.c_float)) + +print("\n X1 X2 │ Target │ Predicted") +print(" " + "─" * 38) +inputs = [0,0, 0,1, 1,0, 1,1] +targets = [0, 1, 1, 0] +for i in range(4): + print(f" {inputs[i*2]:.0f} {inputs[i*2+1]:.0f} │ {targets[i]:.0f} │ {pred_ptr[i]:.4f}") + +# ── 7. Cleanup ──────────────────────────────────────────────── +lib.sc_graph_destroy(graph) +for p in [pool, meta, gcpu, ggpu, gpu]: + lib.sc_pool_destroy(p) +``` + +--- + +## C/C++ Usage Example + +See **`main.cpp`** in the repository root — it trains an XOR network +using exclusively `sc_*` calls and is heavily commented as a walkthrough. + +The key pattern is: + +```c +#include "soft-cuda/python/soft_cuda_python.h" /* single include */ + +/* 1. Pools */ +sc_pool_t *pool = sc_pool_create(1*1024*1024, 0); + +/* 2. Tensors (SC_DTYPE_FLOAT32 = 4, grad=1 for weights) */ +sc_tensor_t *W = sc_tensor_create(pool, SC_DTYPE_FLOAT32, 2, dims, NULL, 1); +sc_tensor_fill_random_normal(W, 0.0f, 0.1f); + +/* 3. Lazy ops */ +sc_tensor_t *out = sc_tensor_mul_naive(pool, X, W); + +/* 4. Build graph (one call) */ +sc_graph_t *g = sc_build_graph(meta, gpu, gcpu, ggpu, loss, SC_BACKEND_CPU); + +/* 5. Train */ +for (int i = 0; i < 10000; i++) + sc_graph_step(pool, gpu, g, 0.05f); + +/* 6. Read loss */ +float l = sc_graph_get_loss(g); + +/* 7. Cleanup */ +sc_graph_destroy(g); +sc_pool_destroy(pool); +``` + +--- + +## Memory Model + +soft-cuda uses **arena (bump) allocation** — all objects are allocated +from pre-sized pools: + +``` +┌─────────────────────────────────────────────────────┐ +│ sc_pool_t (e.g. 1 MB) │ +│ │ +│ ┌────┬────┬────┬────┬────┬───────────────────────┐ │ +│ │ T1 │ T2 │ T3 │ T4 │ T5 │ ← free space → │ │ +│ └────┴────┴────┴────┴────┴───────────────────────┘ │ +│ ▲ ▲ │ +│ base bump pointer │ +│ │ +│ sc_pool_zero() → resets bump to base (O(1) reset) │ +│ sc_pool_destroy() → returns memory to the OS │ +└─────────────────────────────────────────────────────┘ +``` + +**Recommended pool layout:** + +| Pool | `on_device` | Purpose | +|---|---|---| +| `pool` | 0 | Tensor data + op nodes | +| `pool_meta` | 0 | `execution_node_t` objects (graph metadata) | +| `pool_grad_cpu` | 0 | CPU-side gradient tensors | +| `pool_gpu` | 1 | GPU VRAM for forward-pass device data | +| `pool_grad_gpu` | 1 | GPU VRAM for backward-pass gradient buffers | + +> [!CAUTION] +> `sc_pool_zero()` invalidates **all** tensors allocated from that pool. +> Use it only on pools whose contents you're sure you no longer need +> (e.g. ephemeral scratch pools, not your weight pool). + +--- + +## Design Decisions + +### Why a separate bridge layer (instead of fixing `api.h` directly)? + +The existing `api.h` uses C++ features (`enum class`, `std::vector`, +default argument values) that Python's FFI (ctypes, cffi) cannot parse. +Rather than stripping those features from the core library (which would +break the natural C++ ergonomics for C++ users), we added a thin +translation layer. + +### Why `sc_graph_t` instead of exposing `std::vector` as an opaque pointer? + +A raw `std::vector*` could work, but: + +1. We'd need a separate `sc_graph_push` / `sc_graph_get` / `sc_graph_len` + anyway — the bridge functions need typed access. +2. Bundling the vector inside a named struct gives us a place to add + additional state later (e.g. caching the loss value, storing pool + references for automatic cleanup). +3. `typedef struct sc_graph sc_graph_t;` makes the intent clear in the + header — Python users see "this is a graph handle" not "void pointer + to something". + +### Why `placement-new` in `sc_graph_create`? + +`malloc` gives us a C-compatible, ABI-stable allocation. `placement-new` +properly constructs the `std::vector` inside that allocation. +This avoids exposing `new`/`delete` at the ABI boundary while ensuring +the vector's internal state is correctly initialised. + +### Why `SC_DTYPE_*` constants instead of an enum? + +Python `ctypes` can't parse C `enum` declarations. `#define` constants +are just integers — they work everywhere. The bridge function casts +them back to `tensor_dtype_t` internally. + +### Why Layer 1 + Layer 2? + +Layer 1 gives full control (e.g. run forward without backward, inspect +intermediate nodes, use a custom optimizer). Layer 2 covers the 90% +case (training loop) in fewer calls. Both are available simultaneously. diff --git a/main.cpp b/main.cpp index f56d1cd..fbaa0ce 100644 --- a/main.cpp +++ b/main.cpp @@ -1,127 +1,336 @@ -#include "soft-cuda/tensor/api.h" -#include "soft-cuda/tensor/debug_api.h" -#include -#include -#include +/** + * soft-cuda — XOR demo using the flat-C Python bridge API + * + * This file demonstrates how to build and train a tiny neural network + * (2 → 4 → 1, XOR problem) using only the sc_* bridge functions. + * + * The exact same function calls work from Python via ctypes/cffi + * because every sc_* symbol has extern "C" linkage and uses only + * C-safe primitives (no std::vector, no enum class, no default args). + * + * Network architecture: + * + * X [4×2] + * │ + * ┌────┴────┐ + * │ matmul │ W1 [2×4] + * └────┬────┘ + * │ + * ┌────┴────┐ + * │ + b1 │ b1 [1×4] + * └────┬────┘ + * │ + * ┌────┴────┐ + * │ ReLU │ + * └────┬────┘ + * │ + * ┌────┴────┐ + * │ matmul │ W2 [4×1] + * └────┬────┘ + * │ + * ┌────┴────┐ + * │ + b2 │ b2 [1×1] + * └────┬────┘ + * │ + * Y_pred [4×1] + * │ + * ┌────┴────┐ + * │ MSE │─── Y (ground truth) + * └────┬────┘ + * │ + * loss (scalar) + */ -using namespace std; +#include "soft-cuda/python/soft_cuda_python.h" /* the ONLY header you need */ +#include +#include -int main() { - tensor_pool_t *pool = tensor_pool_create(1024 * 1024); - tensor_pool_t *pool_meta = tensor_pool_create(1024 * 1024); - tensor_pool_t *pool_grad_cpu = tensor_pool_create(1024 * 1024); +/* ═══════════════════════════════════════════════════════════════════════════ + * Helpers — nice printing without pulling in + * ═══════════════════════════════════════════════════════════════════════════ */ - tensor_pool_t *pool_gpu = tensor_pool_create(1024 * 1024, true); - tensor_pool_t *pool_grad_gpu = tensor_pool_create(1024 * 1024, true); +static void banner(const char *msg) { + printf("\n"); + printf("╔══════════════════════════════════════════════╗\n"); + printf("║ %-44s║\n", msg); + printf("╚══════════════════════════════════════════════╝\n"); +} + +static void separator(void) { + printf("──────────────────────────────────────────────\n"); +} + +/* ═══════════════════════════════════════════════════════════════════════════ + * main + * ═══════════════════════════════════════════════════════════════════════════ */ + +int main(void) { + + /* ───────────────────────────────────────────── + * 1 ) CREATE MEMORY POOLS + * + * soft-cuda uses arena-style allocation. + * You need separate pools for: + * • data tensors (pool) + * • graph metadata (pool_meta) + * • GPU VRAM (pool_gpu) ← on_device = 1 + * • CPU gradients (pool_grad_cpu) + * • GPU gradients (pool_grad_gpu) ← on_device = 1 + * ───────────────────────────────────────────── */ + + banner("1. Creating memory pools"); - assert(pool != NULL); - assert(pool_gpu != NULL); + sc_pool_t *pool = sc_pool_create(1024 * 1024, 0); /* 1 MB CPU */ + sc_pool_t *pool_meta = sc_pool_create(1024 * 1024, 0); /* 1 MB CPU */ + sc_pool_t *pool_grad_cpu = sc_pool_create(1024 * 1024, 0); /* 1 MB CPU */ + sc_pool_t *pool_gpu = sc_pool_create(1024 * 1024, 1); /* 1 MB VRAM */ + sc_pool_t *pool_grad_gpu = sc_pool_create(1024 * 1024, 1); /* 1 MB VRAM */ - cout << "=========================================== \n"; - cout << "========= XOR IMPLEMENTATION (4-NEURON) ==== \n"; + assert(pool != NULL); + assert(pool_meta != NULL); + assert(pool_grad_cpu != NULL); + assert(pool_gpu != NULL); + assert(pool_grad_gpu != NULL); - float val_X[8]{0,0, 0,1, 1,0, 1,1}; - float val_Y[4]{0, 1, 1, 0}; - - float val_W1[8]{}; - float val_W2[4]{}; - float val_b1[4]{}; - float val_b2[1]{}; + printf(" pool : %zu bytes total, %zu used\n", + sc_pool_size(pool), sc_pool_used(pool)); + + /* ───────────────────────────────────────────── + * 2 ) CREATE INPUT / TARGET TENSORS + * + * sc_tensor_create(pool, dtype, ndims, dims, data, grad) + * + * • dtype: SC_DTYPE_FLOAT32 (= 4) + * • grad = 0 for data tensors (X, Y) + * • grad = 1 for trainable weights (W, b) + * ───────────────────────────────────────────── */ + + banner("2. Creating data tensors"); + + /* XOR truth table */ + float val_X[] = { 0,0, 0,1, 1,0, 1,1 }; + float val_Y[] = { 0, 1, 1, 0 }; uint32_t dims_X[] = {4, 2}; uint32_t dims_Y[] = {4, 1}; - uint32_t dims_W1[] = {2, 4}; + + sc_tensor_t *X = sc_tensor_create(pool, SC_DTYPE_FLOAT32, 2, dims_X, val_X, 0); + sc_tensor_t *Y = sc_tensor_create(pool, SC_DTYPE_FLOAT32, 2, dims_Y, val_Y, 0); + + assert(X != NULL); + assert(Y != NULL); + + printf(" X tensor id = %u (ndims=%u, dims=[%u, %u])\n", + sc_tensor_id(X), + sc_tensor_get_ndims(X), + sc_tensor_get_dims(X)[0], + sc_tensor_get_dims(X)[1]); + + printf(" Y tensor id = %u (ndims=%u, dims=[%u, %u])\n", + sc_tensor_id(Y), + sc_tensor_get_ndims(Y), + sc_tensor_get_dims(Y)[0], + sc_tensor_get_dims(Y)[1]); + + /* ───────────────────────────────────────────── + * 3 ) CREATE TRAINABLE WEIGHT TENSORS + * + * Pass NULL for data → initialized to zero. + * Then fill with random normals for gradient-friendly init. + * ───────────────────────────────────────────── */ + + banner("3. Creating trainable weights"); + + uint32_t dims_W1[] = {2, 4}; /* input → hidden */ uint32_t dims_b1[] = {1, 4}; - uint32_t dims_W2[] = {4, 1}; + uint32_t dims_W2[] = {4, 1}; /* hidden → output */ uint32_t dims_b2[] = {1, 1}; - tensor_t *X = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_X, val_X, false); - tensor_t *Y = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_Y, val_Y, false); - - tensor_t *W1 = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_W1, val_W1, true); - tensor_t *W2 = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_W2, val_W2, true); - tensor_t *b1 = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_b1, val_b1, true); - tensor_t *b2 = tensor_create(pool, tensor_dtype_t::FLOAT32_T, 2, dims_b2, val_b2, true); - - tensor_fill_random_normal(W1, 0.5, 0.2); - tensor_fill_random_normal(W2, 0.5, 0.2); - tensor_fill_random_normal(b1, 0.0, 0.1); - tensor_fill_random_normal(b2, 0.0, 0.1); - - cout << "=========================================== \n"; - cout << "============== DEFINING OPS =============== \n"; - - tensor_t *H_pred_bef = tensor_mul_naive(pool, X, W1); - tensor_t *H_pred = tensor_add(pool, H_pred_bef, b1); - tensor_t *H = tensor_relu(pool, H_pred); - tensor_t *Y_pred_bef = tensor_mul_naive(pool, H, W2); - tensor_t *Y_pred = tensor_add(pool, Y_pred_bef, b2); - - tensor_t *L_sub = tensor_sub(pool, Y_pred, Y); - tensor_t *L_squ = tensor_square(pool, L_sub); - tensor_t *mse = tensor_mean(pool, L_squ); - - cout << "=========================================== \n"; - cout << "============== LAZY EVAL DONE ============= \n"; - - cout << "=========================================== \n"; - cout << "=========== PREPARING THE KITCHEN ========= \n"; - std::vector seq; - bool oki = verifyIfDAG(pool_meta, mse, seq); - assignBackendGraph(pool_gpu, seq, backend_mode::HYBRID); - std::cout << "TESTING OUT "; - assignGradMemory(pool_grad_cpu, pool_grad_gpu, seq); - // tensor_graph_forward_evaluate(pool, pool_gpu, seq); - - if (oki) { - cout << "=========================================== \n"; - cout << "============= STARTING TRAINING =========== \n"; - - for (int i = 0; i <= 10000; i++) { - tensor_graph_forward_evaluate(pool, pool_gpu, seq); - autogradGpuMemTranfer(seq); - gradInitializer(seq); - tensor_graph_backward(seq); - - if (i % 1000 == 0) { - std::cout << "EPOCH " << i << "\n"; - execution_node_t *mse_node = seq.back(); - printExecutionNode(mse_node); - } - - tensor_sgd(seq, 0.05); - } + sc_tensor_t *W1 = sc_tensor_create(pool, SC_DTYPE_FLOAT32, 2, dims_W1, NULL, 1); + sc_tensor_t *b1 = sc_tensor_create(pool, SC_DTYPE_FLOAT32, 2, dims_b1, NULL, 1); + sc_tensor_t *W2 = sc_tensor_create(pool, SC_DTYPE_FLOAT32, 2, dims_W2, NULL, 1); + sc_tensor_t *b2 = sc_tensor_create(pool, SC_DTYPE_FLOAT32, 2, dims_b2, NULL, 1); + + /* Xavier-ish init */ + sc_tensor_fill_random_normal(W1, 0.5f, 0.2f); + sc_tensor_fill_random_normal(W2, 0.5f, 0.2f); + sc_tensor_fill_random_normal(b1, 0.0f, 0.1f); + sc_tensor_fill_random_normal(b2, 0.0f, 0.1f); + + printf(" W1 [%u×%u] grad=ON id=%u\n", + sc_tensor_get_dims(W1)[0], sc_tensor_get_dims(W1)[1], sc_tensor_id(W1)); + printf(" b1 [%u×%u] grad=ON id=%u\n", + sc_tensor_get_dims(b1)[0], sc_tensor_get_dims(b1)[1], sc_tensor_id(b1)); + printf(" W2 [%u×%u] grad=ON id=%u\n", + sc_tensor_get_dims(W2)[0], sc_tensor_get_dims(W2)[1], sc_tensor_id(W2)); + printf(" b2 [%u×%u] grad=ON id=%u\n", + sc_tensor_get_dims(b2)[0], sc_tensor_get_dims(b2)[1], sc_tensor_id(b2)); + + /* ───────────────────────────────────────────── + * 4 ) DEFINE THE COMPUTATION GRAPH (LAZY) + * + * These calls do NOT compute anything yet — + * they construct op-nodes that record the + * operation and its operand pointers. + * + * The graph is: + * H = relu( X·W1 + b1 ) + * Y_pred = H·W2 + b2 + * loss = mean( (Y_pred − Y)² ) + * ───────────────────────────────────────────── */ + + banner("4. Defining computation graph (lazy)"); + + sc_tensor_t *H_pre = sc_tensor_mul_naive(pool, X, W1); + sc_tensor_t *H_bias = sc_tensor_add(pool, H_pre, b1); + sc_tensor_t *H = sc_tensor_relu(pool, H_bias); + + sc_tensor_t *O_pre = sc_tensor_mul_naive(pool, H, W2); + sc_tensor_t *Y_pred = sc_tensor_add(pool, O_pre, b2); + + sc_tensor_t *diff = sc_tensor_sub(pool, Y_pred, Y); + sc_tensor_t *sq = sc_tensor_square(pool, diff); + sc_tensor_t *loss = sc_tensor_mean(pool, sq); - tensor_graph_forward_evaluate(pool, pool_gpu, seq); + printf(" %d op-nodes defined (X→…→loss)\n", 8); + printf(" No computation has happened yet.\n"); - float* inputs = (float*)tensor_get_data(X); - float* targets = (float*)tensor_get_data(Y); - float* predictions = (float*)tensor_get_data(Y_pred); + /* ───────────────────────────────────────────── + * 5 ) BUILD THE GRAPH (Layer-2 one-liner) + * + * sc_build_graph does everything at once: + * 1. verifyIfDAG → topological sort + * 2. assignBackendGraph → CPU / GPU / HYBRID dispatch + * 3. assignGradMemory → allocate gradient buffers + * + * Returns a ready sc_graph_t* handle. + * ───────────────────────────────────────────── */ - cout << "=========================================== \n"; - cout << "============= XOR(4 NEURONS RESULT) ============= \n"; - cout << "X1\tX2\t|\tTarget\t|\tPredicted\n"; - cout << "---------------------------------------------------\n"; + banner("5. Building execution graph"); - for (int i = 0; i < 4; i++) { - float x1 = inputs[i * 2 + 0]; - float x2 = inputs[i * 2 + 1]; - float y_true = targets[i]; - float y_pred = predictions[i]; + sc_graph_t *graph = sc_build_graph( + pool_meta, + pool_gpu, + pool_grad_cpu, + pool_grad_gpu, + loss, + SC_BACKEND_CPU /* use SC_BACKEND_GPU or SC_BACKEND_HYBRID if CUDA is available */ + ); + assert(graph != NULL); - cout << x1 << "\t" << x2 << "\t|\t" << y_true << "\t|\t" << y_pred << "\n"; + printf(" Graph nodes : %zu\n", sc_graph_size(graph)); + printf(" Backend : CPU\n"); + printf(" Pool usage : data %zu / %zu bytes\n", + sc_pool_used(pool), sc_pool_size(pool)); + printf(" meta %zu / %zu bytes\n", + sc_pool_used(pool_meta), sc_pool_size(pool_meta)); + + /* ───────────────────────────────────────────── + * 6 ) TRAINING LOOP + * + * Option A — manual loop (Layer-1 calls): + * sc_graph_forward(...) + * sc_autograd_gpu_transfer(...) + * sc_grad_initializer(...) + * sc_backward(...) + * sc_sgd(...) + * + * Option B — one-liner (Layer-2): + * sc_graph_step(pool, pool_gpu, graph, lr) + * + * We'll use Option B for brevity. + * ───────────────────────────────────────────── */ + + banner("6. Training (10000 epochs, lr=0.05)"); + + int epochs = 10000; + float lr = 0.05f; + + for (int i = 0; i <= epochs; i++) { + + /* ── single training step ────────────── */ + sc_graph_step(pool, pool_gpu, graph, lr); + + /* ── log every 1000 epochs ───────────── */ + if (i % 1000 == 0) { + float l = sc_graph_get_loss(graph); + printf(" epoch %5d loss = %.8f\n", i, l); } - cout << "=========================================== \n"; + } + + /* ───────────────────────────────────────────── + * 7 ) INFERENCE — run one more forward pass + * and read back the predictions + * ───────────────────────────────────────────── */ + + banner("7. Final predictions"); + + sc_graph_forward(pool, pool_gpu, graph); + + float *inputs = (float *)sc_tensor_get_data(X); + float *targets = (float *)sc_tensor_get_data(Y); + float *predictions = (float *)sc_tensor_get_data(Y_pred); + + separator(); + printf(" X1 X2 │ Target │ Predicted\n"); + separator(); + + for (int i = 0; i < 4; i++) { + float x1 = inputs[i * 2 + 0]; + float x2 = inputs[i * 2 + 1]; - } else { - cout << "WARNING: DAG Verification failed. Aborting expedition.\n"; + printf(" %4.1f %4.1f │ %4.1f │ %7.4f\n", + x1, x2, targets[i], predictions[i]); } - tensor_pool_destroy(pool); - tensor_pool_destroy(pool_meta); - tensor_pool_destroy(pool_grad_cpu); - tensor_pool_destroy(pool_grad_gpu); - tensor_pool_destroy(pool_gpu); + separator(); + printf(" Final loss : %.8f\n", sc_graph_get_loss(graph)); + + /* ───────────────────────────────────────────── + * 8 ) SAVE & LOAD (round-trip demo) + * + * sc_save_model / sc_load_model persist raw + * float data — no shape metadata is stored, + * so you must recreate tensors of the correct + * shapes before loading. + * ───────────────────────────────────────────── */ + + banner("8. Save / Load model weights"); + + sc_tensor_t *weights[] = { W1, b1, W2, b2 }; + int saved = sc_save_model("xor_weights.bin", weights, 4); + printf(" save_model → %s\n", saved ? "OK" : "FAILED"); + + /* Zero out W1 to prove load really works */ + sc_tensor_fill_random_normal(W1, 0.0f, 0.0001f); + + int loaded = sc_load_model("xor_weights.bin", weights, 4); + printf(" load_model → %s\n", loaded ? "OK" : "FAILED"); + + /* Forward pass after reloading to confirm predictions match */ + sc_graph_forward(pool, pool_gpu, graph); + + printf(" Post-reload loss : %.8f\n", sc_graph_get_loss(graph)); + + /* ───────────────────────────────────────────── + * 9 ) CLEANUP + * + * Always destroy pools when done. + * sc_graph_destroy frees the graph handle + * but NOT the pools themselves. + * ───────────────────────────────────────────── */ + + banner("9. Cleanup"); + + sc_graph_destroy(graph); + + sc_pool_destroy(pool); + sc_pool_destroy(pool_meta); + sc_pool_destroy(pool_grad_cpu); + sc_pool_destroy(pool_grad_gpu); + sc_pool_destroy(pool_gpu); + + printf(" All resources freed.\n\n"); return 0; } From 4b0cc728ad0eb7eaab98b7e226d8a94174572d12 Mon Sep 17 00:00:00 2001 From: souls-syntax Date: Tue, 21 Apr 2026 02:19:02 +0530 Subject: [PATCH 25/27] Some author defination --- docs/PYTHON_BRIDGE.md | 5 +++++ include/soft-cuda/python/soft_cuda_python.h | 5 +++++ include/soft-cuda/python/tensor_core.h | 5 +++++ include/soft-cuda/python/tensor_graph.h | 5 +++++ include/soft-cuda/python/tensor_io.h | 5 +++++ include/soft-cuda/python/tensor_ops.h | 5 +++++ include/soft-cuda/python/tensor_pool.h | 5 +++++ main.cpp | 5 +++++ src/python/sc_bridge.cpp | 5 +++++ 9 files changed, 45 insertions(+) diff --git a/docs/PYTHON_BRIDGE.md b/docs/PYTHON_BRIDGE.md index 0cd87a7..974d950 100644 --- a/docs/PYTHON_BRIDGE.md +++ b/docs/PYTHON_BRIDGE.md @@ -1,3 +1,8 @@ + + # soft-cuda Python Bridge — API Reference > **`include/soft-cuda/python/soft_cuda_python.h`** — the only header you need. diff --git a/include/soft-cuda/python/soft_cuda_python.h b/include/soft-cuda/python/soft_cuda_python.h index 6ce212f..0306190 100644 --- a/include/soft-cuda/python/soft_cuda_python.h +++ b/include/soft-cuda/python/soft_cuda_python.h @@ -1,3 +1,8 @@ +/* ═══════════════════════════════════════════════════════════════════════════ + * Written by : Antigravity (AI Coding Assistant) + * Date : 2026-04-21 02:12 IST + * ═══════════════════════════════════════════════════════════════════════════ */ + /** * @file soft_cuda_python.h * @brief Master include for the soft-cuda Python/ctypes bridge layer. diff --git a/include/soft-cuda/python/tensor_core.h b/include/soft-cuda/python/tensor_core.h index 23f04b1..0370b49 100644 --- a/include/soft-cuda/python/tensor_core.h +++ b/include/soft-cuda/python/tensor_core.h @@ -1,3 +1,8 @@ +/* ═══════════════════════════════════════════════════════════════════════════ + * Written by : Antigravity (AI Coding Assistant) + * Date : 2026-04-21 02:12 IST + * ═══════════════════════════════════════════════════════════════════════════ */ + /** * @file tensor_core.h * @brief Flat-C tensor lifecycle API — Python/ctypes compatible. diff --git a/include/soft-cuda/python/tensor_graph.h b/include/soft-cuda/python/tensor_graph.h index d53166f..1ce9b3e 100644 --- a/include/soft-cuda/python/tensor_graph.h +++ b/include/soft-cuda/python/tensor_graph.h @@ -1,3 +1,8 @@ +/* ═══════════════════════════════════════════════════════════════════════════ + * Written by : Antigravity (AI Coding Assistant) + * Date : 2026-04-21 02:12 IST + * ═══════════════════════════════════════════════════════════════════════════ */ + /** * @file tensor_graph.h * @brief Flat-C graph building and training primitives — Python/ctypes compatible. diff --git a/include/soft-cuda/python/tensor_io.h b/include/soft-cuda/python/tensor_io.h index 892d34e..4e81252 100644 --- a/include/soft-cuda/python/tensor_io.h +++ b/include/soft-cuda/python/tensor_io.h @@ -1,3 +1,8 @@ +/* ═══════════════════════════════════════════════════════════════════════════ + * Written by : Antigravity (AI Coding Assistant) + * Date : 2026-04-21 02:12 IST + * ═══════════════════════════════════════════════════════════════════════════ */ + /** * @file tensor_io.h * @brief Flat-C model persistence API — Python/ctypes compatible. diff --git a/include/soft-cuda/python/tensor_ops.h b/include/soft-cuda/python/tensor_ops.h index 13c753a..87bb1fc 100644 --- a/include/soft-cuda/python/tensor_ops.h +++ b/include/soft-cuda/python/tensor_ops.h @@ -1,3 +1,8 @@ +/* ═══════════════════════════════════════════════════════════════════════════ + * Written by : Antigravity (AI Coding Assistant) + * Date : 2026-04-21 02:12 IST + * ═══════════════════════════════════════════════════════════════════════════ */ + /** * @file tensor_ops.h * @brief Flat-C forward ops + evaluate — Python/ctypes compatible. diff --git a/include/soft-cuda/python/tensor_pool.h b/include/soft-cuda/python/tensor_pool.h index c70e0a4..618f3fa 100644 --- a/include/soft-cuda/python/tensor_pool.h +++ b/include/soft-cuda/python/tensor_pool.h @@ -1,3 +1,8 @@ +/* ═══════════════════════════════════════════════════════════════════════════ + * Written by : Antigravity (AI Coding Assistant) + * Date : 2026-04-21 02:12 IST + * ═══════════════════════════════════════════════════════════════════════════ */ + /** * @file tensor_pool.h * @brief Flat-C arena management API — Python/ctypes compatible. diff --git a/main.cpp b/main.cpp index fbaa0ce..1620acb 100644 --- a/main.cpp +++ b/main.cpp @@ -1,3 +1,8 @@ +/* ═══════════════════════════════════════════════════════════════════════════ + * Written by : Antigravity (AI Coding Assistant) + * Date : 2026-04-21 02:12 IST + * ═══════════════════════════════════════════════════════════════════════════ */ + /** * soft-cuda — XOR demo using the flat-C Python bridge API * diff --git a/src/python/sc_bridge.cpp b/src/python/sc_bridge.cpp index beeb2f8..4969248 100644 --- a/src/python/sc_bridge.cpp +++ b/src/python/sc_bridge.cpp @@ -1,3 +1,8 @@ +/* ═══════════════════════════════════════════════════════════════════════════ + * Written by : Antigravity (AI Coding Assistant) + * Date : 2026-04-21 02:12 IST + * ═══════════════════════════════════════════════════════════════════════════ */ + /** * @file sc_bridge.cpp * @brief Implementation of the soft-cuda Python C bridge. From 9fb678c249ed095c0b9cf3b987e3c2a2299b1ad4 Mon Sep 17 00:00:00 2001 From: Aakarsh Kashyap Date: Tue, 21 Apr 2026 11:34:38 +0530 Subject: [PATCH 26/27] Some resolution as well as hoping to correct the API.h in next sitting as there is some graurd resoltion i.e. not happening i think i can do it like this --- CMakeLists.txt | 4 ++-- include/soft-cuda/tensor/api.h | 2 +- src/internal_header.h | 1 + src/python/sc_bridge.cpp | 2 ++ 4 files changed, 6 insertions(+), 3 deletions(-) diff --git a/CMakeLists.txt b/CMakeLists.txt index 693b0d6..972d242 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -42,9 +42,7 @@ target_compile_options(soft_lib PRIVATE $<$:-g>) # target_link_options(soft_lib PUBLIC # $<$:-fsanitize=address> # ) -add_executable(soft main.cpp) -target_link_libraries(soft PRIVATE soft_lib) # ───────────────────────────────────────────────────────────────────────────── # Python C bridge — flat-C shared library for use with ctypes / cffi @@ -64,7 +62,9 @@ target_include_directories(soft_cuda_python PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/src" ) +add_executable(soft main.cpp) target_link_libraries(soft_cuda_python PRIVATE soft_lib) +target_link_libraries(soft PRIVATE soft_cuda_python) # Ensure all sc_* symbols are exported on Linux/macOS if(NOT WIN32) diff --git a/include/soft-cuda/tensor/api.h b/include/soft-cuda/tensor/api.h index 1394417..12f562a 100644 --- a/include/soft-cuda/tensor/api.h +++ b/include/soft-cuda/tensor/api.h @@ -19,7 +19,7 @@ extern "C" { // DONE enum class tensor_dtype_t { UINT32_T, - INT33_T, + INT32_T, UINT64_T, INT64_T, FLOAT32_T, diff --git a/src/internal_header.h b/src/internal_header.h index 625597a..bdebfab 100644 --- a/src/internal_header.h +++ b/src/internal_header.h @@ -2,6 +2,7 @@ // PUBLIC DECLARATIONS #include "../include/soft-cuda/tensor/api.h" #include "../include/soft-cuda/tensor/debug_api.h" +#include "../include/soft-cuda/python/soft_cuda_python.h" // PRIVATE DECLARATIONS #include "./core/include/tensor/tensor.h" diff --git a/src/python/sc_bridge.cpp b/src/python/sc_bridge.cpp index 4969248..c51f21e 100644 --- a/src/python/sc_bridge.cpp +++ b/src/python/sc_bridge.cpp @@ -24,6 +24,8 @@ // Pull in everything: public + private headers & all internal APIs #include "../include/soft-cuda/tensor/api.h" #include "../include/soft-cuda/python/soft_cuda_python.h" +#include "internal_header.h" + #include #include /* memcpy */ From eb76cf574475be9896b718297a16558ae6d63fc6 Mon Sep 17 00:00:00 2001 From: Aakarsh Kashyap Date: Tue, 21 Apr 2026 12:00:38 +0530 Subject: [PATCH 27/27] Working: Corrected the linker error --- include/soft-cuda/tensor/api.h | 14 +++++++------- src/backend_cpu/math/mse.cpp | 6 ++++++ 2 files changed, 13 insertions(+), 7 deletions(-) diff --git a/include/soft-cuda/tensor/api.h b/include/soft-cuda/tensor/api.h index 12f562a..150eee3 100644 --- a/include/soft-cuda/tensor/api.h +++ b/include/soft-cuda/tensor/api.h @@ -3,9 +3,9 @@ #include #include -#ifdef __cplusplus -extern "C" { -#endif +// #ifdef __cplusplus +// extern "C" { +// #endif /////////////////////////////////////////////// @@ -219,7 +219,7 @@ tensor_t *tensor_add(tensor_pool_t *pool, tensor_t *x, tensor_t *y); // Explicit broadcasting for layers (e.g., Y = XW + b) // DONE -tensor_t *tensor_add_bias(tensor_pool_t *pool, const tensor_t *xw, const tensor_t *bias); +tensor_t *tensor_add_bias(tensor_pool_t *pool, tensor_t *xw, tensor_t *bias); /* * Do matrix subtraction @@ -387,6 +387,6 @@ void autogradGpuMemTranfer(std::vector &nodes); bool save_model(const std::string& filepath, const std::vector& weights); bool load_model(const std::string& filepath, const std::vector& weights); -#ifdef __cplusplus -} -#endif +// #ifdef __cplusplus +// } +// #endif diff --git a/src/backend_cpu/math/mse.cpp b/src/backend_cpu/math/mse.cpp index 82bba6a..6aef5c6 100644 --- a/src/backend_cpu/math/mse.cpp +++ b/src/backend_cpu/math/mse.cpp @@ -1,2 +1,8 @@ #include "internal_header.h" +tensor_t *tensor_mse_loss(tensor_pool_t *pool, tensor_t *Y_pred, tensor_t *Y) { + tensor_t *L_sub = tensor_sub(pool, Y_pred, Y); + tensor_t *L_squ = tensor_square(pool, L_sub); + tensor_t *mse = tensor_mean(pool, L_squ); + return mse; +}