ThrowIrrecoverable lacks graceful cancellation and resource cleanup mechanism

[staheri14]

Problem Description

The ThrowIrrecoverable method in modules/throwable/context.go:27-35 currently lacks a proper cancellation mechanism and graceful shutdown logic. When an irrecoverable error occurs, the method propagates the error to the root context and then calls log.Fatal(), which immediately terminates the program without allowing for proper resource cleanup.

Current Behavior

• Irrecoverable errors propagate up the context chain via recursive calls
• At the root level, log.Fatal() is called, causing immediate program termination
• No opportunity for graceful shutdown or resource cleanup
• No cancellation signal is sent to other goroutines or components

Issues with Current Implementation

1. Ungraceful Termination: log.Fatal() calls os.Exit(1) internally, bypassing any cleanup logic
2. Resource Leaks: Network connections, file handles, database connections, and other resources may not be properly closed
3. No Cancellation Signal: Other running goroutines are not notified of the fatal error
4. Lost Context: The cancellation doesn't propagate through the normal Go context cancellation mechanism

Suggested Implementation Approach

1. Add Cancellation Mechanism

Enhance the Context struct to include a cancellation function:

type Context struct {
    ctx    context.Context
    cancel context.CancelFunc
}

2. Implement Graceful Shutdown

Replace direct log.Fatal() with a graceful shutdown mechanism:

func (t *Context) ThrowIrrecoverable(err error) {
    // Cancel the context to signal all dependent operations
    if t.cancel != nil {
        t.cancel()
    }
    
    // Propagate to parent if exists
    if parent, ok := t.ctx.(*Context); ok {
        parent.ThrowIrrecoverable(err)
        return
    }
    
    // At root level: log error and initiate graceful shutdown
    log.Printf("irrecoverable error occurred: %v", err)
    
    // Allow brief grace period for cleanup
    time.Sleep(100 * time.Millisecond)
    
    // Exit after cleanup opportunity
    os.Exit(1)
}

3. Factory Function Updates

Update the constructor to properly set up cancellation:

func NewContext(ctx context.Context) *Context {
    childCtx, cancel := context.WithCancel(ctx)
    return &Context{
        ctx:    childCtx,
        cancel: cancel,
    }
}

func NewRootContext() *Context {
    ctx, cancel := context.WithCancel(context.Background())
    return &Context{
        ctx:    ctx,
        cancel: cancel,
    }
}

4. Consider Shutdown Hook Pattern

For more sophisticated cleanup, consider implementing a shutdown hook registry:

type shutdownHook func() error

func (t *Context) RegisterShutdownHook(hook shutdownHook) {
    // Add to cleanup registry
}

func (t *Context) executeShutdownHooks() {
    // Execute all registered cleanup functions
}

Benefits of Proposed Changes

1. Graceful Shutdown: Allows components to clean up resources before termination
2. Context Cancellation: Properly cancels all child contexts and dependent operations
3. Resource Management: Prevents resource leaks through proper cleanup
4. Debugging: Better error logging and state preservation during shutdown
5. Testability: Shutdown behavior can be tested without actually terminating the process

Testing Considerations

• Add tests for cancellation propagation
• Test resource cleanup scenarios
• Mock testing for shutdown hooks
• Verify timeout behavior during cleanup

Priority

This issue affects system reliability and resource management, making it a medium to high priority item for system stability.