Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 3 additions & 4 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,18 +1,17 @@

name: CI

on:
push:
branches: [ master ]
branches: [master]
pull_request:
branches: [ master ]
branches: [master]

jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
go: [ '1.25' ]
go: ["1.26"]
steps:
- uses: actions/checkout@v3

Expand Down
2 changes: 1 addition & 1 deletion .golangci.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
version: "2"

run:
go: "1.25"
go: "1.26"
timeout: 5m
tests: false
issues-exit-code: 1
Expand Down
28 changes: 28 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Agent instructions

## Repository map

- The root module is `go.osspkg.com/do`; its source and tests live at the repository root.
- `mapreduce`, `monad`, `pipline`, `words`, and `workerpool` are separate importable packages.
- `pipline` is the existing package and import path spelling. Preserve it unless a deliberate compatibility change is requested.
- `README.md` is the user-facing overview; keep examples and documented behavior aligned with the exported API.
- For usage questions or changes that use this module, read `skills/go-do-usage/SKILL.md` and the relevant linked reference.

## Go version and commands

Run commands from the repository root. `go.mod` requires Go 1.26.0.

- Run all tests with `go test ./...`.
- For changes to the core helpers, worker pool, or map-reduce package, run `go test . ./workerpool ./mapreduce` first.
- Format changed Go files with `gofmt -w <files>`.
- The GitHub Actions workflow is `.github/workflows/ci.yml`; it invokes `make ci`.
- `make ci` runs `make pre-commit`, which installs the latest `goppy` tool, runs `goppy setup-lib`, and then runs license, lint, test, and build targets. Review those setup and generated-file side effects before running it locally.
- `go.mod` requires Go 1.26.0.

## Change guidance

- Add regression tests for changed behavior. For concurrency changes, cover cancellation, shutdown, and channel ownership; use bounded waits in tests that could hang.
- Keep package boundaries intact. The root package contains generic slice/map helpers, conditionals, numeric helpers, panic recovery, and async helpers; the subpackages provide focused APIs.
- Update `README.md` when installation, public behavior, or usage examples change.
- Avoid changing exported names or import paths without considering downstream compatibility.
- Do not run publication, deployment, or release commands as part of local validation.
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
BSD 3-Clause License

Copyright (c) 2024-2025, Mikhail Knyazhev <markus621@yandex.com>
Copyright (c) 2024-2026, Mikhail Knyazhev <markus621@yandex.com>

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
Expand Down
139 changes: 138 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1 +1,138 @@
# go-do
# go-do

[![Go version](https://img.shields.io/github/go-mod/go-version/osspkg/go-do)](https://go.dev/doc/install)
[![CI](https://github.com/osspkg/go-do/actions/workflows/ci.yml/badge.svg?branch=master)](https://github.com/osspkg/go-do/actions/workflows/ci.yml)
[![Go Reference](https://pkg.go.dev/badge/go.osspkg.com/do.svg)](https://pkg.go.dev/go.osspkg.com/do)
[![License](https://img.shields.io/github/license/osspkg/go-do)](LICENSE)

`go-do` is a Go utility library with generic helpers for slices and maps, numeric operations, conditional expressions, panic recovery, and asynchronous work. It also provides focused packages for worker pools, map-reduce, state pipelines, result handling, and word tokenization.

## Requirements

- Go 1.26.0 or newer, as declared in [`go.mod`](go.mod).

## Installation

```sh
go get go.osspkg.com/do
```

Import the root package and any subpackage you need:

```go
import (
"go.osspkg.com/do"
"go.osspkg.com/do/mapreduce"
"go.osspkg.com/do/workerpool"
)
```

## Quick start

```go
package main

import (
"fmt"

"go.osspkg.com/do"
)

func main() {
values := do.Filter([]int{1, 2, 3, 4, 5}, func(value, index int) bool {
return value%2 == 1
})
labels := do.Convert(values, func(value, index int) string {
return fmt.Sprintf("item-%d", value)
})

fmt.Println(labels) // [item-1 item-3 item-5]
}
```

## Packages and features

### Root package: `go.osspkg.com/do`

| Area | Functions and types | Notes |
| --- | --- | --- |
| Slices | `Each`, `Convert`, `Join`, `Chunk`, `Entries`, `Reduce`, `Filter`, `Treat`, `TreatValue`, `Diff`, `Unique`, `IndexOf`, `LastIndexOf`, `Include`, `Exclude`, `Copy`, `Splice`, `Pop`, `Push`, `Shift`, `Unshift`, `Reverse`, `ToMap` | `Splice`, `Push`, `Unshift`, `Pop`, `Shift`, and `Reverse` mutate the supplied slice or slice pointer. |
| Maps | `EachMap`, `ConvertMap`, `FilterMap`, `TreatMap`, `TreatMapValue`, `Keys`, `Values`, `JoinMap`, `FlipMap`, `DivideMap`, `CombineMap`, `ReduceMap`, `ToSlice` | `Keys`, `Values`, `DivideMap`, `ReduceMap`, and `ToSlice` use sorted keys. Other map transformations may follow Go's unspecified map iteration order. |
| Numeric helpers | `MinMax`, `MinMaxTime`, `Range`, `Sum`, `Average` | `Range` includes both endpoints when reached and requires a positive, progressing step; otherwise it returns an empty slice or stops when the value cannot advance. |
| Conditional helpers | `If`, `IfFunc`, `IfElse`, `IfElseFunc`, `DoIf` | `DoIf.ElseIf`, `Else`, `ElseIfFunc`, and `ElseFunc` form a conditional chain; function variants evaluate only the selected branch. |
| Panic and async helpers | `Recovery`, `Trace`, `Try`, `Async`, `AsyncGroup` | `AsyncGroup` waits for all supplied functions and returns their errors. |

The `Summable` and `Comparable` type constraints define the numeric and ordered types accepted by the generic helpers.

### `go.osspkg.com/do/workerpool`

A bounded worker pool with generic task and result types.

| API | Use |
| --- | --- |
| `Task[I, T]`, `Result[I, T]` | Carry a task ID and input data, or a task ID, result value, and error. |
| `Pool[I, T, R]` | Own the task and result channels and worker lifecycle. |
| `New[I, T, R]` | Create a pool with a fixed worker count and task handler. |
| `Pool.Start` | Start the workers; repeated calls have no effect. |
| `Pool.Send` | Submit a task; returns `false` after shutdown or when the pool is closing. |
| `Pool.Receive` | Get the result channel. |
| `Pool.Close` | Cancel workers and close the result channel; safe to call more than once. |

Read results while work is running to avoid filling the bounded result buffer during normal processing.

### `go.osspkg.com/do/mapreduce`

| API | Use |
| --- | --- |
| `New[T, R, O]` | Map items concurrently and pass completed results to a reducer. |

Set `workers` to a positive number. Mapping and reducing order is nondeterministic when more than one worker is used, so reducers should not depend on input order. A reducer error cancels the mapping context and waits for mapper goroutines to return; mapper functions should honor their context.

### `go.osspkg.com/do/pipline`

| API | Use |
| --- | --- |
| `Pipe[S, T]` | Define a state transition and its context-aware handler. |
| `Pipline[S, T]` | Store transitions keyed by their current state. |
| `New[S, T]` | Create an empty pipeline. |
| `Pipline.Set` | Register a transition; returns an error for a nil handler, duplicate current state, or identical current and next states. |
| `Pipline.Do` | Run handlers from a starting state until no transition exists, the context is canceled, or a handler returns an error. |

### `go.osspkg.com/do/monad`

| API | Use |
| --- | --- |
| `Result[T]` | Hold a value and an error for chained operations. |
| `Some[T]` | Create a successful result from a value. |
| `Bind[T, R]` | Apply a function to a successful result and propagate errors. |
| `Result.Return` | Retrieve the result's value and error. |

### `go.osspkg.com/do/words`

| API | Use |
| --- | --- |
| `Words` | Interface for tokenizing to strings or byte slices and configuring token categories. |
| `NewString`, `NewBytes` | Create a tokenizer with default rune detectors. |
| `Strings`, `Bytes` | Tokenize a string or byte slice with the default block and symbol detectors. |
| `Words.Strings`, `Words.Bytes` | Tokenize using the instance's current detector configuration. |
| `UseDefaultBlock`, `SetBlock` | Select default or custom runes that form word blocks. |
| `UseDefaultDigital`, `SetDigital` | Select default or custom digit runes. |
| `UseDefaultSymbol`, `SetSymbol` | Select default or custom symbol runes. |

## Testing

Run the complete test suite from the repository root:

```sh
go test ./...
```

The project Makefile also provides `make tests`, `make lint`, `make build`, and `make ci`. `make ci` installs the latest `goppy` tool and runs setup, license, lint, test, and build targets; see the [Makefile](Makefile) before running it locally.

## Contributing

Pull requests are welcome. Include tests for behavior changes and run `go test ./...` before submitting. Keep public API and import-path compatibility in mind.

## License

This project is licensed under the [BSD 3-Clause License](LICENSE).
2 changes: 1 addition & 1 deletion async.go
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* Copyright (c) 2024-2025 Mikhail Knyazhev <markus621@yandex.com>. All rights reserved.
* Copyright (c) 2024-2026 Mikhail Knyazhev <markus621@yandex.com>. All rights reserved.
* Use of this source code is governed by a BSD 3-Clause license that can be found in the LICENSE file.
*/

Expand Down
2 changes: 1 addition & 1 deletion async_test.go
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* Copyright (c) 2024-2025 Mikhail Knyazhev <markus621@yandex.com>. All rights reserved.
* Copyright (c) 2024-2026 Mikhail Knyazhev <markus621@yandex.com>. All rights reserved.
* Use of this source code is governed by a BSD 3-Clause license that can be found in the LICENSE file.
*/

Expand Down
2 changes: 1 addition & 1 deletion common.go
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* Copyright (c) 2024-2025 Mikhail Knyazhev <markus621@yandex.com>. All rights reserved.
* Copyright (c) 2024-2026 Mikhail Knyazhev <markus621@yandex.com>. All rights reserved.
* Use of this source code is governed by a BSD 3-Clause license that can be found in the LICENSE file.
*/

Expand Down
6 changes: 3 additions & 3 deletions go.mod
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
module go.osspkg.com/do

go 1.25.0
go 1.26.0

require (
go.osspkg.com/casecheck v0.3.0
golang.org/x/sync v0.21.0
go.osspkg.com/casecheck v0.3.2
golang.org/x/sync v0.23.0
)
8 changes: 4 additions & 4 deletions go.sum
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
go.osspkg.com/casecheck v0.3.0 h1:x15blEszElbrHrEH5H02JIIhGIg/lGZzIt1kQlD3pwM=
go.osspkg.com/casecheck v0.3.0/go.mod h1:TRFXDMFJEOtnlp3ET2Hix3osbxwPWhvaiT/HfD3+gBA=
golang.org/x/sync v0.21.0 h1:HLII4xRRTtCRkxYp4HNFF0Js/Og6q2i++KXbg0gHCwM=
golang.org/x/sync v0.21.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
go.osspkg.com/casecheck v0.3.2 h1:KDdtEsEnGcDKjtg8FKL0hjnVZ7kusmXBD/VS/5IKSjE=
go.osspkg.com/casecheck v0.3.2/go.mod h1:nf1vimi3VPl1o0hV+bKZsy3+1Qi8wNRRWiNGqivFV1A=
golang.org/x/sync v0.23.0 h1:KameEIfc1IkluZyXWLn39Wd4tURc6GbCiISGiZm2bQk=
golang.org/x/sync v0.23.0/go.mod h1:sUUOizhqBxiL6pEWpqNLUiaJn1ShEbZ6BBqskPbjZm0=
2 changes: 1 addition & 1 deletion if.go
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* Copyright (c) 2024-2025 Mikhail Knyazhev <markus621@yandex.com>. All rights reserved.
* Copyright (c) 2024-2026 Mikhail Knyazhev <markus621@yandex.com>. All rights reserved.
* Use of this source code is governed by a BSD 3-Clause license that can be found in the LICENSE file.
*/

Expand Down
2 changes: 1 addition & 1 deletion if_test.go
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* Copyright (c) 2024-2025 Mikhail Knyazhev <markus621@yandex.com>. All rights reserved.
* Copyright (c) 2024-2026 Mikhail Knyazhev <markus621@yandex.com>. All rights reserved.
* Use of this source code is governed by a BSD 3-Clause license that can be found in the LICENSE file.
*/

Expand Down
2 changes: 1 addition & 1 deletion map.go
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* Copyright (c) 2024-2025 Mikhail Knyazhev <markus621@yandex.com>. All rights reserved.
* Copyright (c) 2024-2026 Mikhail Knyazhev <markus621@yandex.com>. All rights reserved.
* Use of this source code is governed by a BSD 3-Clause license that can be found in the LICENSE file.
*/

Expand Down
2 changes: 1 addition & 1 deletion map_test.go
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* Copyright (c) 2024-2025 Mikhail Knyazhev <markus621@yandex.com>. All rights reserved.
* Copyright (c) 2024-2026 Mikhail Knyazhev <markus621@yandex.com>. All rights reserved.
* Use of this source code is governed by a BSD 3-Clause license that can be found in the LICENSE file.
*/

Expand Down
16 changes: 15 additions & 1 deletion mapreduce/mapreduce.go
Original file line number Diff line number Diff line change
@@ -1,7 +1,13 @@
/*
* Copyright (c) 2024-2026 Mikhail Knyazhev <markus621@yandex.com>. All rights reserved.
* Use of this source code is governed by a BSD 3-Clause license that can be found in the LICENSE file.
*/

package mapreduce

import (
"context"
"errors"

"golang.org/x/sync/errgroup"
)
Expand All @@ -14,7 +20,13 @@ func New[T, R, O any](
initial O,
workers int,
) (O, error) {
g, ctx := errgroup.WithContext(ctx)
if workers <= 0 {
return initial, errors.New("workers must be greater than zero")
}

workCtx, cancel := context.WithCancel(ctx)
defer cancel()
g, ctx := errgroup.WithContext(workCtx)
g.SetLimit(workers)

results := make(chan R, len(items))
Expand Down Expand Up @@ -45,6 +57,8 @@ func New[T, R, O any](
var err error
acc, err = reducer(ctx, acc, r)
if err != nil {
cancel()
_ = g.Wait()
return acc, err
}
}
Expand Down
49 changes: 49 additions & 0 deletions mapreduce/mapreduce_test.go
Original file line number Diff line number Diff line change
@@ -1,8 +1,14 @@
/*
* Copyright (c) 2024-2026 Mikhail Knyazhev <markus621@yandex.com>. All rights reserved.
* Use of this source code is governed by a BSD 3-Clause license that can be found in the LICENSE file.
*/

package mapreduce

import (
"context"
"errors"
"fmt"
"sync"
"testing"
"time"
Expand Down Expand Up @@ -256,3 +262,46 @@ func TestUnit_MapReduceWaitForAllGoroutines(t *testing.T) {
t.Errorf("expected %d results, got %d", len(items), len(result))
}
}

func TestUnit_MapReduceRejectsNonPositiveWorkers(t *testing.T) {
mapper := func(ctx context.Context, value int) (int, error) { return value, nil }
reducer := func(ctx context.Context, acc, value int) (int, error) { return acc + value, nil }

for _, workers := range []int{0, -1} {
t.Run(fmt.Sprintf("workers=%d", workers), func(t *testing.T) {
got, err := New(context.Background(), []int{1}, mapper, reducer, 7, workers)
if err == nil {
t.Fatal("expected an error for non-positive workers")
}
if got != 7 {
t.Fatalf("initial value: got %d, want 7", got)
}
})
}
}

func TestUnit_MapReduceReducerErrorCancelsMappers(t *testing.T) {
wantErr := errors.New("reducer error")
canceled := make(chan struct{})
mapper := func(ctx context.Context, value int) (int, error) {
if value == 1 {
return value, nil
}
<-ctx.Done()
close(canceled)
return 0, ctx.Err()
}
reducer := func(ctx context.Context, acc, value int) (int, error) {
return acc, wantErr
}

_, err := New(context.Background(), []int{1, 2}, mapper, reducer, 0, 2)
if !errors.Is(err, wantErr) {
t.Fatalf("got error %v, want %v", err, wantErr)
}
select {
case <-canceled:
case <-time.After(time.Second):
t.Fatal("mapper did not observe cancellation")
}
}
Loading
Loading