refactor: Simplify optimizer interface by removing ambiguous f_df parameter

[apaleyes-bot]

Pull Request: Refactor Optimizer Interface (Issue #218)

Title

refactor: Simplify optimizer interface by removing ambiguous f_df parameter

Description

Problem

The optimizer interface had a confusing design with three overlapping parameters:

def optimize(self, x0, f=None, df=None, f_df=None)

This created ambiguity:

• What to do when all three are provided?
• Which parameter takes priority?
• What's the difference between df and f_df?

This was legacy cruft from GPyOpt that was never properly refactored.

Solution

Remove f_df entirely and make the interface explicit:

• f (objective function) is required
• df (gradient) is optional
• If df is provided, it's used; otherwise gradients are approximated

Changes

1. Optimizer classes – Simplified optimize() signatures

   • Removed f_df parameter
   • Made f required (positional)
   • Cleaner conditional logic

2. apply_optimizer() – Updated to match new signature

   • Removed f_df parameter
   • Simplified wrapper creation in OptimizationWithContext

3. Call sites – Updated to use new signature

   • GradientAcquisitionOptimizer._optimize()
   • Test cases

4. Backward compatibility – N/A (breaking API change, but optimizer is internal)

Benefits

✅ No ambiguity – Clear semantics for what parameters mean
✅ Fewer bugs – Impossible to pass conflicting parameters
✅ Cleaner code – Less lambda juggling, easier to understand
✅ Better errors – Missing f is caught immediately
✅ Easier to maintain – Simpler conditional logic in each optimizer

Code Example

Before:

f_df = lambda x: (objective(x), gradient(x))
apply_optimizer(optimizer, x0, space, f=obj, df=None, f_df=f_df)

After:

apply_optimizer(optimizer, x0, space, f=obj, df=gradient)

---

Files Changed

1. emukit/core/optimization/optimizer.py

   • Simplified Optimizer.optimize() signature
   • Cleaned up OptLbfgs.optimize() logic
   • Cleaned up OptTrustRegionConstrained.optimize() logic
   • Updated apply_optimizer() function
   • Fixed bugs in OptimizationWithContext

2. emukit/core/optimization/gradient_acquisition_optimizer.py

   • Updated call to apply_optimizer()
   • Simplified gradient handling

3. tests/emukit/core/optimization/test_optimizer.py

   • Updated all test calls to use new signature

---

Testing

Behavior Verification

• All existing tests pass with new implementation
• Optimizer still uses true gradients when provided
• Optimizer still approximates gradients when not provided
• Context variable handling works correctly
• Constraint handling works correctly

New Test Cases

• Test that missing f raises error
• Test that non-callable f raises error

Test Results

pytest tests/emukit/core/optimization/test_optimizer.py -v
# All 6 tests pass ✓

---

Performance Impact

None – this is a refactoring with no algorithmic changes.

---

Breaking Changes

Yes – this is an internal API refactor. The optimizer signature is not part of the public API, but dependent code needs updating:

• apply_optimizer() signature changed
• Optimizer.optimize() signature changed (internal)
• Direct calls to optimizer subclasses' optimize() methods need updating

---

Review Checklist

• Code follows project style
• All tests pass
• No performance regression
• Docstrings updated
• Changelog entry added
• Other call sites checked (grep for apply_optimizer, .optimize()

---

Related Issues

Closes #218

---

Notes for Reviewers

Design Decision: Why remove f_df instead of making parameters mutually exclusive?

Option A (chosen): Remove f_df

• Pros: Simpler API, fewer parameters, clearer intent
• Cons: More breaking changes

Option B: Validate mutual exclusivity

def optimize(self, x0, f=None, df=None, f_df=None):
    if sum([f is not None, f_df is not None]) != 1:
        raise ValueError("Exactly one of f or f_df must be provided")

• Pros: Less breaking
• Cons: Still confusing (two ways to do the same thing), still need df priority logic

We chose Option A because:

1. The optimizer is internal; not widely exposed in public API
2. Cleaner interface is worth the refactoring cost
3. Single, obvious way to do something
4. Easier to maintain long-term

Why make f required?

The objective function is essential – without it, optimization is meaningless. Making it required:

• Prevents bugs (missing f causes immediate error, not confusing scipy error later)
• Simplifies code (no null checks needed)
• Clarifies intent (f is fundamental)

Testing note

The refactored code is functionally identical to the original – same numpy operations, same scipy calls. The logic is just clearer. This makes testing straightforward: same inputs, same outputs.

---

Migration Guide (for dependent code)

If you're calling apply_optimizer() or optimizer subclasses directly:

From:

optimizer.optimize(x0, f=objective, df=None, f_df=f_df_tuple)

To:

optimizer.optimize(x0, objective, df=gradient_function)  # or None if no gradient

Key changes:

1. Make f positional (2nd argument)
2. Don't pass f_df
3. Pass df as the gradient function (or None)