JDD 2018: Optimal control, no! Than what? (...) by Tomasz sikora

## Overview Tomasz Sikora discusses the evolution from computational exactness to practical optimization, using Go and AlphaGo as a pivotal example of when exact computation must yield to approximate methods. He argues that many real-world business problems cannot be solved with pure brute force, and presents a spectrum of optimization algorithms suited for different scenarios.

## Key Problems and Context The speaker identifies a recurring business challenge: given a table with many attributes (columns) and potentially billions of rows, find the minimum or maximum value across a dataset as efficiently as possible. Traditional database indexing may not work if calculations involve third-party API calls or complex logic. He emphasizes that before adopting deep reinforcement learning or complex frameworks, practitioners should explore simpler metaheuristic methods.

## Optimization Algorithms Presented

**Simulated Annealing**: Inspired by metallurgy, this approach performs random search through a parameter space while gradually reducing randomness (lowering temperature). Tested on a highly multimodal benchmark function in 2D, 5D, and 30D spaces, it checked ~360,000 points in 30-dimensional space to approximate the global minimum efficiently in real-time.

**Differential Evolution**: A population-based method using genetic operators (crossover, mutation). Unlike genetic algorithms, it simplifies implementation by avoiding explicit operator design. On the same test function, it checked ~300,000 points in 3D space with proper parameter tuning (300 genes, population of 100). Supports multi-threading for parallel execution.

**Particle Swarm Optimization (PSO)**: Simulates social cooperation among agents (particles) exploring the search space. Introduced approximately 22 years prior to the talk. With 25 particles, it found good approximations using 250,000 checks in 2D. Scaling to 30 dimensions with 50 particles and reduced iterations required only 5,000 tries, demonstrating efficiency in higher-dimensional spaces.

## Reinforcement Learning and Deep Learning The speaker briefly covers reinforcement learning, where agents take actions, observe state changes, and receive rewards or punishments. He presents DeepMind's Atari game solver (solving 260 games) as an example of end-to-end learning using convolutional neural networks. The Atari example involves 84×84 pixel inputs with 256 colors and movement tracking—a massive search space (10^8,68,000 possibilities). However, he cautions that deep reinforcement learning should not be the default choice for business problems.

## Key Takeaways 1. **Abandon perfectionism**: The goal is a "good enough" solution, not global optimality, especially given computational constraints. 2. **Start simple**: Before investing in heavy frameworks or AWS infrastructure, test lightweight implementations (often 4–5 classes on GitHub). 3. **Unit test with benchmarks**: Use continuous benchmark functions to validate algorithm selection for specific business problems. 4. **Match method to problem**: Simulated annealing, differential evolution, PSO, and ABC each have strengths depending on dimensionality and function characteristics. 5. **Java limitations**: Java is not ideal for deep learning; consider Python microservices called via REST instead. 6. **Accept risk**: Local minima are an inherent risk, but interactive refinement with business stakeholders can help define acceptable solutions.

## Q&A Highlights Audience questions addressed real-world applicability (table with 20 columns, 1 million rows), the requirement for continuous/differentiable functions (gradient methods require this; metaheuristics do not), and how to detect sufficient training convergence. Sikora emphasizes that discrete-space and non-differentiable problems favor metaheuristic (Monte Carlo-based) methods, and that visualization and dimensionality analysis guide algorithm selection.