Evaluation Criteria & Competition Rules

This section presents the evaluation framework and official competition rules for the WavePiston Passive Control (WAPPAC) Competition.

It is self-contained for participants’ convenience, while details on formulations and implementations are provided in the relevant sections of the WAPPAC Competition Documentation.

---

Evaluation Criteria

The WAPPAC Competition challenges participants to design a passive control strategy that maximizes a performance index balancing absorbed power, physical constraints, and capacity factor.

All controllers are evaluated using the official WAPPAC simulation platform, under identical numerical setups, model configurations, and sea-state data to ensure fairness and reproducibility.

The performance index \(\mathcal{G}\) to be maximized across three predefined sea states is:

\[\max_{F_{pto}(t)} \; \mathcal{G}\!\left(F_{pto}(t); t \in [T_0,t_{end}] \right) = \frac{\bar{P}_{pto}} {2 + \frac{\left[|x(t)|\right]_{98}}{x_{\max}} + \frac{\left[|F_{pto}(t)|\right]_{98}}{F_{u,\max}} - \frac{\bar{P}_{pto}}{\left[p_{pto}(t)\right]_{98}}}\]

\[\begin{split}\text{s.t.}& \quad \text{WavePiston dynamics (1),} \quad \\ & \quad p_{pto}(t) = F_{pto}(t) \dot{x}(t) \ge 0 \quad \forall t \in [t_{init},t_{end}]\end{split}\]

A full derivation and explanation of \(\mathcal{G}\) and its terms are provided in Control Problem Definition.

---

Key Time Intervals

When defining and testing their controllers, participants must account for two key time intervals:

Figure 10 Key time intervals for the performance index evaluation.

Important

• Scoring interval (\(t \ge 30\) s):

  • The performance index \(\mathcal{G}\) is computed only during the scoring interval \(t \in [T_0, t_{end}]\), where \(T_0 = 30\) s.

• Full simulation time span (\(t \ge 0\) s):

  • The controller must remain defined and stable for the entire simulation duration \(t \in [t_{init}, t_{end}]\).

  • The passivity constraint \(p_{pto}(t) \ge 0\) must hold for the entire time span.

  • Any violation of the passivity (hard) constraint results in disqualification of the corresponding sea state evaluation (see Competition Rules below).

---

Participant’s Final Competition Score

Each of the three predefined sea states is evaluated independently, producing scores \(\mathcal{G}_1\), \(\mathcal{G}_2\), and \(\mathcal{G}_3\). The final competition score is computed as:

(2)\[\mathcal{G}_{\text{total}} \;=\; \mathcal{G}_1 \;+\; \mathcal{G}_2 \;+\; \mathcal{G}_3\]

The participant achieving the highest \(\mathcal{G}_{\text{total}}\) will be declared the winner of the WAPPAC Competition. In case of a tie, the organizers may apply secondary criteria such as lowest computational cost or earliest valid submission.

---

Numerical Consistency Requirement

The WAPPAC simulator represents a numerical approximation of a continuous-time physical system, subject to inherent limitations introduced by the discretization of the governing equations. Controllers that induce time responses approaching or exceeding the simulation timestep may surpass the model numerical resolution, leading to unrealistic dynamics.

Performance scores obtained from numerically inconsistent or non-physically meaningful simulations will not be considered valid for evaluation. Controllers must be stable and yield consistent dynamic behavior across the predefined sea states to be eligible for scoring.

For practical guidance on maintaining numerical consistency and mitigating these effects, refer to Numerical Implementation and Writing Your Controller.

---

Competition Rules

1. Evaluation Mode & Reproducibility

   • Final evaluation is performed by setting eval_flag = true in my_sim_input_file.json (see Simulation Input File). This automatically executes all three official sea-state scenarios in a randomized order, each using fixed internal seeds. The controller is not informed of which scenario is active and should therefore adapt its response accordingly.

   • The process generates three encrypted output files (one per sea state) in the evaluation_outputs/ directory. These files are part of the official submission documents.

   • The COER team will re-run each participant’s controller using the official WAPPAC simulation platform, under identical solvers, hydrodynamic models, and conditions. This ensures reproducibility and fairness across all submissions.

   • Any missing, non-functional, or non-reproducible controller for a given sea state will result in the exclusion of that sea state’s score \(\mathcal{G}_i\) from \(\mathcal{G}_{\text{total}}\). Therefore, only valid evaluations are counted, however, incomplete submissions may be deemed invalid at the organizers’ discretion.

2. Passivity (Hard) Constraint Handling

   • The PTO power must remain non-negative throughout the entire simulation: \(p_{pto}(t) = F_{pto}(t)\dot{x}(t) \ge 0 \; \forall t\).

   • Any violation of this constraint leads to exclusion of the affected sea-state score \(\mathcal{G}_i\) from the total \(\mathcal{G}_{\text{total}}\).

3. Position and Force (Soft) Constraints Handling

   • Position and PTO force limits are treated as soft constraints. Occasional exceedances are permitted but penalized within \(\mathcal{G}\) (see Control Problem Definition).

4. Integrity and Evaluation Environment

   • All evaluations are executed on the official, precompiled WAPPAC binaries, which run in a network-isolated, CPU-only environment. Internet access, external API calls, or runtime package installations are not permitted during evaluation. Any attempt to perform such actions will cause evaluation failure. Controllers must run fully self-contained, and not rely on internet or LAN communication (local function calls or internal APIs are, of course, allowed).

   • Results exhibiting numerical instability or unrealistic dynamics (see Numerical Implementation and Writing Your Controller) may be subject to manual review. If the reported performance is determined to arise from numerical artifacts rather than consistent dynamic behavior, the corresponding results will be excluded from the official evaluation, regardless of their performance index. This ensures fairness and consistency with the intended physical modeling assumptions and the numerical limitations of the WAPPAC simulation platform.

   • No modifications to the simulation binaries, hydrodynamic model, solver settings, or wave data are permitted:

   Any attempt to alter or tamper with the simulation platform, inputs, or encrypted outputs constitutes a violation of the competition integrity and results in immediate disqualification and reporting the participant.

5. Unit System and Conventions

   • All physical quantities are expressed in SI units.

   • Participants must ensure their controller adheres to the variable naming and normalization conventions defined in the documentation for compatibility and correctness.

---

Quick Reference

Topic	Summary
Objective	Maximize the performance index \(\mathcal{G}\) across three sea-state scenarios
Scoring	\(\mathcal{G}_{\text{total}} = \mathcal{G}_1 + \mathcal{G}_2 + \mathcal{G}_3\)
Hard Constraint	\(p_{pto}(t) = F_{pto}(t)\dot{x}(t) \ge 0\) — violations disqualify the corresponding run
Soft Constraints	Position and force limits are penalized within \(\mathcal{G}\) (occasional exceedances allowed)
Evaluation Order	Sea states are simulated in aleatory order during eval_flag = true; controller should be able to adapt.
Environment	Offline, CPU-only; no network/API access or runtime installs allowed during evaluation.
Reproducibility	Organizers will re-run controllers using the official WAPPAC binaries under fixed conditions
Fairness	All participants evaluated under identical physics, solvers, and wave data
Units	All variables and parameters expressed in SI units

---

Related Sections for Further Details

• Modeling Framework

• Control Problem Definition

• Performance Index & Metrics

• Simulation Framework

• Submission Guidelines