The maintenance problem with safety-critical components is significant for the economical benefit of companies. Motivated by a practical asset maintenance project, a new joint replacement maintenance problem is introduced in this paper. The dynamics of the problem are modelled as a Markov decision process, whose action space increases exponentially with the number of safety-critical components in the asset. To deal with the curse of dimensionality, we identify a key property of the optimal solution: the optimal performance can always be achieved in a class of policies which satisfy the so-called shortest-remaininglifetime- first (SRLF) rule. It reduces the action space from O(2n ) to (On ), where is the number of safety-critical components. To further speed up the optimization procedure, some interesting properties of the optimal policy are derived. Combining the SRLF rule and the neuro-dynamic programming (NDP) methodology, we develop an efficient on-line algorithm to optimize this maintenance problem. This algorithm can handle the difficulties of large state space and large action space. Besides the theoretical proof, the optimality and efficiency of the SRLF rule and the properties of the optimal policy are also illustrated by numerical examples. This work can shed some insights to the maintenance problems in a more general situation.

Finite-Stage Markov Decision Process (MDP) supplies a general framework for many practical problems when only the performance in a finite duration is of interest. Dynamic programming (DP) supplies a general way to find the optimal policies but is usually practically infeasible, due to the exponentially increasing policy space. Approximating the finitestage MDP by an infinite-stage MDP reduces the search space but usually does not find the optimal stationary policy, due to the approximation error. We develop a method that finds the optimal stationary policies for the finite-stage MDP. The method is based on performance potentials, which can be estimated through sample paths and thus suits practical application.

The pervasive application of digital computer in control and optimization techniques forces us to consider the constraint of limited memory space when dealing with large scale practical systems. As an example, we consider the famous Witsenhausen counterexample with the new constraint of limited memory space in this paper. The main difficulty is how to sample strategies that can be stored in the given memory space efficiently. The concept of Kolmogorov complexity measures the minimal memory space to store a strategy (i.e., simple strategies), but is incomputable. To overcome this difficulty, we propose a method based on ordered binary decision diagram to sample only simple strategies. Besides the high sampling efficiency which is demonstrated by numerical testing, the proposed sampling method can be easily combined with optimization algorithms and performance evaluation techniques. As an example, we show how to combine ordinal optimization, numerical integration, and the proposed sampling method to solve the Witsenhausen problem with the constraint of limited memory space. We hope this work can shed some insights to computer-based optimization problems with memory space constraint in a more general situation.

Due to the abundant application background, the optimization of maintenance problem has been extensively studied in the past decades. Besides the well-known difficulty of large state space and large action space, the pervasive application of digital computers forces us to consider the new constraint of limited memory space. The given memory space restricts what strategies can be explored during the optimization procedure. By explicitly quantifying the minimal memory space to store a strategy using support vector machine, we propose to describe simple strategies exactly and only approximate complex strategies. This selective approximation can best utilize the given memory space for any description mechanism. We use numerical results on illustrative examples to show how the selective approximation improves the solution quality. We hope this work sheds some insights to best utilize the memory space for practical engine maintenance strategy optimization problems.