Simultaneous Planning of Production , Setup and Maintenance for an Unreliable Multiple Products Manufacturing System

DOI: http://dx.doi.org/10.24018/ejers.2021.6.5.2495 Vol 6 | Issue 5 | July 2021 24 Abstract — The work presented in this paper addresses the problem of joint optimization of the production, setup and corrective maintenance activities of a manufacturing system. This system consists of a machine subject to breakdowns and repairs and producing two types of parts. A corrective maintenance strategy whose repair rate depends on the number of setup operations already performed on the production system is considered in this work. The objective of this research is to propose a policy that controls production, setup, and corrective maintenance. The contribution of this paper is through the control of the repair rate, combined with the planning of production and setup in a dynamic and stochastic context. Optimality conditions in the form of Hamilton-Jacoby-Bellman (HJB) equations are obtained and a numerical approach is proposed in order to deal with the joint optimization issues. Extensive simulations are performed to address many scenarios that illustrate the interactions between production, setup and maintenance activities.

I. INTRODUCTION 1 In manufacturing industries, the corrective maintenance function is a priority and is at the center of industrial concerns as it plays a crucial role in maintaining the production infrastructure. Much of the control policies developed so far assume that the dynamics of machine failures and repairs are described by Markov processes. However, the frequency of occurrence of machine failures increases over time as corrective maintenance actions are not perfect. Dellagi et al. [1] studied joint production and maintenance strategies considering that the system deteriorates in the presence of imperfect maintenance activities. In the same context, Ayed et al [2] treated a manufacturing system that operates in an imperfect environment whose failure rate depends on both the age of the machine and the rate of production. Depending on the state of the system defined by the levels of machine deterioration and by the quality of the parts produced, Njike et al. [3] and Rivera-Gomez et al. [4] developed joint optimization models in an uncertain environment and thus contributed to the development of new decision support tools for uncertain manufacturing systems.
In the industrial context, setup operations usually involve significant time and cost. These can have a significant impact on the competitiveness of the manufacturing company. The flexibility and performance of production systems are therefore limited by costs and setup shutdowns without added values. Indeed, the setup actions generate significant losses in production. Hence, it is important to control and reduce the number of setup actions. The problem is to propose a global and effective structure of the setup operations considering all the interactions between the production and the maintenance. Thus, important research work has been done in production management of several products when the configuration change (setup) requires a significant time and cost. The goal of this work is to use a joint optimal production and setup control policy that increases productivity and resource availability as well as minimizes the total cost incurred by the system.
Since the 1990, production systems consisting of machines producing two types of parts and requiring setup activities to switch from one mode of production to another have been studied [5]. A joint optimization of production, setup and maintenance policies based on stochastic dynamic programming and numerical methods was presented by Boukas and Kenne [6] in a dynamic context. Subsequently, Hajji et al. [7] and Gharbi et al. [8] analyzed in detail the systems producing two types of products subject to random failures and repairs in the presence of the setup. They showed that the joint optimal production, setup and maintenance policies are of the "Modified Hedging Corridor Policy or MHCP" type, more adapted in practice than those of the "Hedging Corridor Policy or HCP" type. An analysis of multi-product production systems, in the presence of setup and preventive maintenance, based on an extension of the "Multi-Hedging Point Policy or MHPP " policy is discussed in Assid et al. [9].
In order to determine the optimal production policies for different classes of manufacturing systems subject to breakdowns and random repairs, Berthaut et al. [10] have developed preventive maintenance strategies regardless of production. They have shown the benefits of combining maintenance strategies with production control policies. This combination makes it possible to simultaneously optimize the parameters related to the production and preventive maintenance policies, and also to study the variation of the inventory levels of parts produced according to the interventions of the preventive or corrective maintenance (Boulet et al. [11]). In this context, Gharbi et al. [8] combined the critical threshold control policy with the age replacement policy (ARP). The manufacturing system studied consists of a single machine producing a single type of finished products and whose time and cost of maintenance interventions are non-negligible. Boulet et al. [12] extended the policy of Gharbi et al. [13] by using a modified Modified Block Replacement Policy (MBRP). Based on the total cost incurred, Berthaut et al. [10] compared the effect of cost parameters on the MBRP. block replacement policy (BRP) and ARP strategies, when combined with the conventional critical threshold policy (Hedging Point Policy or HPP).
In the literature, no work has addressed the combination of production control and setup policies with corrective maintenance strategies for a machine producing two types of products. Note that the duration of corrective maintenance interventions is non-negligible. The main objective of this study is based on the work of Berthaut et al. [14], Ayed et al [2], Kenne et al. [15] and Dellagi et al. [1] and consists of proposing a suboptimal joint control policies that simultaneously integrates production, setup, and corrective maintenance for this class of manufacturing systems. Due to the complexity of the problem, a new modelling approach has been proposed based on the scientific literature, notably by extending the Bai and Elhafsi [16] approaches, Gharbi et al. [8] and Berthaut et al. [14] in the case of systems degraded by setup operations. This policy will combine production and setup planning with a corrective maintenance control strategy based on the number of setup operations. This combination allows us to achieve even more savings and increase the level of customer service. This concept aims to better control the transition rate from failure mode to operational mode (repair rate) when setup operations require the interruption of production activity for a non-negligible duration and degrade the dynamics of the system.
The rest of the article is organized as follows: Section 2 presents the general context of the study. Section 3 presents the definition of the problem of joint planning of production, setup, and corrective maintenance. The optimality conditions and the numerical approach are presented in section 4. Section 5 describes the numerical example and the analysis of the results. The article ends with a conclusion in section 6.

II. NOTATION AND MAIN ASSUMPTIONS
In this section, we present the notations and the main modelling assumptions of the optimal control problem under study.

A. Notations
The following notations will be used in the rest of this article for all i,j I=1,2: Pi : Type of products; ij : Setup time to switch the production from Pi to Pj; Kij : Setup cost to switch the production from Pi to Pj; di : Demand rate for products type Pi; x(t) : Vector of inventory levels for products type Pi; ui(t) : Production rate for products type Pi; + : Maximal production rate for products type Pi; Zi : Optimal inventory threshold level for products type Pi; (t) : Stochastic process describing the dynamics of the machine; Sij : Setup policy to switch the production from Pi to Pj;  : Transition rate from mode  to mode ; − : Backlog cost for products type Pi; + : Inventory cost for products type Pi; ρ : Discount rate;

B. Modelling Assumptions
The main assumptions on which the mathematical model presented in this article is based are listed below.
-The machine can produce two types of parts with a time and cost of setup not insignificant when reconfiguring the machine to switch production from one type of parts to another.
-The machine is subject to random breakdowns and repairs that can be described by stochastic process with known constant failure rates.
-When the machine breaks down, a corrective maintenance activity is immediately undertaken to repair it and put it back into operation. The machine repair activities are minimal and restore its age to as-bad-as-old conditions.
-The repair time depends on the number of setups and increases each time a setup activity is performed on the machine.
-The repair rate of the machine depends on the number of setups and decreases each time a setup activity is performed on the machine. Hence, the corrective maintenance activity on the machine is imperfect and characterized by the fact that the repair time increases gradually with the number of setups.
-The customer demand rate for each product is constant and known.
The maximum production rate of the machine for each product is known and constant.

III. PROBLEM STATEMENT
The production system considered represents a common problem in the manufacturing industry. The system consists of a single non-flexible machine producing two types of products. This machine is subject to random breakdowns and repairs that can lead to backlog situations. In order to switch the production from one type of products to another, a time of reconfiguration of the machine is associated with the setup activity. The production rate of the first and second product, the decision to proceed with the setup of the first product to the second product and the second product to the first product, in normal operation, constitute the decision variables. These decision variables respectively influence the inventory and system capacity. The structure of the system studied is shown in Fig. 1. In the proposed optimization model, we integrated the notion of repair rate control to take into account the degradation of the system with the setup activities and to improve the availability of the machine. The stochastic process resulting from this integration is then a transition ratecontrolled process (non-homogeneous Markov process). The system is described by a hybrid state consisting of a continuous part (inventory dynamics) and a discrete part (machine modes). The dynamics of the stock is described by the differential equation given by the expression (1). For both types of products P1 and P2, production rates are designated by 1 (. ) et 2 (. ) such 0 ≤ 1 (. ) ≤ 1 + and 0 ≤ 2 (. ) ≤ 2 + ,with 1 + and 2 + describing the maximal production rate of P1 and P2 respectively. The production must satisfy the equation (1) ,u2(t)) and d = (d1,d2). The system is described by a hybrid state consisting of a continuous part (inventory dynamics) and a discrete part (machine modes). The dynamics of the stock is described by the differential equation given by the expression (1). For both types of products P1 and P2, production rates are designated by 1 (. ) et 2 (. ) such 0 ≤ 1 (. ) ≤ 1 + and 0 ≤ 2 (. ) ≤ 2 + ,with 1 + and 2 + describing the maximal production rate of P1 and P2 respectively. The production must satisfy the equation (1) with where d1 and d2 are the demand rates of products type P1 and P2. The state transition diagram of the non-homogeneous Markov process associated with the dynamics of the machine is shown in Fig. 2. With  describing the transition rate from mode  to mode , we can set 12 = and 21 = .
When the repair rate is controlled and we denote by (. ) the associated decision variable, then 21 = . The system dynamics can be described by the stochastic process The matrix of transition rates corresponding to the dynamics of breakdowns and repairs of the machine is then given by the expression (3).
( ) 12 12 r rr qq Qu uu and denote the minimum and maximum repair rates of the machine respectively.
The decision variable of the joint optimization problem, considered in this paper, are as follows: • The production rates u1 and u2 for product types P1 and P2, • The sequence of setup denoted  = τij which will allow us to know when to switch the production from products type Pi to products type Pj . • The repair rate of the machine, with values in [ , ] according to the actual repair rate which depends on the number of setups according to the following model: where n is the number of setup and b1, b2, k2 and  are given constants. The expression (4) corresponds to the representation of a decreasing repair rate as a function of the number of setups that degrades the repair process of the machine. This representation is inspired by the increasing failure rate model used by Love et al. [17] and Dehayem-Nodem et al. [18] in the case of degradations that affect the reliability of the machine. Figure 3(a) shows the evolution of the repair rate according to the number of setup activities and figure 3(b) represents 21 ( ) defined by equation (4) Fig. 3(a) presents two zones associated with these values. In zone 1, the repair could be done at the minimum repair rate r U when the actual repair rate has not reached the switching limit due to the variation of the repair time depending on the number of setups. In zone 2, since the repair rate has decreased significantly, corresponding to a higher average repair time, the repair could be done at the maximum repair rate r U . Using the illustration of Fig. 3(a), we propose to jointly control production, setup, and corrective maintenance (Zone 1 vs. Zone 2). This is a new problem (not treated in the literature) and which uses stochastic processes at controlled rates, as mentioned above. In this case, the control issue is the control of the repair rate, combined with the planning of production and setup. In this context, we must demonstrate the existence of the optimal solution (optimality conditions) and then develop the corresponding HJB equations. For this optimization problem, the admissible control domain is given by the following equation: The instantaneous cost associated with the optimization problem considered is given by the expression (6): By setting, The setup cost is defined by the expression (7): The total discounted cost over an infinite horizon can be defined by the following expression (8): The optimization problem that we propose to solve is to find an admissible decision belonging to the set A (.) defined by equation (5). The value of the cost function when the optimal control policy is adopted is represented by the value function given by equation (9).
To demonstrate the convexity of the value function, we will formulate three hypotheses and state the convexity theorem.
For the purpose of simplification, the value function ( , , ) can be expressed in terms of ( , , ) and ( − , 1) as follows: The properties of the value function described by (9) leads us to optimality conditions described by the Hamilton-Jacobi Bellman equations (HJB) presented in the next section.

IV. OPTIMALITY CONDITIONS AND NUMERICAL METHODS
In the context where setup activities degrade the machine repair process, and under the assumption that "setup is done in both machine modes and can be run concurrently with corrective maintenance", we have defined and developed the following probabilities of occurrence: 12 -the probability that the machine is in mode 1 at the end of setup if it was in mode 1 at the beginning of setup: 12 12 -the probability that the machine is in mode 2 at the end of setup if it was in mode 1 at the beginning of setup: 12 22 -the probability that the machine is in mode 2 at the end of setup if it was in mode 2 at the beginning of setup: 12 21 -the probability that the machine is in mode 1 at the end of setup if it was in mode 1 at the beginning of setup: 12 With these expressions of occurrence probabilities and giving the convexity of the value function (theorem 1), the HJB equations describing the optimality conditions are as follows.
To approximate the solution of the HJB equations and to solve the optimal control problem under study, we used numerical methods based on the Kushner's approach (Kushner and Dupuis [19]). For more details on this approach, the reader is referred to the work of Boukas and Kenné [6], Hajji et al. [7], Kibouka et al [20] and Polotski et al. [21]. Let h denote the length of the finite difference interval of the variable x. The value function ( , ) is then approximated by ℎ ( , ) and the partial derivatives ( ) ( , ) by: By introducing the expression (16) into the HJB (15), (17) and (18) ➢ Mode 2 In the case of a machine producing two types of parts, the discretized (17) and (18) are transformed into two equations each, giving the (19), (20), (21) and (22) These four expressions (19), (20), (21) and (22) differ from those developed in the literature (Kibouka et al., [20]). This difference is justified by the integration of the control of the repair rate of the machine according to the number of setups activities.

V. NUMERICAL EXAMPLE AND RESULTS ANALYSIS
In this section, we present the numerical results for the example described in this section. In order to characterize the optimal production, maintenance and setup policies, we simulated four different cases. In order to focus our efforts on an effective characterization of the control policies, our analyzes focused on two products with the same parameters. Only the technico-economic parameters related to the products (setup time) have been changed as part of the analysis of the results. These parameters are directly related to the total cost function given by (8). The operating parameters set for all case studies are presented in Table I. For setup parameters, we considered three cases (θ12= θ21=0,5; θ12=θ21=1 et θ12 = θ21 =1,5) with the results obtained  Fig. 4-7, represent the production and maintenance policies for the first case (θ12=θ21=0.5) when the machine is setup for product P1. Fig. 8-11, represent the production and maintenance policies for the second case (θ12=θ21=1), when the machine is setup for product P1. Fig. 12-15 illustrate the production policies related to product P1 and product P2 for the third case (θ12=θ21=1.5). On the other hand, Fig. 16-19 illustrate the maintenance policies with respect to the products P1 and P2, all relating to the third case.            All these illustrations are the starting point for the determination phase of the optimal control policies to be used for the control of the production system under consideration. The other cases made it possible to carry out experiments and sensitivity analyzes in order to reveal the effects of variation of some parameters on the production, setup and maintenance policies. The analysis of the results obtained shows that the optimal production policy is to produce the part type i (i = 1.2) at the maximum production rate as long as its stock level remains in the production zone, and that the machine is configured for the same part type (Zone I in the figures). Outside of this zone, the production policy orders to stop the machine and to produce nothing by fixing the production rate equal to zero or by carrying out the setup. It should be noted that the obtained optimal policy is of the "Hedging Point Policy" type; which means that the production rate can be equal to one of three levels: (i) minimal level which results in a zero production rate, (ii) the extreme level which results in a maximum production rate; (iii) an intermediate level which results in production at the rate of demand.
Indeed, in the case of the identical products, Fig. 4, 6, 8, 10, 12, 13, 14 and 15 show that the optimal production policy recommend to produce at the maximum production rate in zone I belonging to the positive space of stocks. This means that at any time, the level of stock to be reached changes depending on the stock level of the other product. This level is not necessarily the optimal level (Hedging point); which shows that the production of the two products is interconnected in the case where the setup is not negligible. This obtained threshold policy requires the system to produce at the maximum rate as long as the stock of the current product is below a target level regardless of the stock status of the I   I   III   II   III   II   III   III other product. The resulting production policy is then expressed as follows: ( ) ( ) where 1 and 2 denote the stock thresholds of product types P1 and P2. Since the repair rate depends on the number of setups (according to (4)) and as shown in Fig. 3, the optimal stock levels of both types of products increase with the number of setup operations already performed on the machine (the higher the number of setups, the higher the optimal inventory level also increases). Hence, as the number of setup operations increases, the average repair time also increases, and thus the repair rate decreases. The system is then forced to produce a lot to increase the optimal stock level when it is in operational mode. When the system goes into failure mode, it will remain there for a long time (for duration equal to the MTTR (n)). This could lead to the risk of both products entering the shortage if the system spends much time in the failure mode (under repair). The proposed control policy for corrective maintenance stipulates that a major repair (As Good As New or AGAN) must be done for a large number of setup operations and that a minimum repair is made before this number. This corrective maintenance policy is described by the following equation: where zone II and zone III are the areas where the corrective maintenance is done at the minimum and maximum rates respectively for a value of (n). When the repair rate is equal to , the corrective maintenance is done at the rate The corrective maintenance policy divides the inventory grid into two regions. Looking at Fig. 5,7,9,11,16,17,18 and 19, it appears that zone III is always dominant in the negative region of the stock level (zone in which the repair is made at the maximum rate). On the other hand, in the positive stock region, zone II (corresponding to a minimum repair rate) is predominant and increases when (n) increases.
The optimal setup policy divides the inventory grid into three regions as in Fig. 20 and 21: -Region of production at the maximum rate for the product being produced (RI). -Region of the setup of the machine to the other product (RII). -Region of zero production rates (RIII).  These product setup policies from type 1 to product type 2 and from product type 2 to product type 1 are given by expressions (25) To validate the structures of the control policies obtained, we have shown that the characteristic parameters of these policies move in the right direction when certain parameters of the studied system vary. We first focused our efforts on the technical parameters (setup time) deemed most relevant; and the others (demand, production, outage, and repair rates) are RII RIII RI RI RIII RII kept constant and correspond to the values in Table I. The values of the obtained optimal stock levels, for n = 10 are shown in Table II. The values of the critical stock thresholds as a function of n and for θ12 = θ21 = 1.5, are shown in Table  III.
The values of the parameters 1 , 1 , 2 2 and 2 can be approximated graphically from Fig. 20 and 21 for a given setup number (n). It is interesting to note in Fig. 20 and 21 that the setup policies depend on the number of setup (n). For n = 4, 6, 10 and 15, it is clearly seen in Fig. 20 and 21 that the switching zones of product P1 to P2 or product P2 to product P1 change with the number of setups, thus giving more appropriate policies than those presented in the literature.
To validate the structures of the control policies obtained, we have shown that the characteristic parameters of these policies move in the right direction when certain parameters of the studied system vary. We first focused our efforts on the technical parameters (setup time) deemed most relevant; and the others (listed in Table I) are kept constant. The values of the obtained optimal stock levels, for n = 10 are shown in Table II. The values of the stock thresholds as a function of (n) and for θ12 = θ21 = 1.5, are shown in Table III TABLE III: PRODUCTION THRESHOLDS LEVEL DEPENDING ON THE  NUMBER OF SETUPS  n  11  12  13  14  15  16  17  18  19  20  Z1 et Z2  3  6  6  6 6.5 6.5 6.5 6.5 6.5 6.5 The obtained control policies, described by (23)-(26) are based on the mean time to failure (MTTR) which depends on the number of setup operations (n) already executed on the production system. Hence, the consideration of the number of setups in this work leads to new structures of the control policies including corrective maintenance.

VI. CONCLUSION
This paper has allowed us to introduce corrective maintenance strategies based on the number of setups. We considered a single-machine system capable of producing two types of finished products with a significant setup time and cost. Such a machine is prone to random failures and repairs. We have found a new control law structure that allows joint control of production, setup, and corrective maintenance. This work aimed to study the effect of a wide range of system configurations on the optimal parameters of the various control policies considered, minimizing the total cost incurred. The complexity of the problem has led us to analyze simpler situations up to the consideration of the interactions of the control policies presented. This study has mainly shown that when the system is degraded according to the setup activities, it is possible to improve the performance of the system by more appropriate production policies, characterized by a joint structure. The policies obtained are of the type of modified hedging point policy, given that they depend on the number of setup activities. The results in this paper make a significant contribution to the control literature of production systems due to the fact that such a problem has never been addressed.