A coordinated operation method of wind-PV-hydrogenstorage multi-agent energy system

0 Introduction

1) Research background

Hydrogen energy is considered a promising secondary energy source for realizing clean,low-carbon,and sustainable energy transformation;therefore,many countries around the world have mapped their hydrogen development paths to promote the development of hydrogen-related technologies and industries [1].

In recent years,with the rapid growth in wind power and installed PV capacities,the utilization of renewable power has become of significant importance.Renewable power has intermittence and randomness,which pose great challenges to the stable and reliable operation of power systems [2].In this context,the conversion of renewable power to hydrogen is considered a promising method for promoting the utilization of renewable power and achieving carbon neutrality.By converting wind or PV power to hydrogen energy,intermittent and probabilistic renewable power can be stored on a large scale and in the long term,as well as transported over long distances.Moreover,green hydrogen can play an important role in the decarbonization process of hard-toabate sectors,such as heavy and transportation industries,which promotes carbon neutrality [3].The role of hydrogen in promoting the utilization of renewable energy and carbon neutrality has been thoroughly investigated and discussed.For example,Zhang et al.[4] proposed a power system simulation model to analyze the development pathway and influencing factors of hydrogen energy storage to accommodate the growth of renewable energy.Li et al.[5] summarized and discussed the latest approaches to green hydrogen as a potential source to promote the utilization of renewable energy.Zheng et al.[6] analyzed the nexus between hydrogen technology and carbon neutrality and examined the role of hydrogen technology in achieving carbon neutrality targets in the six top economies involved in innovative and polluting hydrogen technologies.Yang et al.[7] employed a multi-model comparison method to analyze and discuss the role of hydrogen in China’s carbon neutrality pathways.

The wind-PV-hydrogen-storage integrated energy system is composed of renewable power generators,hydrogen production systems,and energy storage systems,where renewable power can be stored and converted to hydrogen.It is considered a promising energy unit to promote renewable power utilization and achieve carbon neutrality [8].However,the investment costs of wind power,PV,energy storage,and power-to-hydrogen systems are significant,and wind power plants,PV plants,energy storage stations,and hydrogen production stations usually require different investors.Meanwhile,transactive energy or peer-to-peer (P2P) power trading technologies can connect power suppliers and consumers [9].Therefore,the distributed energy system is gradually exhibiting multi-agent characteristics.Because different agents normally belong to different energy types or energy utilization links and usually pursue self-interest rationally,the coordinated operation issue of this type of multi-agent energy system is different from that of the traditional single-agent distributed energy system;hence,further investigation is urgently needed.

2) Literature review

Currently,wind-PV-hydrogen-storage energy systems are drawing wide attention,and research is mainly focused on economic operation [10-14],optimal control [15-19]and planning issues [20-24].To build a more efferent energy system,some studies have focused on the operation or planning issues of multi-agent energy systems from the perspective of cooperative games,the key problem of which is benefit distribution.For example,to promote the consumption of wind power,Wu et al.proposed a cooperative operation model for wind power plants and multi-hydrogen fueling stations,and Nash bargaining theory was employed to address energy trading and benefit-sharing issues [25].Li et al.proposed a collaborative operation method that considered P2P power trading to promote interactions between the power network and hydrogenfueling stations,which could reduce operational costs[26].Ma et al.proposed a cooperative operation method for a wind-solar-hydrogen multi-agent energy system to improve the operational revenues of wind and PV agents and reduce hydrogen production costs [27].Hamed et al.proposed a multi-objective day-ahead cooperative operation method for PV and hydrogen systems in a distribution network,and the proposed model was formulated as a mixed-integer linear programming problem and solved via centralized optimization [28].Wu et al.designed a marketbased demand response trade mechanism based on an auction model,and the bargaining game method was used to inspire demand response agents and power-to-hydrogen agents to participate in the demand response trade [29].In addition,to reduce investment and operational costs,Ma et al.proposed a cooperative planning and operation method for a wind-hydrogen-heat multi-agent energy system,and the Nash bargaining method was employed to distribute the cooperative profits among the multi-agents [30].

Furthermore,protecting the privacy of each agent when coordinating their operation strategies is an important issue when constructing an efficient and safe multi-agent energy system.The alternating direction method of multipliers(ADMM) is a distributed algorithm that is performed in a distributed interactive manner,which is widely applied to protect the privacy of multi-agent energy systems [31].For example,the adaptive uncertainty quantification ADMM method was adopted in [32] to solve the P2P trading problem,considering the carbon trading of largescale interconnected microgrid systems,which is efficient in protecting the privacy of each microgrid agent and calculating the power-trading problem.In [33],an efficient distributed algorithm was proposed based on the ADMM to solve the distributed transactive energy trading problem of multiple prosumers and protect their privacy.In [34],a P2P power and hydrogen trading model was proposed for an integrated energy system,where the ADMM was adopted to solve the trading problem and protect the privacy of each participant.In [35],the ADMM was applied to solve the nested P2P energy trading problem of networked microgrids and to protect the privacy of each microgrid.

As discussed above,the existing research has laid a good foundation for the cooperative operation or planning issues of wind-PV-hydrogen-storage multi-agent energy systems.However,most existing methods consider the entire energy system as one agent or investigate the operation,control,or planning problems from the viewpoint of the microgrid,which ignores multi-agent characteristics.Although some studies have investigated the cooperative operation or planning issues of multi-agent energy systems from the viewpoint of cooperative games,the key problem of the cooperative game method is benefit distribution,and the cooperative operation model cannot be used to directly solve the coordinated operation problems of the wind-PVhydrogen-storage multi-agent energy system.

The coordinated operation of the wind-PV-hydrogenstorage multi-agent energy system is complex and not yet fully understood in the following aspects: 1) how to coordinate operation strategies among renewable power agents,energy storage agents,and hydrogen agents,and 2) how to clarify the complex power trading and energy management problem and protect the privacy of each agent simultaneously.Therefore,coordinating the operation and electricity trading strategies of multiple agents to realize efficient economic operation and energy management remains an open problem.

3) Contributions and organization of this paper

To overcome these shortcomings,this study proposed a coordinated operation method for a wind-PV-hydrogenstorage multi-agent energy system that can exploit the advantages of each agent and coordinate their operation strategies to achieve efficient and economic operation.The main contributions of this study are summarized as follows:

(1) A coordinated operation method was proposed for the wind-PV-hydrogen-storage multi-agent energy system that can effectively motivate multiple agents to trade power with each other and improve their operational revenues.

(2) A distributed interactive algorithm based on the ADMM was proposed to solve the coordinated operation strategies of each agent and protect the privacy of each agent.

(3) The proposed coordinated operation method was compared with the traditional operation model and other multi-agent energy systems with different structures,and its advantages,applicability,and effectiveness were demonstrated.

The remainder of this paper is organized as follows.Section 1 presents the coordinated operation models for multi-agent energy systems.Section 2 proposes a distributed coordinated operation mechanism for a multi-agent energy system and a distributed interactive algorithm.Section 3 presents a case study,and Section 4 presents the conclusions.

1 Coordinated operation model for a wind-PVhydrogen-storage multi-agent energy system

Figure 1 illustrates a representative wind-PV-hydrogenstorage multi-agent energy system composed of a wind power plant,PV plant,hydrogen production station,and energy storage power station.In a multi-agent energy system,P2P power trading between different agents is permitted so that they can coordinate their operation strategies through P2P power trading.The four agents exchange power-trading information through the communication network by sending the expected trading power to their trading partners.

Fig.1 Schematic of wind-PV-hydrogen-storage multi-agent energy systems

Figure 2 presents a schematic of the potential powertrading paths for each agent.Each agent can choose to trade either with a power company or directly with other agents.Specifically,the energy storage agent can trade power with the power company at the time-of-use (TOU) price of the power company or directly trade with other agents at bilateral trading prices.Wind power and PV agents can sell power to the power company at their feed-in tariffs or trade directly with energy storage or hydrogen agents at bilateral trading prices.The hydrogen agent can purchase power from the power company at the TOU price of the power company or directly purchase power from other agents at bilateral power-trading prices.Therefore,the key concern of the coordinated operation problem of a multi-agent energy system is how to optimize the operation strategies of each agent and their power-trading strategies with the power company and other agents to maximize their operational profits.

Fig.2 Schematic of the potential power trading paths of each agent

1.1 Optimal operation model of the energy storage agent

The energy storage power station can be considered a shared energy storage of the multi-agent energy system,because the energy storage agent can trade with all other agents.As a rational individual,the operational objective of an energy storage agent is to maximize its operational revenue,which is formulated as follows:

The constraints of the energy storage agent include the power trading constraints (2) and operation constraints (3)of the energy storage power station.

1.2 Optimal operation model of the wind power and PV agents

Wind power and PV agents can sell power to power companies at their feed-in tariffs,to the energy storage and hydrogen agents at bilateral trading prices,or even abandon redundant power.Therefore,the optimal operation models of the wind power and PV agents are similar;the operation objective of the wind power or PV agent is to maximize the operational revenue,and the objective function can be formulated as (4).

The operation model of the wind power and PV agents mainly contains the trading constraints (5).

where denotes the maximum feed-in power of the wind power or PV agent.

1.3 Optimal operation model of the hydrogen agent

A hydrogen agent can purchase power from power companies,wind power agents,PV agents,or energy storage agents to produce hydrogen and meet the hydrogen demand.The operational objective of the hydrogen agent is to maximize the operational revenue,which can be formulated as the negative of the operational cost,that is,(6).

The operational constraints of the hydrogen agent include power-trading constraints and operational constraints of the hydrogen production system.The powertrading constraints are formulated as (7).

As illustrated in Fig.3,the power of the hydrogen system is mainly composed of an electrolyzer,a hydrogen compressor,and a hydrogen storage tank.The electrolyzer electrolyzes water to produce hydrogen.The constantpressure hydrogen is then compressed into high-pressure hydrogen and stored in a hydrogen tank.

Fig.3 Diagram of a power-to-hydrogen system

1) Electrolyzer operation model

The operation model of an electrolyzer can be formulated as follows [36]:

2) Hydrogen compressor operation model

A hydrogen compressor compresses hydrogen into highpressure hydrogen for storage.The operational constraints of the compressor are formulated as (9) [37].

3) Hydrogen storage tank

The state of the stored hydrogen mass in the hydrogen tank is formulated as (10) [38]:

1.4 Coordinated operation and market clearing model of the multi-agent energy system

In Section 1.2.,the uncertainty of the wind power output of the PV agent was modeled using typical scenarios and probabilities.Thus,it is clear that the following formulas hold true:

From the perspective of social welfare maximization,the optimal operation objective of a wind-PV-hydrogenstorage multi-agent energy system is to maximize the total operational revenue W.The coordinated operation and market-clearing model of the entire multi-agent energy system can then be formulated as follows:

The model in (12) can be solved using a centralized optimization method;however,it requires the operation information of all agents,which violates the privacy of the agents.Moreover,from (12),we can see that the trading paymentsandcancel out when calculating the total operational revenue W;hence,it is difficult to calculate the bilateral trading prices and payments.Therefore,a novel method that can simultaneously protect privacy and provide clear P2P trading results is required.

2 Distributed coordinated operation mechanism of the multi-agent energy system

Then,the coupling variables can be decoupled,and the objective function of model (12) can be reformulated with the auxiliary variables as follows:

Constraint (2) can be reformulated as (15) using the auxiliary variables.

The constraint in (7) can be reformulated using the auxiliary variables as follows:

Then,model (12) can be reformulated as (17).

where ρ ＞0 is the penalty parameter.The augmented Lagrange function in (18) can be separated with respect to the decision variables of each agent.Therefore,the original model (17) can be decomposed into subproblems for the energy storage,wind power,PV,and hydrogen agents based on the augmented Lagrangian function of (17) and the ADMM decomposition method [39].Thecan be interpreted as the trading prices between the energy storage and hydrogen agents,the PV and energy storage agents,the wind power and hydrogen agents,and the PV and hydrogen agents,respectively [31,40].

• The distributed operation subproblem for the energy storage agent

• The distributed operation subproblem for the wind power agent

• The distributed operation subproblem for the PV agent

• The distributed operation subproblem for the hydrogen agent

To protect the privacy of each agent,a distributed iterative algorithm,Algorithm 1,was proposed based on the ADMM.Algorithm 1 indicates that only a limited information exchange is required between different trading agents.They only need to report their expected trading powers to their trading partners without disclosing private information on operations or equipment.Specifically,the energy storage agent reports only its expected trading powerto the wind power,PV,and hydrogen agents,respectively.The wind power agent only sends its expected trading powerto the energy storage and hydrogen agents,respectively.The PV agent only reports its expected trading powerandto the energy storage and hydrogen agents,respectively.The hydrogen agent sends only its expected trading powerto the energy storage,wind power,and PV agents,respectively.

Table 1

3 Case study

As described in this section,the wind-PV-hydrogenstorage multi-agent energy system illustrated in Fig.1 was tested to demonstrate the effectiveness of the proposed coordinated operation method.The stochastic programming method was employed to simulate the uncertainties in the output power of PV and wind power plants [30,41].The Monte Carlo simulation method was used to generate 1000 output power scenarios for the PV and wind power plants,and six typical output power scenarios and their probabilities were obtained using the k-means clustering method [42].Typical output power scenarios of wind and PV power plants are illustrated in Figs.4 and 5,respectively.The parameters for the wind power,PV,energy storage,and hydrogen agents are listed in Table 2.The industrial and commercial TOU power prices are listed in Table 3.In this study,the power purchase prices for the energy storage and hydrogen agents,and the feed-in tariff of the energy storage agent were all set as the power prices given in Table 3.

Table 2 Parameters for the multi-agent energy system and algorithm [27,28,40]

Table 3 Industrial and commercial TOU power prices [43]

Fig.4 Typical output power scenarios of a wind power plant

Fig.5 Typical output power scenarios of a PV power plant

Fig.6 Typical hydrogen load scenario of a hydrogen production station

3.1 Simulation results and discussions

In this section,the convergence of the proposed distributed interactive algorithms is analyzed and discussed.The power-trading prices and coordinated operation strategies are then discussed.

3.1.1 Convergence analysis of the proposed distributed interactive algorithm

The test model of the wind-PV-hydrogen-storage multiagent energy system had four agents;the simulation time step was one hour,and the simulation duration was 24 h.The numbers of variables and constraints of the distributed optimization model for the energy storage agent were 216 and 265,respectively.The number of variables and constraints of the distributed optimization model for the wind power and PV agents were 432 and 144,respectively.The number of variables and constraints of the distributed optimization model for the hydrogen agent were 192 and 265,respectively.The simulation was conducted on a PC with an Intel Core i7-8750H CPU @2.20GHz with 16 GB RAM,the distributed algorithm was performed on MATLAB software,and each distributed operation subproblem was solved with the Gurobi optimizer.

Fig.7 illustrates the convergence characteristics of the operational revenue and cost of each agent.The algorithm converged after 50 iterations,and the convergence speed of the proposed distributed iterative algorithm was fast.The convergence error tolerance was less than 10-4.Therefore,the proposed distributed iterative algorithm exhibited good convergence and could effectively solve the coordinated operation problem and protect the privacy of each agent.

Fig.7 Convergence characteristics of the operational revenue and cost

3.1.2 Power-trading prices

Fig.8 illustrates the power-trading prices of the four agents,which were determined by the Lagrangian multipliers in (18).Fig.8 (a) shows that the power-trading prices between the PV/wind power and hydrogen agents during the valley periods were lower than the feed-in tariff of the PV/wind power agent.The power-trading prices between the PV/wind power and hydrogen agents during peak and flat periods were between the TOU prices and the feed-in tariff of the PV/wind power agent.Furthermore,the power-trading prices between the PV/wind power and hydrogen agents were zero when there was no power trading.The power-trading prices between the PV/wind power/hydrogen agent and the energy storage agent shown in Fig.8 (b) had characteristics similar to those in Fig.8 (a).In summary,the bilateral trading prices of the agents are beneficial for improving their profits compared to the TOU price or feed-in tariff.

Fig.8 Power-trading prices between the PV,wind power,hydrogen,and energy storage agents

3.1.3 Coordinated operation strategies

Figs.9,10,11,and 12 illustrate the coordinated operation strategies of the wind power,PV,hydrogen,and energy storage agents,respectively.The following characteristics can be determined by analyzing the coordinated operation strategies.

Fig.9 Coordinated operation strategies of the wind power agent

Fig.10 Coordinated operation strategies of the PV agent

Fig.11 Coordinated operation strategies of the hydrogen agent

Fig.12 Coordinated operation strategies of the energy storage agent

During the valley periods (01:00-07:00),the feed-in tariffs of wind and PV power (4.876￠/kWh and 5.572￠/kWh,respectively) were higher than the power prices of the power company;therefore,the wind power and PV agents mainly sold their power to the power company to improve operational revenue,and the hydrogen agent mainly purchased power from power company to reduce operational cost.In some periods (2:00,4:00-7:00),limited by its maximum trading power,the wind power agent sold surplus power to the hydrogen or energy storage agent to avoid curtailment.

During the peak and flat periods (8:00-24:00),the TOU prices were higher than the feed-in tariffs of wind power and PV power,while the power-trading prices between the PV/wind power and hydrogen/energy storage agents were between the TOU prices and the feed-in tariff of the PV/wind power agent;therefore,the wind power and the PV agents mainly sold their power to the hydrogen agent or energy storage agent to improve their operational revenues and reduce the hydrogen production cost.Fig.10 shows that all the required power of the hydrogen agent was purchased from the wind power and PV agents during the peak periods (8:00-11:00,18:00-22:00).However,during flat periods (12:00-17:00 and 23:00-24:00),the hydrogen agent purchased power from wind power agent,PV agent,power company,or energy storage agent to satisfy its power demands.

The energy storage agent mainly charged in valley or flat periods and discharged during peak periods for arbitrage.Specifically,because the power prices of the power company were low during valley periods,it mainly purchased power from power company for charging purposes.During some peak periods (11h,19h),it also charged by purchasing power from the wind power and PV agents to reduce charging costs.During flat periods (12-17h,23h),power was purchased from wind power agents,PV agents,or the power company for charging.The energy storage agent mainly discharged during peak periods (8-10h,18h,20-22h) and sold its power to the power company to make a profit.Furthermore,through coordinated operation strategies,there were no curtailments in wind and PV power.The energy storage states illustrated in Fig.13 demonstrate the coordinated operational strategies of the energy storage agent.

Fig.13 Energy storage state of the energy storage agent

Therefore,the proposed coordinated model is effective in coordinating the operation strategies of wind power,PV,energy storage,and hydrogen agents,which can improve the operational efficiency of the entire multi-agent energy system.

3.2 Comparisons with other operation model and structures

As shown in this section,the proposed coordinated operation method was compared with the independent operation model and applied to other multi-agent energy systems to verify its applicability,scalability,and validity.

3.2.1 Comparisons with the independent operation model

The coordinated operation model was compared with the independent operation model.In the independent operation model,the energy storage,wind power,PV,and hydrogen agents trade only with the power company,and bilateral power trading between them is not considered.The operational revenues for each agent are listed in Table 4,where a negative value denotes an operational cost.Simulation results indicate that the proposed coordinated operation method can significantly improve the operational revenue of each agent.A detailed discussion is given below.

Table 4 Operational revenue of each agent in different operation models

The operational revenues of the wind power,PV,and energy storage agents improved significantly.The operational cost of the hydrogen agent was also reduced.Specifically,compared to the independent operation model,the operational revenue of the wind power agent in the coordinated operation model improved by 1,270.53$,that is,approximately 59.19%.The operational revenue of the PV agent in the coordinated operation model was 1,851.28$,which is a significant increase of approximately 233.28%compared to that of the independent operation model.The operational revenue of the energy storage agent was 3,132.28$,which is a significant increase of approximately 145.56% compared to that of the independent operation model.In addition,the operational cost of the hydrogen agent in the coordinated operation model decreased by approximately 16.75% compared to that of the independent operation model.Therefore,the proposed coordinated operation method improves the operational revenue of each agent.The main reason for this is that the hydrogen agent can purchase cheaper power from the wind power,PV,or energy storage agent to reduce the power cost,whereas the wind power and PV agents can sell their power at higher prices than their feed-in tariffs.Furthermore,the energy storage agent can charge cheaper wind or PV power and sell it to the power company at high prices for arbitrage.

Fig.14 shows the curtailments of the PV and wind power in the independent operation model.The total curtailment of PV power was approximately 2.956 MWh,which was approximately 13.26% of the total expected output power.The total curtailment of wind power was approximately 2.977 MWh,which was approximately 6.33% of the total expected output power.Therefore,the proposed coordinated operation method can reduce or avoid the curtailment of PV and wind power.

Fig.14 Operation strategies of the PV and wind power agents in the independent operation model

3.2.2 Comparisons with other multi-agent energy system

To demonstrate the applicability and scalability of the proposed coordinated operation model on different structures of multi-agent energy systems,the coordinated operation issues of two multi-agent energy systems with different structures were solved using the proposed coordinated operation method,and the operational revenues were compared with Case 1 (the wind-PV-hydrogen-storage energy systems illustrated in Fig.1).Case 2 did not include the energy storage agent,whereas Case 3 did not contain the hydrogen agent;the other structures and parameters were set to be the same as in Case 1.The structures of Cases 2 and 3 are shown in Fig.15.

Fig.15 Structures of other multi-agent energy systems

Table 5 lists the operational revenues of each agent for the different cases.Comparing the three cases and the independent model,the following results were obtained:

Table 5 Operational revenues of each agent in different cases

The operational revenues of the wind power and PV agents in Case 1 improved significantly compared to those in Case 3,and the improved ratios were approximately 52.62% and 59.80%,respectively.The operational revenue of the energy storage agent in Case 1 decreased by approximately 17.49% compared to that in Case 3.The total revenues of the wind power,PV,and energy storage agents in Cases 1 and 3 were 8,400.54$ and 7,193.64$,respectively;therefore,the total revenue in Case 1 improved by approximately 16.78%.Compared to Case 2,the operational revenues of the wind power and PV agents improved by 1.40% and 4.75%,respectively,and the operational cost of the hydrogen agent decreased by approximately 0.81%.In general,Case 1 achieved better operational revenues.

The coordinated operation model can be used to study multi-agent energy systems with different structures.The iterations of the proposed distributed interactive algorithm for Cases 2 and 3 were 42 and 24,respectively,demonstrating the applicability and scalability of the proposed coordinated operation model.Furthermore,the operational revenues of the agent in Cases 2 and 3 also improved significantly compared with the independent operation model (the operational revenues are listed in Table 3),and the feasibility and validity of the proposed coordinated operation method were further verified for different multi-agent energy systems.

4 Conclusion

A distributed coordinated operation method was proposed to coordinate the operation strategies of a multiagent energy system.Meanwhile,a distributed interactive algorithm was proposed to protect privacy and solve coordinated operation strategies.The following conclusions were drawn:

Compared to the traditional independent operation model,the proposed coordinated operation method can significantly improve the operational revenue of each agent.Specifically,the operational revenues of wind power,PV,and energy storage agents improved by approximately 59.19%,233.28%,and 145.56%,respectively.The operational cost of the hydrogen agent in the coordinated operation model decreased by approximately 16.75%.

The proposed coordinated operation method can also reduce or avoid the curtailment of PV and wind power.Bilateral power-trading prices can efficiently improve the profits of each agent and coordinate the operation strategies of multiple agents.

The proposed distributed interactive algorithm exhibits good convergence,which can protect privacy and efficiently solve coordinated operational strategies.In addition,the proposed coordinated operation method is feasible and applicable for addressing the operational issues of multiagent energy systems with different structures.

Acknowledgement

This work was supported by the Key Research and Development Program of Jiangsu Provincial Department of Science and Technology (BE2020081).

Declaration of Competing Interest

We declare that we have no conflict of interest.