Assessing Risk Management Capability of Public Sector Organizations Related to PPP Scheme Development for Water Supply in Indonesia

The success of Public Private Partnership (PPP) for water supply investment is inseparable from the capability of risk management of the parties within the project. This study investigates the risk management capability of Indonesian local public sector organizations that are potentially involved in PPP schemes for water supply. A risk management maturity model based assessment tool probing the culture, process, experience, application and partnership aspects is used in the survey. The model describes risk management capability in four levels (adhoc, initial, competent, excellent). The survey shows that their risk management capability is still in-average at the initial stage (level 2), meaning that the adopted risk management postures are mostly supported only by unstructured, ad-hoc and non-formal processes. The result of this study can help decision makers in choosing appropriate risk management methods and tools to be used by the local public authorities for managing risks in PPP schemes.


Introduction
The Government of Indonesia (GoI) currently seeks to involve private sector in water supply infrastructure investment, through Public Private Partnership (PPP) scheme.PPP is now seen as an option to overcome the gaps in investment, technology and expertise.By involving the private sector, it is expected that the operational efficiency of water supply utilities can be improved [1].However, international practices show that water supply infrastructure PPP projects, mostly dominated by concession scheme, are risky in terms of business, due to the uniqueness of its investment mode.Investment in water supply infrastructure requires large initial capital cost.The majority of its assets are underground, difficult to assess in order to ensure its proper valuation.[2].
From the investor's point of view, it is difficult to withdraw the investment once it is commenced.The future revenue stream that has to be secured by the investors is spread out over a long period of time (more than 25 years).Its security would also depend on the legal system and the good faith of the host country [3].These issues imply that for a water supply PPP scheme, good risk management within the project is inseparable from its success.
There are studies confirming that risk management is one of critical success factors in PPP schemes [4,5].From the public sector's point of view, in particular the PPP procurement authority, an effective risk management system is very important to improve the quality of decision-making.It may benefit the project by [6]: (i) enhancing performance delivery through early risk identification, structured and systematic consideration of risks, effective risk allocation and monitoring; (ii) providing an accountable platform for project planning (i.e.improved value for money through appropriate and efficient risk allocation and management, reduction in project costs through development of a standardized process of allocating risk); and (iii) supporting a robust decision making process.Within this context, risk management should not only be limited to the process of identifying, analyzing and planning for response, but it should also be integrated with the other processes within the organization.Implementing risk management should not be just as an ad-hoc process, despite the fact that some public sector practices in PPP transactions in United Kingdom [7] and Nigeria [8] still implement it on an ad-hoc basis, reflected by their unstructured applications.In many cases, however, due to the complexity of the issues, risks are often under-estimated and allocated to parties who do not have adequate knowledge and capabilities to manage them effectively, resulting in increased costs, project delays and services that fail to deliver value for money [9].
This study investigates the capability of risk management of the public authority bodies related to PPP scheme development in infrastructure provision in Indonesia, taking water supply infrastructure as the case study.It is based on the assumption that every organization should be able to measure its risk management capability, considering that there are many areas in risk management that need to be developed in a comprehensive manner to build a strong foundation towards its effective implementation [10].If the project eventually reaches financial closure and is then executed, risk management will continue to play a substantial role in project management over the life of the long-term contract.This concern is actually addressed by the the Government of Indonesia (GoI) through the Presidential Regulation No. 67 year 2005 concerning PPP in infrastructure provision [11], which stipulates that risks should be allocated to parties which are best to control and manage them in order to ensure project efficiency and effectiveness.The result of the investigation of the risk management capability of the organizations can be used for better defining which type of risk management methods and tools are more appropriate for the organizations in implementing PPP schemes.

Research Approach
This study utilized the Risk Management Capability Maturity (RMCM) Model [12] for assessing the risk management capability of the public sector organizations that deal with the development of PPP concession scheme for water supply in Indonesia.It is based on the understanding that a management maturity model can be used to describe the organizational readiness in introducing and/or implementing a particular management process and practice, as exemplified by concurrent engineering readiness in construction industry [13], stakeholder management relationship readiness in project management [14], and project management maturity in construction firm [15].Within this context, risk management maturity is defined as the description of the stages in the development of an organizational capability in introducing and implementing risk management process and practices, which can serve as a benchmark for other organizations [16].The assessment tool, questionnaire form and data analysis, and sample of respondents are described bellow.
The levels of maturity are defined based on four attributes of assessment, namely: culture, process, experience, and application.Level 1 refers to the situation where the organization is unaware of the need for risk management and has no structured approach to dealing with uncertainty.The risk process is reactive, with little or no attempt to learn from the experience in the past or to prepare for future uncertainties.Level 2 describes a situation where the organization is experimenting with the application of risk management, usually through a small number of nominated individuals within specific projects.Although the organization is aware, to a certain level, of the potential benefits of managing its project risks, there is no effective, organization-wide risk management implementation.Level 3 describes a situation where the organization has integrated risk management into their routine business processes and implements risk management in most, if not all, of its projects.Generic risk policies and procedures are formalized and widespread, and the benefits are understood at all levels of the organization, although they may not be consistently achieved in all cases.The highest is level 4, where the organization has established a risk-aware culture that requires a proactive approach to the management of risks in all aspects of the organization.Risk information is continually developed and actively used to improve all organization processes and to increase the probability of success in projects and operations.A standard risk management process (or processes) is (are) documented and used across the organization.
The proposed RMCM Model was developed based on the similar approach, where it divides risk management capability maturity into four levels, i.e. level 1 (ad-hoc), level 2 (initial), level 3 (competent), and level 4 (excellent) (Figure 1).RMM and RMML are recognized as generic models that can be applied to all organizations.The proposed RMCM Model adopts five attributes i.e. culture, process, experience, application, and partnership as measuring instruments by adapting partnership aspect from Stakeholder Relationship Management Maturity [14] as the fifth attribute.
For a PPP scheme that involves different stakeholders with different agendas i.e. government, investor, designer, and contractor, organization awareness in risk management is very crucial, as transferring and sharing risk are an important part of the business.The five attributes are sub-divided into eighteen subattributes as shown in Table 1.Table 1 contains the detailed structure of the model in matrix form, where its elements provide the indicators for each sub attributes representing the stages of risk management maturity.

Questionnaire Form and Data Analysis
A self-assessment questionnaire was used for collecting respondents' opinion on the current organizational risk management status, adapting a model for measuring the maturity level of organizational project management [17].The closed-type questionnaire comprises 18 questions reflecting the subattributes as depicted in the matrix (Table 1), each question has 4 possible answer statements (Table 2), each of them is given a score ranging from one (Level 1) to four (Level 4).
In practice, respondents were asked to choose one statement that best describes their current organization situation from the provided list of answers.As an exception, respondents may choose two adjacent options (as a compromise) if they are not sure in selecting only one option, i.e. options 1 & 2, 2 & 3, or 3 & 4. The maturity index value of each attribute is the average value of the scores for the attribute (the total of the scores from all the questions divided by the number of the questions in the attribute).

Sampling of Respondents
The

Survey Respondents
The survey was conducted in the period of January-March 2010.The surveyed respondents consist of 20 organizations representing the local government agencies (Office of Public Works and BAPPEDA) and PDAM.More than two-third of the survey were done by direct interview, while the rest were communicated through mail.Some interviews involved only one respondent, while in other interviews, more than one person were involved at the request from the interviewee in order to ensure the accuracy of the responses.Respondents come from five regions, i.e.Sumatera (15%), West Java (45%), Banten (15%), Central Java (10%), and East Java (15%).Distribution of organizations represented by respondents is BAPPEDA (30%), Office of Public Works (20%) and PDAM (15%).

Validity and Reliability of the Questionnaire Survey
The quality of the questionnaire is examined by conducting a (construct) validity and reliability analysis.The validity test is to see the consistency between components.If components are consistent with each other then the component is valid.Validity test is done by calculating the correlation between each question using the Pearson correlation formula [19].Validity test using SPSS software [20] showed that the statements contained in the self-assessment questionnaire have a strong correlation to each other.In this case about 33% of couple correlations (between sub-attribute) is very significant at confidence level > 95%.To ensure the reliability of used scale (1-4) that will produce the same results each time, the internal consistency method is used.Based on this method, the high and low reliability value is empirically demonstrated by a number called coefficient of reliability.Theoretically, reliability coefficients ranged from 0.00 to 1.00, despite the fact that the coefficient of 1.00 will never be achieved in the measurement, as human being as an object of psychological measurement is a source of potential inconsistency.Cronbach Alpha [20] is performed to test the internal consistency of the scale, the higher the alpha the higher the coefficient of reliability.Despite the absence of a specific rule on Alpha value representing the level of acceptance, a general rule that is widely used [20] is adopted: Excellent (0.9); Good (0.8), Acceptable (0.7); Questionable (0.6); Poor (0.5), and Unacceptable if alpha <0.5.The result of reliability test with SPSS software showed that the value of Cronbach Alpha for the overall assessment of sub-attributes contained in the self assessment questionnaire is 0.8839, indicating that the respondents' answers had good to excellent internal consistency.

Results and Discussion
Table 3 summarizes the obtained maturity index for each surveyed organization (the jurisdiction name of each organization is not disclosed for confidentiality).Table 3 also contains general information of the surveyed organizations such as number of employees, annual turnover, and the largest project value ever undertaken in the last five years.From the 20 organizations surveyed, three of them (Group C) were not in an advance stage of PPP implementation, but have conducted studies on opportunities for PPP scheme.
As shown in Table 3, the values of risk management maturity index of the surveyed organizations vary significantly.All (except for three organizations showing Level 3 for the culture attribute) are found to be below Level 3 (Competent).The average value of maturity index is 1.99, indicating that the risk management capability for all the surveyed organizations is at the Initial level (Level 2).The highest and lowest values are respectively 2.43 and 1.16, obtained by a local water company (PDAM) and a BAPPEDA.In general (as has been detailed in Table 1), it can be said that the risk management process in the surveyed organizations is more or less informal and its implementation has not yet been consistent.Some of the personnel are able to take a risk management initiative and implement some specific applications, but this capacity was obtained in a self-taught manner, with no formal qualification.Consequently, the use of methods and tools continues to be on an ad-hoc basis, and the collected risk information are not utilized in an optimal way; there is no risk registration system available.Although contingency is recognized as one of the risk response mechanism that is often opted, it is often done without a comprehensive analysis due to their limited knowledge in risk management methodology.The assessment results for each attribute are summarized bellow.

Culture
There are 12 (60%) organizations that have maturity index of culture attribute above the average (2.48), and three of them exceed level 3 (Competent).Based on their responses in the self-assessment questionnaire, the three organizations claimed that awareness of risk has been the concern of most of the staff and fully supported by the management.They also believe that risk management will add value to the organization.Conversely, two (10%) organizations scored below level 2 (Initial) for the attribute of culture.

Question
Possible answer Maturity level Sub-attribute: Perception on value of risk management "Which statement best describes current situation in your organization's, associated with the attitude of senior management to the idea of risk management?" No support from upper management, tend to be hostile 1 Upper management encourages passively (no real action to ensure its implementation).RM reports are not used 2 RM idea accepted and supported by upper management, direct participation to promote it.RM reports used, although not consistent 3 Top-down commitment to RM with leadership.Risk information used in decision-making.Organization provides reward for proactive efforts in term of RM 4 Sub-attribute: Risk management strategy and policy "Which statement best describes your current organizations, associated with risk management strategy and policy?" No RM strategies and policies 1 RM strategy and policies are not yet consistently applied 2 RM strategy policies are accepted and applied in all organizational processes 3 RM strategy and policies are regularly evaluated and refined to suit the various changes that occur

Process
The average of maturity index for attribute of process for all organizations was 1.98 (slightly below the Initial level), which indicates that all (risk management) processes are still informal, not yet consistent and limited to a specific project or activity (if risk management process is applied).This level also indicates that the existence of risk management team is only for a specific project or operation.Implementation of risk management strategies and policies is still limited to the processes which are functionally related, such as those associated with issues of loss, safety/hazard, and so on.In terms of process effectiveness, implementation of risk management is still very dependent on the skill of the project team (due to limited human resources with adequate qualification) and the availability of external support (special consultant of risk management).

Experience
The average score of maturity index for the attribute of experience was 1.97 or below the Initial Level (2.0).In general, at this level, there is no personnel in the organization that has special abilities relevant to risk management and there is no specific program of risk management training to improve staff skill and expertise in risk management.However, the maturity indexes of 55% of the organizations are greater than 1.97.Even one of them almost reach Level 3 (Competent), which means that the majority of personnel within the organization already have the basic skill for participating in the risk management issues.There are also a number of key personnel who are able to take risk management initiatives, as well as acting as trainers who are able to improve their ability independently.In addition to formal and periodic risk management training, in that organization with Level 3 maturity, some risk management methods and tools have been developed.

Application
The scores of maturity index for the attribute of application averaged only 1.60, which demonstrates that the ability of risk management applications of the surveyed organizations is still in the Level 1 (Ad-Hoc).They do not have structured risk management application, no tools and risk metric information (e.g.beta, value-at-risk, etc), nor dedicated resources for risk management application.Some of the organizations have never conducted any risk assessment.The survey recorded only a small-scale use of techniques such as criticality assessment of risk for qualitative risk method and sensitivity analysis for quantitative  risk method.In the case of risk response strategy, risk mitigation options are often used.

Partnership
The scores of maturity index for partnership attribute are averaged 1.91 or close to the Level 2 (Initial).At least 13 organizations assess themselves as organizations that are aware of the importance of stakeholder involvement in risk management.They also describe that risk communication has been running within the organization, either individually or across departments/divisions.However, the study indicated that in most of the organizations, there are no standardized processes to support stakeholder management practices, which are done more in an informal way.Most of respondents describes that they are aware of the risk issue, indicated by the maturity index of culture attribute that approaching level 3.In this level, risk management idea is accepted and supported by upper management.Although not consistently, risk management reports have been used and the benefit of risk management are recognized, expected, and believed to be useful and provides added value.In-depth interviews with the respondents indicated that there is a tendency that most of the respondents felt that their organizations were in the right track in this regard.They insisted that the risk management principles have implicitly existed in their organization policy, while in fact the survey result does not indicate that the concept of risk management has really been formally accepted.It seems that some organizations understood "risk management" as "another area of management", leading to the impression that the basic concept of risk management is not yet well grasped.Some local government organizations even argued that the existing current regulations did not enable them to promote a drastic innovation on risk management outside their authorized boundary.Basically, the respondents did not like to be considered as "naïve", they just said that their internal policy is opposing to the effort to anticipate any change toward improvement.Responses from some organizations described their risk awareness, for example, as the existence of concerns over potential problems that may accompany any bidding process, such as the inability of the contractor to complete the work in accordance with the requirements, or the slow decision to execute the project which results in project cost escalation.Actually, just having these kinds of concern will not guarantee that risk management principles have been applied, unless concrete actions are taken.Within this context, the survey could not demonstrate that any product of internal policies that specifically describes the general and/or specific procedures to deal with uncertainties or any potential future problem exists within the organizations of the respondents.

Conclusion
It has been introduced previously that risk management plays a substantial role in a PPP project management over the life of the long-term contract.It should not only be limited to the process of identifying, analyzing, and planning for response, in an ad-hoc manner, but need to be carried out in an integrated manner with the other processes within the organization.It is important that public organizations dealing with PPP schemes have an adequate capacity to implement it.This study investigates the capability of risk management of the public authority bodies related to PPP scheme development in water supply provision in Indonesia, using a risk management capability maturity model that describes four levels of capability (ad-hoc, initial, competent, excellent) as the function of five assessment attributes (culture, process, experience, application, partnership).The study found that the risk management maturity level of the surveyed organizations is mostly at the Initial stage (level 2), meaning that risk management principles have been applied but still limited to a small number of individuals and not supported by a formal or structured process.At this stage, only simple, relatively easy to understand and implement risk assessment methods should be applied by the organizations, instead of more robust and complex risk management methods, because most of the personnel in the organization do not have adequate knowledge and basic skills to apply them.Initiatives to implement risk management here will not be easy, as the organizations are still lacking in terms of risk management strategy and policies as well as the formalization of processes and knowledge.In this situation, the application of risk assessment methodology is more often to be just an adjunct and not as part of an integrated process.This study indicates that there are challenges for local government organizations in Indonesia in managing the risks of PPP projects in water supply infrastructure effectively and they have yet to develop their capability in order to manage a PPP project successfully.

Figure 1 .
Figure 1.RMCM Model 4 Levels of Competence (s) in operation in the jurisdiction.B) PPP scheme(s) in offering stage in the jurisdiction.C) Potential PPP schemes exist in the jurisdiction.Cult (Culture); Pro (Process); App (Application); Exp (Experience); Part (Partnership); Ave (Average).N/A (not available) Rp 1.000 = approximately USD 0.11 at the time of the study (Jan -March 2010).

Figure 2
Figure 2 illustrates a spider diagram showing the comparison of average maturity index of each assessment attributes for the three groups of respondents (Offices of Public Works, Bappedas, PDAMs), all showing that neither of them are at the competent level, with the Public Works group showing a little bit better for the culture attribute.The diagrams show that all of the groups have a more or less similar pattern of risk management maturity.

Figure 2 .
Figure 2. Comparison of Maturity Index of Grouped Organizations [18]s based on the assumption that there are two approaches in implementing PPP projects in Indonesia for water supply sector.The first case is a greenfield project (no prior service in the area) where the contracting authority is the Mayor or Regent.The PPP procurement authority is usually a group of several agencies forming a local PPP unit.For example, the local Development Planning Agency (BAPPEDA) and the local Office of Public Works (DPU) join together to set up a local PPP unit.The second case is a PPP project in an existing service area, where the contracting authority is the PDAM.In practice, the PPP unit of a PDAM usually communicates with the PPP unit of the local authority.Based on that practice, samples of survey were taken from jurisdictions where at least a PPP scheme has operated and/or ready to be offered and/or listed in the national priority according to Agency for Drinking Water Supply Provision Development Support (BPPSPAM)-Ministry of Public Works[18].
targeted respondents of the survey are individuals representing local government units from Municipalities and/or Regencies related to PPP project procurement and Local Drinking Water Companies (PDAM).

Table 1 .
RMCM Model for Assessing Risk Management (RM) Maturity Levels

Table 3 .
Value of Maturity Index for all Surveyed Organizations