Capital construction projects completing front end planning during COVID-19 are facing some of the most challenging circumstances in recent history. What does this mean for organizations that rely on PDRI reviews as a means of ensuring high-quality scope definition during front end planning?
Current Challenges in Front End Planning
Part of our journey at Valency this year has been engaging with our clients who have implemented PDRI to understand the specific challenges that COVID-19 has introduced that need to be addressed. We conducted a survey in June and again in August, finding that the most prominent challenges from the perspective of our PDRI facilitators were:
- Supporting projects that have either been paused or stopped due to COVID-19
- Maintaining team alignment
- Working through new or increased risks facing their projects
- The quality of deliverables developed in the front-end planning
Leveraging PDRI Reviews to Address Challenges Posed by COVID-19
We had the opportunity to discuss this topic with Valerie Weaver, a project management subject matter expert for a large Midwestern refining company, during a recent webinar hosted by the Construction Industry Institute (CII). Valerie leads project assurance activities for large projects, develops and implements new best practices for project execution, and is a CII Implementation Champion within her organization. She frequently facilitates Project Definition Rating Index (PDRI) reviews to determine if large projects are ready for gate approval and full funding. Valerie has also introduce CII’s latest PDRI template – PDRI MATRS for Industrial projects.
In this post, we will:
- Introduce the new PDRI MATRS template and Valerie’s perspective on the impact to the PDRI process.
- Walk through some of the key risks to scope maturity in front end planning, and how to ensure these risks are discussed in your review of PDRI elements.
- Discuss the new Accuracy factors and their use to assess how a team is functioning amid COVID-19.
- Summarized the Questions and Answers posted by your peers at the live Q&A session during the CII webinar “Leveraging PDRI MATRS & Accuracy Factors for Projects in Front End Planning During COVID-19”.
Watch the full presentation:
Evolution of PDRI Templates
The evolution of PDRI templates has spanned the past 25 years with the publication of seven PDRI templates by the Construction Industry Institute (CII).
One of the more recent templates published in 2019 is an enhancement to the PDRI Industrial template. It’s referred to as PDRI MATRS Industrial. This template introduced new secondary measures, ideal for large industrial projects:
Maturity Score
A maturity score for Front End Engineering Design (FEED) deliverables, along with definition level scoring guidance for each element has helped improve the consistency in scoring across projects and organizations.
Accuracy Score
The other addition was an accuracy score. The accuracy section provides a means to evaluate the degree of confidence the team has in the overall scope definition package. Accuracy factors relate to the:
- The project leadership team,
- Project execution team,
- Project management processes, and
- Project resources
The combination of the Maturity Score and the Accuracy Score comprised the new secondary metric called the MATRS Score.
Learn more: PDRI MATRS – Taking Front End Planning to the Next Level
Scope Maturity Elements at Risk as a result of COVID-19
Regardless of which PDRI templates you utilize today, there are common “at-risk” areas in capital construction projects that can be systematically reviewed using the PDRI process.
Market Strategy
The first element we will review from the PDRI MATRS Industrial template is Market Strategy (B.2). Video reference: 11:26
Depending on which PDRI template you use today, there will be an equivalent element:
- In the buildings template, it is Business Plan and Economic Analysis
- In the infrastructure template, it is Need & Purpose Documentation
SANDRA: Valerie, have you seen a shift in the type of discussion that teams are having when you get to the market strategy element?
VALERIE: I would say this is probably one of the biggest areas of focus in our PDRI reviews. Historically when we have had these reviews, the business case has already been developed pretty conclusively but what we are seeing now is a lot more churn particularly in my industry which is refining – demand has shifted, so pricing has shifted and there has been an intense need to revisit the business economics. So, I see a lot of discussion on this topic, the need to consider sensitivity cases far more than in the past. The other thing that comes up with this particular element is with projects where there is a need to go to market quickly has historically been more of a schedule-driven objective so we are seeing a big shift to a cost-driven project objective so that tends to also create a pretty significant change in strategy with how the projects are executed so lots of good conversation on this element.
Site Characteristics
The second scope element we would like to highlight is Site Characteristics Available vs Required (D.3). Again, we have similar elements across all PDRI templates. Video reference: 14:07
SANDRA: Valerie, what impact has COVID-19 had on teams related to assessing site characteristics?
VALERIE: Site characteristics are typically one of the most important things to define early in project development to get a good solid estimate for project funding but what we are finding is that project teams have limited access to the site and so they have become a bit more dependent on past 3D models and we are seeing some delays on confirming measurements. That also translates into a reduced ability to determine the right physical characteristics: you have got pipe racks – are they full, do you have underground interferences, there is just a whole host of issues that we historically address through site visits. So with that, there is a lot more dependency on institutional knowledge and the need for people with a lot of site knowledge and experience. We’re seeing these things delayed from perhaps they were done in FEL 2 previously, and they are being done in FEL 3, still trying to do it before full project funding but later than what I would consider ideal.
Using Accuracy Factors (How a Project Team is Functioning) to Identify Risk
The definition levels used to evaluate accuracy factors are different than those used for scope related elements, but the intended application is the same.
- Zero is used for not applicable
- 1 means the team is “high performing” concerning the factor criteria,
- 5 means the results are “not acceptable”
SANDRA: Valerie, you shared that you have developed a strategy for using the accuracy scoring to identify risks. Can you tell us a bit about your approach?
VALERIE: We have been convening virtually obviously given the circumstances, we will have the extended team virtually complete the PDRI in its entirety and at the conclusion of that, we will always reduce the number of people that are participating in the PDRI to a much smaller group and then proceed on to do this accuracy assessment. Sometimes we will even break and reconvene on a separate day but I think the big thing is you want to reduce the participants down to the leadership folks – typically the PM on the engineering side, the PM on the owner project side, and then possibly with their supervisors, maybe somebody who is responsible for the business relationship on the engineering contractor sides and then the engineering supervisor on the project side now that is the smaller group that we would then go through these particular questions with.
SANDRA: Are you focused on the aggregate score overall or do you look for outliers? Do you have any guidance on that side?
VALERIE: Yes, typically we are looking for things that are going to put the project in jeopardy. So the 3’s and the 4’s – those are the ones that really stimulate the most conversation and I have often found that these are things the project leaders are already aware of but they have not stopped to talk about so it allows you to put added emphasis on what I call some of the ‘project team fundamentals’ that are really essential for delivering the project in the next phase, developing some strategies to possibly improve those areas.
And often the things that come up are turnover of personnel, turnover is particularly damaging right now when we already have a lot of things at play. And then communication issues often surface here as well.
Accuracy Factors at Risk During PDRI Reviews
Let us dive a little deeper and discuss the accuracy factors that are evaluated, with a focus on those significantly impacted by COVID-19.
Co-Location & Communication
The first two we would like to highlight are Co-location (2f) and Communication (3a). Video reference: 21:37
SANDRA: Valerie, this has clearly been a challenging issue over the past several months. Through your recent PDRI assessments, have teams acknowledged any impacts from not being able to co-locate?
VALERIE: So as an owner, we have always put a lot of emphasis on having our people co-located at the engineering office so it has created a big shift from the norm and I have kind of seen two groups – I actually did a PDRI last week with a very seasoned team that has worked with a particular engineering office for a long time and so they know one another, they are accustomed to working with one another, they have developed some more advanced communication approaches and they seem to have been very effective in working through the issues. I think the big communication issue that they are always concerned about is a disconnect between the disciplines – those are often conversations across desks or at lunchtime and they have overcome that with daily update communications-type approaches virtually. And the other camp is groups that have less experienced owner personnel and project teams because we have seen turnover or perhaps, they are using an engineering office for the first time and they are developing a relationship with that particular office. I think that the second case is the one where we have more conversations around how we can work through to improve essentially the communication plan to overcome that co-location issue.
SANDRA: So in the accuracy factors, this next factor here – communication closely follows on the heels of the co-location discussion so it does naturally help that “what do we do about it?” if it is an issue discussion.
VALERIE: It does, and somebody said at a recent PDRI “well you can always improve communication”. Having that conversation is always a benefit but now I think it is even more important that we focus on that.
Priorities
Another accuracy factor that is very valuable to assess is related to Priorities (3c). Did the team feel they had clear priorities for cost, schedule, and required project features? Video Reference: 24:33
SANDRA: Valerie, what issues have COVID-19 introduced related to this factor?
VALERIE: Yes, I think all of the above have changed – we have gone from (as I mentioned earlier) from schedule to cost so that is a big shift and that is further exacerbated by perhaps some downtime when they say “we are going to shut this project down for a year and then we are going to restart it” so that brings into play ‘how do you most efficiently shut down a project for a short period of time so that you can efficiently pick it back up and not create a lot of rework?’ And then on the project features side, a lot of value engineering going on to make sure that the project scope meets our new business needs. So typically, that is creating churn – something that we try to avoid later in project development. So yes – all the above priorities are seeing a lot of shifts across the board.
Local Knowledge
SANDRA: Another one here, and I think you helped us unpack this one a little bit earlier, but I think it’s just really valuable for how it relates to local knowledge and sometimes this is having an understanding of the site history but most of the time it also means being able to access and visit and evaluate that site. And the principal behind this one is that access to the site gives the team that hands on review, allows them to do field verification, site characteristics. So Valerie, you shared a little bit about what this has looked like. Have people come up with strategies to help work through this or work together, work cooperatively with operations personnel?
VALERIE: Yes, I think the key is you just really need to have experienced personnel if you can have it. And then in some cases where it’s not enough, we’ve devised some strategies where we can access the site through one person perhaps or small groups, really trying to minimize that particular factor, but it’s just really important that the group is tracking their assumptions and they have to have some definitive plan on when they’re going to be able to shore up that gap even if it occurs late. So this is an ongoing issue and it’s hard to substitute for experience in this case.
SANDRA: Well it sounds like a really valuable discussion to have in FEL2 in that first review.
VALERIE: Oh absolutely and I’d say having construction personnel even more involved – we’ve always tried to really drive construction driven engineering type approach but even more important here is to get that input early from people who have had that construction knowledge and experience for that particular location.
How to Assess Accuracy in any PDRI Session
At Valency, our team has been utilizing PDRI MATRS for about 18 months. Early on, we concluded that evaluating Accuracy factors adds value to any PDRI session.
Below are the steps we are using and would recommend for any organization that sees value in stepping back and having a healthy conversation about team dynamics.
- Conduct your normal PDRI session using your current template.
- Break – can be the same day or the next day.
- Reconvene with the core project team for the “Accuracy” session. We recommend setting aside 90 minutes for this portion of the assessment.
- Use the PDRI Summary report to pull both the PDRI total score and Accuracy score results together.
Peer Q&A from August 11, 2020, CII Webinar
Due to the shortfall in face-to-face interaction, are companies changing the way they document planning efforts, or are they simply reemphasizing existing documentation and record-keeping protocols?
VALERIE: In general, I believe we have been documenting things as much as we have in the past. I think some of the ways we are meeting is perhaps changing how that information is stored and shared. You know, we are using Teams meeting platform as a way to share documents so that we can modify them as a group but generally speaking our document control practices are unchanged.
SANDRA: Yes, that might be the type of thing that takes another six months or so for it to work itself through that people acknowledge the change and maybe in the next year’s budgeting cycle that ends up being a priority for us too.
We have found that reviewing the accuracy criteria with the team early in the FEL process helps a better understanding of what makes a better-quality project environment. Would you agree?
SANDRA: Absolutely
VALERIE: I would agree with that. What we do is we use the other CII tool that is the ‘Alignment Thermometer”, and a lot of the questions are very similar. We try to do the Alignment Thermometer once in FEL 2 and again in FEL 3 to gauge how the extended team is working together. So yes, I think it would be very comparable to that but more of a workshop-type environment. That is a great idea.
How do you verify the objectivity of the accuracy definition scoring?
VALERIE: I have always found the project leaders to be open and honest. I don’t worry about the scores so much – the actual score that comes about. I mean it is good to be able to provide an overall assessment in a package that is going to go to leadership for funding approval to show how you benchmark against other projects in the portfolio but I usually look for those little nuggets of information that come out of the discussions around the elements that’s where I think the value is. So, I do not worry so much about the actual number that is assigned to the various elements. I don’t know, Sandra, what do you think about that?
SANDRA: I was kind of reflecting on how you make sure it is objective, you and I have both used a technique of actually asking the engineering contractor that is in the room what their feeling was in terms of the processes and how well they were followed or how well they were defined that the owner had provided them. So, kind of having the opportunity requires you to have a safe environment where they feel open to sharing that but that is part of that smaller group. That has been one technique that I think has been effective to get that objectivity at the table.
VALERIE: I agree with that, in fact during a recent session, I started with the second group of questions which was more team-based and asked the questions directly of the engineering company first before we started the owner question bank and I felt like there was a lot of objectivity but have not experienced a problem in that area.
Do the poor scoring criteria find their way onto a risk register or issues register for tracking and resolution?
VALERIE: Well our documented procedures are that the action items that come out of any of our project assurance processes which PDRI is one of them, they have to be formally incorporated into what we call a Gap Closure Plan and that Gap Closure Plan has to be stewarded by the site supervisor to closure so I am not sure if that is what the question was but I agree, you have to follow up on the actions and drive them to closure. But that said, you know not every action that gets identified in the PDRI has the same value towards risk reduction so we do encourage our PMs to put that additional thought into it and if there are some actions of lower importance, then we do handle those separately.
SANDRA: Yes, I think I have seen good examples come out of the accuracy factors that were added to the risk register. They were not necessarily actions that the team could change but they were a risk that needed to be managed. One of the good examples that I had was a team where they had seconded the senior project manager and some of the senior engineering resources through the front end planning phase from one of their sister companies and that small group was going to depart the team when they headed into the execution phase. So there was some risk in terms of loss of knowledge and experience on the team that came through the discussion that ended up on the risk register and hopefully there are some good mitigating actions that could be taken as a follow-up.
VALERIE: Along those same lines, looking at it from that perspective, I have also seen gaps in what is considered more matrix-type organizations where functional groups are not collaborating as effectively as necessary say perhaps construction management versus project engineering and so that has led to some very successful follow-up action as well so much along the lines of what Sandra noted there.
Valerie, are most of the projects that you are talking about that you have referenced, are they brownfield or greenfield?
VALERIE: The vast majority of the projects we do are retrofit revamp jobs, not a lot of greenfield. Some, but not a lot.
Who would you include for the core group in the Accuracy session and how many people?
VALERIE: So, my smallest group might be two – it would be the PM from the engineering group, from the Engineering office, and the PM from the project. But usually, I encourage the leaders from their respective organizations. So often it is that business lead contact on the Engineering Contractor’s side who is responsible for managing the relationship between our respective organizations and then the supervision on the project engineering owner’s side or perhaps the sponsor of the project, so depending on if it is a $100 million project, you are going to get more interaction and involvement than a $25 million project.
What is the bottom-line takeaway you give the project team when you have the accuracy score?
VALERIE: Well, we go through the full bank and then we always, at the conclusion of the PDRI and the Accuracy session, go back to revisit the low scoring areas and just validate that we are all in alignment and that what we have documented is consistent with the message that we all have agreed to. So, I think that would be the bottom line, it is just driving consensus on what is going to appear in the final report and that everybody is in agreement with those findings.
As part of the project closeout, are the PDRI results tied back to the actual project results? And what lessons learned can you share?
SANDRA: Yes, I think this is probably one of the greatest areas of opportunity within the industry and some of the most exciting conversations that we have been having over the past six months has been “we have got so much data that we can leverage in terms of areas that scored well or areas that did not score so well, how do we start to leverage machine learning and AI technologies to be able to take that data and be able to drive learnings from it?” It doesn’t replace the human element from lessons learned, but it certainly gives us a lot more objective information that we can use as we look at project results. So, I am quite excited about that, but I would say there is a lot of room for improvement in the industry.
VALERIE: Yes, I think, on a more fundamental basis, where we are at, annually, we will take a look at the PDRI findings and try to identify those top five areas that scored the lowest across the board and try to develop action planning and initiatives to improve the performance in those areas across our system, but long term PDRI would be great benchmarking metrics to use in our overall analysis of project outcome. We are just not that far. We have got those activities under development but have not fully implemented them yet.
Getting Started with PDRI
The first step to implementing PDRI within your Front-End Planning practices is becoming a trained facilitator. Valency offers industry-leading training (PDRI Principles & Practices), held virtually several times each year.
The Valency team has trained hundreds of PDRI facilitators and offers ongoing support and facilitation tools to ensure your PDRI reviews are productive and effective.
Questions about PDRI?
Please contact our Business Development Team at 1-519-883-7136 to schedule a time with one of our senior consultants.
More resources:
- White paper: Introduction to the Project Definition Rating Index (PDRI)
- Datasheet: PDRI MATRS
- Blog: PDRI MATRS – Taking Front End Planning to the Next Level
- CII Report: RS-331 Assessing the Maturity and Accuracy of Front End Engineering Design, CII
- CII Report: RT-361 Development of the PDRI Maturity & Accuracy Total Rating System (MATRS), CII