Model validation is an important aspect of risk management and adherence to regulatory requirements. Guidance from federal and regulatory entities requires banks to document procedures and policies surrounding their risk management practices, and actions are taken to prove models are consistently reliable.
Meeting CECL model validation requirements can, however, be difficult for financial institutions. Many banks and credit unions rely on third-party vendors for risk modeling. They often don’t have the resources for an internal team of experts to perform model validations.
Regulatory requirements provide little flexibility due to a lack of resources. Financial institutions are still responsible for documenting and applying best practices or assumptions consistently, such as those made on prepayment speeds and reversion methods.
This article outlines the key steps needed in CECL model validation, along with common challenges and tips. These will ensure your bank does not run into any issues if audited by an examiner or other third party.
What Is CECL Model Validation?
CECL model validation is an independent review of your expected credit loss framework. This includes the model, data, assumptions, governance, and outcomes. Validation ensures a model’s logic is suitable given a bank’s specific risk profile.
Outputs are expected to meet accuracy and documentation standards defined by the Federal Deposit Insurance Corporation’s (FDIC’s) Interagency Policy Statement on Allowances for Credit Losses:
- Conceptual soundness: The design, including segmentation, methodologies, assumptions, and data sources, must reasonably estimate lifetime expected credit losses under CECL based on the bank’s circumstances and risk characteristics.
- Ongoing monitoring: Data feeds must be accurate, and controls, overrides, and reporting should align with policy and how the model was designed to be used.
- Outcome analysis: Forecasts of losses should make sense compared to prior actual losses and credit trends.
According to guidance from the Office of the Comptroller of the Currency (OCC), validation must be conducted by parties independent from the model development and day-to-day use, to the extent appropriate for the institution’s size, risk, and complexity. While each firm has its own approach, external validators typically focus on:
- Governance and policies.
- Model design and methodology.
- Data and implementation.
- Assumptions and overlay.
- Monitoring and outcomes analysis.
Your institution’s CECL use, assumptions, and governance must be sound enough that an internal team or external third party can validate the model. This ultimately builds investor and examiner confidence.
Pro Tip: Treat CECL validation like a board presentation. Your model should tell a clear, defensible story backed by data.
CECL Model Validation Requirements for Community Banks
Regulatory guidance from the Federal Reserve and the OCC states that institutions must validate their models regularly. This is to ensure modeling methods are sound and reasonable, so outcomes can be considered reliable.
While CECL model validation is conducted by a third party, management still owns the methodology, assumptions, and controls of their models. Validation simply confirms if the institutions using CECL have the following characteristics:
- Consistency With Risk Profile and Lending Activities: Assumptions and data used in the model should be a reflection of the institution’s mix of loans and credit behavior.
- Documentation of Inputs, Assumptions, and Sources: Data and estimation methods should all be explained and documented.
- Back-Testing and Performance Monitoring: The differences between predicted and actual credit losses should be identified to continuously update models to perform as intended.
- Independent Review: Validation should be conducted by internal or external teams, as long as it’s not by staff who were involved in the development of the original model.
CECL guidance adds that these expectations apply to all banks, credit unions, and financial institutions and are commensurate on size, complexity, and risk profile. This protects smaller banks that may not have as many resources. Validation is also required even if an institution utilizes a third-party CECL calculation platform or other software for its modeling.
Pro Tip: Use Empyrean CECL solutions to document and export validation-ready reports automatically for auditors and examiners.
Common CECL Model Validation Pitfalls
Even if a CECL model is designed well, it may still not function as intended if the underlying data or validation processes are faulty. Banks looking to have a certified CECL model validation should be aware of the following challenges:
- Weak or Fragmented Documentation: Documented explanations of CECL methodology, segmentation, and forecast logic are missing, limited, or outdated, which can lead to inaccuracy.
- Data Quality Issues: Limited monitoring and back-testing can result in missing and inconsistent data, which can lead to inaccurate model estimates.
- Inconsistent Reversion Logic: Misalignment in forecasting period and reversion methods tends to damage a model’s accuracy.
- Vendor Overreliance: Institutions must understand how different types of financial models operate, rather than simply relying on third-party vendors to ensure accuracy.
- Overfitting and Complexity: Models that are too complex in relation to the bank’s risk profile can create an unnecessary burden when it comes to documenting and explaining assumptions and methodologies.
- Incorrect Prepayment Speed Assumptions: Prepayment rates not tailored for a bank’s specific risk profile or customer behavior can result in inaccurate estimates of estimated credit losses.
Pro Tip: Flag any assumption that’s unclear to your team. It’s likely to draw examiner attention later.
How to Prepare: A Simple Framework for Third-Party Validation
Think of this as your validation readiness checklist. You’re organizing your information, so an independent validator can evaluate and certify it. The primary goal is to show that your model utilizes sound logic, reasoning, reversion methodology, and economic drivers and assumptions.
1. Document Assumptions and Data Sources
List every data input and assumption being used for your model. Common items include:
- Portfolio segmentation and risk characteristics used for pooling.
- Historical loss data sources and any adjustments applied.
- Key assumptions, like prepayment speeds and recovery rates, and their rationale.
- Forecast period, reversion methodology, and economic drivers.
This allows any third party to understand how the results were produced, along with ensuring consistency with your CECL standard.
2. Preassess Model Logic and Outputs
Evaluate the model’s calculations and outputs to determine if they align with the OCC’s three pillars: conceptual soundness, ongoing monitoring, and outcomes analysis.
Instead of blindly trusting the output, review the results yourself. You want to see if figures like loss estimates seem accurate with the trends present in your current portfolio.
Flag anything that doesn’t align. Be prepared to discuss with validators why you are comfortable with it or how you plan to remediate.
3. Organize Evidence of Monitoring and Testing
Validators will look for proof that you actually manage the model. For this, you should prepare:
- Back-testing analyses that compare expected versus actual losses by segment.
- Override logs, where managers changed or ignored model outputs, and performance of those decisions.
- Control documentation, including reconciliations, change-management records, and validation of any spreadsheets or user-developed applications used in CECL.
4. Prepare Management and Board for Validator Interviews
During an independent review, validators will review board and management responsibilities for overseeing credit loss and model risk.
This means the chief finance officer, chief revenue officer, controller, and credit leadership need to be able to understand and explain:
- How CECL works at your institution.
- Key assumptions and why they’re reasonable.
- How results are reviewed and challenged.
Make sure your board is also prepared to demonstrate how they have received and reviewed CECL information for discussions of trends and risks.
Pro Tip: Integrate CECL validation with your asset and liability management systems to align credit loss forecasts with liquidity and capital planning.
Best Practices for Small Institutions
For smaller lending institutions, credit unions, and community banks, the CECL model validation should focus on transparency, consistency, and clarity. The complexity of a model should not be a factor.
A critical part of a well-designed CECL model has processes examiners can easily understand for your bank’s risk profile:
- Keep Models Simple but Transparent: Less complex models are easier to document, update, and maintain. This also makes them easier to validate and explain.
- Refresh Validation Annually or After Major Assumption Changes: Regularly updating the model ensures it stays relevant for current portfolio trends and economic shifts.
- Maintain Version Control and Change Logs: Tracking all changes made to the model creates a clear audit trail, which makes it easier to reverse changes if the model stops functioning as intended.
- Provide Board-Level Summaries Focused on Trends: Reports should be customized towards decision-makers in your institution. To this end, highlight outcomes and key risks.
- Ensure Your CECL Model Ties to Broader Performance Metrics: Tie credit loss forecasts to revenue and profitability. Doing so strengthens strategic insight and credibility.
Pro Tip: Simplicity is strength. Clear documentation beats complex math for examiner confidence.
From Compliance to Confidence: How Empyrean Helps
Small banks often face an unfair playing field in the sense that they’re expected to meet similar regulatory expectations as large institutions, but with fewer resources. That’s where Empyrean CECL can bridge that gap.
Empyrean CECL reduces the staffing needs by automating many functions needed to prepare for the validation process. This includes guidance with assumptions and reporting, allowing banks to spend less time on manual data entry.
Empyrean’s platform generates transparent and auditable loss estimates to ensure documentation meets the expectations of examiners and regulators. Seamless integration is also available with Empyrean’s ALM and budgeting tools, allowing for an even greater level of consistency across forecasts.
The result is alignment with CECL and supervisory guidance that provides reliable forecasts without burdening staff.
Ready to take the next step?
See how Empyrean CECL simplifies model validation when you request a demo today.
Get a DemoFAQ: CECL Model Validation
What Are CECL Model Validation Requirements?
Regulators expect CECL models to be tested and validated by an independent party on a regular basis.
Validation processes must confirm that assumptions, data, and outputs are sound and reasonable. Transparency is also expected, with a sufficient amount of documentation showing any changes or updates made over a period of time.
How Often Should CECL Models Be Validated?
CECL models should be tested at least annually and more frequently if warranted by model changes or economic shifts. They should also be evaluated whenever there are major shifts in assumptions or data sources.
For example, updates to economic conditions or CECL reasonable and supportable forecasts are events that should trigger a new evaluation process.
What Are Common CECL Model Validation Challenges?
Prepayment speed, data quality, limited time to document updates, and vendor dependency are common challenges faced, especially by smaller banks. Each can cause auditors and other stakeholders to question the integrity of the model.
What Tools Help Simplify CECL Validation?
The Empyrean CECL platform automates many of the steps involved in validating the CECL model. It eliminates the need to increase staffing, while still producing defensible results to regulators.