Implementing CECL successfully is a major accomplishment, but it still requires ongoing monitoring to ensure it remains relevant and accurate as economic and business conditions change. CECL backtesting provides the ongoing validation needed to identify deficiencies so institutions can maintain confidence in their expected credit loss estimates.

An effective CECL backtesting process requires reliable data, thorough documentation, and consistency in review processes. Institutions that understand what is CECL also recognize that strong data requirements are foundational to the standard. From there, they can implement best practices to support the ongoing reliability of their CECL models.

What Is CECL Backtesting?

CECL backtesting involves comparing previous assumptions or estimates of credit losses with actual results. This helps evaluate the accuracy of previous loss assumptions and estimates. When used as part of an ongoing assessment process, this can help banks and credit unions continue to refine models to improve performance and the accuracy of loss estimates.

CECL backtesting involves more than just validating calculations. It also involves identifying trends, data issues, and whether qualitative adjustments and model assumptions remain appropriate. This helps banks and credit unions determine whether model refinements are needed and is a key part of more accurate CECL accounting by ensuring estimates remain valid for the current business environment.

Backtesting is also critical for CECL model validation. This is because regulatory CECL guidance requires models to be continuously monitored, results documented, and institutions to be able to demonstrate that estimates remain reasonable and supportable. Backtesting is a key piece of Model Risk Management guidance surrounding “effective challenge” and provides financial institutions the capability to provide documentation on how they are approaching challenging the assumptions. For additional guidance, see our article on CECL reasonable and supportable forecasts.

Why CECL Backtesting Matters

CECL backtesting is necessary in order to maintain a reliable and defensible audit-ready system. Even if CECL models are using reliable and reasonable assumptions, those assumptions should still be evaluated against actual loan performance. Regular backtesting helps financial institutions determine the accuracy of their models over time as market conditions continue to change. In the context of regulatory CECL compliance, backtesting:

• Demonstrates that organizations are actively monitoring models over time, instead of relying on the same set of assumptions that may no longer reflect changes to customer behavior or economic events.
• Supports sound governance by providing a process for reviewing results, identifying discrepancies, and implementing changes to improve performance.
• Improves forecast accuracy by comparing projected and actual credit losses, so institutions can refine model assumptions over time.
• Improves audit readiness by creating a clear record of model performance, assumptions, and historical changes.

How CECL Backtesting Works

An effective CECL backtesting workflow is structured, allowing institutions to evaluate performance and make adjustments over time. Backtesting also provides the greatest amount of value when used as an ongoing process, rather than a one-time exercise.

Compare projected versus actual credit losses

The first step is to compare historical credit loss estimates with actual losses. This helps establish a baseline for how the initial CECL model performed based on the market conditions and assumptions at the time of use.

Analyze model performance

Once results have been compiled, banks should identify performance trends across portfolios, product lines, geographic regions, and customer segments. This step is intended to identify which areas experienced the largest variances in projected and actual losses.

Investigate variances

Once variances have been identified, institutions must then determine whether they were significant enough to be categorized as a model deficiency. Small variances are expected as no forecasting model is expected to be perfectly accurate, but large variances may indicate critical oversights in economic conditions.

Update assumptions when appropriate

If backtesting shows significant variances in models, they should be updated to address the factors contributing to those variances. This can include qualitative and quantitative adjustments, forecasting inputs, and other model assumptions.

Document results and governance

To ensure continued transparency and audit-readiness, updates should be documented. This includes changes made to backtesting methodologies, findings, and the underlying reasons for model changes.

Common Challenges That Reduce Backtesting Accuracy

A CECL model can become less reliable if operational challenges hinder the underlying backtesting process. Addressing these issues with routine monitoring is key to maintaining a reliable and accurate CECL model:

• Insufficient Historical Data: Missing data across time periods, loan products, or customer segments makes it difficult to accurately compare expected and actual credit losses.
• Inaccurate Data and Inconsistent Segmentation: Inconsistent data or loan groupings across reporting periods make it difficult to identify meaningful trends and compare results.
• Changing Economic Conditions: Rapid shifts in economic conditions or customer behavior may require model updates, but changes should be validated to maintain reliable credit loss estimates.
• Poor Documentation: Missing records of model changes, assumptions, approvals, or backtesting results make it difficult to explain and defend CECL estimates during audits or regulatory reviews.

How Better CECL Data Improves Backtesting

Quality data is fundamental to quality backtesting processes. With that said, teams seeking a detailed walkthrough of the required CECL data fields and documentation should visit our guide to CECL data requirements.

Quality CECL data management allows institutions to compare historical credit loss estimates with actual loan performance. Having a complete loan history is key to fully understanding loan performance trends over time.

With a complete loan history, banks should also ensure consistency in segmentation to ensure an apples-to-apples comparison across time periods. Without these, it becomes difficult to determine if variances reflect changes in loan performance or if they’re indicative of issues in data quality.

Strong data governance with a centralized data environment further improves backtesting with clear ownership for data management, including data collection, validation, and quality control. When teams can rely on a unified source of trusted data, they spend less time reconciling discrepancies and more time analyzing model performance.

Best Practices for Building an Effective CECL Backtesting Process

An effective CECL backtesting process requires transparency, consistency, and a willingness to regularly monitor and make adjustments for improved performance. With established and repeatable processes, banks also benefit from streamlined regulatory and examiner reviews.

Establish consistent review schedules

Backtesting should be done on a pre-defined schedule, preferably one that also aligns with the bank’s CECL reporting cycle. Regular backtesting helps identify trends more quickly and also ensures that model performance is regularly monitored for issues.

Monitor key performance metrics

Financial institutions should track how estimates of credit losses compare against actual losses. Tracking forecast accuracy and variances over time helps banks identify whether model estimates are reliable and where adjustments may be needed.

Maintain strong documentation

Each stage of backtesting should be documented. This should include the rationale for assumptions, methodologies used, performance logging, and variance analyses. Having well-documented processes here simplifies internal and external reviews by providing a clear audit trail.

Improve collaboration across finance and risk teams

An effective backtesting process requires input from multiple departments, such as finance, risk, credit, and accounting teams. Having a high level of collaboration across each function ensures consistency in decision-making, increases transparency, and reduces duplicate work.

Continuously refine assumptions

Backtesting should be used not just as a compliance exercise, but also as a tool to improve model performance. As market conditions change, backtesting can inform teams what factors need to be adjusted in order to maintain and improve the reliability of future CECL estimates.

Simplify CECL Backtesting With Empyrean

An effective CECL backtesting process involves more than just periodic reviews. It also requires connected data, consistent and repeatable workflows, and strong governance. Empyrean CECL provides each of those in a unified platform that can help banks strengthen their model validation framework.

Empyrean CECL centralizes data management, providing teams with consistent data, connected workflows, and stronger governance. Integrated scenario management and audit-ready documentation also make it easy to track model assumptions and performance over time.

Learn more about Empyrean CECL to see how its integrated platform can improve your institution’s modeling processes.

Alternatively, request a demo to see how Empyrean can help boost confidence and accuracy in forecasting workflows.

FAQ: CECL Backtesting