Loans Held for Investment - Collateral Dependent Loans by Collateral Type (Details) $ in Thousands	Mar. 31, 2020 USD ($)
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	$ 6,030
Office Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	590
Industrial Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	756
Retail Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Hotel Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	710
Multifamily Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	1,802
Various Business Assets
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	2,172
Investor loans secured by real estate
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	2,512
Investor loans secured by real estate | Office Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Investor loans secured by real estate | Industrial Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Investor loans secured by real estate | Retail Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Investor loans secured by real estate | Hotel Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	710
Investor loans secured by real estate | Multifamily Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	1,802
Investor loans secured by real estate | Various Business Assets
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Investor loans secured by real estate | Commercial real estate (“CRE”) non-owner-occupied
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	318
Investor loans secured by real estate | Commercial real estate (“CRE”) non-owner-occupied | Office Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Investor loans secured by real estate | Commercial real estate (“CRE”) non-owner-occupied | Industrial Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Investor loans secured by real estate | Commercial real estate (“CRE”) non-owner-occupied | Retail Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Investor loans secured by real estate | Commercial real estate (“CRE”) non-owner-occupied | Hotel Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	318
Investor loans secured by real estate | Commercial real estate (“CRE”) non-owner-occupied | Multifamily Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Investor loans secured by real estate | Commercial real estate (“CRE”) non-owner-occupied | Various Business Assets
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Investor loans secured by real estate | Multifamily
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Investor loans secured by real estate | Multifamily | Office Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Investor loans secured by real estate | Multifamily | Industrial Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Investor loans secured by real estate | Multifamily | Retail Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Investor loans secured by real estate | Multifamily | Hotel Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Investor loans secured by real estate | Multifamily | Multifamily Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Investor loans secured by real estate | Multifamily | Various Business Assets
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Investor loans secured by real estate | Construction and land
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	1,802
Investor loans secured by real estate | Construction and land | Office Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Investor loans secured by real estate | Construction and land | Industrial Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Investor loans secured by real estate | Construction and land | Retail Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Investor loans secured by real estate | Construction and land | Hotel Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Investor loans secured by real estate | Construction and land | Multifamily Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	1,802
Investor loans secured by real estate | Construction and land | Various Business Assets
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Investor loans secured by real estate | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	392
Investor loans secured by real estate | SBA secured by real estate | Office Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Investor loans secured by real estate | SBA secured by real estate | Industrial Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Investor loans secured by real estate | SBA secured by real estate | Retail Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Investor loans secured by real estate | SBA secured by real estate | Hotel Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	392
Investor loans secured by real estate | SBA secured by real estate | Multifamily Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Investor loans secured by real estate | SBA secured by real estate | Various Business Assets
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Business loans secured by real estate
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	1,033
Business loans secured by real estate | Office Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	277
Business loans secured by real estate | Industrial Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	756
Business loans secured by real estate | Retail Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Business loans secured by real estate | Hotel Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Business loans secured by real estate | Multifamily Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Business loans secured by real estate | Various Business Assets
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Business loans secured by real estate | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	1,033
Business loans secured by real estate | SBA secured by real estate | Office Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	277
Business loans secured by real estate | SBA secured by real estate | Industrial Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	756
Business loans secured by real estate | SBA secured by real estate | Retail Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Business loans secured by real estate | SBA secured by real estate | Hotel Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Business loans secured by real estate | SBA secured by real estate | Multifamily Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Business loans secured by real estate | SBA secured by real estate | Various Business Assets
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Business loans secured by real estate | CRE owner-occupied
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Business loans secured by real estate | CRE owner-occupied | Office Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Business loans secured by real estate | CRE owner-occupied | Industrial Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Business loans secured by real estate | CRE owner-occupied | Retail Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Business loans secured by real estate | CRE owner-occupied | Hotel Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Business loans secured by real estate | CRE owner-occupied | Multifamily Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Business loans secured by real estate | CRE owner-occupied | Various Business Assets
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Business loans secured by real estate | Franchise real estate secured
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Business loans secured by real estate | Franchise real estate secured | Office Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Business loans secured by real estate | Franchise real estate secured | Industrial Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Business loans secured by real estate | Franchise real estate secured | Retail Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Business loans secured by real estate | Franchise real estate secured | Hotel Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Business loans secured by real estate | Franchise real estate secured | Multifamily Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Business loans secured by real estate | Franchise real estate secured | Various Business Assets
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Commercial loans
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	2,485
Commercial loans | Office Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	313
Commercial loans | Industrial Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Commercial loans | Retail Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Commercial loans | Hotel Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Commercial loans | Multifamily Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Commercial loans | Various Business Assets
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	2,172
Commercial loans | Commercial and industrial
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	1,608
Commercial loans | Commercial and industrial | Office Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	313
Commercial loans | Commercial and industrial | Industrial Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Commercial loans | Commercial and industrial | Retail Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Commercial loans | Commercial and industrial | Hotel Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Commercial loans | Commercial and industrial | Multifamily Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Commercial loans | Commercial and industrial | Various Business Assets
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	1,295
Commercial loans | Franchise non-real estate secured
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Commercial loans | Franchise non-real estate secured | Office Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Commercial loans | Franchise non-real estate secured | Industrial Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Commercial loans | Franchise non-real estate secured | Retail Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Commercial loans | Franchise non-real estate secured | Hotel Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Commercial loans | Franchise non-real estate secured | Multifamily Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Commercial loans | Franchise non-real estate secured | Various Business Assets
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Commercial loans | SBA non-real estate secured
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	877
Commercial loans | SBA non-real estate secured | Office Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Commercial loans | SBA non-real estate secured | Industrial Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Commercial loans | SBA non-real estate secured | Retail Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Commercial loans | SBA non-real estate secured | Hotel Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Commercial loans | SBA non-real estate secured | Multifamily Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Commercial loans | SBA non-real estate secured | Various Business Assets
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	877
Retail loans
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Retail loans | Office Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Retail loans | Industrial Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Retail loans | Retail Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Retail loans | Hotel Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Retail loans | Multifamily Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Retail loans | Various Business Assets
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Retail loans | Single family residential
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Retail loans | Single family residential | Office Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Retail loans | Single family residential | Industrial Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Retail loans | Single family residential | Retail Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Retail loans | Single family residential | Hotel Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Retail loans | Single family residential | Multifamily Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Retail loans | Single family residential | Various Business Assets
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Retail loans | Consumer
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Retail loans | Consumer | Office Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Retail loans | Consumer | Industrial Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Retail loans | Consumer | Retail Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Retail loans | Consumer | Hotel Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Retail loans | Consumer | Multifamily Properties
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	0
Retail loans | Consumer | Various Business Assets
Financing Receivable, Past Due [Line Items]
Collateral dependent loans	$ 0