Consolidated Statements of Cash Flows - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2021	Dec. 31, 2020	Dec. 31, 2019
Cash flows from operating activities
Net income (loss) before attribution of noncontrolling interest	$ 110,311	$ (1,722)	$ 51,334
Adjustments to reconcile net (loss) income to net cash provided by operating activities
Reversal of (provision for) loan losses	(16,500)	88,620	(3,150)
Net premium amortization on securities	12,596	14,868	14,299
Depreciation and amortization	7,269	9,385	7,094
Stock-based compensation expense	4,782	2,307	6,365
Change in cash surrender value of bank owned life insurance	(5,459)	(5,695)	(5,710)
Securities gains, net	(3,740)	(26,990)	(2,605)
Gains on sale of loans, net	(4,276)	0	0
Net gain on sale of headquarters building	(62,387)	0	0
Net loss (gain) on sale of premises and equipment	71	1,729	(2,795)
Deferred taxes and others	6,000	(11,513)	525
Loss on early extinguishment of advances from the FHLB, net	2,488	73	886
Proceeds from sales and repayments of mortgage loans originated for sale (at fair value)	20,859	0	0
Originations of mortgage loans originated for sale (at fair value)	(35,108)	0	0
Net changes in operating assets and liabilities
Accrued interest receivable and other assets	(4,432)	(446)	15,426
Account payable, accrued liabilities and other liabilities	34,957	(13,369)	(3,277)
Net cash provided by operating activities	67,431	57,247	78,392
Purchases of investment securities:
Available for sale	(425,864)	(399,202)	(445,892)
Held to maturity	(100,403)	0	0
Federal Home Loan Bank stock	(4,565)	(9,843)	(43,232)
Equity securities with readily determinable fair value not held for trading	0	(29)	0
Purchases of investment securities	(530,832)	(409,074)	(489,124)
Maturities, sales, calls, paydowns and redemptions of investment securities:
Available for sale	446,436	781,983	497,709
Held to maturity	39,695	15,056	10,747
Federal Home Loan Bank stock	22,110	18,742	40,487
Equity securities with readily determinable fair value not held for trading	23,470	0	0
Maturities, sales, calls and paydowns of investment securities	531,711	815,781	548,943
Net proceeds from sale of headquarters building	132,360	0	0
Net decrease (increase) in loans	93,321	(199,910)	(98,262)
Proceeds from loan portfolio sales	166,329	71,639	267,765
Purchases of premises and equipment	(6,577)	(5,573)	(14,262)
Proceeds from sales of premises and equipment and others	44	13,476	5,173
Cash paid in business acquisition, net	(1,037)	0	(14,390)
Net cash provided by investing activities	385,319	286,339	205,843
Cash flows from financing activities
Net increase (decrease) in demand, savings and money market accounts	602,950	353,277	(308,751)
Net (decrease) increase in time deposits	(703,722)	(378,777)	18,822
Proceeds from advances from the Federal Home Loan Bank	485,500	750,000	1,800,000
Repayments of advances from the Federal Home Loan Bank	(729,618)	(935,073)	(1,731,886)
Proceeds from issuance of Senior Notes, net of issuance costs	0	58,412	0
Repayments of Subordinated Debt	0	(28,068)	(25,864)
Proceeds from common stock issued - Class A	0	0	29,218
Repurchase of common stock - Class A	(36,332)	0	0
Repurchase of common stock - Class B	(9,563)	(69,378)	(28,465)
Common stock surrendered	(2,143)	(917)	(1,695)
Net cash used in financing activities	(392,928)	(250,524)	(248,621)
Net increase in cash and cash equivalents	59,822	93,062	35,614
Cash and cash equivalents
Beginning of period	214,386	121,324	85,710
End of period	274,208	214,386	121,324
Supplemental disclosures of cash flow information
Cash paid for Interest	46,327	73,349	99,958
Cash paid for Income taxes	14,538	10,576	7,544
Initial recognition of operating lease right-of-use assets	55,670	0	0
Initial recognition of operating lease liabilities	56,024	0	0
Right-of-use assets obtained in exchange for new lease obligations	91,797	0	0
Noncash investing activities:
Loans held for investment transferred to loans held for sale	256,154	0	0
Net transfers from premises and equipments to operating lease right-of-use assets	69,931	0	0
Loans transferred to other assets	$ 9,400	$ 400	$ 42