Debt - Narrative (Details)		3 Months Ended	12 Months Ended
	Jan. 13, 2022 USD ($)	Aug. 31, 2020 USD ($)	Dec. 31, 2023 USD ($)	Dec. 31, 2022 USD ($)	Dec. 31, 2021 USD ($)	Aug. 31, 2023 USD ($)
Debt Conversion [Line Items]
Loans pledged to the FHLB			$ 1,674,096,000	$ 1,630,939,000
Total borrowing capacity with the FHLB			1,192,022,000	1,139,356,000
Additional borrowing availability with the FHLB			706,367,000
Convertible notes payable, net			0	5,355,000
Amortization of issuance costs on subordinated debt			147,000	$ 145,000	$ 93,000
Trust preferred securities, aggregate liquidation valuation amount			419,000
Trust Preferred Securities | New Mexico Banquest Capital Trust I (NMBCT I)
Debt Conversion [Line Items]
Debt instrument, face amount			9,279,000
Trust Preferred Securities | New Mexico Banquest Capital Trust II (NMBCT II)
Debt Conversion [Line Items]
Debt instrument, face amount			$ 4,640,000
LIBOR | Trust Preferred Securities | New Mexico Banquest Capital Trust I (NMBCT I)
Debt Conversion [Line Items]
Interest rate margin on variable rate basis			3.35%
Rate			8.94%	7.02%
LIBOR | Trust Preferred Securities | New Mexico Banquest Capital Trust II (NMBCT II)
Debt Conversion [Line Items]
Interest rate margin on variable rate basis			2.00%
Rate			7.64%	6.69%
Convertible Notes Payable | Convertible Debt
Debt Conversion [Line Items]
Convertible notes payable, net				$ 5,456,000		$ 5,456,000
Debt instrument, interest rate						3.29%
Convertible debt, conversion ratio			0.0156717
Amortization of debt discount			$ 101,000	1,131,000	746,000
Subordinated Notes Due July 1, 2030 | Subordinated Debt
Debt Conversion [Line Items]
Debt instrument, face amount		$ 40,000,000
Debt instrument, interest rate		6.00%
Debt, non-redeemable period		5 years
Costs related to the issuance of the subordinated notes		$ 933,000	608,000
Amortization of issuance costs on subordinated debt			93,000	94,000	93,000
Subordinated Notes Due July 1, 2030 | Subordinated Debt | SOFR
Debt Conversion [Line Items]
Interest rate margin on variable rate basis		5.89%
Subordinated Notes Due January 15, 2032 | Subordinated Debt
Debt Conversion [Line Items]
Debt instrument, face amount	$ 25,000,000
Debt instrument, interest rate	3.375%
Debt, non-redeemable period	5 years
Costs related to the issuance of the subordinated notes	$ 534,000		429,000
Amortization of issuance costs on subordinated debt			54,000	51,000
Subordinated Notes Due January 15, 2032 | Subordinated Debt | SOFR
Debt Conversion [Line Items]
Interest rate margin on variable rate basis	2.03%
Subordinated Debt related to Trust Preferred Securities | Subordinated Debt
Debt Conversion [Line Items]
Debt instrument, face amount			13,919,000
Debt discount on the convertible notes			4,293,000
Amortization of debt discount			286,000	$ 254,000	$ 256,000
Costs related to the issuance of the subordinated notes			2,570,000
Line of Credit | Other Financial Institutions
Debt Conversion [Line Items]
Line of credit facility, maximum borrowing capacity			110,000,000
Line of credit outstanding			0
Line of Credit | Federal Reserve Bank stock
Debt Conversion [Line Items]
Line of credit facility, maximum borrowing capacity			$ 2,028,410,000