I’m having trouble finding any documentations regarding supported key lengths for RSA certs to encrypt SQL connections. Our internal CA is issuing certs with a 4096 key length and that seems to work with our 2019 SQL servers, but I cannot seem to get anything lower than 2019 to see the certs in Configuration Manager. My hunch is that 2017 and below do not support a 4096 key length but I’m not able to find any documentation saying what is actually supported so it’s only a guess right now. Does anybody have any documentation showing what is supported by each version? Thanks.
It looks like maybe the issue was not an issue with the key length but rather the type of cert. The below link has this and it looks like when we created a new cert template to use the legacy CSP it worked. So, it looks like maybe SQL 2019 supports KSP, but 2017 and below only support CSP.
After releasing Windows Vista and Windows Server 2008, there is a brand new CSP subsystem called Cryptography Next Generation (CNG). CNG is was written from scratch and is not compatible with legacy CSP subsystem. CNG brings a lot of improvements to CryptoAPI, but the idea is still the same. Also, CNG ships a new class of CSPs called Key Storage Provider or KSP. In general, KSP and CSP are similar in terms of their purpose. CSP refers to legacy CryptoAPI 1.0 and KSP refers to CNG or CAPI2.