After playing with the code I have received from ZZ Coder, sylvarking and Software Monkey, I have found a solution that works:

First, I wrote a X509KeyManager that works combines a custom keystore and a default keystore.

class MultiKeyStoreManager implements X509KeyManager {
 private static final Logger logger = Logger.getLogger(MultiKeyStoreManager.class); 
 private final X509KeyManager jvmKeyManager;
 private final X509KeyManager customKeyManager;

 public MultiKeyStoreManager(X509KeyManager jvmKeyManager, X509KeyManager customKeyManager ) {
  this.jvmKeyManager = jvmKeyManager;
  this.customKeyManager = customKeyManager;  
 }

 @Override
 public String chooseClientAlias(String[] keyType, Principal[] issuers, Socket socket) {
  // try the first key manager
  String alias = customKeyManager.chooseClientAlias(keyType, issuers, socket);
  if( alias == null ) {
   alias = jvmKeyManager.chooseClientAlias(keyType, issuers, socket);
   logger.warn("Reverting to JVM CLIENT alias : " + alias);
  }

  return alias;

 }

 @Override
 public String chooseServerAlias(String keyType, Principal[] issuers, Socket socket) {
  // try the first key manager
  String alias = customKeyManager.chooseServerAlias(keyType, issuers, socket);
  if( alias == null ) {
   alias =  jvmKeyManager.chooseServerAlias(keyType, issuers, socket);
   logger.warn("Reverting to JVM Server alias : " + alias);
  } 
  return alias;
 }

 @Override
 public X509Certificate[] getCertificateChain(String alias) {
  X509Certificate[] chain = customKeyManager.getCertificateChain(alias);
  if( chain == null || chain.length == 0) {
   logger.warn("Reverting to JVM Chain : " + alias);
   return jvmKeyManager.getCertificateChain(alias);
  } else {
   return chain;
  }  
 }

 @Override
 public String[] getClientAliases(String keyType, Principal[] issuers) {
  String[] cAliases = customKeyManager.getClientAliases(keyType, issuers);
  String[] jAliases = jvmKeyManager.getClientAliases(keyType, issuers);
  logger.warn("Supported Client Aliases Custom: " + cAliases.length + " JVM : " + jAliases.length);
  return ArrayUtils.join(cAliases,jAliases);
 }

 @Override
 public PrivateKey getPrivateKey(String alias) {
  PrivateKey key = customKeyManager.getPrivateKey(alias);
  if( key == null ) {
   logger.warn("Reverting to JVM Key : " + alias);
   return jvmKeyManager.getPrivateKey(alias);
  } else {
   return key;
  }
 }

 @Override
 public String[] getServerAliases(String keyType, Principal[] issuers) {
  String[] cAliases = customKeyManager.getServerAliases(keyType, issuers);
  String[] jAliases = jvmKeyManager.getServerAliases(keyType, issuers);
  logger.warn("Supported Server Aliases Custom: " + cAliases.length + " JVM : " + jAliases.length);
  return ArrayUtils.join(cAliases,jAliases);
 }

}

Then, you can use this keystore manager when creating an SSL Context or SocketFactory. The code needs some refactoring and tidying up but it works perfectly.

 /**
  * Returns an array of KeyManagers, set up to use the required keyStore.
  * This method does the bulk of the work of setting up the custom trust managers.
  * 
  * @param props 
  * 
  * @return an array of KeyManagers set up accordingly.
  */
 private static KeyManager[] getKeyManagers(Properties props) throws IOException, GeneralSecurityException {
  // First, get the default KeyManagerFactory.
  String alg = KeyManagerFactory.getDefaultAlgorithm();
  KeyManagerFactory kmFact = KeyManagerFactory.getInstance(alg);   
  // Next, set up the KeyStore to use. We need to load the file into
  // a KeyStore instance.
  FileInputStream fis = new FileInputStream(props.getProperty(SSL_KEYSTORE));
  logger.info("Loaded keystore");
  KeyStore ks = KeyStore.getInstance("jks");
  String keyStorePassword = props.getProperty(SSL_KEYSTORE_PASSWORD);
  ks.load(fis, keyStorePassword.toCharArray());
  fis.close();
  // Now we initialise the KeyManagerFactory with this KeyStore
  kmFact.init(ks, keyStorePassword.toCharArray());

  // default
  KeyManagerFactory dkmFact = KeyManagerFactory.getInstance(alg); 
  dkmFact.init(null,null);  

  // Get the first X509KeyManager in the list
  X509KeyManager customX509KeyManager = getX509KeyManager(alg, kmFact);
  X509KeyManager jvmX509KeyManager = getX509KeyManager(alg, dkmFact);

  KeyManager[] km = { new MultiKeyStoreManager(jvmX509KeyManager, customX509KeyManager) };   
  logger.debug("Number of key managers registered:" + km.length);  
  return km;
 }


 /**
  * Find a X509 Key Manager compatible with a particular algorithm
  * @param algorithm
  * @param kmFact
  * @return
  * @throws NoSuchAlgorithmException
  */
 private static X509KeyManager getX509KeyManager(String algorithm, KeyManagerFactory kmFact)
   throws NoSuchAlgorithmException {
  KeyManager[] keyManagers = kmFact.getKeyManagers();

  if (keyManagers == null || keyManagers.length == 0) {
   throw new NoSuchAlgorithmException("The default algorithm :" + algorithm + " produced no key managers");
  }

  X509KeyManager x509KeyManager = null;

  for (int i = 0; i < keyManagers.length; i++) {
   if (keyManagers[i] instanceof X509KeyManager) {
    x509KeyManager = (X509KeyManager) keyManagers[i];
    break;
   }
  }

  if (x509KeyManager == null) {
   throw new NoSuchAlgorithmException("The default algorithm :"+ algorithm + " did not produce a X509 Key manager");
  }
  return x509KeyManager;
 }




 private static void initialiseManager(Properties props) throws IOException, GeneralSecurityException { 
  // Next construct and initialise a SSLContext with the KeyStore and
  // the TrustStore. We use the default SecureRandom.
  SSLContext context = SSLContext.getInstance("SSL");
  context.init(getKeyManagers(props), getTrustManagers(props), null);
  SSLContext.setDefault(context);

 }

Let me know if anyone has any question or need any demonstration codes.

Raz s answer was a great start, but wasn t quite flexible enough to meet my needs. The MultiStoreKeyManager explicitly checks the custom KeyManager and then falls back to the jvm KeyManager if an operation fails. I actually want to check jvm certs first; the best solution should be able to handle either case. Additionally, the answer fails to provide a working TrustManager.

I ve written a couple more flexible classes, CompositeX509KeyManager and CompositeX509TrustManager, which add support for any number of keystores in an arbitrary order.

CompositeX509KeyManager

package com.mycompany.ssl;

import java.net.Socket;
import java.security.Principal;
import java.security.PrivateKey;
import java.security.cert.X509Certificate;
import java.util.List;

import javax.annotation.Nullable;
import javax.net.ssl.X509KeyManager;

import com.google.common.collect.ImmutableList;
import com.google.common.collect.Iterables;

/**
 * Represents an ordered list of {@link X509KeyManager}s with most-preferred managers first.
 *
 * This is necessary because of the fine-print on {@link SSLContext#init}:
 *     Only the first instance of a particular key and/or trust manager implementation type in the
 *     array is used. (For example, only the first javax.net.ssl.X509KeyManager in the array will be used.)
 *
 * @author codyaray
 * @since 4/22/2013
 * @see http://stackoverflow.com/questions/1793979/registering-multiple-keystores-in-jvm
 */
public class CompositeX509KeyManager implements X509KeyManager {

  private final List keyManagers;

  /**
   * Creates a new {@link CompositeX509KeyManager}.
   *
   * @param keyManagers the X509 key managers, ordered with the most-preferred managers first.
   */
  public CompositeX509KeyManager(List keyManagers) {
    this.keyManagers = ImmutableList.copyOf(keyManagers);
  }

  /**
   * Chooses the first non-null client alias returned from the delegate
   * {@link X509TrustManagers}, or {@code null} if there are no matches.
   */
  @Override
  public @Nullable String chooseClientAlias(String[] keyType, Principal[] issuers, Socket socket) {
    for (X509KeyManager keyManager : keyManagers) {
      String alias = keyManager.chooseClientAlias(keyType, issuers, socket);
      if (alias != null) {
        return alias;
      }
    }
    return null;
  }

  /**
   * Chooses the first non-null server alias returned from the delegate
   * {@link X509TrustManagers}, or {@code null} if there are no matches.
   */
  @Override
  public @Nullable String chooseServerAlias(String keyType, Principal[] issuers, Socket socket) {
    for (X509KeyManager keyManager : keyManagers) {
      String alias = keyManager.chooseServerAlias(keyType, issuers, socket);
      if (alias != null) {
        return alias;
      }
    }
    return null;
  }

  /**
   * Returns the first non-null private key associated with the
   * given alias, or {@code null} if the alias can t be found.
   */
  @Override
  public @Nullable PrivateKey getPrivateKey(String alias) {
    for (X509KeyManager keyManager : keyManagers) {
      PrivateKey privateKey = keyManager.getPrivateKey(alias);
      if (privateKey != null) {
        return privateKey;
      }
    }
    return null;
  }

  /**
   * Returns the first non-null certificate chain associated with the
   * given alias, or {@code null} if the alias can t be found.
   */
  @Override
  public @Nullable X509Certificate[] getCertificateChain(String alias) {
    for (X509KeyManager keyManager : keyManagers) {
      X509Certificate[] chain = keyManager.getCertificateChain(alias);
      if (chain != null && chain.length > 0) {
        return chain;
      }
    }
    return null;
  }

  /**
   * Get all matching aliases for authenticating the client side of a
   * secure socket, or {@code null} if there are no matches.
   */
  @Override
  public @Nullable String[] getClientAliases(String keyType, Principal[] issuers) {
    ImmutableList.Builder aliases = ImmutableList.builder();
    for (X509KeyManager keyManager : keyManagers) {
      aliases.add(keyManager.getClientAliases(keyType, issuers));
    }
    return emptyToNull(Iterables.toArray(aliases.build(), String.class));
  }

  /**
   * Get all matching aliases for authenticating the server side of a
   * secure socket, or {@code null} if there are no matches.
   */
  @Override
  public @Nullable String[] getServerAliases(String keyType, Principal[] issuers) {
    ImmutableList.Builder aliases = ImmutableList.builder();
    for (X509KeyManager keyManager : keyManagers) {
      aliases.add(keyManager.getServerAliases(keyType, issuers));
    }
    return emptyToNull(Iterables.toArray(aliases.build(), String.class));
  }

  @Nullable
  private static <T> T[] emptyToNull(T[] arr) {
    return (arr.length == 0) ? null : arr;
  }

}

CompositeX509TrustManager

package com.mycompany.ssl;

import java.security.cert.CertificateException;
import java.security.cert.X509Certificate;
import java.util.List;

import javax.net.ssl.X509TrustManager;

import com.google.common.collect.ImmutableList;
import com.google.common.collect.Iterables;

/**
 * Represents an ordered list of {@link X509TrustManager}s with additive trust. If any one of the
 * composed managers trusts a certificate chain, then it is trusted by the composite manager.
 *
 * This is necessary because of the fine-print on {@link SSLContext#init}:
 *     Only the first instance of a particular key and/or trust manager implementation type in the
 *     array is used. (For example, only the first javax.net.ssl.X509KeyManager in the array will be used.)
 *
 * @author codyaray
 * @since 4/22/2013
 * @see http://stackoverflow.com/questions/1793979/registering-multiple-keystores-in-jvm
 */
public class CompositeX509TrustManager implements X509TrustManager {

  private final List trustManagers;

  public CompositeX509TrustManager(List trustManagers) {
    this.trustManagers = ImmutableList.copyOf(trustManagers);
  }

  @Override
  public void checkClientTrusted(X509Certificate[] chain, String authType) throws CertificateException {
    for (X509TrustManager trustManager : trustManagers) {
      try {
        trustManager.checkClientTrusted(chain, authType);
        return; // someone trusts them. success!
      } catch (CertificateException e) {
        // maybe someone else will trust them
      }
    }
    throw new CertificateException("None of the TrustManagers trust this certificate chain");
  }

  @Override
  public void checkServerTrusted(X509Certificate[] chain, String authType) throws CertificateException {
    for (X509TrustManager trustManager : trustManagers) {
      try {
        trustManager.checkServerTrusted(chain, authType);
        return; // someone trusts them. success!
      } catch (CertificateException e) {
        // maybe someone else will trust them
      }
    }
    throw new CertificateException("None of the TrustManagers trust this certificate chain");
  }

  @Override
  public X509Certificate[] getAcceptedIssuers() {
    ImmutableList.Builder certificates = ImmutableList.builder();
    for (X509TrustManager trustManager : trustManagers) {
      certificates.add(trustManager.getAcceptedIssuers());
    }
    return Iterables.toArray(certificates.build(), X509Certificate.class);
  }

}

Usage

For the standard case of one keystore + jvm keystore, you can wire it up like this. I’m using Guava again, but in a Guicey wrapper this time:

@Provides @Singleton
SSLContext provideSSLContext(KeyStore keystore, char[] password) {
  String defaultAlgorithm = KeyManagerFactory.getDefaultAlgorithm();
  X509KeyManager customKeyManager = getKeyManager("SunX509", keystore, password);
  X509KeyManager jvmKeyManager = getKeyManager(defaultAlgorithm, null, null);
  X509TrustManager customTrustManager = getTrustManager("SunX509", keystore);
  X509TrustManager jvmTrustManager = getTrustManager(defaultAlgorithm, null);

  KeyManager[] keyManagers = { new CompositeX509KeyManager(ImmutableList.of(jvmKeyManager, customKeyManager)) };
  TrustManager[] trustManagers = { new CompositeX509TrustManager(ImmutableList.of(jvmTrustManager, customTrustManager)) };

  SSLContext context = SSLContext.getInstance("SSL");
  context.init(keyManagers, trustManagers, null);
  return context;
}

private X509KeyManager getKeyManager(String algorithm, KeyStore keystore, char[] password) {
  KeyManagerFactory factory = KeyManagerFactory.getInstance(algorithm);
  factory.init(keystore, password);
  return Iterables.getFirst(Iterables.filter(
      Arrays.asList(factory.getKeyManagers()), X509KeyManager.class), null);
}

private X509TrustManager getTrustManager(String algorithm, KeyStore keystore) {
  TrustManagerFactory factory = TrustManagerFactory.getInstance(algorithm);
  factory.init(keystore);
  return Iterables.getFirst(Iterables.filter(
      Arrays.asList(factory.getTrustManagers()), X509TrustManager.class), null); 
}

I extracted this from my blog post about this problem which has a bit more detail, motivation, etc. All the code is there though, so its standalone. :)

Maybe I am 10 years too late to answer this question, but it could be maybe helpful for other developers too. I also ran into the same challenge of loading multiple keystores as keymaterial/trustmaterial. I discovered this page and the answer which Cody A. Ray has provided. After using the same snippet for multiple projects, I thought it would be handy to create a library and also make it publicly available to contribute back to the community. Please have a look here: Github - SSLContext-Kickstart The code snippet of Cody A. Ray for the CompositeKeyManager and CompositeTrustManager are both included.

Usage:

import nl.altindag.ssl.SSLFactory;

import javax.net.ssl.SSLContext;

public class App {

    public static void main(String[] args) {
        String keyStorePathOne = ...;
        String keyStorePathTwo = ...;
        String trustStorePathOne = ...;
        String trustStorePathTwo = ...;
        char[] password = "password".toCharArray();


        SSLFactory sslFactory = SSLFactory.builder()
                .withDefaultTrustMaterial() // loads JDK trusted certificates
                .withIdentityMaterial(keyStorePathOne, password)
                .withIdentityMaterial(keyStorePathTwo, password)
                .withTrustMaterial(trustStorePathOne, password)
                .withTrustMaterial(trustStorePathTwo, password)
                .build();

        SSLContext sslContext = sslFactory.getSslContext();
    }

}

I wasn t quite sure if I should post this here, because it could also be seen as a way to promote "my library" but I thought it could be helpful for developers.

You can add the dependency with the following snippet:

<dependency>
    <groupId>io.github.hakky54</groupId>
    <artifactId>sslcontext-kickstart</artifactId>
    <version>7.3.0</version>
</dependency>

Check out my answer to this question,

How can I have multiple SSL certificates for a Java server

If you use the MyKeyManager, you can have multiple keystores or you can use a single keystore for multiple contexts.