我解决了，谢谢。如果有人感兴趣，bouncycastle会骗人的，只是由于我缺乏知识和文档，我花了一些时间。这是代码：

var bytesToDecrypt = Convert.FromBase64String("la0Cz.....D43g=="); // string to decrypt, base64 encoded

AsymmetricCipherKeyPair keyPair;

using (var reader = File.OpenText(@"c:\myprivatekey.pem")) // file containing RSA PKCS1 private key
    keyPair = (AsymmetricCipherKeyPair) new PemReader(reader).ReadObject();

var decryptEngine = new Pkcs1Encoding(new RsaEngine());
decryptEngine.Init(false, keyPair.Private);

var decrypted = Encoding.UTF8.GetString(decryptEngine.ProcessBlock(bytesToDecrypt, 0, bytesToDecrypt.Length));