<div dir="ltr"><div>I hadn't tried that, but tried, didn't change anything. I noticed things specifically related to StrongSWAN aren't working since the update to Bullseye and swanctl is not a recognised command. StrongSWAN is installed via apt, version 5.9.1-1<br></div><div class="gmail_quote"><br>swanctl doesn't exist as a command and there is no service called strongswan anymore. I'm not sure how weird that is.<br><br><br>Just been trawling more logs and spotted something else which should be a massive clue;<br><br>Oct 6 16:43:55 VPN-Server charon: 00[CFG] loading secrets from '/etc/ipsec.secrets'<br>Oct 6 16:43:55 VPN-Server charon: 00[LIB] opening '/etc/letsencrypt/live/vpn.my-hostname/privkey.pem' failed: Permission denied<br>Oct 6 16:43:55 VPN-Server charon: 00[LIB] building CRED_PRIVATE_KEY - RSA failed, tried 11 builders<br>Oct 6 16:43:55 VPN-Server charon: 00[CFG] loading private key from '/etc/letsencrypt/live/vpn.my-hostname/privkey.pem' failed<br><br><br>So yeah it looks like it's a simple permissions issue, I'm guessing the dist upgrade has changed the user the service runs as and that uid doesn't have read access to the privkey. I should have thought of that. For some reason I thought it just ran as root.<br><br>Oh..so no, it does run as root, but It's AppArmor, interfering with Charon apparently - the PEM files are created by certbot with symlinks (from 'live' to 'archive') as it rotates through and creates new ones, keeping the old, the newest versions are always symlinked.</div><div class="gmail_quote"><br></div><div class="gmail_quote">Debian Stretch didn't have AppArmor but it's been enabled by default in Debian since Buster. So yeah, the dist-upgrade kinda broke things.</div><div class="gmail_quote"><br></div><div class="gmail_quote">Thanks to Simon Deziel in this old thread from years ago; <a href="https://lists.strongswan.org/pipermail/users/2017-February/010537.html">https://lists.strongswan.org/pipermail/users/2017-February/010537.html</a><br></div><div class="gmail_quote"><br></div><div class="gmail_quote"><br></div><div class="gmail_quote">I've not quite yet figured out how I want to fix it (there are a few options) but at least I know why it does not work. </div><div class="gmail_quote"><br></div><div class="gmail_quote"><br><br>On Wed, 6 Oct 2021 at 16:02, Noel Kuntze <noel.kuntze+strongswan-users-ml@thermi.consulting> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>Hi,<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
</blockquote><br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>Have you tried ipsec stroke rereadsecrets? (Btw, better switch to swanctl)<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
</blockquote><br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>Kind regards<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>Noel<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
</blockquote><br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>Am 06.10.21 um 16:54 schrieb Philip Veale:<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> So about a week about, one of the CAs in the chain Let'sEncrypt use (DST Root CA X3) expired. This shouldn't have been a problem for most clients, as it was cross signed with a CA that had not expired (ISRG Root X1) which most modern clients and devices should trust, though some older ones may not which was (AIUI) why they kept the DST Root CA X3 in there too.<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>><br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> I use a Let'sEncrypt certificate with StrongSWAN and for years it has mostly just worked (mostly being, certbot very usefully renewed the certificate dutifully every so many days, but it didn't get ipsec to re-read this so I'd have to manually punt it about 4 times a year. I could have fixed that but never bothered, anyway, I digress...)<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>><br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> Recently I found the StrongSWAN client on my Android 10 phone wouldn't connect, and at first I thought I just needed to punt it again to re-read the cert, but then I realised it was reporting it had expired even after I'd manually run certbot. And thus I discovered that something that the Android Client is looking at doesn't trust the cert the server was offering as one CA was expired even though another was still valid. This I don't entirely understand as the Let'sEncrypt people seem pretty adamant it should all still work.<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>><br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> I don't entirely understand why new certificates issued today are still being signed with a cert that expired last week, though.<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>><br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> Anyway, moving on. I figured the best way to solve this was to request a cert from LE that was only signed by ISRG Root X1 and NOT by DST Root CA X3 as well, which is not the default behaviour but can be achieved by passing a switch to certbot to ask it to do it that way.<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>><br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> My system was running Debian Switch and I wanted to continue to use certbot, and I didn't want to pollute the system with certbot's suggested tool snap, which imports a little bit of Ubuntu stuff.<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>><br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> Looking at repositories' versions of the certbot package, it was clear I had to upgrade from Stretch to Bullseye, via Buster in between.<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>><br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> So, done that, everything all up to date, got new cert, all should be well now, except it fails, client and server both reporting basically the same thing;<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>><br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> Oct 6 15:20:30 VPN-Server charon: 10[IKE] no private key found for '<a href="http://vpn.my-hostname.net" target="_blank">vpn.my-hostname.net</a>' <br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> Oct 6 15:20:30 VPN-Server charon: 10[ENC] generating IKE_AUTH response 1 [ N(AUTH_FAILED) ]<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>><br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> I've not modified ipsec.secrets, it's still intact and contains the correct reference to the privkey.pem file, which pans out as running the below command brings up the expected result;<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>><br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> # ipsec listcerts<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>><br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> List of X.509 End Entity Certificates<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>><br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> subject: "CN=<a href="http://vpn.my-hostname.net" target="_blank">vpn.my-hostname.net</a>"<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> issuer: "C=US, O=Let's Encrypt, CN=R3"<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> validity: not before Oct 06 14:03:36 2021, ok<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> not after Jan 04 13:03:35 2022, ok (expires in 89 days)<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> serial: [redacted]<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> altNames: <a href="http://vpn.my-hostname.net" target="_blank">vpn.my-hostname.net</a><br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> flags: serverAuth clientAuth<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> OCSP URIs: <a href="http://r3.o.lencr.org" target="_blank">http://r3.o.lencr.org</a><br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> certificatePolicies:<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> 2.23.140.1.2.1<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> 1.3.6.1.4.1.44947.1.1.1<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> CPS: <a href="http://cps.letsencrypt.org" target="_blank">http://cps.letsencrypt.org</a><br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> [other lines trimmed]<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>><br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>><br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> all looks correct to me. All certs present and correct by my reckoning, config unaltered from previous working state before certificate trouble started within the last week.<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>><br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> I say unaltered, I've obviously gone up TWO Debian release versions which might have some bearing on it, but I can't see how and the logs and pointing the finger at a certificate issue which seems more likely.<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>><br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> Only other thing I can thing of is at some point in the past I had manually imported a cacert from Let'sEncrypt onto the system such that "ipsec listcacerts" produced some output, they are gone now so this produces nothing.<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>><br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> Not sure how they'd be needed, though<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>><br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>><br>
> Can anyone spot or think of what I've missed, or has anyone else been through similar recently due to what's happened with Let'sEncrypt ?<br>
</div></div>
</blockquote></div></div>