Secure your selfhosted services NOW – the Log4j2 issue

If you have not heard about the Log4j2 issues then it is time to act! Secure your selfhosted service right now or risk loosing all your data. Here is what you can do.

The issue with Log4j2

Log4j2 is a logging library that is widely popular in the Java world and used in many Java based services. Unfortunately recent an issue with this library surfaced that allows any attacker that can inject a certain string, which can be easily done by providing simple input to the service, to take over the whole server the service is running on. This is less of a bug and more of a strange design but needs to be patched either way.

Attackers are already out there scanning every server for this vulnerability so everyone using public facing Java based software (even indirectly via other systems) should update right now! If your service uses:

• log4j2 in the version 2.0 to 2.14.1
• and does not have set log4j2.formatMsgNoLookups to true

you are affected. If your service is using log4j in a 1.x version you should be safe (but you should also update as this might have other issues).

This means you should keep an eye on messages from any web services you are using (I saw news about Steam and iCloud being vulnerable for some time for example) and if you run your own services (like you still should in my opinion) you need to secure those.

Which (self hosted) services are affected?

You should check your server for all your running services and disable anything that you not 100% need right now. Then go over the list to make sure that they are still secure. Here is a list of common services and my current status for them (no guarantees and this might change in the future).

• OpenHab is affected! Read the details here. You should make sure that your instance is not reachable from the web and update! If you run it via docker then adding log4j2.formatMsgNoLookups to your environment should help too.
• Unifi controller – install 6.5.54 6.6.55 at least (the first one fixes the major issue but the second one closes a small one too)
• Minecraft – this is where the issue was found, update now!
• Elasticsearch – if you you this for logging make sure to update
• Plex should also be updated
• Solr should also be updated if you use it (some DMS systems need it)
• Jitsi Meet also seems to be affected
• Jenkins runs on Java but should not use Log4j

You can find a nice updated list of services here.

Other software including these should be fine:

• Nextcloud – not using Java but check your plugins (if they use Elasticsearch for example)
• Traefik should be safe as it is written in Go
• Home Assistant is mostly Python so should be safe too

Same for most other software. Be sure to check GitHub if the service is open source to see if it is using Java. You should see a small bar on the right side indicating the languages used in the repository. This is how it looks like for Home Assistant.

How to secure your services

The best solutions are mitigation and updates. Try to stop any Internet access for systems you do not really need to make the attack surface smaller. It might be a good idea to make many services only available via VPN (which is what I will also do for most of my services).

For anything else that is affected the easiest solution is to update to a version with a fix. This should include at least log4j2 version 2.15.0. If the service is running at least version 2.10 you can also mitigate the issue by setting the environment variable or JVM argument log4j2.formatMsgNoLookups to true. If you use docker-compose like I do for my smart home server you can add this to the environment by using

serviceName:
  environment:
    - LOG4J_FORMAT_MSG_NO_LOOKUPS=true

It is not over (yet)

Unfortunately the fixes described above did not remove all issues. While they apparently stop your server from running arbitrary code loaded from the Internet there is still a remaining issue that allows attackers to run a denial of service (DOS) attack on it. This only works under certain conditions but wherever possible you should update to version 2.16.0 of log4j2 or install updates for your respective services.

P.S. This is still not over! They found another issue still not fixed by the first two updates and now there is version 2.17.0. While the most important part is still to update to version 2.16.0 you should still be careful and keep updating.

Stay safe and update your services and systems!