Maybe simply use static linking? Also, one of the problems is that glibc does very little efforts (if any) at keeping backward ABI compatibility. If you can afford it, and go the static linking way as much as possible, maybe try to statically link toward muslc.
For complex languages (aka: not C) this can be harder to do, depending on features or libs one relies on, though, but that should not affect you much.
Using a statically linked binary have it's cons, of course: you can run into libs which have incompatible licencing, there's a link order to respect, and it's not the standard way to do things, but it also have good chances to be the smallest portable build you can do.
If you feel that motivated, you could even do a single binary with all the resources: https://nixers.net/Thread-Nixers-net-Conf-2020 see "Everything about embedding static data in UNIX binaries" but that's mostly if you have time to waste :) (or require it for some reason)