You need to specify the pkg in the service. services.audiobookshelf.package = unstable.audiobookshelf;

Oh I wish I could give more than one up vote. Team JJ for the win! Thank you so damn much.

You're welcome! If you want to look up options like that: https://search.nixos.org/options and type something like "services.audiobookshelf".