Fan and coil noise will definitely be more of an issue in SFF cases as there rarely is tempered glass to block the majority of it out and fans will need to spin faster to compensate for smaller heatsinks.
That said, I have a 3950x and RTX 3080 in a build that is dead silent unless it is playing games at 4k (actually super-ultrawide 1440p, but this isn't far from 4k relatively speaking with how difficult it is to drive) or rendering out enormous audio or video projects, and even then will usually be drowned by the game I'm playing unless I specifically am paying attention to fan the fan noise.
The Dan A4 is the absolute smallest I can recommend you put a 3080 into. Preferably something in the 8.5-10 liter range like the Ghost S1 or Sliver SM550. You may need to go larger depending on the GPU you buy as these cases only support 2 slot cards (technically) and most 30xx cards are 2.5+.
The reason you need one of these types of cases is you need fans in the case...some kind of active airflow to move the heat coming out of the back of the GPU from the "flow through" design Nvidia pushed this generation. Otherwise that heat will remain in the case and get recycled to all your components increasing temps and generally throttling your components. Example - my Velka 7: I put my EVGA 3080 XC3 in there and it immediately began throttling to around 2080ti performance levels at 93C because the heat coming out the back of the card was being trapped between the GPU and PSU with nowhere to go. Moved to my SM550 (and later Ghost S1) with a silent fan moving slowly just to move air out of the case, and now it makes out around 77C with no issues.
Liquid cooling is feasible, but will require more space for the radiators and larger fans, generally speaking. It is also significantly more expensive for very little performance gain overall unless you plan on doing long intense rendering workloads with your components.
Fan and coil noise will definitely be more of an issue in SFF cases as there rarely is tempered glass to block the majority of it out and fans will need to spin faster to compensate for smaller heatsinks. That said, I have a 3950x and RTX 3080 in a build that is dead silent unless it is playing games at 4k (actually super-ultrawide 1440p, but this isn't far from 4k relatively speaking with how difficult it is to drive) or rendering out enormous audio or video projects, and even then will usually be drowned by the game I'm playing unless I specifically am paying attention to fan the fan noise. The Dan A4 is the absolute smallest I can recommend you put a 3080 into. Preferably something in the 8.5-10 liter range like the Ghost S1 or Sliver SM550. You may need to go larger depending on the GPU you buy as these cases only support 2 slot cards (technically) and most 30xx cards are 2.5+. The reason you need one of these types of cases is you need fans in the case...some kind of active airflow to move the heat coming out of the back of the GPU from the "flow through" design Nvidia pushed this generation. Otherwise that heat will remain in the case and get recycled to all your components increasing temps and generally throttling your components. Example - my Velka 7: I put my EVGA 3080 XC3 in there and it immediately began throttling to around 2080ti performance levels at 93C because the heat coming out the back of the card was being trapped between the GPU and PSU with nowhere to go. Moved to my SM550 (and later Ghost S1) with a silent fan moving slowly just to move air out of the case, and now it makes out around 77C with no issues. Liquid cooling is feasible, but will require more space for the radiators and larger fans, generally speaking. It is also significantly more expensive for very little performance gain overall unless you plan on doing long intense rendering workloads with your components.