So, I was able to get this working in prod (which is the most important). After looking again at the settings in webpack.config.js, I realized that the line: { from: 'static', to: outDir, ignore: ['.*'] } is actually copying my favicon.ico to the root directory – not into a folder named static.
I had to update my link tag to:
<link rel="icon" href="favicon.ico" type="image/ico">
That worked.
I then noticed that in the wwwroot folder of my project, there is a favicon.ico file. I replaced that with my file and - voila! - it now works in dev as well!