nginx hogs cpu when proxying large files
I have a server where nginx is used as frontend for Apache. nginx serves static content and Apache serves PHP pages. This is a common setup.
Today I migrated stuff to a new server and needed to copy a 7GB database file to another server. I figured HTTP would be fastest way to do it. Unfortunatelly, DNS change already went thought so I could not serve the file on the static domain nginx was configured for.
nevermind, placed file under one of those domains handled by Apache and started the download. It was going fine at 11MB/s for some time. However, soon it started to crawl at 850KB/s. I suspected network problems, but everything else was running fine. I looked at process list and whoa, nginx using 99% of CPU. Because of this single download, the server was brought to its knees and no other client could even get a simple
Hello World page.
I stopped the download at the client side and nginx soon recovered (not restart needed). Then I edited /etc/hosts and place the old IP address of static domain and continued the download (wget -c). It finished few minutes later with 11MB/s average.Tweet to @mbabuskov Tweet Milan Babuškov, 2011-12-01