By Chris Barretto


2011-02-08 22:44:24 8 Comments

I have my config setup to handle a bunch of GET requests which render pixels that work fine to handle analytics and parse query strings for logging. With an additional third party data stream, I need to handle a POST request to a given url that has JSON in an expected loggable format inside of it's request body. I don't want to use a secondary server with proxy_pass and just want to log the whole response into an associated log file like what it does with GET requests. A snippet of some code that I'm using looks like the following:

GET request (which works great):

location ^~ /rl.gif {
  set $rl_lcid $arg_lcid;
  if ($http_cookie ~* "lcid=(.*\S)")
  {
    set $rl_lcid $cookie_lcid;
  }
  empty_gif;
  log_format my_tracking '{ "guid" : "$rl_lcid", "data" : "$arg__rlcdnsegs" }';
  access_log  /mnt/logs/nginx/my.access.log my_tracking;
  rewrite ^(.*)$ http://my/url?id=$cookie_lcid? redirect;
}

Here is kinda what I am trying to do: POST request (which does not work):

location /bk {
  log_format bk_tracking $request_body;
  access_log  /mnt/logs/nginx/bk.access.log bk_tracking;
}

Curling curl http://myurl/bk -d name=example gives me a 404 page not found.

Then I tried:

location /bk.gif {
  empty_gif;
  log_format bk_tracking $request_body;
  access_log  /mnt/logs/nginx/bk.access.log bk_tracking;
}

Curling curl http://myurl/bk.gif -d name=example gives me a 405 Not Allowed.

My current version is nginx/0.7.62. Any help in the right direction is very much appreciated! Thanks!

UPDATE So now my post looks like this:

location /bk {
  if ($request_method != POST) {
    return 405;
  }
  proxy_pass $scheme://127.0.0.1:$server_port/dummy;
  log_format my_tracking $request_body;
  access_log  /mnt/logs/nginx/my.access.log my_tracking;
}
location /dummy { set $test 0; }

It is logging the post data correctly, but returns a 404 on the requesters end. If I change the above code to return a 200 like so:

location /bk {
  if ($request_method != POST) {
    return 405;
  }
  proxy_pass $scheme://127.0.0.1:$server_port/dummy;
  log_format my_tracking $request_body;
  access_log  /mnt/logs/nginx/my.access.log my_tracking;
  return 200;
}
location /dummy { set $test 0; }

Then it return the 200 correctly, but no longer records the post data.

ANOTHER UPDATE Kinda found a working solution. Hopefully this can help other on their way.

6 comments

@ahofmann 2011-09-19 15:45:24

This solution works like a charm (updated in 2017 to honor that log_format needs to be in the http part of the nginx config):

log_format postdata $request_body;

server {
    # (...)

    location = /post.php {
       access_log  /var/log/nginx/postdata.log  postdata;
       fastcgi_pass php_cgi;
    }
}

I think the trick is making nginx believe that you will call a cgi script.

@Oleg Neumyvakin 2013-02-15 13:09:47

I have to move "log_format postdata $request_body;" to server part because nginx says "log_format directive is not allowed here"

@Matt 2013-05-07 15:44:31

This is the right answer. Except, like @OlegNeumyvakin mentioned, I had to move the log_format part. But in my case, I had to move it to inside the "http" block, instead of inside the "server" block, for it to work. But this did the trick!

@Greg Dubicki 2015-04-08 22:15:23

@OlegNeumyvakin & Matt : I have put your remarks into the answer itself (awaiting review currently). Thank you!

@TheFisch 2016-01-13 11:32:30

I even had to move the log_format line above the server part. But then it worked well. ( nginx/1.9.3 )

@ahofmann 2017-12-20 14:39:20

yes, log_format is only allowed in the http part of the nginx config. I changed the example accordingly.

@NoamG 2015-01-13 10:12:44

nginx log format taken from here: http://nginx.org/en/docs/http/ngx_http_log_module.html

no need to install anything extra

worked for me for GET and POST requests:

upstream my_upstream {
   server upstream_ip:upstream_port;
}

location / {
    log_format postdata '$remote_addr - $remote_user [$time_local] '
                       '"$request" $status $bytes_sent '
                       '"$http_referer" "$http_user_agent" "$request_body"';
    access_log /path/to/nginx_access.log postdata;
    proxy_set_header Host $http_host;
    proxy_pass http://my_upstream;
    }
}

just change upstream_ip and upstream_port

@user2096933 2013-02-21 19:47:59

FWIW, this config worked for me:

location = /logpush.html {
  if ($request_method = POST) {
    access_log /var/log/nginx/push.log push_requests;
    proxy_pass $scheme://127.0.0.1/logsink;
    break;
  }   
  return 200 $scheme://$host/serviceup.html;
}   
#
location /logsink {
  return 200;
}

@boqapt 2012-12-25 23:39:41

Try echo_read_request_body.

"echo_read_request_body ... Explicitly reads request body so that the $request_body variable will always have non-empty values (unless the body is so big that it has been saved by Nginx to a local temporary file)."

location /log {
  log_format postdata $request_body;
  access_log /mnt/logs/nginx/my_tracking.access.log postdata;
  echo_read_request_body;
}

@Fluffy 2014-01-09 09:28:19

This worked better than the accepted answer, since it did not need fastcgi to be installed

@NoamG 2015-01-13 09:38:54

@Fluffy yes, but it need echo_read_request_body to be installed...

@firelynx 2015-08-17 12:28:58

install by apt-get install nginx-full

@Vladimir Kovalchuk 2016-07-23 20:38:59

@user4I can see dash only in my logs. What's the problem? I use echo_read_request_body. and echo module installed

@qreba47jhqb4e3lstrujvvdx 2017-09-17 23:15:43

for me, if I redirect (eg 301/302), request_body disappears

@Dale Anderson 2017-09-18 22:09:44

@dcousens at what point are you serving the 301/302? If you're setting the 301/302 redirect right in the nginx location directive, then the behaviour you're seeing makes sense. If it's happening after some php processing, that would be a little odd. You should start a new stack overflow question so someone can help you.

@qreba47jhqb4e3lstrujvvdx 2017-09-20 11:57:25

in the location directive. I don't hesitate to announce that my experience with nginx is relatively low, but it wasn't immediately clear to me why the request_body would disappear, or why that would be sensical.

@epicsmile 2012-06-14 13:44:27

I had a similar problem. GET requests worked and their (empty) request bodies got written to the the log file. POST requests failed with a 404. Experimenting a bit, I found that all POST requests were failing. I found a forum posting asking about POST requests and the solution there worked for me. That solution? Add a proxy_header line right before the proxy_pass line, exactly like the one in the example below.

server {
    listen       192.168.0.1:45080;
    server_name  foo.example.org;

    access_log  /path/to/log/nginx/post_bodies.log post_bodies;
    location / {
      ### add the following proxy_header line to get POSTs to work
      proxy_set_header Host $http_host;
      proxy_pass   http://10.1.2.3;
    }
}

(This is with nginx 1.2.1 for what it is worth.)

@Chris Barretto 2011-07-05 18:50:49

Ok. So finally I was able to log the post data and return a 200. It's kind of a hacky solution that I'm not to proud of which basically overrides the natural behavior for error_page, but my inexperience of nginx plus timelines lead me to this solution:

location /bk {
  if ($request_method != POST) {
    return 405;
  }
  proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;
  proxy_set_header Host $host;
  proxy_set_header X-Real-IP $remote_addr;
  proxy_redirect off;
  proxy_pass $scheme://127.0.0.1:$server_port/success;
  log_format my_tracking $request_body;
  access_log  /mnt/logs/nginx/my_tracking.access.log my_tracking;
}
location /success {
  return 200;
}
error_page   500 502 503 504  /50x.html;
location = /50x.html {
  root   /var/www/nginx-default;
  log_format my_tracking $request_body;
  access_log  /mnt/logs/nginx/my_tracking.access.log my_tracking_2;
}

Now according to that config, it would seem that the proxy pass would return a 200 all the time. Occasionally I would get 500 but when I threw in an error_log to see what was going on, all of my request_body data was in there and I couldn't see a problem. So I caught that and wrote to the same log. Since nginx doesn't like the same name for the tracking variable, I just used my_tracking_2 and wrote to the same log as when it returns a 200. Definitely not the most elegant solution and I welcome any better solution. I've seen the post module, but in my scenario, I couldn't recompile from source.

@turtlemonvh 2015-10-28 18:39:29

The solution you came up with is very similar to the accepted answer here: stackoverflow.com/questions/17609472/… Since nginx doesn't parse the body unless sending to a proxy or a fast-cgi server, proxying to itself works.

@arganzheng 2016-05-31 08:58:11

we use a similar solution, by proxy_pass the POST request to the another location.

Related Questions

Sponsored Content

26 Answered Questions

[SOLVED] How to process POST data in Node.js?

  • 2010-11-28 07:16:28
  • Ming-Tang
  • 715146 View
  • 552 Score
  • 26 Answer
  • Tags:   node.js http-post

8 Answered Questions

[SOLVED] How to have git log show filenames like svn log -v

  • 2009-08-04 21:46:36
  • jes5199
  • 327932 View
  • 848 Score
  • 8 Answer
  • Tags:   svn git logging

12 Answered Questions

[SOLVED] How do I manually fire HTTP POST requests with Firefox or Chrome?

12 Answered Questions

[SOLVED] Hide strange unwanted Xcode logs

2 Answered Questions

[SOLVED] Express - req.ip returns 127.0.0.1

0 Answered Questions

nginx and the 405 error

2 Answered Questions

[SOLVED] How to debug: Client gets 502 but nginx logs 200

3 Answered Questions

[SOLVED] Making a POST request to an external URL from a django + gunicorn + nginx setup

2 Answered Questions

[SOLVED] POST from REST console perceived as GET by server

  • 2015-09-15 08:40:29
  • JackalopeZero
  • 251 View
  • 1 Score
  • 2 Answer
  • Tags:   symfony nginx

0 Answered Questions

tornado stop response after about 37000 requests

Sponsored Content