| Posted by
| Nick Gammon
Australia (23,166 posts) Bio
Forum Administrator |
| Message
| I have spent a couple of hours working out this problem, so this post is sharing my findings with the rest of the world, as it seems to be a bit of an ongoing issue with Safari - the web browser distributed with Apple Macintosh.
The problem
I had been developing some dynamic web pages using scripts documented on this page:
http://www.gammon.com.au/forum/?id=6498
They seemed to work perfectly when tested using Firefox (on the PC), however certain pages just "hung" when tested with Safari, which eventually gave this message:
Safari can't open the page.
Safari can't open the page "http://10.0.0.2/cgi-bin/cgiutils.cgi?form=events&action=Edit&eventid=1". The error was: "lost network connection" (NSURLErrorDomain:-1005) Please choose Report Bug to Apple from the Safari menu, note the error number, and describe what you did before you saw this message.
The problem only appeared to occur on POST pages (that is when you fill in a form and hit a "submit" button) rather than GET pages. A GET page occurs when you simply type in a URL in the address bar, or click on a hyperlink.
Now, in CGI scripts POST data is obtained by reading from stdin, as there is likely to be a lot of it. Compare this to GET data which is simply supplied in environment variables.
In fact, in Lua, I was obtaining the GET data like this:
s = os.getenv ("QUERY_STRING")
I obtained the cookie data like this:
s = os.getenv ("HTTP_COOKIE")
And I obtained the POST data like this:
s = io.read ("*a")
My first problem was that, after changing from Apache web server to thttpd, all attempts to access CGI files were hanging, because thttpd was not even supplying stdin data unless it was for a POST form. So, this fixed up that part:
if os.getenv ("REQUEST_METHOD") == "POST" then
s = os.getenv ("HTTP_COOKIE")
end -- if POST
However I still had the problem with Safari, and POST data, even with that change in place.
Troubleshooting
Time to look at the difference between what Safari was doing, and what Firefox was doing, that caused one to work and not the other one.
After a bit of mucking around looking at the documentation for tcpdump, I worked out this:
$ tcpdump -X -s 0 'src host 10.0.0.3 and dst port 80'
0x0000 4500 00ff 8341 4000 8006 62b3 0a00 0003 E....A@...b.....
0x0010 0a00 0002 0c1c 2382 00e0 583e eec3 647f ......#...X>..d.
0x0020 5018 2238 f8b1 0000 436f 6e74 656e 742d P."8....Content-
0x0030 5479 7065 3a20 6170 706c 6963 6174 696f Type:.applicatio
0x0040 6e2f 782d 7777 772d 666f 726d 2d75 726c n/x-www-form-url
0x0050 656e 636f 6465 640d 0a43 6f6e 7465 6e74 encoded..Content
0x0060 2d4c 656e 6774 683a 2031 3433 0d0a 0d0a -Length:.143....
0x0070 7375 6d6d 6172 793d 5061 7265 6e74 2d74 summary=Parent-t
0x0080 6561 6368 6572 2b6e 6967 6874 2665 7665 eacher+night&eve
0x0090 6e74 3d4d 616e 792b 7061 7265 6e74 732b nt=Many+parents+
0x00a0 6174 7465 6e64 6564 2b74 6865 2b6e 6967 attended+the+nig
0x00b0 6874 2b61 6161 6262 6226 6576 656e 7464 ht+aaabbb&eventd
0x00c0 6174 653d 3230 3036 2d31 312d 3034 2b30 ate=2006-11-04+0
0x00d0 3025 3341 3030 2533 4130 3026 666f 726d 0%3A00%3A00&form
0x00e0 3d65 7665 6e74 7326 6576 656e 7469 643d =events&eventid=
0x00f0 3126 6163 7469 6f6e 3d43 6861 6e67 65 1&action=Change
The address 10.0.0.3 was the PC with Firefox on it, and the "dst port 80" means attempts to access port 80 (the HTTP port).
That showed that the POST data was in the same packet as some of the HTTP headers. I reran it with the -w option, like this:
$ tcpdump -X -s 0 'src host 10.0.0.3 and dst port 80' -w firefox_file
Then edited this file, and cut out the unreadable stuff, to give the actual headers:
Host: 10.0.0.2:80
User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-GB; rv:1.7.12) Gecko/20050919 Firefox/1.0.7
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-gb,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: http://10.0.0.2:80/cgi-bin/cgiutils.cgi?form=events&action=Edit&eventid=1
Cookie: token=0959F66083E33D90FD91A333CCAF9D2965B1E0AAD1062A081A8C9839A505E8E3
---> new packet here <----
Content-Length: 143
summary=Parent-teacher+night&event=Many+parents+attended .. blah blah
Then I compared that (which worked) to the results from Safari (which didn't):
$ tcpdump -X -s 0 'src host 10.0.0.12 and dst port 80'
0x0000 4500 00c3 30ea 4000 4006 f53d 0a00 000c E...0.@.@..=....
0x0010 0a00 0002 c08e 2382 ee9d 6892 092b 8b7b ......#...h..+.{
0x0020 8018 ffff 2839 0000 0101 080a 32e2 085f ....(9......2.._
0x0030 2f5a f875 7375 6d6d 6172 793d 5061 7265 /Z.usummary=Pare
0x0040 6e74 2d74 6561 6368 6572 2b6e 6967 6874 nt-teacher+night
0x0050 2665 7665 6e74 3d4d 616e 792b 7061 7265 &event=Many+pare
0x0060 6e74 732b 6174 7465 6e64 6564 2b74 6865 nts+attended+the
0x0070 2b6e 6967 6874 2b61 6161 6262 6226 6576 +night+aaabbb&ev
0x0080 656e 7464 6174 653d 3230 3036 2d31 312d entdate=2006-11-
0x0090 3034 2b30 3025 3341 3030 2533 4130 3026 04+00%3A00%3A00&
0x00a0 666f 726d 3d65 7665 6e74 7326 6576 656e form=events&even
0x00b0 7469 643d 3126 6163 7469 6f6e 3d43 6861 tid=1&action=Cha
0x00c0 6e67 65 nge
And, saving the data to file to see the difference:
Accept: */*
Accept-Language: en
Accept-Encoding: gzip, deflate
Cookie: token=B7B1AF1014D21E82581C1EEE33E584E202407F6598B9B967AA0F586E9FAB9E79
Referer: http://10.0.0.2:80/cgi-bin/cgiutils.cgi?form=events&action=Edit&eventid=1
User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/418.8 (KHTML, like Gecko) Safari/419.3
Content-Type: application/x-www-form-urlencoded
Content-Length: 143
Connection: keep-alive
Host: 10.0.0.2:80
---> new packet here <----
summary=Parent-teacher+night&event=Many+parents+attended .. blah blah
What does it all mean?
The one big difference here is that the Firefox request bundled up the POST data in the same packet as some of the other headers (in this case, Content-Length: 143) whereas Safari put it into a new packet.
Now, looking back at the code for thttpd, it seems that there is a big difference in how it handles the POST data in these 2 cases:
- If some of the post data has already arrived (the Firefox case) it cannot simply let the CGI process redirect from stdin to the socket, because it would miss out on the first part. Thus it makes a special "interposer process" that buffers up the first part of the POST data, and then supplies the rest from the socket.
Plus, and this is the important bit, it automatically stops supplying data when the "Content-Length" bytes have arrived (in my case 143 bytes). Thus my "read all from stdin" reaches end-of-file and my CGI program proceeds successfully.
- If none of the post data has arrived (the Safari case) it simply redirects the incoming socket to be stdin for the CGI process, and lets the CGI process read it directly. However as the connection is marked "keep-alive" it does not terminate the input stream after 143 bytes, and the CGI program sits there indefinitely trying to read from stdin. Eventually thttpd kills the CGI process (after 30 seconds) and Safari gives an error message about "lost network connection".
The solution
What has worked for me is to make sure that, when reading the POST data, you stop after reading "Content-Length" bytes. This change worked for me:
local post_length = tonumber (os.getenv ("CONTENT_LENGTH")) or 0
if os.getenv ("REQUEST_METHOD") == "POST" and post_length > 0 then
s = io.read (post_length)
end -- if post
Now the io.read is only reading the correct number of bytes, so it no longer hangs. :)
Problem solved. |
- Nick Gammon
www.gammon.com.au, www.mushclient.com | | Top |
|
![[Go to top]](/images/gototop.gif){kind=small}
top