Mac Os X - Curl empty reply from server problem

Dear friends,

In my country, Brazil, we have new rules that oblige us to submmit our invoices to a server before we can ship goods out.

I will try to be very brief, basically we send a signed XML file containing the data to a server that belongs to our local taxing department, and the server replies authorizing the use of the invoice.

It is a SOAP-RPC web service, under SSL and certificates are exchanged.

I am currently using an opensource package developed in Brazil called nfephp, that uses curl and nusoap for this. (nusoap ends using curl...)

The code s functional, and I have discovered a particular issue with my Mac Os X operating system (I believe so).

Basically I can send XML file to a server that is just for tests or to another server that is the "real" server, and has legal value. The tests server is used only for development.

The URLs are:

the legal value one:
https://nfe.sefazvirtual.rs.gov.br

the one for tests only:
https://homologacao.nfe.sefazvirtual.rs.gov.br

The problem:

We have been in the past 20 days with a curl timeout problem that alternates from
these two servers. Most of the time I can comunicate with one of these two servers, sometimes none,
but never both as I did for the previous 6 months.

Let me make myself clear:

My webservice can comunicate with https://nfe.sefazvirtual.rs.gov.br but not
with https://homologacao.nfe.sefazvirtual.rs.gov.br at this moment, and yeasterday the problem
was inversed. And it goes on like this...

We have been doing a lot of checking in the php code that uses curl, and it is fully functional once
we move it to another system, such as windows or linux. But the problem remains on Mac Os 10.6.5.

I have come to down to a very, very simple test, I type on terminal.app the following command:


==== Comand begins:

*$ curl -vv --url "https://nfe.sefazvirtual.rs.gov.br/ws/nferecepcao/NfeRecepcao.asmx"*
** About to connect() to nfe.sefazvirtual.rs.gov.br port 443 (#0)*
** Trying 200.233.3.104... connected*
** Connected to nfe.sefazvirtual.rs.gov.br (200.233.3.104) port 443 (#0)*
** SSLv3, TLS handshake, Client hello (1):*
** SSLv3, TLS handshake, Server hello (2):*
** SSLv3, TLS handshake, CERT (11):*
** SSLv3, TLS handshake, Request CERT (13):*
** SSLv3, TLS handshake, Server finished (14):*
** SSLv3, TLS handshake, CERT (11):*
** SSLv3, TLS handshake, Client key exchange (16):*
** SSLv3, TLS change cipher, Client hello (1):*
** SSLv3, TLS handshake, Finished (20):*
** SSLv3, TLS change cipher, Client hello (1):*
** SSLv3, TLS handshake, Finished (20):*
** SSL connection using RC4-MD5*
** Server certificate:*
** subject: C=BR; O=ICP-Brasil; OU=Autoridade Certificadora SERPROACF; OU=CONTRIBUINTE; OU=Equipamento A1; CN=nfe.sefazvirtual.rs.gov.br*
** start date: 2010-06-23 20:08:03 GMT*
** expire date: 2011-06-23 19:35:53 GMT*
** common name: nfe.sefazvirtual.rs.gov.br (matched)*
** issuer: C=BR; O=ICP-Brasil; OU=Servico Federal de Processamento de Dados - SERPRO; OU=CSPB-1; CN=Autoridade Certificadora do SERPRO Final v2*
** SSL certificate verify ok.*
*> GET /ws/nferecepcao/NfeRecepcao.asmx HTTP/1.1*
*> User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3*
*> Host: nfe.sefazvirtual.rs.gov.br*
*> Accept: /*
>
*< HTTP/1.1 403 Forbidden*
*< Content-Length: 89*
*< Content-Type: text/html*
*< Server: Microsoft-IIS/6.0*
*< X-Powered-By: ASP.NET*
*< Date: Thu, 03 Feb 2011 22:36:30 GMT*
<
*<HTML>HTTP Error 403.7 - Forbidden: SSL client certificate is required.</HTML>*

* *


** Connection #0 to host nfe.sefazvirtual.rs.gov.br left intact*
** Closing connection #0*
** SSLv3, TLS alert, Client hello (1):*

====== Command ends


And as you see I have a reply that asks me for a client certificate.

I can of course add the certificate to the command line, but it really doesn't make any difference to this simple test.

But when I try the test server:

===== Command begins

*$ curl -vv --url "https://homologacao.nfe.sefazvirtual.rs.gov.br/ws/nferecepcao/NfeRecepcao.asmx "*
** About to connect() to homologacao.nfe.sefazvirtual.rs.gov.br port 443 (#0)*
** Trying 200.233.3.103... connected*
** Connected to homologacao.nfe.sefazvirtual.rs.gov.br (200.233.3.103) port 443 (#0)*
** SSLv3, TLS handshake, Client hello (1):*
** SSLv3, TLS handshake, Server hello (2):*
** SSLv3, TLS handshake, CERT (11):*
** SSLv3, TLS handshake, Server finished (14):*
** SSLv3, TLS handshake, Client key exchange (16):*
** SSLv3, TLS change cipher, Client hello (1):*
** SSLv3, TLS handshake, Finished (20):*
** SSLv3, TLS change cipher, Client hello (1):*
** SSLv3, TLS handshake, Finished (20):*
** SSL connection using RC4-MD5*
** Server certificate:*
** subject: C=BR; O=ICP-Brasil; OU=Autoridade Certificadora SERPROACF; OU=CONTRIBUINTE; OU=Equipamento A1; CN=homologacao.nfe.sefazvirtual.rs.gov.br*
** start date: 2010-06-23 20:03:27 GMT*
** expire date: 2011-06-23 19:32:40 GMT*
** common name: homologacao.nfe.sefazvirtual.rs.gov.br (matched)*
** issuer: C=BR; O=ICP-Brasil; OU=Servico Federal de Processamento de Dados - SERPRO; OU=CSPB-1; CN=Autoridade Certificadora do SERPRO Final v2*
** SSL certificate verify ok.*
*> GET /ws/nferecepcao/NfeRecepcao.asmx HTTP/1.1*
*> User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3*
*> Host: homologacao.nfe.sefazvirtual.rs.gov.br*
*> Accept: /*
>
** SSLv3, TLS handshake, Hello request (0):*

===== Command ends


The command times out with no reply.

And this alternates every day from one server to the other...

Sometimes I can not comunicate with one server, sometimes another server...

===== Curl version:
*curl --version*
*curl 7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3*
*Protocols: tftp ftp telnet dict ldap http file https ftps*
*Features: GSS-Negotiate IPv6 Largefile NTLM SSL libz*
==== end version

I have also for tests a free Java Applications that is supplied by the taxing office, and when I can not send my invoices using my routine, I import the XML into it, and send the invoice away without any problems, using the very same system.

+Safari and Firefox can talk to both servers using my certificate all the time User uploaded file+


I apreciate your attention in reading this long email, thank you,


Bernardo Höhl
Rio de Janeiro - Brazil

MacBook Air, Mac OS X (10.6.5)

Posted on Feb 4, 2011 5:08 AM

Reply
13 replies

Feb 4, 2011 8:20 AM in response to Bernardo Hohl

It sounds like a DNS problem. The Mac is probably caching the results of the old query instead of trying to lookup the server fresh each time. Since the server is live and now horribly overloaded, they are using DNS to load balance. The Mac is trying to talk to the machine that went down yesterday and needs to be rebooted.

It should be pretty easy to verify my theory with ping.

This is a fairly common issue but not one that I've paid much attention to. See if you can disable DNS caching for those two domains.

Feb 4, 2011 8:48 AM in response to etresoft

Hi etresoft!


Thanks for replying.

I tried:

dscacheutil -flushcache

But the problem is still there.

From this morning till now, the problem has switched servers.

Yeasterday I could talk to:

https://nfe.sefazvirtual.rs.gov.br/ws/nferecepcao/NfeRecepcao.asmx

Today I cann't...

I can only talk today to:

https://homologacao.nfe.sefazvirtual.rs.gov.br/ws/nferecepcao/NfeRecepcao.asmx

Thanks again,

If there is any thing I could try, please help.

Feb 4, 2011 10:49 AM in response to Bernardo Hohl

Yes, the US passed a law ( unpopular ) that businesses have to report any expense over $600 sometime in the future. Seems like the US law will be repealed.

---------------

You could try the numeric ip addresses of 200.233.3.104 or 200.233.3.103. This might get around any DNS lookup problem. I say might because they could change addresses & you'd be out of luck.

Safari had a problem connecting to site.

User uploaded file

Didn't seem to respond to ping:
mac $ ping -c 4 apple.com
PING apple.com (17.112.152.57): 56 data bytes
64 bytes from 17.112.152.57: icmp_seq=0 ttl=243 time=99.232 ms
64 bytes from 17.112.152.57: icmp_seq=1 ttl=243 time=97.822 ms
64 bytes from 17.112.152.57: icmp_seq=2 ttl=243 time=99.063 ms
64 bytes from 17.112.152.57: icmp_seq=3 ttl=243 time=98.304 ms

--- apple.com ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max/stddev = 97.822/98.605/99.232/0.572 ms

mac $ ping -c 4 homologacao.nfe.sefazvirtual.rs.gov.br
PING homologacao.nfe.sefazvirtual.rs.gov.br (200.233.3.103): 56 data bytes

--- homologacao.nfe.sefazvirtual.rs.gov.br ping statistics ---
4 packets transmitted, 0 packets received, 100% packet loss

mac $ ping -c 4 nfe.sefazvirtual.rs.gov.br
PING nfe.sefazvirtual.rs.gov.br (200.233.3.104): 56 data bytes

--- nfe.sefazvirtual.rs.gov.br ping statistics ---
4 packets transmitted, 0 packets received, 100% packet loss
mac $

Feb 4, 2011 12:48 PM in response to Bernardo Hohl

If Java is working correctly from the same box, then the bug here likely isn't directly involving DNS; the Mac OS X resolver code is going to be common.

I was going to suggest the [NFePHP Google group|https://groups.google.com/group/nfephp?hl=pt], but it looks like you're already posting this into that group. It looks like there might be issues with the version of the WSDL and possibly with a revision that's been made to the servers? There's another recent thread pointing to an error within the def_ws2.xml configuration, too. (If I have translated that correctly. My grasp of Portuguese is poor.)

Folks here aren't in a good position to go test with that server, either. I certainly don't want to, um, pique the interest of the Brazilian department that deals with revenue and taxation.

Feb 5, 2011 10:26 AM in response to rccharles

Hi rccharles,

Thanks for replying.

In order to skip the certificate error window you have to add the remote certificate to your keychain and mark it as "always trust".

Using the IP address instead of the name will result in a failure to verify the remote certificate, since that inside the certificate is not the IP address, but the host name.

The server is configured to run under stealth mode, so it won't answer a ping request.

Feb 5, 2011 10:33 AM in response to MrHoffman

Thanks Mr. Hoffman,

I am really lost in this.

If I add the servers certificate to my keychain and tell it to allways trust, and import my company's certificate to keychain and set a preferences to this url, then, I can use Safari to go straight thru the webservice and list services.

This is certainly a curl issue under Mac Os X system.

My guess is that some security patch under Mac Os X is blocking the passage, please note that the tests server is a subdomain of the "legal value" server... I don't know how Mac Os X keychain will understand this...

Message was edited by: Bernardo Hoehl

Feb 5, 2011 3:30 PM in response to MrHoffman

*Now just check how interesting this is:*

*I have set preferences for my certificate under keychain, so I can login into these urls without having to be prompted to choose my client certificate, Safari just goes all the way thru to the service.*

*These tests I run into a server that will time out on a curl request.*

*Check the pictures I attach.*

*It seems to me I can send my invoices using AppleScript and Safari...*

*What do you think my friends?*


User uploaded file


User uploaded file

Feb 5, 2011 4:03 PM in response to Bernardo Hohl

I don't think curl looks for SSL certificates in the Keychain. That's why you're able to connect with Safari but not with curl. That's also why you can connect with curl running on Linux; there, all certificates are stored in one place. You need to store the certificates in a file somewhere and pass their location to curl on the command line, as detailed in the man page.

Feb 5, 2011 6:51 PM in response to Linc Davis

Hi Linc,

Thanks for replying.

My example shows that the server will not reply and times out.

It should return an error that it requires a client certificate.

I could use this command:

curl -vv -s -k --cert /Library/WebServer/Documents/nfe/producao/certs/USINABRASILEIRA.pem:password_he re --url "https://nfe.fazenda.sp.gov.br/nfeWEB/services/NfeRecepcao.asmx?WSDL"

which results in the same timeout in one server and results in a good reply on another server.

But this command could not be tested by others without a proper certificate.

Yes, there is something wrong either with curl or the underlying layer of darwin that makes my application fail...

I know the problem for sure, but don't have the solution yet.

Feb 6, 2011 12:03 PM in response to Bernardo Hohl

Note: I am running Tiger 10.4.11.
My take on it today. I'm not an SSL expert or anyway near so.
1) One or more servers is failing.
2) the curl statement doesn't include ssl information.



This is kind of weird. Yesterday, I was able to access this site:
https://nfe.sefazvirtual.rs.gov.br/ws/nferecepcao/NfeRecepcao.asmx
I was in FireFox & told FireFox to allow the site for one session. Today I get this message:
HTTP Error 403.7 - Forbidden: SSL client certificate is required.


There seems to be random failure occurring at one or both sites.

-----
1) Failure from Safari. Notice how the address changes on failure.

User uploaded file
--------
2) curl fails on safari working site:
User uploaded file
-------------
This statement failed security test for me.

curl -vv --url " https://nfe.sefazvirtual.rs.gov.br/ws/nferecepcao/NfeRecepcao.asmx "


...
curl: (60) SSL certificate problem, verify that the CA cert is OK. Details:
error:14090086:SSL routines:SSL3 GET_SERVERCERTIFICATE:certificate verify failed
More details here: http://curl.haxx.se/docs/sslcerts.html
...

Interesting comment:
"Until 7.18.0, curl bundled a severely outdated ca bundle file that was
installed by default. These days, the curl archives include no ca certs at
all. You need to get them elsewhere. "
http://curl.haxx.se/docs/sslcerts.html

--------------

It would be interesting to get a trace on the data flow on a working & failure case.
Macintosh-HD -> Applications -> Utilities -> Terminal
sudo tcpdump -s 1024 -w ou2t.txt -i ppp0

I haven't figured out tcpdump but it's an idea.

Robert
PS. I'm a knowledgeable user at all this.

This thread has been closed by the system or the community team. You may vote for any posts you find helpful, or search the Community for additional answers.

Mac Os X - Curl empty reply from server problem

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple Account.